Re: [tcpm] On the implementation of TCP urgent data (IETF Internet Draft)

[David Borman <david.borman at windriver.com>]

On Mar 3, 2009, at 10:50 AM, Fernando Gont wrote:

> Jerry Leichter wrote:
>
> >   You refer to the data pointed to by the urgent pointer as the  
> > urgent
> > data.  This is incorrect.  TCP doesn't directly have a concept of  
> > urgent
> > *data*; what it has is an urgent *mode*.
> [....]
>
> This was the intent of the text. I'll go through the I-D again and  
> will
> try to remove any instances of what you describe.

RFC 793 specifies all three: the urgent pointer, urgent mode, and  
urgent data.  Urgent data is *all* the data before the urgent  
pointer.  Urgent mode is when you are reading the urgent data.  What  
TCP does not have is out of band (OOB) data.  Writing just one byte in  
urgent mode makes all the previously written and unread data urgent  
data.


> > Note that this is a fine concept, but *it's completely unsupported by
> > the socket API*.  While the socket API can tell you that you've  
> > reached
> > the end of the urgent data (a read() stops at the urgent byte, and
> > there's an ioctl() that tells you that you've just read it),  
> > there's no
> > way to know when you've *entered* urgent mode.  At best, you can  
> > get a
> > signal, but there's no way to figure out how that relates to the
> > position in the TCP stream.
>
> Not sure what you mean.

select() should tell you when you have entered urgent mode, through  
the errorfds descriptor set (which is exceptional conditions, not just  
errors).


> > Given all that, I think your suggestion that there is no OOB data,  
> > only
> > Urgent Mode, is exactly backwards.
>
> Isn't this the converse of what you stated at the very beginning of  
> your
> post??
>
>
>
> > Unless I'm inside of the TCP stack,
> > Urgent Mode is a meaningless concept.  I can't control when it  
> > starts, I
> > can't detect when it starts, I can't even ask if I'm *in* urgent  
> > mode.
>
> huh? What about SIOCATMARK?

The sender controls when the connection leaves urgent mode, and by  
doing so, implicitly makes all the previously unread data urgent  
data.  On the receiver, select() will tell you when you've gone into  
urgent mode, and SIOCATMARK will tell you when you are at the mark.


> >   You comment that based on tests under Cygwin, OOB data is always
> > delivered in-line.  If so, this is a Cygwin issue - I can tell you  
> > quite
> > definitely that if you use the Windows socket implementation, by  
> > default
> > OOB data does *not* get delivered in-line.
>
> Will try this.

There is no OOB data in TCP.  BSD was wrong when it implemented it as  
such, but there were existing applications (telnet) that were using  
the OOB interface, so in order to not break existing binaries, the  
SO_OOBINLINE socket option was created so that you could turn off the  
broken OOB semantics.  That option has been around since at least  
4.4BSD.  Any application that uses the TCP urgent pointer and does  
*not* turn on SO_OOBINLINE has to deal with that decision, but it is  
out of scope for the IETF.


> >   You recommend that applications that use OOB data request that it  
> > be
> > delivered in line.  This is pointless - it fixes nothing.
>
> This is the intented semantincs for TCP urgent mode. And nevertheless,
> the code should be prepared to handle "urgent data" in-band, because
> some middle-boxes while clear the URG bit, thus effectively putting
> "urgent data" back "in-line".

There is no OOB data in TCP.  Applications should be setting  
SO_OOBINLINE and using select to find out when they have transitioned  
into urgent mode, and using SIOCATMARK to determine when they are at  
the end of urgent mode.


> >   The RFC's today *do not allow* middle boxes to turn of the UP bit.
>
> And what? These are widely-deployed middleboxed. If your app depends  
> on
> tcp urgent mode, it will break.
>
>
>
> > Either we change the spec to completely eliminate the whole concept  
> > of
> > urgent data, or we assert that the spec is right, and these middle  
> > boxes
> > don't conform.  It's absurd to specify something and then tell
> > applications "well, it's in the spec, but you can't rely on it".
>
> It's absurd to neglect what's a fact.

Middle boxes shouldn't be turning off the URG bit.

Applications shouldn't be using or relying the OOB socket interface on  
TCP connections.

TCP provides the urgent pointer as mechanism that the applications can  
use to indicate an "interesting" point in the TCP data stream.  It is  
not a record marker.  It is not one-to-one (if you don't read up to  
the mark before a new urgent pointer arrives, you lose the knowledge  
of the earlier urgent pointer).  It is up to the application to define  
what "interesting" means, what it should do with the data it reads  
while in urgent mode, and what to do once it gets to the mark.  An  
application can even be written so that it tolerates whether the  
implementation implemented the urgent pointer as a pointer to the last  
byte of urgent data, or the first byte of non-urgent data.


> > And,
> > frankly, if NDIS's can't get this right - too bad.  Fix them.
>
> This is impossible. The issue here is that there's the semantics of  
> the
> UP as specified by the IETF specs, and there are the semantics that  
> real
> implementations are using (which can be overridden by a system-wide  
> toggle).
>
>
>
> >   BTW, Microsoft actually *relies* on OOB data.  I don't know the
> > details - something to do with aborting database transactions.  I  
> > expect
> > you'd see significant opposition from them to an attempt to remove
> > Urgent Mode from the specs.
>
> Microsoft's implementation of TCP urgent mode is completely broken.
> Shame on them.

OOB != Urgent mode.  TCP does not have OOB data.  You can't get rid of  
something that was never there.

> >   When you come right down to it, Urgent Mode as specified by TCP is
> > unambiguous (ignoring the age-old RFC793/RFC961 which is irrelevant  
> > in
> > today's world:  Everyone has agreed on the same interpretation).  The
> > actual problems are:
> >
> >   - Middle-box implementations that *illegally* modify data in  
> > transit;
>
> huh? You claim that RFC 793 is irrelevant, and then claim that  
> clearing
> the UP is illegal????
>
>
>
>
> > None of these can be fixed at the TCP spec level, since that's not  
> > where
> > the problem is.  (Well, the first is a matter of politics:  Getting
> > implementers to actually implement the specs as they exist!)
>
> This is completely wrong. If you get implementers to implement the  
> specs
> as they exist, you'd be e.g. changing the semantics of the urgent
> pointer (UP), which would basically break all those applications that
> have not yet been broken by middleboxes clearing the URG bit.
>
> Our draft aims at:
>
> * Clarifying the concept of TCP urgent mode (no "out of band" data
> channel. Simply a mark that signals whether you are in urgent mode  
> or not)
>
> * Changing the specs so that the semantics of the UP accommodate all
> real implementations (i.e., "poinst to the last byte of urgent data"  
> vs.
> "points to the byte following the last byte of urgent data".
>
> * Raising awareness about "urgent mode" not being reliable in the
> current Internet (e.g., Cisco PIX clears the URG bit).

Yep, those are the issues.

> > So while I
> > think it's important to raise these issues with IETF, this document  
> > is
> > aimed at the wrong target.
>
> I'm not sure what's the target you think our I-D should aim at.

TCPM is the right place to discuss whether the urgent pointer is the  
last byte of urgent data, or the first byte of non-urgent data.

			-David Borman
> Thanks!
>
> Kind regards,
> --
> Fernando Gont
> e-mail: fernando at gont.com.ar || fgont at acm.org
> PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
>
>
>
>
>
> _______________________________________________
> tcpm mailing list
> tcpm at ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm