idnits 2.17.1 draft-gont-tcpm-urgent-data-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 2, 2009) is 5561 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'UNPv1' is defined on line 340, but no explicit reference was found in the text ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 961 (Obsoleted by RFC 991) Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TCP Maintenance and Minor F. Gont 3 Extensions (tcpm) UTN/FRH 4 Internet-Draft A. Yourtchenko 5 Intended status: Standards Track Cisco 6 Expires: August 6, 2009 February 2, 2009 8 On the implementation of TCP urgent data 9 draft-gont-tcpm-urgent-data-01.txt 11 Status of this Memo 13 This Internet-Draft is submitted to IETF in full conformance with the 14 provisions of BCP 78 and BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt. 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 This Internet-Draft will expire on August 6, 2009. 34 Copyright Notice 36 Copyright (c) 2009 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. 46 Abstract 48 This document analyzes how current TCP implementations process TCP 49 urgent indications, and how the behavior of some widely-deployed 50 middle-boxes affect how urgent indications are processed by end 51 systems. This document updates the relevant specifications such that 52 they accommodate current practice in processing TCP urgent 53 indications, and raises awareness about the reliability of TCP urgent 54 indications in the current Internet. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Specification of TCP urgent data . . . . . . . . . . . . . . . 3 60 2.1. Semantics of urgent inications . . . . . . . . . . . . . . 3 61 2.2. Semantics of the Urgent Pointer . . . . . . . . . . . . . 4 62 2.3. Allowed length of urgent data . . . . . . . . . . . . . . 4 63 3. Current implementation practice of TCP urgent data . . . . . . 4 64 3.1. Semantics of urgent indications . . . . . . . . . . . . . 4 65 3.2. Semantics of the Urgent Pointer . . . . . . . . . . . . . 5 66 3.3. Allowed length of urgent data . . . . . . . . . . . . . . 5 67 3.4. Interaction of middle-boxes with urgent data . . . . . . . 6 68 4. Updating RFC 1122 . . . . . . . . . . . . . . . . . . . . . . 6 69 5. Security Considerations . . . . . . . . . . . . . . . . . . . 6 70 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 71 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 7 72 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7 73 8.1. Normative References . . . . . . . . . . . . . . . . . . . 7 74 8.2. Informative References . . . . . . . . . . . . . . . . . . 8 75 Appendix A. Survey of the processing of urgent data by some 76 popular implementations . . . . . . . . . . . . . . . 8 77 A.1. FreeBSD . . . . . . . . . . . . . . . . . . . . . . . . . 8 78 A.2. Linux . . . . . . . . . . . . . . . . . . . . . . . . . . 9 79 A.3. NetBSD . . . . . . . . . . . . . . . . . . . . . . . . . . 9 80 A.4. OpenBSD . . . . . . . . . . . . . . . . . . . . . . . . . 9 81 A.5. Cisco IOS, versions 12.2(18)SXF7, 12.4(15)T7 . . . . . . . 9 82 A.6. Microsoft Windows 2000, Service Pack 4 . . . . . . . . . . 10 83 A.7. Microsoft Windows 2008 . . . . . . . . . . . . . . . . . . 10 84 A.8. Microsoft Windows 95 . . . . . . . . . . . . . . . . . . . 10 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 10 87 1. Introduction 89 TCP incorporates a "urgent mechanism" that allows the sending user to 90 stimulate the receiving user to accept some urgent data and to permit 91 the receiving TCP to indicate to the receiving user when all the 92 currently known urgent data has been received by the user. This 93 mechanism permits a point in the data stream to be designated as the 94 end of urgent information. Whenever this point is in advance of the 95 receive sequence number (RCV.NXT) at the receiving TCP, that TCP must 96 tell the user to go into "urgent mode"; when the receive sequence 97 number catches up to the urgent pointer, the TCP must tell user to go 98 into "normal mode" [RFC0793]. 100 The URG control flag indicates that the "Urgent Pointer" field is 101 meaningful and must be added to the segment sequence number to yield 102 the urgent pointer. The absence of this flag indicates that there is 103 no urgent data outstanding [RFC0793]. 105 This document analyzes how current TCP implementations process TCP 106 urgent indications, and how the behavior of some widely-deployed 107 middle-boxes affect the processing of urgent indications by hosts. 108 This document updates the relevant specifications such that they 109 accommodate current practice in processing TCP urgent indications, 110 and also raises awareness about the reliability of TCP urgent 111 indications in the current Internet. 113 Section 2 describes what the current IETF secifications state with 114 respect to TCP urgent indications. Section 3 describes how current 115 TCP implementations actually process TCP urgent indications. 116 Section 4 updates RFC 1122 [RFC1122] such that it accommodates 117 current practice in processing TCP urgent indications. 119 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 120 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 121 document are to be interpreted as described in RFC 2119 [RFC2119]. 123 2. Specification of TCP urgent data 125 2.1. Semantics of urgent inications 127 As discussed in Section 1, the TCP urgent mechanism permits a point 128 in the data stream to be designated as the end of urgent information. 129 Whenever this point is in advance of the receive sequence number 130 (RCV.NXT) at the receiving TCP, that TCP must tell the user to go 131 into "urgent mode"; when the receive sequence number catches up to 132 the urgent pointer, the TCP must tell user to go into "normal mode". 133 This means, for example, that data that were received as "normal 134 data" might become "urgent data" if an urgent indication is received 135 in some successive TCP segment before those data are consumed by the 136 TCP user. 138 The TCP urgent mechanism is NOT a mechanism for sending "out-of-band" 139 data: the "urgent data" should be delivered "in-line" to the TCP 140 user. 142 2.2. Semantics of the Urgent Pointer 144 There is some ambiguity in RFC 793 [RFC0793] with respect to the 145 semantics of the Urgent Pointer. Section 3.1 (page 17) of RFC 793 146 [RFC0793] states that the Urgent Pointer "communicates the current 147 value of the urgent pointer as a positive offset from the sequence 148 number in this segment. The urgent pointer points to the sequence 149 number of the octet following the urgent data. This field is only be 150 interpreted in segments with the URG control bit set." However, 151 Section 3.9 (page 56) of RFC 793 [RFC0793] states, when describing 152 the processing of the SEND call in the ESTABLISHED and CLOSE-WAIT 153 states, that "If the urgent flag is set, then SND.UP <- SND.NXT-1 and 154 set the urgent pointer in the outgoing segments". 156 RFC 961 [RFC0961] clarified this ambiguity in RFC 793 stating that 157 "Page 17 is wrong. The urgent pointer points to the last octet of 158 urgent data (not to the first octet of non-urgent data)". RFC 1122 159 [RFC1122] formally updated RFC 793 by stating, in Section 4.2.2.4 160 (page 84), that "the urgent pointer points to the sequence number of 161 the LAST octet (not LAST+1) in a sequence of urgent data." 163 2.3. Allowed length of urgent data 165 RFC 793 [RFC0793] allows TCP peers to send urgent data of any length, 166 as the TCP urgent mechanism simply provides a pointer to an 167 interesting point in the data stream. In this respect, Section 168 4.2.2.4 (page 84) of RFC 1122 explicitly states that "A TCP MUST 169 support a sequence of urgent data of any length". 171 3. Current implementation practice of TCP urgent data 173 3.1. Semantics of urgent indications 175 As discussed in Section 1, the TCP urgent mechanism simply permits a 176 point in the data stream to be designated as the end of urgent 177 information, but does NOT provide a mechanism for sending out of band 178 data. 180 Unfortunately, virtually all TCP implementations process TCP urgent 181 data differently. By default, the "last byte of urgent data" is 182 delivered "out of band" to the application. That is, it is not 183 delivered as part of the normal data stream. For example, the "out 184 of band" byte is read by an application when a recv(2) system call 185 with the MSG_OOB flag set is issued. 187 Most implementations provide a socket option (SO_OOBINLINE) that 188 allows an application to override the default processing of urgent 189 data, so that they are delivered "in band" to the application, thus 190 providing the semantics intended by the IETF specifications. 192 3.2. Semantics of the Urgent Pointer 194 All the popular implementations that the authors of this document 195 have been able to test interpret the semantics of the TCP Urgent 196 Pointer as specified in Section 3.1 of RFC 793. This means that even 197 when RFC 1122 officially updated RFC 793 to clarify the ambiguity in 198 the semantics of the Urgent Pointer, this clarification never 199 reflected into actual implementations (i.e., virtually all 200 implementations default to the semantics of the urgent pointer 201 specified in Section 3.1 of RFC 793). 203 Some operating systems provide a system-wide toggle to override this 204 behavior, and interpret the semantics of the Urgent Pointer as 205 clarified in RFC 1122. However, this system-wide toggle has been 206 found to be inconsistent. For example, Linux provides a the sysctl 207 "tcp_stdurg" (e.g., net.ivp4.tcp_stdurg) that, when set, supposedly 208 changes the system behavior to interpret the semantics of the TCP 209 Urgent Pointer as described in RFC 1122. However, this sysctl 210 changes the semantics of the Urgent Pointer only for incoming 211 segments, but not for outgoing segments. This means that if this 212 sysctl is set, an application might be unable to interoperate with 213 itself. 215 3.3. Allowed length of urgent data 217 While Section 4.2.2.4 (page 84) of RFC 1122 explicitly states that "A 218 TCP MUST support a sequence of urgent data of any length", in 219 practice all those implementations that interpret TCP urgent 220 indications as a mechanism for sending out-of-band data keep a buffer 221 of a single byte for storing the "last byte of urgent data". Thus, 222 if successive indications of urgent data are received before the 223 application reads the pending "out of band" byte, that pending byte 224 will be discarded (i.e., overwritten by the new byte of urgent data). 226 In order to avoid urgent data from being discarded, some 227 implementations queue each of the received "urgent bytes", so that 228 even if another urgent indication is received before the pending 229 urgent data are consumed by the application, those bytes do not need 230 to be discarded. Some of these implementations have been known to 231 fail to enforce any limits on the amount of urgent data that they 232 queue, thus resulting vulnerable to trivial resource exhaustion 233 attacks [CPNI-TCP]. 235 3.4. Interaction of middle-boxes with urgent data 237 As a result of the publication of Network Intrusion Detection (NIDs) 238 evasion techniques based on urgent data [phrack] , some middle-boxes 239 modify the TCP data stream such that urgent data is put "in band", 240 that is, they are accessible by the read(2) or recv(2) calls without 241 the MSG_OOB flag. Examples of such middle-boxes are Cisco PIX 242 firewall [Cisco-PIX]. This should discourage applications to depend 243 on urgent data for their corect operation, as urgent data may not be 244 not reliable in the current Internet. 246 4. Updating RFC 1122 248 Firstly, considering that as long as both the TCP sender and the TCP 249 receiver implement the same semantics for the Urgent Pointer there is 250 no functional difference in having the Urgent Pointer point to "the 251 sequence number of the octet following the urgent data" vs. "the last 252 octet of urgent data", and since all known implementations interpret 253 the semantics of the Urgent Pointer as pointing to "the sequence 254 number of the octet following the urgent data", we propose that RFC 255 1122 [RFC1122] be updated such that "the urgent pointer points to the 256 sequence number of the octet following the urgent data" (in segments 257 with the URG control bit set), thus accommodating virtually all 258 existing TCP implementations. 260 Secondly, we strongly encourage applications that employ Sockets API 261 to set the SO_OOBINLINE socket option, such that urgent data is 262 delivered inline, as intended by the IETF specifications. 263 Furthermore, we discourage the use of the MSG_OOB flag in recv(2) 264 calls to retrieve the "urgent data". 266 Finally, considering the discussion in Section 3.4, we discourage 267 applications to depend on the TCP urgent mechanism for correct 268 operation, as urgent data may not be reliable in the current 269 Internet. 271 5. Security Considerations 273 Given that there are two different interpretations of the semantics 274 of the Urgent Pointer in current implementations, and that either 275 middle-boxes (such as packet scrubbers) or the end-systems themselves 276 could cause the urgent data to be processed "in band", there exists 277 ambiguity in how TCP urgent data sent by a TCP will be processed by 278 the intended recipient. This might make it difficult for a Network 279 Intrusion Detection System (NIDS) to track the application-layer data 280 transferred to the destination system, and thus lead to false 281 negatives or false positives in the NIDS [CPNI-TCP]. 283 Probably the best way to avoid the security implications of TCP 284 urgent data is to avoid having application protocols depend on the 285 use of TCP urgent data altogether. Packet scrubbers could probably 286 be configured to clear the URG bit, and set the Urgent Pointer to 287 zero. This would basically cause the urgent data to be put "in 288 band". However, this might cause interoperability problems or 289 undesired behavior in the applications running on top of TCP. 291 6. IANA Considerations 293 This document has no actions for IANA. 295 7. Acknowledgements 297 The authors of this document would like to thank (in alphabetical 298 order) David Borman, Alfred Hoenes, Carlos Pignataro, Anantha 299 Ramaiah, Joe Touch, and Dan Wing for providing valuable feedback on 300 earlier versions of this document. 302 Additionally, Fernando would like to thank David Borman and Joe Touch 303 for a fruitful discussion about TCP urgent mode at IETF 73 304 (Minneapolis). 306 8. References 308 8.1. Normative References 310 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 311 RFC 793, September 1981. 313 [RFC1122] Braden, R., "Requirements for Internet Hosts - 314 Communication Layers", STD 3, RFC 1122, October 1989. 316 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 317 Requirement Levels", BCP 14, RFC 2119, March 1997. 319 8.2. Informative References 321 [CPNI-TCP] 322 CPNI, "Security Assessment of the Transmission Control 323 Protocol (TCP)", (to be published) . 325 [Cisco-PIX] 326 Cisco PIX, "http://www.cisco.com/en/US/docs/security/asa/ 327 asa70/command/reference/tz.html#wp1288756". 329 [FreeBSD] The FreeBSD project, "http://www.freebsd.org". 331 [Linux] The Linux Project, "http://www.kernel.org". 333 [NetBSD] The NetBSD project, "http://www.netbsd.org". 335 [OpenBSD] The OpenBSD project, "http://www.openbsd.org". 337 [RFC0961] Reynolds, J. and J. Postel, "Official ARPA-Internet 338 protocols", RFC 961, December 1985. 340 [UNPv1] Stevens, W., "UNIX Network Programming, Volume 1. 341 Networking APIs: Sockets and XTI", Prentice Hall PTR , 342 1997. 344 [Windows2000] 345 Microsoft Windows 2000, "http://technet.microsoft.com/ 346 en-us/library/bb726981(printer).aspx". 348 [Windows95] 349 Microsoft Windows 95, 350 "ftp://ftp.demon.co.uk/pub/mirrors/win95netfaq/ 351 faq-c.html". 353 [phrack] Ko, Y., Ko, S., and M. Ko, "NIDS Evasion Method named 354 "SeolMa"", Phrack Magazine, Volume 0x0b, Issue 0x39, Phile 355 #0x03 of 0x12 http://www.phrack.org/ 356 issues.html?issue=57&id=3#article, 2001. 358 Appendix A. Survey of the processing of urgent data by some popular 359 implementations 361 A.1. FreeBSD 363 FreeBSD [FreeBSD] interprets the semantics of the urgent pointer as 364 specified in RFC 793. It does not provide any sysctl to override 365 this behavior. However, it provides the SO_OOBINLINE that when set 366 causes TCP urgent data to be put "in band". That is, it will be 367 accessible by the read(2) or recv(2) calls without the MSG_OOB flag. 369 FreeBSD supports only one byte of urgent data. That is, only the 370 byte preceding the Urgent Pointer is considered as "urgent data". 372 A.2. Linux 374 Linux [Linux] interprets the semantics of the urgent pointer as 375 specified in RFC 793. It provides the net.ipv4.tcp_stdurg sysctl to 376 override this behavior to interpret the Urgent Pointer as specified 377 by RFC 1122. However, this sysctl only affects the processing of 378 incoming segments (the Urgent Pointer in outgoing segments will still 379 be set as specified in RFC 793). 381 Linux supports only one byte of urgent data. That is, only the byte 382 preceding the Urgent Pointer is considered as "urgent data". 384 A.3. NetBSD 386 NetBSD [NetBSD] interprets the semantics of the urgent pointer as 387 specified in RFC 793. It does not provide any sysctl to override 388 this behavior. However, it provides the SO_OOBINLINE that when set 389 causes TCP urgent data to be put "in band". That is, it will be 390 accessible by the read(2) or recv(2) calls without the MSG_OOB flag. 392 NetBSD supports only one byte of urgent data. That is, only the byte 393 preceding the Urgent Pointer is considered as "urgent data". 395 A.4. OpenBSD 397 OpenBSD [OpenBSD] interprets the semantics of the urgent pointer as 398 specified in RFC 793. It does not provide any sysctl to override 399 this behavior. However, it provides the SO_OOBINLINE that when set 400 causes TCP urgent data to be put "in band". That is, it will be 401 accessible by the read(2) or recv(2) calls without the MSG_OOB flag. 403 OpenBSD supports only one byte of urgent data. That is, only the 404 byte preceding the Urgent Pointer is considered as "urgent data". 406 A.5. Cisco IOS, versions 12.2(18)SXF7, 12.4(15)T7 408 Cisco IOS, versions 12.2(18)SXF7, 12.4(15)T7 interpret the semantics 409 of the urgent pointer as specified in RFC 793. However, tests 410 performed with an HTTP server running on Cisco IOS version 411 12.2(18)SXF7 and 12.4(15)T7 suggest that urgent data is processed "in 412 band". That is, they are accessible together with "normal" data. 413 The TCP debugs on the Cisco IOS device do explicitly mention the 414 presence of urgent data, and thus we infer that the behavior is 415 different depending on the application. 417 A.6. Microsoft Windows 2000, Service Pack 4 419 Microsoft Windows 2000 [Windows2000] interprets the semantics of the 420 urgent pointer as specified in RFC 793. It provides the 421 TcpUseRFC1122UrgentPointer system-wide variable to override this 422 behavior to interpret the Urgent Pointer as specified by RFC 1122. 423 However, the tests performed with the sample server application 424 compiled using the cygwin environment, has shown that the default 425 behavior is to return the urgent data "in band". 427 A.7. Microsoft Windows 2008 429 Microsoft Windows 2008 interprets the semantics of the urgent pointer 430 as specified in RFC 793. 432 A.8. Microsoft Windows 95 434 Microsoft Windows 95 interprets the semantics of the urgent pointer 435 as specified in RFC 793. It provides the BSDUrgent system-wide 436 variable to override this behavior to interpret the Urgent Pointer as 437 specified by RFC 1122. Windows 95 supports only one byte of urgent 438 data. That is, only the byte preceding the Urgent Pointer is 439 considered as "urgent data". [Windows95] 441 Authors' Addresses 443 Fernando Gont 444 Universidad Tecnologica Nacional / Facultad Regional Haedo 445 Evaristo Carriego 2644 446 Haedo, Provincia de Buenos Aires 1706 447 Argentina 449 Phone: +54 11 4650 8472 450 Email: fernando@gont.com.ar 451 URI: http://www.gont.com.ar 452 Andrew Yourtchenko 453 Cisco 454 De Kleetlaan, 7 455 Diegem B-1831 456 Belgium 458 Phone: +32 2 704 5494 459 Email: ayourtch@cisco.com