idnits 2.17.1 draft-ietf-tcpm-fastopen-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 8 characters in excess of 72. ** There are 2 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 30, 2014) is 3581 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Experimental ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 1323 (Obsoleted by RFC 7323) -- Obsolete informational reference (is this intentional?): RFC 1644 (Obsoleted by RFC 6247) -- Obsolete informational reference (is this intentional?): RFC 2460 (Obsoleted by RFC 8200) -- Obsolete informational reference (is this intentional?): RFC 6013 (Obsoleted by RFC 7805) == Outdated reference: A later version (-10) exists of draft-ietf-tcpm-fastopen-01 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft Y. Cheng 3 draft-ietf-tcpm-fastopen-09.txt J. Chu 4 Intended status: Experimental S. Radhakrishnan 5 Expiration date: January, 2015 A. Jain 6 Google, Inc. 7 June 30, 2014 9 TCP Fast Open 11 Abstract 13 This document describes an experimental TCP mechanism TCP Fast Open 14 (TFO). TFO allows data to be carried in the SYN and SYN-ACK packets 15 and consumed by the receiving end during the initial connection 16 handshake, thus saving up to one full round trip time (RTT) compared 17 to the standard TCP, which requires a three-way handshake (3WHS) to 18 complete before data can be exchanged. However TFO deviates from the 19 standard TCP semantics since the data in the SYN could be replayed to 20 an application in some rare circumstances. Applications should not 21 use TFO unless they can tolerate this issue detailed in the 22 Applicability section. 24 Status of this Memo 26 Distribution of this memo is unlimited. 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF), its areas, and its working groups. Note that other 33 groups may also distribute working documents as Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 The list of current Internet-Drafts can be accessed at 41 http://www.ietf.org/1id-abstracts.html 43 The list of Internet-Draft Shadow Directories can be accessed at 44 http://www.ietf.org/shadow.html 46 Copyright Notice 48 Copyright (c) 2014 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 65 2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 2.1 Relaxing TCP Semantics on Duplicated SYNs . . . . . . . . . 4 67 2.2. SYNs with Spoofed IP Addresses . . . . . . . . . . . . . . 4 68 3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5 69 4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 7 70 4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 7 71 4.1.1. TCP Options . . . . . . . . . . . . . . . . . . . . . . 7 72 4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 8 73 4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 9 74 4.1.3.1 Client Caching Negative Responses . . . . . . . . . 9 75 4.2. Fast Open Protocol . . . . . . . . . . . . . . . . . . . . 11 76 4.2.1. Fast Open Cookie Request . . . . . . . . . . . . . . . 11 77 4.2.2. TCP Fast Open . . . . . . . . . . . . . . . . . . . . . 12 78 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 14 79 5.1. Resource Exhaustion Attack by SYN Flood with Valid 80 Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . 14 81 5.1.1 Attacks from behind Shared Public IPs (NATs) . . . . . . 15 82 5.2. Amplified Reflection Attack to Random Host . . . . . . . . 16 83 6. TFO's Applicability . . . . . . . . . . . . . . . . . . . . . . 17 84 6.1 Duplicate Data in SYNs . . . . . . . . . . . . . . . . . . . 17 85 6.2 Potential Performance Improvement . . . . . . . . . . . . . 17 86 6.3. Example: Web Clients and Servers . . . . . . . . . . . . . 18 87 6.3.1. HTTP Request Replay . . . . . . . . . . . . . . . . . . 18 88 6.3.2. Speculative Connections by the Applications . . . . . . 18 89 6.3.3. HTTP over TLS (HTTPS) . . . . . . . . . . . . . . . . . 18 90 6.3.4. Comparison with HTTP Persistent Connections . . . . . . 18 91 7. Open Areas for Experimentation . . . . . . . . . . . . . . . . 19 92 7.1. Performance impact due to middle-boxes and NAT . . . . . . 19 93 7.2. Cookie-less Fast Open . . . . . . . . . . . . . . . . . . . 20 94 7.3 Impact on congestion control . . . . . . . . . . . . . . . . 20 95 8. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 21 96 8.1. T/TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 97 8.2. Common Defenses Against SYN Flood Attacks . . . . . . . . . 21 98 8.3. TCP Cookie Transaction (TCPCT) . . . . . . . . . . . . . . 21 100 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 21 101 10. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 23 102 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 103 11.1. Normative References . . . . . . . . . . . . . . . . . . . 23 104 11.2. Informative References . . . . . . . . . . . . . . . . . . 23 105 Appendix A. Example Socket API Changes to support TFO . . . . . . 25 106 A.1 Active Open . . . . . . . . . . . . . . . . . . . . . . . . 25 107 A.2 Passive Open . . . . . . . . . . . . . . . . . . . . . . . . 25 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 110 1. Introduction 112 TCP Fast Open (TFO) is an experimental update to TCP that enables 113 data to be exchanged safely during TCP's connection handshake. This 114 document describes a design that enables applications to save a round 115 trip while avoiding severe security ramifications. At the core of TFO 116 is a security cookie used by the server side to authenticate a client 117 initiating a TFO connection. This document covers the details of 118 exchanging data during TCP's initial handshake, the protocol for TFO 119 cookies, potential new security vulnerabilities and their mitigation, 120 and the new socket API. 122 TFO is motivated by the performance needs of today's Web 123 applications. Current TCP only permits data exchange after the 3-way 124 handshake (3WHS)[RFC793], which adds one RTT to network latency. For 125 short Web transfers this additional RTT is a significant portion of 126 overall network latency, even when HTTP persistent connection is 127 widely used. For example, the Chrome browser [Chrome] keeps TCP 128 connections idle for up to 5 minutes but 35% of HTTP requests are 129 made on new TCP connections [RCCJR11]. For such Web and Web-like 130 applications placing data in the SYN can yield significant latency 131 improvements. Next we describe how we resolve the challenges that 132 arise upon doing so. 134 1.1 Terminology 136 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 137 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 138 document are to be interpreted as described in RFC 2119 [RFC2119]. 140 TFO refers to TCP Fast Open. Client refers to the TCP's active open 141 side and server refers to the TCP's passive open side. 143 2. Data In SYN 145 Standard TCP already allows data to be carried in SYN packets 146 ([RFC793], section 3.4) but forbids the receiver from delivering it 147 to the application until 3WHS is completed. This is because TCP's 148 initial handshake serves to capture old or duplicate SYNs. 150 To enable applications exchange data in TCP handshake, TFO removes 151 the constraint and allows data in SYN packets to be delivered to the 152 application. This change of TCP semantic raises two issues discussed 153 in the following subsections, making TFO unsuitable for certain 154 applications. 156 Therefore TCP implementations MUST NOT use TFO by default, but only 157 use TFO if requested explicitly by the application on a per service 158 port basis. Applications need to evaluate TFO applicability described 159 in Section 6 before using TFO. 161 2.1 Relaxing TCP Semantics on Duplicated SYNs 163 TFO allows data to be delivered to the application before the 3WHS 164 is completed, thus opening itself to a data integrity issue in either 165 of the two cases below: 167 a) the receiver host receives data in a duplicate SYN after it has 168 forgotten it received the original SYN (e.g. due to a reboot); 170 b) the duplicate is received after the connection created by the 171 original SYN has been closed and the close was initiated by the 172 sender (so the receiver will not be protected by the 2MSL TIMEWAIT 173 state). 175 The now obsoleted T/TCP [RFC1644] attempted to address these issues. 176 It was not successful and not deployed due to various vulnerabilities 177 as described in the Related Work section. Rather than trying to 178 capture all dubious SYN packets to make TFO 100% compatible with TCP 179 semantics, we made a design decision early on to accept old SYN 180 packets with data, i.e., to restrict TFO use to a class of 181 applications (Section 6) that are tolerant of duplicate SYN packets 182 with data. We believe this is the right design trade-off balancing 183 complexity with usefulness. 185 2.2. SYNs with Spoofed IP Addresses 187 Standard TCP suffers from the SYN flood attack [RFC4987] because SYN 188 packets with spoofed source IP addresses can easily fill up a 189 listener's small queue, causing a service port to be blocked 190 completely until timeouts. 192 TFO goes one step further to allow server-side TCP to send up data to 193 the application layer before 3WHS is completed. This opens up serious 194 new vulnerabilities. Applications serving ports that have TFO enabled 195 may waste lots of CPU and memory resources processing the requests 196 and producing the responses. If the response is much larger than the 197 request, the attacker can further mount an amplified reflection 198 attack against victims of choice beyond the TFO server itself. 200 Numerous mitigation techniques against regular SYN flood attacks 201 exist and have been well documented [RFC4987]. Unfortunately none are 202 applicable to TFO. We propose a server-supplied cookie to mitigate 203 these new vulnerabilities in Section 3 and evaluate the effectiveness 204 of the defense in Section 7. 206 3. Protocol Overview 208 The key component of TFO is the Fast Open Cookie (cookie), a message 209 authentication code (MAC) tag generated by the server. The client 210 requests a cookie in one regular TCP connection, then uses it for 211 future TCP connections to exchange data during 3WHS: 213 Requesting a Fast Open Cookie: 215 1. The client sends a SYN with a Fast Open Cookie Request option. 217 2. The server generates a cookie and sends it through the Fast Open 218 Cookie option of a SYN-ACK packet. 220 3. The client caches the cookie for future TCP Fast Open connections 221 (see below). 223 Performing TCP Fast Open: 225 1. The client sends a SYN with Fast Open Cookie option and data. 227 2. The server validates the cookie: 228 a. If the cookie is valid, the server sends a SYN-ACK 229 acknowledging both the SYN and the data. The server then 230 delivers the data to the application. 232 b. Otherwise, the server drops the data and sends a SYN-ACK 233 acknowledging only the SYN sequence number. 235 3. If the server accepts the data in the SYN packet, it may send the 236 response data before the handshake finishes. The maximum amount is 237 governed by the TCP's congestion control [RFC5681]. 239 4. The client sends an ACK acknowledging the SYN and the server data. 241 If the client's data is not acknowledged, the client retransmits 242 the data in the ACK packet. 244 5. The rest of the connection proceeds like a normal TCP connection. 245 The client can repeat many Fast Open operations once it acquires a 246 cookie (until the cookie is expired by the server). Thus TFO is 247 useful for applications that have temporal locality on client and 248 server connections. 250 Requesting Fast Open Cookie in connection 1: 252 TCP A (Client) TCP B(Server) 253 ______________ _____________ 254 CLOSED LISTEN 256 #1 SYN-SENT ----- ----------> SYN-RCVD 258 #2 ESTABLISHED <---- ---------- SYN-RCVD 259 (caches cookie C) 261 Performing TCP Fast Open in connection 2: 263 TCP A (Client) TCP B(Server) 264 ______________ _____________ 265 CLOSED LISTEN 267 #1 SYN-SENT ----- ----> SYN-RCVD 269 #2 ESTABLISHED <---- ---- SYN-RCVD 271 #3 ESTABLISHED <---- ---- SYN-RCVD 273 #4 ESTABLISHED ----- --------------------> ESTABLISHED 275 #5 ESTABLISHED --- ----------> ESTABLISHED 277 4. Protocol Details 279 4.1. Fast Open Cookie 281 The Fast Open Cookie is designed to mitigate new security 282 vulnerabilities in order to enable data exchange during handshake. 283 The cookie is a message authentication code tag generated by the 284 server and is opaque to the client; the client simply caches the 285 cookie and passes it back on subsequent SYN packets to open new 286 connections. The server can expire the cookie at any time to enhance 287 security. 289 4.1.1. TCP Options 291 Fast Open Cookie Option 293 The server uses this option to grant a cookie to the client in the 294 SYN-ACK packet; the client uses it to pass the cookie back to the 295 server in subsequent SYN packets. 297 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 298 | Kind | Length | 299 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 300 | | 301 ~ Cookie ~ 302 | | 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 305 Kind 1 byte: constant-TBD (to be assigned by IANA) 306 Length 1 byte: range 6 to 18 (bytes); limited by 307 remaining space in the options field. 308 The number MUST be even. 309 Cookie 4 to 16 bytes (Length - 2) 310 Options with invalid Length values or without SYN flag set MUST be 311 ignored. The minimum Cookie size is 4 bytes. Although the diagram 312 shows a cookie aligned on 32-bit boundaries, alignment is not 313 required. 315 Fast Open Cookie Request Option 317 The client uses this option in the SYN packet to request a cookie 318 from a TFO-enabled server 319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 320 | Kind | Length | 321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 323 Kind 1 byte: constant-TBD (same value as the Fast Open 324 Cookie option) 326 Length 1 byte: constant 2. This distinguishes the option 327 from the Fast Open cookie option. 328 Options with invalid Length values, without SYN flag set, or with ACK 329 flag set MUST be ignored. 331 4.1.2. Server Cookie Handling 333 The server is in charge of cookie generation and authentication. The 334 cookie SHOULD be a message authentication code tag with the following 335 properties: 336 1. The cookie authenticates the client's (source) IP address of the 337 SYN packet. The IP address may be an IPv4 or IPv6 address. 339 2. The cookie can only be generated by the server and can not be 340 fabricated by any other parties including the client. 342 3. The generation and verification are fast relative to the rest of 343 SYN and SYN-ACK processing. 345 4. A server may encode other information in the cookie, and accept 346 more than one valid cookie per client at any given time. But this 347 is server implementation dependent and transparent to the 348 client. 350 5. The cookie expires after a certain amount of time. The reason for 351 cookie expiration is detailed in the "Security Consideration" 352 section. This can be done by either periodically changing the 353 server key used to generate cookies or including a timestamp when 354 generating the cookie. 356 To gradually invalidate cookies over time, the server can 357 implement key rotation to generate and verify cookies using 358 multiple keys. This approach is useful for large-scale servers to 359 retain Fast Open rolling key updates. We do not specify a 360 particular mechanism because the implementation is server 361 specific. 363 The server supports the cookie generation and verification 364 operations: 366 - GetCookie(IP_Address): returns a (new) cookie 368 - IsCookieValid(IP_Address, Cookie): checks if the cookie is valid, 369 i.e., it has not expired and it authenticates the client IP address. 371 Example Implementation: a simple implementation is to use AES_128 to 372 encrypt the IPv4 (with padding) or IPv6 address and truncate to 64 373 bits. The server can periodically update the key to expire the 374 cookies. AES encryption on recent processors is fast and takes only a 375 few hundred nanoseconds [RCCJR11]. 377 If only one valid cookie is allowed per-IP and the server can 378 regenerate the cookie independently, the best validation process is 379 to simply regenerate a valid cookie and compare it against the 380 incoming cookie. In that case if the incoming cookie fails the check, 381 a valid cookie is readily available to be sent to the client. 383 4.1.3. Client Cookie Handling 385 The client MUST cache cookies from servers for later Fast Open 386 connections. For a multi-homed client, the cookies are dependent on 387 the client and server IP addresses. Hence the client should cache at 388 most one (most recently received) cookie per client and server IP 389 addresses pair. 391 Beside the cookie we RECOMMEND that the client caches the MSS to the 392 server to enhance performance. The MSS advertised by the server is 393 stored in the cache to determine the maximum amount of data that can 394 be supported in the SYN packet. This information is needed because 395 data is sent before the server announces its MSS in the SYN-ACK 396 packet. Without this information, the data size in the SYN packet is 397 limited to the default MSS of 536 bytes for IPv4 [RFC1122] and 1240 398 bytes for IPv6 [RFC2460]. In particular it's known an IPv4 receiver 399 advertised MSS less than 536 bytes would result in transmission of an 400 unexpected large segment. If the cache MSS is larger than the typical 401 1460 bytes, the extra large data in SYN segment may have issues that 402 offsets the performance benefit of Fast Open. For examples, the 403 super-size SYN may trigger IP fragmentation and may confuse firewall 404 or middle-boxes, causing SYN retransmission and other side-effects. 405 Therefore the client MAY limit the cached MSS to 1460 bytes. 407 4.1.3.1 Client Caching Negative Responses 409 The client MUST cache negative responses from the server in order to 410 avoid potential connection failures. Negative responses include 411 server not acknowledging the data in SYN, ICMP error messages, and 412 most importantly no response (SYN/ACK) from the server at all, i.e., 413 connection timeout. The last case is likely due to incompatible 414 middle-boxes or firewall blocking the connection completely after it 415 sees data in SYN. If the client does not react to these negative 416 responses and continue to retry Fast Open, the client may never be 417 able to connect to the specific server. 419 For any negative responses, the client SHOULD disable Fast Open on 420 the specific path (the source and destination IP addresses and ports) 421 at least temporarily. Since TFO is enabled on a per-service port 422 basis but cookies are independent of service ports, the client's 423 cache should include remote port numbers too. 425 4.2. Fast Open Protocol 427 One predominant requirement of TFO is to be fully compatible with 428 existing TCP implementations, both on the client and the server 429 sides. 431 The server keeps two variables per listening socket (IP address & 432 port): 434 FastOpenEnabled: default is off. It MUST be turned on explicitly by 435 the application. When this flag is off, the server does not perform 436 any TFO related operations and MUST ignore all cookie options. 438 PendingFastOpenRequests: tracks number of TFO connections in SYN-RCVD 439 state. If this variable goes over a preset system limit, the server 440 MUST disable TFO for all new connection requests until 441 PendingFastOpenRequests drops below the system limit. This variable 442 is used for defending some vulnerabilities discussed in the "Security 443 Considerations" section. 445 The server keeps a FastOpened flag per connection to mark if a 446 connection has successfully performed a TFO. 448 4.2.1. Fast Open Cookie Request 450 Any client attempting TFO MUST first request a cookie from the server 451 with the following steps: 453 1. The client sends a SYN packet with a Fast Open Cookie Request 454 option. 456 2. The server SHOULD respond with a SYN-ACK based on the procedures 457 in the "Server Cookie Handling" section. This SYN-ACK SHOULD 458 contain a Fast Open Cookie option if the server currently supports 459 TFO for this listener port. 461 3. If the SYN-ACK contains a Fast Open Cookie option, the client 462 replaces the cookie and other information as described in the 463 "Client Cookie Handling" section. Otherwise, if the SYN-ACK is 464 first seen, i.e., not a (spurious) retransmission, the client MAY 465 remove the server information from the cookie cache. If the SYN- 466 ACK is a spurious retransmission without valid Fast Open Cookie 467 Option, the client does nothing to the cookie cache for the 468 reasons below. 470 The network or servers may drop the SYN or SYN-ACK packets with the 471 new cookie options, which will cause SYN or SYN-ACK timeouts. We 472 RECOMMEND both the client and the server to retransmit SYN and SYN- 473 ACK without the cookie options on timeouts. This ensures the 474 connections of cookie requests will go through and lowers the latency 475 penalty (of dropped SYN/SYN-ACK packets). The obvious downside for 476 maximum compatibility is that any regular SYN drop will fail the 477 cookie (although one can argue the delay in the data transmission 478 till after 3WHS is justified if the SYN drop is due to network 479 congestion). The next section describes a heuristic to detect such 480 drops when the client receives the SYN-ACK. 482 We also RECOMMEND the client to record the set of servers that failed 483 to respond to cookie requests and only attempt another cookie request 484 after certain period. 486 An alternate proposal is to request a TFO cookie in the FIN instead, 487 since FIN-drop by incompatible middle-boxes does not affect latency. 488 However paths that block SYN cookies may be more likely to drop a 489 later SYN packet with data, and many applications close a connection 490 with RST instead anyway. 492 Although cookie-in-FIN may not improve robustness, it would give 493 clients using a single connection a latency advantage over clients 494 opening multiple parallel connections. If experiments with TFO find 495 that it leads to increased connection-sharding, cookie-in-FIN may 496 prove to be a useful alternative. 498 4.2.2. TCP Fast Open 500 Once the client obtains the cookie from the target server, it can 501 perform subsequent TFO connections until the cookie is expired by the 502 server. 504 Client: Sending SYN 506 To open a TFO connection, the client MUST have obtained a cookie from 507 the server: 509 1. Send a SYN packet. 511 a. If the SYN packet does not have enough option space for the 512 Fast Open Cookie option, abort TFO and fall back to regular 3WHS. 514 b. Otherwise, include the Fast Open Cookie option with the cookie 515 of the server. Include any data up to the cached server MSS or 516 default 536 bytes. 518 2. Advance to SYN-SENT state and update SND.NXT to include the data 519 accordingly. 521 To deal with network or servers dropping SYN packets with payload or 522 unknown options, when the SYN timer fires, the client SHOULD 523 retransmit a SYN packet without data and Fast Open Cookie options. 525 Server: Receiving SYN and responding with SYN-ACK 527 Upon receiving the SYN packet with Fast Open Cookie option: 529 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 530 is false, go to step 5. 532 2. If PendingFastOpenRequests is over the system limit, go to step 5. 534 3. If IsCookieValid() in section 4.1.2 returns false, go to step 5. 536 4. Buffer the data and notify the application. Set FastOpened flag 537 and increment PendingFastOpenRequests. 539 5. Send the SYN-ACK packet. The packet MAY include a Fast Open 540 Option. If FastOpened flag is set, the packet acknowledges the SYN 541 and data sequence. Otherwise it acknowledges only the SYN 542 sequence. The server MAY include data in the SYN-ACK packet if the 543 response data is readily available. Some application may favor 544 delaying the SYN-ACK, allowing the application to process the 545 request in order to produce a response, but this is left up to the 546 implementation. 548 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 549 server MUST follow [RFC5681] (based on [RFC3390]) to set the 550 initial congestion window for sending more data packets. 552 If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK 553 segment with neither data nor Fast Open Cookie options for 554 compatibility reasons. 556 A special case is simultaneous open where the SYN receiver is a 557 client in SYN-SENT state. The protocol remains the same because 558 [RFC793] already supports both data in SYN and simultaneous open. But 559 the client's socket may have data available to read before it's 560 connected. This document does not cover the corresponding API change. 562 Client: Receiving SYN-ACK 564 The client SHOULD perform the following steps upon receiving the SYN- 565 ACK: 567 1. If the SYN-ACK has a Fast Open Cookie Option or MSS option or both, 568 update the corresponding cookie and MSS information in the cookie cache. 570 2. Send an ACK packet. Set acknowledgment number to RCV.NXT and 571 include the data after SND.UNA if data is available. 573 3. Advance to the ESTABLISHED state. 575 Note there is no latency penalty if the server does not acknowledge 576 the data in the original SYN packet. The client SHOULD retransmit any 577 unacknowledged data in the first ACK packet in step 2. The data 578 exchange will start after the handshake like a regular TCP 579 connection. 581 If the client has timed out and retransmitted only regular SYN 582 packets, it can heuristically detect paths that intentionally drop 583 SYN with Fast Open option or data. If the SYN-ACK acknowledges only 584 the initial sequence and does not carry a Fast Open cookie option, 585 presumably it is triggered by a retransmitted (regular) SYN and the 586 original SYN or the corresponding SYN-ACK was lost. 588 Server: Receiving ACK 590 Upon receiving an ACK acknowledging the SYN sequence, the server 591 decrements PendingFastOpenRequests and advances to the ESTABLISHED 592 state. No special handling is required further. 594 5. Security Considerations 596 The Fast Open cookie stops an attacker from trivially flooding 597 spoofed SYN packets with data to burn server resources or to mount an 598 amplified reflection attack on random hosts. The server can defend 599 against spoofed SYN floods with invalid cookies using existing 600 techniques [RFC4987]. We note that although generating bogus cookies 601 is cost-free, the cost of validating the cookies, inherent to any 602 authentication scheme, may be substantial compared to processing a 603 regular SYN packet. We describe these new vulnerabilities of TFO and 604 the countermeasures in detail below. 606 5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies 608 An attacker may still obtain cookies from some compromised hosts, 609 then flood spoofed SYN with data and "valid" cookies (from these 610 hosts or other vantage points). Like regular TCP handshakes, TFO is 611 vulnerable to such an attack. But the potential damage can be much 612 more severe. Besides causing temporary disruption to service ports 613 under attack, it may exhaust server CPU and memory resources. Such an 614 attack will show up on application server logs as a application level 615 DoS from Bot-nets, triggering other defenses and alerts. 617 To protect the server it is important to limit the maximum number of 618 total pending TFO connection requests, i.e., PendingFastOpenRequests 619 (Section 4.2). When the limit is exceeded, the server temporarily 620 disables TFO entirely as described in "Server Cookie Handling". Then 621 subsequent TFO requests will be downgraded to regular connection 622 requests, i.e., with the data dropped and only SYN acknowledged. This 623 allows regular SYN flood defense techniques [RFC4987] like SYN- 624 cookies to kick in and prevent further service disruption. 626 The main impact of SYN floods against the standard TCP stack is not 627 directly from the floods themselves costing TCP processing overhead 628 or host memory, but rather from the spoofed SYN packets filling up 629 the often small listener's queue. 631 On the other hand, TFO SYN floods can cause damage directly if 632 admitted without limit into the stack. The RST packets from the 633 spoofed host will fuel rather than defeat the SYN floods as compared 634 to the non-TFO case, because the attacker can flood more SYNs with 635 data to cost more data processing resources. For this reason, a TFO 636 server needs to monitor the connections in SYN-RCVD being reset in 637 addition to imposing a reasonable max queue length. Implementations 638 may combine the two, e.g., by continuing to account for those 639 connection requests that have just been reset against the listener's 640 PendingFastOpenRequests until a timeout period has passed. 642 Limiting the maximum number of pending TFO connection requests does 643 make it easy for an attacker to overflow the queue, causing TFO to be 644 disabled. We argue that causing TFO to be disabled is unlikely to be 645 of interest to attackers because the service will remain intact 646 without TFO hence there is hardly any real damage. 648 5.1.1 Attacks from behind Shared Public IPs (NATs) 650 An attacker behind a NAT can easily obtain valid cookies to launch 651 the above attack to hurt other clients that share the path. 652 [BRISCOE12] suggested that the server can extend cookie generation to 653 include the TCP timestamp---GetCookie(IP_Address, Timestamp)---and 654 implement it by encrypting the concatenation of the two values to 655 generate the cookie. The client stores both the cookie and its 656 corresponding timestamp, and echoes both in the SYN. The server then 657 implements IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting 658 the IP and timestamp data and comparing it with the cookie value. 660 This enables the server to issue different cookies to clients that 661 share the same IP address, hence can selectively discard those 662 misused cookies from the attacker. However the attacker can simply 663 repeat the attack with new cookies. The server would eventually need 664 to throttle all requests from the IP address just like the current 665 approach. Moreover this approach requires modifying [RFC1323] to send 666 non-zero Timestamp Echo Reply in SYN, potentially cause firewall 667 issues. Therefore we believe the benefit does not outweigh the 668 drawbacks. 670 5.2. Amplified Reflection Attack to Random Host 672 Limiting PendingFastOpenRequests with a system limit can be done 673 without Fast Open Cookies and would protect the server from resource 674 exhaustion. It would also limit how much damage an attacker can cause 675 through an amplified reflection attack from that server. However, it 676 would still be vulnerable to an amplified reflection attack from a 677 large number of servers. An attacker can easily cause damage by 678 tricking many servers to respond with data packets at once to any 679 spoofed victim IP address of choice. 681 With the use of Fast Open Cookies, the attacker would first have to 682 steal a valid cookie from its target victim. This likely requires the 683 attacker to compromise the victim host or network first. But in some 684 case it may be relatively easy. 686 The attacker here has little interest in mounting an attack on the 687 victim host that has already been compromised. But it may be 688 motivated to disrupt the victim's network. Since a stolen cookie is 689 only valid for a single server, it has to steal valid cookies from a 690 large number of servers and use them before they expire to cause 691 sufficient damage without triggering the defense. 693 One can argue that if the attacker has compromised the target network 694 or hosts, it could perform a similar but simpler attack by injecting 695 bits directly. The degree of damage will be identical, but TFO- 696 specific attack allows the attacker to remain anonymous and disguises 697 the attack as from other servers. 699 For example with DHCP an attacker can obtain cookies when he (or the 700 host he has compromised) owns a particular IP address by performing 701 regular Fast Open to servers supporting TFO and collect valid 702 cookies. The attacker then actively or passively releases his IP 703 address. When the IP address is re-assigned to a victim, the attacker 704 now owning a different IP address, floods spoofed Fast Open requests 705 to perform an amplified reflection attack on the victim. 707 The best defense is for the server not to respond with data until 708 handshake finishes. In this case the risk of amplification reflection 709 attack is completely eliminated. But the potential latency saving 710 from TFO may diminish if the server application produces responses 711 earlier before the handshake completes. 713 6. TFO's Applicability 715 This section is to help applications considering TFO to evaluate 716 TFO's benefits and drawbacks using the Web client and server 717 applications as an example throughout. Applications here refer 718 specifically to the process that writes data into the socket, i.e., a 719 JavaScript process that sends data to the server. A proposed socket 720 API change is in the Appendix. 722 6.1 Duplicate Data in SYNs 724 It is possible that using TFO results in the first data written to a 725 socket to be delivered more than once to the application on the 726 remote host (Section 2.1). This replay potential only applies to data 727 in the SYN but not subsequent data exchanges. 729 Empirically [JIDKT07] showed the packet duplication on a Tier-1 730 network is rare. Since the replay only happens specifically when the 731 SYN data packet is duplicated and also the duplicate arrives after 732 the receiver has cleared the original SYN's connection state, the 733 replay is thought to be uncommon in practice. Neverthless a client 734 that cannot handle receiving the same SYN data more than once MUST 735 NOT enable TFO to send data in a SYN. Similarly a server that cannot 736 accept receiving the same SYN data more than once MUST NOT enable TFO 737 to receive data in a SYN. Further investigation is needed to judge 738 about the probability of receiving duplicated SYN or SYN-ACK with 739 data in non-Tier 1 networks. 741 6.2 Potential Performance Improvement 743 TFO is designed for latency-conscious applications that are sensitive 744 to TCP's initial connection setup delay. To benefit from TFO, the 745 first application data unit (e.g., an HTTP request) needs to be no 746 more than TCP's maximum segment size (minus options used in SYN). 747 Otherwise the remote server can only process the client's application 748 data unit once the rest of it is delivered after the initial 749 handshake, diminishing TFO's benefit. 751 To the extent possible, applications SHOULD reuse the connection to 752 take advantage of TCP's built-in congestion control and reduce 753 connection setup overhead. An application that employs too many 754 short-lived connections will negatively impact network stability, as 755 these connections often exit before TCP's congestion control 756 algorithm takes effect. 758 6.3. Example: Web Clients and Servers 760 6.3.1. HTTP Request Replay 762 While TFO is motivated by Web applications, the browser should not 763 use TFO to send requests in SYNs if those requests cannot tolerate 764 replays. One example is POST requests without application-layer 765 transaction protection (e.g., a unique identifier in the request 766 header). 768 On the other hand, TFO is particularly useful for GET requests. 769 Although not all GET requests are idem-potent, GETs are frequently 770 replayed today across striped TCP connections: after a server 771 receives an HTTP request but before the ACKs of the requests reach 772 the browser, the browser may timeout and retry the same request on 773 another (possibly new) TCP connection. This differs from a TFO replay 774 only in that the replay is initiated by the browser, not by the TCP 775 stack. 777 6.3.2. Speculative Connections by the Applications 779 Some Web browsers maintain a history of the domains for frequently 780 visited web pages. The browsers then speculatively pre-open TCP 781 connections to these domains before the user initiates any requests 782 for them [BELSHE11]. While this technique also saves the handshake 783 latency, it wastes server and network resources by initiating and 784 maintaining idle connections. 786 6.3.3. HTTP over TLS (HTTPS) 788 For TLS over TCP, it is safe and useful to include TLS CLIENT_HELLO 789 in the SYN packet to save one RTT in TLS handshake. There is no 790 concern about violating idem-potency. In particular it can be used 791 alone with the speculative connection above. 793 6.3.4. Comparison with HTTP Persistent Connections 795 Is TFO useful given the wide deployment of HTTP persistent 796 connections? The short answer is yes. Studies [RCCJR11][AERG11] show 797 that the average number of transactions per connection is between 2 798 and 4, based on large-scale measurements from both servers and 799 clients. In these studies, the servers and clients both kept idle 800 connections up to several minutes, well into "human think" time. 802 Keeping connections open and idle even longer risks a greater 803 performance penalty. [HNESSK10][MQXMZ11] show that the majority of 804 home routers and ISPs fail to meet the the 124-minute idle timeout 805 mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs silently 806 timeout idle connections within 30 minutes. End hosts, unaware of 807 silent middle-box timeouts, suffer multi-minute TCP timeouts upon 808 using those long-idle connections. 810 To circumvent this problem, some applications send frequent TCP keep- 811 alive probes. However, this technique drains power on mobile devices 812 [MQXMZ11]. In fact, power has become such a prominent issue in modern 813 LTE devices that mobile browsers close HTTP connections within 814 seconds or even immediately [SOUDERS11]. 816 [RCCJR11] studied Chrome browser [Chrome] performance based on 28 817 days of global statistics. The Chrome browser keeps idle HTTP 818 persistent connections for 5 to 10 minutes. However the average 819 number of the transactions per connection is only 3.3 and TCP 3WHS 820 accounts for up to 25% of the HTTP transaction network latency. The 821 authors estimated that TFO improves page load time by 10% to 40% on 822 selected popular Web sites. 824 7. Open Areas for Experimentation 826 We now outline some areas that need experimentation in the Internet 827 and under different network scenarios. These experiments should help 828 the community evaluate Fast Open benefits and risks towards further 829 standardization and implementation of Fast Open and its related 830 protocols. 832 7.1. Performance impact due to middle-boxes and NAT 834 [MAF04] found that some middle-boxes and end-hosts may drop packets 835 with unknown TCP options. Studies [LANGLEY06, HNRGHT11] both found 836 that 6% of the probed paths on the Internet drop SYN packets with 837 data or with unknown TCP options. The TFO protocol deals with this 838 problem by falling back to regular TCP handshake and re-transmitting 839 SYN without data or cookie options after the initial SYN timeout. 840 Moreover the implementation is recommended to negatively cache such 841 incidents to avoid recurring timeouts. Further study is required to 842 evaluate the performance impact of these malicious drop behaviors. 844 Another interesting study is the (loss of) TFO performance benefit 845 behind certain carrier-grade NAT. Typically hosts behind a NAT 846 sharing the same IP address will get the same cookie for the same 847 server. This will not prevent TFO from working. But on some carrier- 848 grade NAT configurations where every new TCP connection from the same 849 physical host uses a different public IP address, TFO does not 850 provide latency benefits. However, there is no performance penalty 851 either, as described in Section "Client: Receiving SYN-ACK". 853 7.2. Cookie-less Fast Open 855 The cookie mechanism mitigates resource exhaustion and amplification 856 attacks. However cookies are not necessary if the server has 857 application-level protection or is immune to these attacks. For 858 example a Web server that only replies with a simple HTTP redirect 859 response that fits in the SYN-ACK packet may not care about resource 860 exhaustion. For such an application, the server could decide to 861 disable TFO cookie checks. 863 Disabling cookies (i.e., no Fast Open TCP options in SYN and SYN/ACK) 864 simplifies both the client and the server, as the client no longer 865 needs to request a cookie and the server no longer needs to check or 866 generate cookies. Disabling cookies also potentially simplifies 867 configuration, as the server no longer needs a key. It may be 868 preferable to enable SYN cookies and disable TFO [RFC4987] when a 869 server is overloaded by a large-scale Bot-net attack. 871 Careful experimentation is necessary to evaluate if cookie-less TFO 872 is practical. The implementation can provide an experimental feature 873 to allow zero length, or null, cookies as opposed to the minimum 4 874 bytes cookies. Thus the server may return a null cookie and the 875 client will send data in SYN with it subsequently. If the server 876 believes it's under a DoS attack through other defense mechanisms, it 877 can switch to regular Fast Open for listener sockets. 879 7.3 Impact on congestion control 881 Although TFO does not directly change the congestion control, there 882 are subtle cases that it may. When SYN-ACK times out, regular TCP 883 reduces the initial congestion window before sending any data 884 [RFC5681]. However in TFO the server may have already sent up to an 885 initial window of data. 887 If the server serves mostly short connections then the losses of SYN- 888 ACKs are not as effective as regular TCP on reducing the congestion 889 window. This could result in an unstable network condition. The 890 connections that experience losses may attempt again and add more 891 load under congestion. A potential solution is to temporarily disable 892 Fast Open if the server observes many SYN-ACK or data losses during 893 the handshake across connections. Further experimentation regarding 894 the congestion control impact will be useful. 896 8. Related Work 898 8.1. T/TCP 900 TCP Extensions for Transactions [RFC1644] attempted to bypass the 901 three-way handshake, among other things, hence shared the same goal 902 but also the same set of issues as TFO. It focused most of its effort 903 battling old or duplicate SYNs, but paid no attention to security 904 vulnerabilities it introduced when bypassing 3WHS [PHRACK98]. 906 As stated earlier, we take a practical approach to focus TFO on the 907 security aspect, while allowing old, duplicate SYN packets with data 908 after recognizing that 100% TCP semantics is likely infeasible. We 909 believe this approach strikes the right tradeoff, and makes TFO much 910 simpler and more appealing to TCP implementers and users. 912 8.2. Common Defenses Against SYN Flood Attacks 914 [RFC4987] studies on mitigating attacks from regular SYN flood, i.e., 915 SYN without data. But from the stateless SYN-cookies to the stateful 916 SYN Cache, none can preserve data sent with SYN safely while still 917 providing an effective defense. 919 The best defense may be to simply disable TFO when a host is 920 suspected to be under a SYN flood attack, e.g., the SYN backlog is 921 filled. Once TFO is disabled, normal SYN flood defenses can be 922 applied. The "Security Consideration" section contains a thorough 923 discussion on this topic. 925 8.3. TCP Cookie Transaction (TCPCT) 927 TCPCT [RFC6013] eliminates server state during initial handshake and 928 defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK 929 packets to carry data. But the server can only send up to MSS bytes 930 of data during the handshake instead of the initial congestion window 931 unlike TFO. Therefore the latency of applications such as Web may be 932 worse than with TFO. 934 9. IANA Considerations 936 IANA is requested to allocate one value from the TCP Option Kind 937 Numbers: The constant-TBD in Section Section 4.1.1 has to be replaced 938 with the newly assigned value. The length of the new TCP option Kind 939 is variable and the Meaning should be set to "TCP Fast Open Cookie". 940 Early implementation before the IANA allocation SHOULD follow 941 [RFC6994] and use experimental option 254 and magic number 0xF989 (16 942 bits), then migrate to the new option after the allocation 943 accordingly. 945 10. Acknowledgement 947 We thank Bob Briscoe, Michael Scharf, Gorry Fairhurst, Rick Jones, 948 Roberto Peon, William Chan, Adam Langley, Neal Cardwell, Eric 949 Dumazet, and Matt Mathis for their feedbacks. We especially thank 950 Barath Raghavan for his contribution on the security design of Fast 951 Open and proofreading this draft numerous times. 953 11. References 955 11.1. Normative References 957 [RFC793] Postel, J. "Transmission Control Protocol", RFC 793, 958 September 1981. 960 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 961 Communication Layers", STD 3, RFC 1122, October 1989. 963 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 964 Requirement Levels", BCP 14, RFC 2119, March 1997. 966 [RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh, 967 P., "NAT Behavioral Requirements for TCP", RFC 5382 969 [RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion 970 Control", RFC 5681, September 2009 972 [RFC6994] Touch, Joe, "Shared Use of Experimental TCP Options", 973 RFC6994, August 2013. 975 [RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's 976 Initial Window", RFC 3390, October 2002. 978 11.2. Informative References 980 [AERG11] Al-Fares, M., Elmeleegy, K., Reed, B., Gashinsky, I., 981 "Overclocking the Yahoo! CDN for Faster Web Page Loads". 982 In Proceedings of Internet Measurement Conference, 983 November 2011. 985 [HNESSK10] Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S., 986 Sarolahti, P., Kojo., M., "An Experimental Study of Home 987 Gateway Characteristics". In Proceedings of Internet 988 Measurement Conference. October 2010 990 [HNRGHT11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., 991 Handley, M., Tokuda, H., "Is it Still Possible to 992 Extend TCP?". In Proceedings of Internet Measurement 993 Conference. November 2011. 995 [LANGLEY06] Langley, A, "Probing the viability of TCP extensions", 996 URL http://www.imperialviolet.org/binary/ecntest.pdf 998 [MAF04] Medina, A., Allman, M., and S. Floyd, "Measuring 999 Interactions Between Transport Protocols and 1000 Middleboxes". In Proceedings of Internet Measurement 1001 Conference, October 2004. 1003 [MQXMZ11] Wang, Z., Qian, Z., Xu, Q., Mao, Z., Zhang, M., 1004 "An Untold Story of Middleboxes in Cellular Networks". 1005 In Proceedings of SIGCOMM. August 2011. 1007 [PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue 1008 53 artical 6. July 8, 1998. URL 1009 http://www.phrack.com/issues.html?issue=53&id=6 1011 [RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A., 1012 Raghavan, B., "TCP Fast Open". In Proceedings of 7th 1013 ACM CoNEXT Conference, December 2011. 1015 [RFC1323] Jacobson, V., Braden, R., Borman, D., "TCP Extensions for 1016 High Performance", RFC 1323, May 1992. 1018 [RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions 1019 Functional Specification", RFC 1644, July 1994. 1021 [RFC2460] Deering, S., Hinden, R., "Internet Protocol, Version 6 1022 (IPv6) Specification", RFC 2460, December 1998. 1024 [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common 1025 Mitigations", RFC 4987, August 2007. 1027 [RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013, 1028 January 2011. 1030 [SOUDERS11] Souders, S., "Making A Mobile Connection". 1031 http://www.stevesouders.com/blog/2011/09/21/making-a- 1032 mobile-connection/ 1034 [BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm- 1035 fastopen-01", tcpm list, 1036 http://www.ietf.org/mail-archive/web/tcpm/ 1037 current/msg07192.html 1039 [BELSHE11] Belshe, M., "The era of browser preconnect.", 1040 http://www.belshe.com/2011/02/10/ 1041 the-era-of-browser-preconnect/ 1043 [JIDKT07] Jaiswal, S., Iannaccone, G., Diot, C., Kurose, J., 1044 Towsley, D., "Measurement and classification of 1045 out-of-sequence packets in a tier-1 IP backbone.". 1046 IEEE/ACM Transactions on Networking (TON), 15(1), 54-66. 1048 [Chrome] Chrome Browser. https://www.google.com/intl/en-US/chrome/browser/ 1050 Appendix A. Example Socket API Changes to support TFO 1052 A.1 Active Open 1054 The active open side involves changing or replacing the connect() 1055 call, which does not take a user data buffer argument. We recommend 1056 replacing connect() call to minimize API changes and hence 1057 applications to reduce the deployment hurdle. 1059 One solution implemented in Linux 3.7 is introducing a new flag 1060 MSG_FASTOPEN for sendto() or sendmsg(). MSG_FASTOPEN marks the 1061 attempt to send data in SYN like a combination of connect() and 1062 sendto(), by performing an implicit connect() operation. It blocks 1063 until the handshake has completed and the data is buffered. 1065 For non-blocking socket it returns the number of bytes buffered and 1066 sent in the SYN packet. If the cookie is not available locally, it 1067 returns -1 with errno EINPROGRESS, and sends a SYN with TFO cookie 1068 request automatically. The caller needs to write the data again when 1069 the socket is connected. On errors, it returns the same errno as 1070 connect() if the handshake fails. 1072 An implementation may prefer not to change the sendmsg() because TFO 1073 is a TCP specific feature. A solution is to add a new socket option 1074 TCP_FASTOPEN for TCP sockets. When the option is enabled before a 1075 connect operation, sendmsg() or sendto() will perform Fast Open 1076 operation similar to the MSG_FASTOPEN flag described above. This 1077 approach however requires an extra setsockopt() system call. 1079 A.2 Passive Open 1081 The passive open side change is simpler compared to active open side. 1082 The application only needs to enable the reception of Fast Open 1083 requests via a new TCP_FASTOPEN setsockopt() socket option before 1084 listen(). 1086 The option enables Fast Open on the listener socket. The option value 1087 specifies the PendingFastOpenRequests threshold, i.e., the maximum 1088 length of pending SYNs with data payload. Once enabled, the TCP 1089 implementation will respond with TFO cookies per request. 1091 Traditionally accept() returns only after a socket is connected. But 1092 for a Fast Open connection, accept() returns upon receiving a SYN 1093 with a valid Fast Open cookie and data, and the data is available to 1094 be read through, e.g., recvmsg(), read(). 1096 Authors' Addresses 1098 Yuchung Cheng 1099 Google, Inc. 1100 1600 Amphitheatre Parkway 1101 Mountain View, CA 94043, USA 1102 EMail: ycheng@google.com 1104 Jerry Chu 1105 Google, Inc. 1106 1600 Amphitheatre Parkway 1107 Mountain View, CA 94043, USA 1108 EMail: hkchu@google.com 1110 Sivasankar Radhakrishnan 1111 Department of Computer Science and Engineering 1112 University of California, San Diego 1113 9500 Gilman Dr 1114 La Jolla, CA 92093-0404 1115 EMail: sivasankar@cs.ucsd.edu 1117 Arvind Jain 1118 Google, Inc. 1119 1600 Amphitheatre Parkway 1120 Mountain View, CA 94043, USA 1121 EMail: arvind@google.com