idnits 2.17.1 draft-ietf-tcpm-fastopen-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 29, 2014) is 3496 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Experimental ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 1323 (Obsoleted by RFC 7323) -- Obsolete informational reference (is this intentional?): RFC 1644 (Obsoleted by RFC 6247) -- Obsolete informational reference (is this intentional?): RFC 2460 (Obsoleted by RFC 8200) -- Obsolete informational reference (is this intentional?): RFC 6013 (Obsoleted by RFC 7805) == Outdated reference: A later version (-10) exists of draft-ietf-tcpm-fastopen-01 Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft Y. Cheng 3 draft-ietf-tcpm-fastopen-10.txt J. Chu 4 Intended status: Experimental S. Radhakrishnan 5 Expiration date: April, 2015 A. Jain 6 Google 7 September 29, 2014 9 TCP Fast Open 11 Abstract 13 This document describes an experimental TCP mechanism TCP Fast Open 14 (TFO). TFO allows data to be carried in the SYN and SYN-ACK packets 15 and consumed by the receiving end during the initial connection 16 handshake, and saves up to one full round trip time (RTT) compared to 17 the standard TCP, which requires a three-way handshake (3WHS) to 18 complete before data can be exchanged. However TFO deviates from the 19 standard TCP semantics since the data in the SYN could be replayed to 20 an application in some rare circumstances. Applications should not 21 use TFO unless they can tolerate this issue detailed in the 22 Applicability section. 24 Status of this Memo 26 Distribution of this memo is unlimited. 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF), its areas, and its working groups. Note that other 33 groups may also distribute working documents as Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 The list of current Internet-Drafts can be accessed at 41 http://www.ietf.org/1id-abstracts.html 43 The list of Internet-Draft Shadow Directories can be accessed at 44 http://www.ietf.org/shadow.html 46 Copyright Notice 48 Copyright (c) 2014 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 65 2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2.1 Relaxing TCP Semantics on Duplicated SYNs . . . . . . . . . 4 67 2.2. SYNs with Spoofed IP Addresses . . . . . . . . . . . . . . 4 68 3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5 69 4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 6 70 4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 6 71 4.1.1. Fast Open option . . . . . . . . . . . . . . . . . . . 7 72 4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 7 73 4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 8 74 4.1.3.1 Client Caching Negative Responses . . . . . . . . . 9 75 4.2. Fast Open Protocol . . . . . . . . . . . . . . . . . . . . 9 76 4.2.1. Fast Open Cookie Request . . . . . . . . . . . . . . . 10 77 4.2.2. TCP Fast Open . . . . . . . . . . . . . . . . . . . . . 11 78 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 13 79 5.1. Resource Exhaustion Attack by SYN Flood with Valid 80 Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . 13 81 5.1.1 Attacks from behind Shared Public IPs (NATs) . . . . . . 14 82 5.2. Amplified Reflection Attack to Random Host . . . . . . . . 14 83 6. TFO's Applicability . . . . . . . . . . . . . . . . . . . . . . 15 84 6.1 Duplicate Data in SYNs . . . . . . . . . . . . . . . . . . . 15 85 6.2 Potential Performance Improvement . . . . . . . . . . . . . 16 86 6.3. Example: Web Clients and Servers . . . . . . . . . . . . . 16 87 6.3.1. HTTP Request Replay . . . . . . . . . . . . . . . . . . 16 88 6.3.2. HTTP over TLS (HTTPS) . . . . . . . . . . . . . . . . . 16 89 6.3.3. Comparison with HTTP Persistent Connections . . . . . . 17 90 6.3.4. Load Balancers and Server farms . . . . . . . . . . . . 17 91 7. Open Areas for Experimentation . . . . . . . . . . . . . . . . 17 92 7.1. Performance impact due to middle-boxes and NAT . . . . . . 18 93 7.2. Impact on congestion control . . . . . . . . . . . . . . . 18 94 7.3. Cookie-less Fast Open . . . . . . . . . . . . . . . . . . . 18 95 8. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 19 96 8.1. T/TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 97 8.2. Common Defenses Against SYN Flood Attacks . . . . . . . . . 19 98 8.3. Speculative Connections by the Applications . . . . . . . . 19 99 8.4. Fast Open Cookie in FIN . . . . . . . . . . . . . . . . . . 19 100 8.5. TCP Cookie Transaction (TCPCT) . . . . . . . . . . . . . . 20 101 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 20 102 10. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 20 103 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 104 11.1. Normative References . . . . . . . . . . . . . . . . . . . 20 105 11.2. Informative References . . . . . . . . . . . . . . . . . . 21 106 Appendix A. Example Socket API Changes to support TFO . . . . . . 22 107 A.1 Active Open . . . . . . . . . . . . . . . . . . . . . . . . 22 108 A.2 Passive Open . . . . . . . . . . . . . . . . . . . . . . . . 23 109 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 111 1. Introduction 113 TCP Fast Open (TFO) is an experimental update to TCP that enables 114 data to be exchanged safely during TCP's connection handshake. This 115 document describes a design that enables applications to save a round 116 trip while avoiding severe security ramifications. At the core of TFO 117 is a security cookie used by the server side to authenticate a client 118 initiating a TFO connection. This document covers the details of 119 exchanging data during TCP's initial handshake, the protocol for TFO 120 cookies, potential new security vulnerabilities and their mitigation, 121 and the new socket API. 123 TFO is motivated by the performance needs of today's Web 124 applications. Current TCP only permits data exchange after the 3-way 125 handshake (3WHS)[RFC793], which adds one RTT to network latency. For 126 short Web transfers this additional RTT is a significant portion of 127 overall network latency, even when HTTP persistent connection is 128 widely used. For example, the Chrome browser [Chrome] keeps TCP 129 connections idle for up to 5 minutes but 35% of HTTP requests are 130 made on new TCP connections [RCCJR11]. For such Web and Web-like 131 applications placing data in the SYN can yield significant latency 132 improvements. Next we describe how we resolve the challenges that 133 arise upon doing so. 135 1.1. Terminology 137 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 138 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 139 document are to be interpreted as described in RFC 2119 [RFC2119]. 141 TFO refers to TCP Fast Open. Client refers to the TCP's active open 142 side and server refers to the TCP's passive open side. 144 2. Data In SYN 145 Standard TCP already allows data to be carried in SYN packets 146 ([RFC793], section 3.4) but forbids the receiver from delivering it 147 to the application until 3WHS is completed. This is because TCP's 148 initial handshake serves to capture old or duplicate SYNs. 150 To enable applications exchange data in TCP handshake, TFO removes 151 the constraint and allows data in SYN packets to be delivered to the 152 application. This change of TCP semantic raises two issues discussed 153 in the following subsections, making TFO unsuitable for certain 154 applications. 156 Therefore TCP implementations MUST NOT use TFO by default, but only 157 use TFO if requested explicitly by the application on a per service 158 port basis. Applications need to evaluate TFO applicability described 159 in Section 6 before using TFO. 161 2.1 Relaxing TCP Semantics on Duplicated SYNs 163 TFO allows data to be delivered to the application before the 3WHS 164 is completed, thus opening itself to a data integrity issue in either 165 of the two cases below: 167 a) the receiver host receives data in a duplicate SYN after it has 168 forgotten it received the original SYN (e.g. due to a reboot); 170 b) the duplicate is received after the connection created by the 171 original SYN has been closed and the close was initiated by the 172 sender (so the receiver will not be protected by the 2MSL TIMEWAIT 173 state). 175 The now obsoleted T/TCP [RFC1644] attempted to address these issues. 176 It was not successful and not deployed due to various vulnerabilities 177 as described in the Related Work section. Rather than trying to 178 capture all dubious SYN packets to make TFO 100% compatible with TCP 179 semantics, we made a design decision early on to accept old SYN 180 packets with data, i.e., to restrict TFO use to a class of 181 applications (Section 6) that are tolerant of duplicate SYN packets 182 with data. We believe this is the right design trade-off balancing 183 complexity with usefulness. 185 2.2. SYNs with Spoofed IP Addresses 187 Standard TCP suffers from the SYN flood attack [RFC4987] because SYN 188 packets with spoofed source IP addresses can easily fill up a 189 listener's small queue, causing a service port to be blocked 190 completely until timeouts. 192 TFO goes one step further to allow server-side TCP to send up data to 193 the application layer before 3WHS is completed. This opens up serious 194 new vulnerabilities. Applications serving ports that have TFO enabled 195 may waste lots of CPU and memory resources processing the requests 196 and producing the responses. If the response is much larger than the 197 request, the attacker can further mount an amplified reflection 198 attack against victims of choice beyond the TFO server itself. 200 Numerous mitigation techniques against regular SYN flood attacks 201 exist and have been well documented [RFC4987]. Unfortunately none are 202 applicable to TFO. We propose a server-supplied cookie to mitigate 203 these new vulnerabilities in Section 3 and evaluate the effectiveness 204 of the defense in Section 7. 206 3. Protocol Overview 208 The key component of TFO is the Fast Open Cookie (cookie), a message 209 authentication code (MAC) tag generated by the server. The client 210 requests a cookie in one regular TCP connection, then uses it for 211 future TCP connections to exchange data during 3WHS: 213 Requesting a Fast Open Cookie: 215 1. The client sends a SYN with a Fast Open option with an empty 216 cookie field to request a cookie. 218 2. The server generates a cookie and sends it through the Fast Open 219 option of a SYN-ACK packet. 221 3. The client caches the cookie for future TCP Fast Open connections 222 (see below). 224 Performing TCP Fast Open: 226 1. The client sends a SYN with data and the cookie in the Fast Open 227 option. 229 2. The server validates the cookie: 230 a. If the cookie is valid, the server sends a SYN-ACK 231 acknowledging both the SYN and the data. The server then 232 delivers the data to the application. 234 b. Otherwise, the server drops the data and sends a SYN-ACK 235 acknowledging only the SYN sequence number. 237 3. If the server accepts the data in the SYN packet, it may send the 238 response data before the handshake finishes. The maximum amount is 239 governed by the TCP's congestion control [RFC5681]. 241 4. The client sends an ACK acknowledging the SYN and the server data. 242 If the client's data is not acknowledged, the client retransmits 243 the data in the ACK packet. 245 5. The rest of the connection proceeds like a normal TCP connection. 246 The client can repeat many Fast Open operations once it acquires a 247 cookie (until the cookie is expired by the server). Thus TFO is 248 useful for applications that have temporal locality on client and 249 server connections. 251 Requesting Fast Open Cookie in connection 1: 253 TCP A (Client) TCP B(Server) 254 ______________ _____________ 255 CLOSED LISTEN 257 #1 SYN-SENT ----- ----------> SYN-RCVD 259 #2 ESTABLISHED <---- ---------- SYN-RCVD 260 (caches cookie C) 262 Performing TCP Fast Open in connection 2: 264 TCP A (Client) TCP B(Server) 265 ______________ _____________ 266 CLOSED LISTEN 268 #1 SYN-SENT ----- ----> SYN-RCVD 270 #2 ESTABLISHED <---- ---- SYN-RCVD 272 #3 ESTABLISHED <---- ---- SYN-RCVD 274 #4 ESTABLISHED ----- --------------------> ESTABLISHED 276 #5 ESTABLISHED --- ----------> ESTABLISHED 278 4. Protocol Details 280 4.1. Fast Open Cookie 282 The Fast Open Cookie is designed to mitigate new security 283 vulnerabilities in order to enable data exchange during handshake. 284 The cookie is a message authentication code tag generated by the 285 server and is opaque to the client; the client simply caches the 286 cookie and passes it back on subsequent SYN packets to open new 287 connections. The server can expire the cookie at any time to enhance 288 security. 290 4.1.1. Fast Open option 292 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 293 | Kind | Length | 294 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 295 | | 296 ~ Cookie ~ 297 | | 298 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 300 Kind 1 byte: constant-TBD (to be assigned by IANA) 301 Length 1 byte: range 6 to 18 (bytes); limited by 302 remaining space in the options field. 303 The number MUST be even. 304 Cookie 0, or 4 to 16 bytes (Length - 2) 306 The Fast Open option is used to request or to send a Fast Open 307 Cookie. When cookie is not present or empty, the option is used by 308 the client to request a cookie from the server. When the cookie is 309 present, the option is used to pass the cookie from the server to the 310 client or from the client back to the server (to perform a Fast 311 Open). 313 The minimum Cookie size is 4 bytes. Although the diagram shows a 314 cookie aligned on 32-bit boundaries, alignment is not required. 315 Options with invalid Length values or without SYN flag set MUST be 316 ignored. 318 4.1.2. Server Cookie Handling 320 The server is in charge of cookie generation and authentication. The 321 cookie SHOULD be a message authentication code tag with the following 322 properties. We use SHOULD because in some cases the cookie may be 323 trivially generated as discussed in Section 7.3. 325 1. The cookie authenticates the client's (source) IP address of the 326 SYN packet. The IP address may be an IPv4 or IPv6 address. 328 2. The cookie can only be generated by the server and can not be 329 fabricated by any other parties including the client. 331 3. The generation and verification are fast relative to the rest of 332 SYN and SYN-ACK processing. 334 4. A server may encode other information in the cookie, and accept 335 more than one valid cookie per client at any given time. But this 336 is server implementation dependent and transparent to the 337 client. 339 5. The cookie expires after a certain amount of time. The reason for 340 cookie expiration is detailed in the "Security Consideration" 341 section. This can be done by either periodically changing the 342 server key used to generate cookies or including a timestamp when 343 generating the cookie. 345 To gradually invalidate cookies over time, the server can 346 implement key rotation to generate and verify cookies using 347 multiple keys. This approach is useful for large-scale servers to 348 retain Fast Open rolling key updates. We do not specify a 349 particular mechanism because the implementation is server 350 specific. 352 The server supports the cookie generation and verification 353 operations: 355 - GetCookie(IP_Address): returns a (new) cookie 357 - IsCookieValid(IP_Address, Cookie): checks if the cookie is valid, 358 i.e., it has not expired and it authenticates the client IP address. 360 Example Implementation: a simple implementation is to use AES_128 to 361 encrypt the IPv4 (with padding) or IPv6 address and truncate to 64 362 bits. The server can periodically update the key to expire the 363 cookies. AES encryption on recent processors is fast and takes only a 364 few hundred nanoseconds [RCCJR11]. 366 If only one valid cookie is allowed per-IP and the server can 367 regenerate the cookie independently, the best validation process is 368 to simply regenerate a valid cookie and compare it against the 369 incoming cookie. In that case if the incoming cookie fails the check, 370 a valid cookie is readily available to be sent to the client. 372 4.1.3. Client Cookie Handling 374 The client MUST cache cookies from servers for later Fast Open 375 connections. For a multi-homed client, the cookies are dependent on 376 the client and server IP addresses. Hence the client should cache at 377 most one (most recently received) cookie per client and server IP 378 addresses pair. 380 When caching cookies, we recommend that the client also cache the 381 Maximum Segment Size (MSS) advertised by the server. The client can 382 cache the MSS advertised by the server in order to determine the 383 maximum amount of data that the client can fit in the SYN packet in 384 subsequent TFO connections. Caching the server MSS is useful because 385 with Fast Open a client sends data in the SYN packet before the 386 server announces its MSS in the SYN-ACK packet. If the client sends 387 more data in the SYN packet than the server will accept, this will 388 likely require the client to retransmit some or all of the data. 389 Hence caching the server MSS can enhance performance. 391 Without a cached server MSS, the amount of data in the SYN packet is 392 limited to the default MSS of 536 bytes for IPv4 [RFC1122] and 1240 393 bytes for IPv6 [RFC2460]. Even if the client complies with this limit 394 when sending the SYN, it is known that an IPv4 receiver advertising 395 an MSS less than 536 bytes can receive a segment larger than it is 396 expecting. 398 If the cached MSS is larger than the typical size (1460 bytes for 399 IPv4, or 1440 bytes for IPv6), then the excess data in the SYN packet 400 may cause problems that offset the performance benefit of Fast Open. 401 For example, the unusually large SYN may trigger IP fragmentation and 402 may confuse firewalls or middleboxes, causing SYN retransmission and 403 other side effects. Therefore the client MAY limit the cached MSS to 404 1460 bytes for IPv4 or 1440 for IPv6. 406 4.1.3.1 Client Caching Negative Responses 408 The client MUST cache negative responses from the server in order to 409 avoid potential connection failures. Negative responses include 410 server not acknowledging the data in SYN, ICMP error messages, and 411 most importantly no response (SYN/ACK) from the server at all, i.e., 412 connection timeout. The last case is likely due to incompatible 413 middle-boxes or firewall blocking the connection completely after it 414 sees data in SYN. If the client does not react to these negative 415 responses and continue to retry Fast Open, the client may never be 416 able to connect to the specific server. 418 For any negative responses, the client SHOULD disable Fast Open on 419 the specific path (the source and destination IP addresses and ports) 420 at least temporarily. Since TFO is enabled on a per-service port 421 basis but cookies are independent of service ports, the client's 422 cache should include remote port numbers too. 424 4.2. Fast Open Protocol 426 One predominant requirement of TFO is to be fully compatible with 427 existing TCP implementations, both on the client and the server 428 sides. 430 The server keeps two variables per listening socket (IP address & 431 port): 433 FastOpenEnabled: default is off. It MUST be turned on explicitly by 434 the application. When this flag is off, the server does not perform 435 any TFO related operations and MUST ignore all cookie options. 437 PendingFastOpenRequests: tracks number of TFO connections in SYN-RCVD 438 state. If this variable goes over a preset system limit, the server 439 MUST disable TFO for all new connection requests until 440 PendingFastOpenRequests drops below the system limit. This variable 441 is used for defending some vulnerabilities discussed in the "Security 442 Considerations" section. 444 The server keeps a FastOpened flag per connection to mark if a 445 connection has successfully performed a TFO. 447 4.2.1. Fast Open Cookie Request 449 Any client attempting TFO MUST first request a cookie from the server 450 with the following steps: 452 1. The client sends a SYN packet with a Fast Open option with a 453 length field of 0 (empty cookie field). 455 2. The server responds with a SYN-ACK based on the procedures 456 in the "Server Cookie Handling" section. This SYN-ACK may 457 contain a Fast Open option if the server currently supports 458 TFO for this listener port. 460 3. If the SYN-ACK has a Fast Open option with a cookie, the client 461 replaces the cookie and other information as described in the 462 "Client Cookie Handling" section. Otherwise, if the SYN-ACK is 463 first seen, i.e., not a (spurious) retransmission, the client MAY 464 remove the server information from the cookie cache. If the 465 SYN-ACK is a spurious retransmission, the client does nothing to 466 the cookie cache for the reasons below. 468 The network or servers may drop the SYN or SYN-ACK packets with the 469 new cookie options, which will cause SYN or SYN-ACK timeouts. We 470 RECOMMEND both the client and the server to retransmit SYN and SYN- 471 ACK without the cookie options on timeouts. This ensures the 472 connections of cookie requests will go through and lowers the latency 473 penalty (of dropped SYN/SYN-ACK packets). The obvious downside for 474 maximum compatibility is that any regular SYN drop will fail the 475 cookie (although one can argue the delay in the data transmission 476 till after 3WHS is justified if the SYN drop is due to network 477 congestion). The next section describes a heuristic to detect such 478 drops when the client receives the SYN-ACK. 480 We also RECOMMEND the client to record the set of servers that failed 481 to respond to cookie requests and only attempt another cookie request 482 after certain period. 484 4.2.2. TCP Fast Open 486 Once the client obtains the cookie from the target server, it can 487 perform subsequent TFO connections until the cookie is expired by the 488 server. 490 Client: Sending SYN 492 To open a TFO connection, the client MUST have obtained a cookie from 493 the server: 495 1. Send a SYN packet. 497 a. If the SYN packet does not have enough option space for the 498 Fast Open option, abort TFO and fall back to regular 3WHS. 500 b. Otherwise, include the Fast Open option with the cookie 501 of the server. Include any data up to the cached server MSS or 502 default 536 bytes. 504 2. Advance to SYN-SENT state and update SND.NXT to include the data 505 accordingly. 507 To deal with network or servers dropping SYN packets with payload or 508 unknown options, when the SYN timer fires, the client SHOULD 509 retransmit a SYN packet without data and Fast Open options. 511 Server: Receiving SYN and responding with SYN-ACK 513 Upon receiving the SYN packet with Fast Open option: 515 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 516 is false, go to step 5. 518 2. If PendingFastOpenRequests is over the system limit, go to step 5. 520 3. If IsCookieValid() in section 4.1.2 returns false, go to step 5. 522 4. Buffer the data and notify the application. Set FastOpened flag 523 and increment PendingFastOpenRequests. 525 5. Send the SYN-ACK packet. The packet MAY include a Fast Open 526 Option. If FastOpened flag is set, the packet acknowledges the SYN 527 and data sequence. Otherwise it acknowledges only the SYN 528 sequence. The server MAY include data in the SYN-ACK packet if the 529 response data is readily available. Some application may favor 530 delaying the SYN-ACK, allowing the application to process the 531 request in order to produce a response, but this is left up to the 532 implementation. 534 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 535 server MUST follow [RFC5681] (based on [RFC3390]) to set the 536 initial congestion window for sending more data packets. 538 If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK 539 segment with neither data nor Fast Open options for compatibility 540 reasons. 542 A special case is simultaneous open where the SYN receiver is a 543 client in SYN-SENT state. The protocol remains the same because 544 [RFC793] already supports both data in SYN and simultaneous open. But 545 the client's socket may have data available to read before it's 546 connected. This document does not cover the corresponding API change. 548 Client: Receiving SYN-ACK 550 The client SHOULD perform the following steps upon receiving the SYN- 551 ACK: 553 1. If the SYN-ACK has a Fast Open option or MSS option or 554 both, update the corresponding cookie and MSS information in the 555 cookie cache. 557 2. Send an ACK packet. Set acknowledgment number to RCV.NXT and 558 include the data after SND.UNA if data is available. 560 3. Advance to the ESTABLISHED state. 562 Note there is no latency penalty if the server does not acknowledge 563 the data in the original SYN packet. The client SHOULD retransmit any 564 unacknowledged data in the first ACK packet in step 2. The data 565 exchange will start after the handshake like a regular TCP 566 connection. 568 If the client has timed out and retransmitted only regular SYN 569 packets, it can heuristically detect paths that intentionally drop 570 SYN with Fast Open option or data. If the SYN-ACK acknowledges only 571 the initial sequence and does not carry a Fast Open cookie option, 572 presumably it is triggered by a retransmitted (regular) SYN and the 573 original SYN or the corresponding SYN-ACK was lost. 575 Server: Receiving ACK 577 Upon receiving an ACK acknowledging the SYN sequence, the server 578 decrements PendingFastOpenRequests and advances to the ESTABLISHED 579 state. No special handling is required further. 581 5. Security Considerations 583 The Fast Open cookie stops an attacker from trivially flooding 584 spoofed SYN packets with data to burn server resources or to mount an 585 amplified reflection attack on random hosts. The server can defend 586 against spoofed SYN floods with invalid cookies using existing 587 techniques [RFC4987]. We note that although generating bogus cookies 588 is cost-free, the cost of validating the cookies, inherent to any 589 authentication scheme, may be substantial compared to processing a 590 regular SYN packet. We describe these new vulnerabilities of TFO and 591 the countermeasures in detail below. 593 5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies 595 An attacker may still obtain cookies from some compromised hosts, 596 then flood spoofed SYN with data and "valid" cookies (from these 597 hosts or other vantage points). Like regular TCP handshakes, TFO is 598 vulnerable to such an attack. But the potential damage can be much 599 more severe. Besides causing temporary disruption to service ports 600 under attack, it may exhaust server CPU and memory resources. Such an 601 attack will show up on application server logs as an application 602 level DoS from Bot-nets, triggering other defenses and alerts. 604 To protect the server it is important to limit the maximum number of 605 total pending TFO connection requests, i.e., PendingFastOpenRequests 606 (Section 4.2). When the limit is exceeded, the server temporarily 607 disables TFO entirely as described in "Server Cookie Handling". Then 608 subsequent TFO requests will be downgraded to regular connection 609 requests, i.e., with the data dropped and only SYN acknowledged. This 610 allows regular SYN flood defense techniques [RFC4987] like SYN- 611 cookies to kick in and prevent further service disruption. 613 The main impact of SYN floods against the standard TCP stack is not 614 directly from the floods themselves costing TCP processing overhead 615 or host memory, but rather from the spoofed SYN packets filling up 616 the often small listener's queue. 618 On the other hand, TFO SYN floods can cause damage directly if 619 admitted without limit into the stack. The RST packets from the 620 spoofed host will fuel rather than defeat the SYN floods as compared 621 to the non-TFO case, because the attacker can flood more SYNs with 622 data to cost more data processing resources. For this reason, a TFO 623 server needs to monitor the connections in SYN-RCVD being reset in 624 addition to imposing a reasonable max queue length. Implementations 625 may combine the two, e.g., by continuing to account for those 626 connection requests that have just been reset against the listener's 627 PendingFastOpenRequests until a timeout period has passed. 629 Limiting the maximum number of pending TFO connection requests does 630 make it easy for an attacker to overflow the queue, causing TFO to be 631 disabled. We argue that causing TFO to be disabled is unlikely to be 632 of interest to attackers because the service will remain intact 633 without TFO hence there is hardly any real damage. 635 5.1.1 Attacks from behind Shared Public IPs (NATs) 637 An attacker behind a NAT can easily obtain valid cookies to launch 638 the above attack to hurt other clients that share the path. 639 [BRISCOE12] suggested that the server can extend cookie generation to 640 include the TCP timestamp---GetCookie(IP_Address, Timestamp)---and 641 implement it by encrypting the concatenation of the two values to 642 generate the cookie. The client stores both the cookie and its 643 corresponding timestamp, and echoes both in the SYN. The server then 644 implements IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting 645 the IP and timestamp data and comparing it with the cookie value. 647 This enables the server to issue different cookies to clients that 648 share the same IP address, hence can selectively discard those 649 misused cookies from the attacker. However the attacker can simply 650 repeat the attack with new cookies. The server would eventually need 651 to throttle all requests from the IP address just like the current 652 approach. Moreover this approach requires modifying [RFC1323] to send 653 non-zero Timestamp Echo Reply in SYN, potentially causing firewall 654 issues. Therefore we believe the benefit does not outweigh the 655 drawbacks. 657 5.2. Amplified Reflection Attack to Random Host 659 Limiting PendingFastOpenRequests with a system limit can be done 660 without Fast Open Cookies and would protect the server from resource 661 exhaustion. It would also limit how much damage an attacker can cause 662 through an amplified reflection attack from that server. However, it 663 would still be vulnerable to an amplified reflection attack from a 664 large number of servers. An attacker can easily cause damage by 665 tricking many servers to respond with data packets at once to any 666 spoofed victim IP address of choice. 668 With the use of Fast Open Cookies, the attacker would first have to 669 steal a valid cookie from its target victim. This likely requires the 670 attacker to compromise the victim host or network first. But in some 671 cases it may be relatively easy. 673 The attacker here has little interest in mounting an attack on the 674 victim host that has already been compromised. But it may be 675 motivated to disrupt the victim's network. Since a stolen cookie is 676 only valid for a single server, it has to steal valid cookies from a 677 large number of servers and use them before they expire to cause 678 sufficient damage without triggering the defense. 680 One can argue that if the attacker has compromised the target network 681 or hosts, it could perform a similar but simpler attack by injecting 682 bits directly. The degree of damage will be identical, but TFO- 683 specific attack allows the attacker to remain anonymous and disguises 684 the attack as from other servers. 686 For example with DHCP an attacker can obtain cookies when he (or the 687 host he has compromised) owns a particular IP address by performing 688 regular Fast Open to servers supporting TFO and collect valid 689 cookies. The attacker then actively or passively releases his IP 690 address. When the IP address is re-assigned to a victim, the attacker 691 now owning a different IP address, floods spoofed Fast Open requests 692 to perform an amplified reflection attack on the victim. 694 The best defense is for the server not to respond with data until 695 handshake finishes. In this case the risk of amplification reflection 696 attack is completely eliminated. But the potential latency saving 697 from TFO may diminish if the server application produces responses 698 earlier before the handshake completes. 700 6. TFO's Applicability 702 This section is to help applications considering TFO to evaluate 703 TFO's benefits and drawbacks using the Web client and server 704 applications as an example throughout. Applications here refer 705 specifically to the process that writes data into the socket. For 706 example a JavaScript process that sends data to the server. A 707 proposed socket API change is in the Appendix. 709 6.1 Duplicate Data in SYNs 711 It is possible that using TFO results in the first data written to a 712 socket to be delivered more than once to the application on the 713 remote host (Section 2.1). This replay potential only applies to data 714 in the SYN but not subsequent data exchanges. 716 Empirically [JIDKT07] showed the packet duplication on a Tier-1 717 network is rare. Since the replay only happens specifically when the 718 SYN data packet is duplicated and also the duplicate arrives after 719 the receiver has cleared the original SYN's connection state, the 720 replay is thought to be uncommon in practice. Nevertheless a client 721 that cannot handle receiving the same SYN data more than once MUST 722 NOT enable TFO to send data in a SYN. Similarly a server that cannot 723 accept receiving the same SYN data more than once MUST NOT enable TFO 724 to receive data in a SYN. Further investigation is needed to judge 725 about the probability of receiving duplicated SYN or SYN-ACK with 726 data in non-Tier 1 networks. 728 6.2 Potential Performance Improvement 730 TFO is designed for latency-conscious applications that are sensitive 731 to TCP's initial connection setup delay. To benefit from TFO, the 732 first application data unit (e.g., an HTTP request) needs to be no 733 more than TCP's maximum segment size (minus options used in SYN). 734 Otherwise the remote server can only process the client's application 735 data unit once the rest of it is delivered after the initial 736 handshake, diminishing TFO's benefit. 738 To the extent possible, applications SHOULD reuse the connection to 739 take advantage of TCP's built-in congestion control and reduce 740 connection setup overhead. An application that employs too many 741 short-lived connections will negatively impact network stability, as 742 these connections often exit before TCP's congestion control 743 algorithm takes effect. 745 6.3. Example: Web Clients and Servers 747 6.3.1. HTTP Request Replay 749 While TFO is motivated by Web applications, the browser should not 750 use TFO to send requests in SYNs if those requests cannot tolerate 751 replays. One example is POST requests without application-layer 752 transaction protection (e.g., a unique identifier in the request 753 header). 755 On the other hand, TFO is particularly useful for GET requests. GET 756 requests replay could happen across striped TCP connections: after a 757 server receives an HTTP request but before the ACKs of the requests 758 reach the browser, the browser may timeout and retry the same request 759 on another (possibly new) TCP connection. This differs from a TFO 760 replay only in that the replay is initiated by the browser, not by 761 the TCP stack. 763 6.3.2. HTTP over TLS (HTTPS) 764 For TLS over TCP, it is safe and useful to include TLS CLIENT_HELLO 765 in the SYN packet to save one RTT in TLS handshake. There is no 766 concern about violating idem-potency. In particular it can be used 767 alone with the speculative connection above. 769 6.3.3. Comparison with HTTP Persistent Connections 771 Is TFO useful given the wide deployment of HTTP persistent 772 connections? The short answer is yes. Studies [RCCJR11][AERG11] show 773 that the average number of transactions per connection is between 2 774 and 4, based on large-scale measurements from both servers and 775 clients. In these studies, the servers and clients both kept idle 776 connections up to several minutes, well into "human think" time. 778 Keeping connections open and idle even longer risks a greater 779 performance penalty. [HNESSK10][MQXMZ11] show that the majority of 780 home routers and ISPs fail to meet the 124-minute idle timeout 781 mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs silently 782 timeout idle connections within 30 minutes. End hosts, unaware of 783 silent middle-box timeouts, suffer multi-minute TCP timeouts upon 784 using those long-idle connections. 786 To circumvent this problem, some applications send frequent TCP keep- 787 alive probes. However, this technique drains power on mobile devices 788 [MQXMZ11]. In fact, power has become such a prominent issue in modern 789 LTE devices that mobile browsers close HTTP connections within 790 seconds or even immediately [SOUDERS11]. 792 [RCCJR11] studied Chrome browser [Chrome] performance based on 28 793 days of global statistics. The Chrome browser keeps idle HTTP 794 persistent connections for 5 to 10 minutes. However the average 795 number of the transactions per connection is only 3.3 and TCP 3WHS 796 accounts for up to 25% of the HTTP transaction network latency. The 797 authors estimated that TFO improves page load time by 10% to 40% on 798 selected popular Web sites. 800 6.3.4. Load Balancers and Server farms 802 Servers behind a load balancers that accept connection requests to 803 the same server IP address should use the same key such that they 804 generate identical Fast Open Cookies for a particular client IP 805 address. Otherwise a client may get different cookies across 806 connections; its Fast Open attempts would fall back to regular 3WHS. 808 7. Open Areas for Experimentation 810 We now outline some areas that need experimentation in the Internet 811 and under different network scenarios. These experiments should help 812 the community evaluate Fast Open benefits and risks towards further 813 standardization and implementation of Fast Open and its related 814 protocols. 816 7.1. Performance impact due to middle-boxes and NAT 818 [MAF04] found that some middle-boxes and end-hosts may drop packets 819 with unknown TCP options. Studies [LANGLEY06, HNRGHT11] both found 820 that 6% of the probed paths on the Internet drop SYN packets with 821 data or with unknown TCP options. The TFO protocol deals with this 822 problem by falling back to regular TCP handshake and re-transmitting 823 SYN without data or cookie options after the initial SYN timeout. 824 Moreover the implementation is recommended to negatively cache such 825 incidents to avoid recurring timeouts. Further study is required to 826 evaluate the performance impact of these drop behaviors. 828 Another interesting study is the loss of TFO performance benefit 829 behind certain carrier-grade NAT. Typically hosts behind a NAT 830 sharing the same IP address will get the same cookie for the same 831 server. This will not prevent TFO from working. But on some carrier- 832 grade NAT configurations where every new TCP connection from the same 833 physical host uses a different public IP address, TFO does not 834 provide latency benefits. However, there is no performance penalty 835 either, as described in Section "Client: Receiving SYN-ACK". 837 7.2. Impact on congestion control 839 Although TFO does not directly change the congestion control, there 840 are subtle cases that it may. When SYN-ACK times out, regular TCP 841 reduces the initial congestion window before sending any data 842 [RFC5681]. However in TFO the server may have already sent up to an 843 initial window of data. 845 If the server serves mostly short connections then the losses of SYN- 846 ACKs are not as effective as regular TCP on reducing the congestion 847 window. This could result in an unstable network condition. The 848 connections that experience losses may attempt again and add more 849 load under congestion. A potential solution is to temporarily disable 850 Fast Open if the server observes many SYN-ACK or data losses during 851 the handshake across connections. Further experimentation regarding 852 the congestion control impact will be useful. 854 7.3. Cookie-less Fast Open 856 The cookie mechanism mitigates resource exhaustion and amplification 857 attacks. However cookies are not necessary if the server has 858 application-level protection or is immune to these attacks. For 859 example a Web server that only replies with a simple HTTP redirect 860 response that fits in the SYN-ACK packet may not care about resource 861 exhaustion. 863 For such applications the server may choose to generate a trivial or 864 even a zero-length cookie to improve performance by avoiding the 865 cookie generation and verification. If the server believes it's under 866 a DoS attack through other defense mechanisms, it can switch to 867 regular Fast Open for listener sockets. 869 8. Related Work 871 8.1. T/TCP 873 TCP Extensions for Transactions [RFC1644] attempted to bypass the 874 three-way handshake, among other things, hence shared the same goal 875 but also the same set of issues as TFO. It focused most of its effort 876 battling old or duplicate SYNs, but paid no attention to security 877 vulnerabilities it introduced when bypassing 3WHS [PHRACK98]. 879 As stated earlier, we take a practical approach to focus TFO on the 880 security aspect, while allowing old, duplicate SYN packets with data 881 after recognizing that 100% TCP semantics is likely infeasible. We 882 believe this approach strikes the right tradeoff, and makes TFO much 883 simpler and more appealing to TCP implementers and users. 885 8.2. Common Defenses Against SYN Flood Attacks 887 [RFC4987] studies on mitigating attacks from regular SYN flood, i.e., 888 SYN without data. But from the stateless SYN-cookies to the stateful 889 SYN Cache, none can preserve data sent with SYN safely while still 890 providing an effective defense. 892 The best defense may be to simply disable TFO when a host is 893 suspected to be under a SYN flood attack, e.g., the SYN backlog is 894 filled. Once TFO is disabled, normal SYN flood defenses can be 895 applied. The "Security Consideration" section contains a thorough 896 discussion on this topic. 898 8.3. Speculative Connections by the Applications 900 Some Web browsers maintain a history of the domains for frequently 901 visited web pages. The browsers then speculatively pre-open TCP 902 connections to these domains before the user initiates any requests 903 for them [BELSHE11]. While this technique also saves the handshake 904 latency, it wastes server and network resources by initiating and 905 maintaining idle connections. 907 8.4. Fast Open Cookie in FIN 908 An alternate proposal is to request a TFO cookie in the FIN instead, 909 since FIN-drop by incompatible middle-boxes does not affect latency. 910 However paths that block SYN cookies may be more likely to drop a 911 later SYN packet with data, and many applications close a connection 912 with RST instead anyway. 914 Although cookie-in-FIN may not improve robustness, it would give 915 clients using a single connection a latency advantage over clients 916 opening multiple parallel connections. If experiments with TFO find 917 that it leads to increased connection-sharding, cookie-in-FIN may 918 prove to be a useful alternative. 920 8.5. TCP Cookie Transaction (TCPCT) 922 TCPCT [RFC6013] eliminates server state during initial handshake and 923 defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK 924 packets to carry data. But the server can only send up to MSS bytes 925 of data during the handshake instead of the initial congestion window 926 unlike TFO. Therefore the latency of applications such as Web may be 927 worse than with TFO. 929 9. IANA Considerations 931 IANA is requested to allocate one value from the TCP Option Kind 932 Numbers: The constant-TBD in Section 4.1.1 has to be replaced with 933 the newly assigned value. The length of the new TCP option Kind is 934 variable and the Meaning should be set to "TCP Fast Open Cookie". 935 Early implementation before the IANA allocation SHOULD follow 936 [RFC6994] and use experimental option 254 and magic number 0xF989 (16 937 bits), then migrate to the new option after the allocation 938 accordingly. 940 10. Acknowledgement 942 We thank Bob Briscoe, Michael Scharf, Gorry Fairhurst, Rick Jones, 943 Roberto Peon, William Chan, Adam Langley, Neal Cardwell, Eric 944 Dumazet, and Matt Mathis for their feedbacks. We especially thank 945 Barath Raghavan for his contribution on the security design of Fast 946 Open and proofreading this draft numerous times. 948 11. References 950 11.1. Normative References 952 [RFC793] Postel, J. "Transmission Control Protocol", RFC 793, 953 September 1981. 955 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 956 Communication Layers", STD 3, RFC 1122, October 1989. 958 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 959 Requirement Levels", BCP 14, RFC 2119, March 1997. 961 [RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh, 962 P., "NAT Behavioral Requirements for TCP", RFC 5382 964 [RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion 965 Control", RFC 5681, September 2009 967 [RFC6994] Touch, Joe, "Shared Use of Experimental TCP Options", 968 RFC6994, August 2013. 970 [RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's 971 Initial Window", RFC 3390, October 2002. 973 11.2. Informative References 975 [AERG11] Al-Fares, M., Elmeleegy, K., Reed, B., Gashinsky, I., 976 "Overclocking the Yahoo! CDN for Faster Web Page Loads". 977 In Proceedings of Internet Measurement Conference, 978 November 2011. 980 [HNESSK10] Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S., 981 Sarolahti, P., Kojo., M., "An Experimental Study of Home 982 Gateway Characteristics". In Proceedings of Internet 983 Measurement Conference. October 2010 985 [HNRGHT11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., 986 Handley, M., Tokuda, H., "Is it Still Possible to 987 Extend TCP?". In Proceedings of Internet Measurement 988 Conference. November 2011. 990 [LANGLEY06] Langley, A, "Probing the viability of TCP extensions", 991 URL http://www.imperialviolet.org/binary/ecntest.pdf 993 [MAF04] Medina, A., Allman, M., and S. Floyd, "Measuring 994 Interactions Between Transport Protocols and 995 Middleboxes". In Proceedings of Internet Measurement 996 Conference, October 2004. 998 [MQXMZ11] Wang, Z., Qian, Z., Xu, Q., Mao, Z., Zhang, M., 999 "An Untold Story of Middleboxes in Cellular Networks". 1000 In Proceedings of SIGCOMM. August 2011. 1002 [PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue 1003 53 artical 6. July 8, 1998. URL 1004 http://www.phrack.com/issues.html?issue=53&id=6 1006 [RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A., 1007 Raghavan, B., "TCP Fast Open". In Proceedings of 7th 1008 ACM CoNEXT Conference, December 2011. 1010 [RFC1323] Jacobson, V., Braden, R., Borman, D., "TCP Extensions for 1011 High Performance", RFC 1323, May 1992. 1013 [RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions 1014 Functional Specification", RFC 1644, July 1994. 1016 [RFC2460] Deering, S., Hinden, R., "Internet Protocol, Version 6 1017 (IPv6) Specification", RFC 2460, December 1998. 1019 [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common 1020 Mitigations", RFC 4987, August 2007. 1022 [RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013, 1023 January 2011. 1025 [SOUDERS11] Souders, S., "Making A Mobile Connection". 1026 http://www.stevesouders.com/blog/2011/09/21/making-a- 1027 mobile-connection/ 1029 [BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm- 1030 fastopen-01", tcpm list, 1031 http://www.ietf.org/mail-archive/web/tcpm/ 1032 current/msg07192.html 1034 [BELSHE11] Belshe, M., "The era of browser preconnect.", 1035 http://www.belshe.com/2011/02/10/ 1036 the-era-of-browser-preconnect/ 1038 [JIDKT07] Jaiswal, S., Iannaccone, G., Diot, C., Kurose, J., 1039 Towsley, D., "Measurement and classification of 1040 out-of-sequence packets in a tier-1 IP backbone.". 1041 IEEE/ACM Transactions on Networking (TON), 15(1), 54-66. 1043 [Chrome] Chrome. https://www.google.com/intl/en-US/chrome/browser/ 1045 Appendix A. Example Socket API Changes to support TFO 1047 A.1 Active Open 1049 The active open side involves changing or replacing the connect() 1050 call, which does not take a user data buffer argument. We recommend 1051 replacing connect() call to minimize API changes and hence 1052 applications to reduce the deployment hurdle. 1054 One solution implemented in Linux 3.7 is introducing a new flag 1055 MSG_FASTOPEN for sendto() or sendmsg(). MSG_FASTOPEN marks the 1056 attempt to send data in SYN like a combination of connect() and 1057 sendto(), by performing an implicit connect() operation. It blocks 1058 until the handshake has completed and the data is buffered. 1060 For non-blocking socket it returns the number of bytes buffered and 1061 sent in the SYN packet. If the cookie is not available locally, it 1062 returns -1 with errno EINPROGRESS, and sends a SYN with TFO cookie 1063 request automatically. The caller needs to write the data again when 1064 the socket is connected. On errors, it returns the same errno as 1065 connect() if the handshake fails. 1067 An implementation may prefer not to change the sendmsg() because TFO 1068 is a TCP specific feature. A solution is to add a new socket option 1069 TCP_FASTOPEN for TCP sockets. When the option is enabled before a 1070 connect operation, sendmsg() or sendto() will perform Fast Open 1071 operation similar to the MSG_FASTOPEN flag described above. This 1072 approach however requires an extra setsockopt() system call. 1074 A.2 Passive Open 1076 The passive open side change is simpler compared to active open side. 1077 The application only needs to enable the reception of Fast Open 1078 requests via a new TCP_FASTOPEN setsockopt() socket option before 1079 listen(). 1081 The option enables Fast Open on the listener socket. The option value 1082 specifies the PendingFastOpenRequests threshold, i.e., the maximum 1083 length of pending SYNs with data payload. Once enabled, the TCP 1084 implementation will respond with TFO cookies per request. 1086 Traditionally accept() returns only after a socket is connected. But 1087 for a Fast Open connection, accept() returns upon receiving a SYN 1088 with a valid Fast Open cookie and data, and the data is available to 1089 be read through, e.g., recvmsg(), read(). 1091 Authors' Addresses 1093 Yuchung Cheng 1094 Google, Inc. 1095 1600 Amphitheatre Parkway 1096 Mountain View, CA 94043, USA 1097 EMail: ycheng@google.com 1099 Jerry Chu 1100 Google, Inc. 1101 1600 Amphitheatre Parkway 1102 Mountain View, CA 94043, USA 1103 EMail: hkchu@google.com 1105 Sivasankar Radhakrishnan 1106 Department of Computer Science and Engineering 1107 University of California, San Diego 1108 9500 Gilman Dr 1109 La Jolla, CA 92093-0404 1110 EMail: sivasankar@cs.ucsd.edu 1112 Arvind Jain 1113 Google, Inc. 1114 1600 Amphitheatre Parkway 1115 Mountain View, CA 94043, USA 1116 EMail: arvind@google.com