idnits 2.17.1 draft-bagnulo-tcpm-generalized-ecn-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: If a TCP client wishes to use ECN in a given connection and it also wishes to enable the ECN marking of the TCP SYN packet, then the TCP client MUST send the TCP SYN with the ECT(0) or ECT(1) codepoints in the IP header and the NS, the CWR and the ECE flags set to 1 in the TCP header. The server SHOULD not set any of the ECT codepoints if the server is included in the cache as not supporting ECN in the SYN packet. Shall we define the lifetime of the entries and how are they extended? The client SHOULD use an initial retransmission timeout value of 500ms or less for this connection. -- The document date (September 29, 2016) is 2737 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-28) exists of draft-ietf-tcpm-accurate-ecn-01 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Bagnulo 3 Internet-Draft UC3M 4 Intended status: Informational B. Briscoe 5 Expires: April 2, 2017 Simula Research Lab 6 September 29, 2016 8 Adding Explicit Congestion Notification (ECN) to TCP control packets and 9 TCP retransmissions 10 draft-bagnulo-tcpm-generalized-ecn-00 12 Abstract 14 This document describes an experimental modification to ECN to allow 15 the use of ECN to the following TCP packets: SYNs, Pure ACKs, Window 16 probes, FINs, RSTs and retransmissions. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at http://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on April 2, 2017. 35 Copyright Notice 37 Copyright (c) 2016 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (http://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 3 54 1.2. Experiment goals . . . . . . . . . . . . . . . . . . . . 3 55 1.3. Document structure . . . . . . . . . . . . . . . . . . . 4 56 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 57 3. Specification . . . . . . . . . . . . . . . . . . . . . . . . 5 58 3.1. Network behaviour . . . . . . . . . . . . . . . . . . . . 5 59 3.2. Endpoint behaviour . . . . . . . . . . . . . . . . . . . 6 60 3.2.1. SYN . . . . . . . . . . . . . . . . . . . . . . . . . 6 61 3.2.2. Pure ACK . . . . . . . . . . . . . . . . . . . . . . 8 62 3.2.3. Window Probe . . . . . . . . . . . . . . . . . . . . 9 63 3.2.4. FIN . . . . . . . . . . . . . . . . . . . . . . . . . 10 64 3.2.5. RST . . . . . . . . . . . . . . . . . . . . . . . . . 10 65 3.2.6. Retransmissions . . . . . . . . . . . . . . . . . . . 11 66 4. Discussion about the arguments in RFC3168 . . . . . . . . . . 12 67 4.1. The reliability argument . . . . . . . . . . . . . . . . 12 68 4.2. TCP SYNs . . . . . . . . . . . . . . . . . . . . . . . . 13 69 4.3. Pure ACKs. . . . . . . . . . . . . . . . . . . . . . . . 16 70 4.4. Retransmitted packets. . . . . . . . . . . . . . . . . . 18 71 4.5. Window probe packets . . . . . . . . . . . . . . . . . . 20 72 5. Security considerations . . . . . . . . . . . . . . . . . . . 21 73 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 74 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 75 8. Informative References . . . . . . . . . . . . . . . . . . . 21 76 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 78 1. Introduction 80 RFC3168 [RFC3168] specifies the support of Explicit Congestion 81 Notification (ECN) to IP. By using the ECN capability, switches 82 performing Active Queue Management (AQM) can use ECN marks instead of 83 packets drops to signal congestion to the endpoints of a 84 communication. This results in lower packet loss and increased 85 performance. However, RFC3168 specifies the support of ECN in TCP 86 data packets, but precludes the use of ECN in TCP control packets 87 (TCP SYN, TCP SYN/ACK, pure ACKs, Window probes) and in retransmitted 88 packets. RFC3168 is silent about the use of ECN in RST and FIN 89 packets. RFC 5562 [RFC5562] is an experimental extension to ECN that 90 enables the ECN support for TCP SYN/ACK packets. This document 91 defines an experimental modification to ECN that enables the ECN 92 support in all the aforementioned packet types. 94 1.1. Motivation 96 The inability of using ECN in TCP control packets and retransmissions 97 has a potential harmful effect, especially in environments where ECN 98 support is pervasive. For example, [judd-nsdi] shows that in a data 99 center (DC) environment where DCTCP is used (in conjunction with 100 ECN), the the probability of being able to establish a new connection 101 using a non-ECT-marked SYN packet drops to close to 0 when there are 102 16 ongoing TCP flows transmitting at full speed. In this particular 103 context of a datacenter using DCTCP, the issue is that the proposed 104 AQM aggressively marks packets to keep the buffer queues small and 105 this implies that non-ECT-marked packets are in turn dropped 106 aggressively as well, rendering nearly impossible to establish new 107 connection when there is ongoing traffic. 109 These limitations are not limited to the data center environment. In 110 any ECN deployment, non ECT marked packets suffer a penalty when they 111 traverse a congested bottleneck. For instance, with a drop 112 probability of 1%, 1% of connection attempts suffer a timeout before 113 the SYN is retransmitted, which is very detrimental to the 114 performance of short flows. Dropping TCP control traffic, such as 115 TCP SYNs and pure ACKs have a negative effect on the overall 116 performance of the communication, so it is beneficial to avoid it. 118 Finally, there are ongoing efforts to promote the adoption of DCTCP 119 (and similar transports) over the Internet to achieve low latency for 120 all communications [I-D.briscoe-tsvwg-aqm-tcpm-rmcat-l4s-problem]. 121 In such approach, ECN capable packets are treated more favorably, as 122 they are likely to experience less delay and lower packet drop 123 probability. Preventing TCP control packets, which are critical for 124 TCP performance, to obtain the benefits of ECN would result in 125 degraded performance. 127 1.2. Experiment goals 129 The goal of the experimental extensions defined in this document is 130 to allow the use of ECN (both ECT and CE codepoints) in the public 131 Internet as well as in controlled environments so we can find out 132 about the following issues: 134 How SYN, Window probes, pure ACKs, FINs, RSTs and retransmissions 135 that carry the ECT(0), ECT(1) or CE codepoints are processed by 136 the TCP endpoints and the network (including routers, firewalls 137 and other middleboxes). In particular we would like to learn if 138 these packets are frequently blocked or if these packets are 139 usually forwarded and processed. This will affect the design of 140 the support of the different packet types considered. 142 The scale of deployment of the different flavors of ECN, including 143 [RFC3168], [RFC5562], [RFC3540] and [I-D.ietf-tcpm-accurate-ecn]. 144 Depending of how pervasive is the deployment of each option, the 145 design of adding ECN support to the different packet types 146 considered in this document can vary greatly. 148 How much the performance of the TCP communications is improved by 149 allowing the ECN marking of these packets, for each of the 150 different packet types. 152 Identify any issues (including security issues) that enabling the 153 ECN marking of these packets may imply. 155 The data gathered through the experiments described in this document 156 will help in the design of the final mechanism (if any) to add ECN 157 support to the different packet types considered in this document. 158 Whenever data input is needed to assist in a design choice, it is 159 spelled out throughout the document. 161 Success criteria: If we manage to obtain enough data to have a 162 clearer view of the deployability, the benefits and any other issues 163 of ECN marking of the considered packets, the experiment will be a 164 success. If the results of the experiment show that it is feasible 165 to deploy such changes, there are gains to be achieved though the 166 changes described in this specification and no other major issues 167 that may interfere with the deployment of the proposed changes, then 168 it would be reasonable to attempt to update RFC3168 to adopt the 169 proposed changes in a standards track specification. 171 1.3. Document structure 173 The remaining of this document is structured as follows. In section 174 Section 2, we present the terminology used in the rest of the 175 document. In section Section 3, we specify the modifications to 176 provide ECN support to TCP SYNs, pure ACKs, Window probes, FINs, RSTs 177 and retransmissions. We describe both the network behaviour and the 178 endpoint behaviour (this last one detailed for both the TCP sender 179 and TCP receiver). RFC3168 does not prevents from using ECN in TCP 180 control packets lightly. It provides a number of specific reasons 181 for each packet type. In this Section 4, we revisit each of the 182 arguments provided by RFC3168 and explore possibilities to enable the 183 ECN capability in the different packet types. 185 2. Terminology 187 The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, 188 SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL, when they appear in this 189 document, are to be interpreted as described in [RFC2119]. 191 Pure ACK: A TCP segment with the ACK flag set and no data payload. 193 SYN: A TCP segment with the SYN flag set. It may carry data if TCP 194 Fast Open is used. 196 Window probe: Defined in [RFC1122], Window probe is a TCP segment 197 with only one byte of data sent to learn is the the receiver window 198 is still zero or else. 200 FIN: A TCP segment with the FIN flag set. 202 RST: A TCP segment with the RST flag set. 204 Retransmission: A TCP segment that has been retransmitted by the TCP 205 sender because it determined that the original segment was lost, 206 which may or may not be the case. 208 3. Specification 210 3.1. Network behaviour 212 If a router or any other middlebox along the path, receives a SYN, or 213 a Window probe or a Pure ACK or FIN or a RST or a Retransmission with 214 the ECT(0) or the ECT(1) codepoint set, then: 216 if the router is not congested (i.e. if in the same situation and 217 if the received packet was the same packet but with the non-ECT 218 codepoint, then the router would forward such packet), the router 219 SHOULD forward the packet. 221 It could be possible to use a MUST instead of a SHOULD. The 222 reason to use a SHOULD is that it may be the case that some 223 firewalls or other middleboxes may decide to block some of 224 these packets, such as ECT marked SYNs if they detect an 225 ongoing attack, which would then qualify the SHOULD and would 226 allow these boxes to drop the packets. This issue is to be 227 discussed. 229 if the router is congested, (i.e. if in the same situation and if 230 the received packet was the same packet but with the non-ECT 231 codepoint, then the router would drop the packet), then the router 232 MAY set the CE codepoint in the packet instead of dropping the 233 packet. If other behaviour alternative to drop is defined in 234 another specification (e.g. for ECT(1)) then this text should be 235 updated to allow this alternative behaviour with a proper link to 236 that specification. 238 3.2. Endpoint behaviour 240 3.2.1. SYN 242 A first design choice that needs to be taken when proposing the 243 experimental extensions to support ECN marking in SYN packets is 244 whether the ECN marking of the SYN packets is done only for AccECN 245 endpoints [I-D.ietf-tcpm-accurate-ecn] or also RFC3168 endpoints. 247 As far as AccECN capable endpoints, [I-D.ietf-tcpm-accurate-ecn] 248 defines the wire protocol for supporting ECN marking of the SYN as 249 well as feeding back the congestion signal to the sender if a SYN 250 packet with the CE codepoint was delivered to the receiver. In the 251 mechanism described next follows the wire format defined in 252 [I-D.ietf-tcpm-accurate-ecn] and also complements it with the ECT 253 marking of the SYN as well as the endpoint behaviour both of which 254 are left undefined in [I-D.ietf-tcpm-accurate-ecn]. 256 The mechanism described below does not support ECN marking of SYNs 257 with RFC3168 endpoints. Whether this is needed is left for 258 discussion in the WG. 260 3.2.1.1. TCP client behaviour 262 In this section we specify the behaviour of a TCP client that wishes 263 to support ECN marking in the SYN. The proposed behaviour is fully 264 compliant with [I-D.ietf-tcpm-accurate-ecn]. We call W0 the default 265 value for the Initial Window supported by the TCP endpoint. 266 According to current specifications, W0 can be any value between 2 267 and 10. A TCP endpoint supporting this specification SHOULD have a 268 cache where it stores information about the type of ECN support 269 (RFC3168 ECN, AccECN, ECN in the SYN, no-ECN). 271 If a TCP client wishes to use ECN in a given connection and it also 272 wishes to enable the ECN marking of the TCP SYN packet, then the TCP 273 client MUST send the TCP SYN with the ECT(0) or ECT(1) codepoints in 274 the IP header and the NS, the CWR and the ECE flags set to 1 in the 275 TCP header. The server SHOULD not set any of the ECT codepoints if 276 the server is included in the cache as not supporting ECN in the SYN 277 packet. Shall we define the lifetime of the entries and how are they 278 extended? The client SHOULD use an initial retransmission timeout 279 value of 500ms or less for this connection. 281 DISCUSSION: Should (may?) the sender send also a SYN with the non- 282 ECT codepoint, perhaps slightly delayed after this first one? 283 This would reduce the penalty of adoption the ECN marking of SYNs 284 when communicating with TCP receivers that silently drop ECT SYNs. 285 In the text below, we first consider the case where the client 286 only sends the ECT marked SYN and then we consider the case where 287 the client sends first an ECT marked SYN followed by a regular 288 SYN. 290 After sending the ECT marked SYN, the client may receive the 291 following different replies and will react as follows: 293 If the client receives a SYN/ACK with the CWR and the ECE flags set 294 to 1 and the NS flag set to 0, (this means that the server supports 295 AccECN and that the SYN was not marked with the CE codepoint while 296 being forwarded though the network) then the client must continue 297 with the connection establishment using an Initial Window of W0 and 298 it must use AccECN for this connection (as defined in 299 [I-D.ietf-tcpm-accurate-ecn]). The client SHOULD cache that this 300 server supports ECN in the SYN packet and AccECN. 302 If the client receives a SYN/ACK with the CWR, the ECE and the NS 303 flags set to 1, (this means that the server supports AccECN and that 304 the SYN was marked with the CE codepoint while being forwarded though 305 the network) then the client must continue with the connection 306 establishment using an Initial Window of 1 SMSS and it must use 307 AccECN for this connection (as defined in 308 [I-D.ietf-tcpm-accurate-ecn]).The client SHOULD cache that this 309 server supports ECN in the SYN packet and AccECN. 311 If the client receives a SYN/ACK with the ECE flag set to 1 and the 312 CWR flag set to 0 (the value of the NS flag can be either 1 or 0), 313 (this means that the server supports RFC3168 ECN but does not support 314 AccECN nor this specification) then the client must continue with the 315 connection establishment and it must use RFC3168 ECN for this 316 connection The client SHOULD cache that this server does not support 317 ECN in the SYN packet nor AccECN but it supports RFC3168 ECN. 319 DISCUSSION: Initial Window. Because the server does not support 320 ECT marking of the SYN message, it is not possible for the client 321 to know if the SYN was marked with the CE codepoint while 322 transiting to the server. We can either assume that the SYN was 323 marked with CE and impose an initial window of 1 MSS, or we can 324 assume it was not marked and use an initial window of W0 or 325 something in between (e.g. W0/2). If we impose a smaller initial 326 window, we are penalizing those that implement this specification, 327 which is in itself an optimization. Also, caching will help in 328 this case, as after the first contact a client will use the 329 appropriate mode to contact a server. 331 If the client receives a SYN/ACK with the ECE, the CWR and the NS 332 flags set to 0, (this means that the server does not support ECN) 333 then the client must continue with the connection establishment and 334 it must not use ECN for this connection The client SHOULD cache that 335 this server does not support ECN. 337 DISCUSSION: Same discussion about the Initial Window as in the 338 previous case. 340 If the client receives a SYN/ACK with the ECE, the CWR and the NS 341 flags set to 1, (this means that the server does not support ECN and 342 it is not compliant with current standards) then the client may 343 continue with the connection establishment and it must not use ECN 344 for this connection The client SHOULD cache that this server does not 345 support ECN. See Appendix B of [I-D.ietf-tcpm-accurate-ecn] for 346 ideas about how these tow last cases can be used. 348 DISCUSSION: Same discussion about the Initial Window as in the 349 previous case. 351 If the client receives a SYN, then the client must behave as defined 352 in [I-D.ietf-tcpm-accurate-ecn], which implies sending some form of 353 SYN/ACK. After doing so, the client may receive some form os SYN/ACK 354 back. the processing of the SYN/ACK will follow the rules specified 355 above. This is the case of simultaneous open. 357 If the retransmission timer times out, then the client SHOULD send a 358 TCP SYN with the No-ECT codepoint in the IP header. The initial 359 retransmission timeout MUST be set to 1 sec. 361 3.2.1.2. TCP server 363 The TCP server behaviour is defined in [I-D.ietf-tcpm-accurate-ecn]. 364 In addition, the TCP server SHOULD cache the type of endpoint of the 365 client, for future connections. 367 3.2.2. Pure ACK 369 3.2.2.1. TCP sender behaviour 371 A TCP endpoint MAY set the ECT(0) or the ECT(1) codepoints in the IP 372 header of packets carrying a TCP pure ACK. 374 In the case that a TCP endpoint is only sending pure ACKs and it 375 decides to mark them as ECT, the endpoint may receive a congestion 376 signal back indicating that one or more of the pure ACKs it has sent 377 have experienced congestion. when this happens, the endpoint will 378 react in the same way than it would if any other packet has 379 experienced congestion i.e. it will reduce its congestion window 380 accordingly. However, if the endpoint is only sending pure ACKs this 381 will have no effect in the load offered to the network and hence it 382 will not help in reducing the congestion. It may be possible to 383 explore some ways to help reducing congestion in this scenario. For 384 instance, one possibility would be for the endpoint to increase the 385 maximum number of data packets that the endpoint can wait until 386 sending a delayed ACK. Current specifications (RFC 1122 and RFC 387 5681) mandate that at most one ACK must be sent every two full-sized 388 segments. Upon congestion notification, the endpoint could increase 389 the number of segments required to send an ACK (while preserving the 390 timeout value unchanged). This would reduce the number of ACKs sent 391 by the endpoint and hence the offered load. It is up for discussion 392 in the WG if it is worth it. Also, it should be noted than if an ACK 393 is dropped due to congestion the sender of the ACK does not react by 394 reducing the load in any way. 396 3.2.2.2. TCP receiver behaviour 398 Upon reception of a pure ACK with the ECT(0), ECT(1) or CE 399 codepoints, the TCP receiver will process it as if it were any other 400 legitimate packet (e.g. a data packet). The exact treatment depends 401 on the flavour of ECN that the endpoint implements (either RFC3168 or 402 AccECN). In particular for AccECN, a CE marked pure ACK would 403 increase the CE packet counter and would not increase the CE byte 404 counter. 406 3.2.3. Window Probe 408 3.2.3.1. TCP sender behaviour 410 A TCP endpoint MAY set the ECT(0) or the ECT(1) codepoints in the IP 411 header of a packet carrying a zero window probe (ZWP) packet. 413 According to RFC793, a TCP endpoint that has an ongoing connection 414 for which the other endpoint has announced a receiver window equal to 415 zero will send periodic ZWP every two minutes until a non zero window 416 is announced by the other endpoint of the connection. In case a TCP 417 endpoint that has an ongoing connection for which the other endpoint 418 has announced a receiver window equal to zero and it receives 419 incoming packets that include a congestion notification signal, the 420 only option for the endpoint to reduce the offered load is to 421 increase the time between ZWP messages e.g. do an exponential back- 422 off or increment the retransmission timer in some other way. 423 However, given that the current retransmission timer is pretty long, 424 it is not clear if this is an effective decrease of the offered load 425 from a congestion perspective. Note that the endpoint receiving the 426 congestion notification will reduce its congestion window, so that 427 when the receiver window opens, it will transmit with such a 428 congestion window. 430 3.2.3.2. TCP receiver behaviour 432 Upon reception of a ZWP with the ECT(0), ECT(1) or CE codepoints, the 433 TCP receiver will process it as if it were any other legitimate 434 packet (e.g. a data packet). The exact treatment depends on the 435 flavour of ECN that the endpoint implements (either RFC3168 or 436 AccECN). 438 3.2.4. FIN 440 3.2.4.1. TCP sender behaviour 442 A TCP endpoint MAY set the ECT(0) or the ECT(1) codepoints in the IP 443 header of a packet carrying the FIN flag of the TCP header set. 445 After sending the FIN, the endpoint will not send any more data in 446 the connection. It may send one or more pure ACKs, so if the 447 endpoint that has set the ECT codepoint in the FIN receives feedback 448 from the other endpoint that the FIN was receives with the CE 449 codepoint, there is little it can do to reduce the load offered to 450 the network. It is pointless to reduce the congestion window as the 451 endpoint will not send any more data. It can try to reduce the 452 amount of pure ACKs it sends, by using a similar approach as the one 453 suggested in Section 3.2.2 about incrementing the number of ACKs 454 accumulated before sending a delayed ACK. 456 3.2.4.2. TCP receiver behaviour 458 Upon reception of a FIN with the ECT(0), ECT(1) or CE codepoints, the 459 TCP receiver will process it as if it were any other legitimate 460 packet (e.g. a data packet). The exact treatment depends on the 461 flavour of ECN that the endpoint implements (either RFC3168 or 462 AccECN). 464 3.2.5. RST 466 A RST message is hardly a useful vehicle to convey congestion 467 notification information. The reason for this is that the endpoint 468 generating the RST message does not have an open connection after 469 sending it (either because there was no such connection when the 470 packet that triggered the RST message was received or because the 471 packet that triggered the RST message also triggered the closure of 472 the connection). So, if a congestion notification signal is fed back 473 to the sender to the RST message, the sender will not be able to do 474 anything about it. Moreover, the the perspective of the receiver of 475 the RST message with the CE bit set, it can either accept the RST 476 message and close the connection, so there is no point in echoing the 477 congestion notification signal received or it can discard the RST 478 message (e.g. because the sequence number is out of window) so it 479 probably makes sense also to discard the CE signal as well. So, from 480 the receiver perspective, there is no reaction to the reception of a 481 CE marked RST message. 483 So, the only motivation for marking the RST message with the ECT 484 codepoint is to reduce the chances of the RST message getting 485 dropped. The question whether it is useful to provide more reliable 486 delivery of RST messages is also non trivial. RST messages are used 487 to both create and mitigate attacks. Spoofed RST messages are used 488 by attackers to terminate ongoing connections. Legitimate RST 489 messages allow endpoints to inform their peers to eliminate existing 490 state that correspond to non existing connections, liberating 491 resources e.g. in DoS attacks scenarios. 493 So, with all this, probably the recommendation should be that: 495 for senders, stacks MUST allow for administrators to configure 496 whether the RST messages are marked with the ECT(0) or ECT(1) 497 codepoints. We should define a default behaviour, not sure which 498 that one should be. 500 for receivers, ECT and CE codepoints are ignored. 502 3.2.6. Retransmissions 504 3.2.6.1. TCP sender behaviour 506 A TCP endpoint MAY set the ECT(0) or the ECT(1) codepoints in the IP 507 header of a packet carrying a retransmitted segment. 509 Upon reception of congestion notification that the retransmitted 510 packet was marked with CE, the sender will react as with it would do 511 if it received congestion notification feedback concerning any other 512 data packet. 514 3.2.6.2. TCP receiver behaviour 516 The receiver of a retransmitted packet marked with the ECT(0), ECT(1) 517 or CE codepoints, reacts as it would do with any other data packet. 518 In particular, the condition of ignoring ECN information for packets 519 outside the receiver window still hold. This means that for those 520 retransmitted packets that the original packet was properly received, 521 the ECN information will be ignored. There is no problem with that, 522 since allowing the ECN marking of retransmitted packets still 523 increases the reliability of their transmission. 525 4. Discussion about the arguments in RFC3168 527 This section goes through each of the arguments presented in RFC3168 528 to prevent the ECN marking of the different packet types and provides 529 counter-arguments for each of them. 531 4.1. The reliability argument 533 While for each type of packet RFC 3168 provides a set of specific 534 arguments for preventing their marking, RFC3168 presents the reliable 535 delivery of the congestion signal as an overarching argument that 536 needs to be consider when trying to enable the ECT marking of TCP 537 control packets. In particular, Section 5.2 of RFC3168 states: 539 To ensure the reliable delivery of the congestion indication of 540 the CE codepoint, an ECT codepoint MUST NOT be set in a packet 541 unless the loss of that packet in the network would be detected by 542 the end nodes and interpreted as an indication of congestion. 544 We believe this argument is overly conservative. The overall 545 principle that should determine the level of reliability required for 546 ECN capable packets should be the one of "do not harm". Reliable 547 delivery of the CE codepoint is indeed paramount but the level of 548 reliability required should be the one of the original congestion 549 signal (i.e. the detection of the loss of the original packet). In 550 other words, the situation without ECN is that when a packet is to be 551 transmitted through a congested link, the packet may be dropped and 552 that is the congestion signal sent to the endpoint. When ECN is 553 introduced, the reliability of the delivery of the congestion signal 554 should be no worse than without ECN. In particular, setting the CE 555 codepoint in the very same packet seem to fulfill this criteria, 556 since either the packet is delivered and the CE codepoint signal is 557 delivered to the endpoint, or the packet is dropped, so the original 558 congestion signal through the packet loss is delivered to the 559 endpoint. Requiring more than this implies that the ECN congestion 560 signal is delivered more reliably than the current situation, which 561 is not a bad thing per se, but, as we describe in this memo, it 562 results in performance penalties that should be reconsidered in the 563 view of current deployments. 565 In addition, the reliability of the delivery of the congestion signal 566 is used an argument for not setting the ECT codepoint in TCP control 567 packets, which effectively reduced the reliability of the 568 transmission of these TCP control packets. There is the then a 569 tradeoff between the reliability of the delivery of the congestion 570 signal and the reliability of the delivery of TCP control packets. 571 As currently specified, ECN adoption implies an increased reliability 572 of the ECN congestion signal and a decrease in the reliability in the 573 TCP control packets. We believe that it is possible and desirable to 574 restore the tradeoff existent in non ECN capable networks in terms of 575 reliability, where the congestion signal delivery is as reliable as 576 in a non ECN capable network and so it is the delivery of TCP control 577 packets. 579 4.2. TCP SYNs 581 We next describe he arguments given by current specifications for 582 precluding ECT on SYN packets. 584 RFC 5562 presents two arguments against ECT marking of SYN packets 585 (quoted verbatim): 587 There are several reasons why an ECN-Capable codepoint must not be 588 set in the IP header of the initiating TCP SYN packet. First, 589 when the TCP SYN packet is sent, there are no guarantees that the 590 other TCP endpoint (node B in Figure 2) is ECN-Capable, or that it 591 would be able to understand and react if the ECN CE codepoint was 592 set by a congested router. 594 Second, the ECN-Capable codepoint in TCP SYN packets could be 595 misused by malicious clients to "improve" the well-known TCP SYN 596 attack. By setting an ECN-Capable codepoint in TCP SYN packets, a 597 malicious host might be able to inject a large number of TCP SYN 598 packets through a potentially congested ECN-enabled router, 599 congesting it even further. 601 We next go through all the arguments stated above to enable ECT 602 marking of SYN packets. 604 Argument 1: Unknown ECN capability at the responder. The initiator 605 does not know what the responder will do if an ECT or CE SYN arrives. 607 In a controlled environment, this argument does not hold because the 608 administrator can make sure that servers support ECN and in 609 particular ECN-capable SYN packets. Examples of controlled 610 environments are single-tenant DCs, and possibly multi-tenant DCs if 611 we assume that each tenant mostly communicates with its own VMs. 613 However, in the public Internet context, it cannot be assumed that 614 all TCP responders support ECN, and much less that they support ECT 615 marked SYN packets. It is possible that the responder will check 616 that the SYN complies with RFC 3168, which says a host "MUST NOT" set 617 ECT on a SYN. RFC 3168 does not say what the responder should do if 618 an ECN-capable SYN arrives. Some implementation might ignore the SYN 619 (either silently or by returning a RST). Also some middleboxes (e.g. 621 firewalls) might take either of these actions on behalf of the 622 responder. 624 Silent losses lead to much longer delays than resets by the following 625 reasoning. The responder sends a reset immediately, then the 626 initiator falls back to retransmitting a non-ECT SYN (and possibly 627 falls back from negotiating ECN in the TCP flags as well). However, 628 after a silent discard, the initiator has to wait longer for a 629 timeout. Then it might immediately fall back to retransmitting a 630 non-ECT SYN, or it might retransmit an unchanged SYN first, in case 631 the loss was simply due to congestion. 633 Ironically, the benefit of making SYNs ECN-capable is to avoid the 634 delays when SYNs are lost due to congestion. Policy-based discard of 635 ECN-capable SYNs would merely replace congestion as a cause of these 636 delays. So for ECT SYNs to be worthwhile it seems that the 637 percentage loss due to policy would have to be less than that due to 638 congestion. However, unlike congestion loss, policy loss is 639 predictable, so the initiator can avoid it by caching those sites 640 that do not support ECN-capable SYNs. 642 According to a study using 2014 data [ecn-pam] from a limited range 643 of vantage points, out of the top 1M Alexa web sites, 4791 (0,82%) 644 IPv4 sites and 104 (0,61%) IPv6 sites failed to establish a 645 connection when they received a TCP SYN with any ECN codepoint set in 646 the IP header and the appropriate ECN flags in the TCP header. Of 647 these, about 41% failed to establish a connection due to the ECN 648 flags in the TCP header even with a Not-ECT ECN field in the IP 649 header (i.e. despite full compliance with RFC 3168). 651 One option, would be to first send an ECT SYN and then a non-ECT SYN 652 (possibly with a small delay between them) and only accept the non- 653 ECT connection if it returned first. Nonetheless, even a cache of a 654 dozen or so sites would avoid performance problems with roughly the 655 Alexa top thousand, so it is questionable whether the level of 656 failure of ECT on SYNs warrants always sending two SYNs, particularly 657 given failures at well-maintained sites could reduce if ECT SYNs are 658 standardized. 660 Argument 2: Loss of congestion notification in the SYN packet due to 661 lack of support from the responder. If an ECT SYN packet is marked 662 as CE by a congested router along the path but the responder cannot 663 feed back CE marks on SYN packets, the congestion information will be 664 lost. 666 Currently, neither the TCP nor the DCTCP protocol provides space in 667 the SYN/ACK to send feed back in response CE on the SYN. The problem 668 is that there are two mutually exclusive uses of ECE on the SYN/ACK: 670 i) the responder has to set ECE=0 to agree to use ECN as part of the 671 3-way hand-shake; ii) both TCP and DCTCP use ECE=1 to feed back CE. 673 The accurate ECN (AccECN) proposal [I-D.ietf-tcpm-accurate-ecn] 674 suggests a two-pronged solution to this problem. First AccECN 675 provides a way for the responder to feed back whether there was CE on 676 the SYN, and second AccECN introduces a different combination of TCP 677 header flags on the SYN/ACK so that the initiator knows whether or 678 not the responder supports AccECN. Then if the responder does 679 indicate that it supports AccECN the initiator can be sure that, if 680 there is no CE feedback on the SYNACK, then there really was no CE on 681 the SYN. 683 If the responder's SYN/ACK shows that it does not support AccECN, the 684 initiator can take a conservative approach and assume the SYN was 685 marked with CE and reduce its initial window. However, the initiator 686 knows that congestion is not pathological enough for a router to have 687 had to turn off ECN, because it knows that both the SYN and the SYN/ 688 ACK have been delivered through the network. Therefore, even a 689 conservative initiator would not have to reduce its initial window as 690 much as it would in response to a timeout following no response to 691 its SYN. 693 Nonetheless, even a slight conservative reduction in initial window 694 might be a significant penalty, especially in the early days of 695 deployment, when little support for ECT SYN packets will be 696 available. This could be mitigated by caching previous experience of 697 which servers support AccECN. 699 Argument 3: DoS attacks. [RFC5562] says that ECT SYN packets could 700 be misused by malicious clients to augment "the well-known TCP SYN 701 attack". It goes on to say "a malicious host might be able to inject 702 a large number of TCP SYN packets through a potentially congested 703 ECN-enabled router, congesting it even further." 705 We assume this is a reference to the TCP SYN flood attack (see 706 https://en.wikipedia.org/wiki/SYN_flood), which is an attack against 707 a responder end point. We assume the idea of this attack is to use 708 ECT to get more packets through an ECN-enabled router in preference 709 to other non-ECN traffic so that they can go on to use the SYN 710 flooding attack to inflict more damage on the responder end point. 711 This argument could apply to flooding with any type of packet, but we 712 assume SYNs are singled out because their source address is easier to 713 spoof, whereas floods of other types of packets are easier to block . 715 Mandating Not-ECT in an RFC does not stop attackers using ECT for 716 flooding. Nonetheless, if a standard says SYNs are not meant to be 717 ECT it would make it legitimate for firewalls to discard them. 719 However this would negate the considerable benefit of ECT SYNs for 720 compliant transports and seems unnecessary because RFC 3168 already 721 provides the means to address this concern. In section 7 is says 722 that an AQM MUST turn off ECN support if under persistent overload, 723 and this advice is repeated in [RFC7567] (section 4.2.1). This makes 724 it hard for flooding packets to gain from ECT, but more experiments 725 are needed to see how much might be gained by an attacker flying 726 "just under the radar". 728 Alternative behaviour. The initiator can set ECT on a SYN as long as 729 it also negotiates for the use of AccECN [I-D.ietf-tcpm-accurate-ecn] 730 and as long as it conservatively reduces its initial window if the 731 SYN/ACK shows that the responder does not support AccECN. The 732 reduction in initial window need not be as great as that required in 733 response to a timeout, because the return of a SYN/ACK proves that 734 congestion is not severe. In controlled environments like data 735 centres, universal support for AccECN could be arranged. 737 Further experiments are needed to test how much malicious hosts can 738 use ECT to augment flooding attacks without triggering AQMs to turn 739 off ECN support (as mandated by RFC 3168 and RFC 7567). If it is 740 found that ECT can only slightly augment flooding attacks, the risk 741 of such attacks will need to be weighed against the performance 742 benefits of ECT SYNs. 744 4.3. Pure ACKs. 746 RFC3168 gives the following arguments for not allowing the ECT 747 marking of pure ACKs (ACKs not piggy-backed on data). In section 5.2 748 it reads: 750 To ensure the reliable delivery of the congestion indication of 751 the CE codepoint, an ECT codepoint MUST NOT be set in a packet 752 unless the loss of that packet in the network would be detected by 753 the end nodes and interpreted as an indication of congestion. 755 Transport protocols such as TCP do not necessarily detect all 756 packet drops, such as the drop of a "pure" ACK packet; for 757 example, TCP does not reduce the arrival rate of subsequent ACK 758 packets in response to an earlier dropped ACK packet. Any 759 proposal for extending ECN- Capability to such packets would have 760 to address issues such as the case of an ACK packet that was 761 marked with the CE codepoint but was later dropped in the network. 762 We believe that this aspect is still the subject of research, so 763 this document specifies that at this time, "pure" ACK packets MUST 764 NOT indicate ECN-Capability. 766 Later on, in section 6.1.4 it reads: 768 For the current generation of TCP congestion control algorithms, 769 pure acknowledgement packets (e.g., packets that do not contain 770 any accompanying data) MUST be sent with the not-ECT codepoint. 771 Current TCP receivers have no mechanisms for reducing traffic on 772 the ACK-path in response to congestion notification. Mechanisms 773 for responding to congestion on the ACK-path are areas for current 774 and future research. (One simple possibility would be for the 775 sender to reduce its congestion window when it receives a pure ACK 776 packet with the CE codepoint set). For current TCP 777 implementations, a single dropped ACK generally has only a very 778 small effect on the TCP's sending rate. 780 We next address each of the arguments presented above. 782 The first argument is about lack of reliability while conveying 783 congestion notification information when carried in pure ACKs. This 784 is the specific instance for the pure ACK messages of the reliability 785 argument discussed in Section 4.1. In some cases, the loss of pure 786 ACKs is not detected by the endpoints, losing the congestion 787 notification information indadvertedly if it was to be carried in 788 those packets. As we argued before, the bar for deciding if a packet 789 can be marked with the ECT codepoint i.e. if it is suitable for 790 carrying congestion notification information is that the congestion 791 signal communication should be as reliable as dropping the packet. 792 After all, the alternative of setting the CE bit in the packet is 793 dropping the packet. So, the question is whether carrying congestion 794 information in a pure ACK conveys the congestion information as 795 reliably as when the pure ACK is dropped and it is obvious that the 796 answer to that question is clearly yes. If the pure ACK carrying the 797 ECT and the CE bits set is later dropped by the network, it will be 798 essentially falling back to the use of drop as congestion signal. 800 The second argument given in RFC3168 is the lack of means in a sender 801 of pure ACKs to reduce the load that is creating the congestion. 802 Again, marking pure ACKs with the ECT codepoint to allow them to 803 carry congestion marks would be no worse than not doing so (and it 804 would be detrimental from a performance perspective). The TCP 805 receiver does not ACK pure ACKs so the sender of the pure ACK will 806 receive no echo of any congestion notification. However, this is no 807 worse than if a pure ACK is dropped, which cannot even be detected by 808 the remote end. 810 The proposed AccECN modification to TCP feedback 811 [I-D.ietf-tcpm-accurate-ecn] involves a data receiver repeatedly 812 sending a count of received congestion marks. So AccECN could 813 include marks on pure ACKs in this count, even though it does not ACK 814 pure ACKs themselves. Nonetheless, if the original sender of the 815 pure ACK does not respond to this feedback, or if it is decided that 816 AccECN will not provide this information, it will still make sense to 817 set ECT on pure ACKs, because the congestion situation will be no 818 worse than it is today with non-ECT pure ACKs. 820 So, overall, we believe that in terms of conveying and reacting to 821 congestion, allowing ECT (and CE) to be set on pure ACKs is no worse 822 than not doing so (and dropping the pure ACK). ANd not setting ECT 823 on pure ACKs is certainly detrimental to performance because when a 824 pure ACK is lost it can prevent the release of new data. 826 4.4. Retransmitted packets. 828 RFC3168 does not allow setting the ECT codepoint in retransmitted 829 packets. The arguments presented in the specification for supporting 830 this design choice are the following ones (the text is quite long, 831 not sure if we should keep it all): 833 This document specifies ECN-capable TCP implementations MUST NOT 834 set either ECT codepoint (ECT(0) or ECT(1)) in the IP header for 835 retransmitted data packets, and that the TCP data receiver SHOULD 836 ignore the ECN field on arriving data packets that are outside of 837 the receiver's current window. This is for greater security 838 against denial-of-service attacks, as well as for robustness of 839 the ECN congestion indication with packets that are dropped later 840 in the network. 842 First, we note that if the TCP sender were to set an ECT codepoint 843 on a retransmitted packet, then if an unnecessarily-retransmitted 844 packet was later dropped in the network, the end nodes would never 845 receive the indication of congestion from the router setting the 846 CE codepoint. Thus, setting an ECT codepoint on retransmitted 847 data packets is not consistent with the robust delivery of the 848 congestion indication even for packets that are later dropped in 849 the network. 851 In addition, an attacker capable of spoofing the IP source address 852 of the TCP sender could send data packets with arbitrary sequence 853 numbers, with the CE codepoint set in the IP header. On receiving 854 this spoofed data packet, the TCP data receiver would determine 855 that the data does not lie in the current receive window, and 856 return a duplicate acknowledgement. We define an out-of-window 857 packet at the TCP data receiver as a data packet that lies outside 858 the receiver's current window. On receiving an out-of-window 859 packet, the TCP data receiver has to decide whether or not to 860 treat the CE codepoint in the packet header as a valid indication 861 of congestion, and therefore whether to return ECN-Echo 862 indications to the TCP data sender. If the TCP data receiver 863 ignored the CE codepoint in an out-of-window packet, then the TCP 864 data sender would not receive this possibly- legitimate indication 865 of congestion from the network, resulting in a violation of end- 866 to-end congestion control. On the other hand, if the TCP data 867 receiver honors the CE indication in the out-of-window packet, and 868 reports the indication of congestion to the TCP data sender, then 869 the malicious node that created the spoofed, out-of- window packet 870 has successfully "attacked" the TCP connection by forcing the data 871 sender to unnecessarily reduce (halve) its congestion window. To 872 prevent such a denial-of-service attack, we specify that a 873 legitimate TCP data sender MUST NOT set an ECT codepoint on 874 retransmitted data packets, and that the TCP data receiver SHOULD 875 ignore the CE codepoint on out-of-window packets. 877 One drawback of not setting ECT(0) or ECT(1) on retransmitted 878 packets is that it denies ECN protection for retransmitted 879 packets. However, for an ECN-capable TCP connection in a fully- 880 ECN-capable environment with mild congestion, packets should 881 rarely be dropped due to congestion in the first place, and so 882 instances of retransmitted packets should rarely arise. If 883 packets are being retransmitted, then there are already packet 884 losses (from corruption or from congestion) that ECN has been 885 unable to prevent. 887 We note that if the router sets the CE codepoint for an ECN- 888 capable data packet within a TCP connection, then the TCP 889 connection is guaranteed to receive that indication of congestion, 890 or to receive some other indication of congestion within the same 891 window of data, even if this packet is dropped or reordered in the 892 network. We consider two cases, when the packet is later 893 retransmitted, and when the packet is not later retransmitted. 895 In the first case, if the packet is either dropped or delayed, and 896 at some point retransmitted by the data sender, then the 897 retransmission is a result of a Fast Retransmit or a Retransmit 898 Timeout for either that packet or for some prior packet in the 899 same window of data. In this case, because the data sender 900 already has retransmitted this packet, we know that the data 901 sender has already responded to an indication of congestion for 902 some packet within the same window of data as the original packet. 903 Thus, even if the first transmission of the packet is dropped in 904 the network, or is delayed, if it had the CE codepoint set, and is 905 later ignored by the data receiver as an out- of-window packet, 906 this is not a problem, because the sender has already responded to 907 an indication of congestion for that window of data. 909 In the second case, if the packet is never retransmitted by the 910 data sender, then this data packet is the only copy of this data 911 received by the data receiver, and therefore arrives at the data 912 receiver as an in-window packet, regardless of how much the packet 913 might be delayed or reordered. In this case, if the CE codepoint 914 is set on the packet within the network, this will be treated by 915 the data receiver as a valid indication of congestion. 917 There are essentially three arguments for not ECT marking 918 retransmitted packets, namely, reliability, DoS attacks and over- 919 reaction to congestion. We address all of them next in order. 921 About reliability, as described in Section 4.1, we believe that the 922 bar should be that the congestion signal should be delivered as 923 reliably as if it was a packet drop. So, if a retransmitted packet 924 is dropped and this goes by unnoticed by the receiver, then the 925 congestion signal expressed as a drop would be lost. The same 926 applies to the congestion signal resulting from marking with ECT and 927 CE the very same retransmitted packet which later is dropped. 929 About the possibility of DoS attacks, the protection against the DoS 930 attack does not result from not allowing retransmitted packets to be 931 ECT marked. If an attacker decided to launch such an attack, it 932 would craft the packet with the ECT codepoint set. Effectively, the 933 protection against the described DoS attack comes from the 934 requirement that the receiver should not ignore the CE codepoint in 935 out-of-window packets. We proposed to allow ECT marking of 936 retransmitted packets, in order reduces the chances of it being 937 dropped, but keep the requirement to ignore the CE codepoint in out- 938 of-window packets. 940 Finally, the third argument is about over-reacting to congestion. 941 The argument goes that, if a retransmitted packet is dropped, the 942 sender will not detect it, so it will not react again to congestion 943 (it would have reduced its congestion window already when it 944 retransmitted the packet). Whereas, if retransmitted packets can be 945 CE tagged instead of dropped, senders could potentially react more 946 than once to congestion. 948 However, we argue that it is legitimate to respond again to 949 congestion if it still persists in subsequent round trip(s). So it 950 is not incorrect to set ECT on retransmissions. 952 4.5. Window probe packets 954 RFC3168 presents only the reliability argument for preventing setting 955 the ECT codepoint in Window Probe packets. Specifically, it states: 957 When the TCP data receiver advertises a zero window, the TCP data 958 sender sends window probes to determine if the receiver's window 959 has increased. Window probe packets do not contain any user data 960 except for the sequence number, which is a byte. If a window 961 probe packet is dropped in the network, this loss is not detected 962 by the receiver. Therefore, the TCP data sender MUST NOT set 963 either an ECT codepoint or the CWR bit on window probe packets. 965 However, because window probes use exact sequence numbers, they 966 cannot be easily spoofed in denial-of-service attacks. Therefore, 967 if a window probe arrives with the CE codepoint set, then the 968 receiver SHOULD respond to the ECN indications. 970 The reliability argument has been addressed in Section 4.1. dropping 971 the window probe message in the case the conditions for the Silly 972 Window Syndrome are on, basically implies that the sender will be 973 stalled until the new Window Probe message reaches the receiver, 974 which agains results in a performance penalty. 976 On the bright side, receivers should respond to ECN messages in these 977 packets, so changing the behaviour should be less painful than for 978 other packet types. 980 5. Security considerations 982 There are several security arguments presented in RFC 3168 for 983 preventing the ECN marking of TCP control packets and retransmitted 984 segments. We believe all of them have been properly addressed in 985 Section 4. 987 6. IANA Considerations 989 There are no IANA considerations in this memo. 991 7. Acknowledgments 993 TBD 995 8. Informative References 997 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 998 of Explicit Congestion Notification (ECN) to IP", 999 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1000 . 1002 [RFC5562] Kuzmanovic, A., Mondal, A., Floyd, S., and K. 1003 Ramakrishnan, "Adding Explicit Congestion Notification 1004 (ECN) Capability to TCP's SYN/ACK Packets", RFC 5562, 1005 DOI 10.17487/RFC5562, June 2009, 1006 . 1008 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 1009 Recommendations Regarding Active Queue Management", 1010 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 1011 . 1013 [RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit 1014 Congestion Notification (ECN) Signaling with Nonces", 1015 RFC 3540, DOI 10.17487/RFC3540, June 2003, 1016 . 1018 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1019 Requirement Levels", BCP 14, RFC 2119, 1020 DOI 10.17487/RFC2119, March 1997, 1021 . 1023 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 1024 Communication Layers", STD 3, RFC 1122, 1025 DOI 10.17487/RFC1122, October 1989, 1026 . 1028 [I-D.briscoe-tsvwg-aqm-tcpm-rmcat-l4s-problem] 1029 Briscoe, B., Schepper, K., and M. Bagnulo, "Low Latency, 1030 Low Loss, Scalable Throughput (L4S) Internet Service: 1031 Problem Statement", draft-briscoe-tsvwg-aqm-tcpm-rmcat- 1032 l4s-problem-02 (work in progress), July 2016. 1034 [I-D.ietf-tcpm-accurate-ecn] 1035 Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More 1036 Accurate ECN Feedback in TCP", draft-ietf-tcpm-accurate- 1037 ecn-01 (work in progress), June 2016. 1039 [judd-nsdi] 1040 Judd, G., "Attaining the promise and avoiding the pitfalls 1041 of TCP in the Datacenter", NSDI 2015, 2015. 1043 [ecn-pam] Brian, B., Mirja, M., Damiano, D., Iain, I., Gorry, G., 1044 and R. Richard, "Enabling Internet-Wide Deployment of 1045 Explicit Congestion Notification", PAM 2015, 2015. 1047 Authors' Addresses 1048 Marcelo Bagnulo 1049 Universidad Carlos III de Madrid 1050 Av. Universidad 30 1051 Leganes, Madrid 28911 1052 SPAIN 1054 Phone: 34 91 6249500 1055 Email: marcelo@it.uc3m.es 1056 URI: http://www.it.uc3m.es 1058 Bob Briscoe 1059 Simula Research Lab 1061 Email: ietf@bobbriscoe.net 1062 URI: http://bobbriscoe.net/