idnits 2.17.1 draft-ietf-tsvwg-ecn-l4s-id-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1620 has weird spacing: '...initial even...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: o it SHOULD not lead to some packets of a transport-layer flow being served by a different queue from others. -- The document date (November 5, 2018) is 1996 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Outdated reference: A later version (-28) exists of draft-ietf-tcpm-accurate-ecn-07 == Outdated reference: A later version (-15) exists of draft-ietf-tcpm-generalized-ecn-03 == Outdated reference: A later version (-15) exists of draft-ietf-tcpm-rack-04 == Outdated reference: A later version (-25) exists of draft-ietf-tsvwg-aqm-dualq-coupled-08 == Outdated reference: A later version (-22) exists of draft-ietf-tsvwg-ecn-encap-guidelines-10 == Outdated reference: A later version (-20) exists of draft-ietf-tsvwg-l4s-arch-03 == Outdated reference: A later version (-07) exists of draft-stewart-tsvwg-sctpecn-05 -- Obsolete informational reference (is this intentional?): RFC 2309 (Obsoleted by RFC 7567) -- Obsolete informational reference (is this intentional?): RFC 4960 (Obsoleted by RFC 9260) -- Obsolete informational reference (is this intentional?): RFC 8312 (Obsoleted by RFC 9438) Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Transport Services (tsv) K. De Schepper 3 Internet-Draft Nokia Bell Labs 4 Intended status: Experimental B. Briscoe, Ed. 5 Expires: May 9, 2019 CableLabs 6 November 5, 2018 8 Identifying Modified Explicit Congestion Notification (ECN) Semantics 9 for Ultra-Low Queuing Delay (L4S) 10 draft-ietf-tsvwg-ecn-l4s-id-05 12 Abstract 14 This specification defines the identifier to be used on IP packets 15 for a new network service called low latency, low loss and scalable 16 throughput (L4S). It is similar to the original (or 'Classic') 17 Explicit Congestion Notification (ECN). 'Classic' ECN marking was 18 required to be equivalent to a drop, both when applied in the network 19 and when responded to by a transport. Unlike 'Classic' ECN marking, 20 for packets carrying the L4S identifier, the network applies marking 21 more immediately and more aggressively than drop, and the transport 22 response to each mark is reduced and smoothed relative to that for 23 drop. The two changes counterbalance each other so that the 24 throughput of an L4S flow will be roughly the same as a 'Classic' 25 flow under the same conditions. However, the much more frequent 26 control signals and the finer responses to them result in ultra-low 27 queuing delay without compromising link utilization, and low delay is 28 maintained during high load. Examples of new active queue management 29 (AQM) marking algorithms and examples of new transports (whether TCP- 30 like or real-time) are specified separately. The new L4S identifier 31 is the key piece that enables them to interwork and distinguishes 32 them from 'Classic' traffic. It gives an incremental migration path 33 so that existing 'Classic' TCP traffic will be no worse off, but it 34 can be prevented from degrading the ultra-low delay and loss of the 35 new scalable transports. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at https://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on May 9, 2019. 54 Copyright Notice 56 Copyright (c) 2018 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (https://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 72 1.1. Problem . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 6 74 1.3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 6 75 2. Consensus Choice of L4S Packet Identifier: Requirements . . . 7 76 3. L4S Packet Identification at Run-Time . . . . . . . . . . . . 8 77 4. Prerequisite Transport Layer Behaviour . . . . . . . . . . . 8 78 4.1. Prerequisite Codepoint Setting . . . . . . . . . . . . . 8 79 4.2. Prerequisite Transport Feedback . . . . . . . . . . . . . 8 80 4.3. Prerequisite Congestion Response . . . . . . . . . . . . 9 81 5. Prerequisite Network Node Behaviour . . . . . . . . . . . . . 10 82 5.1. Prerequisite Classification and Re-Marking Behaviour . . 10 83 5.2. The Meaning of L4S CE Relative to Drop . . . . . . . . . 11 84 5.3. Exception for L4S Packet Identification by Network Nodes 85 with Transport-Layer Awareness . . . . . . . . . . . . . 12 86 5.4. Interaction of the L4S Identifier with other Identifiers 12 87 5.4.1. Examples of Other Identifiers Complementing L4S 88 Identifiers . . . . . . . . . . . . . . . . . . . . . 12 89 5.4.1.1. Inclusion of Additional Traffic with L4S . . . . 12 90 5.4.1.2. Exclusion of Traffic From L4S Treatment . . . . . 13 91 5.4.2. Generalized Combination of L4S and Other Identifiers 14 92 6. L4S Experiments . . . . . . . . . . . . . . . . . . . . . . . 15 93 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 94 8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 95 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 15 96 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 97 10.1. Normative References . . . . . . . . . . . . . . . . . . 16 98 10.2. Informative References . . . . . . . . . . . . . . . . . 16 99 Appendix A. The 'TCP Prague Requirements' . . . . . . . . . . . 21 100 A.1. Requirements for Scalable Transport Protocols . . . . . . 22 101 A.1.1. Use of L4S Packet Identifier . . . . . . . . . . . . 22 102 A.1.2. Accurate ECN Feedback . . . . . . . . . . . . . . . . 22 103 A.1.3. Fall back to Reno-friendly congestion control on 104 packet loss . . . . . . . . . . . . . . . . . . . . . 22 105 A.1.4. Fall back to Reno-friendly congestion control on 106 classic ECN bottlenecks . . . . . . . . . . . . . . . 23 107 A.1.5. Reduce RTT dependence . . . . . . . . . . . . . . . . 24 108 A.1.6. Scaling down to fractional congestion windows . . . . 24 109 A.1.7. Measuring Reordering Tolerance in Time Units . . . . 25 110 A.2. Scalable Transport Protocol Optimizations . . . . . . . . 27 111 A.2.1. Setting ECT in TCP Control Packets and 112 Retransmissions . . . . . . . . . . . . . . . . . . . 27 113 A.2.2. Faster than Additive Increase . . . . . . . . . . . . 27 114 A.2.3. Faster Convergence at Flow Start . . . . . . . . . . 27 115 Appendix B. Alternative Identifiers . . . . . . . . . . . . . . 28 116 B.1. ECT(1) and CE codepoints . . . . . . . . . . . . . . . . 28 117 B.2. ECN Plus a Diffserv Codepoint (DSCP) . . . . . . . . . . 30 118 B.3. ECN capability alone . . . . . . . . . . . . . . . . . . 32 119 B.4. Protocol ID . . . . . . . . . . . . . . . . . . . . . . . 34 120 B.5. Source or destination addressing . . . . . . . . . . . . 34 121 B.6. Summary: Merits of Alternative Identifiers . . . . . . . 34 122 Appendix C. Potential Competing Uses for the ECT(1) Codepoint . 35 123 C.1. Integrity of Congestion Feedback . . . . . . . . . . . . 35 124 C.2. Notification of Less Severe Congestion than CE . . . . . 36 125 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 127 1. Introduction 129 This specification defines the identifier to be used on IP packets 130 for a new network service called low latency, low loss and scalable 131 throughput (L4S). It is similar to the original (or 'Classic') 132 Explicit Congestion Notification (ECN [RFC3168]). 'Classic' ECN 133 marking was required to be equivalent to a drop, both when applied in 134 the network and when responded to by a transport. Unlike 'Classic' 135 ECN marking, the network applies L4S marking more immediately and 136 more aggressively than drop, and the transport response to each mark 137 is reduced and smoothed relative to that for drop. The two changes 138 counterbalance each other so that the throughput of an L4S flow will 139 be roughly the same as a 'Classic' flow under the same conditions. 140 Nonetheless, the much more frequent control signals and the finer 141 responses to them result in ultra-low queuing delay without 142 compromising link utilization, and low delay is maintained during 143 high load. 145 An example of a scalable transport that would enable the L4S service 146 is Data Centre TCP (DCTCP), which until now has been applicable 147 solely to controlled environments like data centres [RFC8257], 148 because it is too aggressive to co-exist with existing TCP. The 149 DualQ Coupled AQM, which is defined in a complementary experimental 150 specification [I-D.ietf-tsvwg-aqm-dualq-coupled], is an AQM framework 151 that enables scalable transports like DCTCP to co-exist with existing 152 traffic, each getting roughly the same flow rate when they compete 153 under similar conditions. Note that a transport such as DCTCP is 154 still not safe to deploy on the Internet unless it satisfies the 155 requirements listed in Section 4. 157 The new L4S identifier is the key piece that enables L4S hosts and 158 L4S network nodes to interwork and distinguishes their traffic from 159 'Classic' traffic. It gives an incremental migration path so that 160 existing 'Classic' TCP traffic will be no worse off, but it can be 161 prevented from degrading the ultra-low delay and loss of the new 162 scalable transports. The performance improvement is so great that it 163 is motivating initial deployment of the separate parts of this 164 system. 166 1.1. Problem 168 Latency is becoming the critical performance factor for many (most?) 169 applications on the public Internet, e.g. interactive Web, Web 170 services, voice, conversational video, interactive video, interactive 171 remote presence, instant messaging, online gaming, remote desktop, 172 cloud-based applications, and video-assisted remote control of 173 machinery and industrial processes. In the developed world, further 174 increases in access network bit-rate offer diminishing returns, 175 whereas latency is still a multi-faceted problem. In the last decade 176 or so, much has been done to reduce propagation time by placing 177 caches or servers closer to users. However, queuing remains a major 178 intermittent component of latency. 180 The Diffserv architecture provides Expedited Forwarding [RFC3246], so 181 that low latency traffic can jump the queue of other traffic. 182 However, on access links dedicated to individual sites (homes, small 183 enterprises or mobile devices), often all traffic at any one time 184 will be latency-sensitive. Then Diffserv is of little use. Instead, 185 we need to remove the causes of any unnecessary delay. 187 The bufferbloat project has shown that excessively-large buffering 188 ('bufferbloat') has been introducing significantly more delay than 189 the underlying propagation time. These delays appear only 190 intermittently--only when a capacity-seeking (e.g. TCP) flow is long 191 enough for the queue to fill the buffer, making every packet in other 192 flows sharing the buffer sit through the queue. 194 Active queue management (AQM) was originally developed to solve this 195 problem (and others). Unlike Diffserv, which gives low latency to 196 some traffic at the expense of others, AQM controls latency for _all_ 197 traffic in a class. In general, AQMs introduce an increasing level 198 of discard from the buffer the longer the queue persists above a 199 shallow threshold. This gives sufficient signals to capacity-seeking 200 (aka. greedy) flows to keep the buffer empty for its intended 201 purpose: absorbing bursts. However, RED [RFC2309] and other 202 algorithms from the 1990s were sensitive to their configuration and 203 hard to set correctly. So, AQM was not widely deployed. 205 More recent state-of-the-art AQMs, e.g. fq_CoDel [RFC8290], 206 PIE [RFC8033], Adaptive RED [ARED01], are easier to configure, 207 because they define the queuing threshold in time not bytes, so it is 208 invariant for different link rates. However, no matter how good the 209 AQM, the sawtoothing rate of TCP will either cause queuing delay to 210 vary or cause the link to be under-utilized. Even with a perfectly 211 tuned AQM, the additional queuing delay will be of the same order as 212 the underlying speed-of-light delay across the network. Flow-queuing 213 can isolate one flow from another, but it cannot isolate a TCP flow 214 from the delay variations it inflicts on itself, and it has other 215 problems - it overrides the flow rate decisions of variable rate 216 video applications, it does not recognise the flows within IPSec VPN 217 tunnels and it is relatively expensive to implement. 219 Latency is not our only concern: It was known when TCP was first 220 developed that it would not scale to high bandwidth-delay products 221 [TCP-CA]. Given regular broadband bit-rates over WAN distances are 222 already [RFC3649] beyond the scaling range of 'Classic' TCP Reno, 223 'less unscalable' Cubic [RFC8312] and 224 Compound [I-D.sridharan-tcpm-ctcp] variants of TCP have been 225 successfully deployed. However, these are now approaching their 226 scaling limits. Unfortunately, fully scalable TCPs such as DCTCP 227 [RFC8257] cause 'Classic' TCP to starve itself, which is why they 228 have been confined to private data centres or research testbeds 229 (until now). 231 It turns out that a TCP algorithm like DCTCP that solves the latency 232 problem also solves TCP's scalability problem. The finer sawteeth 233 have low amplitude, so they cause very little queuing delay variation 234 and the number of sawteeth per round trip remains invariant, which 235 maintains constant tight control as flow-rate scales. A supporting 236 paper [DCttH15] gives the full explanation of why the design solves 237 both the latency and the scaling problems, both in plain English and 238 in more precise mathematical form. The explanation is summarised 239 without the maths in the L4S architecture document 240 [I-D.ietf-tsvwg-l4s-arch]. 242 1.2. Terminology 244 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 245 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 246 "OPTIONAL" in this document are to be interpreted as described in 247 [RFC2119]. In this document, these words will appear with that 248 interpretation only when in ALL CAPS. Lower case uses of these words 249 are not to be interpreted as carrying RFC-2119 significance. 251 Classic service: The 'Classic' service is intended for all the 252 behaviours that currently co-exist with TCP Reno (e.g. TCP Cubic, 253 Compound, SCTP, etc). 255 Low-Latency, Low-Loss and Scalable (L4S) service: The 'L4S' service 256 is intended for traffic from scalable TCP algorithms such as Data 257 Centre TCP. But it is also more general--it allows the set of 258 congestion controls with similar scaling properties to DCTCP to 259 evolve (e.g. Relentless TCP [Mathis09] and the L4S variant of 260 SCREAM for real-time media [RFC8298]. 262 Both Classic and L4S services can cope with a proportion of 263 unresponsive or less-responsive traffic as well, as long as it 264 does not build a queue (e.g. DNS, VoIP, game sync datagrams, 265 etc). 267 Classic ECN: The original Explicit Congestion Notification (ECN) 268 protocol [RFC3168]. 270 1.3. Scope 272 The new L4S identifier defined in this specification is applicable 273 for IPv4 and IPv6 packets (as for classic ECN [RFC3168]). It is 274 applicable for the unicast, multicast and anycast forwarding modes. 276 The L4S identifier is an orthogonal packet classification to the 277 Differentiated Services Code Point (DSCP [RFC2474]). Section 5.4 278 explains what this means in practice. 280 This document is intended for experimental status, so it does not 281 update any standards track RFCs. Therefore it depends on [RFC8311], 282 which is a standards track specification that: 284 o updates the ECN proposed standard [RFC3168] to allow experimental 285 track RFCs to relax the requirement that an ECN mark must be 286 equivalent to a drop, both when applied by the network, and when 287 responded to by the sender; 289 o changes the status of the experimental ECN nonce [RFC3540] to 290 historic; 292 o makes consequent updates to the following additional proposed 293 standard RFCs to reflect the above two bullets: 295 * ECN for RTP [RFC6679]; 297 * the congestion control specifications of various DCCP 298 congestion control identifier (CCID) profiles [RFC4341], 299 [RFC4342], [RFC5622]. 301 2. Consensus Choice of L4S Packet Identifier: Requirements 303 This subsection briefly records the process that led to a consensus 304 choice of L4S identifier, selected from all the alternatives in 305 Appendix B. 307 Ideally, the identifier for packets using the Low Latency, Low Loss, 308 Scalable throughput (L4S) service ought to meet the following 309 requirements: 311 o it SHOULD survive end-to-end between source and destination 312 applications: across the boundary between host and network, 313 between interconnected networks, and through middleboxes; 315 o it SHOULD be common to IPv4 and IPv6 and transport-agnostic; 317 o it SHOULD be incrementally deployable; 319 o it SHOULD enable an AQM to classify packets encapsulated by outer 320 IP or lower-layer headers; 322 o it SHOULD consume minimal extra codepoints; 324 o it SHOULD not lead to some packets of a transport-layer flow being 325 served by a different queue from others. 327 Whether the identifier would be recoverable if the experiment failed 328 is a factor that could be taken into account. However, this has not 329 been made a requirement, because that would favour schemes that would 330 be easier to fail, rather than those more likely to succeed. 332 It is recognised that the chosen identifier is unlikely to satisfy 333 all these requirements, particularly given the limited space left in 334 the IP header. Therefore a compromise will be necessary, which is 335 why all the requirements are expressed with the word 'SHOULD' not 336 'MUST'. Appendix B discusses the pros and cons of the compromises 337 made in various competing identification schemes against the above 338 requirements. 340 On the basis of this analysis, "ECT(1) and CE codepoints" is the best 341 compromise. Therefore this scheme is defined in detail in the 342 following sections, while Appendix B records the rationale for this 343 decision. 345 3. L4S Packet Identification at Run-Time 347 The L4S treatment is an experimental track alternative packet marking 348 treatment [RFC4774] to the classic ECN treatment [RFC3168], which has 349 been updated by [RFC8311] to allow this experiment (amongst others). 350 Like classic ECN, L4S ECN identifies both network and host behaviour: 351 it identifies the marking treatment that network nodes are expected 352 to apply to L4S packets, and it identifies packets that have been 353 sent from hosts that are expected to comply with a broad type of 354 sending behaviour. 356 For a packet to receive L4S treatment as it is forwarded, the sender 357 sets the ECN field in the IP header to the ECT(1) codepoint. See 358 Section 4 for full transport layer behaviour requirements, including 359 feedback and congestion response. 361 A network node that implements the L4S service normally classifies 362 arriving ECT(1) and CE packets for L4S treatment. See Section 5 for 363 full network element behaviour requirements, including 364 classification, ECN-marking and interaction of the L4S identifier 365 with other identifiers and per-hop behaviours. 367 4. Prerequisite Transport Layer Behaviour 369 4.1. Prerequisite Codepoint Setting 371 For a packet to receive L4S treatment as it is forwarded, the sender 372 MUST set the ECN field in the IP header (v4 or v6) to the ECT(1) 373 codepoint. 375 4.2. Prerequisite Transport Feedback 377 In general, a scalable congestion control needs feedback of the 378 extent of CE marking on the forward path. Due to the history of TCP 379 development, when ECN was added TCP reported no more than one CE mark 380 per round trip. Some transport protocols derived from TCP mimic this 381 behaviour while others report the accurate extent of TCP marking. 382 This means that some transport protocols will need to be updated as a 383 prerequisite for scalable congestion control. The position for a few 384 well-known transport protocols is given below. 386 TCP: Support for accurate ECN feedback (AccECN 387 [I-D.ietf-tcpm-accurate-ecn]) by both ends is a prerequisite for 388 scalable congestion control. Therefore, the presence of ECT(1) in 389 the IP headers even in one direction of a TCP connection will 390 imply that both ends support AccECN. However, the converse does 391 not apply. So even if both ends support AccECN, either of the two 392 ends can choose not to use a scalable congestion control, whatever 393 the other end's choice. 395 SCTP: An ECN feedback protocol such as that specified in 396 [I-D.stewart-tsvwg-sctpecn] would be a prerequisite for scalable 397 congestion control. That draft would update the ECN feedback 398 protocol sketched out in Appendix A of the standards track 399 specification of SCTP [RFC4960] by adding a field to report the 400 number of CE marks. 402 RTP over UDP: A prerequisite for scalable congestion control is for 403 both (all) ends of one media-level hop to signal ECN support using 404 the ecn-capable-rtp attribute [RFC6679]. Therefore, the presence 405 of ECT(1) implies that both (all) ends of that hop support ECN. 406 However, the converse does not apply, so each end of a media-level 407 hop can independently choose not to use a scalable congestion 408 control, even if both ends support ECN. 410 DCCP: The ACK vector in DCCP [RFC4340] is already sufficient to 411 report the extent of CE marking as needed by a scalable congestion 412 control. 414 4.3. Prerequisite Congestion Response 416 As a condition for a host to send packets with the L4S identifier 417 (ECT(1)), it SHOULD implement a congestion control behaviour that 418 ensures the flow rate is inversely proportional to the proportion of 419 bytes in packets marked with the CE codepoint. This is termed a 420 scalable congestion control, because the number of control signals 421 (ECN marks) per round trip remains roughly constant for any flow 422 rate. As with all transport behaviours, a detailed specification 423 will need to be defined for each type of transport or application, 424 including the timescale over which the proportionality is averaged, 425 and control of burstiness. The inverse proportionality requirement 426 above is worded as a 'SHOULD' rather than a 'MUST' to allow 427 reasonable flexibility when defining these specifications. 429 Data Center TCP (DCTCP [RFC8257]) is an example of a scalable 430 congestion control. 432 Each sender in a session can use a scalable congestion control 433 independently of the congestion control used by the receiver(s) when 434 they send data. Therefore there might be ECT(1) packets in one 435 direction and ECT(0) or Not-ECT in the other. 437 In order to coexist safely with other Internet traffic, a scalable 438 congestion control MUST NOT tag its packets with the ECT(1) codepoint 439 unless it complies with the following bulleted requirements. The 440 specification of a particular scalable congestion control MUST 441 describe in detail how it satisfies each requirement: 443 o A scalable congestion control MUST react to packet loss in a way 444 that will coexist safely with a TCP Reno congestion control 445 [RFC5681] (see Appendix A.1.3 for rationale). 447 o A scalable congestion control MUST react to ECN marking from a 448 non-L4S but ECN-capable bottleneck in a way that will coexist with 449 a TCP Reno congestion control [RFC5681] (see Appendix A.1.4 for 450 rationale). 452 o A scalable congestion control MUST reduce or eliminate RTT bias 453 over as wide a range of RTTs as possible, or at least over the 454 typical range of RTTs that will interact in the intended 455 deployment scenario (see Appendix A.1.5 for rationale). 457 o A scalable congestion control MUST remain responsive to congestion 458 when the RTT is significantly smaller than in the current public 459 Internet (see Appendix A.1.6 for rationale). 461 o A scalable congestion control MUST detect loss by counting in 462 units of time, which is scalable, and MUST NOT count in units of 463 packets (as in the 3 DupACK rule of traditional TCP), which is not 464 scalable (see Appendix A.1.7 for rationale). 466 5. Prerequisite Network Node Behaviour 468 5.1. Prerequisite Classification and Re-Marking Behaviour 470 A network node that implements the L4S service MUST classify arriving 471 ECT(1) packets for L4S treatment and it SHOULD classify arriving CE 472 packets for L4S treatment as well. Section 5.3 describes a possible 473 exception to this latter rule for some CE packets. 475 An L4S AQM treatment follows similar codepoint transition rules to 476 those in RFC 3168. Specifically, the ECT(1) codepoint MUST NOT be 477 changed to any other codepoint than CE, and CE MUST NOT be changed to 478 any other codepoint. An ECT(1) packet is classified as ECN-capable 479 and, if congestion increases, an L4S AQM algorithm will mark the ECN 480 field as CE for an increasing proportion of packets, otherwise 481 forwarding packets unchanged as ECT(1). Necessary conditions for an 482 L4S marking treatment are defined in Section 5.2. Under persistent 483 overload an L4S marking treatment SHOULD turn off ECN marking, using 484 drop as a congestion signal until the overload episode has subsided, 485 as recommended for all AQMs in [RFC7567] (Section 4.2.1), which 486 follows the similar advice in RFC 3168 (Section 7). 488 For backward compatibility in uncontrolled environments, a network 489 node that implements the L4S treatment MUST also implement a classic 490 AQM treatment. It MUST classify arriving ECT(0) and Not-ECT packets 491 for treatment by the Classic AQM (see the discussion of the 492 classifier for the dual-queue coupled AQM in 493 [I-D.ietf-tsvwg-aqm-dualq-coupled]). Classic treatment means that 494 the AQM will mark ECT(0) packets under the same conditions as it 495 would drop Not-ECT packets [RFC3168]. 497 5.2. The Meaning of L4S CE Relative to Drop 499 The likelihood that an AQM drops a Not-ECT Classic packet (p_C) MUST 500 be roughly proportional to the square of the likelihood that it would 501 have marked it if it had been an L4S packet (p_L). That is 503 p_C ~= (p_L / k)^2 505 The constant of proportionality (k) does not have to be standardised 506 for interoperability, but a value of 2 is RECOMMENDED. 508 [I-D.ietf-tsvwg-aqm-dualq-coupled] specifies the essential aspects of 509 an L4S AQM, as well as recommending other aspects. It gives example 510 implementations in appendices. 512 The term 'likelihood' is used above to allow for marking and dropping 513 to be either probabilistic or deterministic. The example AQMs in 514 [I-D.ietf-tsvwg-aqm-dualq-coupled] drop and mark probabilistically, 515 so the drop probability is arranged to be the square of the marking 516 probability. Nonetheless, an alternative AQM that dropped and marked 517 deterministically would be valid, as long as the dropping frequency 518 was proportional to the square of the marking frequency. 520 Note that, contrary to RFC 3168, a Dual AQM implementing the L4S and 521 Classic treatments does not mark an ECT(1) packet under the same 522 conditions that it would have dropped a Not-ECT packet, as allowed by 523 [RFC8311], which updates RFC 3168. However, it does mark an ECT(0) 524 packet under the same conditions that it would have dropped a Not-ECT 525 packet. 527 5.3. Exception for L4S Packet Identification by Network Nodes with 528 Transport-Layer Awareness 530 To implement the L4S treatment, a network node does not need to 531 identify transport-layer flows. Nonetheless, if an implementer is 532 willing to identify transport-layer flows at a network node, and if 533 the most recent ECT packet in the same flow was ECT(0), the node MAY 534 classify CE packets for classic ECN [RFC3168] treatment. In all 535 other cases, a network node MUST classify CE packets for L4S 536 treatment. Examples of such other cases are: i) if no ECT packets 537 have yet been identified in a flow; ii) if it is not desirable for a 538 network node to identify transport-layer flows; or iii) if the most 539 recent ECT packet in a flow was ECT(1). 541 If an implementer uses flow-awareness to classify CE packets, to 542 determine whether the flow is using ECT(0) or ECT(1) it only uses the 543 most recent ECT packet of a flow (this advice will need to be 544 verified as part of L4S experiments). This is because a sender might 545 have to switch from sending ECT(1) (L4S) packets to sending ECT(0) 546 (Classic) packets, or back again, in the middle of a transport-layer 547 flow. Such a switch-over is likely to be very rare, but It could be 548 necessary if the path bottleneck moves from a network node that 549 supports L4S to one that only supports Classic ECN. A host ought to 550 be able to detect such a change from a change in RTT variation. 552 5.4. Interaction of the L4S Identifier with other Identifiers 554 5.4.1. Examples of Other Identifiers Complementing L4S Identifiers 556 5.4.1.1. Inclusion of Additional Traffic with L4S 558 In a typical case for the public Internet a network element that 559 implements L4S might want to classify some low-rate but unresponsive 560 traffic (e.g. DNS, voice, game sync packets) into the low latency 561 queue to mix with L4S traffic. Such non-ECN-based packet types MUST 562 be safe to mix with L4S traffic without harming the low latency 563 service. 565 In this case it would not be appropriate to call the queue an L4S 566 queue, because it is shared by L4S and non-L4S traffic. Instead it 567 will be called the low latency or L queue. The L queue then offers 568 two different treatments: 570 o The L4S treatment, which is a combination of the L4S AQM treatment 571 and a priority scheduling treatment; 573 o The low latency treatment, which is solely the priority scheduling 574 treatment, without ECN-marking by the AQM. 576 To identify packets for just the scheduling treatment, it would be 577 inappropriate to use the L4S ECT(1) identifier, because such traffic 578 is unresponsive to ECN marking. Therefore, a network element that 579 implements L4S MAY classify additional packets into the L queue if 580 they carry certain non-ECN identifiers. For instance: 582 o addresses of specific applications or hosts configured to be safe 583 (but for example cannot set the ECN field for some temporary 584 reason); 586 o certain protocols that are usually lightweight (e.g. ARP, DNS); 588 o specific Diffserv codepoints that indicate traffic with limited 589 burstiness such as the EF (Expedited Forwarding) and Voice-Admit 590 service classes or equivalent local-use DSCPs (see 591 [I-D.briscoe-tsvwg-l4s-diffserv]). 593 For clarity, non-ECN identifiers, such as the examples itemized 594 above, might be used by some network operators who believe they 595 identify non-L4S traffic that would be safe to mix with L4S traffic. 596 They are not alternative ways for a host to indicate that it is 597 sending L4S packets. Only the ECT(1) and CE ECN codepoints indicate 598 to a network element that a host is sending L4S packets - 599 specifically that the host claims its behaviour satisfies the pre- 600 requisite transport requirements in Section 4. 602 5.4.1.2. Exclusion of Traffic From L4S Treatment 604 To extend the above example, an operator might want to exclude some 605 traffic from the L4S treatment for policy reason, e.g. security 606 (traffic from malicious sources) or commercial (initially the 607 operator may wish to confine the benefits of L4S to business 608 customers). 610 In this exclusion case, the operator MUST classify on the relevant 611 locally-used identifiers (e.g. source addresses) before classifying 612 the non-matching traffic on the end-to-end L4S ECN identifier. 614 The operator MUST NOT re-mark the end-to-end L4S identifier, because 615 its decision to exclude certain traffic from L4S treatment is local- 616 only. The end-to-end L4S identifier then survives for other 617 operators to use, or indeed, they can apply their own policy, 618 independently based on their own choice of locally-used identifiers. 619 This approach also allows any operator to remove its locally-applied 620 exclusions in future, e.g. if it wishes to widen the benefit of the 621 L4S treatment to all its customers. 623 5.4.2. Generalized Combination of L4S and Other Identifiers 625 L4S concerns low latency, which it can provide for all traffic 626 without differentiation and without affecting bandwidth allocation. 627 Diffserv provides for differentiation of both bandwidth and low 628 latency, but its control of latency depends on its control of 629 bandwidth. The two can be combined if a network operator wants to 630 control bandwidth allocation but it also wants to provide low latency 631 for any amount of traffic within one of these allocations of 632 bandwidth (rather than only providing low latency by limiting 633 bandwidth) [I-D.briscoe-tsvwg-l4s-diffserv]. 635 The examples above were framed in the context of splitting the 636 default Best Efforts Per-Hop Behaviour (PHB) into a Low Latency (L) 637 queue and a Classic (C) Queue. But, more generally, an operator 638 might choose to control bandwidth allocation through a hierarchy of 639 Diffserv PHBs at a node, and to split one or more of these PHBs into 640 a low latency and a classic variant of that PHB. 642 In the first case, where there are no other PHBs except the DualQ, if 643 a packet carries ECT(1) or CE, a network element would classify it 644 for the L4S treatment irrespective of its DSCP. And, if a packet 645 carried (say) the EF DSCP, the network element could classify it for 646 into L queue irrespective of its ECN codepoint. However, where the 647 DualQ is in a hierarchy of other PHBs, the classifier would classify 648 some traffic into other PHBs based on DSCP before classifying between 649 the latency and classic queues (based on ECT(1), CE and perhaps the 650 EF DSCP or other identifiers as in the above example). 652 [I-D.briscoe-tsvwg-l4s-diffserv] describes how an operator might use 653 L4S to offer low latency for all L4S traffic as well as using 654 Diffserv for bandwidth differentiation. It identifies two main types 655 of approach, which can be combined: the operator might split certain 656 Diffserv PHBs between L4S and a corresponding Classic service. Or it 657 might split the L4S and/or the Classic service into multiple Diffserv 658 PHBs. In any of these cases, a packet would have to be classified on 659 its Diffserv and ECN codepoints. 661 In summary, there are numerous ways in which the L4S ECN identifier 662 (ECT(1) and CE) could be combined with other identifiers to achieve 663 particular objectives. The following categorization articulates 664 those that are valid, but it is not necessarily exhaustive. Those 665 tagged 'Global-use' could be set by the sending host or a network. 666 Those tagged 'Local-use' would only be set by a network: 668 1. Identifiers Complementing the L4S Identifier 670 A. Including More Traffic in the L Queue 671 (Global-use or Local-use) 673 B. Excluding Certain Traffic from the L Queue 674 (Local-use only) 676 2. Identifiers to place L4S classification in a PHB Hierarchy 677 (Global-use or Local-use) 679 A. PHBs Before L4S ECN Classification 681 B. PHBs After L4S ECN Classification 683 6. L4S Experiments 685 [I-D.ietf-tsvwg-aqm-dualq-coupled] sets operational and management 686 requirements for experiments with DualQ Coupled AQMs. General 687 operational and management requirements for experiments with L4S 688 congestion controls are given in Section 4 and Section 5 above, e.g. 689 co-existence and scaling requirements, incremental deployment 690 arrangements. The specification of each scalable congestion control 691 will need to include protocol-specific requirements for configuration 692 and monitoring performance during experiments. Appendix A of 693 [RFC5706] provides a helpful checklist. 695 7. IANA Considerations 697 This specification contains no IANA considerations. 699 8. Security Considerations 701 Approaches to assure the integrity of signals using the new identifer 702 are introduced in Appendix C.1. 704 9. Acknowledgements 706 Thanks to Richard Scheffenegger, John Leslie, David Taeht, Jonathan 707 Morton, Gorry Fairhurst, Michael Welzl, Mikael Abrahamsson and Andrew 708 McGregor for the discussions that led to this specification. Ing-jyh 709 (Inton) Tsang was a contributor to the early drafts of this document. 710 Appendix A listing the TCP Prague Requirements is based on text 711 authored by Marcelo Bagnulo Braun that was originally an appendix to 712 [I-D.ietf-tsvwg-l4s-arch]. That text was in turn based on the 713 collective output of the attendees listed in the minutes of a 'bar 714 BoF' on DCTCP Evolution during IETF-94 [TCPPrague]. 716 The authors' contributions were part-funded by the European Community 717 under its Seventh Framework Programme through the Reducing Internet 718 Transport Latency (RITE) project (ICT-317700). Bob Briscoe was also 719 part-funded by the Research Council of Norway through the TimeIn 720 project. The views expressed here are solely those of the authors. 722 10. References 724 10.1. Normative References 726 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 727 Requirement Levels", BCP 14, RFC 2119, 728 DOI 10.17487/RFC2119, March 1997, 729 . 731 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 732 of Explicit Congestion Notification (ECN) to IP", 733 RFC 3168, DOI 10.17487/RFC3168, September 2001, 734 . 736 [RFC4774] Floyd, S., "Specifying Alternate Semantics for the 737 Explicit Congestion Notification (ECN) Field", BCP 124, 738 RFC 4774, DOI 10.17487/RFC4774, November 2006, 739 . 741 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 742 and K. Carlberg, "Explicit Congestion Notification (ECN) 743 for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 744 2012, . 746 10.2. Informative References 748 [Alizadeh-stability] 749 Alizadeh, M., Javanmard, A., and B. Prabhakar, "Analysis 750 of DCTCP: Stability, Convergence, and Fairness", ACM 751 SIGMETRICS 2011 , June 2011. 753 [ARED01] Floyd, S., Gummadi, R., and S. Shenker, "Adaptive RED: An 754 Algorithm for Increasing the Robustness of RED's Active 755 Queue Management", ACIRI Technical Report , August 2001, 756 . 758 [DCttH15] De Schepper, K., Bondarenko, O., Briscoe, B., and I. 759 Tsang, "'Data Centre to the Home': Ultra-Low Latency for 760 All", RITE Project Technical Report , 2015, 761 . 763 [I-D.briscoe-tsvwg-l4s-diffserv] 764 Briscoe, B., "Interactions between Low Latency, Low Loss, 765 Scalable Throughput (L4S) and Differentiated Services", 766 draft-briscoe-tsvwg-l4s-diffserv-02 (work in progress), 767 November 2018. 769 [I-D.ietf-tcpm-accurate-ecn] 770 Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More 771 Accurate ECN Feedback in TCP", draft-ietf-tcpm-accurate- 772 ecn-07 (work in progress), July 2018. 774 [I-D.ietf-tcpm-generalized-ecn] 775 Bagnulo, M. and B. Briscoe, "ECN++: Adding Explicit 776 Congestion Notification (ECN) to TCP Control Packets", 777 draft-ietf-tcpm-generalized-ecn-03 (work in progress), 778 October 2018. 780 [I-D.ietf-tcpm-rack] 781 Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "RACK: 782 a time-based fast loss detection algorithm for TCP", 783 draft-ietf-tcpm-rack-04 (work in progress), July 2018. 785 [I-D.ietf-tsvwg-aqm-dualq-coupled] 786 Schepper, K., Briscoe, B., Bondarenko, O., and I. Tsang, 787 "DualQ Coupled AQMs for Low Latency, Low Loss and Scalable 788 Throughput (L4S)", draft-ietf-tsvwg-aqm-dualq-coupled-08 789 (work in progress), November 2018. 791 [I-D.ietf-tsvwg-ecn-encap-guidelines] 792 Briscoe, B., Kaippallimalil, J., and P. Thaler, 793 "Guidelines for Adding Congestion Notification to 794 Protocols that Encapsulate IP", draft-ietf-tsvwg-ecn- 795 encap-guidelines-10 (work in progress), March 2018. 797 [I-D.ietf-tsvwg-l4s-arch] 798 Briscoe, B., Schepper, K., and M. Bagnulo, "Low Latency, 799 Low Loss, Scalable Throughput (L4S) Internet Service: 800 Architecture", draft-ietf-tsvwg-l4s-arch-03 (work in 801 progress), October 2018. 803 [I-D.sridharan-tcpm-ctcp] 804 Sridharan, M., Tan, K., Bansal, D., and D. Thaler, 805 "Compound TCP: A New TCP Congestion Control for High-Speed 806 and Long Distance Networks", draft-sridharan-tcpm-ctcp-02 807 (work in progress), November 2008. 809 [I-D.stewart-tsvwg-sctpecn] 810 Stewart, R., Tuexen, M., and X. Dong, "ECN for Stream 811 Control Transmission Protocol (SCTP)", draft-stewart- 812 tsvwg-sctpecn-05 (work in progress), January 2014. 814 [Mathis09] 815 Mathis, M., "Relentless Congestion Control", PFLDNeT'09 , 816 May 2009, . 819 [Paced-Chirping] 820 Misund, J., "Rapid Acceleration in TCP Prague", Masters 821 Thesis , May 2018, 822 . 825 [QV] Briscoe, B. and P. Hurtig, "Up to Speed with Queue View", 826 RITE Technical Report D2.3; Appendix C.2, August 2015, 827 . 830 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 831 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 832 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 833 S., Wroclawski, J., and L. Zhang, "Recommendations on 834 Queue Management and Congestion Avoidance in the 835 Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998, 836 . 838 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 839 "Definition of the Differentiated Services Field (DS 840 Field) in the IPv4 and IPv6 Headers", RFC 2474, 841 DOI 10.17487/RFC2474, December 1998, 842 . 844 [RFC2983] Black, D., "Differentiated Services and Tunnels", 845 RFC 2983, DOI 10.17487/RFC2983, October 2000, 846 . 848 [RFC3246] Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec, 849 J., Courtney, W., Davari, S., Firoiu, V., and D. 850 Stiliadis, "An Expedited Forwarding PHB (Per-Hop 851 Behavior)", RFC 3246, DOI 10.17487/RFC3246, March 2002, 852 . 854 [RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit 855 Congestion Notification (ECN) Signaling with Nonces", 856 RFC 3540, DOI 10.17487/RFC3540, June 2003, 857 . 859 [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", 860 RFC 3649, DOI 10.17487/RFC3649, December 2003, 861 . 863 [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram 864 Congestion Control Protocol (DCCP)", RFC 4340, 865 DOI 10.17487/RFC4340, March 2006, 866 . 868 [RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion 869 Control Protocol (DCCP) Congestion Control ID 2: TCP-like 870 Congestion Control", RFC 4341, DOI 10.17487/RFC4341, March 871 2006, . 873 [RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for 874 Datagram Congestion Control Protocol (DCCP) Congestion 875 Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC 4342, 876 DOI 10.17487/RFC4342, March 2006, 877 . 879 [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", 880 RFC 4960, DOI 10.17487/RFC4960, September 2007, 881 . 883 [RFC5562] Kuzmanovic, A., Mondal, A., Floyd, S., and K. 884 Ramakrishnan, "Adding Explicit Congestion Notification 885 (ECN) Capability to TCP's SYN/ACK Packets", RFC 5562, 886 DOI 10.17487/RFC5562, June 2009, 887 . 889 [RFC5622] Floyd, S. and E. Kohler, "Profile for Datagram Congestion 890 Control Protocol (DCCP) Congestion ID 4: TCP-Friendly Rate 891 Control for Small Packets (TFRC-SP)", RFC 5622, 892 DOI 10.17487/RFC5622, August 2009, 893 . 895 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 896 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 897 . 899 [RFC5706] Harrington, D., "Guidelines for Considering Operations and 900 Management of New Protocols and Protocol Extensions", 901 RFC 5706, DOI 10.17487/RFC5706, November 2009, 902 . 904 [RFC6077] Papadimitriou, D., Ed., Welzl, M., Scharf, M., and B. 905 Briscoe, "Open Research Issues in Internet Congestion 906 Control", RFC 6077, DOI 10.17487/RFC6077, February 2011, 907 . 909 [RFC6660] Briscoe, B., Moncaster, T., and M. Menth, "Encoding Three 910 Pre-Congestion Notification (PCN) States in the IP Header 911 Using a Single Diffserv Codepoint (DSCP)", RFC 6660, 912 DOI 10.17487/RFC6660, July 2012, 913 . 915 [RFC7560] Kuehlewind, M., Ed., Scheffenegger, R., and B. Briscoe, 916 "Problem Statement and Requirements for Increased Accuracy 917 in Explicit Congestion Notification (ECN) Feedback", 918 RFC 7560, DOI 10.17487/RFC7560, August 2015, 919 . 921 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 922 Recommendations Regarding Active Queue Management", 923 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 924 . 926 [RFC8033] Pan, R., Natarajan, P., Baker, F., and G. White, 927 "Proportional Integral Controller Enhanced (PIE): A 928 Lightweight Control Scheme to Address the Bufferbloat 929 Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017, 930 . 932 [RFC8257] Bensley, S., Thaler, D., Balasubramanian, P., Eggert, L., 933 and G. Judd, "Data Center TCP (DCTCP): TCP Congestion 934 Control for Data Centers", RFC 8257, DOI 10.17487/RFC8257, 935 October 2017, . 937 [RFC8290] Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys, 938 J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler 939 and Active Queue Management Algorithm", RFC 8290, 940 DOI 10.17487/RFC8290, January 2018, 941 . 943 [RFC8298] Johansson, I. and Z. Sarker, "Self-Clocked Rate Adaptation 944 for Multimedia", RFC 8298, DOI 10.17487/RFC8298, December 945 2017, . 947 [RFC8311] Black, D., "Relaxing Restrictions on Explicit Congestion 948 Notification (ECN) Experimentation", RFC 8311, 949 DOI 10.17487/RFC8311, January 2018, 950 . 952 [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and 953 R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", 954 RFC 8312, DOI 10.17487/RFC8312, February 2018, 955 . 957 [TCP-CA] Jacobson, V. and M. Karels, "Congestion Avoidance and 958 Control", Laurence Berkeley Labs Technical Report , 959 November 1988, . 961 [TCP-sub-mss-w] 962 Briscoe, B. and K. De Schepper, "Scaling TCP's Congestion 963 Window for Small Round Trip Times", BT Technical Report 964 TR-TUB8-2015-002, May 2015, 965 . 968 [TCPPrague] 969 Briscoe, B., "Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 970 2015, 17:40, Prague", tcpprague mailing list archive , 971 July 2015, . 974 [VCP] Xia, Y., Subramanian, L., Stoica, I., and S. Kalyanaraman, 975 "One more bit is enough", Proc. SIGCOMM'05, ACM CCR 976 35(4)37--48, 2005, 977 . 979 Appendix A. The 'TCP Prague Requirements' 981 This appendix is informative, not normative. It gives a list of 982 modifications to current scalable transport protocols so that they 983 can be deployed over the public Internet and coexist safely with 984 existing traffic. The list complements the normative requirements in 985 Section 4 that a sender has to comply with before it can set the L4S 986 identifier in packets it sends into the Internet. As well as 987 necessary safety improvements (requirements) this appendix also 988 includes preferable performance improvements (optimizations). 990 These recommendations have become know as the TCP Prague 991 Requirements, because they were originally identified at an ad hoc 992 meeting during IETF-94 in Prague [TCPPrague]. The wording has been 993 generalized to apply to all scalable congestion controls, not just 994 TCP congestion control specifically. 996 DCTCP [RFC8257] is currently the most widely used scalable transport 997 protocol. In its current form, DCTCP is specified to be deployable 998 only in controlled environments. Deploying it in the public Internet 999 would lead to a number of issues, both from the safety and the 1000 performance perspective. The modifications and additional mechanisms 1001 listed in this section will be necessary for its deployment over the 1002 global Internet. Where an example is needed, DCTCP is used as a 1003 base, but it is likely that most of these requirements equally apply 1004 to other scalable transport protocols. 1006 A.1. Requirements for Scalable Transport Protocols 1008 A.1.1. Use of L4S Packet Identifier 1010 Description: A scalable congestion control needs to distinguish the 1011 packets it sends from those sent by classic congestion controls. 1013 Motivation: It needs to be possible for a network node to classify 1014 L4S packets without flow state into a queue that applies an L4S ECN 1015 marking behaviour and isolates L4S packets from the queuing delay of 1016 classic packets. 1018 A.1.2. Accurate ECN Feedback 1020 Description: A scalable transport protocol needs to provide timely, 1021 accurate feedback about the extent of ECN marking experienced by all 1022 packets. 1024 Motivation: Classic congestion controls only need feedback about the 1025 existence of a congestion episode within a round trip, not precisely 1026 how many packets were marked with ECN or dropped. Therefore, in 1027 2001, when ECN feedback was added to TCP [RFC3168], it could not 1028 inform the sender of more than one ECN mark per RTT. Since then, 1029 requirements for more accurate ECN feedback in TCP have been defined 1030 in [RFC7560] and [I-D.ietf-tcpm-accurate-ecn] specifies an 1031 experimental change to the TCP wire protocol to satisfy these 1032 requirements. Most other transport protocols already satisfy this 1033 requirement. 1035 A.1.3. Fall back to Reno-friendly congestion control on packet loss 1037 Description: A scalable congestion control needs to react to packet 1038 loss in a way that will coexist safely with a TCP Reno congestion 1039 control [RFC5681]. 1041 Motivation: Part of the safety conditions for deploying a scalable 1042 congestion control on the public Internet is to make sure that it 1043 behaves properly when it builds a queue at a network bottleneck that 1044 has not been upgraded to support L4S. Packet loss can have many 1045 causes, but it usually has to be conservatively assumed that it is a 1046 sign of congestion. Therefore, on detecting packet loss, a scalable 1047 congestion control will need to fall back to classic congestion 1048 control behaviour. If it does not comply with this requirement it 1049 could starve classic traffic. 1051 A scalable congestion control can be used for different types of 1052 transport, e.g. for real-time media or for reliable bulk transport 1053 like TCP. Therefore, the particular classic congestion control 1054 behaviour to fall back on will need to be part of the congestion 1055 control specification of the relevant transport. In the particular 1056 case of DCTCP, the current DCTCP specification states that "It is 1057 RECOMMENDED that an implementation deal with loss episodes in the 1058 same way as conventional TCP." For safe deployment of a scalable 1059 transport in the public Internet, the above requirement would need to 1060 be defined as a "MUST". 1062 Packet loss might (rarely) occur in the case that the bottleneck is 1063 L4S capable. In this case, the sender may receive a high number of 1064 packets marked with the CE bit set and also experience a loss. 1065 Current DCTCP implementations react differently to this situation. 1066 At least one implementation reacts only to the drop signal (e.g. by 1067 halving the CWND) and at least another DCTCP implementation reacts to 1068 both signals (e.g. by halving the CWND due to the drop and also 1069 further reducing the CWND based on the proportion of marked packet). 1070 We believe that further experimentation is needed to understand what 1071 is the best behaviour for the public Internet, which may or not be 1072 one of these existing approaches. 1074 A.1.4. Fall back to Reno-friendly congestion control on classic ECN 1075 bottlenecks 1077 Description: A scalable congestion control needs to react to ECN 1078 marking from a non-L4S but ECN-capable bottleneck in a way that will 1079 coexist with a TCP Reno congestion control [RFC5681]. 1081 Motivation: Similarly to the requirement in Appendix A.1.3, this 1082 requirement is a safety condition to ensure a scalable congestion 1083 control behaves properly when it builds a queue at a network 1084 bottleneck that has not been upgraded to support L4S. On detecting 1085 classic ECN marking (see below), a scalable congestion control will 1086 need to fall back to classic congestion control behaviour. If it 1087 does not comply with this requirement it could starve classic 1088 traffic. 1090 It would take time for endpoints to distinguish classic and L4S ECN 1091 marking. An increase in queuing delay or in delay variation would be 1092 a tell-tale sign, but it is not yet clear where a line would be drawn 1093 between the two behaviours. It might be possible to cache what was 1094 learned about the path to help subsequent attempts to detect the type 1095 of marking. 1097 A.1.5. Reduce RTT dependence 1099 Description: A scalable congestion control needs to reduce or 1100 eliminate RTT bias over as wide a range of RTTs as possible, or at 1101 least over the typical range of RTTs that will interact in the 1102 intended deployment scenario. 1104 Motivation: Classic TCP's throughput is known to be inversely 1105 proportional to RTT, so one would expect flows over very low RTT 1106 paths to nearly starve flows over larger RTTs. However, Classic TCP 1107 has never allowed a very low RTT path to exist because it induces a 1108 large queue. For instance, consider two paths with base RTT 1ms and 1109 100ms. If Classic TCP induces a 100ms queue, it turns these RTTs 1110 into 101ms and 200ms leading to a throughput ratio of about 2:1. 1111 Whereas if a Scalable TCP induces only a 1ms queue, the ratio is 1112 2:101, leading to a throughput ratio of about 50:1. 1114 Therefore, with very small queues, long RTT flows will essentially 1115 starve, unless scalable congestion controls comply with this 1116 requirement. 1118 A.1.6. Scaling down to fractional congestion windows 1120 Description: A scalable congestion control needs to remain responsive 1121 to congestion when RTTs are significantly smaller than in the current 1122 public Internet. 1124 Motivation: As currently specified, the minimum required congestion 1125 window of TCP (and its derivatives) is set to 2 maximum segment sizes 1126 (MSS) (see equation (4) in [RFC5681]). Once the congestion window 1127 reaches this minimum, all current TCP algorithms become unresponsive 1128 to congestion signals. No matter how much drop or ECN marking, the 1129 congestion window no longer reduces. Instead, TCP forces the queue 1130 to grow, overriding any AQM and increasing queuing delay. 1132 L4S mechanisms significantly reduce queueing delay so, over the same 1133 path, the RTT becomes lower. Then this problem becomes surprisingly 1134 common [TCP-sub-mss-w]. This is because, for the same link capacity, 1135 smaller RTT implies a smaller window. For instance, consider a 1136 residential setting with an upstream broadband Internet access of 8 1137 Mb/s, assuming a max segment size of 1500 B. Two upstream flows will 1138 each have the minimum window of 2 MSS if the RTT is 6ms or less, 1139 which is quite common when accessing a nearby data centre. So, any 1140 more than two such parallel TCP flows will become unresponsive and 1141 increase queuing delay. 1143 Unless scalable congestion controls are required to comply with this 1144 requirement from the start, they will frequently become unresponsive, 1145 negating the low latency benefit of L4S, for themselves and for 1146 others. One possible sub-MSS window mechanism is described in 1147 [TCP-sub-mss-w], and other approaches are likely to be feasible. 1149 A.1.7. Measuring Reordering Tolerance in Time Units 1151 Description: A scalable congestion control needs to detect loss by 1152 counting in units of time, which is scalable, rather than counting in 1153 units of packets, which is not. 1155 Motivation: If it is known that all L4S senders using a link obey 1156 this rule, then link technologies that support L4S can remove the 1157 head-of-line blocking delay they have to introduce while trying to 1158 keep packets in tight order to avoid triggering loss detection based 1159 on counting packets. 1161 End-systems cannot know whether a missing packet is due to loss or 1162 reordering, except in hindsight - if it appears later. If senders 1163 deem that loss has occurred by counting reordered packets (e.g. the 3 1164 Duplicate ACK rule of Classic TCP), the time over which the network 1165 has to keep packets in order scales down as packet rates scale up 1166 over the years. In contrast, if senders allow a reordering window in 1167 units of time before they deem there has been a loss, the time over 1168 which the network has to keep packets in order stays constant. 1170 Tolerance of reordering over a small duration will allow parallel 1171 (e.g. bonded-channel) link technologies to relax their need to 1172 deliver packets strictly in order. Such links typically give 1173 arriving packets a link-level sequence number and introduce delay 1174 while buffering packets at the receiving end until they can be 1175 delivered in the same order. For radio links, this delay usually 1176 includes the time allowed for link-layer retransmissions. 1178 For receivers that need their packets in order, it would seem that 1179 relaxing network ordering would simply shift this reordering delay 1180 from the network to the receiver. However, that is not true in the 1181 general case because links generally do not recognize transport layer 1182 flows and often cannot even see application layer streams within the 1183 flows (as in SCTP, HTTP/2 or QUIC). So a link will often be holding 1184 back packets from one flow or stream while waiting for those from 1185 another. Relaxing strict ordering in the network will remove this 1186 head-of-line blocking delay. {ToDo: this is being quantified 1187 experimentally - will need to add the figures here.} 1188 Classic TCP implementations are switching over to the time-based 1189 approach of RACK (Recent ACKnowledgements [I-D.ietf-tcpm-rack]). 1190 However, it will be many years (decades?) before networks no longer 1191 have to allow for the presence of traditional TCP senders still using 1192 the 3 DupACK rule. This specification (Section 4.3) says that 1193 senders are not entitled to identify packets as L4S in the IP/ECN 1194 field unless they use the time-based approach. Then networks that 1195 identify L4S traffic separately (e.g. using 1196 [I-D.ietf-tsvwg-aqm-dualq-coupled]) can know for certain that all L4S 1197 traffic is using the scalable time-based approach. 1199 This will allow networks to remove head-of-line blocking delay 1200 immediately, but only for L4S traffic. But Classic traffic will have 1201 to wait for many years until incremental deployment of RACK has 1202 become near-universal. Nonetheless, experience with RACK will 1203 determine how much reordering tolerance networks will be able to 1204 allow for L4S traffic. 1206 Performance Optimization as well as Safety Improvement: The delay 1207 benefit would be lost if any L4S sender did not follow the time-based 1208 approach. Therefore, the time-based approach is made a normative 1209 requirement (a necessary safety improvement). Nonetheless, the time- 1210 based approach also enables a throughput benefit that a flow can 1211 enjoy independently of others (a performance optimization), explained 1212 next. 1214 Given the requirement for a scalable congestion control to fall-back 1215 to Reno or Cubic on a loss (see Appendix A.1.3), it is important that 1216 a scalable congestion control does not deem that a loss has occurred 1217 too soon. If, later within the same round trip, an out-of-order 1218 acknowledgement fills the gap, the sender would have halved its rate 1219 spuriously (as well as retransmitting spuriously). With a RACK-like 1220 approach, allowing longer before a loss is deemed to have occurred 1221 maintains higher throughput in the presence of reordering {ToDo: 1222 Quantify this statement}. 1224 On the other hand, it is also important not to wait too long before 1225 deeming that a gap is due to a loss (termed a long reordering 1226 window), otherwise loss recovery would be slow. 1228 The speed of loss recovery is much more significant for short flows 1229 than long, therefore a good compromise would adapt the reordering 1230 window; from a small fraction of the RTT at the start of a flow, to a 1231 larger fraction of the RTT for flows that continue for many round 1232 trips. This is the approach adopted by TCP RACK (Recent 1233 ACKnowledgements) [I-D.ietf-tcpm-rack] and recommended for all L4S 1234 senders, whether using TCP or another transport protocol. 1236 A.2. Scalable Transport Protocol Optimizations 1238 A.2.1. Setting ECT in TCP Control Packets and Retransmissions 1240 Description: To improve performance, scalable transport protocols 1241 ought to enable ECN at the IP layer in TCP control packets (SYN, SYN- 1242 ACK, pure ACKs, etc.) and in retransmitted packets. The same is true 1243 for derivatives of TCP, e.g. SCTP. 1245 Motivation: RFC 3168 prohibits the use of ECN on these types of TCP 1246 packet, based on a number of arguments. This means these packets are 1247 not protected from congestion loss by ECN, which considerably harms 1248 performance, particularly for short flows. 1249 [I-D.ietf-tcpm-generalized-ecn] counters each argument in RFC 3168 in 1250 turn, showing it was over-cautious. Instead it proposes experimental 1251 use of ECN on all types of TCP packet. 1253 A.2.2. Faster than Additive Increase 1255 Description: It would improve performance if scalable congestion 1256 controls did not limit their congestion window increase to the 1257 traditional additive increase of 1 MSS per round trip [RFC5681] 1258 during congestion avoidance. The same is true for derivatives of TCP 1259 congestion control. 1261 Motivation: As currently defined, DCTCP uses the traditional TCP Reno 1262 additive increase in congestion avoidance phase. When the available 1263 capacity suddenly increases (e.g. when another flow finishes, or if 1264 radio capacity increases) it can take very many round trips to take 1265 advantage of the new capacity. In the steady state, DCTCP induces 1266 about 2 ECN marks per round trip, so it should be possible to quickly 1267 detect when these signals have disappeared and seek available 1268 capacity more rapidly. It will of course be necessary to minimize 1269 the impact on other flows (classic and scalable). 1271 TCP Cubic was designed to solve this problem, but as flow rates have 1272 continued to increase, the delay accelerating into available capacity 1273 has become prohibitive. For instance, with RTT=20 ms, to increase 1274 flow rate from 100Mb/s to 200Mb/s Cubic takes between 50 and 100 1275 round trips. Every 8x increase in flow rate leads to 2x more 1276 acceleration delay. 1278 A.2.3. Faster Convergence at Flow Start 1280 Description: Particularly when a flow starts, scalable congestion 1281 controls need to converge (reach their steady-state share of the 1282 capacity) at least as fast as classic TCP and preferably faster. 1283 This does not just affect TCP Prague, but also the flow start 1284 behaviour of any L4S congestion control derived from a Classic 1285 transport that uses TCP slow start. 1287 Motivation: As an example, a new DCTCP flow takes longer than classic 1288 TCP to obtain its share of the capacity of the bottleneck when there 1289 are already ongoing flows using the bottleneck capacity. In a data 1290 centre environment DCTCP takes about a factor of 1.5 to 2 longer to 1291 converge due to the much higher typical level of ECN marking that 1292 DCTCP background traffic induces, which causes new flows to exit slow 1293 start early [Alizadeh-stability]. In testing for use over the public 1294 Internet the convergence time of DCTCP relative to regular TCP is 1295 even less favourable [Paced-Chirping]). It is exacerbated by the 1296 typically greater mismatch between the link rate of the sending host 1297 and typical Internet access bottlenecks, in combination with the 1298 shallow ECN marking threshold needed for TCP Prague. This problem is 1299 detrimental in general, but would particularly harm the performance 1300 of short flows relative to classic TCP. 1302 Appendix B. Alternative Identifiers 1304 This appendix is informative, not normative. It records the pros and 1305 cons of various alternative ways to identify L4S packets to record 1306 the rationale for the choice of ECT(1) (Appendix B.1) as the L4S 1307 identifier. At the end, Appendix B.6 summarises the distinguishing 1308 features of the leading alternatives. It is intended to supplement, 1309 not replace the detailed text. 1311 The leading solutions all use the ECN field, sometimes in combination 1312 with the Diffserv field. Both the ECN and Diffserv fields have the 1313 additional advantage that they are no different in either IPv4 or 1314 IPv6. A couple of alternatives that use other fields are mentioned 1315 at the end, but it is quickly explained why they are not serious 1316 contenders. 1318 B.1. ECT(1) and CE codepoints 1320 Definition: 1322 Packets with ECT(1) and conditionally packets with CE would 1323 signify L4S semantics as an alternative to the semantics of 1324 classic ECN [RFC3168], specifically: 1326 * The ECT(1) codepoint would signify that the packet was sent by 1327 an L4S-capable sender; 1329 * Given shortage of codepoints, both L4S and classic ECN sides of 1330 an AQM would have to use the same CE codepoint to indicate that 1331 a packet had experienced congestion. If a packet that had 1332 already been marked CE in an upstream buffer arrived at a 1333 subsequent AQM, this AQM would then have to guess whether to 1334 classify CE packets as L4S or classic ECN. Choosing the L4S 1335 treatment would be a safer choice, because then a few classic 1336 packets might arrive early, rather than a few L4S packets 1337 arriving late; 1339 * Additional information might be available if the classifier 1340 were transport-aware. Then it could classify a CE packet for 1341 classic ECN treatment if the most recent ECT packet in the same 1342 flow had been marked ECT(0). However, the L4S service ought 1343 not to need tranport-layer awareness; 1345 Cons: 1347 Consumes the last ECN codepoint: The L4S service is intended to 1348 supersede the service provided by classic ECN, therefore using 1349 ECT(1) to identify L4S packets could ultimately mean that the 1350 ECT(0) codepoint was 'wasted' purely to distinguish one form of 1351 ECN from its successor; 1353 ECN hard in some lower layers: It is not always possible to support 1354 ECN in an AQM acting in a buffer below the IP layer 1355 [I-D.ietf-tsvwg-ecn-encap-guidelines]. In such cases, the L4S 1356 service would have to drop rather than mark frames even though 1357 they might contain an ECN-capable packet. However, such cases 1358 would be unusual. 1360 Risk of reordering classic CE packets: Having to classify all CE 1361 packets as L4S risks some classic CE packets arriving early, which 1362 is a form of reordering. Reordering can cause the TCP sender to 1363 retransmit spuriously. However, one or two packets delivered 1364 early does not cause any spurious retransmissions because the 1365 subsequent packets continue to move the cumulative acknowledgement 1366 boundary forwards. Anyway, the risk of reordering would be low, 1367 because: i) it is quite unusual to experience more than one 1368 bottleneck queue on a path; ii) even then, reordering would only 1369 occur if there was simultaneous mixing of classic and L4S traffic, 1370 which would be more unlikely in an access link, which is where 1371 most bottlenecks are located; iii) even then, spurious 1372 retransmissions would only occur if a contiguous sequence of three 1373 or more classic CE packets from one bottleneck arrived at the 1374 next, which should in itself happen very rarely with a good AQM. 1375 The risk would be completely eliminated in AQMs that were 1376 transport-aware (but they should not need to be); 1378 Non-L4S service for control packets: The classic ECN RFCs [RFC3168] 1379 and [RFC5562] require a sender to clear the ECN field to Not-ECT 1380 for retransmissions and certain control packets specifically pure 1381 ACKs, window probes and SYNs. When L4S packets are classified by 1382 the ECN field alone, these control packets would not be classified 1383 into an L4S queue, and could therefore be delayed relative to the 1384 other packets in the flow. This would not cause re-ordering 1385 (because retransmissions are already out of order, and the control 1386 packets carry no data). However, it would make critical control 1387 packets more vulnerable to loss and delay. To address this 1388 problem, [I-D.ietf-tcpm-generalized-ecn] proposes an experiment in 1389 which all TCP control packets and retransmissions are ECN-capable. 1391 Pros: 1393 Should work e2e: The ECN field generally works end-to-end across the 1394 Internet. Unlike the DSCP, the setting of the ECN field is at 1395 least forwarded unchanged by networks that do not support ECN, and 1396 networks rarely clear it to zero; 1398 Should work in tunnels: Unlike Diffserv, ECN is defined to always 1399 work across tunnels. However, tunnels do not always implement ECN 1400 processing as they should do, particularly because IPsec tunnels 1401 were defined differently for a few years. 1403 Could migrate to one codepoint: If all classic ECN senders 1404 eventually evolve to use the L4S service, the ECT(0) codepoint 1405 could be reused for some future purpose, but only once use of 1406 ECT(0) packets had reduced to zero, or near-zero, which might 1407 never happen. 1409 B.2. ECN Plus a Diffserv Codepoint (DSCP) 1411 Definition: 1413 For packets with a defined DSCP, all codepoints of the ECN field 1414 (except Not-ECT) would signify alternative L4S semantics to those 1415 for classic ECN [RFC3168], specifically: 1417 * The L4S DSCP would signifiy that the packet came from an L4S- 1418 capable sender; 1420 * ECT(0) and ECT(1) would both signify that the packet was 1421 travelling between transport endpoints that were both ECN- 1422 capable; 1424 * CE would signify that the packet had been marked by an AQM 1425 implementing the L4S service. 1427 Use of a DSCP is the only approach for alternative ECN semantics 1428 given as an example in [RFC4774]. However, it was perhaps considered 1429 more for controlled environments than new end-to-end services; 1431 Cons: 1433 Consumes DSCP pairs: A DSCP is obviously not orthogonal to Diffserv. 1434 Therefore, wherever the L4S service is applied to multiple 1435 Diffserv scheduling behaviours, it would be necessary to replace 1436 each DSCP with a pair of DSCPs. 1438 Uses critical lower-layer header space: The resulting increased 1439 number of DSCPs might be hard to support for some lower layer 1440 technologies, e.g. 802.1p and MPLS both offer only 3-bits for a 1441 maximum of 8 traffic class identifiers. Although L4S should 1442 reduce and possibly remove the need for some DSCPs intended for 1443 differentiated queuing delay, it will not remove the need for 1444 Diffserv entirely, because Diffserv is also used to allocate 1445 bandwidth, e.g. by prioritising some classes of traffic over 1446 others when traffic exceeds available capacity. 1448 Not end-to-end (host-network): Very few networks honour a DSCP set 1449 by a host. Typically a network will zero (bleach) the Diffserv 1450 field from all hosts. Sometimes networks will attempt to identify 1451 applications by some form of packet inspection and, based on 1452 network policy, they will set the DSCP considered appropriate for 1453 the identified application. Network-based application 1454 identification might use some combination of protocol ID, port 1455 numbers(s), application layer protocol headers, IP address(es), 1456 VLAN ID(s) and even packet timing. 1458 Not end-to-end (network-network): Very few networks honour a DSCP 1459 received from a neighbouring network. Typically a network will 1460 zero (bleach) the Diffserv field from all neighbouring networks at 1461 an interconnection point. Sometimes bilateral arrangements are 1462 made between networks, such that the receiving network remarks 1463 some DSCPs to those it uses for roughly equivalent services. The 1464 likelihood that a DSCP will be bleached or ignored depends on the 1465 type of DSCP: 1467 Local-use DSCP: These tend to be used to implement application- 1468 specific network policies, but a bilateral arrangement to 1469 remark certain DSCPs is often applied to DSCPs in the local-use 1470 range simply because it is easier not to change all of a 1471 network's internal configurations when a new arrangement is 1472 made with a neighbour; 1474 Global-use DSCP: These do not tend to be honoured across network 1475 interconnections more than local-use DSCPs. However, if two 1476 networks decide to honour certain of each other's DSCPs, the 1477 reconfiguration is a little easier if both of their globally 1478 recognised services are already represented by the relevant 1479 global-use DSCPs. 1481 Note that today a global-use DSCP gives little more assurance 1482 of end-to-end service than a local-use DSCP. In future the 1483 global-use range might give more assurance of end-to-end 1484 service than local-use, but it is unlikely that either 1485 assurance will be high, particularly given the hosts are 1486 included in the end-to-end path. 1488 Not all tunnels: Diffserv codepoints are often not propagated to the 1489 outer header when a packet is encapsulated by a tunnel header. 1490 DSCPs are propagated to the outer of uniform mode tunnels, but not 1491 pipe mode [RFC2983], and pipe mode is fairly common. 1493 ECN hard in some lower layers:: Because this approach uses both the 1494 Diffserv and ECN fields, an AQM wil only work at a lower layer if 1495 both can be supported. If individual network operators wished to 1496 deploy an AQM at a lower layer, they would usually propagate an IP 1497 Diffserv codepoint to the lower layer, using for example IEEE 1498 802.1p. However, the ECN capability is harder to propagate down 1499 to lower layers because few lower layers support it. 1501 Pros: 1503 Could migrate to e2e: If all usage of classic ECN migrates to usage 1504 of L4S, the DSCP would become redundant, and the ECN capability 1505 alone could eventually identify L4S packets without the 1506 interconnection problems of Diffserv detailed above, and without 1507 having permanently consumed more than one codepoint in the IP 1508 header. Although the DSCP does not generally function as an end- 1509 to-end identifier (see above), it could be used initially by 1510 individual ISPs to introduce the L4S service for their own locally 1511 generated traffic; 1513 B.3. ECN capability alone 1515 Definition: 1517 This approach uses ECN capability alone as the L4S identifier. It 1518 is only feasible if classic ECN is not widely deployed. The 1519 specific definition of codepoints would be: 1521 * Any ECN codepoint other than Not-ECT would signify an L4S- 1522 capable sender; 1524 * ECN codepoints would not be used for classic [RFC3168] ECN, and 1525 the classic network service would only be used for Not-ECT 1526 packets. 1528 This approach would only be feasible if 1530 A. it was generally agreed that there was little chance of any 1531 classic [RFC3168] ECN deployment in any network nodes; 1533 B. it was generally agreed that there was little chance of any 1534 client devices being deployed with classic [RFC3168] TCP-ECN 1535 on by default (note that classic TCP-ECN is already on-by- 1536 default on many servers); 1538 C. for TCP connections, developers of client OSs would all have 1539 to agree not to encourage further deployment of classic ECN. 1540 Specifically, at the start of a TCP connection classic ECN 1541 could be disabled during negotation of the ECN capability: 1543 + an L4S-capable host would have to disable ECN if the 1544 corresponding host did not support accurate ECN feedback 1545 [RFC7560], which is a prerequisite for the L4S service; 1547 + developers of operating systems for user devices would only 1548 enable ECN by default for TCP once the stack implemented 1549 L4S and accurate ECN feedback [RFC7560] including 1550 requesting accurate ECN feedback by default. 1552 Cons: 1554 Near-infeasible deployment constraints: The constraints for 1555 deployment above represent a highly unlikely, but not completely 1556 impossible, set of circumstances. If, despite the above measures, 1557 a pair of hosts did negotiate to use classic ECN, their packets 1558 would be classified into the same queue as L4S traffic, and if 1559 they had to compete with a long-running L4S flow they would get a 1560 very small capacity share; 1562 ECN hard in some lower layers: See the same issue with "ECT(1) and 1563 CE codepoints" (Appendix B.1); 1565 Non-L4S service for control packets: See the same issue with "ECT(1) 1566 and CE codepoints" (Appendix B.1). 1568 Pros: 1570 Consumes no additional codepoints: The ECT(1) codepoint and all 1571 spare Diffserv codepoints would remain available for future use; 1573 Should work e2e: As with "ECT(1) and CE codepoints" (Appendix B.1); 1575 Should work in tunnels: As with "ECT(1) and CE codepoints" 1576 (Appendix B.1). 1578 B.4. Protocol ID 1580 It has been suggested that a new ID in the IPv4 Protocol field or the 1581 IPv6 Next Header field could identify L4S packets. However this 1582 approach is ruled out by numerous problems: 1584 o A new protocol ID would need to be paired with the old one for 1585 each transport (TCP, SCTP, UDP, etc.); 1587 o In IPv6, there can be a sequence of Next Header fields, and it 1588 would not be obvious which one would be expected to identify a 1589 network service like L4S; 1591 o A new protocol ID would rarely provide an end-to-end service, 1592 because It is well-known that new protocol IDs are often blocked 1593 by numerous types of middlebox; 1595 o The approach is not a solution for AQMs below the IP layer; 1597 B.5. Source or destination addressing 1599 Locally, a network operator could arrange for L4S service to be 1600 applied based on source or destination addressing, e.g. packets from 1601 its own data centre and/or CDN hosts, packets to its business 1602 customers, etc. It could use addressing at any layer, e.g. IP 1603 addresses, MAC addresses, VLAN IDs, etc. Although addressing might 1604 be a useful tactical approach for a single ISP, it would not be a 1605 feasible approach to identify an end-to-end service like L4S. Even 1606 for a single ISP, it would require packet classifiers in buffers to 1607 be dependent on changing topology and address allocation decisions 1608 elsewhere in the network. Therefore this approach is not a feasible 1609 solution. 1611 B.6. Summary: Merits of Alternative Identifiers 1613 Table 1 provides a very high level summary of the pros and cons 1614 detailed against the schemes described respectively in Appendix B.2, 1615 Appendix B.3 and Appendix B.1, for six issues that set them apart. 1617 +--------------+--------------------+---------+--------------------+ 1618 | Issue | DSCP + ECN | ECN | ECT(1) + CE | 1619 +--------------+--------------------+---------+--------------------+ 1620 | | initial eventual | initial | initial eventual | 1621 | | | | | 1622 | end-to-end | N . . . ? . | . . Y | . . Y . . Y | 1623 | tunnels | . O . . O . | . . ? | . . ? . . Y | 1624 | lower layers | N . . . ? . | . O . | . O . . . ? | 1625 | codepoints | N . . . . ? | . . Y | N . . . . ? | 1626 | reordering | . . Y . . Y | . . Y | . O . . . ? | 1627 | ctrl pkts | . . Y . . Y | . O . | . O . . . ? | 1628 | | | | | 1629 | | | Note 1 | | 1630 +--------------+--------------------+---------+--------------------+ 1632 Note 1: Only feasible if classic ECN is obsolete. 1634 Table 1: Comparison of the Merits of Three Alternative Identifiers 1636 The schemes are scored based on both their capabilities now 1637 ('initial') and in the long term ('eventual'). The 'ECN' scheme 1638 shares the 'eventual' scores of the 'ECT(1) + CE' scheme. The scores 1639 are one of 'N, O, Y', meaning 'Poor', 'Ordinary', 'Good' 1640 respectively. The same scores are aligned vertically to aid the eye. 1641 A score of "?" in one of the positions means that this approach might 1642 optimisitically become this good, given sufficient effort. The table 1643 summarises the text and is not meant to be understandable without 1644 having read the text. 1646 Appendix C. Potential Competing Uses for the ECT(1) Codepoint 1648 The ECT(1) codepoint of the ECN field has already been assigned once 1649 for the ECN nonce [RFC3540], which has now been categorized as 1650 historic [RFC8311]. ECN is probably the only remaining field in the 1651 Internet Protocol that is common to IPv4 and IPv6 and still has 1652 potential to work end-to-end, with tunnels and with lower layers. 1653 Therefore, ECT(1) should not be reassigned to a different 1654 experimental use (L4S) without carefully assessing competing 1655 potential uses. These fall into the following categories: 1657 C.1. Integrity of Congestion Feedback 1659 Receiving hosts can fool a sender into downloading faster by 1660 suppressing feedback of ECN marks (or of losses if retransmissions 1661 are not necessary or available otherwise). 1663 The historic ECN nonce protocol [RFC3540] proposed that a TCP sender 1664 could set either of ECT(0) or ECT(1) in each packet of a flow and 1665 remember the sequence it had set. If any packet was lost or 1666 congestion marked, the receiver would miss that bit of the sequence. 1667 An ECN Nonce receiver had to feed back the least significant bit of 1668 the sum, so it could not suppress feedback of a loss or mark without 1669 a 50-50 chance of guessing the sum incorrectly. 1671 The ECN Nonce RFC [RFC3540] as been reclassified as historic, partly 1672 because other ways have been developed to protect TCP feedback 1673 integrity [RFC8311] that do not consume a codepoint in the IP header. 1674 So it is highly unlikely that ECT(1) will be needed for integrity 1675 protection in future. 1677 C.2. Notification of Less Severe Congestion than CE 1679 Various researchers have proposed to use ECT(1) as a less severe 1680 congestion notification than CE, particularly to enable flows to fill 1681 available capacity more quickly after an idle period, when another 1682 flow departs or when a flow starts, e.g. VCP [VCP], Queue View (QV) 1683 [QV]. 1685 Before assigning ECT(1) as an identifer for L4S, we must carefully 1686 consider whether it might be better to hold ECT(1) in reserve for 1687 future standardisation of rapid flow acceleration, which is an 1688 important and enduring problem [RFC6077]. 1690 Pre-Congestion Notification (PCN) is another scheme that assigns 1691 alternative semantics to the ECN field. It uses ECT(1) to signify a 1692 less severe level of pre-congestion notification than CE [RFC6660]. 1693 However, the ECN field only takes on the PCN semantics if packets 1694 carry a Diffserv codepoint defined to indicate PCN marking within a 1695 controlled environment. PCN is required to be applied solely to the 1696 outer header of a tunnel across the controlled region in order not to 1697 interfere with any end-to-end use of the ECN field. Therefore a PCN 1698 region on the path would not interfere with any of the L4S service 1699 identifiers proposed in Appendix B. 1701 Authors' Addresses 1703 Koen De Schepper 1704 Nokia Bell Labs 1705 Antwerp 1706 Belgium 1708 Email: koen.de_schepper@nokia.com 1709 URI: https://www.bell-labs.com/usr/koen.de_schepper 1710 Bob Briscoe (editor) 1711 CableLabs 1712 UK 1714 Email: ietf@bobbriscoe.net 1715 URI: http://bobbriscoe.net/