idnits 2.17.1 draft-briscoe-tsvwg-l4s-diffserv-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (July 2, 2018) is 2124 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-25) exists of draft-ietf-tsvwg-aqm-dualq-coupled-04 == Outdated reference: A later version (-29) exists of draft-ietf-tsvwg-ecn-l4s-id-02 == Outdated reference: A later version (-20) exists of draft-ietf-tsvwg-l4s-arch-02 == Outdated reference: A later version (-10) exists of draft-ietf-tsvwg-le-phb-04 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Transport Area Working Group B. Briscoe 3 Internet-Draft CableLabs 4 Intended status: Informational July 2, 2018 5 Expires: January 3, 2019 7 Interactions between Low Latency, Low Loss, Scalable Throughput (L4S) 8 and Differentiated Services 9 draft-briscoe-tsvwg-l4s-diffserv-01 11 Abstract 13 L4S and Diffserv offer somewhat overlapping services (low latency and 14 low loss), but bandwidth allocation is out of scope for L4S. 15 Therefore there is scope for the two approaches to complement each 16 other, but also to conflict. This informational document explains 17 how the two approaches interact, how they can be arranged to 18 complement each other and in which cases one can stand alone without 19 needing the other. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on January 3, 2019. 38 Copyright Notice 40 Copyright (c) 2018 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 57 1.2. Document Roadmap . . . . . . . . . . . . . . . . . . . . 4 58 2. Architectural Comparison of L4S and Diffserv . . . . . . . . 4 59 2.1. Overlaps between L4S and Diffserv . . . . . . . . . . . . 4 60 2.2. Differences between L4S and Diffserv . . . . . . . . . . 4 61 3. Low Latency Diffserv Classes within a DualQ Bandwidth Pool . 5 62 4. DualQ Bandwidth Pool within a Hierarchy of (Diffserv) 63 Bandwidth Queues . . . . . . . . . . . . . . . . . . . . . . 9 64 4.1. DualQ Complemented by an Assured Bandwidth Service . . . 10 65 4.2. DualQ Complemented by a Guaranteed Low Latency Service . 12 66 4.3. DualQ Complemented by a Scavenger Service . . . . . . . . 13 67 5. Coupling More than Two AQMs within a Bandwidth Pool . . . . . 14 68 6. Best Practice for Classification and Marking . . . . . . . . 14 69 6.1. Never Re-Mark a DSCP . . . . . . . . . . . . . . . . . . 14 70 6.2. Classification Order . . . . . . . . . . . . . . . . . . 15 71 6.2.1. Classification Order: Problem . . . . . . . . . . . . 15 72 6.2.2. Classification Order: Solutions . . . . . . . . . . . 15 73 7. Policing and Traffic Conditioning . . . . . . . . . . . . . . 15 74 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 75 9. Security Considerations . . . . . . . . . . . . . . . . . . . 16 76 10. Comments Solicited . . . . . . . . . . . . . . . . . . . . . 16 77 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 16 78 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 79 12.1. Normative References . . . . . . . . . . . . . . . . . . 16 80 12.2. Informative References . . . . . . . . . . . . . . . . . 16 81 Appendix A. Open Issues . . . . . . . . . . . . . . . . . . . . 18 82 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 18 84 1. Introduction 86 The Low Latency Low Loss Scalable throughput (L4S) Internet service 87 [I-D.ietf-tsvwg-l4s-arch] provides a new Internet service that could 88 eventually replace best efforts, but with ultra-low queuing delay and 89 loss. A structure called the Dual-Queue Coupled AQM manages to 90 provide the L4S service alongside a second queue for Classic Internet 91 traffic, but without prejudging the bandwidth allocations between 92 them. L4S is orthogonal to allocation of bandwidth, so it can be 93 complemented by various bandwidth allocation approaches without 94 prejudging which one. 96 The Differentiated Services (Diffserv) architecture [RFC2475] 97 provides for various service classes, some defined globally, others 98 defined locally per network domain. Certain of these service classes 99 offer low latency and low loss, as well as differentiated allocation 100 of bandwidth. 102 Thus, L4S and Diffserv offer somewhat overlapping services (low 103 latency and low loss), but bandwidth allocation is out of scope for 104 L4S. Therefore there is scope for the two approaches to complement 105 each other, but also to conflict. This informational document 106 explains how the two approaches interact, how they can be arranged to 107 complement each other and in which cases one can stand alone without 108 needing the other. 110 1.1. Terminology 112 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 113 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 114 document are to be interpreted as described in [RFC2119]. In this 115 document, these words will appear with that interpretation only when 116 in ALL CAPS. Lower case uses of these words are not to be 117 interpreted as carrying RFC-2119 significance. 119 Classic service: The 'Classic' service is intended for all the 120 congestion control behaviours that currently co-exist with TCP 121 Reno [RFC5681] (e.g. TCP Cubic, Compound, SCTP, etc). 123 Low-Latency, Low-Loss and Scalable (L4S) service: The 'L4S' service 124 is intended for traffic from scalable congestion control 125 algorithms such as Data Centre TCP [RFC8257]. But it is also more 126 general--it will allow a set of congestion controls with similar 127 scaling properties to DCTCP to evolve. 129 Both Classic and L4S services can cope with a proportion of 130 unresponsive or less-responsive traffic as well (e.g. DNS, VoIP, 131 etc). 133 Pure L4S: L4S without unresponsive traffic. 135 Scalable Congestion Control: See [I-D.ietf-tsvwg-l4s-arch] for 136 definition. 138 Classic Congestion Control: See [I-D.ietf-tsvwg-l4s-arch] for 139 definition. 141 DualQ: Abbreviation for Dual-Queue Coupled AQM 142 [I-D.ietf-tsvwg-aqm-dualq-coupled], which is not a specific AQM, 143 but a framework for coupling two AQMs in order to provide L4S 144 service while doing no harm to 'Classic' traffic from traditional 145 sources. 147 ECN field: The Explicit Congestion Notification field [RFC3168] in 148 the IP header (v4 or v6). [RFC8311] has relaxed some of the 149 restrictions that RFC 3168 placed on the use of ECN, in order to 150 enable experiments like L4S, among others. 152 Site: A home, mobile device, small enterprise or campus, where the 153 network bottleneck is typically the access link to the site. Not 154 all network arrangements fit this model but it is a useful, widely 155 applicable generalisation. 157 1.2. Document Roadmap 159 {ToDo} 161 2. Architectural Comparison of L4S and Diffserv 163 This section compares the L4S architecture [I-D.ietf-tsvwg-l4s-arch] 164 with the Diffserv architecture [RFC2475]. 166 L4S uses an identifier [I-D.ietf-tsvwg-ecn-l4s-id] in the ECN field 167 in IP packet headers that is orthogonal to the Diffserv field 168 [RFC2474]. This is because the two approaches can either overlap or 169 complement each other, as outlined in the following two subsections. 171 2.1. Overlaps between L4S and Diffserv 173 L4S provides a low queuing latency, low loss Internet Service. 174 Specific Diffserv service classes also provide low latency and low 175 loss. 177 This means that it is possible to mix traffic from certain Diffserv 178 classes in the same queue as L4S traffic (see Section 3). 180 2.2. Differences between L4S and Diffserv 182 Bandwidth allocation: L4S is orthogonal to allocation of bandwidth, 183 so it can be complemented by various bandwidth allocation 184 approaches without prejudging which one. In contrast, with 185 Diffserv it was never possible to completely separate control of 186 latency and loss from allocation of bandwidth. The only 187 bandwidth-related aspect of L4S is that it ensures that the 188 capacity seeking behaviour of end-systems can scale with 189 increasing flow rate. 191 Differentiation vs. General improvement: Diffserv concerns give and 192 take of bandwidth, latency and loss between traffic classes. In 193 contrast, the separation of L4S from Classic traffic in separate 194 queues concerns incremental deployment of a general improvement in 195 latency and loss, without taking from the other queue. 197 Open vs. closed loop control: The Diffserv architecture requires the 198 source to keep traffic within a contract and, failing that, it has 199 mechanisms to enforce the contract. In this respect, Diffserv is 200 an open-loop control system that is primarily concerned with 201 keeping traffic within capacity limits. Nonetheless, there is an 202 element of closed-loop control in Diffserv. The weighted AQM 203 (e.g. WRED) used for Assured Forwarding [RFC2597] expects traffic 204 to seek to fill capacity and exploits the response to feedback of 205 congestion controllers at traffic sources (closed-loop). 206 Nonetheless, the Diffserv architecture still provides for traffic 207 conditioners that tag traffic that is outside the bandwidth 208 contract for each AF class (open-loop). Then out-of-contract 209 traffic can be discarded if it would otherwise lead to congestion. 211 L4S uses a similar closed-loop mechanism to the weighted AQM used 212 in Diffserv AF in order to ensure roughly equal per-flow 213 throughput between the L4S and Classic queues. That is, L4S 214 relies on the source's closed-loop response to feedback, not any 215 open-loop obligation of each source to keep within a traffic 216 contract. With L4S, any enforcement of per-flow throughput 217 (whether open-loop or closed) is set aside as a separate issue 218 that may or may not be addressed by separate mechanisms, dependent 219 on policy. 221 Per bottleneck vs. per domain: L4S can be independently and 222 incrementally deployed at certain bottlenecks. In contrast a 223 Diffserv system is domain-based consisting of the per-hop 224 behaviour of interior nodes and the traffic conditioning behaviour 225 of boundary nodes, which have to be deployed as a coordinated 226 whole. 228 Degree of multiplexing: Diffserv components such as traffic 229 conditioning are less applicable in access networks where 230 statistical multiplexing is low, whereas L4S was initially 231 designed for access networks, but is also applicable at larger 232 pinch-points (e.g. public peerings). 234 3. Low Latency Diffserv Classes within a DualQ Bandwidth Pool 236 The experimental Dual-Queue Coupled AQM 237 [I-D.ietf-tsvwg-aqm-dualq-coupled] consists of a pair of queues. One 238 provides a low latency low loss service but both have full access to 239 the same pool of bandwidth. When Diffserv was defined no mechanism 240 like this was available that could provide low latency without also 241 requiring bandwidth controls. All Diffserv's mechanisms for low 242 latency and low loss use some form of priority over bandwidth, then 243 apply a bandwidth constraint to prevent the lower priority traffic 244 from being starved. 246 This Diffserv bandwidth constraint has a flip side - it can also 247 provide a bandwidth assurance. However, in turn, bandwidth assurance 248 has both positive and negative aspects. It certainly prevents other 249 traffic encroaching on the bandwidth of the low latency class, but it 250 also carves off a partition within which low latency sessions are 251 more prone to encroach on each other. 253 The DualQ offers an alternative where low latency traffic can access 254 the whole pool of bandwidth (in effect, the largest possible 255 bandwidth constraint). This is expected to be preferred by many 256 network operators and users who would rather not set a bandwidth 257 limit for their low latency traffic - particularly at links in access 258 networks where the very low level of flow multiplexing makes the 259 bandwidth shares of different traffic classes nearly impossible to 260 predict. Nonetheless, if a bandwidth partition is required for 261 bandwidth assurance purposes, it can still be provided separately 262 (see Section 4). 264 The DualQ classifies packets with the ECN field set to ECT(1) or CE 265 into the low latency low loss (L) queue. The L queue maintains a low 266 latency low loss service primarily because an L4S source paces its 267 packets and is linearly responsive to ECN markings, which earns it 268 the right to set the ECT(1) codepoint [I-D.ietf-tsvwg-ecn-l4s-id] 269 [RFC8311]. 271 Nonetheless, a low level of non-L4S traffic can share the L queue 272 without compromising the low latency and low loss of the service. 273 Certain existing Diffserv classes are already intended as low latency 274 and low loss services. An operator could use the DualQ instead of 275 traditional Diffserv queues to give a few of these classes the 276 benefit of low latency and access to the whole pool of bandwidth. 278 However, that would only be safe for those Diffserv service classes 279 that would not risk ruining the low latency of the service. 280 Therefore, an operator must take care to only classify a Diffserv 281 traffic class into the L queue if it is expected to send smoothly 282 without multi-packet bursts. Below we give examples of classes that 283 should (and should not) be safe to mix into the L queue. 285 Table 1 lists the Diffserv service classes that have been allocated 286 global use Diffserv codepoints (DSCPs) from Pool 1. They are 287 described in RFC 4594 [RFC4594] and its updates ([RFC5865] and 288 [I-D.ietf-tsvwg-le-phb] so far). An operator that only deploys a 289 DualQ [I-D.ietf-tsvwg-aqm-dualq-coupled] but not the relevant 290 Diffserv PHBs could classify those with an 'L' in the 'Coupled Queue' 291 column (or local use DSCPs with similar characteristics) into its L 292 queue, irrespective of the setting of the ECN field. 294 +--------------------+-------------+--------+-------+---------------+ 295 | Service Class Name | DSCP Name | DSCP | AQM? | Coupled Queue | 296 +--------------------+-------------+--------+-------+---------------+ 297 | Network Control{1} | CS7 | 111000 | Y & N | L if L4S | 298 | Network Control | CS6 | 110000 | Y & N | L if L4S | 299 | OAM | CS2 | 010000 | Y & N | L if L4S | 300 | Signalling | CS5 | 101000 | N | L if L4S{2} | 301 | Telephony | EF | 101110 | N | L | 302 | RFC 5865 | Voice-Admit | 101100 | N | L{3} | 303 | R-T Interactive | CS4 | 100000 | N | L if L4S{4} | 304 | MM Conferencing | AF4n | 100nn0 | Y | L if L4S | 305 | Broadcast Video | CS3 | 011000 | N | L if L4S{4} | 306 | MM Streaming | AF3n | 011nn0 | Y | L if L4S | 307 | Low Latency Data | AF2n | 010nn0 | Y | L if L4S | 308 | High Thru'put Data | AF1n | 001nn0 | Y | L if L4S{5} | 309 | Standard | DF (CS0) | 000000 | Y | L if L4S | 310 | Low Priority Data | LE{6} | 000001 | Y | L if L4S{7} | 311 +--------------------+-------------+--------+-------+---------------+ 313 Some service class names have been abbreviated to fit. Abbreviations 314 are expanded in RFC 4594 or its updates. For the assured forwarding 315 (AF) DSCP names, the digit 'n' represents 1, 2 or 3 and the 316 corresponding binary digits 'nn' in the DSCP value represent 01,10 or 317 11. The 'Coupled Queue' column is explained in the text. 319 Table 1: Mapping of RFC4594 Diffserv Service Classes in a Coupled AQM 321 Notes for Table 1: 323 {1}: Reserved by RFC 2474 [RFC2474]. 325 {2}: Superficially, CS5 is a candidate for classification into the 326 L queue irrespective of its ECN field, given application 327 signalling is bursty but usually lightweight. However, at 328 least one major equipment vendor uses CS5 by default to 329 indicate unresponsive broadcast video traffic (to which RFC 330 4594 allocates CS3). 332 {3}: Voice-Admit [RFC5865] could be given priority over Expedited 333 Forwarding (EF) [RFC3246]. 335 {4}: The Real-Time Interactive and Broadcast Video service classes 336 (or any equivalent local-use classes) are intended for 337 inelastic traffic. Therefore they would not be expected to 338 mark themselves as ECN-capable. If they did they would be 339 claiming to be elastic and therefore eligible for 340 classification into the L queue (subject to any policing). 341 These classes should not be classified into the L queue on the 342 basis of DSCP alone, because high bandwidth unresponsive 343 traffic with potentially variable rate is not compatible with 344 the L4S service. 346 {5}: High Throughput Data (or any equivalent local-use class) might 347 use the L4S service because of its support for scalable 348 congestion control. 350 {6}: [I-D.ietf-tsvwg-le-phb] updates RFC 4594 to deprecate using 351 CS1 for Lower Effort (LE). 353 {7}: If a packet is marked LE and ECT(1) and the operator has 354 solely provided a DualQ, this recommends that the packet is 355 classified into the L queue. This could result in LE traffic 356 competing for bandwidth with other classes of traffic in the L 357 queue, but at least it should not harm the latency of other 358 traffic. This is because the ECT(1) marking means the source 359 "MUST" use a scalable congestion control 360 [I-D.ietf-tsvwg-ecn-l4s-id], but the LE marking only means it 361 "SHOULD" use an LBE congestion control 362 [I-D.ietf-tsvwg-le-phb]. 364 Those classes with an 'L' in the 'DualQ-Coupled' column would not be 365 expected to have the ECT(1) codepoint set because they are generally 366 unresponsive to congestion. Nonetheless, they could coexist in the 367 same queue as L4S traffic because traffic in all of these classes is 368 expected to arrive smoothly, not in bursts of more than a few 369 packets. Therefore an operator could configure a DualQ Coupled AQM 370 to classify such packets into the L queue solely based on their DSCP, 371 irrespective of their ECN codepoint [I-D.ietf-tsvwg-ecn-l4s-id]. 373 Otherwise, [I-D.ietf-tsvwg-ecn-l4s-id] requires that any other DSCP 374 has no effect on classification into the L queue. Thus a packet of 375 any other DSCP will not be classified into the L queue unless it 376 carries an ECT(1) or CE codepoint in the ECN field. This is shown as 377 'L if L4S' in in the 'DualQ-Coupled' column of Table 1. 379 4. DualQ Bandwidth Pool within a Hierarchy of (Diffserv) Bandwidth 380 Queues 382 The DualQ Coupled AQM offers an L queue that provides low latency low 383 loss service but it pools bandwidth with the Classic (C) service as 384 if they shared a single FIFO. As explained earlier, unlike previous 385 Diffserv low latency mechanisms, the L queue can offer low latency 386 without needing to limit its bandwidth. 388 Typically the DualQ will be able to use all the bandwidth available 389 to a customer site, e.g. a household, a campus or a mobile node, as a 390 single pool. However, this section considers scenarios where the 391 network operator might want to carve off a fraction of a site's 392 bandwidth for other purposes, for instance: 394 1. to ensure that a particularly demanding application (e.g. a 395 virtual reality session) survives even if excess traffic 396 overloads the remainder of the site's bandwidth; 398 2. to give guaranteed low latency to a particular application (e.g. 399 industrial process control), if the statistically assured low 400 latency of the L queue is insufficiently stable; 402 3. to provide a bandwidth scavenger service that will have no effect 403 on any other applications at the site, but will scavenge any 404 unused bandwidth, for instance to transfer backups or large data 405 sets. 407 In all cases, it is assumed that the DualQ has to be able to borrow 408 back any of the carved off bandwidth that is unused by the other 409 service. 411 The following three subsections present solutions for each of the 412 above scenarios. Depending on the reader's viewpoint, each scenario 413 can be seen as: 415 o either taking a queue within an existing Diffserv hierarchy and 416 splitting it into L4S and Classic queues; 418 o or building a queuing hierarchy around a pre-existing dual L4S/ 419 Classic queue. 421 In each case, the DualQ remains as an indivisible 'atomic' component 422 as if it were a single queue with a single pool of bandwidth (but 423 that can either be used for low latency or classic service). 425 The three examples represent the three main ways that this queue-like 426 'atom' can be included in a hierarchy of other queues. Without loss 427 of generality only one other queue complements the DualQ in each 428 case, but it would be straightforward to extend the examples with 429 more queues. 431 Although these examples are framed in the context of IP and Diffserv, 432 similar queuing hierarchies could be constructed at a lower layer, as 433 long as it supported a similar capability to ECN and a similar 434 Traffic Class identifier to Diffserv. 436 4.1. DualQ Complemented by an Assured Bandwidth Service 438 Figure 1 shows a DualQ complemented by an additional queue to add a 439 bandwidth assured service. It is assumed that the operator 440 classifies certain packets into the assured bandwidth queue, perhaps 441 by class of service, source address or 5-tuple flow ID. 443 ---------+--+ 444 Assured b/w | |-----------. 445 ---------+--+ \ Weighted 446 w\.-.scheduler 447 , -----------++ ( )---> 448 | L .->||---. /`-' 449 DualQ | -------/---++ c\.-. / 450 b/w < (Coupling ( )--' 451 pool | ----+--\----+ /`-'Conditional 452 | C | \ |---' priority 453 ` ----+-------+ scheduler 455 Figure 1: How to Complement a DualQ with an Assured Bandwidth Service 457 The DualQ is used as if it were an indivisible 'atomic' component, 458 unchanged from its original description in 459 [I-D.ietf-tsvwg-aqm-dualq-coupled]: 461 o The outputs of the AQMs in the two queues (L and C) are coupled 462 together so that L4S sources leave enough space for C packets so 463 that all 'standard' flows get roughly equal throughput; 465 o A scheduler recombines the outputs of the two queues, giving 466 conditional priority to L packets (the condition prevents 467 starvation of the C queue if any L traffic misbehaves). 469 A weighted scheduler, e.g. weighted round robin (WRR), is used to 470 combine the outputs of the assured bandwidth queue and the DualQ. It 471 is configured with weight w for the assured bandwidth queue. Then, 472 packets requesting assured bandwidth will have priority access to 473 fraction w of the link capacity. However, whenever the assured 474 bandwidth queue is idle or under-utilized, the DualQ can borrow the 475 balance of the bandwidth. Likewise the assured bandwidth queue can 476 borrow more than fraction w if the DualQ under-utilizes its remaining 477 share. 479 Note that a weighted scheduler such as WRR can be used to implement 480 the conditional priority scheduler between the L and C queues. 481 However, the system will not work as intended if the two weighted 482 schedulers in series are replaced by a single three-input weighted 483 scheduler. This is because, whenever one queue under-uses its 484 weighted share, a weighted scheduler allows the other queue to borrow 485 unused capacity. Whenever traffic is present in the C queue, the 486 coupling ensures that L traffic makes space for it by underutiliizing 487 its share of the first scheduler. If the assured bandwidth queue was 488 also served by the same scheduler, the assured bandwidth service 489 would continually borrow the spare capacity left by the L queue that 490 was intended for the C queue. 492 The assured bandwidth service could itself also support applications 493 using low latency low loss and scalable throughput (L4S). This would 494 be done by serving assured bandwidth traffic with a DualQ (Figure 2) 495 and, as usual, confining legacy queue-building traffic to the C 496 queue. 498 , -----------++ Conditional 499 | L .->||---. priority 500 Assured | -------/---++ c\.-.scheduler 501 b/w < (Coupling ( )--. 502 | ----+--\----+ /`-' \ 503 | C | \ |---' \ Weighted 504 ` ----+-------+ w\.-.scheduler 505 ( )---> 506 , -----------++ /`-' 507 | L .->||---. / 508 DualQ | -------/---++ c\.-. / 509 b/w < (Coupling ( )--' 510 pool | ----+--\----+ /`-'Conditional 511 | C | \ |---' priority 512 ` ----+-------+ scheduler 514 Figure 2: How to Complement a DualQ with an Assured Bandwidth Service 515 that also Supports L4S 517 The symmetry of Figure 2 reveals that both DualQs actually have 518 assured bandwidth. Nonetheless, the label 'Assured bandwidth' is 519 only really meaningful from a per-application perspective if the 520 traffic classified into that DualQ is limited to a small number of 521 application sessions at any one time. 523 4.2. DualQ Complemented by a Guaranteed Low Latency Service 525 Figure 3 shows a DualQ complemented by an additional queue to add a 526 guaranteed latency service. It is assumed that the operator 527 classifies certain packets into the guaranteed latency queue, perhaps 528 by class of service, source address or 5-tuple flow ID. 530 o Token bucket 531 | o |rate/burst limiter 532 |___| 533 |___| -----------++ 534 Guaranteed low latency||-----------. 535 -----------++ \ Priority 536 1\.-.scheduler 537 , -----------++ ( )---> 538 | L .->||---. /`-' 539 DualQ | -------/---++ c\.-. / 540 b/w < (Coupling ( )--' 541 pool | ----+--\----+ /`-'Conditional 542 | C | \ |---' priority 543 ` ----+-------+ scheduler 545 Figure 3: How to Complement a DualQ with a Guaranteed Low Latency 546 Service 548 As in all the previous example, the DualQ is used as if it were an 549 indivisible 'atomic' component. 551 A strict priority scheduler is used to combine the outputs of the 552 guaranteed latency queue and the DualQ. Guaranteed low latency 553 traffic is shown as subject to a token bucket that limits rate and 554 tightly limits burst size, which ensures that: 556 o Excessive guaranteed latency traffic cannot abuse its priority and 557 cause the DualQ to starve; 559 o Guaranteed latency traffic cannot ruin its own latency guarantees 560 - it has to keep to a the traffic contract enforced by the token 561 bucket. 563 In a traditional Diffserv architecture, the token bucket would be 564 deployed at the ingress network edge, to limit traffic at each entry 565 point. Alternatively, the token bucket could be deployed directly in 566 front of the queue, where it would only limit the total traffic from 567 all entry points to the network. For an access link into a network, 568 these two alternative would amount to the same thing. 570 Whenever the guaranteed latency queue is idle or under-utilized, the 571 DualQ can borrow the balance of the bandwidth. However, the 572 guaranteed latency queue cannot borrow more than the token bucket 573 allows, even if the DualQ under-utilizes its remaining share. 575 4.3. DualQ Complemented by a Scavenger Service 577 Figure 3 shows a DualQ complemented by an additional queue to add a 578 bandwidth scavenger service. It is assumed that the operator 579 classifies certain packets into the scavenger queue, probably by 580 class of service, e.g. the global-use Lower Effort (LE) Diffserv 581 codepoint [I-D.ietf-tsvwg-le-phb]. 583 , -----------++ Conditional 584 | L .->||---. priority 585 DualQ | -------/---++ c\.-.scheduler 586 b/w < (Coupling ( )--. 587 pool | ----+--\----+ /`-' \ Priority 588 | C | \ |---' 1\.-.scheduler 589 ` ----+-------+ ( )---> 590 /`-' 591 -+-----------+ / 592 Bandwidth|scavenger |-----------' 593 -+-----------+ 595 Figure 4: How to Complement a DualQ with a Bandwidth Scavenger 596 Service 598 As in all the previous example, the DualQ is used as if it were an 599 indivisible 'atomic' component. 601 A strict priority scheduler is used to combine the outputs of the 602 DualQ and the scavenger service. Section 2 of 603 [I-D.ietf-tsvwg-le-phb] suggests alternative mechanisms. 605 Whenever the DualQ is idle or under-utilized, the scavenger service 606 can borrow the balance of the bandwidth. In contrast to the previous 607 guaranteed latency example, no rate limiter is needed on the DualQ 608 because, by definition, the scavenger service is expected to starve 609 if the higher priority service is using all the capacity. 611 5. Coupling More than Two AQMs within a Bandwidth Pool 613 The Diffserv Assured Forwarding (AF) classes of service [RFC2597] use 614 an AQM with differently weighted outputs, e.g. WRED, to provide 615 weighted congestion feedback to the transport layer. Flows 616 classified to use a higher weight AQM each take more of the available 617 capacity, because the weighted AQM has fooled their congestion 618 controller into detecting that the bottleneck is more lightly loaded. 620 A similar mechanism can be used to add throughput differentiation to 621 either or both of the queues within a DualQ. Figure 5 illustrates an 622 example with an AQM offering three weights within the L queue, where 623 L1 gets the highest throughput per flow. It would be a matter of 624 operator policy to choose which of the three L4S AQMs the Classic AQM 625 would couple to. If it were coupled to L3, then C and L3 flows would 626 get roughly equal throughput, while L2 and L1 flows would get more. 628 , -----------++ 629 | L1 || 630 | L2 ||--. 631 | L3 .-> || \ 632 DualQ | -----/-----++ c\.-. 633 b/w < ( Coupling ( )---> 634 pool | ----+\------+ /`-'Conditional 635 | C | \ |---' priority 636 ` ----+-------+ scheduler 638 Figure 5: Coupling the Classic AQM to Multiple L4S AQMs 640 Note: this structure seems straightforward to implement, but the 641 authors are not aware of any implementation or evaluation of AQMs 642 that are both weighted and coupled to other AQMs. 644 6. Best Practice for Classification and Marking 646 6.1. Never Re-Mark a DSCP 648 It is not a DualQ's job to alter Diffserv codepoints to attempt to 649 make other downstream AQMs classify selected packets in certain ways. 650 Each DualQ Coupled AQM is independently (but hopefully consistently) 651 configured to select certain DSCPs for classification into the L 652 queue. It never alters the DSCP nor the ECN codepoint (except 653 setting CE to indicate that congestion was experienced) 654 [I-D.ietf-tsvwg-aqm-dualq-coupled]. 656 6.2. Classification Order 658 6.2.1. Classification Order: Problem 660 The above wide range of possible structures raises the question of 661 which order it would be more efficient for classifier rules to take: 662 DSCP before ECN, ECN before DSCP or some hybrid. 664 On the one hand, for a structure like that in Figure 1 it would make 665 sense to classify on DSCP first, then ECN. Otherwise, if packets 666 were classified on ECN first, an extra merge stage would be required 667 because the assured bandwidth queue handles all ECN codepoints for a 668 particular DSCP. 670 On the other hand, for a structure like that in Figure 5 it would 671 make sense to classify on ECN first, then DSCP. Otherwise, again an 672 extra merge stage would be needed, because the C queue handles all 673 DSCPs but only some ECN codepoints. 675 A hybrid of these two scenarios would be possible, for instance where 676 the L queue in Figure 1 was further broken down into three weighted 677 AQMs, as in Figure 5. In this case, the ideal matching order would 678 be DSCP, ECN, DSCP. 680 6.2.2. Classification Order: Solutions 682 Probably the most straightforward solution would be to classify in a 683 single stage over all 8 octets of the IPv6 Traffic Class field or the 684 former IPv4 TOS octet, irrespective of the boundary between the 6-bit 685 DS field and the 2-bit ECN field [RFC3260]. As long as hardware 686 supports this, it will be possible because all the inputs to the 687 queues are at the same level of hierarchy, even though the outputs 688 form a multi-level hierarchy of schedulers in some cases. 690 Pre-existing classifier hardware might consider the 6-bit and 2-bit 691 fields as separate. Then it would seem most efficient for the order 692 of the classifiers to depend on the structure of the queues being 693 classified (given the structure has to have been designed before the 694 classifiers are designed). 696 7. Policing and Traffic Conditioning 698 {ToDo: L4S latency policing is discussed in the Security 699 Considerations section of [I-D.ietf-tsvwg-l4s-arch]. This section 700 will compare Diffserv traffic conditioning with L4S latency 701 policing.} 703 8. IANA Considerations 705 This specification contains no IANA considerations. 707 9. Security Considerations 709 {ToDo} 711 10. Comments Solicited 713 Comments and questions are encouraged and very welcome. They can be 714 addressed to the IETF Transport Area working group mailing list 715 , and/or to the authors. 717 11. Acknowledgements 719 Thanks to Greg White, David Black, Wes Eddy and Gorry Fairhurst for 720 their useful discussions prior to this -00 draft. 722 12. References 724 12.1. Normative References 726 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 727 Requirement Levels", BCP 14, RFC 2119, 728 DOI 10.17487/RFC2119, March 1997, 729 . 731 12.2. Informative References 733 [I-D.ietf-tsvwg-aqm-dualq-coupled] 734 Schepper, K., Briscoe, B., Bondarenko, O., and I. Tsang, 735 "DualQ Coupled AQMs for Low Latency, Low Loss and Scalable 736 Throughput (L4S)", draft-ietf-tsvwg-aqm-dualq-coupled-04 737 (work in progress), March 2018. 739 [I-D.ietf-tsvwg-ecn-l4s-id] 740 Schepper, K. and B. Briscoe, "Identifying Modified 741 Explicit Congestion Notification (ECN) Semantics for 742 Ultra-Low Queuing Delay (L4S)", draft-ietf-tsvwg-ecn-l4s- 743 id-02 (work in progress), March 2018. 745 [I-D.ietf-tsvwg-l4s-arch] 746 Briscoe, B., Schepper, K., and M. Bagnulo, "Low Latency, 747 Low Loss, Scalable Throughput (L4S) Internet Service: 748 Architecture", draft-ietf-tsvwg-l4s-arch-02 (work in 749 progress), March 2018. 751 [I-D.ietf-tsvwg-le-phb] 752 Bless, R., "A Lower Effort Per-Hop Behavior (LE PHB)", 753 draft-ietf-tsvwg-le-phb-04 (work in progress), March 2018. 755 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 756 "Definition of the Differentiated Services Field (DS 757 Field) in the IPv4 and IPv6 Headers", RFC 2474, 758 DOI 10.17487/RFC2474, December 1998, 759 . 761 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 762 and W. Weiss, "An Architecture for Differentiated 763 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 764 . 766 [RFC2597] Heinanen, J., Baker, F., Weiss, W., and J. Wroclawski, 767 "Assured Forwarding PHB Group", RFC 2597, 768 DOI 10.17487/RFC2597, June 1999, 769 . 771 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 772 of Explicit Congestion Notification (ECN) to IP", 773 RFC 3168, DOI 10.17487/RFC3168, September 2001, 774 . 776 [RFC3246] Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec, 777 J., Courtney, W., Davari, S., Firoiu, V., and D. 778 Stiliadis, "An Expedited Forwarding PHB (Per-Hop 779 Behavior)", RFC 3246, DOI 10.17487/RFC3246, March 2002, 780 . 782 [RFC3260] Grossman, D., "New Terminology and Clarifications for 783 Diffserv", RFC 3260, DOI 10.17487/RFC3260, April 2002, 784 . 786 [RFC4594] Babiarz, J., Chan, K., and F. Baker, "Configuration 787 Guidelines for DiffServ Service Classes", RFC 4594, 788 DOI 10.17487/RFC4594, August 2006, 789 . 791 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 792 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 793 . 795 [RFC5865] Baker, F., Polk, J., and M. Dolly, "A Differentiated 796 Services Code Point (DSCP) for Capacity-Admitted Traffic", 797 RFC 5865, DOI 10.17487/RFC5865, May 2010, 798 . 800 [RFC8257] Bensley, S., Thaler, D., Balasubramanian, P., Eggert, L., 801 and G. Judd, "Data Center TCP (DCTCP): TCP Congestion 802 Control for Data Centers", RFC 8257, DOI 10.17487/RFC8257, 803 October 2017, . 805 [RFC8311] Black, D., "Relaxing Restrictions on Explicit Congestion 806 Notification (ECN) Experimentation", RFC 8311, 807 DOI 10.17487/RFC8311, January 2018, 808 . 810 Appendix A. Open Issues 812 o The Abstract promises "in which cases one can stand alone without 813 needing the other", but that's TBA in the text. 815 o Answer the interaction question between Diffserv and L4S the other 816 way round as well: Which global PHBs is a DualQ applicable to? 818 o Document Roadmap TBA 820 o Mapping to 802.11 user priorities (or LTE QCIs)? Not strictly 821 within the scope, but perhaps desirable to add, or at least to 822 mention how L4S (experimental) would affect RFC8325 which gives 823 (standards track) mappings between Diffserv and 802.11. 825 o Identify L4S-friendly rate policers 827 o Comparison between L4S policing and Diffserv traffic conditioning 828 is TBA 830 o Security Considerations are TBA (largely depends on the previous 831 bullet) 833 Author's Address 835 Bob Briscoe 836 CableLabs 837 UK 839 Email: ietf@bobbriscoe.net 840 URI: http://bobbriscoe.net/