idnits 2.17.1 draft-white-tsvwg-nqb-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 11, 2019) is 1874 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-20) exists of draft-ietf-tsvwg-l4s-arch-03 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Transport Area Working Group G. White 3 Internet-Draft CableLabs 4 Intended status: Standards Track March 11, 2019 5 Expires: September 12, 2019 7 Identifying and Handling Non Queue Building Flows in a Bottleneck Link 8 draft-white-tsvwg-nqb-01 10 Abstract 12 This draft proposes the definition of a standardized DSCP to identify 13 Non-Queue-Building flows, along with a Per-Hop-Behavior (PHB) that 14 provides a separate queue for such flows. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at https://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on September 12, 2019. 33 Copyright Notice 35 Copyright (c) 2019 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (https://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 51 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 3 52 3. Non-Queue Building Flows . . . . . . . . . . . . . . . . . . 3 53 4. Endpoint Marking and Queue Protection . . . . . . . . . . . . 3 54 5. Non Queue Building PHB and DSCP . . . . . . . . . . . . . . . 4 55 6. End-to-end Support . . . . . . . . . . . . . . . . . . . . . 5 56 7. Relationship to L4S . . . . . . . . . . . . . . . . . . . . . 6 57 8. Comparison to Existing Approaches . . . . . . . . . . . . . . 6 58 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7 59 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 60 11. Security Considerations . . . . . . . . . . . . . . . . . . . 7 61 12. Informative References . . . . . . . . . . . . . . . . . . . 8 62 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 9 64 1. Introduction 66 Residential broadband internet services are commonly configured with 67 a single bottleneck link (the access network link) upon which the 68 service definition is applied. The service definition, typically an 69 upstream/downstream data rate tuple, is implemented as a configured 70 pair of rate shapers that are applied to the user's traffic. In such 71 networks, the quality of service that each application receives, and 72 as a result, the quality of experience that it generates for the user 73 is influenced by the characteristics of the access network link. 75 The vast majority of packets that are carried by residential 76 broadband access networks are managed by an end-to-end congestion 77 control algorithm, such as Reno, Cubic or BBR. These congestion 78 control algorithms attempt to seek the available capacity of the end- 79 to-end path (which in the case of residential broadband networks, can 80 frequently be the access network link capacity), and in doing so 81 generally overshoot the available capacity, causing a queue to build- 82 up at the bottleneck link. This queue build up results in queuing 83 delay that the application experiences as variable latency, and 84 commonly results in packet loss as well. 86 In contrast to congestion-controlled applications, there are a 87 variety of relatively low data rate applications that do not 88 materially contribute to queueing delay, but are nonetheless 89 subjected to it by sharing the same bottleneck link in the access 90 network. Many of these applications may be sensitive to latency or 91 latency variation, as well as packet loss, and thus produce a poor 92 quality of experience in such conditions. 94 Active Queue Management (AQM) mechanisms (such as PIE [RFC8033], 95 DOCSIS-PIE [RFC8034], or CoDel [RFC8289]) can improve the quality of 96 experience for latency sensitive applications, but there are 97 practical limits to the amount of improvement that can be achieved 98 without impacting the throughput of capacity-seeking applications. 100 This document considers differentiating between these two classes of 101 traffic in bottleneck links in order that both classes can deliver 102 exceptional quality of experience for their applications. 104 2. Requirements Language 106 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 107 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 108 document are to be interpreted as described in RFC 2119 [RFC2119]. 110 3. Non-Queue Building Flows 112 There are many applications that send traffic at relatively low data 113 rates and/or in a fairly smooth and consistent manner such that they 114 are highly unlikely to exceed the available capacity of the network 115 path between source and sink. Such applications are ideal candidates 116 to be queued separately from the capacity-seeking applications that 117 cause queue buildups and latency. 119 These Non-queue-building (NQB) flows are typically UDP flows, which 120 send traffic at a lower data rate and don't seek the capacity of the 121 link (examples: online games, voice chat, dns lookups). Here the 122 data rate is essentially limited by the Application itself. In 123 contrast, Queue-building (QB) flows include traffic which uses the 124 Traditional TCP, QUIC, BBR or other TCP variants. 126 There are a lot of great examples of applications that fall very 127 neatly into these two categories, but there are also application 128 flows that may be in a gray area in between (e.g. they are NQB on 129 high-speed links, but QB on low-speed links). 131 4. Endpoint Marking and Queue Protection 133 This memo proposes that application endpoints apply a marking, 134 utilizing the Diffserv field of the IP header, to packets of NQB 135 flows that could then be used by the network to differentiate between 136 QB and NQB flows. It is important for such a marking to be 137 universally agreed upon, rather than being locally defined by the 138 network operator, such that applications could be written to apply 139 the marking without regard to local network policies. 141 Some questions that arise when considering endpoint marking are: How 142 can an application determine whether it is queue building or not, 143 given that the sending application is generally not aware of the 144 available capacity of the path to the receiving endpoint? Even in 145 cases where an application is aware of the capacity of the path, how 146 can it be sure that the available capacity (considering other flows 147 that may be sharing the path) would be sufficient to result in the 148 application's traffic not causing a queue to form? In an unmanaged 149 environment, how can networks trust endpoint marking, why wouldn't 150 all applications mark their packets as NQB? 152 As an answer the last question, it would be worthwhile to note that 153 the NQB designation and marking would be intended to convey 154 verifiable traffic behavior, not needs or wants. Also, it would be 155 important that incentives are aligned correctly, i.e. that there is a 156 benefit to the application in marking its packets correctly, and no 157 benefit for an application in intentionally mismarking its traffic. 158 Thus, a useful property of nodes that support separate queues for NQB 159 and QB flows would be that for NQB flows, the NQB queue provides 160 better performance (considering latency, loss and throughput) than 161 the QB queue; and for QB flows, the QB queue provides better 162 performance (considering latency, loss and throughput) than the NQB 163 queue. 165 Even so, it is possible that due to an implementation error or 166 misconfiguration, a QB flow would end up getting mismarked as NQB, or 167 vice versa. In the case of an NQB flow that isn't marked as NQB and 168 ends up in the QB queue, it would only impact its own quality of 169 service, and so it seems to be of lesser concern. However, a QB flow 170 that is mismarked as NQB would cause queuing delays for all of the 171 other flows that are sharing the NQB queue. 173 To prevent this situation from harming the performance of the real 174 NQB flows, it would likely be valuable to support a "queue 175 protection" function that could identify QB flows that are mismarked 176 as NQB, and reclassify those flows/packets to the QB queue. This 177 would benefit the reclassified flow by giving it access to a large 178 buffer (and thus lower packet loss rate), and would benefit the 179 actual NQB flows by preventing harm (increased latency variability) 180 to them. Such a function should be implemented in an objective and 181 verifiable manner, basing its decisions upon the behavior of the flow 182 rather than on application-level constructs. 184 5. Non Queue Building PHB and DSCP 186 This section uses the DiffServ nomenclature of per-hop-behavior (PHB) 187 to describe how a network node could provide better quality of 188 service for NQB flows without reducing performance of QB flows. 190 A node supporting the NQB PHB MUST provide a queue for non-queue- 191 building traffic separate from the queue used for queue-building 192 traffic. This queue SHOULD support a latency-based queue protection 193 mechanism that is able to identify queue-building behavior in flows 194 that are classified into the queue, and to redirect flows causing 195 queue build-up to a different queue. One example algorithm can be 196 found in Annex P of [DOCSIS-MULPIv3.1]. 198 While there may be some similarities between the characteristics of 199 NQB flows and flows marked with the Expedited Forwarding DSCP, the 200 NQB PHB would differ from the Expedited Forwarding PHB in several 201 important ways. 203 o NQB traffic is not rate limited or rate policed. Rather, the NQB 204 queue would be expected to support a latency-based queue 205 protection mechanism that identifies NQB marked flows that are 206 beginning to cause latency, and redirects packets from those flows 207 to the queue for QB flows. 209 o The node supporting the NQB PHB makes no guarantees on latency or 210 data rate for NQB marked flows, but instead aims to provide sub- 211 millisecond queuing delays for as many such marked flows as it 212 can, and shed load when needed. 214 o EF is commonly used exclusively for voice traffic, for which 215 additional functions are applied, such as admission control, 216 accounting, prioritized delivery, etc. 218 In networks that support the NQB PHB, it may be preferred to also 219 include traffic marked EF (101110b) in the NQB queue. The choice of 220 the 0x2A codepoint (101010b) for NQB would conveniently allow a node 221 to select these two codepoints using a single mask pattern of 222 101x10b. 224 Additionally, WiFi routers and APs that support WiFi MultiMedia 225 commonly use the upper three bits of the DiffServ Field to select the 226 WiFi User Priority. In the case of the 0x2A codepoint, this would 227 map to UP_5 which is in the "Video" Access Category (one level above 228 "Best Effort"). 230 6. End-to-end Support 232 In contrast to the existing standard DSCPs, which are typically only 233 enforced within a DiffServ Domain (e.g. an AS), this DSCP would be 234 intended for end-to-end usage across the Internet. Some access 235 network service providers bleach the Diffserv field on ingress into 236 their network, and in some cases apply their own DSCP for internal 237 usage. Access networks that support the NQB PHB would need to permit 238 the NQB PHB to pass through this bleaching operation such that the 239 PHB can be provided at the access network link. 241 7. Relationship to L4S 243 The dual-queue mechanism described in this draft is similar to, and 244 is intended to be compatible with [I-D.ietf-tsvwg-l4s-arch]. 246 8. Comparison to Existing Approaches 248 Traditional QoS mechanisms focus on prioritization in an attempt to 249 achieve two goals, reduced latency for "latency-sensitive" traffic, 250 and increased bandwidth availability for "important" applications. 251 Applications are generally given priority in proportion to some 252 combination of latency-sensitivity and importance. 254 Downsides to this approach include the difficulties in sorting out 255 what priority level each application should get (making the value 256 judgement as to latency-sensitivity and importance), associating 257 packets to priority levels (lots of classifier state, or trusting 258 endpoint markings and the value judgements that they convey), 259 ensuring that high priority traffic doesn't starve lower priority 260 traffic (admission control, weighted scheduling, etc. are possible 261 solutions). This solution can work in a managed network, where the 262 network operator can control the usage of the QoS mechanisms, but has 263 not been adopted end-to-end across the internet. 265 Flow queueing approaches (such as fq_codel [RFC8290]), on the other 266 hand, achieve latency improvements by associating packets into "flow" 267 queues and then prioritizing "sparse flows", i.e. packets that arrive 268 to an empty flow queue. Flow queueing does not attempt to 269 differentiate between flows on the basis of value (importance or 270 latency-sensitivity), it simply gives preference to sparse flows, and 271 tries to guarantee that the non-sparse flows all get an equal share 272 of the remaining channel capacity and are interleaved with one 273 another. As a result, fq mechanisms could be considered more 274 appropriate for unmanaged environments and general internet traffic. 276 Downsides to this approach include loss of low latency performance 277 due to the possibility of hash collisions (where a sparse flow shares 278 a queue with a bulk data flow), complexity in managing a large number 279 of queues in certain implementations, and the DRR scheduling, which 280 enforces that each non-sparse flow gets an equal fraction of link 281 bandwidth, causes problems with VPNs and other tunnels, exhibits poor 282 behavior with less-aggressive CA algos, e.g. LEDBAT, and exhibits 283 poor behavior with RMCAT CA algos. In effect the network element is 284 making a decision as to what constitutes a flow, and then forcing all 285 such flows to take equal bandwidth at every instant. 287 The Dual-queue approach achieves the main benefit of fq_codel: 288 latency improvement without value judgements, without the downsides. 290 The distinction between NQB flows and QB flows is similar to the 291 distinction made between "sparse flow queues" and "non-sparse flow 292 queues" in fq_codel. In fq_codel, a flow queue is considered sparse 293 if it is drained completely by each packet transmission, and remains 294 empty for at least one cycle of the round robin over the active flows 295 (this is approximately equivalent to saying that it utilizes less 296 than its fair share of capacity). While this definition is 297 convenient to implement in fq_codel, it isn't the only useful 298 definition of sparse flows. 300 The Linux Heavy-Hitter Filter ([HHF]) qdisc and the Cisco Dynamic 301 Packet Prioritization ([DPP]) feature both categorize application 302 flows into "mice" and "elephants", and provide a separate queue that 303 gives high priority to the "mice" flows. In both of these 304 implementations, the definition of a mouse flow is one that falls 305 below a defined number of bytes or packets (respectively). In 306 essence, the first N bytes or packets of every new flow are queued 307 separately, and given priority over other traffic. The HHF 308 implementation defaults to using 128KB for N, whereas the DPP 309 documentation discusses using 120 packets. 311 This approach is relatively simple to implement, but it is making the 312 wrong distinction between flows. To illustrate, an hour-long 60 kbps 313 multiplayer online gaming flow sending 60 packets per second would be 314 classified as an elephant after the first 17 seconds using HFF or 2 315 seconds using DPP, whereas it should be considered as NQB for the 316 entire duration. 318 9. Acknowledgements 320 TBD 322 10. IANA Considerations 324 This draft proposes the registration of a standardized DSCP = 0x2A to 325 denote Non-Queue-Building behavior. 327 11. Security Considerations 329 There is no incentive for an application to mismark its packets as 330 NQB (or vice versa). If a queue-building flow were to mark its 331 packets as NQB, it could experience excessive packet loss (in the 332 case that queue-protection is not supported by a node) or it could 333 receive no benefit (in the case that queue-protection is supported). 334 If a non-queue-building flow were to fail to mark its packets as NQB, 335 it could suffer the latency and loss typical of sharing a queue with 336 capacity seeking traffic. 338 The NQB signal is not integrity protected and could be flipped by an 339 on-path attacker. This might negatively affect the QoS of the 340 tampered flow. 342 12. Informative References 344 [DOCSIS-MULPIv3.1] 345 Cable Television Laboratories, Inc., "MAC and Upper Layer 346 Protocols Interface Specification, CM-SP- 347 MULPIv3.1-I17-190121", January 21, 2019, 348 . 351 [DPP] Cisco, "Intelligent Buffer Management on Cisco Nexus 9000 352 Series Switches White Paper", June 2017, 353 . 357 [HHF] Lam, T., "net-qdisc-hhf: Heavy-Hitter Filter (HHF) qdisc", 358 December 2013, . 360 [I-D.ietf-tsvwg-l4s-arch] 361 Briscoe, B., Schepper, K., and M. Bagnulo, "Low Latency, 362 Low Loss, Scalable Throughput (L4S) Internet Service: 363 Architecture", draft-ietf-tsvwg-l4s-arch-03 (work in 364 progress), October 2018. 366 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 367 Requirement Levels", BCP 14, RFC 2119, 368 DOI 10.17487/RFC2119, March 1997, 369 . 371 [RFC8033] Pan, R., Natarajan, P., Baker, F., and G. White, 372 "Proportional Integral Controller Enhanced (PIE): A 373 Lightweight Control Scheme to Address the Bufferbloat 374 Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017, 375 . 377 [RFC8034] White, G. and R. Pan, "Active Queue Management (AQM) Based 378 on Proportional Integral Controller Enhanced PIE) for 379 Data-Over-Cable Service Interface Specifications (DOCSIS) 380 Cable Modems", RFC 8034, DOI 10.17487/RFC8034, February 381 2017, . 383 [RFC8289] Nichols, K., Jacobson, V., McGregor, A., Ed., and J. 384 Iyengar, Ed., "Controlled Delay Active Queue Management", 385 RFC 8289, DOI 10.17487/RFC8289, January 2018, 386 . 388 [RFC8290] Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys, 389 J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler 390 and Active Queue Management Algorithm", RFC 8290, 391 DOI 10.17487/RFC8290, January 2018, 392 . 394 Author's Address 396 Greg White 397 CableLabs 398 858 Coal Creek Circle 399 Louisville, CO 80027 400 US 402 Email: g.white@cablelabs.com