idnits 2.17.1 draft-ietf-aqm-recommendation-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document obsoletes RFC2309, but the abstract doesn't seem to directly say this. It does mention RFC2309 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 1, 2014) is 3587 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'Jain94' is defined on line 1095, but no explicit reference was found in the text == Unused Reference: 'Nic12' is defined on line 1115, but no explicit reference was found in the text == Unused Reference: 'Sut99' is defined on line 1192, but no explicit reference was found in the text ** Obsolete normative reference: RFC 5405 (Obsoleted by RFC 8085) -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 896 (Obsoleted by RFC 7805) -- Obsolete informational reference (is this intentional?): RFC 2309 (Obsoleted by RFC 7567) -- Obsolete informational reference (is this intentional?): RFC 2460 (Obsoleted by RFC 8200) -- Obsolete informational reference (is this intentional?): RFC 4960 (Obsoleted by RFC 9260) Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group F. Baker, Ed. 3 Internet-Draft Cisco Systems 4 Obsoletes: 2309 (if approved) G. Fairhurst, Ed. 5 Intended status: Best Current Practice University of Aberdeen 6 Expires: January 2, 2015 July 1, 2014 8 IETF Recommendations Regarding Active Queue Management 9 draft-ietf-aqm-recommendation-06 11 Abstract 13 This memo presents recommendations to the Internet community 14 concerning measures to improve and preserve Internet performance. It 15 presents a strong recommendation for testing, standardization, and 16 widespread deployment of active queue management (AQM) in network 17 devices, to improve the performance of today's Internet. It also 18 urges a concerted effort of research, measurement, and ultimate 19 deployment of AQM mechanisms to protect the Internet from flows that 20 are not sufficiently responsive to congestion notification. 22 The note largely repeats the recommendations of RFC 2309, updated 23 after fifteen years of experience and new research. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on January 2, 2015. 42 Copyright Notice 44 Copyright (c) 2014 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 1.1. Congestion Collapse . . . . . . . . . . . . . . . . . . . 3 61 1.2. Active Queue Management to Manage Latency . . . . . . . . 3 62 1.3. Document Overview . . . . . . . . . . . . . . . . . . . . 4 63 1.4. Requirements Language . . . . . . . . . . . . . . . . . . 5 64 2. The Need For Active Queue Management . . . . . . . . . . . . 5 65 2.1. AQM and Multiple Queues . . . . . . . . . . . . . . . . . 9 66 2.2. AQM and Explicit Congestion Marking (ECN) . . . . . . . . 9 67 2.3. AQM and Buffer Size . . . . . . . . . . . . . . . . . . . 9 68 3. Managing Aggressive Flows . . . . . . . . . . . . . . . . . . 10 69 4. Conclusions and Recommendations . . . . . . . . . . . . . . . 13 70 4.1. Operational deployments SHOULD use AQM procedures . . . . 14 71 4.2. Signaling to the transport endpoints . . . . . . . . . . 14 72 4.2.1. AQM and ECN . . . . . . . . . . . . . . . . . . . . . 15 73 4.3. AQM algorithms deployed SHOULD NOT require operational 74 tuning . . . . . . . . . . . . . . . . . . . . . . . . . 17 75 4.4. AQM algorithms SHOULD respond to measured congestion, not 76 application profiles. . . . . . . . . . . . . . . . . . . 18 77 4.5. AQM algorithms SHOULD NOT be dependent on specific 78 transport protocol behaviours . . . . . . . . . . . . . . 19 79 4.6. Interactions with congestion control algorithms . . . . . 19 80 4.7. The need for further research . . . . . . . . . . . . . . 20 81 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 82 6. Security Considerations . . . . . . . . . . . . . . . . . . . 21 83 7. Privacy Considerations . . . . . . . . . . . . . . . . . . . 21 84 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 85 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 86 9.1. Normative References . . . . . . . . . . . . . . . . . . 22 87 9.2. Informative References . . . . . . . . . . . . . . . . . 23 88 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 26 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 91 1. Introduction 93 The Internet protocol architecture is based on a connectionless end- 94 to-end packet service using the Internet Protocol, whether IPv4 95 [RFC0791] or IPv6 [RFC2460]. The advantages of its connectionless 96 design: flexibility and robustness, have been amply demonstrated. 98 However, these advantages are not without cost: careful design is 99 required to provide good service under heavy load. In fact, lack of 100 attention to the dynamics of packet forwarding can result in severe 101 service degradation or "Internet meltdown". This phenomenon was 102 first observed during the early growth phase of the Internet in the 103 mid 1980s [RFC0896][RFC0970], and is technically called "congestive 104 collapse" and was a key focus of RFC2309. 106 Since 1998, when RFC2309 was written, the Internet has become used 107 for a variety traffic of traffic. In the current Internet and low 108 latency is extremely important for many interactive and transaction- 109 based applications. The same type of technology that RFC2309 110 advocated for combating congestion collapse is also effective at 111 limiting delays to reduce the interaction delay experienced by 112 applications. This document replaces RFC2309, and while there is 113 still a need to avoid congestion collapse, there is now also a focus 114 on reducing network latency using the same technology. 116 1.1. Congestion Collapse 118 The original fix for Internet meltdown was provided by Van Jacobsen. 119 Beginning in 1986, Jacobsen developed the congestion avoidance 120 mechanisms [Jacobson88] that are now required for implementations of 121 the Transport Control Protocol (TCP) [RFC0768] [RFC1122]. These 122 mechanisms operate in Internet hosts to cause TCP connections to 123 "back off" during congestion. We say that TCP flows are "responsive" 124 to congestion signals (i.e., marked or dropped packets) from the 125 network. It is primarily these TCP congestion avoidance algorithms 126 that prevent the congestive collapse of today's Internet. Similar 127 algorithms are specified for other non-TCP transports. 129 However, that is not the end of the story. Considerable research has 130 been done on Internet dynamics since 1988, and the Internet has 131 grown. It has become clear that the congestion avoidance mechanisms 132 [RFC5681], while necessary and powerful, are not sufficient to 133 provide good service in all circumstances. Basically, there is a 134 limit to how much control can be accomplished from the edges of the 135 network. Some mechanisms are needed in the network devices to 136 complement the endpoint congestion avoidance mechanisms. These 137 mechanisms may be implemented in network devices that include 138 routers, switches, and other network middleboxes. 140 1.2. Active Queue Management to Manage Latency 142 Internet latency has become a focus of attention to increase the 143 responsiveness of Internet applications and protocols. One major 144 source of delay is the build-up of queues in network devices. 145 Queueing occurs whenever the arrival rate of data at the ingress to a 146 device exceeds the current egress rate. Such queueing is normal in a 147 packet-switched network and often necessary to absorb bursts in 148 transmission and perform statistical multiplexing of traffic, but 149 excessive queueing can lead to unwanted delay, reducing the 150 performance of some Internet applications. 152 Active Queue Management (AQM) is a technology that manages the size 153 of the queues that build in network buffers. Deploying AQM in the 154 network can significantly reduce the latency across an Internet path 155 and since writing RFC2309, this has become a key motivation for using 156 AQM in the Internet. 158 In the context of AQM, it is useful to distinguish between two 159 related classes of algorithms: "queue management" versus "scheduling" 160 algorithms. To a rough approximation, queue management algorithms 161 manage the length of packet queues by marking or dropping packets 162 when necessary or appropriate, while scheduling algorithms determine 163 which packet to send next and are used primarily to manage the 164 allocation of bandwidth among flows. While these two mechanisms are 165 closely related, they address different performance issues and 166 operate on different timescales. Both may be used in combination. 168 1.3. Document Overview 170 This memo highlights two performance issues: 172 The first issue is the need for an advanced form of queue management 173 that we call "Active Queue Management", AQM. Section 2 summarizes 174 the benefits that active queue management can bring. A number of AQM 175 procedures are described in the literature, with different 176 characteristics. This document does not recommend any of them in 177 particular, but does make recommendations that ideally would affect 178 the choice of procedure used in a given implementation. 180 The second issue, discussed in Section 3 of this memo, is the 181 potential for future congestive collapse of the Internet due to flows 182 that are unresponsive, or not sufficiently responsive, to congestion 183 indications. Unfortunately, while scheduling can mitigate some of 184 the side-effects of sharing a network queue with an unresponsive 185 flow, there is currently no consensus solution to controlling the 186 congestion caused by such aggressive flows. Methods such as 187 congestion exposure (ConEx) [RFC6789] offer a framework [CONEX] that 188 can update network devices to alleviate these effects. Significant 189 research and engineering will be required before any solution will be 190 available. It is imperative that work to mitigate the impact of 191 unresponsive flows is energetically pursued, to ensure the future 192 stability of the Internet. 194 Section 4 concludes the memo with a set of recommendations to the 195 Internet community concerning these topics. 197 The discussion in this memo applies to "best-effort" traffic, which 198 is to say, traffic generated by applications that accept the 199 occasional loss, duplication, or reordering of traffic in flight. It 200 also applies to other traffic, such as real-time traffic that can 201 adapt its sending rate to reduce loss and/or delay. It is most 202 effective when the adaption occurs on time scales of a single Round 203 Trip Time (RTT) or a small number of RTTs, for elastic traffic 204 [RFC1633]. 206 [RFC2309] resulted from past discussions of end-to-end performance, 207 Internet congestion, and Random Early Discard (RED) in the End-to-End 208 Research Group of the Internet Research Task Force (IRTF). This 209 update results from experience with this and other algorithms, and 210 the AQM discussion within the IETF[AQM-WG]. 212 1.4. Requirements Language 214 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 215 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 216 document are to be interpreted as described in [RFC2119]. 218 2. The Need For Active Queue Management 220 Active Queue Management (AQM) is a method that allows network devices 221 to control the queue length or the mean time that a packet spends in 222 a queue. Although AQM can be applied across a range of deployment 223 environments, the recommendations in this document are directed to 224 use in the general Internet. It is expected that the principles and 225 guidance are also applicable to a wide range of environments, but may 226 require tuning for specific types of link/network (e.g. to 227 accommodate the traffic patterns found in data centres, the 228 challenges of wireless infrastructure, or the higher delay 229 encountered on satellite Internet links). The remainder of this 230 section identifies the need for AQM and the advantages of deploying 231 the method. 233 The traditional technique for managing the queue length in a network 234 device is to set a maximum length (in terms of packets) for each 235 queue, accept packets for the queue until the maximum length is 236 reached, then reject (drop) subsequent incoming packets until the 237 queue decreases because a packet from the queue has been transmitted. 238 This technique is known as "tail drop", since the packet that arrived 239 most recently (i.e., the one on the tail of the queue) is dropped 240 when the queue is full. This method has served the Internet well for 241 years, but it has two important drawbacks: 243 1. Full Queues 245 The tail drop discipline allows queues to maintain a full (or, 246 almost full) status for long periods of time, since tail drop 247 signals congestion (via a packet drop) only when the queue has 248 become full. It is important to reduce the steady-state queue 249 size, and this is perhaps the most important goal for queue 250 management. 252 The naive assumption might be that there is a simple tradeoff 253 between delay and throughput, and that the recommendation that 254 queues be maintained in a "non-full" state essentially translates 255 to a recommendation that low end-to-end delay is more important 256 than high throughput. However, this does not take into account 257 the critical role that packet bursts play in Internet 258 performance. For example, even though TCP constrains the 259 congestion window of a flow, packets often arrive at network 260 devices in bursts [Leland94]. If the queue is full or almost 261 full, an arriving burst will cause multiple packets to be dropped 262 from the same flow. Bursts of loss can result in a global 263 synchronization of flows throttling back, followed by a sustained 264 period of lowered link utilization, reducing overall throughput 265 [Flo94], [Zha90] 267 The goal of buffering in the network is to absorb data bursts and 268 to transmit them during the (hopefully) ensuing bursts of 269 silence. This is essential to permit transmission of bursts of 270 data. Normally small queues are preferred in network devices, 271 with sufficient queue capacity to absorb the bursts. The 272 counter-intuitive result is that maintaining normally-small 273 queues can result in higher throughput as well as lower end-to- 274 end delay. In summary, queue limits should not reflect the 275 steady state queues we want to be maintained in the network; 276 instead, they should reflect the size of bursts that a network 277 device needs to absorb. 279 2. Lock-Out 281 In some situations tail drop allows a single connection or a few 282 flows to monopolize the queue space starving other connection 283 preventing them from getting room in the queue [Flo92]. 285 3. Mitigating the Impact of Packet Bursts 287 Large burst of packets can delay other packets, disrupting the 288 control loop (e.g. the pacing of flows by the TCP ACK-Clock), and 289 reducing the performance of flows that share a common bottleneck. 291 4. Control loop synchronisation 293 Congestion control, like other end-to-end mechanisms, introduces 294 a control loop between hosts. Sessions that share a common 295 network bottleneck can therefore become synchronised, introducing 296 periodic disruption (e.g. jitter/loss). "lock-out" is often also 297 the result of synchronization or other timing effects 299 Besides tail drop, two alternative queue management disciplines that 300 can be applied when a queue becomes full are "random drop on full" or 301 "head drop on full". When a new packet arrives at a full queue using 302 the random drop on full discipline, the network device drops a 303 randomly selected packet from the queue (which can be an expensive 304 operation, since it naively requires an O(N) walk through the packet 305 queue). When a new packet arrives at a full queue using the head 306 drop on full discipline, the network device drops the packet at the 307 front of the queue [Lakshman96]. Both of these solve the lock-out 308 problem, but neither solves the full-queues problem described above. 310 We know in general how to solve the full-queues problem for 311 "responsive" flows, i.e., those flows that throttle back in response 312 to congestion notification. In the current Internet, dropped packets 313 provide a critical mechanism indicating congestion notification to 314 hosts. The solution to the full-queues problem is for network 315 devices to drop packets before a queue becomes full, so that hosts 316 can respond to congestion before buffers overflow. We call such a 317 proactive approach AQM. By dropping packets before buffers overflow, 318 AQM allows network devices to control when and how many packets to 319 drop. 321 In summary, an active queue management mechanism can provide the 322 following advantages for responsive flows. 324 1. Reduce number of packets dropped in network devices 326 Packet bursts are an unavoidable aspect of packet networks 327 [Willinger95]. If all the queue space in a network device is 328 already committed to "steady state" traffic or if the buffer 329 space is inadequate, then the network device will have no ability 330 to buffer bursts. By keeping the average queue size small, AQM 331 will provide greater capacity to absorb naturally-occurring 332 bursts without dropping packets. 334 Furthermore, without AQM, more packets will be dropped when a 335 queue does overflow. This is undesirable for several reasons. 336 First, with a shared queue and the tail drop discipline, this can 337 result in unnecessary global synchronization of flows, resulting 338 in lowered average link utilization, and hence lowered network 339 throughput. Second, unnecessary packet drops represent a waste 340 of network capacity on the path before the drop point. 342 While AQM can manage queue lengths and reduce end-to-end latency 343 even in the absence of end-to-end congestion control, it will be 344 able to reduce packet drops only in an environment that continues 345 to be dominated by end-to-end congestion control. 347 2. Provide a lower-delay interactive service 349 By keeping a small average queue size, AQM will reduce the delays 350 experienced by flows. This is particularly important for 351 interactive applications such as short web transfers, POP/IMAP, 352 DNS, terminal traffic (telnet, ssh, mosh, RDP, etc), gaming or 353 interactive audio-video sessions, whose subjective (and 354 objective) performance is better when the end-to-end delay is 355 low. 357 3. Avoid lock-out behavior 359 AQM can prevent lock-out behavior by ensuring that there will 360 almost always be a buffer available for an incoming packet. For 361 the same reason, AQM can prevent a bias against low capacity, but 362 highly bursty, flows. 364 Lock-out is undesirable because it constitutes a gross unfairness 365 among groups of flows. However, we stop short of calling this 366 benefit "increased fairness", because general fairness among 367 flows requires per-flow state, which is not provided by queue 368 management. For example, in a network device using AQM with only 369 FIFO scheduling, two TCP flows may receive very different share 370 of the network capacity simply because they have different round- 371 trip times [Floyd91], and a flow that does not use congestion 372 control may receive more capacity than a flow that does. AQM can 373 therefore be combined with a scheduling mechanism that divides 374 network traffic between multiple queues (section 2.1). 376 4. Reduce the probability of control loop synchronisation 378 The probability of network control loop synchronisation can be 379 reduced by introducing randomness in the AQM functions used by 380 network devices that trigger congestion avoidance at the sending 381 host. 383 2.1. AQM and Multiple Queues 385 A network device may use per-flow or per-class queuing with a 386 scheduling algorithm to either prioritise certain applications or 387 classes of traffic, limit the rate of transmission, or to provide 388 isolation between different traffic flows within a common class. For 389 example, a router may maintain per-flow state to achieve general 390 fairness by a per-flow scheduling algorithm such as various forms of 391 Fair Queueing (FQ) [Dem90], including Weighted Fair Queuing (WFQ), 392 Stochastic Fairness Queueing (SFQ) [McK90] Deficit Round Robin (DRR) 393 [Shr96], and/or a Class-Based Queue scheduling algorithm such as CBQ 394 [Floyd95]. Hierarchical queues may also be used e.g., as a part of a 395 Hierarchical Token Bucket (HTB), or Hierarchical Fair Service Curve 396 (HFSC) [Sto97] . These methods are also used to realise a range of 397 Quality of Service (QoS) behaviours designed to meet the need of 398 traffic classes (e.g. using the integrated or differentiated service 399 models). 401 AQM is needed even for network devices that use per-flow or per-class 402 queuing, because scheduling algorithms by themselves do not control 403 the overall queue size or the size of individual queues. AQM 404 mechanisms need to control the overall queue sizes, to ensure that 405 arriving bursts can be accommodated without dropping packets. AQM 406 should also be used to control the queue size for each individual 407 flow or class, so that they do not experience unnecessarily high 408 delay. Using a combination of AQM and scheduling between multiple 409 queues has been shown to offer good results in experimental and some 410 types of operational use. 412 In short, scheduling algorithms and queue management should be seen 413 as complementary, not as replacements for each other. 415 2.2. AQM and Explicit Congestion Marking (ECN) 417 An AQM method may use Explicit Congestion Notification (ECN) 418 [RFC3168] instead of dropping to mark packets under mild or moderate 419 congestion. ECN-marking can allow a network device to signal 420 congestion at a point before a transport experiences congestion loss 421 or additional queuing delay [ECN-Benefit]. Section 4.2.1 describes 422 some of the benefits of using ECN with AQM. 424 2.3. AQM and Buffer Size 426 It is important to differentiate the choice of buffer size for a 427 queue in a switch/router or other network device, and the 428 threshold(s) and other parameters that determine how and when an AQM 429 algorithm operates. One the one hand, the optimum buffer size is a 430 function of operational requirements and should generally be sized to 431 be sufficient to buffer the largest normal traffic burst that is 432 expected. This size depends on the number and burstiness of traffic 433 arriving at the queue and the rate at which traffic leaves the queue. 435 The simplest mechanism starts with a new or building session 436 attacking a queue that is full. One or more sessions, following 437 algorithms similar to those of [RFC5681], maximizes its effective 438 window, maximizing its impact on a queue somewhere in the network and 439 the effect of that queue on both its own latency and that of 440 competing sessions. It also maximizes the probability of loss from 441 that queue. A new session, sending its initial burst, has an 442 enhanced probability of filling the remaining queue and dropping 443 packets. As a result, the new session can be effectively prevented 444 from sharing the queue effectively for a period of many RTTs. One 445 objective of AQM is to minimize the effect of lock-out by minimizing 446 mean queue depth and therefore the probability that competing 447 sessions can materially prevent each other from performing well. 448 Different types of traffic and deployment scenarios will lead to 449 different requirements. 451 AQM frees a designer from having to the limit buffer space to achieve 452 acceptable performance, allowing allocation of sufficient buffering 453 to satisfy the needs of the particular traffic pattern. On the other 454 hand, the choice of AQM algorithm and associated parameters is a 455 function of the way in which congestion is experienced and the 456 required reaction to achieve acceptable performance. This latter 457 topic is the primary topic of the following sections. 459 3. Managing Aggressive Flows 461 One of the keys to the success of the Internet has been the 462 congestion avoidance mechanisms of TCP. Because TCP "backs off" 463 during congestion, a large number of TCP connections can share a 464 single, congested link in such a way that link bandwidth is shared 465 reasonably equitably among similarly situated flows. The equitable 466 sharing of bandwidth among flows depends on all flows running 467 compatible congestion avoidance algorithms, i.e., methods conformant 468 with the current TCP specification [RFC5681]. 470 In this document a flow is known as "TCP-friendly" when it has a 471 congestion response that approximates the average response expected 472 of a TCP flow. One example method of a TCP-friendly scheme is the 473 TCP-Friendly Rate Control algorithm [RFC5348]. In this document, the 474 term is used more generally to describe this and other algorithms 475 that meet these goals. 477 It is convenient to divide flows into three classes: (1) TCP Friendly 478 flows, (2) unresponsive flows, i.e., flows that do not slow down when 479 congestion occurs, and (3) flows that are responsive but are not TCP- 480 friendly. The last two classes contain more aggressive flows that 481 pose significant threats to Internet performance, which we will now 482 discuss. 484 1. TCP-Friendly flows 486 A TCP-friendly flow responds to congestion notification within a 487 small number of path Round Trip Times (RTT), and in steady-state 488 it uses no more capacity than a conformant TCP running under 489 comparable conditions (drop rate, RTT, packet size, etc.). This 490 is described in the remainder of the document. 492 2. Non-Responsive Flows 494 The User Datagram Protocol (UDP) [RFC0768] provides a minimal, 495 best-effort transport to applications and upper-layer protocols 496 (both simply called "applications" in the remainder of this 497 document) and does not itself provide mechanisms to prevent 498 congestion collapse and establish a degree of fairness [RFC5405]. 500 There is a growing set of UDP-based applications whose congestion 501 avoidance algorithms are inadequate or nonexistent (i.e, a flow 502 that does not throttle its sending rate when it experiences 503 congestion). Examples include some UDP streaming applications 504 for packet voice and video, and some multicast bulk data 505 transport. If no action is taken, such unresponsive flows could 506 lead to a new congestive collapse [RFC2309]. 507 In general, UDP-based applications need to incorporate effective 508 congestion avoidance mechanisms [RFC5405]. Further research and 509 development of ways to accomplish congestion avoidance for 510 presently unresponsive applications continue to be important. 511 Network devices need to be able to protect themselves against 512 unresponsive flows, and mechanisms to accomplish this must be 513 developed and deployed. Deployment of such mechanisms would 514 provide an incentive for all applications to become responsive by 515 either using a congestion-controlled transport (e.g. TCP, SCTP 516 [RFC4960] and DCCP [RFC4340].) or by incorporating their own 517 congestion control in the application [RFC5405]. 518 Lastly, some applications (e.g. current web browsers) open a 519 large numbers of short TCP flows for a single session. This can 520 lead to each individual flow spending the majority of time in the 521 exponential TCP slow start phase, rather than in TCP congestion 522 avoidance. The resulting traffic aggregate can therefore be much 523 less responsive than a single standard TCP flow. 525 3. Non-TCP-friendly Transport Protocols 526 A second threat is posed by transport protocol implementations 527 that are responsive to congestion, but, either deliberately or 528 through faulty implementation, are not TCP-friendly. Such 529 applications may gain an unfair share of the available network 530 capacity. 532 For example, the popularity of the Internet has caused a 533 proliferation in the number of TCP implementations. Some of 534 these may fail to implement the TCP congestion avoidance 535 mechanisms correctly because of poor implementation. Others may 536 deliberately be implemented with congestion avoidance algorithms 537 that are more aggressive in their use of capacity than other TCP 538 implementations; this would allow a vendor to claim to have a 539 "faster TCP". The logical consequence of such implementations 540 would be a spiral of increasingly aggressive TCP implementations, 541 leading back to the point where there is effectively no 542 congestion avoidance and the Internet is chronically congested. 544 Another example could be an RTP/UDP video flow that uses an 545 adaptive codec, but responds incompletely to indications of 546 congestion or responds over an excessively long time period. 547 Such flows are unlikely to be responsive to congestion signals in 548 a timeframe comparable to a small number of end-to-end 549 transmission delays. However, over a longer timescale, perhaps 550 seconds in duration, they could moderate their speed, or increase 551 their speed if they determine capacity to be available. 553 Tunneled traffic aggregates carrying multiple (short) TCP flows 554 can be more aggressive than standard bulk TCP. Applications 555 (e.g. web browsers and peer-to-peer file-sharing) have exploited 556 this by opening multiple connections to the same endpoint. 558 The projected increase in the fraction of total Internet traffic for 559 more aggressive flows in classes 2 and 3 clearly poses a threat to 560 future Internet stability. There is an urgent need for measurements 561 of current conditions and for further research into the ways of 562 managing such flows. This raises many difficult issues in 563 identifying and isolating unresponsive or non-TCP-friendly flows at 564 an acceptable overhead cost. Finally, there is as yet little 565 measurement or simulation evidence available about the rate at which 566 these threats are likely to be realized, or about the expected 567 benefit of algorithms for managing such flows. 569 Another topic requiring consideration is the appropriate 570 granugranularity of a "flow" when considering a queue management 571 method. There are a few "natural" answers: 1) a transport (e.g. TCP 572 or UDP) flow (source address/port, destination address/port, 573 protocol); 2) Differentiated Services Code Point, DSCP; 3) a source/ 574 destination host pair (IP address); 4) a given source host or a given 575 destination host, or various combinations of the above. 577 The source/destination host pair gives an appropriate granularity in 578 many circumstances, However, different vendors/providers use 579 different granularities for defining a flow (as a way of 580 "distinguishing" themselves from one another), and different 581 granularities may be chosen for different places in the network. It 582 may be the case that the granularity is less important than the fact 583 that a network device needs to be able to deal with more unresponsive 584 flows at *some* granularity. The granularity of flows for congestion 585 management is, at least in part, a question of policy that needs to 586 be addressed in the wider IETF community. 588 4. Conclusions and Recommendations 590 The IRTF, in publishing [RFC2309], and the IETF in subsequent 591 discussion, has developed a set of specific recommendations regarding 592 the implementation and operational use of AQM procedures. The 593 updated recommendations provided by this document are summarised as: 595 1. Network devices SHOULD implement some AQM mechanism to manage 596 queue lengths, reduce end-to-end latency, and avoid lock-out 597 phenomena within the Internet. 599 2. Deployed AQM algorithms SHOULD support Explicit Congestion 600 Notification (ECN) as well as loss to signal congestion to 601 endpoints. 603 3. The algorithms that the IETF recommends SHOULD NOT require 604 operational (especially manual) configuration or tuning. 606 4. AQM algorithms SHOULD respond to measured congestion, not 607 application profiles. 609 5. AQM algorithms SHOULD NOT interpret specific transport protocol 610 behaviours. 612 6. Transport protocol congestion control algorithms SHOULD maximize 613 their use of available capacity (when there is data to send) 614 without incurring undue loss or undue round trip delay. 616 7. Research, engineering, and measurement efforts are needed 617 regarding the design of mechanisms to deal with flows that are 618 unresponsive to congestion notification or are responsive, but 619 are more aggressive than present TCP. 621 These recommendations are expressed using the word "SHOULD". This is 622 in recognition that there may be use cases that have not been 623 envisaged in this document in which the recommendation does not 624 apply. Therefore, care should be taken in concluding that one's use 625 case falls in that category; during the life of the Internet, such 626 use cases have been rarely if ever observed and reported. To the 627 contrary, available research [Choi04] says that even high speed links 628 in network cores that are normally very stable in depth and behavior 629 experience occasional issues that need moderation. The 630 recommendations are detailed in the following sections. 632 4.1. Operational deployments SHOULD use AQM procedures 634 AQM procedures are designed to minimize the delay and buffer 635 exhaustion induced in the network by queues that have filled as a 636 result of host behavior. Marking and loss behaviors provide a signal 637 that buffers within network devices are becoming unnecessarily full, 638 and that the sender would do well to moderate its behavior. 640 The use of scheduling mechanisms, such as priority queuing, classful 641 queuing, and fair queuing, is often effective in networks to help a 642 network serve the needs of a range of applications. Network 643 operators can use these methods to manage traffic passing a choke 644 point. This is discussed in [RFC2474] and [RFC2475]. When 645 scheduling is used AQM should be applied across the classes or flows 646 as well as within each class or flow: 648 o AQM mechanisms need to control the overall queue sizes, to ensure 649 that arriving bursts can be accommodated without dropping packets. 651 o AQM mechanisms need to allow combination with other mechanisms, 652 such as scheduling, to allow implementation of policies for 653 providing fairness between different flows. 655 o AQM should be used to control the queue size for each individual 656 flow or class, so that they do not experience unnecessarily high 657 delay. 659 4.2. Signaling to the transport endpoints 661 There are a number of ways a network device may signal to the end 662 point that the network is becoming congested and trigger a reduction 663 in rate. The signalling methods include: 665 o Delaying transport segments (packets) in flight, such as in a 666 queue. 668 o Dropping transport segments (packets) in transit. 670 o Marking transport segments (packets), such as using Explicit 671 Congestion Control[RFC3168] [RFC4301] [RFC4774] [RFC6040] 672 [RFC6679]. 674 Increased network latency is used as an implicit signal of 675 congestion. E.g., in TCP additional delay can affect ACK Clocking 676 and has the result of reducing the rate of transmission of new data. 677 In the Real Time Protocol (RTP), network latency impacts the RTCP- 678 reported RTT and increased latency can trigger a sender to adjust its 679 rate. Methods such as Low Extra Delay Background Transport (LEDBAT) 680 [RFC6817] assume increased latency as a primary signal of congestion. 681 Appropriate use of delay-based methods and the implications of AQM 682 presently remains an area for further research. 684 It is essential that all Internet hosts respond to loss [RFC5681], 685 [RFC5405][RFC4960][RFC4340]. Packet dropping by network devices that 686 are under load has two effects: It protects the network, which is the 687 primary reason that network devices drop packets. The detection of 688 loss also provides a signal to a reliable transport (e.g. TCP, SCTP) 689 that there is potential congestion using a pragmatic heuristic; "when 690 the network discards a message in flight, it may imply the presence 691 of faulty equipment or media in a path, and it may imply the presence 692 of congestion. To be conservative, a transport must assume it may be 693 the latter." Unreliable transports (e.g. using UDP) need to 694 similarly react to loss [RFC5405] 696 Network devices SHOULD use an AQM algorithm to determine the packets 697 that are marked or discarded due to congestion. Procedures for 698 dropping or marking packets within the network need to avoid 699 increasing synchronisation events, and hence randomness SHOULD be 700 introduced in the algorithms that generate these congestion signals 701 to the endpoints. 703 Loss also has an effect on the efficiency of a flow and can 704 significantly impact some classes of application. In reliable 705 transports the dropped data must be subsequently retransmitted. 706 While other applications/transports may adapt to the absence of lost 707 data, this still implies inefficient use of available capacity and 708 the dropped traffic can affect other flows. Hence, congestion 709 signalling by loss is not entirely positive; it is a necessary evil. 711 4.2.1. AQM and ECN 713 Explicit Congestion Notification (ECN) [RFC4301] [RFC4774] [RFC6040] 714 [RFC6679] is a network-layer function that allows a transport to 715 receive network congestion information from a network device without 716 incurring the unintended consequences of loss. ECN includes both 717 transport mechanisms and functions implemented in network devices, 718 the latter rely upon using AQM to decider when and whether to ECN- 719 mark. 721 Congestion for ECN-capable transports is signalled by a network 722 device setting the "Congestion Experienced (CE)" codepoint in the IP 723 header. This codepoint is noted by the remote receiving end point 724 and signalled back to the sender using a transport protocol 725 mechanism, allowing the sender to trigger timely congestion control. 726 The decision to set the CE codepoint requires an AQM algorithm 727 configured with a threshold. Non-ECN capable flows (the default) are 728 dropped under congestion. 730 Network devices SHOULD use an AQM algorithm that marks ECN-capable 731 traffic when making decisions about the response to congestion. 732 Network devices need to implement this method by marking ECN-capable 733 traffic or by dropping non-ECN-capable traffic. 735 Safe deployment of ECN requires that network devices drop excessive 736 traffic, even when marked as originating from an ECN-capable 737 transport. This is a necessary safety precaution because: 739 1. A non-conformant, broken or malicious receiver could conceal an 740 ECN mark, and not report this to the sender; 742 2. A non-conformant, broken or malicious sender could ignore a 743 reported ECN mark, as it could ignore a loss without using ECN; 745 3. A malfunctioning or non-conforming network device may "hide" an 746 ECN mark (or fail to correctly set the ECN codepoint at an egress 747 of a network tunnel). 749 In normal operation, such cases should be very uncommon, however 750 overload protection is desirable to protect traffic from 751 misconfigured or malicious use of ECN (e.g. a denial-of-service 752 attack that generates ECN-capable traffic that is unresponsive to CE- 753 marking). 755 An AQM algorithm that supports ECN needs to define the threshold and 756 algorithm for ECN-marking. This threshold MAY differ from that used 757 for dropping packets that are not marked as ECN-capable, and SHOULD 758 be configurable. 760 Network devices SHOULD use an algorithm to drop excessive traffic 761 (e.g. at some level above the threshold for CE-marking), even when 762 the packets are marked as originating from an ECN-capable transport. 764 4.3. AQM algorithms deployed SHOULD NOT require operational tuning 766 A number of AQM algorithms have been proposed. Many require some 767 form of tuning or setting of parameters for initial network 768 conditions. This can make these algorithms difficult to use in 769 operational networks. 771 AQM algorithms need to consider both "initial conditions" and 772 "operational conditions". The former includes values that exist 773 before any experience is gathered about the use of the algorithm, 774 such as the configured speed of interface, support for full duplex 775 communication, interface MTU and other properties of the link. The 776 latter includes information observed from monitoring the size of the 777 queue, experienced queueing delay, rate of packet discard, etc. 779 This document therefore specifies that AQM algorithms that are 780 proposed for deployment in the Internet have the following 781 properties: 783 o SHOULD NOT require tuning of initial or configuration parameters. 784 An algorithm needs to provide a default behaviour that auto-tunes 785 to a reasonable performance for typical network operational 786 conditions. This is expected to ease deployment and operation. 787 Initial conditions, such as the interface rate and MTU size or 788 other values derived from these, MAY be required by an AQM 789 algorithm. 791 o MAY support further manual tuning that could improve performance 792 in a specific deployed network. Algorithms that lack such 793 variables are acceptable, but if such variables exist, they SHOULD 794 be externalized (made visible to the operator). Guidance needs to 795 be provided on the cases where auto-tuning is unlikely to achieve 796 satisfactory performance and to identify the set of parameters 797 that can be tuned. For example, the expected response of an 798 algorithm may need to be configured to accommodate the largest 799 expected Path RTT, since this value can not be known at 800 initialisation. This guidance is expected to enable the algorithm 801 to be deployed in networks that have specific characteristics 802 (paths with variable/larger delay; networks where capacity is 803 impacted by interactions with lower layer mechanisms, etc). 805 o MAY provide logging and alarm signals to assist in identifying if 806 an algorithm using manual or auto-tuning is functioning as 807 expected. (e.g., this could be based on an internal consistency 808 check between input, output, and mark/drop rates over time). This 809 is expected to encourage deployment by default and allow operators 810 to identify potential interactions with other network functions. 812 Hence, self-tuning algorithms are to be preferred. Algorithms 813 recommended for general Internet deployment by the IETF need to be 814 designed so that they do not require operational (especially manual) 815 configuration or tuning. 817 4.4. AQM algorithms SHOULD respond to measured congestion, not 818 application profiles. 820 Not all applications transmit packets of the same size. Although 821 applications may be characterized by particular profiles of packet 822 size this should not be used as the basis for AQM (see next section). 823 Other methods exist, e.g. Differentiated Services queueing, Pre- 824 Congestion Notification (PCN) [RFC5559], that can be used to 825 differentiate and police classes of application. Network devices may 826 combine AQM with these traffic classification mechanisms and perform 827 AQM only on specific queues within a network device. 829 An AQM algorithm should not deliberately try to prejudice the size of 830 packet that performs best (i.e. Preferentially drop/mark based only 831 on packet size). Procedures for selecting packets to mark/drop 832 SHOULD observe the actual or projected time that a packet is in a 833 queue (bytes at a rate being an analog to time). When an AQM 834 algorithm decides whether to drop (or mark) a packet, it is 835 RECOMMENDED that the size of the particular packet should not be 836 taken into account [Byte-pkt]. 838 Applications (or transports) generally know the packet size that they 839 are using and can hence make their judgments about whether to use 840 small or large packets based on the data they wish to send and the 841 expected impact on the delay or throughput, or other performance 842 parameter. When a transport or application responds to a dropped or 843 marked packet, the size of the rate reduction should be proportionate 844 to the size of the packet that was sent [Byte-pkt]. 846 AQM-enabled system MAY instantiate different instances of an AQM 847 algorithm to be applied within the same traffic class. Traffic 848 classes may be differentiated based on an Access Control List (ACL), 849 the packet Differentiated Services Code Point (DSCP) [RFC5559], 850 enabling use of the ECN field (i.e. any of ECT(0), ECT(1) or 851 CE)[RFC3168] [RFC4774], a multi-field (MF) classifier that combines 852 the values of a set of protocol fields (e.g. IP address, transport, 853 ports) or an equivalent codepoint at a lower layer. This 854 recommendation goes beyond what is defined in RFC 3168, by allowing 855 that an implementation MAY use more than one instance of an AQM 856 algorithm to handle both ECN-capable and non-ECN-capable packets. 858 4.5. AQM algorithms SHOULD NOT be dependent on specific transport 859 protocol behaviours 861 In deploying AQM, network devices need to support a range of Internet 862 traffic and SHOULD NOT make implicit assumptions about the 863 characteristics desired by the set transports/applications the 864 network supports. That is, AQM methods should be opaque to the 865 choice of transport and application. 867 AQM algorithms are often evaluated by considering TCP [RFC0793] with 868 a limited number of applications. Although TCP is the predominant 869 transport in the Internet today, this no longer represents a 870 sufficient selection of traffic for verification. There is 871 significant use of UDP [RFC0768] in voice and video services, and 872 some applications find utility in SCTP [RFC4960] and DCCP [RFC4340]. 873 Hence, AQM algorithms should also demonstrate operation with 874 transports other than TCP and need to consider a variety of 875 applications. Selection of AQM algorithms also needs to consider use 876 of tunnel encapsulations that may carry traffic aggregates. 878 AQM algorithms SHOULD NOT target or derive implicit assumptions about 879 the characteristics desired by specific transports/applications. 880 Transports and applications need to respond to the congestion signals 881 provided by AQM (i.e. dropping or ECN-marking) in a timely manner 882 (within a few RTT at the latest). 884 4.6. Interactions with congestion control algorithms 886 Applications and transports need to react to received implicit or 887 explicit signals that indicate the presence of congestion. This 888 section identifies issues that can impact the design of transport 889 protocols when using paths that use AQM. 891 Transport protocols and applications need timely signals of 892 congestion. The time taken to detect and respond to congestion is 893 increased when network devices queue packets in buffers. It can be 894 difficult to detect tail losses at a higher layer and this may 895 sometimes require transport timers or probe packets to detect and 896 respond to such loss. Loss patterns may also impact timely 897 detection, e.g. the time may be reduced when network devices do not 898 drop long runs of packets from the same flow. 900 A common objective of an elastic transport congestion control 901 protocol is to allow an application to deliver the maximum rate of 902 data without inducing excessive delays when packets are queued in a 903 buffers within the network. To achieve this, a transport should try 904 to operate at rate below the inflexion point of the load/delay curve 905 (the bend of what is sometimes called a "hockey-stick" curve). When 906 the congestion window allows the load to approach this bend, the end- 907 to-end delay starts to rise - a result of congestion, as packets 908 probabilistically arrive at non-overlapping times. On the one hand, 909 a transport that operates above this point can experience congestion 910 loss and could also trigger operator activities, such as those 911 discussed in [RFC6057]. On the other hand, a flow may achieve both 912 near-maximum throughput and low latency when it operates close to 913 this knee point, with minimal contribution to router congestion. 914 Choice of an appropriate rate/congestion window can therefore 915 significantly impact the loss and delay experienced by a flow and 916 will impact other flows that share a common network queue. 918 Some applications may send less than permitted by the congestion 919 control window (or rate). Examples include multimedia codecs that 920 stream at some natural rate (or set of rates) or an application that 921 is naturally interactive (e.g., some web applications, gaming, 922 transaction-based protocols). Such applications may have different 923 objectives. They may not wish to maximize throughput, but may desire 924 a lower loss rate or bounded delay. 926 The correct operation of an AQM-enabled network device MUST NOT rely 927 upon specific transport responses to congestion signals. 929 4.7. The need for further research 931 The second recommendation of [RFC2309] called for further research 932 into the interaction between network queues and host applications, 933 and the means of signaling between them. This research has occurred, 934 and we as a community have learned a lot. However, we are not done. 936 We have learned that the problems of congestion, latency and buffer- 937 sizing have not gone away, and are becoming more important to many 938 users. A number of self-tuning AQM algorithms have been found that 939 offer significant advantages for deployed networks. There is also 940 renewed interest in deploying AQM and the potential of ECN. 942 In 2013, an obvious example of further research is the need to 943 consider the use of Map/Reduce applications in data centers; do we 944 need to extend our taxonomy of TCP/SCTP sessions to include not only 945 "mice" and "elephants", but "lemmings"? "Lemmings" are flash crowds 946 of "mice" that the network inadvertently try to signal to as if they 947 were elephant flows, resulting in head of line blocking in data 948 center applications. 950 Examples of other required research include: 952 o Research into new AQM and scheduling algorithms. 954 o Appropriate use of delay-based methods and the implications of 955 AQM. 957 o Research into the use of and deployment of ECN alongside AQM. 959 o Tools for enabling AQM (and ECN) deployment and measuring the 960 performance. 962 o Methods for mitigating the impact of non-conformant and malicious 963 flows. 965 o Research to understand the implications of using new network and 966 transport methods on applications. 968 Hence, this document therefore reiterates the call of RFC 2309: we 969 need continuing research as applications develop. 971 5. IANA Considerations 973 This memo asks the IANA for no new parameters. 975 6. Security Considerations 977 While security is a very important issue, it is largely orthogonal to 978 the performance issues discussed in this memo. 980 Many deployed network devices use queueing methods that allow 981 unresponsive traffic to capture network capacity, denying access to 982 other traffic flows. This could potentially be used as a denial-of- 983 service attack. This threat could be reduced in network devices 984 deploy AQM or some form of scheduling. We note, however, that a 985 denial-of-service attack that results in unresponsive traffic flows 986 may be indistinguishable from other traffic flows (e.g. tunnels 987 carrying aggregates of short flows, high-rate isochronous 988 applications). New methods therefore may remain vulnerable, and this 989 document recommends that ongoing research should consider ways to 990 mitigate such attacks. 992 7. Privacy Considerations 994 This document, by itself, presents no new privacy issues. 996 8. Acknowledgements 998 The original recommendation in [RFC2309] was written by the End-to- 999 End Research Group, which is to say Bob Braden, Dave Clark, Jon 1000 Crowcroft, Bruce Davie, Steve Deering, Deborah Estrin, Sally Floyd, 1001 Van Jacobson, Greg Minshall, Craig Partridge, Larry Peterson, KK 1002 Ramakrishnan, Scott Shenker, John Wroclawski, and Lixia Zhang. This 1003 is an edited version of that document, with much of its text and 1004 arguments unchanged. 1006 The need for an updated document was agreed to in the tsvarea meeting 1007 at IETF 86. This document was reviewed on the aqm@ietf.org list. 1008 Comments were received from Colin Perkins, Richard Scheffenegger, 1009 Dave Taht, John Leslie, David Collier-Brown and many others. 1011 Gorry Fairhurst was in part supported by the European Community under 1012 its Seventh Framework Programme through the Reducing Internet 1013 Transport Latency (RITE) project (ICT-317700). 1015 9. References 1017 9.1. Normative References 1019 [Byte-pkt] 1020 and Internet Engineering Task Force, Work in Progress, 1021 "Byte and Packet Congestion Notification (draft-ietf- 1022 tsvwg-byte-pkt-congest)", July 2013. 1024 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1025 Requirement Levels", BCP 14, RFC 2119, March 1997. 1027 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1028 of Explicit Congestion Notification (ECN) to IP", RFC 1029 3168, September 2001. 1031 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1032 Internet Protocol", RFC 4301, December 2005. 1034 [RFC4774] Floyd, S., "Specifying Alternate Semantics for the 1035 Explicit Congestion Notification (ECN) Field", BCP 124, 1036 RFC 4774, November 2006. 1038 [RFC5405] Eggert, L. and G. Fairhurst, "Unicast UDP Usage Guidelines 1039 for Application Designers", BCP 145, RFC 5405, November 1040 2008. 1042 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1043 Control", RFC 5681, September 2009. 1045 [RFC6040] Briscoe, B., "Tunnelling of Explicit Congestion 1046 Notification", RFC 6040, November 2010. 1048 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 1049 and K. Carlberg, "Explicit Congestion Notification (ECN) 1050 for RTP over UDP", RFC 6679, August 2012. 1052 9.2. Informative References 1054 [AQM-WG] "IETF AQM WG", . 1056 [CONEX] Mathis, M. and B. Briscoe, "The Benefits to Applications 1057 of using Explicit Congestion Notification (ECN)", IETF 1058 (Work-in-Progress) draft-ietf-conex-abstract-mech, March 1059 2014. 1061 [Choi04] Choi, Baek-Young., Moon, Sue., Zhang, Zhi-Li., 1062 Papagiannaki, K., and C. Diot, "Analysis of Point-To-Point 1063 Packet Delay In an Operational Network", March 2004. 1065 [Dem90] Demers, A., Keshav, S., and S. Shenker, "Analysis and 1066 Simulation of a Fair Queueing Algorithm, Internetworking: 1067 Research and Experience", SIGCOMM Symposium proceedings on 1068 Communications architectures and protocols , 1990. 1070 [ECN-Benefit] 1071 Welzl, M. and G. Fairhurst, "The Benefits to Applications 1072 of using Explicit Congestion Notification (ECN)", IETF 1073 (Work-in-Progress) , February 2014. 1075 [Flo92] Floyd, S. and V. Jacobsen, "On Traffic Phase Effects in 1076 Packet-Switched Gateways", 1992. 1078 [Flo94] Floyd, S. and V. Jacobsen, "The Synchronization of 1079 Periodic Routing Messages, 1080 http://ee.lbl.gov/papers/sync_94.pdf", 1994. 1082 [Floyd91] Floyd, S., "Connections with Multiple Congested Gateways 1083 in Packet-Switched Networks Part 1: One-way Traffic.", 1084 Computer Communications Review , October 1991. 1086 [Floyd95] Floyd, S. and V. Jacobson, "Link-sharing and Resource 1087 Management Models for Packet Networks", IEEE/ACM 1088 Transactions on Networking , August 1995. 1090 [Jacobson88] 1091 Jacobson, V., "Congestion Avoidance and Control", SIGCOMM 1092 Symposium proceedings on Communications architectures and 1093 protocols , August 1988. 1095 [Jain94] Jain, Raj., Ramakrishnan, KK., and Chiu. Dah-Ming, 1096 "Congestion avoidance scheme for computer networks", US 1097 Patent Office 5377327, December 1994. 1099 [Lakshman96] 1100 Lakshman, TV., Neidhardt, A., and T. Ott, "The Drop From 1101 Front Strategy in TCP Over ATM and Its Interworking with 1102 Other Control Features", IEEE Infocomm , 1996. 1104 [Leland94] 1105 Leland, W., Taqqu, M., Willinger, W., and D. Wilson, "On 1106 the Self-Similar Nature of Ethernet Traffic (Extended 1107 Version)", IEEE/ACM Transactions on Networking , February 1108 1994. 1110 [McK90] McKenney, PE. and G. Varghese, "Stochastic Fairness 1111 Queuing", 1112 http://www2.rdrop.com/~paulmck/scalability/paper/ 1113 sfq.2002.06.04.pdf , 1990. 1115 [Nic12] Nichols, K., "Controlling Queue Delay", Communications of 1116 the ACM Vol. 55 No. 11, July, 2012, pp.42-50. , July 2002. 1118 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 1119 August 1980. 1121 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1122 1981. 1124 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1125 793, September 1981. 1127 [RFC0896] Nagle, J., "Congestion control in IP/TCP internetworks", 1128 RFC 896, January 1984. 1130 [RFC0970] Nagle, J., "On packet switches with infinite storage", RFC 1131 970, December 1985. 1133 [RFC1122] Braden, R., "Requirements for Internet Hosts - 1134 Communication Layers", STD 3, RFC 1122, October 1989. 1136 [RFC1633] Braden, B., Clark, D., and S. Shenker, "Integrated 1137 Services in the Internet Architecture: an Overview", RFC 1138 1633, June 1994. 1140 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 1141 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 1142 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 1143 S., Wroclawski, J., and L. Zhang, "Recommendations on 1144 Queue Management and Congestion Avoidance in the 1145 Internet", RFC 2309, April 1998. 1147 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1148 (IPv6) Specification", RFC 2460, December 1998. 1150 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1151 "Definition of the Differentiated Services Field (DS 1152 Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1153 1998. 1155 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1156 and W. Weiss, "An Architecture for Differentiated 1157 Services", RFC 2475, December 1998. 1159 [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram 1160 Congestion Control Protocol (DCCP)", RFC 4340, March 2006. 1162 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 1163 4960, September 2007. 1165 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1166 Friendly Rate Control (TFRC): Protocol Specification", RFC 1167 5348, September 2008. 1169 [RFC5559] Eardley, P., "Pre-Congestion Notification (PCN) 1170 Architecture", RFC 5559, June 2009. 1172 [RFC6057] Bastian, C., Klieber, T., Livingood, J., Mills, J., and R. 1173 Woundy, "Comcast's Protocol-Agnostic Congestion Management 1174 System", RFC 6057, December 2010. 1176 [RFC6789] Briscoe, B., Woundy, R., and A. Cooper, "Congestion 1177 Exposure (ConEx) Concepts and Use Cases", RFC 6789, 1178 December 2012. 1180 [RFC6817] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, 1181 "Low Extra Delay Background Transport (LEDBAT)", RFC 6817, 1182 December 2012. 1184 [Shr96] Shreedhar, M. and G. Varghese, "Efficient Fair Queueing 1185 Using Deficit Round Robin", IEEE/ACM Transactions on 1186 Networking Vol 4, No. 3 , July 1996. 1188 [Sto97] Stoica, I. and H. Zhang, "A Hierarchical Fair Service 1189 Curve algorithm for Link sharing, real-time and priority 1190 services", ACM SIGCOMM , 1997. 1192 [Sut99] Suter, B., "Buffer Management Schemes for Supporting TCP 1193 in Gigabit Routers with Per-flow Queueing", IEEE Journal 1194 on Selected Areas in Communications Vol. 17 Issue 6, June, 1195 1999, pp. 1159-1169. , 1999. 1197 [Willinger95] 1198 Willinger, W., Taqqu, M., Sherman, R., Wilson, D., and V. 1199 Jacobson, "Self-Similarity Through High-Variability: 1200 Statistical Analysis of Ethernet LAN Traffic at the Source 1201 Level", SIGCOMM Symposium proceedings on Communications 1202 architectures and protocols , August 1995. 1204 [Zha90] Zhang, L. and D. Clark, "Oscillating Behavior of Network 1205 Traffic: A Case Study Simulation, 1206 http://groups.csail.mit.edu/ana/Publications/Zhang-DDC- 1207 Oscillating-Behavior-of-Network-Traffic-1990.pdf", 1990. 1209 Appendix A. Change Log 1211 Initial Version: March 2013 1213 Minor update of the algorithms that the IETF recommends SHOULD NOT 1214 require operational (especially manual) configuration or tuningdate: 1216 April 2013 1218 Major surgery. This draft is for discussion at IETF-87 and expected 1219 to be further updated. 1220 July 2013 1222 -00 WG Draft - Updated transport recommendations; revised deployment 1223 configuration section; numerous minor edits. 1224 Oct 2013 1226 -01 WG Draft - Updated transport recommendations; revised deployment 1227 configuration section; numerous minor edits. 1228 Jan 2014 - Feedback from WG. 1230 -02 WG Draft - Minor edits Feb 2014 - Mainly language fixes. 1232 -03 WG Draft - Minor edits Feb 2013 - Comments from David Collier- 1233 Brown and David Taht. 1235 -04 WG Draft - Minor edits May 2014 - Comments during WGLC: Provided 1236 some introductory subsections to help people (with subsections and 1237 better text). - Written more on the role scheduling. - Clarified 1238 that ECN mark threshold needs to be configurable. - Reworked your 1239 "knee" para. Various updates in response to feedback. 1241 -05 WG Draft - Minor edits June 2014 - New text added to address 1242 further comments, and improve introduction - adding context, 1243 reference to Conex, linking between sections, added text on 1244 synchronisation. 1246 -06 WG Draft - Minor edits July 2014 - Reorganised the introduction 1247 following WG feedback to better explain how this relates to the 1248 original goals of RGFC2309. Added item on packet bursts. Various 1249 minor corrections incorporatd - no change to main recommendations. 1251 Authors' Addresses 1253 Fred Baker (editor) 1254 Cisco Systems 1255 Santa Barbara, California 93117 1256 USA 1258 Email: fred@cisco.com 1260 Godred Fairhurst (editor) 1261 University of Aberdeen 1262 School of Engineering 1263 Fraser Noble Building 1264 Aberdeen, Scotland AB24 3UE 1265 UK 1267 Email: gorry@erg.abdn.ac.uk 1268 URI: http://www.erg.abdn.ac.uk