idnits 2.17.1 draft-ietf-tcpm-rfc8312bis-03.txt: -(492): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(494): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 25 instances of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. ** The abstract seems to contain references ([RFC8312]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. -- The draft header indicates that this document obsoletes RFC8312, but the abstract doesn't seem to directly say this. It does mention RFC8312 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 538 has weird spacing: '... est est...' == Line 686 has weird spacing: '... max max...' -- The document date (26 July 2021) is 998 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 4960 (Obsoleted by RFC 9260) -- Obsolete informational reference (is this intentional?): RFC 8312 (Obsoleted by RFC 9438) Summary: 2 errors (**), 0 flaws (~~), 4 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TCPM L. Xu 3 Internet-Draft UNL 4 Obsoletes: 8312 (if approved) S. Ha 5 Intended status: Standards Track Colorado 6 Expires: 27 January 2022 I. Rhee 7 Bowery 8 V. Goel 9 Apple Inc. 10 L. Eggert, Ed. 11 NetApp 12 26 July 2021 14 CUBIC for Fast and Long-Distance Networks 15 draft-ietf-tcpm-rfc8312bis-03 17 Abstract 19 CUBIC is a standard TCP congestion control algorithm that uses a 20 cubic function instead of the linear window increase function on the 21 sender side to improve scalability and stability over fast and long- 22 distance networks. CUBIC has been adopted as the default TCP 23 congestion control algorithm by the Linux, Windows, and Apple stacks. 25 This document updates the specification of CUBIC to include 26 algorithmic improvements based on these implementations and recent 27 academic work. Based on the extensive deployment experience with 28 CUBIC, it also moves the specification to the Standards Track, 29 obsoleting [RFC8312]. 31 Note to Readers 33 Discussion of this draft takes place on the TCPM working group 34 mailing list (mailto:tcpm@ietf.org), which is archived at 35 https://mailarchive.ietf.org/arch/browse/tcpm/. 37 Working Group information can be found at 38 https://datatracker.ietf.org/wg/tcpm/; source code and issues list 39 for this draft can be found at https://github.com/NTAP/rfc8312bis. 41 Note to the RFC Editor 43 xml2rfc currently renders "" in the XML by surrounding the 44 corresponding text with underscores. This is highly distracting; 45 please manually remove the underscores when doing the final edits to 46 the text version of this document. 48 (There is an issue open against xml2rfc to stop doing this in the 49 future: https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/596) 51 Also, please manually change "Figure" to "Equation" for all artwork 52 with anchors beginning with "eq" - xml2rfc doesn't seem to be able to 53 do this. 55 Status of This Memo 57 This Internet-Draft is submitted in full conformance with the 58 provisions of BCP 78 and BCP 79. 60 Internet-Drafts are working documents of the Internet Engineering 61 Task Force (IETF). Note that other groups may also distribute 62 working documents as Internet-Drafts. The list of current Internet- 63 Drafts is at https://datatracker.ietf.org/drafts/current/. 65 Internet-Drafts are draft documents valid for a maximum of six months 66 and may be updated, replaced, or obsoleted by other documents at any 67 time. It is inappropriate to use Internet-Drafts as reference 68 material or to cite them other than as "work in progress." 70 This Internet-Draft will expire on 27 January 2022. 72 Copyright Notice 74 Copyright (c) 2021 IETF Trust and the persons identified as the 75 document authors. All rights reserved. 77 This document is subject to BCP 78 and the IETF Trust's Legal 78 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 79 license-info) in effect on the date of publication of this document. 80 Please review these documents carefully, as they describe your rights 81 and restrictions with respect to this document. Code Components 82 extracted from this document must include Simplified BSD License text 83 as described in Section 4.e of the Trust Legal Provisions and are 84 provided without warranty as described in the Simplified BSD License. 86 Table of Contents 88 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 89 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 5 90 3. Design Principles of CUBIC . . . . . . . . . . . . . . . . . 5 91 3.1. Principle 1 for the CUBIC Increase Function . . . . . . . 5 92 3.2. Principle 2 for AIMD Friendliness . . . . . . . . . . . . 6 93 3.3. Principle 3 for RTT Fairness . . . . . . . . . . . . . . 7 94 3.4. Principle 4 for the CUBIC Decrease Factor . . . . . . . . 7 95 4. CUBIC Congestion Control . . . . . . . . . . . . . . . . . . 8 96 4.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 8 97 4.1.1. Constants of Interest . . . . . . . . . . . . . . . . 8 98 4.1.2. Variables of Interest . . . . . . . . . . . . . . . . 8 99 4.2. Window Increase Function . . . . . . . . . . . . . . . . 9 100 4.3. AIMD-Friendly Region . . . . . . . . . . . . . . . . . . 11 101 4.4. Concave Region . . . . . . . . . . . . . . . . . . . . . 13 102 4.5. Convex Region . . . . . . . . . . . . . . . . . . . . . . 13 103 4.6. Multiplicative Decrease . . . . . . . . . . . . . . . . . 13 104 4.7. Fast Convergence . . . . . . . . . . . . . . . . . . . . 14 105 4.8. Timeout . . . . . . . . . . . . . . . . . . . . . . . . . 15 106 4.9. Spurious Congestion Events . . . . . . . . . . . . . . . 15 107 4.10. Slow Start . . . . . . . . . . . . . . . . . . . . . . . 17 108 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 17 109 5.1. Fairness to AIMD TCP . . . . . . . . . . . . . . . . . . 18 110 5.2. Using Spare Capacity . . . . . . . . . . . . . . . . . . 20 111 5.3. Difficult Environments . . . . . . . . . . . . . . . . . 21 112 5.4. Investigating a Range of Environments . . . . . . . . . . 21 113 5.5. Protection against Congestion Collapse . . . . . . . . . 21 114 5.6. Fairness within the Alternative Congestion Control 115 Algorithm . . . . . . . . . . . . . . . . . . . . . . . 21 116 5.7. Performance with Misbehaving Nodes and Outside 117 Attackers . . . . . . . . . . . . . . . . . . . . . . . 21 118 5.8. Behavior for Application-Limited Flows . . . . . . . . . 21 119 5.9. Responses to Sudden or Transient Events . . . . . . . . . 22 120 5.10. Incremental Deployment . . . . . . . . . . . . . . . . . 22 121 6. Security Considerations . . . . . . . . . . . . . . . . . . . 22 122 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 123 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 124 8.1. Normative References . . . . . . . . . . . . . . . . . . 22 125 8.2. Informative References . . . . . . . . . . . . . . . . . 23 126 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 26 127 Appendix B. Evolution of CUBIC . . . . . . . . . . . . . . . . . 26 128 B.1. Since draft-ietf-tcpm-rfc8312bis-02 . . . . . . . . . . . 26 129 B.2. Since draft-ietf-tcpm-rfc8312bis-01 . . . . . . . . . . . 26 130 B.3. Since draft-ietf-tcpm-rfc8312bis-00 . . . . . . . . . . . 26 131 B.4. Since draft-eggert-tcpm-rfc8312bis-03 . . . . . . . . . . 26 132 B.5. Since draft-eggert-tcpm-rfc8312bis-02 . . . . . . . . . . 27 133 B.6. Since draft-eggert-tcpm-rfc8312bis-01 . . . . . . . . . . 27 134 B.7. Since draft-eggert-tcpm-rfc8312bis-00 . . . . . . . . . . 27 135 B.8. Since RFC8312 . . . . . . . . . . . . . . . . . . . . . . 28 136 B.9. Since the Original Paper . . . . . . . . . . . . . . . . 28 137 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29 139 1. Introduction 141 CUBIC has been adopted as the default TCP congestion control 142 algorithm in the Linux, Windows, and Apple stacks, and has been used 143 and deployed globally. Extensive, decade-long deployment experience 144 in vastly different Internet scenarios has convincingly demonstrated 145 that CUBIC is safe for deployment on the global Internet and delivers 146 substantial benefits over classical AIMD congestion control. It is 147 therefore to be regarded as the current standard for TCP congestion 148 control. CUBIC can also be used for other transport protocols such 149 as QUIC [RFC9000] and SCTP [RFC4960] as a default congestion 150 controller. 152 The design of CUBIC was motivated by the well-documented problem 153 classical TCP has with low utilization over fast and long-distance 154 networks [K03][RFC3649]. This problem arises from a slow increase of 155 the congestion window following a congestion event in a network with 156 a large bandwidth-delay product (BDP). [HKLRX06] indicates that this 157 problem is frequently observed even in the range of congestion window 158 sizes over several hundreds of packets. This problem is equally 159 applicable to all Reno-style TCP standards and their variants, 160 including TCP-Reno [RFC5681], TCP-NewReno [RFC6582][RFC6675], SCTP 161 [RFC4960], and TFRC [RFC5348], which use the same linear increase 162 function for window growth. We refer to all Reno-style TCP standards 163 and their variants collectively as "AIMD TCP" below because they use 164 the Additive Increase and Multiplicative Decrease algorithm (AIMD). 166 CUBIC, originally proposed in [HRX08], is a modification to the 167 congestion control algorithm of classical AIMD TCP to remedy this 168 problem. This document describes the most recent specification of 169 CUBIC. Specifically, CUBIC uses a cubic function instead of the 170 linear window increase function of AIMD TCP to improve scalability 171 and stability under fast and long-distance networks. 173 Binary Increase Congestion Control (BIC-TCP) [XHR04], a predecessor 174 of CUBIC, was selected as the default TCP congestion control 175 algorithm by Linux in the year 2005 and had been used for several 176 years by the Internet community at large. 178 CUBIC uses a similar window increase function as BIC-TCP and is 179 designed to be less aggressive and fairer to AIMD TCP in bandwidth 180 usage than BIC-TCP while maintaining the strengths of BIC-TCP such as 181 stability, window scalability, and round-trip time (RTT) fairness. 183 In the following sections, we first briefly explain the design 184 principles of CUBIC, then provide the exact specification of CUBIC, 185 and finally discuss the safety features of CUBIC following the 186 guidelines specified in [RFC5033]. 188 2. Conventions 190 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 191 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 192 "OPTIONAL" in this document are to be interpreted as described in 193 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 194 capitals, as shown here. 196 3. Design Principles of CUBIC 198 CUBIC is designed according to the following design principles: 200 Principle 1: For better network utilization and stability, CUBIC 201 uses both the concave and convex profiles of a cubic function to 202 increase the congestion window size, instead of using just a 203 convex function. 205 Principle 2: To be AIMD-friendly, CUBIC is designed to behave like 206 AIMD TCP in networks with short RTTs and small bandwidth where 207 AIMD TCP performs well. 209 Principle 3: For RTT-fairness, CUBIC is designed to achieve linear 210 bandwidth sharing among flows with different RTTs. 212 Principle 4: CUBIC appropriately sets its multiplicative window 213 decrease factor in order to balance between the scalability and 214 convergence speed. 216 3.1. Principle 1 for the CUBIC Increase Function 218 For better network utilization and stability, CUBIC [HRX08] uses a 219 cubic window increase function in terms of the elapsed time from the 220 last congestion event. While most alternative congestion control 221 algorithms to AIMD TCP increase the congestion window using convex 222 functions, CUBIC uses both the concave and convex profiles of a cubic 223 function for window growth. 225 After a window reduction in response to a congestion event detected 226 by duplicate ACKs, Explicit Congestion Notification-Echo (ECN-Echo, 227 ECE) ACKs [RFC3168], TCP RACK [RFC8985] or QUIC loss detection 228 [RFC9002], CUBIC remembers the congestion window size at which it 229 received the congestion event and performs a multiplicative decrease 230 of the congestion window. When CUBIC enters into congestion 231 avoidance, it starts to increase the congestion window using the 232 concave profile of the cubic function. The cubic function is set to 233 have its plateau at the remembered congestion window size, so that 234 the concave window increase continues until then. After that, the 235 cubic function turns into a convex profile and the convex window 236 increase begins. 238 This style of window adjustment (concave and then convex) improves 239 the algorithm stability while maintaining high network utilization 240 [CEHRX07]. This is because the window size remains almost constant, 241 forming a plateau around the remembered congestion window size of the 242 last congestion event, where network utilization is deemed highest. 243 Under steady state, most window size samples of CUBIC are close to 244 that remembered congestion window size, thus promoting high network 245 utilization and stability. 247 Note that congestion control algorithms that only use convex 248 functions to increase the congestion window size have their maximum 249 increments around the remembered congestion window size of the last 250 congestion event, and thus introduce many packet bursts around the 251 saturation point of the network, likely causing frequent global loss 252 synchronizations. 254 3.2. Principle 2 for AIMD Friendliness 256 CUBIC promotes per-flow fairness to AIMD TCP. Note that AIMD TCP 257 performs well over paths with short RTTs and small bandwidths (or 258 small BDPs). There is only a scalability problem in networks with 259 long RTTs and large bandwidths (or large BDPs). 261 A congestion control algorithm designed to be friendly to AIMD TCP on 262 a per-flow basis must increase its congestion window less 263 aggressively in small BDP networks than in large BDP networks. 265 The aggressiveness of CUBIC mainly depends on the maximum window size 266 before a window reduction, which is smaller in small-BDP networks 267 than in large-BDP networks. Thus, CUBIC increases its congestion 268 window less aggressively in small-BDP networks than in large-BDP 269 networks. 271 Furthermore, in cases when the cubic function of CUBIC would increase 272 the congestion window less aggressively than AIMD TCP, CUBIC simply 273 follows the window size of AIMD TCP to ensure that CUBIC achieves at 274 least the same throughput as AIMD TCP in small-BDP networks. We call 275 this region where CUBIC behaves like AIMD TCP the "AIMD-friendly 276 region". 278 3.3. Principle 3 for RTT Fairness 280 Two CUBIC flows with different RTTs have a throughput ratio that is 281 linearly proportional to the inverse of their RTT ratio, where the 282 throughput of a flow is approximately the size of its congestion 283 window divided by its RTT. 285 Specifically, CUBIC maintains a window increase rate independent of 286 RTTs outside the AIMD-friendly region, and thus flows with different 287 RTTs have similar congestion window sizes under steady state when 288 they operate outside the AIMD-friendly region. 290 This notion of a linear throughput ratio is similar to that of AIMD 291 TCP under high statistical multiplexing where packet loss is 292 independent of individual flow rates. However, under low statistical 293 multiplexing, the throughput ratio of AIMD TCP flows with different 294 RTTs is quadratically proportional to the inverse of their RTT ratio 295 [XHR04]. 297 CUBIC always ensures a linear throughput ratio independent of the 298 amount of statistical multiplexing. This is an improvement over AIMD 299 TCP. While there is no consensus on particular throughput ratios for 300 different RTT flows, we believe that over wired Internet paths, use 301 of a linear throughput ratio seems more reasonable than equal 302 throughputs (i.e., the same throughput for flows with different RTTs) 303 or a higher-order throughput ratio (e.g., a quadratical throughput 304 ratio of AIMD TCP under low statistical multiplexing environments). 306 3.4. Principle 4 for the CUBIC Decrease Factor 308 To balance between scalability and convergence speed, CUBIC sets the 309 multiplicative window decrease factor to 0.7, whereas AIMD TCP uses 310 0.5. 312 While this improves the scalability of CUBIC, a side effect of this 313 decision is slower convergence, especially under low statistical 314 multiplexing. This design choice is following the observation that 315 HighSpeed TCP (HSTCP) [RFC3649] and other approaches (e.g., [GV02]) 316 made: the current Internet becomes more asynchronous with less 317 frequent loss synchronizations under high statistical multiplexing. 319 In such environments, even strict Multiplicative-Increase 320 Multiplicative-Decrease (MIMD) can converge. CUBIC flows with the 321 same RTT always converge to the same throughput independent of 322 statistical multiplexing, thus achieving intra-algorithm fairness. 323 We also find that in environments with sufficient statistical 324 multiplexing, the convergence speed of CUBIC is reasonable. 326 4. CUBIC Congestion Control 328 In this section, we discuss how the congestion window is updated 329 during the different stages of the CUBIC congestion controller. 331 4.1. Definitions 333 The unit of all window sizes in this document is segments of the 334 maximum segment size (MSS), and the unit of all times is seconds. 335 Implementations can use bytes to express window sizes, which would 336 require factoring in the maximum segment size wherever necessary and 337 replacing _segments_acked_ with the number of bytes acknowledged in 338 Figure 4. 340 4.1.1. Constants of Interest 342 β__cubic_: CUBIC multiplication decrease factor as described in 343 Section 4.6. 345 α__cubic_: CUBIC additive increase factor used in AIMD-friendly 346 region as described in Section 4.3. 348 _C_: constant that determines the aggressiveness of CUBIC in 349 competing with other congestion control algorithms in high BDP 350 networks. Please see Section 5 for more explanation on how it is 351 set. The unit for _C_ is 353 segment 354 ------- 355 3 356 second 358 4.1.2. Variables of Interest 360 This section defines the variables required to implement CUBIC: 362 _RTT_: Smoothed round-trip time in seconds, calculated as described 363 in [RFC6298]. 365 _cwnd_: Current congestion window in segments. 367 _ssthresh_: Current slow start threshold in segments. 369 _W_max_: Size of _cwnd_ in segments just before _cwnd_ was reduced in 370 the last congestion event when fast convergence is disabled. 371 However, if fast convergence is enabled, the size may be further 372 reduced based on the current saturation point. 374 _K_: The time period in seconds it takes to increase the congestion 375 window size at the beginning of the current congestion avoidance 376 stage to _W_max_. 378 _current_time_: Current time of the system in seconds. 380 _epoch_start_: The time in seconds at which the current congestion 381 avoidance stage started. 383 _cwnd_start_: The _cwnd_ at the beginning of the current congestion 384 avoidance stage, i.e., at time _epoch_start_. 386 W_cubic(_t_): The congestion window in segments at time _t_ in 387 seconds based on the cubic increase function, as described in 388 Section 4.2. 390 _target_: Target value of congestion window in segments after the 391 next RTT, that is, W_cubic(_t_ + _RTT_), as described in Section 4.2. 393 _W_est_: An estimate for the congestion window in segments in the 394 AIMD-friendly region, that is, an estimate for the congestion window 395 of AIMD TCP. 397 _segments_acked_: Number of MSS-sized segments acked when an ACK is 398 received. This number will be a decimal value when an ACK 399 acknowledges an amount of data that is not MSS-sized. Specifically, 400 it can be less than 1 when an ACK acknowledges a segment smaller than 401 the MSS. 403 4.2. Window Increase Function 405 CUBIC maintains the acknowledgment (ACK) clocking of AIMD TCP by 406 increasing the congestion window only at the reception of an ACK. It 407 does not make any changes to the TCP Fast Recovery and Fast 408 Retransmit algorithms [RFC6582][RFC6675]. 410 During congestion avoidance, after a congestion event is detected by 411 mechanisms described in Section 3.1, CUBIC changes the window 412 increase function of AIMD TCP. 414 CUBIC uses the following window increase function: 416 3 417 W (t) = C * (t - K) + W 418 cubic max 420 Figure 1 422 where _t_ is the elapsed time in seconds from the beginning of the 423 current congestion avoidance stage, that is, 425 t = current_time - epoch 426 start 428 and where _epoch_start_ is the time at which the current congestion 429 avoidance stage starts. _K_ is the time period that the above 430 function takes to increase the congestion window size at the 431 beginning of the current congestion avoidance stage to _W_max_ if 432 there are no further congestion events and is calculated using the 433 following equation: 435 ________________ 436 /W - cwnd 437 3 / max start 438 K = | / ---------------- 439 |/ C 441 Figure 2 443 where _cwnd_start_ is the congestion window at the beginning of the 444 current congestion avoidance stage. For example, right after a 445 congestion event, _cwnd_start_ is equal to the new cwnd calculated as 446 described in Section 4.6. 448 Upon receiving an ACK during congestion avoidance, CUBIC computes the 449 _target_ congestion window size after the next _RTT_ using Figure 1 450 as follows, where _RTT_ is the smoothed round-trip time. The lower 451 and upper bounds below ensure that CUBIC's congestion window increase 452 rate is non-decreasing and is less than the increase rate of slow 453 start. 455 / 456 | if W (t + RTT) < cwnd 457 |cwnd cubic 458 | 459 | 460 | 461 target = < if W (t + RTT) > 1.5 * cwnd 462 |1.5 * cwnd cubic 463 | 464 | 465 |W (t + RTT) 466 | cubic otherwise 467 \ 469 Depending on the value of the current congestion window size _cwnd_, 470 CUBIC runs in three different regions: 472 1. The AIMD-friendly region, which ensures that CUBIC achieves at 473 least the same throughput as AIMD TCP. 475 2. The concave region, if CUBIC is not in the AIMD-friendly region 476 and _cwnd_ is less than _W_max_. 478 3. The convex region, if CUBIC is not in the AIMD-friendly region 479 and _cwnd_ is greater than _W_max_. 481 Below, we describe the exact actions taken by CUBIC in each region. 483 4.3. AIMD-Friendly Region 485 AIMD TCP performs well in certain types of networks, for example, 486 under short RTTs and small bandwidths (or small BDPs). In these 487 networks, CUBIC remains in the AIMD-friendly region to achieve at 488 least the same throughput as AIMD TCP. 490 The AIMD-friendly region is designed according to the analysis in 491 [FHP00], which studies the performance of an AIMD algorithm with an 492 additive factor of α (segments per _RTT_) and a multiplicative factor 493 of β, denoted by AIMD(α, β). _p_ is the packet loss rate. 494 Specifically, the average congestion window size of AIMD(α, β) can be 495 calculated using Figure 3. 497 _______________ 498 / α * (1 + β) 499 AVG_AIMD(α, β) = | / --------------- 500 |/ 2 * (1 - β) * p 502 Figure 3 504 By the same analysis, to achieve the same average window size as AIMD 505 TCP that uses AIMD(1, 0.5), α must be equal to, 507 1 - β 508 3 * ----- 509 1 + β 511 Thus, CUBIC uses Figure 4 to estimate the window size _W_est_ in the 512 AIMD-friendly region with 514 1 - β 515 cubic 516 α = 3 * ---------- 517 cubic 1 + β 518 cubic 520 which achieves the same average window size as AIMD TCP. When 521 receiving an ACK in congestion avoidance (where _cwnd_ could be 522 greater than or less than _W_max_), CUBIC checks whether W_cubic(_t_) 523 is less than _W_est_. If so, CUBIC is in the AIMD-friendly region and 524 _cwnd_ SHOULD be set to _W_est_ at each reception of an ACK. 526 _W_est_ is set equal to _cwnd_start_ at the start of the congestion 527 avoidance stage. After that, on every ACK, _W_est_ is updated using 528 Figure 4. Note that this equation is for a connection where 529 Appropriate Byte Counting (ABC) [RFC3465] is disabled. For a 530 connection with ABC enabled, this equation SHOULD be adjusted by 531 using the number of acknowledged bytes instead of acknowledged 532 segments. Also note that this equation works for connections with 533 enabled or disabled Delayed ACKs [RFC5681], as _segments_acked_ will 534 be different based on the segments actually acknowledged by an ACK. 536 segments_acked 537 W = W + α * -------------- 538 est est cubic cwnd 540 Figure 4 542 Note that once _W_est_ reaches _W_max_, that is, _W_est_ >= _W_max_, 543 CUBIC needs to start probing to determine the new value of _W_max_. 544 At this point, α__cubic_ SHOULD be set to 1 to ensure that CUBIC can 545 achieve the same congestion window increment as AIMD TCP, which uses 546 AIMD(1, 0.5). 548 4.4. Concave Region 550 When receiving an ACK in congestion avoidance, if CUBIC is not in the 551 AIMD-friendly region and _cwnd_ is less than _W_max_, then CUBIC is 552 in the concave region. In this region, _cwnd_ MUST be incremented by 554 target - cwnd 555 ------------- 556 cwnd 558 for each received ACK, where _target_ is calculated as described in 559 Section 4.2. 561 4.5. Convex Region 563 When receiving an ACK in congestion avoidance, if CUBIC is not in the 564 AIMD-friendly region and _cwnd_ is larger than or equal to _W_max_, 565 then CUBIC is in the convex region. 567 The convex region indicates that the network conditions might have 568 changed since the last congestion event, possibly implying more 569 available bandwidth after some flow departures. Since the Internet 570 is highly asynchronous, some amount of perturbation is always 571 possible without causing a major change in available bandwidth. 573 Unless it is overridden by the AIMD window increase, CUBIC is very 574 careful in this region. The convex profile aims to increase the 575 window very slowly at the beginning when _cwnd_ is around _W_max_ and 576 then gradually increases its rate of increase. We also call this 577 region the "maximum probing phase", since CUBIC is searching for a 578 new _W_max_. In this region, _cwnd_ MUST be incremented by 580 target - cwnd 581 ------------- 582 cwnd 584 for each received ACK, where _target_ is calculated as described in 585 Section 4.2. 587 4.6. Multiplicative Decrease 589 When a congestion event is detected by mechanisms described in 590 Section 3.1, CUBIC updates _W_max_ and reduces _cwnd_ and _ssthresh_ 591 immediately as described below. An implementation MAY set a smaller 592 _ssthresh_ than suggested below to accommodate rate-limited 593 applications as described in [RFC7661]. For both packet loss and 594 congestion detection through ECN, the sender MAY employ a Fast 595 Recovery algorithm to gradually adjust the congestion window to its 596 new reduced _ssthresh_ value. The parameter β__cubic_ SHOULD be set 597 to 0.7. 599 ssthresh = cwnd * β // new slow-start threshold 600 cubic 602 ssthresh = max(ssthresh, 2) // threshold is at least 2 MSS 604 // window reduction 605 cwnd = ssthresh 607 A side effect of setting β__cubic_ to a value bigger than 0.5 is 608 slower convergence. We believe that while a more adaptive setting of 609 β__cubic_ could result in faster convergence, it will make the 610 analysis of CUBIC much harder. 612 4.7. Fast Convergence 614 To improve convergence speed, CUBIC uses a heuristic. When a new 615 flow joins the network, existing flows need to give up some of their 616 bandwidth to allow the new flow some room for growth, if the existing 617 flows have been using all the network bandwidth. To speed up this 618 bandwidth release by existing flows, the following "Fast Convergence" 619 mechanism SHOULD be implemented. 621 With Fast Convergence, when a congestion event occurs, we update 622 _W_max_ as follows, before the window reduction as described in 623 Section 4.6. 625 / 626 | 1 + β 627 | cubic if cwnd < W and fast convergence is enabled, 628 |cwnd * ---------- max 629 | 2 630 W = < 631 max | further reduce W 632 | max 633 | 634 | otherwise, remember cwnd before reduction 635 \cwnd 637 At a congestion event, if the current _cwnd_ is less than _W_max_, 638 this indicates that the saturation point experienced by this flow is 639 getting reduced because of a change in available bandwidth. Then we 640 allow this flow to release more bandwidth by reducing _W_max_ 641 further. This action effectively lengthens the time for this flow to 642 increase its congestion window, because the reduced _W_max_ forces 643 the flow to plateau earlier. This allows more time for the new flow 644 to catch up to its congestion window size. 646 Fast Convergence is designed for network environments with multiple 647 CUBIC flows. In network environments with only a single CUBIC flow 648 and without any other traffic, Fast Convergence SHOULD be disabled. 650 4.8. Timeout 652 In case of a timeout, CUBIC follows AIMD TCP to reduce _cwnd_ 653 [RFC5681], but sets _ssthresh_ using β__cubic_ (same as in 654 Section 4.6) in a way that is different from AIMD TCP [RFC5681]. 656 During the first congestion avoidance stage after a timeout, CUBIC 657 increases its congestion window size using Figure 1, where _t_ is the 658 elapsed time since the beginning of the current congestion avoidance, 659 _K_ is set to 0, and _W_max_ is set to the congestion window size at 660 the beginning of the current congestion avoidance stage. In 661 addition, for the AIMD-friendly region, _W_est_ SHOULD be set to the 662 congestion window size at the beginning of the current congestion 663 avoidance. 665 4.9. Spurious Congestion Events 667 In cases where CUBIC reduces its congestion window in response to 668 having detected packet loss via duplicate ACKs or timeouts, there is 669 a possibility that the missing ACK would arrive after the congestion 670 window reduction and a corresponding packet retransmission. For 671 example, packet reordering could trigger this behavior. A high 672 degree of packet reordering could cause multiple congestion window 673 reduction events, where spurious losses are incorrectly interpreted 674 as congestion signals, thus degrading CUBIC's performance 675 significantly. 677 When there is a congestion event, a CUBIC implementation SHOULD save 678 the current value of the following variables before the congestion 679 window reduction. 681 prior_cwnd = cwnd 683 prior_ssthresh = ssthresh 685 prior_W = W 686 max max 688 prior_K = K 690 prior_epoch = epoch 691 start start 693 prior_W_{est} = W 694 est 696 CUBIC MAY implement an algorithm to detect spurious retransmissions, 697 such as DSACK [RFC3708], Forward RTO-Recovery [RFC5682] or Eifel 698 [RFC3522]. Once a spurious congestion event is detected, CUBIC 699 SHOULD restore the original values of above-mentioned variables as 700 follows if the current _cwnd_ is lower than _prior_cwnd_. Restoring 701 the original values ensures that CUBIC's performance is similar to 702 what it would be without spurious losses. 704 \ 705 cwnd = prior_cwnd | 706 | 707 ssthresh = prior_ssthresh | 708 | 709 W = prior_W | 710 max max | 711 >if cwnd < prior_cwnd 712 K = prior_K | 713 | 714 epoch = prior_epoch | 715 start start| 716 | 717 W = prior_W | 718 est est / 720 In rare cases, when the detection happens long after a spurious loss 721 event and the current _cwnd_ is already higher than _prior_cwnd_, 722 CUBIC SHOULD continue to use the current and the most recent values 723 of these variables. 725 4.10. Slow Start 727 CUBIC MUST employ a slow-start algorithm, when _cwnd_ is no more than 728 _ssthresh_. Among the slow-start algorithms, CUBIC MAY choose the 729 AIMD TCP slow start [RFC5681] in general networks, or the limited 730 slow start [RFC3742] or hybrid slow start [HR08] for fast and long- 731 distance networks. 733 When CUBIC uses hybrid slow start [HR08], it may exit the first slow 734 start without incurring any packet loss and thus _W_max_ is 735 undefined. In this special case, CUBIC switches to congestion 736 avoidance and increases its congestion window size using Figure 1, 737 where _t_ is the elapsed time since the beginning of the current 738 congestion avoidance, _K_ is set to 0, and _W_max_ is set to the 739 congestion window size at the beginning of the current congestion 740 avoidance stage. 742 5. Discussion 744 In this section, we further discuss the safety features of CUBIC 745 following the guidelines specified in [RFC5033]. 747 With a deterministic loss model where the number of packets between 748 two successive packet losses is always _1/p_, CUBIC always operates 749 with the concave window profile, which greatly simplifies the 750 performance analysis of CUBIC. The average window size of CUBIC can 751 be obtained by the following function: 753 ________________ ____ 754 /C * (3 + β ) 3 / 4 755 4 / cubic |/ RTT 756 AVG_W = | / ---------------- * ------- 757 cubic | / 4 * (1 - β ) __ 758 |/ cubic 3 / 4 759 |/ p 761 Figure 5 763 With β__cubic_ set to 0.7, the above formula reduces to: 765 ____ 766 _______ 3 / 4 767 4 /C * 3.7 |/ RTT 768 AVG_W = | / ------- * ------- 769 cubic |/ 1.2 __ 770 3 / 4 771 |/ p 773 Figure 6 775 We will determine the value of _C_ in the following subsection using 776 Figure 6. 778 5.1. Fairness to AIMD TCP 780 In environments where AIMD TCP is able to make reasonable use of the 781 available bandwidth, CUBIC does not significantly change this state. 783 AIMD TCP performs well in the following two types of networks: 785 1. networks with a small bandwidth-delay product (BDP) 787 2. networks with a short RTTs, but not necessarily a small BDP 789 CUBIC is designed to behave very similarly to AIMD TCP in the above 790 two types of networks. The following two tables show the average 791 window sizes of AIMD TCP, HSTCP, and CUBIC. The average window sizes 792 of AIMD TCP and HSTCP are from [RFC3649]. The average window size of 793 CUBIC is calculated using Figure 6 and the CUBIC AIMD-friendly region 794 for three different values of _C_. 796 +=============+=======+========+================+=========+========+ 797 | Loss Rate P | AIMD | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | 798 | | | | | (C=0.4) | (C=4) | 799 +=============+=======+========+================+=========+========+ 800 | 1.0e-02 | 12 | 12 | 12 | 12 | 12 | 801 +-------------+-------+--------+----------------+---------+--------+ 802 | 1.0e-03 | 38 | 38 | 38 | 38 | 59 | 803 +-------------+-------+--------+----------------+---------+--------+ 804 | 1.0e-04 | 120 | 263 | 120 | 187 | 333 | 805 +-------------+-------+--------+----------------+---------+--------+ 806 | 1.0e-05 | 379 | 1795 | 593 | 1054 | 1874 | 807 +-------------+-------+--------+----------------+---------+--------+ 808 | 1.0e-06 | 1200 | 12280 | 3332 | 5926 | 10538 | 809 +-------------+-------+--------+----------------+---------+--------+ 810 | 1.0e-07 | 3795 | 83981 | 18740 | 33325 | 59261 | 811 +-------------+-------+--------+----------------+---------+--------+ 812 | 1.0e-08 | 12000 | 574356 | 105383 | 187400 | 333250 | 813 +-------------+-------+--------+----------------+---------+--------+ 815 Table 1: AIMD TCP, HSTCP, and CUBIC with RTT = 0.1 seconds 817 Table 1 describes the response function of AIMD TCP, HSTCP, and CUBIC 818 in networks with _RTT_ = 0.1 seconds. The average window size is in 819 MSS-sized segments. 821 +=============+=======+========+================+=========+=======+ 822 | Loss Rate P | AIMD | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | 823 | | | | | (C=0.4) | (C=4) | 824 +=============+=======+========+================+=========+=======+ 825 | 1.0e-02 | 12 | 12 | 12 | 12 | 12 | 826 +-------------+-------+--------+----------------+---------+-------+ 827 | 1.0e-03 | 38 | 38 | 38 | 38 | 38 | 828 +-------------+-------+--------+----------------+---------+-------+ 829 | 1.0e-04 | 120 | 263 | 120 | 120 | 120 | 830 +-------------+-------+--------+----------------+---------+-------+ 831 | 1.0e-05 | 379 | 1795 | 379 | 379 | 379 | 832 +-------------+-------+--------+----------------+---------+-------+ 833 | 1.0e-06 | 1200 | 12280 | 1200 | 1200 | 1874 | 834 +-------------+-------+--------+----------------+---------+-------+ 835 | 1.0e-07 | 3795 | 83981 | 3795 | 5926 | 10538 | 836 +-------------+-------+--------+----------------+---------+-------+ 837 | 1.0e-08 | 12000 | 574356 | 18740 | 33325 | 59261 | 838 +-------------+-------+--------+----------------+---------+-------+ 840 Table 2: AIMD TCP, HSTCP, and CUBIC with RTT = 0.01 seconds 842 Table 2 describes the response function of AIMD TCP, HSTCP, and CUBIC 843 in networks with _RTT_ = 0.01 seconds. The average window size is in 844 MSS-sized segments. 846 Both tables show that CUBIC with any of these three _C_ values is 847 more friendly to AIMD TCP than HSTCP, especially in networks with a 848 short _RTT_ where AIMD TCP performs reasonably well. For example, in 849 a network with _RTT_ = 0.01 seconds and p=10^-6, AIMD TCP has an 850 average window of 1200 packets. If the packet size is 1500 bytes, 851 then AIMD TCP can achieve an average rate of 1.44 Gbps. In this 852 case, CUBIC with _C_=0.04 or _C_=0.4 achieves exactly the same rate 853 as AIMD TCP, whereas HSTCP is about ten times more aggressive than 854 AIMD TCP. 856 We can see that _C_ determines the aggressiveness of CUBIC in 857 competing with other congestion control algorithms for bandwidth. 858 CUBIC is more friendly to AIMD TCP, if the value of _C_ is lower. 859 However, we do not recommend setting _C_ to a very low value like 860 0.04, since CUBIC with a low _C_ cannot efficiently use the bandwidth 861 in fast and long-distance networks. Based on these observations and 862 extensive deployment experience, we find _C_=0.4 gives a good balance 863 between AIMD- friendliness and aggressiveness of window increase. 864 Therefore, _C_ SHOULD be set to 0.4. With _C_ set to 0.4, Figure 6 865 is reduced to: 867 ____ 868 3 / 4 869 |/ RTT 870 AVG_W = 1.054 * ------- 871 cubic __ 872 3 / 4 873 |/ p 875 Figure 7 877 Figure 7 is then used in the next subsection to show the scalability 878 of CUBIC. 880 5.2. Using Spare Capacity 882 CUBIC uses a more aggressive window increase function than AIMD TCP 883 for fast and long-distance networks. 885 The following table shows that to achieve the 10 Gbps rate, AIMD TCP 886 requires a packet loss rate of 2.0e-10, while CUBIC requires a packet 887 loss rate of 2.9e-8. 889 +===================+===========+=========+=========+=========+ 890 | Throughput (Mbps) | Average W | AIMD P | HSTCP P | CUBIC P | 891 +===================+===========+=========+=========+=========+ 892 | 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 | 893 +-------------------+-----------+---------+---------+---------+ 894 | 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 | 895 +-------------------+-----------+---------+---------+---------+ 896 | 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 | 897 +-------------------+-----------+---------+---------+---------+ 898 | 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 | 899 +-------------------+-----------+---------+---------+---------+ 900 | 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 | 901 +-------------------+-----------+---------+---------+---------+ 903 Table 3: Required packet loss rate for AIMD TCP, HSTCP, and 904 CUBIC to achieve a certain throughput 906 Table 3 describes the required packet loss rate for AIMD TCP, HSTCP, 907 and CUBIC to achieve a certain throughput. We use 1500-byte packets 908 and an _RTT_ of 0.1 seconds. 910 Our test results in [HKLRX06] indicate that CUBIC uses the spare 911 bandwidth left unused by existing AIMD TCP flows in the same 912 bottleneck link without taking away much bandwidth from the existing 913 flows. 915 5.3. Difficult Environments 917 CUBIC is designed to remedy the poor performance of AIMD TCP in fast 918 and long-distance networks. 920 5.4. Investigating a Range of Environments 922 There is decade-long deployment experience with CUBIC on the 923 Internet. CUBIC has also been extensively studied by using both NS-2 924 simulation and testbed experiments, covering a wide range of network 925 environments. More information can be found in [HKLRX06]. 927 Same as AIMD TCP, CUBIC is a loss-based congestion control algorithm. 928 Because CUBIC is designed to be more aggressive (due to a faster 929 window increase function and bigger multiplicative decrease factor) 930 than AIMD TCP in fast and long-distance networks, it can fill large 931 drop-tail buffers more quickly than AIMD TCP and increases the risk 932 of a standing queue [RFC8511]. In this case, proper queue sizing and 933 management [RFC7567] could be used to reduce the packet queuing 934 delay. 936 5.5. Protection against Congestion Collapse 938 With regard to the potential of causing congestion collapse, CUBIC 939 behaves like AIMD TCP, since CUBIC modifies only the window 940 adjustment algorithm of AIMD TCP. Thus, it does not modify the ACK 941 clocking and timeout behaviors of AIMD TCP. 943 5.6. Fairness within the Alternative Congestion Control Algorithm 945 CUBIC ensures convergence of competing CUBIC flows with the same RTT 946 in the same bottleneck links to an equal throughput. When competing 947 flows have different RTT values, their throughput ratio is linearly 948 proportional to the inverse of their RTT ratios. This is true 949 independently of the level of statistical multiplexing on the link. 951 5.7. Performance with Misbehaving Nodes and Outside Attackers 953 This is not considered in the current CUBIC design. 955 5.8. Behavior for Application-Limited Flows 957 CUBIC does not increase its congestion window size if a flow is 958 currently limited by the application instead of the congestion 959 window. In case of long periods during which _cwnd_ has not been 960 updated due to such an application limit, such as idle periods, _t_ 961 in Figure 1 MUST NOT include these periods; otherwise, W_cubic(_t_) 962 might be very high after restarting from these periods. 964 5.9. Responses to Sudden or Transient Events 966 If there is a sudden congestion, a routing change, or a mobility 967 event, CUBIC behaves the same as AIMD TCP. 969 5.10. Incremental Deployment 971 CUBIC requires only changes to TCP senders, and it does not require 972 any changes at TCP receivers. That is, a CUBIC sender works 973 correctly with the AIMD TCP receivers. In addition, CUBIC does not 974 require any changes to routers and does not require any assistance 975 from routers. 977 6. Security Considerations 979 CUBIC makes no changes to the underlying security of TCP. More 980 information about TCP security concerns can be found in [RFC5681]. 982 7. IANA Considerations 984 This document does not require any IANA actions. 986 8. References 988 8.1. Normative References 990 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 991 Requirement Levels", BCP 14, RFC 2119, 992 DOI 10.17487/RFC2119, March 1997, 993 . 995 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 996 of Explicit Congestion Notification (ECN) to IP", 997 RFC 3168, DOI 10.17487/RFC3168, September 2001, 998 . 1000 [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion 1001 Control Algorithms", BCP 133, RFC 5033, 1002 DOI 10.17487/RFC5033, August 2007, 1003 . 1005 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1006 Friendly Rate Control (TFRC): Protocol Specification", 1007 RFC 5348, DOI 10.17487/RFC5348, September 2008, 1008 . 1010 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1011 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1012 . 1014 [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, 1015 "Computing TCP's Retransmission Timer", RFC 6298, 1016 DOI 10.17487/RFC6298, June 2011, 1017 . 1019 [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The 1020 NewReno Modification to TCP's Fast Recovery Algorithm", 1021 RFC 6582, DOI 10.17487/RFC6582, April 2012, 1022 . 1024 [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., 1025 and Y. Nishida, "A Conservative Loss Recovery Algorithm 1026 Based on Selective Acknowledgment (SACK) for TCP", 1027 RFC 6675, DOI 10.17487/RFC6675, August 2012, 1028 . 1030 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 1031 Recommendations Regarding Active Queue Management", 1032 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 1033 . 1035 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1036 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1037 May 2017, . 1039 [RFC8985] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "The 1040 RACK-TLP Loss Detection Algorithm for TCP", RFC 8985, 1041 DOI 10.17487/RFC8985, February 2021, 1042 . 1044 [RFC9002] Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection 1045 and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, 1046 May 2021, . 1048 8.2. Informative References 1050 [CEHRX07] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic 1051 Ordering for Internet Congestion Control and its 1052 Applications", IEEE INFOCOM 2007 - 26th IEEE International 1053 Conference on Computer Communications, 1054 DOI 10.1109/infcom.2007.111, 2007, 1055 . 1057 [FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of 1058 Equation-Based and AIMD Congestion Control", May 2000, 1059 . 1061 [GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary 1062 Adjustment Algorithms", Technical Report TR2002-29, 1063 Department of Computer Sciences, The University of 1064 Texas at Austin, 11 August 2002, 1065 . 1067 [HKLRX06] Ha, S., Kim, Y., Le, L., Rhee, I., and L. Xu, "A Step 1068 toward Realistic Performance Evaluation of High-Speed TCP 1069 Variants", International Workshop on Protocols for Fast 1070 Long-Distance Networks, February 2006, 1071 . 1073 [HR08] Ha, S. and I. Rhee, "Hybrid Slow Start for High-Bandwidth 1074 and Long-Distance Networks", International Workshop 1075 on Protocols for Fast Long-Distance Networks, March 2008, 1076 . 1079 [HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: a new TCP-friendly 1080 high-speed TCP variant", ACM SIGOPS Operating Systems 1081 Review Vol. 42, pp. 64-74, DOI 10.1145/1400097.1400105, 1082 July 2008, . 1084 [K03] Kelly, T., "Scalable TCP: improving performance in 1085 highspeed wide area networks", ACM SIGCOMM Computer 1086 Communication Review Vol. 33, pp. 83-91, 1087 DOI 10.1145/956981.956989, April 2003, 1088 . 1090 [RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte 1091 Counting (ABC)", RFC 3465, DOI 10.17487/RFC3465, February 1092 2003, . 1094 [RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm 1095 for TCP", RFC 3522, DOI 10.17487/RFC3522, April 2003, 1096 . 1098 [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", 1099 RFC 3649, DOI 10.17487/RFC3649, December 2003, 1100 . 1102 [RFC3708] Blanton, E. and M. Allman, "Using TCP Duplicate Selective 1103 Acknowledgement (DSACKs) and Stream Control Transmission 1104 Protocol (SCTP) Duplicate Transmission Sequence Numbers 1105 (TSNs) to Detect Spurious Retransmissions", RFC 3708, 1106 DOI 10.17487/RFC3708, February 2004, 1107 . 1109 [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large 1110 Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March 1111 2004, . 1113 [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", 1114 RFC 4960, DOI 10.17487/RFC4960, September 2007, 1115 . 1117 [RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata, 1118 "Forward RTO-Recovery (F-RTO): An Algorithm for Detecting 1119 Spurious Retransmission Timeouts with TCP", RFC 5682, 1120 DOI 10.17487/RFC5682, September 2009, 1121 . 1123 [RFC7661] Fairhurst, G., Sathiaseelan, A., and R. Secchi, "Updating 1124 TCP to Support Rate-Limited Traffic", RFC 7661, 1125 DOI 10.17487/RFC7661, October 2015, 1126 . 1128 [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and 1129 R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", 1130 RFC 8312, DOI 10.17487/RFC8312, February 2018, 1131 . 1133 [RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, 1134 "TCP Alternative Backoff with ECN (ABE)", RFC 8511, 1135 DOI 10.17487/RFC8511, December 2018, 1136 . 1138 [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 1139 Multiplexed and Secure Transport", RFC 9000, 1140 DOI 10.17487/RFC9000, May 2021, 1141 . 1143 [SXEZ19] Sun, W., Xu, L., Elbaum, S., and D. Zhao, "Model-Agnostic 1144 and Efficient Exploration of Numerical State Space of 1145 Real-World TCP Congestion Control Implementations", USENIX 1146 NSDI 2019, February 2019, 1147 . 1149 [XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary Increase 1150 Congestion Control (BIC) for Fast Long-Distance Networks", 1151 IEEE INFOCOM 2004, DOI 10.1109/infcom.2004.1354672, March 1152 2004, . 1154 Appendix A. Acknowledgments 1156 Richard Scheffenegger and Alexander Zimmermann originally co-authored 1157 [RFC8312]. 1159 Appendix B. Evolution of CUBIC 1161 B.1. Since draft-ietf-tcpm-rfc8312bis-02 1163 * Decription of packet loss rate _p_ (#65 1164 (https://github.com/NTAP/rfc8312bis/issues/65)) 1166 * Clarification of TCP Friendly Equation for ABC and Delayed ACK 1167 (#66 (https://github.com/NTAP/rfc8312bis/issues/66)) 1169 * add applicability to QUIC and SCTP (#61 1170 (https://github.com/NTAP/rfc8312bis/issues/61)) 1172 * clarity on setting alpha__aimd_ to 1 (#68 1173 (https://github.com/NTAP/rfc8312bis/issues/68)) 1175 * introduce alpha__cubic_ (#64 (https://github.com/NTAP/rfc8312bis/ 1176 issues/64)) 1178 * clarify _cwnd_ growth in convex region (#69 1179 (https://github.com/NTAP/rfc8312bis/issues/69)) 1181 * add guidance for using bytes and mention that segments count is 1182 decimal (#67 (https://github.com/NTAP/rfc8312bis/issues/67)) 1184 * add loss events detected by RACK and QUIC loss detection (#62 1185 (https://github.com/NTAP/rfc8312bis/issues/62)) 1187 B.2. Since draft-ietf-tcpm-rfc8312bis-01 1189 * address Michael Scharf's editorial suggestions. (#59 1190 (https://github.com/NTAP/rfc8312bis/issues/59)) 1192 * add "Note to the RFC Editor" about removing underscores 1194 B.3. Since draft-ietf-tcpm-rfc8312bis-00 1196 * use updated xml2rfc with better text rendering of subscripts 1198 B.4. Since draft-eggert-tcpm-rfc8312bis-03 1200 * fix spelling nits 1201 * rename to draft-ietf 1203 * define _W_max_ more clearly 1205 B.5. Since draft-eggert-tcpm-rfc8312bis-02 1207 * add definition for segments_acked and alpha__aimd_. (#47 1208 (https://github.com/NTAP/rfc8312bis/issues/47)) 1210 * fix a mistake in _W_max_ calculation in the fast convergence 1211 section. (#51 (https://github.com/NTAP/rfc8312bis/issues/51)) 1213 * clarity on setting _ssthresh_ and _cwnd_start_ during 1214 multiplicative decrease. (#53 (https://github.com/NTAP/rfc8312bis/ 1215 issues/53)) 1217 B.6. Since draft-eggert-tcpm-rfc8312bis-01 1219 * rename TCP-Friendly to AIMD-Friendly and rename Standard TCP to 1220 AIMD TCP to avoid confusion as CUBIC has been widely used on the 1221 Internet. (#38 (https://github.com/NTAP/rfc8312bis/issues/38)) 1223 * change introductory text to reflect the significant broader 1224 deployment of CUBIC on the Internet. (#39 1225 (https://github.com/NTAP/rfc8312bis/issues/39)) 1227 * rephrase introduction to avoid referring to variables that have 1228 not been defined yet. 1230 B.7. Since draft-eggert-tcpm-rfc8312bis-00 1232 * acknowledge former co-authors (#15 1233 (https://github.com/NTAP/rfc8312bis/issues/15)) 1235 * prevent _cwnd_ from becoming less than two (#7 1236 (https://github.com/NTAP/rfc8312bis/issues/7)) 1238 * add list of variables and constants (#5 1239 (https://github.com/NTAP/rfc8312bis/issues/5), #6 1240 (https://github.com/NTAP/rfc8312bis/issues/6)) 1242 * update _K_'s definition and add bounds for CUBIC _target_ _cwnd_ 1243 [SXEZ19] (#1 (https://github.com/NTAP/rfc8312bis/issues/1), #14 1244 (https://github.com/NTAP/rfc8312bis/issues/14)) 1246 * update _W_est_ to use AIMD approach (#20 1247 (https://github.com/NTAP/rfc8312bis/issues/20)) 1249 * set alpha__aimd_ to 1 once _W_est_ reaches _W_max_ (#2 1250 (https://github.com/NTAP/rfc8312bis/issues/2)) 1252 * add Vidhi as co-author (#17 (https://github.com/NTAP/rfc8312bis/ 1253 issues/17)) 1255 * note for Fast Recovery during _cwnd_ decrease due to congestion 1256 event (#11 (https://github.com/NTAP/rfc8312bis/issues/11)) 1258 * add section for spurious congestion events (#23 1259 (https://github.com/NTAP/rfc8312bis/issues/23)) 1261 * initialize _W_est_ after timeout and remove variable 1262 _W_(last_max)_ (#28 (https://github.com/NTAP/rfc8312bis/ 1263 issues/28)) 1265 B.8. Since RFC8312 1267 * converted to Markdown and xml2rfc v3 1269 * updated references (as part of the conversion) 1271 * updated author information 1273 * various formatting changes 1275 * move to Standards Track 1277 B.9. Since the Original Paper 1279 CUBIC has gone through a few changes since the initial release 1280 [HRX08] of its algorithm and implementation. Below we highlight the 1281 differences between its original paper and [RFC8312]. 1283 * The original paper [HRX08] includes the pseudocode of CUBIC 1284 implementation using Linux's pluggable congestion control 1285 framework, which excludes system-specific optimizations. The 1286 simplified pseudocode might be a good source to start with and 1287 understand CUBIC. 1289 * [HRX08] also includes experimental results showing its performance 1290 and fairness. 1292 * The definition of beta__cubic_ constant was changed in [RFC8312]. 1293 For example, beta__cubic_ in the original paper was the window 1294 decrease constant while [RFC8312] changed it to CUBIC 1295 multiplication decrease factor. With this change, the current 1296 congestion window size after a congestion event in [RFC8312] was 1297 beta__cubic_ * _W_max_ while it was (1-beta__cubic_) * _W_max_ in 1298 the original paper. 1300 * Its pseudocode used _W_(last_max)_ while [RFC8312] used _W_max_. 1302 * Its AIMD-friendly window was _W_tcp_ while [RFC8312] used _W_est_. 1304 Authors' Addresses 1306 Lisong Xu 1307 University of Nebraska-Lincoln 1308 Department of Computer Science and Engineering 1309 Lincoln, NE 68588-0115 1310 United States of America 1312 Email: xu@unl.edu 1313 URI: https://cse.unl.edu/~xu/ 1315 Sangtae Ha 1316 University of Colorado at Boulder 1317 Department of Computer Science 1318 Boulder, CO 80309-0430 1319 United States of America 1321 Email: sangtae.ha@colorado.edu 1322 URI: https://netstech.org/sangtaeha/ 1324 Injong Rhee 1325 Bowery Farming 1326 151 W 26TH Street, 12TH Floor 1327 New York, NY 10001 1328 United States of America 1330 Email: injongrhee@gmail.com 1331 Vidhi Goel 1332 Apple Inc. 1333 One Apple Park Way 1334 Cupertino, California 95014 1335 United States of America 1337 Email: vidhi_goel@apple.com 1339 Lars Eggert (editor) 1340 NetApp 1341 Stenbergintie 12 B 1342 FI-02700 Kauniainen 1343 Finland 1345 Email: lars@eggert.org 1346 URI: https://eggert.org/