idnits 2.17.1 draft-ietf-tcpm-rfc8312bis-05.txt: -(505): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(507): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(611): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(695): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 25 instances of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 12 characters in excess of 72. -- The draft header indicates that this document obsoletes RFC8312, but the abstract doesn't seem to directly say this. It does mention RFC8312 though, so this could be OK. -- The draft header indicates that this document updates RFC5681, but the abstract doesn't seem to directly say this. It does mention RFC5681 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 552 has weird spacing: '... est est...' == Line 728 has weird spacing: '... max max...' (Using the creation date from RFC5681, updated by this document, for RFC5378 checks: 2006-01-26) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (25 October 2021) is 908 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-14) exists of draft-ietf-tcpm-hystartplusplus-03 -- Obsolete informational reference (is this intentional?): RFC 4960 (Obsoleted by RFC 9260) -- Obsolete informational reference (is this intentional?): RFC 8312 (Obsoleted by RFC 9438) Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TCPM L. Xu 3 Internet-Draft UNL 4 Obsoletes: 8312 (if approved) S. Ha 5 Updates: 5681 (if approved) Colorado 6 Intended status: Standards Track I. Rhee 7 Expires: 28 April 2022 Bowery 8 V. Goel 9 Apple Inc. 10 L. Eggert, Ed. 11 NetApp 12 25 October 2021 14 CUBIC for Fast and Long-Distance Networks 15 draft-ietf-tcpm-rfc8312bis-05 17 Abstract 19 CUBIC is a standard TCP congestion control algorithm that uses a 20 cubic function instead of the linear window increase function on the 21 sender side to improve scalability and stability over fast and long- 22 distance networks. CUBIC has been adopted as the default TCP 23 congestion control algorithm by the Linux, Windows, and Apple stacks. 25 This document updates the specification of CUBIC to include 26 algorithmic improvements based on these implementations and recent 27 academic work. Based on the extensive deployment experience with 28 CUBIC, it also moves the specification to the Standards Track, 29 obsoleting RFC 8312. This also requires updating RFC 5681, to allow 30 for CUBIC's occasionally more aggressive sending behavior. 32 Note to Readers 34 Discussion of this draft takes place on the TCPM working group 35 mailing list (mailto:tcpm@ietf.org), which is archived at 36 https://mailarchive.ietf.org/arch/browse/tcpm/. 38 Working Group information can be found at 39 https://datatracker.ietf.org/wg/tcpm/; source code and issues list 40 for this draft can be found at https://github.com/NTAP/rfc8312bis. 42 Note to the RFC Editor 44 xml2rfc currently renders in the XML by surrounding the 45 corresponding text with underscores. This is highly distracting; 46 please manually remove the underscores when doing the final edits to 47 the text version of this document. 49 (There is an issue open against xml2rfc to stop doing this in the 50 future: https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/596) 52 Also, please manually change "Figure" to "Equation" for all artwork 53 with anchors beginning with "eq" - xml2rfc doesn't seem to be able to 54 do this. 56 Status of This Memo 58 This Internet-Draft is submitted in full conformance with the 59 provisions of BCP 78 and BCP 79. 61 Internet-Drafts are working documents of the Internet Engineering 62 Task Force (IETF). Note that other groups may also distribute 63 working documents as Internet-Drafts. The list of current Internet- 64 Drafts is at https://datatracker.ietf.org/drafts/current/. 66 Internet-Drafts are draft documents valid for a maximum of six months 67 and may be updated, replaced, or obsoleted by other documents at any 68 time. It is inappropriate to use Internet-Drafts as reference 69 material or to cite them other than as "work in progress." 71 This Internet-Draft will expire on 28 April 2022. 73 Copyright Notice 75 Copyright (c) 2021 IETF Trust and the persons identified as the 76 document authors. All rights reserved. 78 This document is subject to BCP 78 and the IETF Trust's Legal 79 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 80 license-info) in effect on the date of publication of this document. 81 Please review these documents carefully, as they describe your rights 82 and restrictions with respect to this document. Code Components 83 extracted from this document must include Simplified BSD License text 84 as described in Section 4.e of the Trust Legal Provisions and are 85 provided without warranty as described in the Simplified BSD License. 87 Table of Contents 89 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 90 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 5 91 3. Design Principles of CUBIC . . . . . . . . . . . . . . . . . 5 92 3.1. Principle 1 for the CUBIC Increase Function . . . . . . . 5 93 3.2. Principle 2 for Reno-Friendliness . . . . . . . . . . . . 6 94 3.3. Principle 3 for RTT Fairness . . . . . . . . . . . . . . 7 95 3.4. Principle 4 for the CUBIC Decrease Factor . . . . . . . . 7 96 4. CUBIC Congestion Control . . . . . . . . . . . . . . . . . . 8 97 4.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 8 98 4.1.1. Constants of Interest . . . . . . . . . . . . . . . . 8 99 4.1.2. Variables of Interest . . . . . . . . . . . . . . . . 8 100 4.2. Window Increase Function . . . . . . . . . . . . . . . . 9 101 4.3. Reno-Friendly Region . . . . . . . . . . . . . . . . . . 11 102 4.4. Concave Region . . . . . . . . . . . . . . . . . . . . . 13 103 4.5. Convex Region . . . . . . . . . . . . . . . . . . . . . . 13 104 4.6. Multiplicative Decrease . . . . . . . . . . . . . . . . . 14 105 4.7. Fast Convergence . . . . . . . . . . . . . . . . . . . . 15 106 4.8. Timeout . . . . . . . . . . . . . . . . . . . . . . . . . 15 107 4.9. Spurious Congestion Events . . . . . . . . . . . . . . . 16 108 4.10. Slow Start . . . . . . . . . . . . . . . . . . . . . . . 17 109 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 17 110 5.1. Fairness to Reno . . . . . . . . . . . . . . . . . . . . 18 111 5.2. Using Spare Capacity . . . . . . . . . . . . . . . . . . 20 112 5.3. Difficult Environments . . . . . . . . . . . . . . . . . 21 113 5.4. Investigating a Range of Environments . . . . . . . . . . 21 114 5.5. Protection against Congestion Collapse . . . . . . . . . 22 115 5.6. Fairness within the Alternative Congestion Control 116 Algorithm . . . . . . . . . . . . . . . . . . . . . . . 22 117 5.7. Performance with Misbehaving Nodes and Outside 118 Attackers . . . . . . . . . . . . . . . . . . . . . . . 22 119 5.8. Behavior for Application-Limited Flows . . . . . . . . . 22 120 5.9. Responses to Sudden or Transient Events . . . . . . . . . 22 121 5.10. Incremental Deployment . . . . . . . . . . . . . . . . . 23 122 6. Security Considerations . . . . . . . . . . . . . . . . . . . 23 123 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 124 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 125 8.1. Normative References . . . . . . . . . . . . . . . . . . 23 126 8.2. Informative References . . . . . . . . . . . . . . . . . 25 127 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 27 128 Appendix B. Evolution of CUBIC . . . . . . . . . . . . . . . . . 28 129 B.1. Since draft-ietf-tcpm-rfc8312bis-04 . . . . . . . . . . . 28 130 B.2. Since draft-ietf-tcpm-rfc8312bis-03 . . . . . . . . . . . 29 131 B.3. Since draft-ietf-tcpm-rfc8312bis-02 . . . . . . . . . . . 29 132 B.4. Since draft-ietf-tcpm-rfc8312bis-01 . . . . . . . . . . . 29 133 B.5. Since draft-ietf-tcpm-rfc8312bis-00 . . . . . . . . . . . 30 134 B.6. Since draft-eggert-tcpm-rfc8312bis-03 . . . . . . . . . . 30 135 B.7. Since draft-eggert-tcpm-rfc8312bis-02 . . . . . . . . . . 30 136 B.8. Since draft-eggert-tcpm-rfc8312bis-01 . . . . . . . . . . 30 137 B.9. Since draft-eggert-tcpm-rfc8312bis-00 . . . . . . . . . . 30 138 B.10. Since RFC8312 . . . . . . . . . . . . . . . . . . . . . . 31 139 B.11. Since the Original Paper . . . . . . . . . . . . . . . . 31 140 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32 142 1. Introduction 144 CUBIC has been adopted as the default TCP congestion control 145 algorithm in the Linux, Windows, and Apple stacks, and has been used 146 and deployed globally. Extensive, decade-long deployment experience 147 in vastly different Internet scenarios has convincingly demonstrated 148 that CUBIC is safe for deployment on the global Internet and delivers 149 substantial benefits over classical Reno congestion control 150 [RFC5681]. It is therefore to be regarded as the currently most 151 widely deployed standard for TCP congestion control. CUBIC can also 152 be used for other transport protocols such as QUIC [RFC9000] and SCTP 153 [RFC4960] as a default congestion controller. 155 The design of CUBIC was motivated by the well-documented problem 156 classical Reno TCP has with low utilization over fast and long- 157 distance networks [K03][RFC3649]. This problem arises from a slow 158 increase of the congestion window following a congestion event in a 159 network with a large bandwidth-delay product (BDP). [HKLRX06] 160 indicates that this problem is frequently observed even in the range 161 of congestion window sizes over several hundreds of packets. This 162 problem is equally applicable to all Reno-style standards and their 163 variants, including TCP-Reno [RFC5681], TCP-NewReno 164 [RFC6582][RFC6675], SCTP [RFC4960], TFRC [RFC5348], and QUIC 165 congestion control [RFC9002], which use the same linear increase 166 function for window growth. We refer to all Reno-style standards and 167 their variants collectively as "Reno" below. 169 CUBIC, originally proposed in [HRX08], is a modification to the 170 congestion control algorithm of classical Reno to remedy this 171 problem. Specifically, CUBIC uses a cubic function instead of the 172 linear window increase function of Reno to improve scalability and 173 stability under fast and long-distance networks. 175 This document updates the specification of CUBIC to include 176 algorithmic improvements based on the Linux, Windows, and Apple 177 implementations and recent academic work. Based on the extensive 178 deployment experience with CUBIC, it also moves the specification to 179 the Standards Track, obsoleting [RFC8312]. This requires an update 180 to [RFC5681], which limits the aggressiveness of Reno TCP 181 implementations in its Section 3. Since CUBIC is occasionally more 182 aggressive than the [RFC5681] algorithms, this document updates 183 [RFC5681] to allow for CUBIC's behavior. 185 Binary Increase Congestion Control (BIC-TCP) [XHR04], a predecessor 186 of CUBIC, was selected as the default TCP congestion control 187 algorithm by Linux in the year 2005 and had been used for several 188 years by the Internet community at large. 190 CUBIC uses a similar window increase function as BIC-TCP and is 191 designed to be less aggressive and fairer to Reno in bandwidth usage 192 than BIC-TCP while maintaining the strengths of BIC-TCP such as 193 stability, window scalability, and round-trip time (RTT) fairness. 195 In the following sections, we first briefly explain the design 196 principles of CUBIC, then provide the exact specification of CUBIC, 197 and finally discuss the safety features of CUBIC following the 198 guidelines specified in [RFC5033]. 200 2. Conventions 202 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 203 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 204 "OPTIONAL" in this document are to be interpreted as described in 205 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 206 capitals, as shown here. 208 3. Design Principles of CUBIC 210 CUBIC is designed according to the following design principles: 212 Principle 1: For better network utilization and stability, CUBIC 213 uses both the concave and convex profiles of a cubic function to 214 increase the congestion window size, instead of using just a 215 convex function. 217 Principle 2: To be Reno-friendly, CUBIC is designed to behave like 218 Reno in networks with short RTTs and small bandwidth where Reno 219 performs well. 221 Principle 3: For RTT-fairness, CUBIC is designed to achieve linear 222 bandwidth sharing among flows with different RTTs. 224 Principle 4: CUBIC appropriately sets its multiplicative window 225 decrease factor in order to balance between the scalability and 226 convergence speed. 228 3.1. Principle 1 for the CUBIC Increase Function 230 For better network utilization and stability, CUBIC [HRX08] uses a 231 cubic window increase function in terms of the elapsed time from the 232 last congestion event. While most alternative congestion control 233 algorithms to Reno increase the congestion window using convex 234 functions, CUBIC uses both the concave and convex profiles of a cubic 235 function for window growth. 237 After a window reduction in response to a congestion event detected 238 by duplicate ACKs, Explicit Congestion Notification-Echo (ECN-Echo, 239 ECE) ACKs [RFC3168], TCP RACK [RFC8985] or QUIC loss detection 240 [RFC9002], CUBIC remembers the congestion window size at which it 241 received the congestion event and performs a multiplicative decrease 242 of the congestion window. When CUBIC enters into congestion 243 avoidance, it starts to increase the congestion window using the 244 concave profile of the cubic function. The cubic function is set to 245 have its plateau at the remembered congestion window size, so that 246 the concave window increase continues until then. After that, the 247 cubic function turns into a convex profile and the convex window 248 increase begins. 250 This style of window adjustment (concave and then convex) improves 251 the algorithm stability while maintaining high network utilization 252 [CEHRX07]. This is because the window size remains almost constant, 253 forming a plateau around the remembered congestion window size of the 254 last congestion event, where network utilization is deemed highest. 255 Under steady state, most window size samples of CUBIC are close to 256 that remembered congestion window size, thus promoting high network 257 utilization and stability. 259 Note that congestion control algorithms that only use convex 260 functions to increase the congestion window size have their maximum 261 increments around the remembered congestion window size of the last 262 congestion event, and thus introduce many packet bursts around the 263 saturation point of the network, likely causing frequent global loss 264 synchronizations. 266 3.2. Principle 2 for Reno-Friendliness 268 CUBIC promotes per-flow fairness to Reno. Note that Reno performs 269 well over paths with short RTTs and small bandwidths (or small BDPs). 270 There is only a scalability problem in networks with long RTTs and 271 large bandwidths (or large BDPs). 273 A congestion control algorithm designed to be friendly to Reno on a 274 per-flow basis must increase its congestion window less aggressively 275 in small BDP networks than in large BDP networks. 277 The aggressiveness of CUBIC mainly depends on the maximum window size 278 before a window reduction, which is smaller in small-BDP networks 279 than in large-BDP networks. Thus, CUBIC increases its congestion 280 window less aggressively in small-BDP networks than in large-BDP 281 networks. 283 Furthermore, in cases when the cubic function of CUBIC would increase 284 the congestion window less aggressively than Reno, CUBIC simply 285 follows the window size of Reno to ensure that CUBIC achieves at 286 least the same throughput as Reno in small-BDP networks. We call 287 this region where CUBIC behaves like Reno the "Reno-friendly region". 289 3.3. Principle 3 for RTT Fairness 291 Two CUBIC flows with different RTTs have a throughput ratio that is 292 linearly proportional to the inverse of their RTT ratio, where the 293 throughput of a flow is approximately the size of its congestion 294 window divided by its RTT. 296 Specifically, CUBIC maintains a window increase rate independent of 297 RTTs outside the Reno-friendly region, and thus flows with different 298 RTTs have similar congestion window sizes under steady state when 299 they operate outside the Reno-friendly region. 301 This notion of a linear throughput ratio is similar to that of Reno 302 under high statistical multiplexing where packet loss is independent 303 of individual flow rates. However, under low statistical 304 multiplexing, the throughput ratio of Reno flows with different RTTs 305 is quadratically proportional to the inverse of their RTT ratio 306 [XHR04]. 308 CUBIC always ensures a linear throughput ratio independent of the 309 amount of statistical multiplexing. This is an improvement over 310 Reno. While there is no consensus on particular throughput ratios 311 for different RTT flows, we believe that over wired Internet paths, 312 use of a linear throughput ratio seems more reasonable than equal 313 throughputs (i.e., the same throughput for flows with different RTTs) 314 or a higher-order throughput ratio (e.g., a quadratical throughput 315 ratio of Reno under low statistical multiplexing environments). 317 3.4. Principle 4 for the CUBIC Decrease Factor 319 To balance between scalability and convergence speed, CUBIC sets the 320 multiplicative window decrease factor to 0.7, whereas Reno uses 0.5. 322 While this improves the scalability of CUBIC, a side effect of this 323 decision is slower convergence, especially under low statistical 324 multiplexing. This design choice is following the observation that 325 HighSpeed TCP (HSTCP) [RFC3649] and other approaches (e.g., [GV02]) 326 made: the current Internet becomes more asynchronous with less 327 frequent loss synchronizations under high statistical multiplexing. 329 In such environments, even strict Multiplicative-Increase 330 Multiplicative-Decrease (MIMD) can converge. CUBIC flows with the 331 same RTT always converge to the same throughput independent of 332 statistical multiplexing, thus achieving intra-algorithm fairness. 333 We also find that in environments with sufficient statistical 334 multiplexing, the convergence speed of CUBIC is reasonable. 336 4. CUBIC Congestion Control 338 In this section, we discuss how the congestion window is updated 339 during the different stages of the CUBIC congestion controller. 341 4.1. Definitions 343 The unit of all window sizes in this document is segments of the 344 maximum segment size (MSS), and the unit of all times is seconds. 345 Implementations can use bytes to express window sizes, which would 346 require factoring in the maximum segment size wherever necessary and 347 replacing _segments_acked_ with the number of bytes acknowledged in 348 Figure 4. 350 4.1.1. Constants of Interest 352 β__cubic_: CUBIC multiplicative decrease factor as described in 353 Section 4.6. 355 α__cubic_: CUBIC additive increase factor used in Reno-friendly 356 region as described in Section 4.3. 358 _C_: constant that determines the aggressiveness of CUBIC in 359 competing with other congestion control algorithms in high BDP 360 networks. Please see Section 5 for more explanation on how it is 361 set. The unit for _C_ is 363 segment 364 ------- 365 3 366 second 368 4.1.2. Variables of Interest 370 This section defines the variables required to implement CUBIC: 372 _RTT_: Smoothed round-trip time in seconds, calculated as described 373 in [RFC6298]. 375 _cwnd_: Current congestion window in segments. 377 _ssthresh_: Current slow start threshold in segments. 379 _W_max_: Size of _cwnd_ in segments just before _cwnd_ was reduced in 380 the last congestion event when fast convergence is disabled. 381 However, if fast convergence is enabled, the size may be further 382 reduced based on the current saturation point. 384 _K_: The time period in seconds it takes to increase the congestion 385 window size at the beginning of the current congestion avoidance 386 stage to _W_max_. 388 _current_time_: Current time of the system in seconds. 390 _epoch_start_: The time in seconds at which the current congestion 391 avoidance stage started. 393 _cwnd_start_: The _cwnd_ at the beginning of the current congestion 394 avoidance stage, i.e., at time _epoch_start_. 396 W_cubic(_t_): The congestion window in segments at time _t_ in 397 seconds based on the cubic increase function, as described in 398 Section 4.2. 400 _target_: Target value of congestion window in segments after the 401 next RTT, that is, W_cubic(_t_ + _RTT_), as described in Section 4.2. 403 _W_est_: An estimate for the congestion window in segments in the 404 Reno-friendly region, that is, an estimate for the congestion window 405 of Reno. 407 _segments_acked_: Number of MSS-sized segments acked when a "new ACK" 408 is received, i.e., an ACK that cumulatively acknowledges the delivery 409 of new data. This number will be a decimal value when a new ACK 410 acknowledges an amount of data that is not MSS-sized. Specifically, 411 it can be less than 1 when a new ACK acknowledges a segment smaller 412 than the MSS. 414 4.2. Window Increase Function 416 CUBIC maintains the acknowledgment (ACK) clocking of Reno by 417 increasing the congestion window only at the reception of a new ACK. 418 It does not make any changes to the TCP Fast Recovery and Fast 419 Retransmit algorithms [RFC6582][RFC6675]. 421 During congestion avoidance, after a congestion event is detected by 422 mechanisms described in Section 3.1, CUBIC changes the window 423 increase function of Reno. 425 CUBIC uses the following window increase function: 427 3 428 W (t) = C * (t - K) + W 429 cubic max 431 Figure 1 433 where _t_ is the elapsed time in seconds from the beginning of the 434 current congestion avoidance stage, that is, 436 t = current_time - epoch 437 start 439 and where _epoch_start_ is the time at which the current congestion 440 avoidance stage starts. _K_ is the time period that the above 441 function takes to increase the congestion window size at the 442 beginning of the current congestion avoidance stage to _W_max_ if 443 there are no further congestion events and is calculated using the 444 following equation: 446 ________________ 447 /W - cwnd 448 3 / max start 449 K = | / ---------------- 450 |/ C 452 Figure 2 454 where _cwnd_start_ is the congestion window at the beginning of the 455 current congestion avoidance stage. 457 Upon receiving a new ACK during congestion avoidance, CUBIC computes 458 the _target_ congestion window size after the next _RTT_ using 459 Figure 1 as follows, where _RTT_ is the smoothed round-trip time. 460 The lower and upper bounds below ensure that CUBIC's congestion 461 window increase rate is non-decreasing and is less than the increase 462 rate of slow start. 464 / 465 | if W (t + RTT) < cwnd 466 |cwnd cubic 467 | 468 | 469 | 470 target = < if W (t + RTT) > 1.5 * cwnd 471 |1.5 * cwnd cubic 472 | 473 | 474 |W (t + RTT) 475 | cubic otherwise 476 \ 478 The elapsed time _t_ in Figure 1 MUST NOT include periods during 479 which _cwnd_ has not been updated due to an application limit (see 480 Section 5.8). 482 Depending on the value of the current congestion window size _cwnd_, 483 CUBIC runs in three different regions: 485 1. The Reno-friendly region, which ensures that CUBIC achieves at 486 least the same throughput as Reno. 488 2. The concave region, if CUBIC is not in the Reno-friendly region 489 and _cwnd_ is less than _W_max_. 491 3. The convex region, if CUBIC is not in the Reno-friendly region 492 and _cwnd_ is greater than _W_max_. 494 Below, we describe the exact actions taken by CUBIC in each region. 496 4.3. Reno-Friendly Region 498 Reno performs well in certain types of networks, for example, under 499 short RTTs and small bandwidths (or small BDPs). In these networks, 500 CUBIC remains in the Reno-friendly region to achieve at least the 501 same throughput as Reno. 503 The Reno-friendly region is designed according to the analysis in 504 [FHP00], which studies the performance of an AIMD algorithm with an 505 additive factor of α (segments per _RTT_) and a multiplicative factor 506 of β, denoted by AIMD(α, β). _p_ is the packet loss rate. 507 Specifically, the average congestion window size of AIMD(α, β) can be 508 calculated using Figure 3. 510 _______________ 511 / α * (1 + β) 512 AVG_AIMD(α, β) = | / --------------- 513 |/ 2 * (1 - β) * p 515 Figure 3 517 By the same analysis, to achieve the same average window size as Reno 518 that uses AIMD(1, 0.5), α must be equal to, 520 1 - β 521 3 * ----- 522 1 + β 524 Thus, CUBIC uses Figure 4 to estimate the window size _W_est_ in the 525 Reno-friendly region with 527 1 - β 528 cubic 529 α = 3 * ---------- 530 cubic 1 + β 531 cubic 533 which achieves the same average window size as Reno. When receiving 534 a new ACK in congestion avoidance (where _cwnd_ could be greater than 535 or less than _W_max_), CUBIC checks whether W_cubic(_t_) is less than 536 _W_est_. If so, CUBIC is in the Reno-friendly region and _cwnd_ 537 SHOULD be set to _W_est_ at each reception of a new ACK. 539 _W_est_ is set equal to _cwnd_start_ at the start of the congestion 540 avoidance stage. After that, on every new ACK, _W_est_ is updated 541 using Figure 4. Note that this equation is for a connection where 542 Appropriate Byte Counting (ABC) [RFC3465] is disabled. For a 543 connection with ABC enabled, this equation SHOULD be adjusted by 544 using the number of acknowledged bytes instead of acknowledged 545 segments. Also note that this equation works for connections with 546 enabled or disabled Delayed ACKs [RFC5681], as _segments_acked_ will 547 be different based on the segments actually acknowledged by a new 548 ACK. 550 segments_acked 551 W = W + α * -------------- 552 est est cubic cwnd 554 Figure 4 556 Note that once _W_est_ reaches _W_max_, that is, _W_est_ >= _W_max_, 557 CUBIC needs to start probing to determine the new value of _W_max_. 558 At this point, α__cubic_ SHOULD be set to 1 to ensure that CUBIC can 559 achieve the same congestion window increment as Reno, which uses 560 AIMD(1, 0.5). 562 4.4. Concave Region 564 When receiving a new ACK in congestion avoidance, if CUBIC is not in 565 the Reno-friendly region and _cwnd_ is less than _W_max_, then CUBIC 566 is in the concave region. In this region, _cwnd_ MUST be incremented 567 by 569 target - cwnd 570 ------------- 571 cwnd 573 for each received new ACK, where _target_ is calculated as described 574 in Section 4.2. 576 4.5. Convex Region 578 When receiving a new ACK in congestion avoidance, if CUBIC is not in 579 the Reno-friendly region and _cwnd_ is larger than or equal to 580 _W_max_, then CUBIC is in the convex region. 582 The convex region indicates that the network conditions might have 583 changed since the last congestion event, possibly implying more 584 available bandwidth after some flow departures. Since the Internet 585 is highly asynchronous, some amount of perturbation is always 586 possible without causing a major change in available bandwidth. 588 Unless it is overridden by the AIMD window increase, CUBIC is very 589 careful in this region. The convex profile aims to increase the 590 window very slowly at the beginning when _cwnd_ is around _W_max_ and 591 then gradually increases its rate of increase. We also call this 592 region the "maximum probing phase", since CUBIC is searching for a 593 new _W_max_. In this region, _cwnd_ MUST be incremented by 595 target - cwnd 596 ------------- 597 cwnd 599 for each received new ACK, where _target_ is calculated as described 600 in Section 4.2. 602 4.6. Multiplicative Decrease 604 When a congestion event is detected by mechanisms described in 605 Section 3.1, CUBIC updates _W_max_ and reduces _cwnd_ and _ssthresh_ 606 immediately as described below. In case of packet loss, the sender 607 MUST reduce _cwnd_ and _ssthresh_ immediately upon entering loss 608 recovery, similar to [RFC5681] (and [RFC6675]). Note that other 609 mechanisms, such as Proportional Rate Reduction [RFC6937], can be 610 used to reduce the sending rate during loss recovery more gradually. 611 The parameter β__cubic_ SHOULD be set to 0.7, which is different from 612 the multiplicative decrease factor used in [RFC5681] (and [RFC6675]) 613 during fast recovery. 615 In Figure 5, _flight_size_ is the amount of outstanding data in the 616 network, as defined in [RFC5681]. Note that a rate-limited 617 application with idle periods or periods when unable to send at the 618 full rate permitted by _cwnd_ may easily encounter notable variations 619 in the volume of data sent from one RTT to another, resulting in 620 _flight_size_ that is significantly less than _cwnd_ on a congestion 621 event. This may decrease _cwnd_ to a much lower value than 622 necessary. To avoid suboptimal performance with such applications, 623 some implementations of CUBIC use _cwnd_ instead of _flight_size_ to 624 calculate the new _ssthresh_ in Figure 5. Alternatively, the 625 mechanisms described in [RFC7661] may also be adopted to mitigate 626 this issue. 628 flight_size * β // new ssthresh 629 ssthresh = cubic 631 /max(ssthresh, 2) // reduction on packet loss, cwnd is at least 2 MSS 632 | 633 cwnd = < 634 |max(ssthresh, 1) // reduction on ECE, cwnd is at least 1 MSS 635 \ 637 max(ssthresh, 2) // ssthresh is at least 2 MSS 638 ssthresh = 640 Figure 5 642 A side effect of setting β__cubic_ to a value bigger than 0.5 is 643 slower convergence. We believe that while a more adaptive setting of 644 β__cubic_ could result in faster convergence, it will make the 645 analysis of CUBIC much harder. 647 Note that CUBIC will continue to reduce _cwnd_ in response to 648 congestion events due to ECN-Echo ACKs until it reaches a value of 1 649 MSS. If congestion persists, a sender with a _cwnd_ of 1 MSS needs 650 to reduce its sending rate even further. It can achieve that by 651 using a retransmission timer with exponential backoff, as described 652 in [RFC3168]. 654 4.7. Fast Convergence 656 To improve convergence speed, CUBIC uses a heuristic. When a new 657 flow joins the network, existing flows need to give up some of their 658 bandwidth to allow the new flow some room for growth, if the existing 659 flows have been using all the network bandwidth. To speed up this 660 bandwidth release by existing flows, the following "Fast Convergence" 661 mechanism SHOULD be implemented. 663 With Fast Convergence, when a congestion event occurs, we update 664 _W_max_ as follows, before the window reduction as described in 665 Section 4.6. 667 / 668 | 1 + β 669 | cubic if cwnd < W and fast convergence is enabled, 670 |cwnd * ---------- max 671 | 2 672 W = < 673 max | further reduce W 674 | max 675 | 676 | otherwise, remember cwnd before reduction 677 \cwnd 679 At a congestion event, if the current _cwnd_ is less than _W_max_, 680 this indicates that the saturation point experienced by this flow is 681 getting reduced because of a change in available bandwidth. Then we 682 allow this flow to release more bandwidth by reducing _W_max_ 683 further. This action effectively lengthens the time for this flow to 684 increase its congestion window, because the reduced _W_max_ forces 685 the flow to plateau earlier. This allows more time for the new flow 686 to catch up to its congestion window size. 688 Fast Convergence is designed for network environments with multiple 689 CUBIC flows. In network environments with only a single CUBIC flow 690 and without any other traffic, Fast Convergence SHOULD be disabled. 692 4.8. Timeout 694 In case of a timeout, CUBIC follows Reno to reduce _cwnd_ [RFC5681], 695 but sets _ssthresh_ using β__cubic_ (same as in Section 4.6) in a way 696 that is different from Reno TCP [RFC5681]. 698 During the first congestion avoidance stage after a timeout, CUBIC 699 increases its congestion window size using Figure 1, where _t_ is the 700 elapsed time since the beginning of the current congestion avoidance, 701 _K_ is set to 0, and _W_max_ is set to the congestion window size at 702 the beginning of the current congestion avoidance stage. In 703 addition, for the Reno-friendly region, _W_est_ SHOULD be set to the 704 congestion window size at the beginning of the current congestion 705 avoidance. 707 4.9. Spurious Congestion Events 709 In cases where CUBIC reduces its congestion window in response to 710 having detected packet loss via duplicate ACKs or timeouts, there is 711 a possibility that the missing ACK would arrive after the congestion 712 window reduction and a corresponding packet retransmission. For 713 example, packet reordering could trigger this behavior. A high 714 degree of packet reordering could cause multiple congestion window 715 reduction events, where spurious losses are incorrectly interpreted 716 as congestion signals, thus degrading CUBIC's performance 717 significantly. 719 When there is a congestion event, a CUBIC implementation SHOULD save 720 the current value of the following variables before the congestion 721 window reduction. 723 prior_cwnd = cwnd 725 prior_ssthresh = ssthresh 727 prior_W = W 728 max max 730 prior_K = K 732 prior_epoch = epoch 733 start start 735 prior_W_{est} = W 736 est 738 CUBIC MAY implement an algorithm to detect spurious retransmissions, 739 such as Forward RTO-Recovery [RFC5682]. Experimental alternatives 740 include DSACK [RFC3708] and Eifel [RFC3522]. Once a spurious 741 congestion event is detected, CUBIC SHOULD restore the original 742 values of above-mentioned variables as follows if the current _cwnd_ 743 is lower than _prior_cwnd_. Restoring the original values ensures 744 that CUBIC's performance is similar to what it would be without 745 spurious losses. 747 \ 748 cwnd = prior_cwnd | 749 | 750 ssthresh = prior_ssthresh | 751 | 752 W = prior_W | 753 max max | 754 >if cwnd < prior_cwnd 755 K = prior_K | 756 | 757 epoch = prior_epoch | 758 start start| 759 | 760 W = prior_W | 761 est est / 763 In rare cases, when the detection happens long after a spurious loss 764 event and the current _cwnd_ is already higher than _prior_cwnd_, 765 CUBIC SHOULD continue to use the current and the most recent values 766 of these variables. 768 4.10. Slow Start 770 CUBIC MUST employ a slow-start algorithm, when _cwnd_ is no more than 771 _ssthresh_. In general, CUBIC SHOULD use the HyStart++ slow start 772 algorithm [I-D.ietf-tcpm-hystartplusplus], or MAY use the Reno TCP 773 slow start algorithm [RFC5681] in the rare cases when HyStart++ is 774 not suitable. Experimental alternatives include hybrid slow start 775 [HR08], a predecessor to HyStart++ that some CUBIC implementations 776 have used as the default for the last decade, and limited slow start 777 [RFC3742]. Whichever start-up algorithm is used, work might be 778 needed to ensure that the end of slow start and the first 779 multiplicative decrease of congestion avoidance work well together. 781 When CUBIC uses HyStart++ [I-D.ietf-tcpm-hystartplusplus], it may 782 exit the first slow start without incurring any packet loss and thus 783 _W_max_ is undefined. In this special case, CUBIC switches to 784 congestion avoidance and increases its congestion window size using 785 Figure 1, where _t_ is the elapsed time since the beginning of the 786 current congestion avoidance, _K_ is set to 0, and _W_max_ is set to 787 the congestion window size at the beginning of the current congestion 788 avoidance stage. 790 5. Discussion 792 In this section, we further discuss the safety features of CUBIC 793 following the guidelines specified in [RFC5033]. 795 With a deterministic loss model where the number of packets between 796 two successive packet losses is always _1/p_, CUBIC always operates 797 with the concave window profile, which greatly simplifies the 798 performance analysis of CUBIC. The average window size of CUBIC can 799 be obtained by the following function: 801 ________________ ____ 802 /C * (3 + β ) 4 / 3 803 4 / cubic |/ RTT 804 AVG_W = | / ---------------- * ------- 805 cubic | / 4 * (1 - β ) __ 806 |/ cubic 4 / 3 807 |/ p 809 Figure 6 811 With β__cubic_ set to 0.7, the above formula reduces to: 813 ____ 814 _______ 4 / 3 815 4 /C * 3.7 |/ RTT 816 AVG_W = | / ------- * ------- 817 cubic |/ 1.2 __ 818 4 / 3 819 |/ p 821 Figure 7 823 We will determine the value of _C_ in the following subsection using 824 Figure 7. 826 5.1. Fairness to Reno 828 In environments where Reno is able to make reasonable use of the 829 available bandwidth, CUBIC does not significantly change this state. 831 Reno performs well in the following two types of networks: 833 1. networks with a small bandwidth-delay product (BDP) 835 2. networks with a short RTTs, but not necessarily a small BDP 837 CUBIC is designed to behave very similarly to Reno in the above two 838 types of networks. The following two tables show the average window 839 sizes of Reno TCP, HSTCP, and CUBIC TCP. The average window sizes of 840 Reno TCP and HSTCP are from [RFC3649]. The average window size of 841 CUBIC is calculated using Figure 7 and the CUBIC Reno-friendly region 842 for three different values of _C_. 844 +=============+=======+========+================+=========+========+ 845 | Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | 846 | | | | | (C=0.4) | (C=4) | 847 +=============+=======+========+================+=========+========+ 848 | 1.0e-02 | 12 | 12 | 12 | 12 | 12 | 849 +-------------+-------+--------+----------------+---------+--------+ 850 | 1.0e-03 | 38 | 38 | 38 | 38 | 59 | 851 +-------------+-------+--------+----------------+---------+--------+ 852 | 1.0e-04 | 120 | 263 | 120 | 187 | 333 | 853 +-------------+-------+--------+----------------+---------+--------+ 854 | 1.0e-05 | 379 | 1795 | 593 | 1054 | 1874 | 855 +-------------+-------+--------+----------------+---------+--------+ 856 | 1.0e-06 | 1200 | 12280 | 3332 | 5926 | 10538 | 857 +-------------+-------+--------+----------------+---------+--------+ 858 | 1.0e-07 | 3795 | 83981 | 18740 | 33325 | 59261 | 859 +-------------+-------+--------+----------------+---------+--------+ 860 | 1.0e-08 | 12000 | 574356 | 105383 | 187400 | 333250 | 861 +-------------+-------+--------+----------------+---------+--------+ 863 Table 1: Reno TCP, HSTCP, and CUBIC with RTT = 0.1 seconds 865 Table 1 describes the response function of Reno TCP, HSTCP, and CUBIC 866 in networks with _RTT_ = 0.1 seconds. The average window size is in 867 MSS-sized segments. 869 +=============+=======+========+================+=========+=======+ 870 | Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | 871 | | | | | (C=0.4) | (C=4) | 872 +=============+=======+========+================+=========+=======+ 873 | 1.0e-02 | 12 | 12 | 12 | 12 | 12 | 874 +-------------+-------+--------+----------------+---------+-------+ 875 | 1.0e-03 | 38 | 38 | 38 | 38 | 38 | 876 +-------------+-------+--------+----------------+---------+-------+ 877 | 1.0e-04 | 120 | 263 | 120 | 120 | 120 | 878 +-------------+-------+--------+----------------+---------+-------+ 879 | 1.0e-05 | 379 | 1795 | 379 | 379 | 379 | 880 +-------------+-------+--------+----------------+---------+-------+ 881 | 1.0e-06 | 1200 | 12280 | 1200 | 1200 | 1874 | 882 +-------------+-------+--------+----------------+---------+-------+ 883 | 1.0e-07 | 3795 | 83981 | 3795 | 5926 | 10538 | 884 +-------------+-------+--------+----------------+---------+-------+ 885 | 1.0e-08 | 12000 | 574356 | 18740 | 33325 | 59261 | 886 +-------------+-------+--------+----------------+---------+-------+ 888 Table 2: Reno TCP, HSTCP, and CUBIC with RTT = 0.01 seconds 890 Table 2 describes the response function of Reno TCP, HSTCP, and CUBIC 891 in networks with _RTT_ = 0.01 seconds. The average window size is in 892 MSS-sized segments. 894 Both tables show that CUBIC with any of these three _C_ values is 895 more friendly to Reno TCP than HSTCP, especially in networks with a 896 short _RTT_ where Reno TCP performs reasonably well. For example, in 897 a network with _RTT_ = 0.01 seconds and p=10^-6, Reno TCP has an 898 average window of 1200 packets. If the packet size is 1500 bytes, 899 then Reno TCP can achieve an average rate of 1.44 Gbps. In this 900 case, CUBIC with _C_=0.04 or _C_=0.4 achieves exactly the same rate 901 as Reno TCP, whereas HSTCP is about ten times more aggressive than 902 Reno TCP. 904 We can see that _C_ determines the aggressiveness of CUBIC in 905 competing with other congestion control algorithms for bandwidth. 906 CUBIC is more friendly to Reno TCP, if the value of _C_ is lower. 907 However, we do not recommend setting _C_ to a very low value like 908 0.04, since CUBIC with a low _C_ cannot efficiently use the bandwidth 909 in fast and long-distance networks. Based on these observations and 910 extensive deployment experience, we find _C_=0.4 gives a good balance 911 between Reno-friendliness and aggressiveness of window increase. 912 Therefore, _C_ SHOULD be set to 0.4. With _C_ set to 0.4, Figure 7 913 is reduced to: 915 ____ 916 4 / 3 917 |/ RTT 918 AVG_W = 1.054 * ------- 919 cubic __ 920 4 / 3 921 |/ p 923 Figure 8 925 Figure 8 is then used in the next subsection to show the scalability 926 of CUBIC. 928 5.2. Using Spare Capacity 930 CUBIC uses a more aggressive window increase function than Reno for 931 fast and long-distance networks. 933 The following table shows that to achieve the 10 Gbps rate, Reno TCP 934 requires a packet loss rate of 2.0e-10, while CUBIC TCP requires a 935 packet loss rate of 2.9e-8. 937 +===================+===========+=========+=========+=========+ 938 | Throughput (Mbps) | Average W | Reno P | HSTCP P | CUBIC P | 939 +===================+===========+=========+=========+=========+ 940 | 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 | 941 +-------------------+-----------+---------+---------+---------+ 942 | 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 | 943 +-------------------+-----------+---------+---------+---------+ 944 | 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 | 945 +-------------------+-----------+---------+---------+---------+ 946 | 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 | 947 +-------------------+-----------+---------+---------+---------+ 948 | 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 | 949 +-------------------+-----------+---------+---------+---------+ 951 Table 3: Required packet loss rate for Reno TCP, HSTCP, and 952 CUBIC to achieve a certain throughput 954 Table 3 describes the required packet loss rate for Reno TCP, HSTCP, 955 and CUBIC to achieve a certain throughput. We use 1500-byte packets 956 and an _RTT_ of 0.1 seconds. 958 Our test results in [HKLRX06] indicate that CUBIC uses the spare 959 bandwidth left unused by existing Reno TCP flows in the same 960 bottleneck link without taking away much bandwidth from the existing 961 flows. 963 5.3. Difficult Environments 965 CUBIC is designed to remedy the poor performance of Reno in fast and 966 long-distance networks. 968 5.4. Investigating a Range of Environments 970 There is decade-long deployment experience with CUBIC on the 971 Internet. CUBIC has also been extensively studied by using both NS-2 972 simulation and testbed experiments, covering a wide range of network 973 environments. More information can be found in [HKLRX06]. 975 Same as Reno, CUBIC is a loss-based congestion control algorithm. 976 Because CUBIC is designed to be more aggressive (due to a faster 977 window increase function and bigger multiplicative decrease factor) 978 than Reno in fast and long-distance networks, it can fill large drop- 979 tail buffers more quickly than Reno and increases the risk of a 980 standing queue [RFC8511]. In this case, proper queue sizing and 981 management [RFC7567] could be used to mitigate the risk to some 982 extent and reduce the packet queuing delay. Also, in large-BDP 983 networks after a congestion event, CUBIC, due its cubic window 984 increase function, recovers quickly to the highest link utilization 985 point. This means that link utilization is less sensitive to an 986 active queue management (AQM) target that is lower than the amplitude 987 of the whole sawtooth. 989 Similar to Reno, the performance of CUBIC as a loss-based congestion 990 control algorithm suffers in networks where a packet loss is not a 991 good indication of bandwidth utilization, such as wireless or mobile 992 networks [LIU16]. 994 5.5. Protection against Congestion Collapse 996 With regard to the potential of causing congestion collapse, CUBIC 997 behaves like Reno, since CUBIC modifies only the window adjustment 998 algorithm of Reno. Thus, it does not modify the ACK clocking and 999 timeout behaviors of Reno. 1001 CUBIC also satisfies the "full backoff" requirement as described in 1002 [RFC5033]. After reducing the sending rate to one packet per RTT in 1003 response to congestion events due to ECN-Echo ACKs, CUBIC then 1004 exponentially increases the transmission timer for each packet 1005 retransmission while congestion persists. 1007 5.6. Fairness within the Alternative Congestion Control Algorithm 1009 CUBIC ensures convergence of competing CUBIC flows with the same RTT 1010 in the same bottleneck links to an equal throughput. When competing 1011 flows have different RTT values, their throughput ratio is linearly 1012 proportional to the inverse of their RTT ratios. This is true 1013 independently of the level of statistical multiplexing on the link. 1015 5.7. Performance with Misbehaving Nodes and Outside Attackers 1017 This is not considered in the current CUBIC design. 1019 5.8. Behavior for Application-Limited Flows 1021 CUBIC does not increase its congestion window size if a flow is 1022 currently limited by the application instead of the congestion 1023 window. Section 4.2 requires that _t_ in Figure 1 does not include 1024 application-limited periods, such as idle periods, otherwise 1025 W_cubic(_t_) might be very high after restarting from these periods. 1027 5.9. Responses to Sudden or Transient Events 1029 If there is a sudden increase in capacity, e.g., due to variable 1030 radio capacity, a routing change, or a mobility event, CUBIC is 1031 designed to utilize the newly available capacity faster than Reno. 1033 On the other hand, if there is a sudden decrease in capacity, CUBIC 1034 reduces more slowly than Reno. This remains true whether or not 1035 CUBIC is in Reno-friendly mode and whether or not fast convergence is 1036 enabled. 1038 5.10. Incremental Deployment 1040 CUBIC requires only changes to the congestion control at the sender, 1041 and it does not require any changes at receivers. That is, a CUBIC 1042 sender works correctly with Reno receivers. In addition, CUBIC does 1043 not require any changes to routers and does not require any 1044 assistance from routers. 1046 6. Security Considerations 1048 CUBIC makes no changes to the underlying security of TCP. More 1049 information about TCP security concerns can be found in [RFC5681]. 1051 7. IANA Considerations 1053 This document does not require any IANA actions. 1055 8. References 1057 8.1. Normative References 1059 [I-D.ietf-tcpm-hystartplusplus] 1060 Balasubramanian, P., Huang, Y., and M. Olson, "HyStart++: 1061 Modified Slow Start for TCP", Work in Progress, Internet- 1062 Draft, draft-ietf-tcpm-hystartplusplus-03, 25 July 2021, 1063 . 1066 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1067 Requirement Levels", BCP 14, RFC 2119, 1068 DOI 10.17487/RFC2119, March 1997, 1069 . 1071 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1072 of Explicit Congestion Notification (ECN) to IP", 1073 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1074 . 1076 [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion 1077 Control Algorithms", BCP 133, RFC 5033, 1078 DOI 10.17487/RFC5033, August 2007, 1079 . 1081 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1082 Friendly Rate Control (TFRC): Protocol Specification", 1083 RFC 5348, DOI 10.17487/RFC5348, September 2008, 1084 . 1086 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1087 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1088 . 1090 [RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata, 1091 "Forward RTO-Recovery (F-RTO): An Algorithm for Detecting 1092 Spurious Retransmission Timeouts with TCP", RFC 5682, 1093 DOI 10.17487/RFC5682, September 2009, 1094 . 1096 [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, 1097 "Computing TCP's Retransmission Timer", RFC 6298, 1098 DOI 10.17487/RFC6298, June 2011, 1099 . 1101 [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The 1102 NewReno Modification to TCP's Fast Recovery Algorithm", 1103 RFC 6582, DOI 10.17487/RFC6582, April 2012, 1104 . 1106 [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., 1107 and Y. Nishida, "A Conservative Loss Recovery Algorithm 1108 Based on Selective Acknowledgment (SACK) for TCP", 1109 RFC 6675, DOI 10.17487/RFC6675, August 2012, 1110 . 1112 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 1113 Recommendations Regarding Active Queue Management", 1114 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 1115 . 1117 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1118 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1119 May 2017, . 1121 [RFC8985] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "The 1122 RACK-TLP Loss Detection Algorithm for TCP", RFC 8985, 1123 DOI 10.17487/RFC8985, February 2021, 1124 . 1126 [RFC9002] Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection 1127 and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, 1128 May 2021, . 1130 8.2. Informative References 1132 [CEHRX07] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic 1133 Ordering for Internet Congestion Control and its 1134 Applications", IEEE INFOCOM 2007 - 26th IEEE International 1135 Conference on Computer Communications, 1136 DOI 10.1109/infcom.2007.111, 2007, 1137 . 1139 [FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of 1140 Equation-Based and AIMD Congestion Control", May 2000, 1141 . 1143 [GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary 1144 Adjustment Algorithms", Technical Report TR2002-29, 1145 Department of Computer Sciences, The University of 1146 Texas at Austin, 11 August 2002, 1147 . 1149 [HKLRX06] Ha, S., Kim, Y., Le, L., Rhee, I., and L. Xu, "A Step 1150 toward Realistic Performance Evaluation of High-Speed TCP 1151 Variants", International Workshop on Protocols for Fast 1152 Long-Distance Networks, February 2006, 1153 . 1155 [HR08] Ha, S. and I. Rhee, "Hybrid Slow Start for High-Bandwidth 1156 and Long-Distance Networks", International Workshop 1157 on Protocols for Fast Long-Distance Networks, March 2008, 1158 . 1161 [HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: a new TCP-friendly 1162 high-speed TCP variant", ACM SIGOPS Operating Systems 1163 Review Vol. 42, pp. 64-74, DOI 10.1145/1400097.1400105, 1164 July 2008, . 1166 [K03] Kelly, T., "Scalable TCP: improving performance in 1167 highspeed wide area networks", ACM SIGCOMM Computer 1168 Communication Review Vol. 33, pp. 83-91, 1169 DOI 10.1145/956981.956989, April 2003, 1170 . 1172 [LIU16] Liu, K. and J. Lee, "On Improving TCP Performance over 1173 Mobile Data Networks", IEEE Transactions on Mobile 1174 Computing Vol. 15, pp. 2522-2536, 1175 DOI 10.1109/tmc.2015.2500227, October 2016, 1176 . 1178 [RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte 1179 Counting (ABC)", RFC 3465, DOI 10.17487/RFC3465, February 1180 2003, . 1182 [RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm 1183 for TCP", RFC 3522, DOI 10.17487/RFC3522, April 2003, 1184 . 1186 [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", 1187 RFC 3649, DOI 10.17487/RFC3649, December 2003, 1188 . 1190 [RFC3708] Blanton, E. and M. Allman, "Using TCP Duplicate Selective 1191 Acknowledgement (DSACKs) and Stream Control Transmission 1192 Protocol (SCTP) Duplicate Transmission Sequence Numbers 1193 (TSNs) to Detect Spurious Retransmissions", RFC 3708, 1194 DOI 10.17487/RFC3708, February 2004, 1195 . 1197 [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large 1198 Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March 1199 2004, . 1201 [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", 1202 RFC 4960, DOI 10.17487/RFC4960, September 2007, 1203 . 1205 [RFC6937] Mathis, M., Dukkipati, N., and Y. Cheng, "Proportional 1206 Rate Reduction for TCP", RFC 6937, DOI 10.17487/RFC6937, 1207 May 2013, . 1209 [RFC7661] Fairhurst, G., Sathiaseelan, A., and R. Secchi, "Updating 1210 TCP to Support Rate-Limited Traffic", RFC 7661, 1211 DOI 10.17487/RFC7661, October 2015, 1212 . 1214 [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and 1215 R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", 1216 RFC 8312, DOI 10.17487/RFC8312, February 2018, 1217 . 1219 [RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, 1220 "TCP Alternative Backoff with ECN (ABE)", RFC 8511, 1221 DOI 10.17487/RFC8511, December 2018, 1222 . 1224 [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 1225 Multiplexed and Secure Transport", RFC 9000, 1226 DOI 10.17487/RFC9000, May 2021, 1227 . 1229 [SXEZ19] Sun, W., Xu, L., Elbaum, S., and D. Zhao, "Model-Agnostic 1230 and Efficient Exploration of Numerical State Space of 1231 Real-World TCP Congestion Control Implementations", USENIX 1232 NSDI 2019, February 2019, 1233 . 1235 [XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary Increase 1236 Congestion Control (BIC) for Fast Long-Distance Networks", 1237 IEEE INFOCOM 2004, DOI 10.1109/infcom.2004.1354672, March 1238 2004, . 1240 Appendix A. Acknowledgments 1242 Richard Scheffenegger and Alexander Zimmermann originally co-authored 1243 [RFC8312]. 1245 These individuals suggested improvements to this document: 1247 * Bob Briscoe 1249 * Christian Huitema 1251 * Gorry Fairhurst 1253 * Jonathan Morton 1255 * Juhamatti Kuusisaari 1257 * Junho Choi 1259 * Markku Kojo 1261 * Martin Thomson 1263 * Matt Olson 1265 * Michael Welzl 1267 * Mirja Kuehlewind 1269 * Mohit P. Tahiliani 1271 * Neal Cardwell 1272 * Praveen Balasubramanian 1274 * Richard Scheffenegger 1276 * Rod Grimes 1278 * Tom Henderson 1280 * Tom Petch 1282 * Wesley Rosenblum 1284 * Yoshifumi Nishida 1286 * Yuchung Cheng 1288 Appendix B. Evolution of CUBIC 1290 B.1. Since draft-ietf-tcpm-rfc8312bis-04 1292 * Fix incorrect math (#106 (https://github.com/NTAP/rfc8312bis/ 1293 issues/106)) 1295 * Update RFC5681 (#99 (https://github.com/NTAP/rfc8312bis/ 1296 issues/99)) 1298 * Rephrase text around algorithmic alternatives, add HyStart++ (#85 1299 (https://github.com/NTAP/rfc8312bis/issues/85), #86 1300 (https://github.com/NTAP/rfc8312bis/issues/86), #90 1301 (https://github.com/NTAP/rfc8312bis/issues/90)) 1303 * Clarify what we mean by "new ACK" and use it in the text in more 1304 places. (#101 (https://github.com/NTAP/rfc8312bis/issues/101)) 1306 * Rewrite the Responses to Sudden or Transient Events section (#98 1307 (https://github.com/NTAP/rfc8312bis/issues/98)) 1309 * Remove confusing text about _cwnd_start_ in Section 4.2 (#100 1310 (https://github.com/NTAP/rfc8312bis/issues/100)) 1312 * Change terminology from "AIMD" to "Reno" (#108 1313 (https://github.com/NTAP/rfc8312bis/issues/108)) 1315 * Moved MUST NOT from app-limited section to main cubic AI section 1316 (#97 (https://github.com/NTAP/rfc8312bis/issues/97)) 1318 * Clarify cwnd decrease during multiplicative decrease (#102 1319 (https://github.com/NTAP/rfc8312bis/issues/102)) 1321 * Clarify text around queuing and slow adaptation of CUBIC in 1322 wireless environments (#94 (https://github.com/NTAP/rfc8312bis/ 1323 issues/94)) 1325 * Set lower bound of cwnd to 1 MSS and use retransmit timer 1326 thereafter (#83 (https://github.com/NTAP/rfc8312bis/issues/83)) 1328 * Use FlightSize instead of cwnd to update ssthresh (#114 1329 (https://github.com/NTAP/rfc8312bis/issues/114)) 1331 B.2. Since draft-ietf-tcpm-rfc8312bis-03 1333 * Remove reference from abstract (#82 1334 (https://github.com/NTAP/rfc8312bis/pull/82)) 1336 B.3. Since draft-ietf-tcpm-rfc8312bis-02 1338 * Description of packet loss rate _p_ (#65 1339 (https://github.com/NTAP/rfc8312bis/issues/65)) 1341 * Clarification of TCP Friendly Equation for ABC and Delayed ACK 1342 (#66 (https://github.com/NTAP/rfc8312bis/issues/66)) 1344 * add applicability to QUIC and SCTP (#61 1345 (https://github.com/NTAP/rfc8312bis/issues/61)) 1347 * clarity on setting alpha__aimd_ to 1 (#68 1348 (https://github.com/NTAP/rfc8312bis/issues/68)) 1350 * introduce alpha__cubic_ (#64 (https://github.com/NTAP/rfc8312bis/ 1351 issues/64)) 1353 * clarify _cwnd_ growth in convex region (#69 1354 (https://github.com/NTAP/rfc8312bis/issues/69)) 1356 * add guidance for using bytes and mention that segments count is 1357 decimal (#67 (https://github.com/NTAP/rfc8312bis/issues/67)) 1359 * add loss events detected by RACK and QUIC loss detection (#62 1360 (https://github.com/NTAP/rfc8312bis/issues/62)) 1362 B.4. Since draft-ietf-tcpm-rfc8312bis-01 1364 * address Michael Scharf's editorial suggestions. (#59 1365 (https://github.com/NTAP/rfc8312bis/issues/59)) 1367 * add "Note to the RFC Editor" about removing underscores 1369 B.5. Since draft-ietf-tcpm-rfc8312bis-00 1371 * use updated xml2rfc with better text rendering of subscripts 1373 B.6. Since draft-eggert-tcpm-rfc8312bis-03 1375 * fix spelling nits 1377 * rename to draft-ietf 1379 * define _W_max_ more clearly 1381 B.7. Since draft-eggert-tcpm-rfc8312bis-02 1383 * add definition for segments_acked and alpha__aimd_. (#47 1384 (https://github.com/NTAP/rfc8312bis/issues/47)) 1386 * fix a mistake in _W_max_ calculation in the fast convergence 1387 section. (#51 (https://github.com/NTAP/rfc8312bis/issues/51)) 1389 * clarity on setting _ssthresh_ and _cwnd_start_ during 1390 multiplicative decrease. (#53 (https://github.com/NTAP/rfc8312bis/ 1391 issues/53)) 1393 B.8. Since draft-eggert-tcpm-rfc8312bis-01 1395 * rename TCP-Friendly to AIMD-Friendly and rename Standard TCP to 1396 AIMD TCP to avoid confusion as CUBIC has been widely used on the 1397 Internet. (#38 (https://github.com/NTAP/rfc8312bis/issues/38)) 1399 * change introductory text to reflect the significant broader 1400 deployment of CUBIC on the Internet. (#39 1401 (https://github.com/NTAP/rfc8312bis/issues/39)) 1403 * rephrase introduction to avoid referring to variables that have 1404 not been defined yet. 1406 B.9. Since draft-eggert-tcpm-rfc8312bis-00 1408 * acknowledge former co-authors (#15 1409 (https://github.com/NTAP/rfc8312bis/issues/15)) 1411 * prevent _cwnd_ from becoming less than two (#7 1412 (https://github.com/NTAP/rfc8312bis/issues/7)) 1414 * add list of variables and constants (#5 1415 (https://github.com/NTAP/rfc8312bis/issues/5), #6 1416 (https://github.com/NTAP/rfc8312bis/issues/6)) 1418 * update _K_'s definition and add bounds for CUBIC _target_ _cwnd_ 1419 [SXEZ19] (#1 (https://github.com/NTAP/rfc8312bis/issues/1), #14 1420 (https://github.com/NTAP/rfc8312bis/issues/14)) 1422 * update _W_est_ to use AIMD approach (#20 1423 (https://github.com/NTAP/rfc8312bis/issues/20)) 1425 * set alpha__aimd_ to 1 once _W_est_ reaches _W_max_ (#2 1426 (https://github.com/NTAP/rfc8312bis/issues/2)) 1428 * add Vidhi as co-author (#17 (https://github.com/NTAP/rfc8312bis/ 1429 issues/17)) 1431 * note for Fast Recovery during _cwnd_ decrease due to congestion 1432 event (#11 (https://github.com/NTAP/rfc8312bis/issues/11)) 1434 * add section for spurious congestion events (#23 1435 (https://github.com/NTAP/rfc8312bis/issues/23)) 1437 * initialize _W_est_ after timeout and remove variable 1438 _W_(last_max)_ (#28 (https://github.com/NTAP/rfc8312bis/ 1439 issues/28)) 1441 B.10. Since RFC8312 1443 * converted to Markdown and xml2rfc v3 1445 * updated references (as part of the conversion) 1447 * updated author information 1449 * various formatting changes 1451 * move to Standards Track 1453 B.11. Since the Original Paper 1455 CUBIC has gone through a few changes since the initial release 1456 [HRX08] of its algorithm and implementation. Below we highlight the 1457 differences between its original paper and [RFC8312]. 1459 * The original paper [HRX08] includes the pseudocode of CUBIC 1460 implementation using Linux's pluggable congestion control 1461 framework, which excludes system-specific optimizations. The 1462 simplified pseudocode might be a good source to start with and 1463 understand CUBIC. 1465 * [HRX08] also includes experimental results showing its performance 1466 and fairness. 1468 * The definition of beta__cubic_ constant was changed in [RFC8312]. 1469 For example, beta__cubic_ in the original paper was the window 1470 decrease constant while [RFC8312] changed it to CUBIC 1471 multiplication decrease factor. With this change, the current 1472 congestion window size after a congestion event in [RFC8312] was 1473 beta__cubic_ * _W_max_ while it was (1-beta__cubic_) * _W_max_ in 1474 the original paper. 1476 * Its pseudocode used _W_(last_max)_ while [RFC8312] used _W_max_. 1478 * Its AIMD-friendly window was _W_tcp_ while [RFC8312] used _W_est_. 1480 Authors' Addresses 1482 Lisong Xu 1483 University of Nebraska-Lincoln 1484 Department of Computer Science and Engineering 1485 Lincoln, NE 68588-0115 1486 United States of America 1488 Email: xu@unl.edu 1489 URI: https://cse.unl.edu/~xu/ 1491 Sangtae Ha 1492 University of Colorado at Boulder 1493 Department of Computer Science 1494 Boulder, CO 80309-0430 1495 United States of America 1497 Email: sangtae.ha@colorado.edu 1498 URI: https://netstech.org/sangtaeha/ 1500 Injong Rhee 1501 Bowery Farming 1502 151 W 26TH Street, 12TH Floor 1503 New York, NY 10001 1504 United States of America 1506 Email: injongrhee@gmail.com 1507 Vidhi Goel 1508 Apple Inc. 1509 One Apple Park Way 1510 Cupertino, California 95014 1511 United States of America 1513 Email: vidhi_goel@apple.com 1515 Lars Eggert (editor) 1516 NetApp 1517 Stenbergintie 12 B 1518 FI-02700 Kauniainen 1519 Finland 1521 Email: lars@eggert.org 1522 URI: https://eggert.org/