idnits 2.17.1 draft-eggert-tcpm-rfc8312bis-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8312]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (15 November 2020) is 1255 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 4960 (Obsoleted by RFC 9260) -- Obsolete informational reference (is this intentional?): RFC 8312 (Obsoleted by RFC 9438) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TCPM L. Xu 3 Internet-Draft UNL 4 Obsoletes: 8312 (if approved) S. Ha 5 Intended status: Standards Track Colorado 6 Expires: 19 May 2021 L. Eggert, Ed. 7 NetApp 8 15 November 2020 10 CUBIC for Fast Long-Distance Networks 11 draft-eggert-tcpm-rfc8312bis-00 13 Abstract 15 CUBIC is an extension to the current TCP standards. It differs from 16 the current TCP standards only in the congestion control algorithm on 17 the sender side. In particular, it uses a cubic function instead of 18 a linear window increase function of the current TCP standards to 19 improve scalability and stability under fast and long-distance 20 networks. CUBIC and its predecessor algorithm have been adopted as 21 defaults by Linux and have been used for many years. This document 22 provides a specification of CUBIC to enable third-party 23 implementations and to solicit community feedback through 24 experimentation on the performance of CUBIC. 26 This documents obsoletes [RFC8312], updating the specification of 27 CUBIC to conform to the current Linux version. 29 Note to Readers 31 Discussion of this draft takes place on the TCPM working group 32 mailing list (mailto:tcpm@ietf.org), which is archived at 33 https://mailarchive.ietf.org/arch/browse/tcpm/. 35 Working Group information can be found at 36 https://datatracker.ietf.org/wg/tcpm/; source code and issues list 37 for this draft can be found at https://github.com/NTAP/rfc8312bis. 39 Status of This Memo 41 This Internet-Draft is submitted in full conformance with the 42 provisions of BCP 78 and BCP 79. 44 Internet-Drafts are working documents of the Internet Engineering 45 Task Force (IETF). Note that other groups may also distribute 46 working documents as Internet-Drafts. The list of current Internet- 47 Drafts is at https://datatracker.ietf.org/drafts/current/. 49 Internet-Drafts are draft documents valid for a maximum of six months 50 and may be updated, replaced, or obsoleted by other documents at any 51 time. It is inappropriate to use Internet-Drafts as reference 52 material or to cite them other than as "work in progress." 54 This Internet-Draft will expire on 19 May 2021. 56 Copyright Notice 58 Copyright (c) 2020 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 63 license-info) in effect on the date of publication of this document. 64 Please review these documents carefully, as they describe your rights 65 and restrictions with respect to this document. Code Components 66 extracted from this document must include Simplified BSD License text 67 as described in Section 4.e of the Trust Legal Provisions and are 68 provided without warranty as described in the Simplified BSD License. 70 Table of Contents 72 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 73 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 4 74 3. Design Principles of CUBIC . . . . . . . . . . . . . . . . . 4 75 4. CUBIC Congestion Control . . . . . . . . . . . . . . . . . . 6 76 4.1. Window Increase Function . . . . . . . . . . . . . . . . 7 77 4.2. TCP-Friendly Region . . . . . . . . . . . . . . . . . . . 8 78 4.3. Concave Region . . . . . . . . . . . . . . . . . . . . . 8 79 4.4. Convex Region . . . . . . . . . . . . . . . . . . . . . . 9 80 4.5. Multiplicative Decrease . . . . . . . . . . . . . . . . . 9 81 4.6. Fast Convergence . . . . . . . . . . . . . . . . . . . . 9 82 4.7. Timeout . . . . . . . . . . . . . . . . . . . . . . . . . 10 83 4.8. Slow Start . . . . . . . . . . . . . . . . . . . . . . . 10 84 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 11 85 5.1. Fairness to Standard TCP . . . . . . . . . . . . . . . . 11 86 5.2. Using Spare Capacity . . . . . . . . . . . . . . . . . . 13 87 5.3. Difficult Environments . . . . . . . . . . . . . . . . . 14 88 5.4. Investigating a Range of Environments . . . . . . . . . . 14 89 5.5. Protection against Congestion Collapse . . . . . . . . . 14 90 5.6. Fairness within the Alternative Congestion Control 91 Algorithm . . . . . . . . . . . . . . . . . . . . . . . 14 92 5.7. Performance with Misbehaving Nodes and Outside 93 Attackers . . . . . . . . . . . . . . . . . . . . . . . 14 94 5.8. Behavior for Application-Limited Flows . . . . . . . . . 15 95 5.9. Responses to Sudden or Transient Events . . . . . . . . . 15 96 5.10. Incremental Deployment . . . . . . . . . . . . . . . . . 15 98 6. Security Considerations . . . . . . . . . . . . . . . . . . . 15 99 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 100 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 15 101 8.1. Normative References . . . . . . . . . . . . . . . . . . 15 102 8.2. Informative References . . . . . . . . . . . . . . . . . 16 103 Appendix A. Changes from RFC8312 . . . . . . . . . . . . . . . . 18 104 A.1. Since RFC8312 . . . . . . . . . . . . . . . . . . . . . . 18 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 107 1. Introduction 109 The low utilization problem of TCP in fast long-distance networks is 110 well documented in [K03] and [RFC3649]. This problem arises from a 111 slow increase of the congestion window following a congestion event 112 in a network with a large bandwidth-delay product (BDP). [HKLRX06] 113 indicates that this problem is frequently observed even in the range 114 of congestion window sizes over several hundreds of packets. This 115 problem is equally applicable to all Reno-style TCP standards and 116 their variants, including TCP-Reno [RFC5681], TCP-NewReno 117 [RFC6582][RFC6675], SCTP [RFC4960], and TFRC [RFC5348], which use the 118 same linear increase function for window growth, which we refer to 119 collectively as "Standard TCP" below. 121 CUBIC, originally proposed in [HRX08], is a modification to the 122 congestion control algorithm of Standard TCP to remedy this problem. 123 This document describes the most recent specification of CUBIC. 124 Specifically, CUBIC uses a cubic function instead of a linear window 125 increase function of Standard TCP to improve scalability and 126 stability under fast and long-distance networks. 128 Binary Increase Congestion Control (BIC-TCP) [XHR04], a predecessor 129 of CUBIC, was selected as the default TCP congestion control 130 algorithm by Linux in the year 2005 and has been used for several 131 years by the Internet community at large. CUBIC uses a similar 132 window increase function as BIC-TCP and is designed to be less 133 aggressive and fairer to Standard TCP in bandwidth usage than BIC-TCP 134 while maintaining the strengths of BIC-TCP such as stability, window 135 scalability, and RTT fairness. CUBIC has already replaced BIC-TCP as 136 the default TCP congestion control algorithm in Linux and has been 137 deployed globally by Linux. Through extensive testing in various 138 Internet scenarios, we believe that CUBIC is safe for testing and 139 deployment in the global Internet. 141 In the following sections, we first briefly explain the design 142 principles of CUBIC, then provide the exact specification of CUBIC, 143 and finally discuss the safety features of CUBIC following the 144 guidelines specified in [RFC5033]. 146 2. Conventions 148 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 149 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 150 "OPTIONAL" in this document are to be interpreted as described in 151 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 152 capitals, as shown here. 154 3. Design Principles of CUBIC 156 CUBIC is designed according to the following design principles: 158 Principle 1: For better network utilization and stability, CUBIC 159 uses both the concave and convex profiles of a cubic function to 160 increase the congestion window size, instead of using just a 161 convex function. 163 Principle 2: To be TCP-friendly, CUBIC is designed to behave like 164 Standard TCP in networks with short RTTs and small bandwidth where 165 Standard TCP performs well. 167 Principle 3: For RTT-fairness, CUBIC is designed to achieve linear 168 bandwidth sharing among flows with different RTTs. 170 Principle 4: CUBIC appropriately sets its multiplicative window 171 decrease factor in order to balance between the scalability and 172 convergence speed. 174 Principle 1: For better network utilization and stability, CUBIC 175 [HRX08] uses a cubic window increase function in terms of the elapsed 176 time from the last congestion event. While most alternative 177 congestion control algorithms to Standard TCP increase the congestion 178 window using convex functions, CUBIC uses both the concave and convex 179 profiles of a cubic function for window growth. After a window 180 reduction in response to a congestion event is detected by duplicate 181 ACKs or Explicit Congestion Notification-Echo (ECN-Echo) ACKs 182 [RFC3168], CUBIC registers the congestion window size where it got 183 the congestion event as W_max and performs a multiplicative decrease 184 of congestion window. After it enters into congestion avoidance, it 185 starts to increase the congestion window using the concave profile of 186 the cubic function. The cubic function is set to have its plateau at 187 W_max so that the concave window increase continues until the window 188 size becomes W_max. After that, the cubic function turns into a 189 convex profile and the convex window increase begins. This style of 190 window adjustment (concave and then convex) improves the algorithm 191 stability while maintaining high network utilization [CEHRX07]. This 192 is because the window size remains almost constant, forming a plateau 193 around W_max where network utilization is deemed highest. Under 194 steady state, most window size samples of CUBIC are close to W_max, 195 thus promoting high network utilization and stability. Note that 196 those congestion control algorithms using only convex functions to 197 increase the congestion window size have the maximum increments 198 around W_max, and thus introduce a large number of packet bursts 199 around the saturation point of the network, likely causing frequent 200 global loss synchronizations. 202 Principle 2: CUBIC promotes per-flow fairness to Standard TCP. Note 203 that Standard TCP performs well under short RTT and small bandwidth 204 (or small BDP) networks. There is only a scalability problem in 205 networks with long RTTs and large bandwidth (or large BDP). An 206 alternative congestion control algorithm to Standard TCP designed to 207 be friendly to Standard TCP on a per-flow basis must operate to 208 increase its congestion window less aggressively in small BDP 209 networks than in large BDP networks. The aggressiveness of CUBIC 210 mainly depends on the maximum window size before a window reduction, 211 which is smaller in small BDP networks than in large BDP networks. 212 Thus, CUBIC increases its congestion window less aggressively in 213 small BDP networks than in large BDP networks. Furthermore, in cases 214 when the cubic function of CUBIC increases its congestion window less 215 aggressively than Standard TCP, CUBIC simply follows the window size 216 of Standard TCP to ensure that CUBIC achieves at least the same 217 throughput as Standard TCP in small BDP networks. We call this 218 region where CUBIC behaves like Standard TCP, the "TCP-friendly 219 region". 221 Principle 3: Two CUBIC flows with different RTTs have their 222 throughput ratio linearly proportional to the inverse of their RTT 223 ratio, where the throughput of a flow is approximately the size of 224 its congestion window divided by its RTT. Specifically, CUBIC 225 maintains a window increase rate independent of RTTs outside of the 226 TCP-friendly region, and thus flows with different RTTs have similar 227 congestion window sizes under steady state when they operate outside 228 the TCP-friendly region. This notion of a linear throughput ratio is 229 similar to that of Standard TCP under high statistical multiplexing 230 environments where packet losses are independent of individual flow 231 rates. However, under low statistical multiplexing environments, the 232 throughput ratio of Standard TCP flows with different RTTs is 233 quadratically proportional to the inverse of their RTT ratio [XHR04]. 234 CUBIC always ensures the linear throughput ratio independent of the 235 levels of statistical multiplexing. This is an improvement over 236 Standard TCP. While there is no consensus on particular throughput 237 ratios of different RTT flows, we believe that under wired Internet, 238 use of a linear throughput ratio seems more reasonable than equal 239 throughputs (i.e., the same throughput for flows with different RTTs) 240 or a higher-order throughput ratio (e.g., a quadratical throughput 241 ratio of Standard TCP under low statistical multiplexing 242 environments). 244 Principle 4: To balance between the scalability and convergence 245 speed, CUBIC sets the multiplicative window decrease factor to 0.7 246 while Standard TCP uses 0.5. While this improves the scalability of 247 CUBIC, a side effect of this decision is slower convergence, 248 especially under low statistical multiplexing environments. This 249 design choice is following the observation that the author of 250 HighSpeed TCP (HSTCP) [RFC3649] has made along with other researchers 251 (e.g., [GV02]): the current Internet becomes more asynchronous with 252 less frequent loss synchronizations with high statistical 253 multiplexing. Under this environment, even strict Multiplicative- 254 Increase Multiplicative-Decrease (MIMD) can converge. CUBIC flows 255 with the same RTT always converge to the same throughput independent 256 of statistical multiplexing, thus achieving intra-algorithm fairness. 257 We also find that under the environments with sufficient statistical 258 multiplexing, the convergence speed of CUBIC flows is reasonable. 260 4. CUBIC Congestion Control 262 The unit of all window sizes in this document is segments of the 263 maximum segment size (MSS), and the unit of all times is seconds. 264 Let cwnd denote the congestion window size of a flow, and ssthresh 265 denote the slow-start threshold. 267 4.1. Window Increase Function 269 CUBIC maintains the acknowledgment (ACK) clocking of Standard TCP by 270 increasing the congestion window only at the reception of an ACK. It 271 does not make any change to the fast recovery and retransmit of TCP, 272 such as TCP-NewReno [RFC6582][RFC6675]. During congestion avoidance 273 after a congestion event where a packet loss is detected by duplicate 274 ACKs or a network congestion is detected by ACKs with ECN-Echo flags 275 [RFC3168], CUBIC changes the window increase function of Standard 276 TCP. Suppose that W_max is the window size just before the window is 277 reduced in the last congestion event. 279 CUBIC uses the following window increase function: 281 W_cubic(t) = C * (t - K)^3 + W_max (Eq. 1) 283 where C is a constant fixed to determine the aggressiveness of window 284 increase in high BDP networks, t is the elapsed time from the 285 beginning of the current congestion avoidance, and K is the time 286 period that the above function takes to increase the current window 287 size to W_max if there are no further congestion events and is 288 calculated using the following equation: 290 K = cubic_root(W_max * (1 - beta_cubic) / C) (Eq. 2) 292 where beta_cubic is the CUBIC multiplication decrease factor, that 293 is, when a congestion event is detected, CUBIC reduces its cwnd to 294 W_cubic(0)=W_max*beta_cubic. We discuss how we set beta_cubic in 295 Section 4.5 and how we set C in Section 5. 297 Upon receiving an ACK during congestion avoidance, CUBIC computes the 298 window increase rate during the next RTT period using Eq. 1. It sets 299 W_cubic(t+RTT) as the candidate target value of the congestion 300 window, where RTT is the weighted average RTT calculated by Standard 301 TCP. 303 Depending on the value of the current congestion window size cwnd, 304 CUBIC runs in three different modes. 306 1. The TCP-friendly region, which ensures that CUBIC achieves at 307 least the same throughput as Standard TCP. 309 2. The concave region, if CUBIC is not in the TCP-friendly region 310 and cwnd is less than W_max. 312 3. The convex region, if CUBIC is not in the TCP-friendly region and 313 cwnd is greater than W_max. 315 Below, we describe the exact actions taken by CUBIC in each region. 317 4.2. TCP-Friendly Region 319 Standard TCP performs well in certain types of networks, for example, 320 under short RTT and small bandwidth (or small BDP) networks. In 321 these networks, we use the TCP-friendly region to ensure that CUBIC 322 achieves at least the same throughput as Standard TCP. 324 The TCP-friendly region is designed according to the analysis 325 described in [FHP00]. The analysis studies the performance of an 326 Additive Increase and Multiplicative Decrease (AIMD) algorithm with 327 an additive factor of alpha_aimd (segments per RTT) and a 328 multiplicative factor of beta_aimd, denoted by AIMD(alpha_aimd, 329 beta_aimd). Specifically, the average congestion window size of 330 AIMD(alpha_aimd, beta_aimd) can be calculated using Eq. 3. The 331 analysis shows that AIMD(alpha_aimd, beta_aimd) with 332 alpha_aimd=3*(1-beta_aimd)/(1+beta_aimd) achieves the same average 333 window size as Standard TCP that uses AIMD(1, 0.5). 335 AVG_W_aimd = [alpha_aimd * (1 + beta_aimd) / 336 (2 * (1 - beta_aimd) * p)]^0.5 (Eq. 3) 338 Based on the above analysis, CUBIC uses Eq. 4 to estimate the window 339 size W_est of AIMD(alpha_aimd, beta_aimd) with 340 alpha_aimd=3*(1-beta_cubic)/(1+beta_cubic) and beta_aimd=beta_cubic, 341 which achieves the same average window size as Standard TCP. When 342 receiving an ACK in congestion avoidance (cwnd could be greater than 343 or less than W_max), CUBIC checks whether W_cubic(t) is less than 344 W_est(t). If so, CUBIC is in the TCP-friendly region and cwnd SHOULD 345 be set to W_est(t) at each reception of an ACK. 347 W_est(t) = W_max * beta_cubic + 348 [3 * (1 - beta_cubic) / (1 + beta_cubic)] * 349 (t / RTT) (Eq. 4) 351 4.3. Concave Region 353 When receiving an ACK in congestion avoidance, if CUBIC is not in the 354 TCP-friendly region and cwnd is less than W_max, then CUBIC is in the 355 concave region. In this region, cwnd MUST be incremented by 356 (W_cubic(t+RTT) - cwnd)/cwnd for each received ACK, where 357 W_cubic(t+RTT) is calculated using Eq. 1. 359 4.4. Convex Region 361 When receiving an ACK in congestion avoidance, if CUBIC is not in the 362 TCP-friendly region and cwnd is larger than or equal to W_max, then 363 CUBIC is in the convex region. The convex region indicates that the 364 network conditions might have been perturbed since the last 365 congestion event, possibly implying more available bandwidth after 366 some flow departures. Since the Internet is highly asynchronous, 367 some amount of perturbation is always possible without causing a 368 major change in available bandwidth. In this region, CUBIC is being 369 very careful by very slowly increasing its window size. The convex 370 profile ensures that the window increases very slowly at the 371 beginning and gradually increases its increase rate. We also call 372 this region the "maximum probing phase" since CUBIC is searching for 373 a new W_max. In this region, cwnd MUST be incremented by 374 (W_cubic(t+RTT) - cwnd)/cwnd for each received ACK, where 375 W_cubic(t+RTT) is calculated using Eq. 1. 377 4.5. Multiplicative Decrease 379 When a packet loss is detected by duplicate ACKs or a network 380 congestion is detected by ECN-Echo ACKs, CUBIC updates its W_max, 381 cwnd, and ssthresh as follows. Parameter beta_cubic SHOULD be set to 382 0.7. 384 W_max = cwnd // save window size before reduction 385 ssthresh = cwnd * beta_cubic // new slow-start threshold 386 ssthresh = max(ssthresh, 2) // threshold is at least 2 MSS 387 cwnd = cwnd * beta_cubic // window reduction 389 A side effect of setting beta_cubic to a value bigger than 0.5 is 390 slower convergence. We believe that while a more adaptive setting of 391 beta_cubic could result in faster convergence, it will make the 392 analysis of CUBIC much harder. This adaptive adjustment of 393 beta_cubic is an item for the next version of CUBIC. 395 4.6. Fast Convergence 397 To improve the convergence speed of CUBIC, we add a heuristic in 398 CUBIC. When a new flow joins the network, existing flows in the 399 network need to give up some of their bandwidth to allow the new flow 400 some room for growth if the existing flows have been using all the 401 bandwidth of the network. To speed up this bandwidth release by 402 existing flows, the following mechanism called "fast convergence" 403 SHOULD be implemented. 405 With fast convergence, when a congestion event occurs, before the 406 window reduction of the congestion window, a flow remembers the last 407 value of W_max before it updates W_max for the current congestion 408 event. Let us call the last value of W_max to be W_last_max. 410 if (W_max < W_last_max) { // should we make room for others 411 W_last_max = W_max // remember the last W_max 412 W_max = W_max * (1.0 + beta_cubic) / 2.0 // further reduce 413 } else { 414 W_last_max = W_max // remember the last W_max 415 } 417 At a congestion event, if the current value of W_max is less than 418 W_last_max, this indicates that the saturation point experienced by 419 this flow is getting reduced because of the change in available 420 bandwidth. Then we allow this flow to release more bandwidth by 421 reducing W_max further. This action effectively lengthens the time 422 for this flow to increase its congestion window because the reduced 423 W_max forces the flow to have the plateau earlier. This allows more 424 time for the new flow to catch up to its congestion window size. 426 The fast convergence is designed for network environments with 427 multiple CUBIC flows. In network environments with only a single 428 CUBIC flow and without any other traffic, the fast convergence SHOULD 429 be disabled. 431 4.7. Timeout 433 In case of timeout, CUBIC follows Standard TCP to reduce cwnd 434 [RFC5681], but sets ssthresh using beta_cubic (same as in 435 Section 4.5) that is different from Standard TCP [RFC5681]. 437 During the first congestion avoidance after a timeout, CUBIC 438 increases its congestion window size using Eq. 1, where t is the 439 elapsed time since the beginning of the current congestion avoidance, 440 K is set to 0, and W_max is set to the congestion window size at the 441 beginning of the current congestion avoidance. 443 4.8. Slow Start 445 CUBIC MUST employ a slow-start algorithm, when the cwnd is no more 446 than ssthresh. Among the slow-start algorithms, CUBIC MAY choose the 447 standard TCP slow start [RFC5681] in general networks, or the limited 448 slow start [RFC3742] or hybrid slow start [HR08] for fast and long- 449 distance networks. 451 In the case when CUBIC runs the hybrid slow start [HR08], it may exit 452 the first slow start without incurring any packet loss and thus W_max 453 is undefined. In this special case, CUBIC switches to congestion 454 avoidance and increases its congestion window size using Eq. 1, where 455 t is the elapsed time since the beginning of the current congestion 456 avoidance, K is set to 0, and W_max is set to the congestion window 457 size at the beginning of the current congestion avoidance. 459 5. Discussion 461 In this section, we further discuss the safety features of CUBIC 462 following the guidelines specified in [RFC5033]. 464 With a deterministic loss model where the number of packets between 465 two successive packet losses is always 1/p, CUBIC always operates 466 with the concave window profile, which greatly simplifies the 467 performance analysis of CUBIC. The average window size of CUBIC can 468 be obtained by the following function: 470 AVG_W_cubic = [C * (3 + beta_cubic) / 471 (4 * (1 - beta_cubic))]^0.25 * 472 (RTT^0.75) / (p^0.75) (Eq. 5) 474 With beta_cubic set to 0.7, the above formula is reduced to: 476 AVG_W_cubic = (C * 3.7 / 1.2)^0.25 * (RTT^0.75) / (p^0.75) 477 (Eq. 6) 479 We will determine the value of C in the following subsection using 480 Eq. 6. 482 5.1. Fairness to Standard TCP 484 In environments where Standard TCP is able to make reasonable use of 485 the available bandwidth, CUBIC does not significantly change this 486 state. 488 Standard TCP performs well in the following two types of networks: 490 1. networks with a small bandwidth-delay product (BDP) 492 2. networks with a short RTTs, but not necessarily a small BDP 493 CUBIC is designed to behave very similarly to Standard TCP in the 494 above two types of networks. The following two tables show the 495 average window sizes of Standard TCP, HSTCP, and CUBIC. The average 496 window sizes of Standard TCP and HSTCP are from [RFC3649]. The 497 average window size of CUBIC is calculated using Eq. 6 and the CUBIC 498 TCP-friendly region for three different values of C. 500 +--------+----------+-----------+------------+-----------+----------+ 501 | Loss | Average | Average | CUBIC | CUBIC | CUBIC | 502 | Rate P | TCP W | HSTCP W | (C=0.04) | (C=0.4) | (C=4) | 503 +--------+----------+-----------+------------+-----------+----------+ 504 | 10^-2 | 12 | 12 | 12 | 12 | 12 | 505 | 10^-3 | 38 | 38 | 38 | 38 | 59 | 506 | 10^-4 | 120 | 263 | 120 | 187 | 333 | 507 | 10^-5 | 379 | 1795 | 593 | 1054 | 1874 | 508 | 10^-6 | 1200 | 12279 | 3332 | 5926 | 10538 | 509 | 10^-7 | 3795 | 83981 | 18740 | 33325 | 59261 | 510 | 10^-8 | 12000 | 574356 | 105383 | 187400 | 333250 | 511 +--------+----------+-----------+------------+-----------+----------+ 513 Table 1 515 Table 1 describes the response function of Standard TCP, HSTCP, and 516 CUBIC in networks with RTT = 0.1 seconds. The average window size is 517 in MSS-sized segments. 519 +--------+-----------+-----------+------------+-----------+---------+ 520 | Loss | Average | Average | CUBIC | CUBIC | CUBIC | 521 | Rate P | TCP W | HSTCP W | (C=0.04) | (C=0.4) | (C=4) | 522 +--------+-----------+-----------+------------+-----------+---------+ 523 | 10^-2 | 12 | 12 | 12 | 12 | 12 | 524 | 10^-3 | 38 | 38 | 38 | 38 | 38 | 525 | 10^-4 | 120 | 263 | 120 | 120 | 120 | 526 | 10^-5 | 379 | 1795 | 379 | 379 | 379 | 527 | 10^-6 | 1200 | 12279 | 1200 | 1200 | 1874 | 528 | 10^-7 | 3795 | 83981 | 3795 | 5926 | 10538 | 529 | 10^-8 | 12000 | 574356 | 18740 | 33325 | 59261 | 530 +--------+-----------+-----------+------------+-----------+---------+ 532 Table 2 534 Table 2 describes the response function of Standard TCP, HSTCP, and 535 CUBIC in networks with RTT = 0.01 seconds. The average window size 536 is in MSS-sized segments. 538 Both tables show that CUBIC with any of these three C values is more 539 friendly to TCP than HSTCP, especially in networks with a short RTT 540 where TCP performs reasonably well. For example, in a network with 541 RTT = 0.01 seconds and p=10^-6, TCP has an average window of 1200 542 packets. If the packet size is 1500 bytes, then TCP can achieve an 543 average rate of 1.44 Gbps. In this case, CUBIC with C=0.04 or C=0.4 544 achieves exactly the same rate as Standard TCP, whereas HSTCP is 545 about ten times more aggressive than Standard TCP. 547 We can see that C determines the aggressiveness of CUBIC in competing 548 with other congestion control algorithms for bandwidth. CUBIC is 549 more friendly to Standard TCP, if the value of C is lower. However, 550 we do not recommend setting C to a very low value like 0.04, since 551 CUBIC with a low C cannot efficiently use the bandwidth in long RTT 552 and high-bandwidth networks. Based on these observations and our 553 experiments, we find C=0.4 gives a good balance between TCP- 554 friendliness and aggressiveness of window increase. Therefore, C 555 SHOULD be set to 0.4. With C set to 0.4, Eq. 6 is reduced to: 557 AVG_W_cubic = 1.054 * (RTT^0.75) / (p^0.75) (Eq. 7) 559 Eq. 7 is then used in the next subsection to show the scalability of 560 CUBIC. 562 5.2. Using Spare Capacity 564 CUBIC uses a more aggressive window increase function than Standard 565 TCP under long RTT and high-bandwidth networks. 567 The following table shows that to achieve the 10 Gbps rate, Standard 568 TCP requires a packet loss rate of 2.0e-10, while CUBIC requires a 569 packet loss rate of 2.9e-8. 571 +------------------+-----------+---------+---------+---------+ 572 | Throughput(Mbps) | Average W | TCP P | HSTCP P | CUBIC P | 573 +------------------+-----------+---------+---------+---------+ 574 | 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 | 575 | 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 | 576 | 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 | 577 | 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 | 578 | 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 | 579 +------------------+-----------+---------+---------+---------+ 581 Table 3 583 Table 3 describes the required packet loss rate for Standard TCP, 584 HSTCP, and CUBIC to achieve a certain throughput. We use 1500-byte 585 packets and an RTT of 0.1 seconds. 587 Our test results in [HKLRX06] indicate that CUBIC uses the spare 588 bandwidth left unused by existing Standard TCP flows in the same 589 bottleneck link without taking away much bandwidth from the existing 590 flows. 592 5.3. Difficult Environments 594 CUBIC is designed to remedy the poor performance of TCP in fast and 595 long-distance networks. 597 5.4. Investigating a Range of Environments 599 CUBIC has been extensively studied by using both NS-2 simulation and 600 test-bed experiments covering a wide range of network environments. 601 More information can be found in [HKLRX06]. 603 Same as Standard TCP, CUBIC is a loss-based congestion control 604 algorithm. Because CUBIC is designed to be more aggressive (due to a 605 faster window increase function and bigger multiplicative decrease 606 factor) than Standard TCP in fast and long-distance networks, it can 607 fill large drop-tail buffers more quickly than Standard TCP and 608 increase the risk of a standing queue [RFC8511]. In this case, 609 proper queue sizing and management [RFC7567] could be used to reduce 610 the packet queuing delay. 612 5.5. Protection against Congestion Collapse 614 With regard to the potential of causing congestion collapse, CUBIC 615 behaves like Standard TCP since CUBIC modifies only the window 616 adjustment algorithm of TCP. Thus, it does not modify the ACK 617 clocking and Timeout behaviors of Standard TCP. 619 5.6. Fairness within the Alternative Congestion Control Algorithm 621 CUBIC ensures convergence of competing CUBIC flows with the same RTT 622 in the same bottleneck links to an equal throughput. When competing 623 flows have different RTTs, their throughput ratio is linearly 624 proportional to the inverse of their RTT ratios. This is true 625 independent of the level of statistical multiplexing in the link. 627 5.7. Performance with Misbehaving Nodes and Outside Attackers 629 This is not considered in the current CUBIC. 631 5.8. Behavior for Application-Limited Flows 633 CUBIC does not raise its congestion window size if the flow is 634 currently limited by the application instead of the congestion 635 window. In case of long periods when cwnd has not been updated due 636 to the application rate limit, such as idle periods, t in Eq. 1 MUST 637 NOT include these periods; otherwise, W_cubic(t) might be very high 638 after restarting from these periods. 640 5.9. Responses to Sudden or Transient Events 642 If there is a sudden congestion, a routing change, or a mobility 643 event, CUBIC behaves the same as Standard TCP. 645 5.10. Incremental Deployment 647 CUBIC requires only the change of TCP senders, and it does not make 648 any changes to TCP receivers. That is, a CUBIC sender works 649 correctly with the Standard TCP receivers. In addition, CUBIC does 650 not require any changes to the routers and does not require any 651 assistance from the routers. 653 6. Security Considerations 655 This proposal makes no changes to the underlying security of TCP. 656 More information about TCP security concerns can be found in 657 [RFC5681]. 659 7. IANA Considerations 661 This document does not require any IANA actions. 663 8. References 665 8.1. Normative References 667 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 668 Requirement Levels", BCP 14, RFC 2119, 669 DOI 10.17487/RFC2119, March 1997, 670 . 672 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 673 of Explicit Congestion Notification (ECN) to IP", 674 RFC 3168, DOI 10.17487/RFC3168, September 2001, 675 . 677 [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion 678 Control Algorithms", BCP 133, RFC 5033, 679 DOI 10.17487/RFC5033, August 2007, 680 . 682 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 683 Friendly Rate Control (TFRC): Protocol Specification", 684 RFC 5348, DOI 10.17487/RFC5348, September 2008, 685 . 687 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 688 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 689 . 691 [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The 692 NewReno Modification to TCP's Fast Recovery Algorithm", 693 RFC 6582, DOI 10.17487/RFC6582, April 2012, 694 . 696 [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., 697 and Y. Nishida, "A Conservative Loss Recovery Algorithm 698 Based on Selective Acknowledgment (SACK) for TCP", 699 RFC 6675, DOI 10.17487/RFC6675, August 2012, 700 . 702 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 703 Recommendations Regarding Active Queue Management", 704 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 705 . 707 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 708 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 709 May 2017, . 711 8.2. Informative References 713 [CEHRX07] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic 714 Ordering for Internet Congestion Control and its 715 Applications", IEEE INFOCOM 2007 - 26th IEEE International 716 Conference on Computer Communications, 717 DOI 10.1109/infcom.2007.111, 2007, 718 . 720 [FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of 721 Equation-Based and AIMD Congestion Control", May 2000, 722 . 724 [GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary 725 Adjustment Algorithms", Technical Report TR2002-29, 726 Department of Computer Sciences, The University of 727 Texas at Austin, 11 August 2002, 728 . 730 [HKLRX06] Ha, S., Kim, Y., Le, L., Rhee, I., and L. Xu, "A Step 731 toward Realistic Performance Evaluation of High-Speed TCP 732 Variants", International Workshop on Protocols for Fast 733 Long-Distance Networks, February 2006, 734 . 736 [HR08] Ha, S. and I. Rhee, "Hybrid Slow Start for High-Bandwidth 737 and Long-Distance Networks", International Workshop 738 on Protocols for Fast Long-Distance Networks, 2008, 739 . 742 [HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: a new TCP-friendly 743 high-speed TCP variant", ACM SIGOPS Operating Systems 744 Review Vol. 42, pp. 64-74, DOI 10.1145/1400097.1400105, 745 July 2008, . 747 [K03] Kelly, T., "Scalable TCP: improving performance in 748 highspeed wide area networks", ACM SIGCOMM Computer 749 Communication Review Vol. 33, pp. 83-91, 750 DOI 10.1145/956981.956989, April 2003, 751 . 753 [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", 754 RFC 3649, DOI 10.17487/RFC3649, December 2003, 755 . 757 [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large 758 Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March 759 2004, . 761 [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", 762 RFC 4960, DOI 10.17487/RFC4960, September 2007, 763 . 765 [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and 766 R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", 767 RFC 8312, DOI 10.17487/RFC8312, February 2018, 768 . 770 [RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, 771 "TCP Alternative Backoff with ECN (ABE)", RFC 8511, 772 DOI 10.17487/RFC8511, December 2018, 773 . 775 [XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary Increase 776 Congestion Control (BIC) for Fast Long-Distance Networks", 777 IEEE INFOCOM 2004, DOI 10.1109/infcom.2004.1354672, March 778 2004, . 780 Appendix A. Changes from RFC8312 782 A.1. Since RFC8312 784 * converted to markdown and xml2rfc v3 786 * updated references (as part of the conversion) 788 * updated author information 790 * various whitespace changes 792 Authors' Addresses 794 Lisong Xu 795 University of Nebraska-Lincoln 796 Department of Computer Science and Engineering 797 Lincoln, NE 68588-0115 798 United States of America 800 Email: xu@unl.edu 801 URI: https://cse.unl.edu/~xu/ 803 Sangtae Ha 804 University of Colorado at Boulder 805 Department of Computer Science 806 Boulder, CO 80309-0430 807 United States of America 809 Email: sangtae.ha@colorado.edu 810 URI: https://netstech.org/sangtaeha/ 811 Lars Eggert (editor) 812 NetApp 813 Stenbergintie 12 B 814 FI-02700 Kauniainen 815 Finland 817 Email: lars@eggert.org 818 URI: https://eggert.org/