idnits 2.17.1 draft-ietf-tcpm-cubic-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 13, 2017) is 2353 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TCP Maintenance and Minor Extensions (TCPM) WG I. Rhee 3 Internet-Draft NCSU 4 Intended status: Informational L. Xu 5 Expires: May 17, 2018 UNL 6 S. Ha 7 Colorado 8 A. Zimmermann 10 L. Eggert 11 NetApp 12 R. Scheffenegger 13 November 13, 2017 15 CUBIC for Fast Long-Distance Networks 16 draft-ietf-tcpm-cubic-07 18 Abstract 20 CUBIC is an extension to the current TCP standards. It differs from 21 the current TCP standards only in the congestion control algorithm in 22 the sender side. In particular, it uses a cubic function instead of 23 a linear window increase function of the current TCP standards to 24 improve scalability and stability under fast and long distance 25 networks. CUBIC and its predecessor algorithm have been adopted as 26 default by Linux and have been used for many years. This document 27 provides a specification of CUBIC to enable third party 28 implementations and to solicit the community feedback through 29 experimentation on the performance of CUBIC. 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at https://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on May 17, 2018. 48 Copyright Notice 50 Copyright (c) 2017 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (https://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 67 3. Design principles of CUBIC . . . . . . . . . . . . . . . . . 4 68 4. CUBIC Congestion Control . . . . . . . . . . . . . . . . . . 6 69 4.1. Window increase function . . . . . . . . . . . . . . . . 6 70 4.2. TCP-friendly region . . . . . . . . . . . . . . . . . . . 7 71 4.3. Concave region . . . . . . . . . . . . . . . . . . . . . 8 72 4.4. Convex region . . . . . . . . . . . . . . . . . . . . . . 8 73 4.5. Multiplicative decrease . . . . . . . . . . . . . . . . . 8 74 4.6. Fast convergence . . . . . . . . . . . . . . . . . . . . 9 75 4.7. Timeout . . . . . . . . . . . . . . . . . . . . . . . . . 9 76 4.8. Slowstart . . . . . . . . . . . . . . . . . . . . . . . . 10 77 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 10 78 5.1. Fairness to standard TCP . . . . . . . . . . . . . . . . 10 79 5.2. Using Spare Capacity . . . . . . . . . . . . . . . . . . 12 80 5.3. Difficult Environments . . . . . . . . . . . . . . . . . 13 81 5.4. Investigating a Range of Environments . . . . . . . . . . 13 82 5.5. Protection against Congestion Collapse . . . . . . . . . 13 83 5.6. Fairness within the Alternative Congestion Control 84 Algorithm. . . . . . . . . . . . . . . . . . . . . . . . 13 85 5.7. Performance with Misbehaving Nodes and Outside Attackers 13 86 5.8. Behavior for Application-Limited Flows . . . . . . . . . 13 87 5.9. Responses to Sudden or Transient Events . . . . . . . . . 14 88 5.10. Incremental Deployment . . . . . . . . . . . . . . . . . 14 89 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 90 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 91 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 92 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 93 9.1. Normative References . . . . . . . . . . . . . . . . . . 14 94 9.2. Informative References . . . . . . . . . . . . . . . . . 15 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 97 1. Introduction 99 The low utilization problem of TCP in fast long-distance networks is 100 well documented in [K03] [RFC3649]. This problem arises from a slow 101 increase of congestion window following a congestion event in a 102 network with a large bandwidth delay product (BDP). Experience 103 [HKLRX06] indicates that this problem is frequently observed even in 104 the range of congestion window sizes over several hundreds of packets 105 especially under a network path with over 100ms round-trip times 106 (RTTs). This problem is equally applicable to all Reno style TCP 107 standards and their variants, including TCP-RENO [RFC5681], TCP- 108 NewReno [RFC6582] [RFC6675], SCTP [RFC4960], TFRC [RFC5348] that use 109 the same linear increase function for window growth, which we refer 110 to collectively as Standard TCP below. 112 CUBIC, originally proposed in [HRX08], is a modification to the 113 congestion control algorithm of Standard TCP to remedy this problem. 114 This document describes the most recent specification of CUBIC. 115 Specifically, CUBIC uses a cubic function instead of a linear window 116 increase function of Standard TCP to improve scalability and 117 stability under fast and long distance networks. 119 BIC-TCP [XHR04], a predecessor of CUBIC, has been selected as the 120 default TCP congestion control algorithm by Linux in the year 2005 121 and been used for several years by the Internet community at large. 122 CUBIC uses a similar window increase function as BIC-TCP and is 123 designed to be less aggressive and fairer to Standard TCP in 124 bandwidth usage than BIC-TCP while maintaining the strengths of BIC- 125 TCP such as stability, window scalability and RTT fairness. CUBIC 126 has already replaced BIC-TCP as the default TCP congestion control 127 algorithm in Linux and has been deployed globally by Linux. Through 128 extensive testing in various Internet scenarios, we believe that 129 CUBIC is safe for testing and deployment in the global Internet. 131 In the following sections, we first briefly explain the design 132 principles of CUBIC, then provide the exact specification of CUBIC, 133 and finally discuss the safety features of CUBIC following the 134 guidelines specified in [RFC5033]. 136 2. Conventions 138 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 139 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 140 document are to be interpreted as described in [RFC2119]. 142 3. Design principles of CUBIC 144 CUBIC is designed according to the following design principles. 146 Principle 1: For better network utilization and stability, CUBIC 147 uses both the concave and convex profiles of a cubic function to 148 increase the congestion window size, instead of using just a 149 convex function. 151 Principle 2: To be TCP-friendly, CUBIC is designed to behave like 152 Standard TCP in networks with short RTTs and small bandwidth where 153 Standard TCP performs well. 155 Principle 3: For RTT-fairness, CUBIC is designed to achieve linear 156 bandwidth share among flows with different RTTs. 158 Principle 4: CUBIC appropriately sets its multiplicative window 159 decrease factor, in order to balance between the scalability and 160 convergence speed. 162 Principle 1: For better network utilization and stability, CUBIC 163 [HRX08] uses a cubic window increase function in terms of the elapsed 164 time from the last congestion event. While most alternative 165 congestion control algorithms to Standard TCP increase the congestion 166 window using convex functions, CUBIC uses both the concave and convex 167 profiles of a cubic function for window growth. After a window 168 reduction in response to a congestion event detected by duplicate 169 ACKs or ECN-Echo ACKs[RFC3168], CUBIC registers the congestion window 170 size where it got the congestion event as W_max and performs a 171 multiplicative decrease of congestion window. After it enters into 172 congestion avoidance, it starts to increase the congestion window 173 using the concave profile of the cubic function. The cubic function 174 is set to have its plateau at W_max so that the concave window 175 increase continues until the window size becomes W_max. After that, 176 the cubic function turns into a convex profile and the convex window 177 increase begins. This style of window adjustment (concave and then 178 convex) improves the algorithm stability while maintaining high 179 network utilization [CEHRX07]. This is because the window size 180 remains almost constant, forming a plateau around W_max where network 181 utilization is deemed highest. Under steady state, most window size 182 samples of CUBIC are close to W_max, thus promoting high network 183 utilization and stability. Note that those congestion control 184 algorithms using only convex functions to increase the congestion 185 window size have the maximum increments around W_max and thus 186 introduce a large number of packet bursts around the saturation point 187 of the network, likely causing frequent global loss synchronizations. 189 Principle 2: CUBIC promotes per-flow fairness to Standard TCP. Note 190 that Standard TCP performs well under short RTT and small bandwidth 191 (or small BDP) networks. Only in long RTT and large bandwidth (or 192 large BDP) networks, it has the scalability problem. An alternative 193 congestion control algorithm to Standard TCP designed to be friendly 194 to Standard TCP at a per-flow basis must operate to increase its 195 congestion window less aggressively in small BDP networks than in 196 large BDP networks. The aggressiveness of CUBIC mainly depends on 197 the maximum window size before a window reduction, which is smaller 198 in small BDP networks than in large BDP networks. Thus, CUBIC 199 increases its congestion window less aggressively in small BDP 200 networks than in large BDP networks. Furthermore, in cases when the 201 cubic function of CUBIC increases its congestion window less 202 aggressively than Standard TCP, CUBIC simply follows the window size 203 of Standard TCP to ensure that CUBIC achieves at least the same 204 throughput as Standard TCP in small BDP networks. We call this 205 region where CUBIC behaves like Standard TCP, the TCP-friendly 206 region. 208 Principle 3: Two CUBIC flows with different RTTs have their 209 throughput ratio linearly proportional to the inverse of their RTT 210 ratio, where the throughput of a flow is approximately its congestion 211 window size divided by its RTT. Specifically, CUBIC maintains a 212 window increase rate independent of RTTs outside of the TCP-friendly 213 region, and thus flows with different RTTs have similar congestion 214 window sizes under steady state when they operate outside the TCP- 215 friendly region. This notion of a linear throughput ratio is similar 216 to that of Standard TCP under high statistical multiplexing 217 environments where packet losses are independent of individual flow 218 rates. However, under low statistical multiplexing environments, the 219 throughput ratio of Standard TCP flows with different RTTs is 220 quadratically proportional to the inverse of their RTT ratio [XHR04]. 221 CUBIC always ensures the linear throughput ratio independent of the 222 levels of statistical multiplexing. This is an improvement over 223 Standard TCP. While there is no consensus on particular throughput 224 ratios of different RTT flows, we believe that under wired Internet, 225 use of a linear throughput ratio seems more reasonable than equal 226 throughputs (i.e., same throughput for flows with different RTTs) or 227 a higher order throughput ratio (e.g., a quadratical throughput ratio 228 of Standard TCP under low statistical multiplexing environments). 230 Principle 4: To balance between the scalability and convergence 231 speed, CUBIC sets the multiplicative window decrease factor to 0.7 232 while Standard TCP uses 0.5. While this improves the scalability of 233 CUBIC, a side effect of this decision is slower convergence 234 especially under low statistical multiplexing environments. This 235 design choice is following the observation that the author of HSTCP 236 [RFC3649] has made along with other researchers (e.g., [GV02]): the 237 current Internet becomes more asynchronous with less frequent loss 238 synchronizations with high statistical multiplexing. Under this 239 environment, even strict Multiplicative-Increase Multiplicative- 240 Decrease (MIMD) can converge. CUBIC flows with the same RTT always 241 converge to the same throughput independent of statistical 242 multiplexing, thus achieving intra-algorithm fairness. We also find 243 that under the environments with sufficient statistical multiplexing, 244 the convergence speed of CUBIC flows is reasonable. 246 4. CUBIC Congestion Control 248 The unit of all window sizes in this document is segments of the 249 maximum segment size (MSS), and the unit of all times is seconds. 250 Let cwnd denote the congestion window size of a flow, and ssthresh 251 denote the slow start threshold. 253 4.1. Window increase function 255 CUBIC maintains the acknowledgment (ACK) clocking of Standard TCP by 256 increasing congestion window only at the reception of ACK. It does 257 not make any change to the fast recovery and retransmit of TCP, such 258 as TCP-NewReno [RFC6582] [RFC6675]. During congestion avoidance 259 after a congestion event where a packet loss is detected by duplicate 260 ACKs or a network congestion is detected by ACKs with ECN-Echo flags 261 [RFC3168], CUBIC changes the window increase function of Standard 262 TCP. Suppose that W_max is the window size just before the window is 263 reduced in the last congestion event. 265 CUBIC uses the following window increase function: 267 W_cubic(t) = C*(t-K)^3 + W_max (Eq. 1) 269 where C is a constant fixed to determine the aggressiveness of window 270 increase in high BDP networks, t is the elapsed time from the 271 beginning of the current congestion avoidance, and K is the time 272 period that the above function takes to increase the current window 273 size to W_max if there are no further congestion events and is 274 calculated using the following equation: 276 K = cubic_root(W_max*(1-beta_cubic)/C) (Eq. 2) 278 where beta_cubic is the CUBIC multiplication decrease factor, that 279 is, when a congestion event is detected, CUBIC reduces its cwnd to 280 W_cubic(0)=W_max*beta_cubic. We discuss how we set beta_cubic in 281 Section 4.5 and how we set C in Section 5. 283 Upon receiving an ACK during congestion avoidance, CUBIC computes the 284 window increase rate during the next RTT period using Eq. 1. It sets 285 W_cubic(t+RTT) as the candidate target value of congestion window, 286 where RTT is the weighted average RTT calculated by Standard TCP. 288 Depending on the value of the current congestion window size cwnd, 289 CUBIC runs in three different modes. 291 1) The TCP-friendly region, which ensures that CUBIC achieves at 292 least the same throughput as Standard TCP. 294 2) The concave region, if CUBIC is not in the TCP-friendly region 295 and cwnd is less than W_max. 297 3) The convex region, if CUBIC is not in the TCP-friendly region 298 and cwnd is greater than W_max. 300 Below, we describe the exact actions taken by CUBIC in each region. 302 4.2. TCP-friendly region 304 Standard TCP performs well in certain types of networks, for example, 305 under short RTT and small bandwidth (or small BDP) networks. In 306 these networks, we use the TCP-friendly region to ensure that CUBIC 307 achieves at least the same throughput as Standard TCP. 309 The TCP-friendly region is designed according to the analysis 310 described in [FHP00]. The analysis studies the performance of an 311 Additive Increase and Multiplicative Decrease (AIMD) algorithm with 312 an additive factor of alpha_aimd (segments per RTT) and a 313 multiplicative factor of beta_aimd, denoted by AIMD(alpha_aimd, 314 beta_aimd). Specifically, the average congestion window size of 315 AIMD(alpha_aimd, beta_aimd) can be calculated using Eq. 3. The 316 analysis shows that AIMD(alpha_aimd, beta_aimd) with 317 alpha_aimd=3*(1-beta_aimd)/(1+beta_aimd) achieves the same average 318 window size as Standard TCP that uses AIMD(1, 0.5). 320 AVG_W_aimd = [ alpha_aimd * (1+beta_aimd) / 321 (2*(1-beta_aimd)*p) ]^0.5 (Eq. 3) 323 Based on the above analysis, CUBIC uses Eq. 4 to estimate the window 324 size W_est of AIMD(alpha_aimd, beta_aimd) with 325 alpha_aimd=3*(1-beta_cubic)/(1+beta_cubic) and beta_aimd=beta_cubic, 326 which achieves the same average window size as Standard TCP. When 327 receiving an ACK in congestion avoidance (cwnd could be greater than 328 or less than W_max), CUBIC checks whether W_cubic(t) is less than 329 W_est(t). If so, CUBIC is in the TCP-friendly region and cwnd SHOULD 330 be set to W_est(t) at each reception of ACK. 332 W_est(t) = W_max*beta_cubic + 333 [3*(1-beta_cubic)/(1+beta_cubic)] * (t/RTT) (Eq. 4) 335 4.3. Concave region 337 When receiving an ACK in congestion avoidance, if CUBIC is not in the 338 TCP-friendly region and cwnd is less than W_max, then CUBIC is in the 339 concave region. In this region, cwnd MUST be incremented by 340 (W_cubic(t+RTT) - cwnd)/cwnd for each received ACK, where 341 W_cubic(t+RTT) is calculated using Eq. 1. 343 4.4. Convex region 345 When receiving an ACK in congestion avoidance, if CUBIC is not in the 346 TCP-friendly region and cwnd is larger than or equal to W_max, then 347 CUBIC is in the convex region. The convex region indicates that the 348 network conditions might have been perturbed since the last 349 congestion event, possibly implying more available bandwidth after 350 some flow departures. Since the Internet is highly asynchronous, 351 some amount of perturbation is always possible without causing a 352 major change in available bandwidth. In this region, CUBIC is being 353 very careful by very slowly increasing its window size. The convex 354 profile ensures that the window increases very slowly at the 355 beginning and gradually increases its increase rate. We also call 356 this region as the maximum probing phase since CUBIC is searching for 357 a new W_max. In this region, cwnd MUST be incremented by 358 (W_cubic(t+RTT) - cwnd)/cwnd for each received ACK, where 359 W_cubic(t+RTT) is calculated using Eq. 1. 361 4.5. Multiplicative decrease 363 When a packet loss is detected by duplicate ACKs or a network 364 congestion is detected by ECN-Echo ACKs, CUBIC updates its W_max, 365 cwnd, and ssthresh (slow start threshold) as follows. Parameter 366 beta_cubic SHOULD be set to 0.7. 368 W_max = cwnd; // save window size before reduction 369 ssthresh = cwnd * beta_cubic; // new slow start threshold 370 ssthresh = max(ssthresh, 2); // threshold is at least 2 MSS 371 cwnd = cwnd * beta_cubic; // window reduction 373 A side effect of setting beta_cubic to a bigger value than 0.5 is 374 slower convergence. We believe that while a more adaptive setting of 375 beta_cubic could result in faster convergence, it will make the 376 analysis of CUBIC much harder. This adaptive adjustment of 377 beta_cubic is an item for the next version of CUBIC. 379 4.6. Fast convergence 381 To improve the convergence speed of CUBIC, we add a heuristic in 382 CUBIC. When a new flow joins the network, existing flows in the 383 network need to give up some of their bandwidth to allow the new flow 384 some room for growth if the existing flows have been using all the 385 bandwidth of the network. To speed up this bandwidth release by 386 existing flows, the following mechanism called fast convergence 387 SHOULD be implemented. 389 With fast convergence, when a congestion event occurs, before the 390 window reduction of the congestion window, a flow remembers the last 391 value of W_max before it updates W_max for the current congestion 392 event. Let us call the last value of W_max to be W_last_max. 394 if (W_max < W_last_max){ // should we make room for others 395 W_last_max = W_max; // remember the last W_max 396 W_max = W_max*(1.0+beta_cubic)/2.0; // further reduce W_max 397 } else { 398 W_last_max = W_max // remember the last W_max 399 } 401 At a congestion event, if the current value of W_max is less than 402 W_last_max, this indicates that the saturation point experienced by 403 this flow is getting reduced because of the change in available 404 bandwidth. Then we allow this flow to release more bandwidth by 405 reducing W_max further. This action effectively lengthens the time 406 for this flow to increase its congestion window because the reduced 407 W_max forces the flow to have the plateau earlier. This allows more 408 time for the new flow to catch up its congestion window size 410 The fast convergence is designed for network environments with 411 multiple CUBIC flows. In network environments with only a single 412 CUBIC flow and without any other traffic, the fast convergence SHOULD 413 be disabled. 415 4.7. Timeout 417 In case of timeout, CUBIC follows Standard TCP to reduce cwnd 418 [RFC5681], but sets ssthresh using beta_cubic (same as in 419 Section 4.5) that is different from Standard TCP [RFC5681]. 421 During the first congestion avoidance after a timeout, CUBIC 422 increases its congestion window size using Eq. 1, where t is the 423 elapsed time since the beginning of the current congestion avoidance, 424 K is set to 0, and W_max is set to the congestion window size at the 425 beginning of the current congestion avoidance. 427 4.8. Slowstart 429 CUBIC MUST employ a slow start algorithm, when the cwnd is no more 430 than ssthresh. Among the slow start algorithms, CUBIC MAY choose the 431 standard TCP slow start [RFC5681] in general networks, or the limited 432 slow start [RFC3742] or hybrid slow start [HR08] for fast and long- 433 distance networks. 435 In the case when CUBIC runs the hybrid slow start [HR08], it may exit 436 the first slow start without incurring any packet loss and thus W_max 437 is undefined. In this special case, CUBIC switches to congestion 438 avoidance and increases its congestion window size using Eq. 1, where 439 t is the elapsed time since the beginning of the current congestion 440 avoidance, K is set to 0, and W_max is set to the congestion window 441 size at the beginning of the current congestion avoidance. 443 5. Discussion 445 In this section, we further discuss the safety features of CUBIC 446 following the guidelines specified in [RFC5033]. 448 With a deterministic loss model where the number of packets between 449 two successive packet losses is always 1/p, CUBIC always operates 450 with the concave window profile which greatly simplifies the 451 performance analysis of CUBIC. The average window size of CUBIC can 452 be obtained by the following function: 454 AVG_W_cubic = [C*(3+beta_cubic)/(4*(1-beta_cubic))]^0.25 * 455 (RTT^0.75) / (p^0.75) (Eq. 5) 457 With beta_cubic set to 0.7, the above formula is reduced to: 459 AVG_W_cubic = (C*3.7/1.2)^0.25 * (RTT^0.75) / (p^0.75) (Eq. 6) 461 We will determine the value of C in the following subsection using 462 Eq. 6. 464 5.1. Fairness to standard TCP 466 In environments where Standard TCP is able to make reasonable use of 467 the available bandwidth, CUBIC does not significantly change this 468 state. 470 Standard TCP performs well in the following two types of networks: 472 1. networks with a small bandwidth-delay product (BDP) 474 2. networks with a short RTT, but not necessarily a small BDP 476 CUBIC is designed to behave very similarly to Standard TCP in the 477 above two types of networks. The following two tables show the 478 average window sizes of Standard TCP, HSTCP, and CUBIC. The average 479 window sizes of Standard TCP and HSTCP are from [RFC3649]. The 480 average window size of CUBIC is calculated using Eq. 6 and the CUBIC 481 TCP friendly region for three different values of C. 483 +--------+----------+-----------+------------+-----------+----------+ 484 | Loss | Average | Average | CUBIC | CUBIC | CUBIC | 485 | Rate P | TCP W | HSTCP W | (C=0.04) | (C=0.4) | (C=4) | 486 +--------+----------+-----------+------------+-----------+----------+ 487 | 10^-2 | 12 | 12 | 12 | 12 | 12 | 488 | 10^-3 | 38 | 38 | 38 | 38 | 59 | 489 | 10^-4 | 120 | 263 | 120 | 187 | 333 | 490 | 10^-5 | 379 | 1795 | 593 | 1054 | 1874 | 491 | 10^-6 | 1200 | 12279 | 3332 | 5926 | 10538 | 492 | 10^-7 | 3795 | 83981 | 18740 | 33325 | 59261 | 493 | 10^-8 | 12000 | 574356 | 105383 | 187400 | 333250 | 494 +--------+----------+-----------+------------+-----------+----------+ 496 Response function of Standard TCP, HSTCP, and CUBIC in networks with 497 RTT = 0.1 seconds. The average window size is in MSS-sized segments. 499 Table 1 501 +--------+-----------+-----------+------------+-----------+---------+ 502 | Loss | Average | Average | CUBIC | CUBIC | CUBIC | 503 | Rate P | TCP W | HSTCP W | (C=0.04) | (C=0.4) | (C=4) | 504 +--------+-----------+-----------+------------+-----------+---------+ 505 | 10^-2 | 12 | 12 | 12 | 12 | 12 | 506 | 10^-3 | 38 | 38 | 38 | 38 | 38 | 507 | 10^-4 | 120 | 263 | 120 | 120 | 120 | 508 | 10^-5 | 379 | 1795 | 379 | 379 | 379 | 509 | 10^-6 | 1200 | 12279 | 1200 | 1200 | 1874 | 510 | 10^-7 | 3795 | 83981 | 3795 | 5926 | 10538 | 511 | 10^-8 | 12000 | 574356 | 18740 | 33325 | 59261 | 512 +--------+-----------+-----------+------------+-----------+---------+ 514 Response function of Standard TCP, HSTCP, and CUBIC in networks with 515 RTT = 0.01 seconds. The average window size is in MSS-sized 516 segments. 518 Table 2 520 Both tables show that CUBIC with any of these three C values is more 521 friendly to TCP than HSTCP, especially in networks with a short RTT 522 where TCP performs reasonably well. For example, in a network with 523 RTT = 0.01 seconds and p=10^-6, TCP has an average window of 1200 524 packets. If the packet size is 1500 bytes, then TCP can achieve an 525 average rate of 1.44 Gbps. In this case, CUBIC with C=0.04 or C=0.4 526 achieves exactly the same rate as Standard TCP, whereas HSTCP is 527 about ten times more aggressive than Standard TCP. 529 We can see that C determines the aggressiveness of CUBIC in competing 530 with other congestion control algorithms for the bandwidth. CUBIC is 531 more friendly to the Standard TCP, if the value of C is lower. 532 However, we do not recommend to set C to a very low value like 0.04, 533 since CUBIC with a low C cannot efficiently use the bandwidth in long 534 RTT and high bandwidth networks. Based on these observations and our 535 experiments, we find C=0.4 gives a good balance between TCP- 536 friendliness and aggressiveness of window increase. Therefore, C 537 SHOULD be set to 0.4. With C set to 0.4, Eq. 6 is reduced to: 539 AVG_W_cubic = 1.054 * (RTT^0.75) / (p^0.75) (Eq. 7) 541 Eq. 7 is then used in the next subsection to show the scalability of 542 CUBIC. 544 5.2. Using Spare Capacity 546 CUBIC uses a more aggressive window increase function than Standard 547 TCP under long RTT and high bandwidth networks. 549 The following table shows that to achieve the 10Gbps rate, Standard 550 TCP requires a packet loss rate of 2.0e-10, while CUBIC requires a 551 packet loss rate of 2.9e-8. 553 +------------------+-----------+---------+---------+---------+ 554 | Throughput(Mbps) | Average W | TCP P | HSTCP P | CUBIC P | 555 +------------------+-----------+---------+---------+---------+ 556 | 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 | 557 | 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 | 558 | 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 | 559 | 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 | 560 | 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 | 561 +------------------+-----------+---------+---------+---------+ 563 Required packet loss rate for Standard TCP, HSTCP, and CUBIC to 564 achieve a certain throughput. We use 1500-byte packets and an RTT of 565 0.1 seconds. 567 Table 3 569 Our test results in [HKLRX06] indicate that CUBIC uses the spare 570 bandwidth left unused by existing Standard TCP flows in the same 571 bottleneck link without taking away much bandwidth from the existing 572 flows. 574 5.3. Difficult Environments 576 CUBIC is designed to remedy the poor performance of TCP in fast and 577 long-distance networks. 579 5.4. Investigating a Range of Environments 581 CUBIC has been extensively studied by using both NS-2 simulation and 582 test-bed experiments covering a wide range of network environments. 583 More information can be found in [HKLRX06]. 585 Same as Standard TCP, CUBIC is a loss-based congestion control 586 algorithm. Because CUBIC is designed to be more aggressive (due to 587 faster window increase function and bigger multiplicative decrease 588 factor) than Standard TCP in fast and long distance networks, it can 589 fill large drop-tail buffers more quickly than Standard TCP and 590 increase the risk of a standing queue[KWAF16]. In this case, proper 591 queue sizing and management [RFC7567] could be used to reduce the 592 packet queueing delay. 594 5.5. Protection against Congestion Collapse 596 With regard to the potential of causing congestion collapse, CUBIC 597 behaves like Standard TCP since CUBIC modifies only the window 598 adjustment algorithm of TCP. Thus, it does not modify the ACK 599 clocking and Timeout behaviors of Standard TCP. 601 5.6. Fairness within the Alternative Congestion Control Algorithm. 603 CUBIC ensures convergence of competing CUBIC flows with the same RTT 604 in the same bottleneck links to an equal throughput. When competing 605 flows have different RTTs, their throughput ratio is linearly 606 proportional to the inverse of their RTT ratios. This is true 607 independent of the level of statistical multiplexing in the link. 609 5.7. Performance with Misbehaving Nodes and Outside Attackers 611 This is not considered in the current CUBIC. 613 5.8. Behavior for Application-Limited Flows 615 CUBIC does not raise its congestion window size if the flow is 616 currently limited by the application instead of the congestion 617 window. In case of long periods when cwnd has not been updated due 618 to the application rate limit, such as idle periods, t in Eq. 1 MUST 619 NOT include these periods; otherwise, W_cubic(t) might be very high 620 after restarting from these periods. 622 5.9. Responses to Sudden or Transient Events 624 In case that there is a sudden congestion, a routing change, or a 625 mobility event, CUBIC behaves the same as Standard TCP. 627 5.10. Incremental Deployment 629 CUBIC requires only the change of TCP senders, and it does not make 630 any changes to TCP receivers. That is, a CUBIC sender works 631 correctly with the Standard TCP receivers. In addition, CUBIC does 632 not require any changes to the routers, and does not require any 633 assistant from the routers. 635 6. Security Considerations 637 This proposal makes no changes to the underlying security of TCP. 638 More information about TCP security concerns can be found in 639 [RFC5681]. 641 7. IANA Considerations 643 There are no IANA considerations regarding this document. 645 8. Acknowledgements 647 Alexander Zimmermann and Lars Eggert have received funding from the 648 European Union's Horizon 2020 research and innovation program 649 2014-2018 under grant agreement No. 644866 (SSICLOPS). This document 650 reflects only the authors' views and the European Commission is not 651 responsible for any use that may be made of the information it 652 contains. 654 9. References 656 9.1. Normative References 658 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 659 Requirement Levels", BCP 14, RFC 2119, 660 DOI 10.17487/RFC2119, March 1997, 661 . 663 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 664 of Explicit Congestion Notification (ECN) to IP", 665 RFC 3168, DOI 10.17487/RFC3168, September 2001, 666 . 668 [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", 669 RFC 3649, DOI 10.17487/RFC3649, December 2003, 670 . 672 [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large 673 Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March 674 2004, . 676 [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", 677 RFC 4960, DOI 10.17487/RFC4960, September 2007, 678 . 680 [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion 681 Control Algorithms", BCP 133, RFC 5033, 682 DOI 10.17487/RFC5033, August 2007, 683 . 685 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 686 Friendly Rate Control (TFRC): Protocol Specification", 687 RFC 5348, DOI 10.17487/RFC5348, September 2008, 688 . 690 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 691 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 692 . 694 [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The 695 NewReno Modification to TCP's Fast Recovery Algorithm", 696 RFC 6582, DOI 10.17487/RFC6582, April 2012, 697 . 699 [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., 700 and Y. Nishida, "A Conservative Loss Recovery Algorithm 701 Based on Selective Acknowledgment (SACK) for TCP", 702 RFC 6675, DOI 10.17487/RFC6675, August 2012, 703 . 705 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 706 Recommendations Regarding Active Queue Management", 707 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 708 . 710 9.2. Informative References 712 [CEHRX07] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic 713 Ordering for Internet Congestion Control and its 714 Applications", In Proceedings of IEEE INFOCOM , May 2007. 716 [FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of 717 Equation-Based and AIMD Congestion Control", May 2000. 719 [GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary 720 Adjustment Algorithms", Technical Report TR2002-29, 721 Department of Computer Sciences , The University of Texas 722 at Austin , August 2002. 724 [HKLRX06] Ha, S., Kim, Y., Le, L., Rhee, I., and L. Xu, "A Step 725 toward Realistic Performance Evaluation of High-Speed TCP 726 Variants", International Workshop on Protocols for Fast 727 Long-Distance Networks , February 2006. 729 [HR08] Ha, S. and I. Rhee, "Hybrid Slow Start for High-Bandwidth 730 and Long-Distance Networks", International Workshop on 731 Protocols for Fast Long-Distance Networks , 2008. 733 [HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: A New TCP-Friendly 734 High-Speed TCP Variant", ACM SIGOPS Operating System 735 Review , 2008. 737 [K03] Kelly, T., "Scalable TCP: Improving Performance in 738 HighSpeed Wide Area Networks", ACM SIGCOMM Computer 739 Communication Review , April 2003. 741 [KWAF16] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, 742 "TCP Alternative Backoff with ECN (ABE)", Internet-draft, 743 IETF work-in-progress draft-khademi-tcpm- 744 alternativebackoff-ecn-01 , October 2016. 746 [XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary Increase 747 Congestion Control for Fast, Long Distance Networks", In 748 Proceedings of IEEE INFOCOM , March 2004. 750 Authors' Addresses 752 Injong Rhee 753 North Carolina State University 754 Department of Computer Science 755 Raleigh, NC 27695-7534 756 US 758 Email: rhee@ncsu.edu 759 Lisong Xu 760 University of Nebraska-Lincoln 761 Department of Computer Science and Engineering 762 Lincoln, NE 68588-0115 763 US 765 Email: xu@unl.edu 767 Sangtae Ha 768 University of Colorado at Boulder 769 Department of Computer Science 770 Boulder, CO 80309-0430 771 US 773 Email: sangtae.ha@colorado.edu 775 Alexander Zimmermann 777 Phone: +49 175 5766838 778 Email: alexander.zimmermann@rwth-aachen.de 780 Lars Eggert 781 NetApp 782 Sonnenallee 1 783 Kirchheim 85551 784 Germany 786 Phone: +49 151 12055791 787 Email: lars@netapp.com 789 Richard Scheffenegger 791 Email: rscheff@gmx.at