idnits 2.17.1 draft-ietf-rmcat-coupled-cc-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 31, 2016) is 2724 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Outdated reference: A later version (-13) exists of draft-ietf-rmcat-nada-03 == Outdated reference: A later version (-10) exists of draft-ietf-rmcat-eval-test-04 == Outdated reference: A later version (-11) exists of draft-ietf-rmcat-sbd-04 == Outdated reference: A later version (-17) exists of draft-ietf-rtcweb-transports-11 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RTP Media Congestion Avoidance S. Islam 3 Techniques (rmcat) M. Welzl 4 Internet-Draft S. Gjessing 5 Intended status: Experimental University of Oslo 6 Expires: May 4, 2017 October 31, 2016 8 Coupled congestion control for RTP media 9 draft-ietf-rmcat-coupled-cc-04 11 Abstract 13 When multiple congestion controlled RTP sessions traverse the same 14 network bottleneck, combining their controls can improve the total 15 on-the-wire behavior in terms of delay, loss and fairness. This 16 document describes such a method for flows that have the same sender, 17 in a way that is as flexible and simple as possible while minimizing 18 the amount of changes needed to existing RTP applications. It 19 specifies how to apply the method for both the NADA and Google 20 congestion control algorithms. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on May 4, 2017. 39 Copyright Notice 41 Copyright (c) 2016 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 3. Limitations . . . . . . . . . . . . . . . . . . . . . . . . . 4 59 4. Architectural overview . . . . . . . . . . . . . . . . . . . . 4 60 5. Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 61 5.1. SBD . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 62 5.2. FSE . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 63 5.3. Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 8 64 5.3.1. Example algorithm 1 - Active FSE . . . . . . . . . . . 8 65 5.3.2. Example algorithm 2 - Conservative Active FSE . . . . 9 66 6. Application . . . . . . . . . . . . . . . . . . . . . . . . . 10 67 6.1. NADA . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 68 6.2. GCC . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 69 6.3. General recommendations . . . . . . . . . . . . . . . . . 11 70 7. Expected feedback from experiments . . . . . . . . . . . . . . 11 71 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 72 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 73 10. Security Considerations . . . . . . . . . . . . . . . . . . . 12 74 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 75 11.1. Normative References . . . . . . . . . . . . . . . . . . . 13 76 11.2. Informative References . . . . . . . . . . . . . . . . . . 13 77 Appendix A. Scheduling . . . . . . . . . . . . . . . . . . . . . 15 78 Appendix B. Example algorithm - Passive FSE . . . . . . . . . . . 15 79 B.1. Example operation (passive) . . . . . . . . . . . . . . . 18 80 Appendix C. Change log . . . . . . . . . . . . . . . . . . . . . 22 81 C.1. draft-welzl-rmcat-coupled-cc . . . . . . . . . . . . . . . 22 82 C.1.1. Changes from -00 to -01 . . . . . . . . . . . . . . . 22 83 C.1.2. Changes from -01 to -02 . . . . . . . . . . . . . . . 22 84 C.1.3. Changes from -02 to -03 . . . . . . . . . . . . . . . 22 85 C.1.4. Changes from -03 to -04 . . . . . . . . . . . . . . . 22 86 C.1.5. Changes from -04 to -05 . . . . . . . . . . . . . . . 22 87 C.2. draft-ietf-rmcat-coupled-cc . . . . . . . . . . . . . . . 23 88 C.2.1. Changes from draft-welzl-rmcat-coupled-cc-05 . . . . . 23 89 C.2.2. Changes from -00 to -01 . . . . . . . . . . . . . . . 23 90 C.2.3. Changes from -01 to -02 . . . . . . . . . . . . . . . 23 91 C.2.4. Changes from -02 to -03 . . . . . . . . . . . . . . . 23 92 C.2.5. Changes from -03 to -04 . . . . . . . . . . . . . . . 23 93 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 95 1. Introduction 97 When there is enough data to send, a congestion controller must 98 increase its sending rate until the path's capacity has been reached; 99 depending on the controller, sometimes the rate is increased further, 100 until packets are ECN-marked or dropped. This process inevitably 101 creates undesirable queuing delay when multiple congestion controlled 102 connections traverse the same network bottleneck. 104 The Congestion Manager (CM) [RFC3124] couples flows by providing a 105 single congestion controller. It is hard to implement because it 106 requires an additional congestion controller and removes all per- 107 connection congestion control functionality, which is quite a 108 significant change to existing RTP based applications. This document 109 presents a method to combine the behavior of congestion control 110 mechanisms that is easier to implement than the Congestion Manager 111 [RFC3124] and also requires less significant changes to existing RTP 112 based applications. It attempts to roughly approximate the CM 113 behavior by sharing information between existing congestion 114 controllers. It is able to honor user-specified priorities, which is 115 required by rtcweb [RFC7478]. 117 2. Definitions 119 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 120 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 121 document are to be interpreted as described in RFC 2119 [RFC2119]. 123 Available Bandwidth: 124 The available bandwidth is the nominal link capacity minus the 125 amount of traffic that traversed the link during a certain time 126 interval, divided by that time interval. 128 Bottleneck: 129 The first link with the smallest available bandwidth along the 130 path between a sender and receiver. 132 Flow: 133 A flow is the entity that congestion control is operating on. 134 It could, for example, be a transport layer connection, an RTP 135 session, or a subsession that is multiplexed onto a single RTP 136 session together with other subsessions. 138 Flow Group Identifier (FGI): 139 A unique identifier for each subset of flows that is limited by 140 a common bottleneck. 142 Flow State Exchange (FSE): 143 The entity that maintains information that is exchanged between 144 flows. 146 Flow Group (FG): 147 A group of flows having the same FGI. 149 Shared Bottleneck Detection (SBD): 150 The entity that determines which flows traverse the same 151 bottleneck in the network, or the process of doing so. 153 3. Limitations 155 Sender-side only: 156 Coupled congestion control as described here only operates 157 inside a single host on the sender side. This is because, 158 irrespective of where the major decisions for congestion 159 control are taken, the sender of a flow needs to eventually 160 decide on the transmission rate. Additionally, the necessary 161 information about how much data an application can currently 162 send on a flow is often only available at the sender side, 163 making the sender an obvious choice for placement of the 164 elements and mechanisms described here. 166 Shared bottlenecks do not change quickly: 167 As per the definition above, a bottleneck depends on cross 168 traffic, and since such traffic can heavily fluctuate, 169 bottlenecks can change at a high frequency (e.g., there can be 170 oscillation between two or more links). This means that, when 171 flows are partially routed along different paths, they may 172 quickly change between sharing and not sharing a bottleneck. 173 For simplicity, here it is assumed that a shared bottleneck is 174 valid for a time interval that is significantly longer than the 175 interval at which congestion controllers operate. Note that, 176 for the only SBD mechanism defined in this document 177 (multiplexing on the same five-tuple), the notion of a shared 178 bottleneck stays correct even in the presence of fast traffic 179 fluctuations: since all flows that are assumed to share a 180 bottleneck are routed in the same way, if the bottleneck 181 changes, it will still be shared. 183 4. Architectural overview 185 Figure 1 shows the elements of the architecture for coupled 186 congestion control: the Flow State Exchange (FSE), Shared Bottleneck 187 Detection (SBD) and Flows. The FSE is a storage element that can be 188 implemented in two ways: active and passive. In the active version, 189 it initiates communication with flows and SBD. However, in the 190 passive version, it does not actively initiate communication with 191 flows and SBD; its only active role is internal state maintenance 192 (e.g., an implementation could use soft state to remove a flow's data 193 after long periods of inactivity). Every time a flow's congestion 194 control mechanism would normally update its sending rate, the flow 195 instead updates information in the FSE and performs a query on the 196 FSE, leading to a sending rate that can be different from what the 197 congestion controller originally determined. Using information 198 about/from the currently active flows, SBD updates the FSE with the 199 correct Flow State Identifiers (FSIs). This document describes both 200 active and passive versions, however the passive version is put into 201 the appendix as it is extremely experimental. Figure 2 shows the 202 interaction between flows and the FSE, using the variable names 203 defined in Section 5.2. 205 ------- <--- Flow 1 206 | FSE | <--- Flow 2 .. 207 ------- <--- .. Flow N 208 ^ 209 | | 210 ------- | 211 | SBD | <-------| 212 ------- 214 Figure 1: Coupled congestion control architecture 216 Flow#1(cc) FSE Flow#2(cc) 217 ---------- --- ---------- 218 #1 JOIN ----register--> REGISTER 220 REGISTER <--register-- JOIN #1 222 #2 CC_R ----UPDATE----> UPDATE (in) 224 #3 NEW RATE <---FSE_R------ UPDATE (out) --FSE_R----> #3 NEW RATE 226 Figure 2: Flow-FSE interaction 228 Since everything shown in Figure 1 is assumed to operate on a single 229 host (the sender) only, this document only describes aspects that 230 have an influence on the resulting on-the-wire behavior. It does, 231 for instance, not define how many bits must be used to represent 232 FSIs, or in which way the entities communicate. Implementations can 233 take various forms: for instance, all the elements in the figure 234 could be implemented within a single application, thereby operating 235 on flows generated by that application only. Another alternative 236 could be to implement both the FSE and SBD together in a separate 237 process which different applications communicate with via some form 238 of Inter-Process Communication (IPC). Such an implementation would 239 extend the scope to flows generated by multiple applications. The 240 FSE and SBD could also be included in the Operating System kernel. 242 5. Roles 244 This section gives an overview of the roles of the elements of 245 coupled congestion control, and provides an example of how coupled 246 congestion control can operate. 248 5.1. SBD 250 SBD uses knowledge about the flows to determine which flows belong in 251 the same Flow Group (FG), and assigns FGIs accordingly. This 252 knowledge can be derived in three basic ways: 254 1. From multiplexing: it can be based on the simple assumption that 255 packets sharing the same five-tuple (IP source and destination 256 address, protocol, and transport layer port number pair) and 257 having the same Differentiated Services Code Point (DSCP) in the 258 IP header are typically treated in the same way along the path. 259 The latter method is the only one specified in this document: SBD 260 MAY consider all flows that use the same five-tuple and DSCP to 261 belong to the same FG. This classification applies to certain 262 tunnels, or RTP flows that are multiplexed over one transport 263 (cf. [transport-multiplex]). Such multiplexing is also a 264 recommended usage of RTP in rtcweb [rtcweb-rtp-usage]. 266 2. Via configuration: e.g. by assuming that a common wireless uplink 267 is also a shared bottleneck. 269 3. From measurements: e.g. by considering correlations among 270 measured delay and loss as an indication of a shared bottleneck. 272 The methods above have some essential trade-offs: e.g., multiplexing 273 is a completely reliable measure, however it is limited in scope to 274 two end points (i.e., it cannot be applied to couple congestion 275 controllers of one sender talking to multiple receivers). A 276 measurement-based SBD mechanism is described in [I-D.ietf-rmcat-sbd]. 278 Measurements can never be 100% reliable, in particular because they 279 are based on the past but applying coupled congestion control means 280 to make an assumption about the future; it is therefore recommended 281 to implement cautionary measures, e.g. by disabling coupled 282 congestion control if enabling it causes a significant increase in 283 delay and/or packet loss. Measurements also take time, which entails 284 a certain delay for turning on coupling (refer to 285 [I-D.ietf-rmcat-sbd] for details). Using system configuration to 286 decide about shared bottlenecks can be more efficient (faster to 287 obtain) than using measurements, but it relies on assumptions about 288 the network environment. 290 5.2. FSE 292 The FSE contains a list of all flows that have registered with it. 293 For each flow, it stores the following: 295 o a unique flow number to identify the flow 297 o the FGI of the FG that it belongs to (based on the definitions in 298 this document, a flow has only one bottleneck, and can therefore 299 be in only one FG) 301 o a priority P, which here is assumed to be represented as a 302 floating point number in the range from 0.1 (unimportant) to 1 303 (very important). 305 o The rate used by the flow in bits per second, FSE_R. 307 Note that the priority does not need to be a floating point value and 308 its value range does not matter for this algorithm: the algorithm 309 works with a flow's priority portion of the sum of all priority 310 values. Priorities can therefore be mapped to the "very-low", "low", 311 "medium" or "high" priority levels described in 312 [I-D.ietf-rtcweb-transports] using the values 1, 2, 4 and 8, 313 respectively. 315 In the FSE, each FG contains one static variable S_CR which is the 316 sum of the calculated rates of all flows in the same FG. This value 317 is used to calculate the sending rate. 319 The information listed here is enough to implement the sample flow 320 algorithm given below. FSE implementations could easily be extended 321 to store, e.g., a flow's current sending rate for statistics 322 gathering or future potential optimizations. 324 5.3. Flows 326 Flows register themselves with SBD and FSE when they start, 327 deregister from the FSE when they stop, and carry out an UPDATE 328 function call every time their congestion controller calculates a new 329 sending rate. Via UPDATE, they provide the newly calculated rate and 330 optionally (if the algorithm supports it) the desired rate. The 331 desired rate is less than the calculated rate in case of application- 332 limited flows; otherwise, it is the same as the calculated rate. 334 Below, two example algorithms are described. While other algorithms 335 could be used instead, the same algorithm must be applied to all 336 flows. Names of variables used in the algorithms are explained 337 below. 339 o CC_R - The rate received from a flow's congestion controller when 340 it calls UPDATE. 342 o FSE_R - The rate calculated by the FSE for a flow. 344 o S_CR - The sum of the calculated rates of all flows in the same 345 FG; this value is used to calculate the sending rate. 347 o FG - A group of flows having the same FGI, and hence sharing the 348 same bottleneck. 350 o P - The priority of a flow which is received from the flow's 351 congestion controller; the FSE uses this variable for calculating 352 FSE R. 354 o S_P - The sum of all the priorities. 356 5.3.1. Example algorithm 1 - Active FSE 358 This algorithm was designed to be the simplest possible method to 359 assign rates according to the priorities of flows. Simulations 360 results in [fse] indicate that it does however not significantly 361 reduce queuing delay and packet loss. 363 (1) When a flow f starts, it registers itself with SBD and the FSE. 364 FSE_R is initialized with the congestion controller's initial 365 rate. SBD will assign the correct FGI. When a flow is assigned 366 an FGI, it adds its FSE_R to S_CR. 368 (2) When a flow f stops or pauses, its entry is removed from the 369 list. 371 (3) Every time the congestion controller of the flow f determines a 372 new sending rate CC_R, the flow calls UPDATE, which carries out 373 the tasks listed below to derive the new sending rates for all 374 the flows in the FG. A flow's UPDATE function uses a local 375 (i.e. per-flow) temporary variable S_P, which is the sum of all 376 the priorities. 378 (a) It updates S_CR. 380 S_CR = S_CR + CC_R - FSE_R(f) 382 (b) It calculates the sum of all the priorities, S_P. 384 S_P = 0 385 for all flows i in FG do 386 S_P = S_P + P(i) 387 end for 389 (c) It calculates the sending rates for all the flows in an FG 390 and distributes them. 392 for all flows i in FG do 393 FSE_R(i) = (P(i)*S_CR)/S_P 394 send FSE_R(i) to the flow i 395 end for 397 5.3.2. Example algorithm 2 - Conservative Active FSE 399 This algorithm extends algorithm 1 to conservatively emulate the 400 behavior of a single flow by proportionally reducing the aggregate 401 rate on congestion. Simulations results in [fse] indicate that it 402 can significantly reduce queuing delay and packet loss. 404 (1) When a flow f starts, it registers itself with SBD and the FSE. 405 FSE_R is initialized with the congestion controller's initial 406 rate. SBD will assign the correct FGI. When a flow is assigned 407 an FGI, it adds its FSE_R to S_CR. 409 (2) When a flow f stops or pauses, its entry is removed from the 410 list. 412 (3) Every time the congestion controller of the flow f determines a 413 new sending rate CC_R, the flow calls UPDATE, which carries out 414 the tasks listed below to derive the new sending rates for all 415 the flows in the FG. A flow's UPDATE function uses a local 416 (i.e. per-flow) temporary variable S_P, which is the sum of all 417 the priorities, and a local variable DELTA, which is used to 418 calculate the difference between CC_R and the previously stored 419 FSE_R. To prevent flows from either ignoring congestion or 420 overreacting, a timer keeps them from changing their rates 421 immediately after the common rate reduction that follows a 422 congestion event. This timer is set to 2 RTTs of the flow that 423 experienced congestion because it is assumed that a congestion 424 event can persist for up to one RTT of that flow, with another 425 RTT added to compensate for fluctuations in the measured RTT 426 value. 428 (a) It updates S_CR based on DELTA. 430 if Timer has expired or not set then 431 DELTA = CC_R - FSE_R(f) 432 if DELTA < 0 then // Reduce S_CR proportionally 433 S_CR = S_CR * CC_R / FSE_R(f) 434 Set Timer for 2 RTTs 435 else 436 S_CR = S_CR + DELTA 437 end if 438 end if 440 (b) It calculates the sum of all the priorities, S_P. 442 S_P = 0 443 for all flows i in FG do 444 S_P = S_P + P(i) 445 end for 447 (c) It calculates the sending rates for all the flows in an FG 448 and distributes them. 450 for all flows i in FG do 451 FSE_R(i) = (P(i)*S_CR)/S_P 452 send FSE_R(i) to the flow i 453 end for 455 6. Application 457 This section specifies how the FSE can be applied to specific 458 congestion control mechanisms and makes general recommendations that 459 facilitate applying the FSE to future congestion controls. 461 6.1. NADA 463 Network-Assisted Dynamic Adapation (NADA) [I-D.ietf-rmcat-nada] is a 464 congestion control scheme for rtcweb. It calculates a reference rate 465 r_ref upon receiving an acknowledgment, and then, based on the 466 reference rate, it calculates a video target rate r_vin and a sending 467 rate for the flows, r_send. 469 When applying the FSE to NADA, the UPDATE function call described in 470 Section 5.3 gives the FSE NADA's reference rate r_ref. The 471 recommended algorithm for NADA is the Active FSE in Section 5.3.1. 472 In step 3 (c), when the FSE_R(i) is "sent" to the flow i, this means 473 updating r_ref(r_vin and r_send) of flow i with the value of 474 FSE_R(i). 476 6.2. GCC 478 Google Congestion Control (GCC) [I-D.ietf-rmcat-gcc] is another 479 congestion control scheme for rtcweb. The rate control of GCC 480 employs two parts: controlling the bandwidth estimate based on delay, 481 and controlling the bandwidth estimate based on loss. Both are 482 designed to estimate the available bandwidth, A_hat. 484 When applying the FSE to GCC, the UPDATE function call described in 485 Section 5.3 gives the FSE GCC's estimate of available bandwidth 486 A_hat. The recommended algorithm for GCC is the Active FSE in 487 Section 5.3.1. In step 3 (c), when the FSE_R(i) is "sent" to the 488 flow i, this means updating A_hat of flow i with the value of 489 FSE_R(i). 491 6.3. General recommendations 493 This section provides general advice for applying the FSE to 494 congestion control mechanisms. 496 Receiver-side calculations: 497 When receiver-side calculations make assumptions about the rate 498 of the sender, the calculations need to be synchronized or the 499 receiver needs to be updated accordingly. This applies to TFRC 500 [RFC5348], for example, where simulations showed somewhat less 501 favorable results when using the FSE without a receiver-side 502 change [fse]. 504 7. Expected feedback from experiments 506 The algorithm described in this memo has so far been evaluated using 507 simulations covering all the tests for more than one flow from 509 [I-D.ietf-rmcat-eval-test] (see [IETF-93], [IETF-94]). Experiments 510 should confirm these results using at least one of the same 511 congestion control algorithms (GCC or NADA) with real-life code 512 (e.g., browsers communicating over an emulated network covering the 513 conditions in [I-D.ietf-rmcat-eval-test]. The tests with real-life 514 code should be repeated afterwards in real network environments and 515 monitored. Experiments should investigate cases where the media 516 coder's output rate is below the rate that is calculated by the 517 coupling algorithm (FSE_R in algorithms 1 and 2, section 5.3). 518 Implementers and testers are invited to document their findings in an 519 Internet draft. 521 8. Acknowledgements 523 This document has benefitted from discussions with and feedback from 524 Andreas Petlund, Anna Brunstrom, David Hayes, David Ros (who also 525 gave the FSE its name), Ingemar Johansson, Karen Nielsen, Kristian 526 Hiorth, Mirja Kuehlewind, Martin Stiemerling, Varun Singh , Xiaoqing 527 Zhu, and Zaheduzzaman Sarker. The authors would like to especially 528 thank Xiaoqing Zhu and Stefan Holmer for helping with NADA and GCC. 530 This work was partially funded by the European Community under its 531 Seventh Framework Programme through the Reducing Internet Transport 532 Latency (RITE) project (ICT-317700). 534 9. IANA Considerations 536 This memo includes no request to IANA. 538 10. Security Considerations 540 In scenarios where the architecture described in this document is 541 applied across applications, various cheating possibilities arise: 542 e.g., supporting wrong values for the calculated rate, the desired 543 rate, or the priority of a flow. In the worst case, such cheating 544 could either prevent other flows from sending or make them send at a 545 rate that is unreasonably large. The end result would be unfair 546 behavior at the network bottleneck, akin to what could be achieved 547 with any UDP based application. Hence, since this is no worse than 548 UDP in general, there seems to be no significant harm in using this 549 in the absence of UDP rate limiters. 551 In the case of a single-user system, it should also be in the 552 interest of any application programmer to give the user the best 553 possible experience by using reasonable flow priorities or even 554 letting the user choose them. In a multi-user system, this interest 555 may not be given, and one could imagine the worst case of an "arms 556 race" situation, where applications end up setting their priorities 557 to the maximum value. If all applications do this, the end result is 558 a fair allocation in which the priority mechanism is implicitly 559 eliminated, and no major harm is done. 561 11. References 563 11.1. Normative References 565 [I-D.ietf-rmcat-gcc] 566 Holmer, S., Lundin, H., Carlucci, G., Cicco, L., and S. 567 Mascolo, "A Google Congestion Control Algorithm for Real- 568 Time Communication", draft-ietf-rmcat-gcc-02 (work in 569 progress), July 2016. 571 [I-D.ietf-rmcat-nada] 572 Zhu, X., Pan, R., Ramalho, M., Cruz, S., Jones, P., Fu, 573 J., D'Aronco, S., and C. Ganzhorn, "NADA: A Unified 574 Congestion Control Scheme for Real-Time Media", 575 draft-ietf-rmcat-nada-03 (work in progress), 576 September 2016. 578 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 579 Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/ 580 RFC2119, March 1997, 581 . 583 [RFC3124] Balakrishnan, H. and S. Seshan, "The Congestion Manager", 584 RFC 3124, DOI 10.17487/RFC3124, June 2001, 585 . 587 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 588 Friendly Rate Control (TFRC): Protocol Specification", 589 RFC 5348, DOI 10.17487/RFC5348, September 2008, 590 . 592 11.2. Informative References 594 [I-D.ietf-rmcat-eval-test] 595 Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test 596 Cases for Evaluating RMCAT Proposals", 597 draft-ietf-rmcat-eval-test-04 (work in progress), 598 October 2016. 600 [I-D.ietf-rmcat-sbd] 601 Hayes, D., Ferlin, S., Welzl, M., and K. Hiorth, "Shared 602 Bottleneck Detection for Coupled Congestion Control for 603 RTP Media.", draft-ietf-rmcat-sbd-04 (work in progress), 604 March 2016. 606 [I-D.ietf-rtcweb-transports] 607 Alvestrand, H., "Transports for WebRTC", 608 draft-ietf-rtcweb-transports-11.txt (work in progress), 609 January 2016. 611 [IETF-93] Islam, S., Welzl, M., and S. Gjessing, "Updates on Coupled 612 Congestion Control for RTP Media", July 2015, 613 . 615 [IETF-94] Islam, S., Welzl, M., and S. Gjessing, "Updates on Coupled 616 Congestion Control for RTP Media", November 2015, 617 . 619 [RFC7478] Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real- 620 Time Communication Use Cases and Requirements", RFC 7478, 621 DOI 10.17487/RFC7478, March 2015, 622 . 624 [fse] Islam, S., Welzl, M., Gjessing, S., and N. Khademi, 625 "Coupled Congestion Control for RTP Media", ACM SIGCOMM 626 Capacity Sharing Workshop (CSWS 2014) and ACM SIGCOMM CCR 627 44(4) 2014; extended version available as a technical 628 report from 629 http://safiquli.at.ifi.uio.no/paper/fse-tech-report.pdf , 630 2014. 632 [fse-noms] 633 Islam, S., Welzl, M., Hayes, D., and S. Gjessing, 634 "Managing Real-Time Media Flows through a Flow State 635 Exchange", IEEE NOMS 2016, Istanbul, Turkey , 2016. 637 [rtcweb-rtp-usage] 638 Perkins, C., Westerlund, M., and J. Ott, "Web Real-Time 639 Communication (WebRTC): Media Transport and Use of RTP", 640 draft-ietf-rtcweb-rtp-usage-26.txt (work in progress), 641 March 2016. 643 [transport-multiplex] 644 Westerlund, M. and C. Perkins, "Multiple RTP Sessions on a 645 Single Lower-Layer Transport", 646 draft-westerlund-avtcore-transport-multiplexing-07.txt 647 (work in progress), October 2013. 649 Appendix A. Scheduling 651 When connections originate from the same host, it would be possible 652 to use only one single sender-side congestion controller which 653 determines the overall allowed sending rate, and then use a local 654 scheduler to assign a proportion of this rate to each RTP session. 655 This way, priorities could also be implemented as a function of the 656 scheduler. The Congestion Manager (CM) [RFC3124] also uses such a 657 scheduling function. 659 Appendix B. Example algorithm - Passive FSE 661 Active algorithms calculate the rates for all the flows in the FG and 662 actively distribute them. In a passive algorithm, UPDATE returns a 663 rate that should be used instead of the rate that the congestion 664 controller has determined. This can make a passive algorithm easier 665 to implement; however, when round-trip times of flows are unequal, 666 shorter-RTT flows will update and react to the overall FSE state more 667 often than longer-RTT flows, which can produce unwanted side effects. 668 This problem is more significant when the congestion control 669 convergence depends on the RTT. While the passive algorithm works 670 better for congestion controls with RTT-independent convergence, it 671 can still produce oscillations on short time scales. The algorithm 672 described below is therefore considered as highly experimental. 673 Results of a simplified passive FSE algorithm with both NADA and GCC 674 can be found in [fse-noms]. 676 This passive version of the FSE stores the following information in 677 addition to the variables described in Section 5.2: 679 o The desired rate DR. This can be smaller than the calculated rate 680 if the application feeding into the flow has less data to send 681 than the congestion controller would allow. In case of a bulk 682 transfer, DR must be set to CC_R received from the flow's 683 congestion module. 685 The passive version of the FSE contains one static variable per FG 686 called TLO (Total Leftover Rate -- used to let a flow 'take' 687 bandwidth from application-limited or terminated flows) which is 688 initialized to 0. For the passive version, S_CR is limited to 689 increase or decrease as conservatively as a flow's congestion 690 controller decides in order to prohibit sudden rate jumps. 692 (1) When a flow f starts, it registers itself with SBD and the FSE. 693 FSE_R and DR are initialized with the congestion controller's 694 initial rate. SBD will assign the correct FGI. When a flow is 695 assigned an FGI, it adds its FSE_R to S_CR. 697 (2) When a flow f stops or pauses, it sets its DR to 0 and sets P to 698 -1. 700 (3) Every time the congestion controller of the flow f determines a 701 new sending rate CC_R, assuming the flow's new desired rate 702 new_DR to be "infinity" in case of a bulk data transfer with an 703 unknown maximum rate, the flow calls UPDATE, which carries out 704 the tasks listed below to derive the flow's new sending rate, 705 Rate. A flow's UPDATE function uses a few local (i.e. per-flow) 706 temporary variables, which are all initialized to 0: DELTA, 707 new_S_CR and S_P. 709 (a) For all the flows in its FG (including itself), it 710 calculates the sum of all the calculated rates, new_S_CR. 711 Then it calculates the difference between FSE_R(f) and 712 CC_R, DELTA. 714 for all flows i in FG do 715 new_S_CR = new_S_CR + FSE_R(i) 716 end for 717 DELTA = CC_R - FSE_R(f) 719 (b) It updates S_CR, FSE_R(f) and DR(f). 721 FSE_R(f) = CC_R 722 if DELTA > 0 then // the flow's rate has increased 723 S_CR = S_CR + DELTA 724 else if DELTA < 0 then 725 S_CR = new_S_CR + DELTA 726 end if 727 DR(f) = min(new_DR,FSE_R(f)) 729 (c) It calculates the leftover rate TLO, removes the terminated 730 flows from the FSE and calculates the sum of all the 731 priorities, S_P. 733 for all flows i in FG do 734 if P(i)<0 then 735 delete flow 736 else 737 S_P = S_P + P(i) 738 end if 739 end for 740 if DR(f) < FSE_R(f) then 741 TLO = TLO + (P(f)/S_P) * S_CR - DR(f)) 742 end if 744 (d) It calculates the sending rate, Rate. 746 Rate = min(new_DR, (P(f)*S_CR)/S_P + TLO) 748 if Rate != new_DR and TLO > 0 then 749 TLO = 0 // f has 'taken' TLO 750 end if 752 (e) It updates DR(f) and FSE_R(f) with Rate. 754 if Rate > DR(f) then 755 DR(f) = Rate 756 end if 757 FSE_R(f) = Rate 759 The goals of the flow algorithm are to achieve prioritization, 760 improve network utilization in the face of application-limited flows, 761 and impose limits on the increase behavior such that the negative 762 impact of multiple flows trying to increase their rate together is 763 minimized. It does that by assigning a flow a sending rate that may 764 not be what the flow's congestion controller expected. It therefore 765 builds on the assumption that no significant inefficiencies arise 766 from temporary application-limited behavior or from quickly jumping 767 to a rate that is higher than the congestion controller intended. 768 How problematic these issues really are depends on the controllers in 769 use and requires careful per-controller experimentation. The coupled 770 congestion control mechanism described here also does not require all 771 controllers to be equal; effects of heterogeneous controllers, or 772 homogeneous controllers being in different states, are also subject 773 to experimentation. 775 This algorithm gives all the leftover rate of application-limited 776 flows to the first flow that updates its sending rate, provided that 777 this flow needs it all (otherwise, its own leftover rate can be taken 778 by the next flow that updates its rate). Other policies could be 779 applied, e.g. to divide the leftover rate of a flow equally among all 780 other flows in the FGI. 782 B.1. Example operation (passive) 784 In order to illustrate the operation of the passive coupled 785 congestion control algorithm, this section presents a toy example of 786 two flows that use it. Let us assume that both flows traverse a 787 common 10 Mbit/s bottleneck and use a simplistic congestion 788 controller that starts out with 1 Mbit/s, increases its rate by 1 789 Mbit/s in the absence of congestion and decreases it by 2 Mbit/s in 790 the presence of congestion. For simplicity, flows are assumed to 791 always operate in a round-robin fashion. Rate numbers below without 792 units are assumed to be in Mbit/s. For illustration purposes, the 793 actual sending rate is also shown for every flow in FSE diagrams even 794 though it is not really stored in the FSE. 796 Flow #1 begins. It is a bulk data transfer and considers itself to 797 have top priority. This is the FSE after the flow algorithm's step 798 1: 800 ---------------------------------------- 801 | # | FGI | P | FSE_R | DR | Rate | 802 | | | | | | | 803 | 1 | 1 | 1 | 1 | 1 | 1 | 804 ---------------------------------------- 805 S_CR = 1, TLO = 0 807 Its congestion controller gradually increases its rate. Eventually, 808 at some point, the FSE should look like this: 810 ----------------------------------------- 811 | # | FGI | P | FSE_R | DR | Rate | 812 | | | | | | | 813 | 1 | 1 | 1 | 10 | 10 | 10 | 814 ----------------------------------------- 815 S_CR = 10, TLO = 0 817 Now another flow joins. It is also a bulk data transfer, and has a 818 lower priority (0.5): 820 ------------------------------------------ 821 | # | FGI | P | FSE_R | DR | Rate | 822 | | | | | | | 823 | 1 | 1 | 1 | 10 | 10 | 10 | 824 | 2 | 1 | 0.5 | 1 | 1 | 1 | 825 ------------------------------------------ 826 S_CR = 11, TLO = 0 828 Now assume that the first flow updates its rate to 8, because the 829 total sending rate of 11 exceeds the total capacity. Let us take a 830 closer look at what happens in step 3 of the flow algorithm. 832 CC_R = 8. new_DR = infinity. 833 3 a) new_S_CR = 11; DELTA = 8 - 10 = -2. 834 3 b) FSE_Rf) = 8. DELTA is negative, hence S_CR = 9; 835 DR(f) = 8. 836 3 c) S_P = 1.5. 837 3 d) new sending rate = min(infinity, 1/1.5 * 9 + 0) = 6. 838 3 e) FSE_R(f) = 6. 840 The resulting FSE looks as follows: 841 ------------------------------------------- 842 | # | FGI | P | FSE_R | DR | Rate | 843 | | | | | | | 844 | 1 | 1 | 1 | 6 | 8 | 6 | 845 | 2 | 1 | 0.5 | 1 | 1 | 1 | 846 ------------------------------------------- 847 S_CR = 9, TLO = 0 849 The effect is that flow #1 is sending with 6 Mbit/s instead of the 8 850 Mbit/s that the congestion controller derived. Let us now assume 851 that flow #2 updates its rate. Its congestion controller detects 852 that the network is not fully saturated (the actual total sending 853 rate is 6+1=7) and increases its rate. 855 CC_R=2. new_DR = infinity. 856 3 a) new_S_CR = 7; DELTA = 2 - 1 = 1. 857 3 b) FSE_R(f) = 2. DELTA is positive, hence S_CR = 9 + 1 = 10; 858 DR(f) = 2. 859 3 c) S_P = 1.5. 860 3 d) new sending rate = min(infinity, 0.5/1.5 * 10 + 0) = 3.33. 861 3 e) DR(f) = FSE_R(f) = 3.33. 863 The resulting FSE looks as follows: 864 ------------------------------------------- 865 | # | FGI | P | FSE_R | DR | Rate | 866 | | | | | | | 867 | 1 | 1 | 1 | 6 | 8 | 6 | 868 | 2 | 1 | 0.5 | 3.33 | 3.33 | 3.33 | 869 ------------------------------------------- 870 S_CR = 10, TLO = 0 872 The effect is that flow #2 is now sending with 3.33 Mbit/s, which is 873 close to half of the rate of flow #1 and leads to a total utilization 874 of 6(#1) + 3.33(#2) = 9.33 Mbit/s. Flow #2's congestion controller 875 has increased its rate faster than the controller actually expected. 876 Now, flow #1 updates its rate. Its congestion controller detects 877 that the network is not fully saturated and increases its rate. 878 Additionally, the application feeding into flow #1 limits the flow's 879 sending rate to at most 2 Mbit/s. 881 CC_R=7. new_DR=2. 882 3 a) new_S_CR = 9.33; DELTA = 1. 883 3 b) FSE_R(f) = 7, DELTA is positive, hence S_CR = 10 + 1 = 11; 884 DR = min(2, 7) = 2. 885 3 c) S_P = 1.5; DR(f) < FSE_R(f), hence TLO = 1/1.5 * 11 - 2 = 5.33. 886 3 d) new sending rate = min(2, 1/1.5 * 11 + 5.33) = 2. 887 3 e) FSE_R(f) = 2. 889 The resulting FSE looks as follows: 890 ------------------------------------------- 891 | # | FGI | P | FSE_R | DR | Rate | 892 | | | | | | | 893 | 1 | 1 | 1 | 2 | 2 | 2 | 894 | 2 | 1 | 0.5 | 3.33 | 3.33 | 3.33 | 895 ------------------------------------------- 896 S_CR = 11, TLO = 5.33 897 Now, the total rate of the two flows is 2 + 3.33 = 5.33 Mbit/s, i.e. 898 the network is significantly underutilized due to the limitation of 899 flow #1. Flow #2 updates its rate. Its congestion controller 900 detects that the network is not fully saturated and increases its 901 rate. 903 CC_R=4.33. new_DR = infinity. 904 3 a) new_S_CR = 5.33; DELTA = 1. 905 3 b) FSE_R(f) = 4.33. DELTA is positive, hence S_CR = 12; 906 DR(f) = 4.33. 907 3 c) S_P = 1.5. 908 3 d) new sending rate: min(infinity, 0.5/1.5 * 12 + 5.33 ) = 9.33. 909 3 e) FSE_R(f) = 9.33, DR(f) = 9.33. 911 The resulting FSE looks as follows: 912 ------------------------------------------- 913 | # | FGI | P | FSE_R | DR | Rate | 914 | | | | | | | 915 | 1 | 1 | 1 | 2 | 2 | 2 | 916 | 2 | 1 | 0.5 | 9.33 | 9.33 | 9.33 | 917 ------------------------------------------- 918 S_CR = 12, TLO = 0 920 Now, the total rate of the two flows is 2 + 9.33 = 11.33 Mbit/s. 921 Finally, flow #1 terminates. It sets P to -1 and DR to 0. Let us 922 assume that it terminated late enough for flow #2 to still experience 923 the network in a congested state, i.e. flow #2 decreases its rate in 924 the next iteration. 926 CC_R = 7.33. new_DR = infinity. 927 3 a) new_S_CR = 11.33; DELTA = -2. 928 3 b) FSE_R(f) = 7.33. DELTA is negative, hence S_CR = 9.33; 929 DR(f) = 7.33. 930 3 c) Flow 1 has P = -1, hence it is deleted from the FSE. 931 S_P = 0.5. 932 3 d) new sending rate: min(infinity, 0.5/0.5*9.33 + 0) = 9.33. 933 3 e) FSE_R(f) = DR(f) = 9.33. 935 The resulting FSE looks as follows: 936 ------------------------------------------- 937 | # | FGI | P | FSE_R | DR | Rate | 938 | | | | | | | 939 | 2 | 1 | 0.5 | 9.33 | 9.33 | 9.33 | 940 ------------------------------------------- 941 S_CR = 9.33, TLO = 0 943 Appendix C. Change log 945 C.1. draft-welzl-rmcat-coupled-cc 947 C.1.1. Changes from -00 to -01 949 o Added change log. 951 o Updated the example algorithm and its operation. 953 C.1.2. Changes from -01 to -02 955 o Included an active version of the algorithm which is simpler. 957 o Replaced "greedy flow" with "bulk data transfer" and "non-greedy" 958 with "application-limited". 960 o Updated new_CR to CC_R, and CR to FSE_R for better understanding. 962 C.1.3. Changes from -02 to -03 964 o Included an active conservative version of the algorithm which 965 reduces queue growth and packet loss; added a reference to a 966 technical report that shows these benefits with simulations. 968 o Moved the passive variant of the algorithm to appendix. 970 C.1.4. Changes from -03 to -04 972 o Extended SBD section. 974 o Added a note about window-based controllers. 976 C.1.5. Changes from -04 to -05 978 o Added a section about applying the FSE to specific congestion 979 control algorithms, with a subsection specifying its use with 980 NADA. 982 C.2. draft-ietf-rmcat-coupled-cc 984 C.2.1. Changes from draft-welzl-rmcat-coupled-cc-05 986 o Moved scheduling section to the appendix. 988 C.2.2. Changes from -00 to -01 990 o Included how to apply the algorithm to GCC. 992 o Updated variable names of NADA to be in line with the latest 993 version. 995 o Added a reference to [I-D.ietf-rtcweb-transports] to make a 996 connection to the prioritization text there. 998 C.2.3. Changes from -01 to -02 1000 o Minor changes. 1002 o Moved references of NADA and GCC from informative to normative. 1004 o Added a reference for the passive variant of the algorithm. 1006 C.2.4. Changes from -02 to -03 1008 o Minor changes. 1010 o Added a section about expected feedback from experiments. 1012 C.2.5. Changes from -03 to -04 1014 o Described the names of variables used in the algorithms. 1016 o Added a diagram to illustrate the interaction between flows and 1017 the FSE. 1019 o Added text on the trade-off of using the configuration based 1020 approach. 1022 o Minor changes to enhance the readability. 1024 Authors' Addresses 1026 Safiqul Islam 1027 University of Oslo 1028 PO Box 1080 Blindern 1029 Oslo, N-0316 1030 Norway 1032 Phone: +47 22 84 08 37 1033 Email: safiquli@ifi.uio.no 1035 Michael Welzl 1036 University of Oslo 1037 PO Box 1080 Blindern 1038 Oslo, N-0316 1039 Norway 1041 Phone: +47 22 85 24 20 1042 Email: michawe@ifi.uio.no 1044 Stein Gjessing 1045 University of Oslo 1046 PO Box 1080 Blindern 1047 Oslo, N-0316 1048 Norway 1050 Phone: +47 22 85 24 44 1051 Email: steing@ifi.uio.no