idnits 2.17.1 draft-ietf-rmcat-nada-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 480 has weird spacing: '... delta x_o...' -- The document date (August 23, 2019) is 1708 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Missing Reference: 'RMIN' is mentioned on line 671, but not defined == Missing Reference: 'RMAX' is mentioned on line 671, but not defined == Unused Reference: 'I-D.ietf-rmcat-video-traffic-model' is defined on line 1064, but no explicit reference was found in the text == Outdated reference: A later version (-09) exists of draft-ietf-avtcore-cc-feedback-message-04 == Outdated reference: A later version (-11) exists of draft-ietf-rmcat-wireless-tests-08 Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group X. Zhu 3 Internet-Draft R. Pan 4 Intended status: Experimental M. Ramalho 5 Expires: February 24, 2020 S. Mena 6 Cisco Systems 7 August 23, 2019 9 NADA: A Unified Congestion Control Scheme for Real-Time Media 10 draft-ietf-rmcat-nada-12 12 Abstract 14 This document describes NADA (network-assisted dynamic adaptation), a 15 novel congestion control scheme for interactive real-time media 16 applications, such as video conferencing. In the proposed scheme, 17 the sender regulates its sending rate based on either implicit or 18 explicit congestion signaling, in a unified approach. The scheme can 19 benefit from explicit congestion notification (ECN) markings from 20 network nodes. It also maintains consistent sender behavior in the 21 absence of such markings, by reacting to queuing delays and packet 22 losses instead. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on February 24, 2020. 41 Copyright Notice 43 Copyright (c) 2019 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 3. System Overview . . . . . . . . . . . . . . . . . . . . . . . 3 61 4. Core Congestion Control Algorithm . . . . . . . . . . . . . . 5 62 4.1. Mathematical Notations . . . . . . . . . . . . . . . . . 5 63 4.2. Receiver-Side Algorithm . . . . . . . . . . . . . . . . . 8 64 4.3. Sender-Side Algorithm . . . . . . . . . . . . . . . . . . 10 65 5. Practical Implementation of NADA . . . . . . . . . . . . . . 13 66 5.1. Receiver-Side Operation . . . . . . . . . . . . . . . . . 13 67 5.1.1. Estimation of one-way delay and queuing delay . . . . 13 68 5.1.2. Estimation of packet loss/marking ratio . . . . . . . 13 69 5.1.3. Estimation of receiving rate . . . . . . . . . . . . 14 70 5.2. Sender-Side Operation . . . . . . . . . . . . . . . . . . 14 71 5.2.1. Rate shaping buffer . . . . . . . . . . . . . . . . . 15 72 5.2.2. Adjusting video target rate and sending rate . . . . 16 73 5.3. Feedback Message Requirements . . . . . . . . . . . . . . 16 74 6. Discussions and Further Investigations . . . . . . . . . . . 17 75 6.1. Choice of delay metrics . . . . . . . . . . . . . . . . . 17 76 6.2. Method for delay, loss, and marking ratio estimation . . 18 77 6.3. Impact of parameter values . . . . . . . . . . . . . . . 18 78 6.4. Sender-based vs. receiver-based calculation . . . . . . . 19 79 6.5. Incremental deployment . . . . . . . . . . . . . . . . . 20 80 7. Reference Implementations . . . . . . . . . . . . . . . . . . 20 81 8. Suggested Experiments . . . . . . . . . . . . . . . . . . . . 21 82 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 83 10. Security Considerations . . . . . . . . . . . . . . . . . . . 22 84 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22 85 12. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 86 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 87 13.1. Normative References . . . . . . . . . . . . . . . . . . 23 88 13.2. Informative References . . . . . . . . . . . . . . . . . 24 89 Appendix A. Network Node Operations . . . . . . . . . . . . . . 27 90 A.1. Default behavior of drop tail queues . . . . . . . . . . 27 91 A.2. RED-based ECN marking . . . . . . . . . . . . . . . . . . 27 92 A.3. Random Early Marking with Virtual Queues . . . . . . . . 28 93 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29 95 1. Introduction 97 Interactive real-time media applications introduce a unique set of 98 challenges for congestion control. Unlike TCP, the mechanism used 99 for real-time media needs to adapt quickly to instantaneous bandwidth 100 changes, accommodate fluctuations in the output of video encoder rate 101 control, and cause low queuing delay over the network. An ideal 102 scheme should also make effective use of all types of congestion 103 signals, including packet loss, queuing delay, and explicit 104 congestion notification (ECN) [RFC3168] markings. The requirements 105 for the congestion control algorithm are outlined in 106 [I-D.ietf-rmcat-cc-requirements]. It highlights that the desired 107 congestion control scheme should avoid flow starvation and attain a 108 reasonable fair share of bandwidth when competing against other 109 flows, adapt quickly, and operate in a stable manner. 111 This document describes an experimental congestion control scheme 112 called network-assisted dynamic adaptation (NADA). The NADA design 113 benefits from explicit congestion control signals (e.g., ECN 114 markings) from the network, yet also operates when only implicit 115 congestion indicators (delay and/or loss) are available. Such a 116 unified sender behavior distinguishes NADA from other congestion 117 control schemes for real-time media. In addition, its core 118 congestion control algorithm is designed to guarantee stability for 119 path round-trip-times (RTTs) below a prescribed bound (e.g., 250ms 120 with default parameter choices). It further supports weighted 121 bandwidth sharing among competing video flows with different 122 priorities. The signaling mechanism consists of standard RTP 123 timestamp [RFC3550] and RTCP feedback reports. The definition of the 124 desired RTCP feedback message is descirbed in detailed in 125 [I-D.ietf-avtcore-cc-feedback-message] so as to support the 126 successful operation of several congestion control schemes for real- 127 time interactive media. 129 2. Terminology 131 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 132 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 133 "OPTIONAL" in this document are to be interpreted as described in BCP 134 14 [RFC2119] [RFC8174] when, and only when, they appear in all 135 capitals, as shown here. 137 3. System Overview 139 Figure 1 shows the end-to-end system for real-time media transport 140 that NADA operates in. Note that there also exist network nodes 141 along the reverse (potentially uncongested) path that the RTCP 142 feedback reports traverse. Those network nodes are not shown in the 143 figure for sake of abrevity. 145 +---------+ r_vin +--------+ +--------+ +----------+ 146 | Media |<--------| RTP | |Network | | RTP | 147 | Encoder |========>| Sender |=======>| Node |====>| Receiver | 148 +---------+ r_vout +--------+ r_send +--------+ +----------+ 149 /|\ | 150 | | 151 +---------------------------------+ 152 RTCP Feedback Report 154 Figure 1: System Overview 156 o Media encoder with rate control capabilities. It encodes raw 157 media (audio and video) frames into compressed bitstream which is 158 later packetized into RTP packets. As discussed in [RFC8593], the 159 actual output rate from the encoder r_vout may fluctuate around 160 the target r_vin. Furthermore, it is possible that the encoder 161 can only react to bit rate changes at rather coarse time 162 intervals, e.g., once every 0.5 seconds. 164 o RTP sender: responsible for calculating the NADA reference rate 165 based on network congestion indicators (delay, loss, or ECN 166 marking reports from the receiver), for updating the video encoder 167 with a new target rate r_vin, and for regulating the actual 168 sending rate r_send accordingly. The RTP sender also generates a 169 sending timestamp for each outgoing packet. 171 o RTP receiver: responsible for measuring and estimating end-to-end 172 delay (based on sender timestamp), packet loss (based on RTP 173 sequence number), ECN marking ratios (based on [RFC6679]), and 174 receiving rate (r_recv) of the flow. It calculates the aggregated 175 congestion signal (x_curr) that accounts for queuing delay, ECN 176 markings, and packet losses. The receiver also determines the 177 mode for sender rate adaptation (rmode) based on whether the flow 178 has encountered any standing non-zero congestion. The receiver 179 sends periodic RTCP reports back to the sender, containing values 180 of x_curr, rmode, and r_recv. 182 o Network node with several modes of operation. The system can work 183 with the default behavior of a simple drop tail queue. It can 184 also benefit from advanced AQM features such as PIE [RFC8033], FQ- 185 CoDel [RFC8290], ECN marking based on RED [RFC7567], and PCN 186 marking using a token bucket algorithm ([RFC6660]). Note that 187 network node operation is out of control for the design of NADA. 189 4. Core Congestion Control Algorithm 191 Like TCP-Friendly Rate Control (TFRC) [Floyd-CCR00] [RFC5348], NADA 192 is a rate-based congestion control algorithm. In its simplest form, 193 the sender reacts to the collection of network congestion indicators 194 in the form of an aggregated congestion signal, and operates in one 195 of two modes: 197 o Accelerated ramp-up: when the bottleneck is deemed to be 198 underutilized, the rate increases multiplicatively with respect to 199 the rate of previously successful transmissions. The rate 200 increase mutliplier (gamma) is calculated based on observed round- 201 trip-time and target feedback interval, so as to limit self- 202 inflicted queuing delay. 204 o Gradual rate update: in the presence of non-zero aggregate 205 congestion signal, the sending rate is adjusted in reaction to 206 both its value (x_curr) and its change in value (x_diff). 208 This section introduces the list of mathematical notations and 209 describes the core congestion control algorithm at the sender and 210 receiver, respectively. Additional details on recommended practical 211 implementations are described in Section 5.1 and Section 5.2. 213 4.1. Mathematical Notations 215 This section summarizes the list of variables and parameters used in 216 the NADA algorithm. Figure 3 also includes the default values for 217 choosing the algorithm parameters either to represent a typical 218 setting in practical applications or based on theoretical and 219 simulation studies. See Section 6.3 for some of the discussions on 220 the impact of parameter values. In the experimental phase of this 221 draft, additional studies in real-world settings will gather further 222 learnings on how to choose and adapt these parameter values in 223 practical deployment. 225 +--------------+-------------------------------------------------+ 226 | Notation | Variable Name | 227 +--------------+-------------------------------------------------+ 228 | t_curr | Current timestamp | 229 | t_last | Last time sending/receiving a feedback | 230 | delta | Observed interval between current and previous | 231 | | feedback reports: delta = t_curr-t_last | 232 | r_ref | Reference rate based on network congestion | 233 | r_send | Sending rate | 234 | r_recv | Receiving rate | 235 | r_vin | Target rate for video encoder | 236 | r_vout | Output rate from video encoder | 237 | d_base | Estimated baseline delay | 238 | d_fwd | Measured and filtered one-way delay | 239 | d_queue | Estimated queuing delay | 240 | d_tilde | Equivalent delay after non-linear warping | 241 | p_mark | Estimated packet ECN marking ratio | 242 | p_loss | Estimated packet loss ratio | 243 | x_curr | Aggregate congestion signal | 244 | x_prev | Previous value of aggregate congestion signal | 245 | x_diff | Change in aggregate congestion signal w.r.t. | 246 | | its previous value: x_diff = x_curr - x_prev | 247 | rmode | Rate update mode: (0 = accelerated ramp-up; | 248 | | 1 = gradual update) | 249 | gamma | Rate increase multiplier in accelerated ramp-up | 250 | | mode | 251 | loss_int | Measured average loss interval in packet count | 252 | loss_exp | Threshold value for setting the last observed | 253 | | packet loss to expiration | 254 | rtt | Estimated round-trip-time at sender | 255 | buffer_len | Rate shaping buffer occupancy measured in bytes | 256 +--------------+-------------------------------------------------+ 258 Figure 2: List of variables. 260 +--------------+----------------------------------+----------------+ 261 | Notation | Parameter Name | Default Value | 262 +--------------+----------------------------------+----------------+ 263 | PRIO | Weight of priority of the flow | 1.0 264 | RMIN | Minimum rate of application | 150 Kbps | 265 | | supported by media encoder | | 266 | RMAX | Maximum rate of application | 1.5 Mbps | 267 | | supported by media encoder | | 268 | XREF | Reference congestion level | 10ms | 269 | KAPPA | Scaling parameter for gradual | 0.5 | 270 | | rate update calculation | | 271 | ETA | Scaling parameter for gradual | 2.0 | 272 | | rate update calculation | | 273 | TAU | Upper bound of RTT in gradual | 500ms | 274 | | rate update calculation | | 275 | DELTA | Target feedback interval | 100ms | 276 +..............+..................................+................+ 277 | LOGWIN | Observation window in time for | 500ms | 278 | | calculating packet summary | | 279 | | statistics at receiver | | 280 | QEPS | Threshold for determining queuing| 10ms | 281 | | delay build up at receiver | | 282 | DFILT | Bound on filtering delay | 120ms | 283 | GAMMA_MAX | Upper bound on rate increase | 0.5 | 284 | | ratio for accelerated ramp-up | | 285 | QBOUND | Upper bound on self-inflicted | 50ms | 286 | | queuing delay during ramp up | | 287 +..............+..................................+................+ 288 | MULTILOSS | Multiplier for self-scaling the | 7. | 289 | | expiration threshold of the last | | 290 | | observed loss (loss_exp) based on| | 291 | | measured average loss interval | | 292 | | (loss_int) | | 293 | QTH | Delay threshold for invoking | 50ms | 294 | | non-linear warping | | 295 | LAMBDA | Scaling parameter in the | 0.5 | 296 | | exponent of non-linear warping | | 297 +..............+..................................+................+ 298 | PLRREF | Reference packet loss ratio | 0.01 | 299 | PMRREF | Reference packet marking ratio | 0.01 | 300 | DLOSS | Reference delay penalty for loss | 10ms | 301 | | when packet loss ratio is at | | 302 | | PLRREF | | 303 | DMARK | Reference delay penalty for ECN | 2ms | 304 | | marking when packet marking | | 305 | | is at PMRREF | | 306 +..............+..................................+................+ 307 | FPS | Frame rate of incoming video | 30 | 308 | BETA_S | Scaling parameter for modulating | 0.1 | 309 | | outgoing sending rate | | 310 | BETA_V | Scaling parameter for modulating | 0.1 | 311 | | video encoder target rate | | 312 | ALPHA | Smoothing factor in exponential | 0.1 | 313 | | smoothing of packet loss and | | 314 | | marking ratios | 315 +--------------+----------------------------------+----------------+ 317 Figure 3: List of algorithm parameters and their default values. 319 4.2. Receiver-Side Algorithm 321 The receiver-side algorithm can be outlined as below: 323 On initialization: 324 set d_base = +INFINITY 325 set p_loss = 0 326 set p_mark = 0 327 set r_recv = 0 328 set both t_last and t_curr as current time in milliseconds 330 On receiving a media packet: 331 obtain current timestamp t_curr from system clock 332 obtain from packet header sending time stamp t_sent 333 obtain one-way delay measurement: d_fwd = t_curr - t_sent 334 update baseline delay: d_base = min(d_base, d_fwd) 335 update queuing delay: d_queue = d_fwd - d_base 336 update packet loss ratio estimate p_loss 337 update packet marking ratio estimate p_mark 338 update measurement of receiving rate r_recv 340 On time to send a new feedback report (t_curr - t_last > DELTA): 341 calculate non-linear warping of delay d_tilde if packet loss exists 342 calculate current aggregate congestion signal x_curr 343 determine mode of rate adaptation for sender: rmode 344 send feedback containing values of: rmode, x_curr, and r_recv 345 update t_last = t_curr 347 In order for a delay-based flow to hold its ground when competing 348 against loss-based flows (e.g., loss-based TCP), it is important to 349 distinguish between different levels of observed queuing delay. For 350 instance, over wired connections, a moderate queuing delay value on 351 the order of tens of milliseonds is likely self-inflicted or induced 352 by other delay-based flows, whereas a high queuing delay value of 353 several hundreds of milliseconds may indicate the presence of a loss- 354 based flow that does not refrain from increased delay. 356 If the last observed packet loss is within the expiration window of 357 loss_exp (measured in terms of packet counts), the estimated queuing 358 delay follows a non-linear warping: 360 / d_queue, if d_queue |||||||||=================> 620 +----------+ -----------+ RTP packets 621 Rate Shaping Buffer 623 Figure 4: NADA Sender Structure 625 5.2.1. Rate shaping buffer 627 The operation of the live video encoder is out of the scope of the 628 design for the congestion control scheme in NADA. Instead, its 629 behavior is treated as a black box. 631 A rate shaping buffer is employed to absorb any instantaneous 632 mismatch between encoder rate output r_vout and regulated sending 633 rate r_send. Its current level of occupancy is measured in bytes and 634 is denoted as buffer_len. 636 A large rate shaping buffer contributes to higher end-to-end delay, 637 which may harm the performance of real-time media communications. 638 Therefore, the sender has a strong incentive to prevent the rate 639 shaping buffer from building up. The mechanisms adopted are: 641 o To deplete the rate shaping buffer faster by increasing the 642 sending rate r_send; and 644 o To limit incoming packets of the rate shaping buffer by reducing 645 the video encoder target rate r_vin. 647 5.2.2. Adjusting video target rate and sending rate 649 If the level of occupancy in the rate shaping buffer is accessible at 650 the sender, such information can be leveraged to further adjust the 651 target rate of the live video encoder r_vin as well as the actual 652 sending rate r_send. The purpose of such adjustments is to mitigate 653 the additional latencies introduced by the rate shaping buffer. The 654 amount of rate adjustment can be calculated as follows: 656 r_diff_v = min(0.05*r_ref, BETA_V*8*buffer_len*FPS). (11) 657 r_diff_s = min(0.05*r_ref, BETA_S*8*buffer_len*FPS). (12) 658 r_vin = max(RMIN, r_ref - r_diff_v). (13) 659 r_send = min(RMAX, r_ref + r_diff_s). (14) 661 In (11) and (12), the amount of adjustment is calculated as 662 proportional to the size of the rate shaping buffer but is bounded by 663 5% of the reference rate r_ref calculated from network congestion 664 feedback alone. This ensures that the adjustment introduced by the 665 rate shaping buffer will not counteract with the core congestion 666 control process. Equations (13) and (14) indicate the influence of 667 the rate shaping buffer. A large rate shaping buffer nudges the 668 encoder target rate slightly below -- and the sending rate slightly 669 above -- the reference rate r_ref. The final video target rate 670 (r_vin) and sending rate (r_send) are further bounded within the 671 original range of [RMIN, RMAX]. 673 Intuitively, the amount of extra rate offset needed to completely 674 drain the rate shaping buffer within the duration of a single video 675 frame is given by 8*buffer_len*FPS, where FPS stands for the 676 reference frame rate of the video. The scaling parameters BETA_V and 677 BETA_S can be tuned to balance between the competing goals of 678 maintaining a small rate shaping buffer and deviating from the 679 reference rate point. Empirical observations show that the rate 680 shaping buffer for a responsive live video encoder typically stays 681 empty and only occasionally holds a large frame (e.g., when an intra- 682 frame is produced) in transit. Therefore, the rate adjustment 683 introduced by this mechanism is expected to be minor. For instance, 684 a rate shaping buffer of 2000 Bytes will lead to a rate adjustment of 685 48 Kbps given the recommended scaling parameters of BETA_V = 0.1 and 686 BETA_S = 0.1 and reference frame rate of FPS = 30. 688 5.3. Feedback Message Requirements 690 The following list of information is required for NADA congestion 691 control to function properly: 693 o Recommended rate adaptation mode (rmode): a 1-bit flag indicating 694 whether the sender should operate in accelerated ramp-up mode 695 (rmode=0) or gradual update mode (rmode=1). 697 o Aggregated congestion signal (x_curr): the most recently updated 698 value, calculated by the receiver according to Section 4.2. This 699 information is expressed with a unit of 100 microsecond (i.e., 700 1/10 of a millisecond) in 15 bits. This allows a maximum value of 701 x_curr at approximately 3.27 second. 703 o Receiving rate (r_recv): the most recently measured receiving rate 704 according to Section 5.1.3. This information is expressed with a 705 unit of bits per second (bps) in 32 bits (unsigned int). This 706 allows a maximum rate of approximately 4.3Gbps, approximately 1000 707 times of the streaming rate of a typical high-definition (HD) 708 video conferencing session today. This field can be expanded 709 further by a few more bytes, in case an even higher rate need to 710 be specified. 712 The above list of information can be accommodated by 48 bits, or 6 713 bytes, in total. They can be either included in the feedback report 714 from the receiver, or, in the case where all receiver-side 715 calculations are moved to the sender, derived from per-packet 716 information from the feedback message as defined in 717 [I-D.ietf-avtcore-cc-feedback-message]. Choice of the feedback 718 message interval DELTA is discussed in Section 6.3. A target 719 feedback interval of DELTA=100ms is recommended. 721 6. Discussions and Further Investigations 723 This section discussed the various design choices made by NADA, 724 potential alternative variants of its implementation, and guidelines 725 on how the key algorithm parameters can be chosen. Section 8 726 recommends additional experimental setups to further explore these 727 topics. 729 6.1. Choice of delay metrics 731 The current design works with relative one-way-delay (OWD) as the 732 main indication of congestion. The value of the relative OWD is 733 obtained by maintaining the minimum value of observed OWD over a 734 relatively long time horizon and subtract that out from the observed 735 absolute OWD value. Such an approach cancels out the fixed 736 difference between the sender and receiver clocks. It has been 737 widely adopted by other delay-based congestion control approaches 738 such as [RFC6817]. As discussed in [RFC6817], the time horizon for 739 tracking the minimum OWD needs to be chosen with care: it must be 740 long enough for an opportunity to observe the minimum OWD with zero 741 standing queue along the path, and sufficiently short so as to timely 742 reflect "true" changes in minimum OWD introduced by route changes and 743 other rare events. 745 The potential drawback in relying on relative OWD as the congestion 746 signal is that when multiple flows share the same bottleneck, the 747 flow arriving late at the network experiencing a non-empty queue may 748 mistakenly consider the standing queuing delay as part of the fixed 749 path propagation delay. This will lead to slightly unfair bandwidth 750 sharing among the flows. 752 Alternatively, one could move the per-packet statistical handling to 753 the sender instead and use relative round-trip-time (RTT) in lieu of 754 relative OWD, assuming that per-packet acknowledgements are 755 available. The main drawback of RTT-based approach is the noise in 756 the measured delay in the reverse direction. 758 Note that the choice of either delay metric (relative OWD vs. RTT) 759 involves no change in the proposed rate adaptation algorithm. 760 Therefore, comparing the pros and cons regarding which delay metric 761 to adopt can be kept as an orthogonal direction of investigation. 763 6.2. Method for delay, loss, and marking ratio estimation 765 Like other delay-based congestion control schemes, performance of 766 NADA depends on the accuracy of its delay measurement and estimation 767 module. Appendix A in [RFC6817] provides an extensive discussion on 768 this aspect. 770 The current recommended practice of applying minimum filter with a 771 window size of 15 samples suffices in guarding against processing 772 delay outliers observed in wired connections. For wireless 773 connections with a higher packet delay variation (PDV), more 774 sophisticated techniques on de-noising, outlier rejection, and trend 775 analysis may be needed. 777 More sophisticated methods in packet loss ratio calculation, such as 778 that adopted by [Floyd-CCR00], will likely be beneficial. These 779 alternatives are part of the experiments this document proposes. 781 6.3. Impact of parameter values 783 In the gradual rate update mode, the parameter TAU indicates the 784 upper bound of round-trip-time (RTT) in feedback control loop. 785 Typically, the observed feedback interval delta is close to the 786 target feedback interval DELTA, and the relative ratio of delta/TAU 787 versus ETA dictates the relative strength of influence from the 788 aggregate congestion signal offset term (x_offset) versus its recent 789 change (x_diff), respectively. These two terms are analogous to the 790 integral and proportional terms in a proportional-integral (PI) 791 controller. The recommended choice of TAU=500ms, DELTA=100ms and ETA 792 = 2.0 corresponds to a relative ratio of 1:10 between the gains of 793 the integral and proportional terms. Consequently, the rate 794 adaptation is mostly driven by the change in the congestion signal 795 with a long-term shift towards its equilibrium value driven by the 796 offset term. Finally, the scaling parameter KAPPA determines the 797 overall speed of the adaptation and needs to strike a balance between 798 responsiveness and stability. 800 The choice of the target feedback interval DELTA needs to strike the 801 right balance between timely feedback and low RTCP feedback message 802 counts. A target feedback interval of DELTA=100ms is recommended, 803 corresponding to a feedback bandwidth of 16Kbps with 200 bytes per 804 feedback message --- approximately 1.6% overhead for a 1 Mbps flow. 805 Furthermore, both simulation studies and frequency-domain analysis in 806 [IETF-95] have established that a feedback interval below 250ms 807 (i.e., more frequently than 4 feedback messages per second) will not 808 break up the feedback control loop of NADA congestion control. 810 In calculating the non-linear warping of delay in (1), the current 811 design uses fixed values of QTH for determining whether to perform 812 the non-linear warping). Its value should be carefully tuned for 813 different operational enviornments (e.g., over wired vs. wireless 814 connections), so as to avoid the potential risk of prematurely 815 discounting the congestion signal level. It is possible to adapt its 816 value based on past observed patterns of queuing delay in the 817 presence of packet losses. It needs to be noted that the non-linear 818 warping mechanism may lead to multiple NADA streams stuck in loss- 819 based mode when competing against each other. 821 In calculating the aggregate congestion signal x_curr, the choice of 822 DMARK and DLOSS influence the steady-state packet loss/marking ratio 823 experienced by the flow at a given available bandwidth. Higher 824 values of DMARK and DLOSS result in lower steady-state loss/marking 825 ratios, but are more susceptible to the impact of individual packet 826 loss/marking events. While the value of DMARK and DLOSS are fixed 827 and predetermined in the current design, this document also 828 encourages futher explorations of a scheme for automatically tuning 829 these values based on desired bandwidth sharing behavior in the 830 presence of other competing loss-based flows (e.g., loss-based TCP). 832 6.4. Sender-based vs. receiver-based calculation 834 In the current design, the aggregated congestion signal x_curr is 835 calculated at the receiver, keeping the sender operation completely 836 independent of the form of actual network congestion indications 837 (delay, loss, or marking) in use. 839 Alternatively, one can shift receiver-side calculations to the 840 sender, whereby the receiver simply reports on per-packet information 841 via periodic feedback messages as defined in 842 [I-D.ietf-avtcore-cc-feedback-message]. Such an approach enables 843 interoperability amongst senders operating on different congestion 844 control schemes, but requires slightly higher overhead in the 845 feedback messages. See additional discussions in 846 [I-D.ietf-avtcore-cc-feedback-message] regarding the desired format 847 of the feedback messages and the recommended feedback intervals. 849 6.5. Incremental deployment 851 One nice property of NADA is the consistent video endpoint behavior 852 irrespective of network node variations. This facilitates gradual, 853 incremental adoption of the scheme. 855 Initially, the proposed congestion control mechanism can be 856 implemented without any explicit support from the network, and relies 857 solely on observed relative one-way delay measurements and packet 858 loss ratios as implicit congestion signals. 860 When ECN is enabled at the network nodes with RED-based marking, the 861 receiver can fold its observations of ECN markings into the 862 calculation of the equivalent delay. The sender can react to these 863 explicit congestion signals without any modification. 865 Ultimately, networks equipped with proactive marking based on token 866 bucket level metering can reap the additional benefits of zero 867 standing queues and lower end-to-end delay and work seamlessly with 868 existing senders and receivers. 870 7. Reference Implementations 872 The NADA scheme has been implemented in both [ns-2] and [ns-3] 873 simulation platforms. The implementation in ns-2 hosts the 874 calculations as descirbed in Section 4.2 at the receiver side, 875 whereas the implementation in ns-3 hosts these receiver-side 876 calculations at the sender for the sake of interoperability. 877 Extensive ns-2 simulation evaluations of an earlier version of the 878 draft are documented in [Zhu-PV13]. An open source implementation of 879 NADA as part of a ns-3 module is available at [ns3-rmcat]. 880 Evaluation results of the current draft based on ns-3 are presented 881 in [IETF-90] and [IETF-91] for wired test cases as documented in 882 [I-D.ietf-rmcat-eval-test]. Evaluation results of NADA over WiFi- 883 based test cases as defined in [I-D.ietf-rmcat-wireless-tests] are 884 presented in [IETF-93]. These simulation-based evaluations have 885 shown that NADA flows can obtain their fair share of bandwidth when 886 competing against each other. They typically adapt fast in reaction 887 to the arrival and departure of other flows, and can sustain a 888 reasonable throughput when competing against loss-based TCP flows. 890 [IETF-90] describes the implemention and evaluation of NADA in a lab 891 setting. Preliminary evaluation results of NADA in single-flow and 892 multi-flow test scenarios have been presented in [IETF-91]. 894 A reference implementation of NADA has been carried out by modifying 895 the WebRTC module embedded in the Mozilla open source browser. 896 Presentations from [IETF-103] and [IETF-105] document real-world 897 evaluations of the modified browser driven by NADA. The experimental 898 setting involve remote connections with endpoints over either home or 899 enterprise wireless networks. These evaluations validate the 900 effectiveness of NADA flows in recovering quickly from throughput 901 drops caused by intermittent delay spikes over the last-hop wirelss 902 connections. 904 8. Suggested Experiments 906 NADA has been extensively evaluated under various test scenarios, 907 including the collection of test cases specified by 908 [I-D.ietf-rmcat-eval-test] and the subset of WiFi-based test cases in 909 [I-D.ietf-rmcat-wireless-tests]. Additional evaluations have been 910 carried out to characterize how NADA interacts with various active 911 queue management (AQM) schemes such as RED, CoDel, and PIE. Most of 912 these evaluations have been carried out in simulators. A few key 913 test cases have been evaluated in lab environments with 914 implementations embedded in video conferencing clients. It is 915 strongly recommended to carry out implementation and experimentation 916 of NADA in real-world settings. Such exercise will provide insights 917 on how to choose or automatically adapte the values of the key 918 algorithm parameters (see list in Figure 3) as discussed in 919 Section 6. 921 Additional experiments are suggested for the following scenarios and 922 preferrably over real-world networks: 924 o Experiments reflecting the set up of a typical WAN connection. 926 o Experiments with ECN marking capability turned on at the network 927 for existing test cases. 929 o Experiments with multiple NADA streams bearing different user- 930 specified priorities. 932 o Experiments with additional access technologies, especially over 933 cellular networks such as 3G/LTE. 935 o Experiments with various media source contents, including audio 936 only, audio and video, and application content sharing (e.g., 937 slide shows). 939 9. IANA Considerations 941 This document makes no request of IANA. 943 10. Security Considerations 945 The rate adaptation mechanism in NADA relies on feedback from the 946 receiver. As such, it is vulnerable to attacks where feedback 947 messages are hijacked, replaced, or intentionally injected with 948 misleading information resulting in denial of service, similar to 949 those that can affect TCP. It is therefore RECOMMENDED that the RTCP 950 feedback message is at least integrity checked. In addition, 951 [I-D.ietf-avtcore-cc-feedback-message] discusses the potential risk 952 of a receiver providing misleading congestion feedback information 953 and the mechanisms for mitigating such risks. 955 The modification of sending rate based on send-side rate shaping 956 buffer may lead to temporary excessive congestion over the network in 957 the presence of a unresponsive video encoder. However, this effect 958 can be mitigated by limiting the amount of rate modification 959 introduced by the rate shaping buffer, bounding the size of the rate 960 shaping buffer at the sender, and maintaining a maximum allowed 961 sending rate by NADA. 963 11. Acknowledgments 965 The authors would like to thank Randell Jesup, Luca De Cicco, Piers 966 O'Hanlon, Ingemar Johansson, Stefan Holmer, Cesar Ilharco Magalhaes, 967 Safiqul Islam, Michael Welzl, Mirja Kuhlewind, Karen Elisabeth Egede 968 Nielsen, Julius Flohr, Roland Bless, Andreas Smas, and Martin 969 Stiemerling for their valuable review comments and helpful input to 970 this specification. 972 12. Contributors 974 The following individuals have contributed to the implementation and 975 evaluation of the proposed scheme, and therefore have helped to 976 validate and substantially improve this specification. 978 Paul E. Jones of Cisco Systems 979 implemented an early version of the NADA congestion control scheme 980 and helped with its lab-based testbed evaluations. 982 Jiantao Fu of Cisco Systems helped with the 983 implementation and extensive evaluation of NADA both in Mozilla 984 web browsers and in earlier simulation-based evaluation efforts. 986 Stefano D'Aronco of ETH Zurich 987 (previously at Ecole Polytechnique Federale de Lausanne when 988 contributing to this work) helped with implementation and 989 evaluation of an early version of NADA in [ns-3]. 991 Charles Ganzhorn contributed to the 992 testbed-based evaluation of NADA during an early stage of its 993 development. 995 13. References 997 13.1. Normative References 999 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1000 Requirement Levels", BCP 14, RFC 2119, 1001 DOI 10.17487/RFC2119, March 1997, 1002 . 1004 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1005 of Explicit Congestion Notification (ECN) to IP", 1006 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1007 . 1009 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1010 Jacobson, "RTP: A Transport Protocol for Real-Time 1011 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1012 July 2003, . 1014 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1015 Friendly Rate Control (TFRC): Protocol Specification", 1016 RFC 5348, DOI 10.17487/RFC5348, September 2008, 1017 . 1019 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 1020 and K. Carlberg, "Explicit Congestion Notification (ECN) 1021 for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 1022 2012, . 1024 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1025 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1026 May 2017, . 1028 13.2. Informative References 1030 [Budzisz-TON11] 1031 Budzisz, L., Stanojevic, R., Schlote, A., Baker, F., and 1032 R. Shorten, "On the Fair Coexistence of Loss- and Delay- 1033 Based TCP", IEEE/ACM Transactions on Networking vol. 19, 1034 no. 6, pp. 1811-1824, December 2011. 1036 [Floyd-CCR00] 1037 Floyd, S., Handley, M., Padhye, J., and J. Widmer, 1038 "Equation-based Congestion Control for Unicast 1039 Applications", ACM SIGCOMM Computer Communications 1040 Review vol. 30, no. 4, pp. 43-56, October 2000. 1042 [I-D.ietf-avtcore-cc-feedback-message] 1043 Sarker, Z., Perkins, C., Singh, V., and M. Ramalho, "RTP 1044 Control Protocol (RTCP) Feedback for Congestion Control", 1045 draft-ietf-avtcore-cc-feedback-message-04 (work in 1046 progress), July 2019. 1048 [I-D.ietf-rmcat-cc-codec-interactions] 1049 Zanaty, M., Singh, V., Nandakumar, S., and Z. Sarker, 1050 "Congestion Control and Codec interactions in RTP 1051 Applications", draft-ietf-rmcat-cc-codec-interactions-02 1052 (work in progress), March 2016. 1054 [I-D.ietf-rmcat-cc-requirements] 1055 Jesup, R. and Z. Sarker, "Congestion Control Requirements 1056 for Interactive Real-Time Media", draft-ietf-rmcat-cc- 1057 requirements-09 (work in progress), December 2014. 1059 [I-D.ietf-rmcat-eval-test] 1060 Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test 1061 Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat- 1062 eval-test-10 (work in progress), May 2019. 1064 [I-D.ietf-rmcat-video-traffic-model] 1065 Zhu, X., Cruz, S., and Z. Sarker, "Video Traffic Models 1066 for RTP Congestion Control Evaluations", draft-ietf-rmcat- 1067 video-traffic-model-07 (work in progress), February 2019. 1069 [I-D.ietf-rmcat-wireless-tests] 1070 Sarker, Z., Johansson, I., Zhu, X., Fu, J., Tan, W., and 1071 M. Ramalho, "Evaluation Test Cases for Interactive Real- 1072 Time Media over Wireless Networks", draft-ietf-rmcat- 1073 wireless-tests-08 (work in progress), July 2019. 1075 [IETF-103] 1076 Zhu, X., Pan, R., Ramalho, M., Mena, S., Jones, P., Fu, 1077 J., and S. D'Aronco, "NADA Implementation in Mozilla 1078 Browser", November 2018, 1079 . 1083 [IETF-105] 1084 Zhu, X., Pan, R., Ramalho, M., Mena, S., Jones, P., Fu, 1085 J., and S. D'Aronco, "NADA Implementation in Mozilla 1086 Browser and Draft Update", July 2019, 1087 . 1090 [IETF-90] Zhu, X., Ramalho, M., Ganzhorn, C., Jones, P., and R. Pan, 1091 "NADA Update: Algorithm, Implementation, and Test Case 1092 Evalua6on Results", July 2014, 1093 . 1096 [IETF-91] Zhu, X., Pan, R., Ramalho, M., Mena, S., Ganzhorn, C., 1097 Jones, P., and S. D'Aronco, "NADA Algorithm Update and 1098 Test Case Evaluations", November 2014, 1099 . 1102 [IETF-93] Zhu, X., Pan, R., Ramalho, M., Mena, S., Ganzhorn, C., 1103 Jones, P., D'Aronco, S., and J. Fu, "Updates on NADA", 1104 July 2015, . 1107 [IETF-95] Zhu, X., Pan, R., Ramalho, M., Mena, S., Jones, P., Fu, 1108 J., D'Aronco, S., and C. Ganzhorn, "Updates on NADA: 1109 Stability Analysis and Impact of Feedback Intervals", 1110 April 2016, . 1113 [ns-2] "The Network Simulator - ns-2", 1114 . 1116 [ns-3] "The Network Simulator - ns-3", . 1118 [ns3-rmcat] 1119 Fu, J., Mena, S., and X. Zhu, "NS3 open source module of 1120 IETF RMCAT congestion control protocols", November 2017, 1121 . 1123 [RFC6660] Briscoe, B., Moncaster, T., and M. Menth, "Encoding Three 1124 Pre-Congestion Notification (PCN) States in the IP Header 1125 Using a Single Diffserv Codepoint (DSCP)", RFC 6660, 1126 DOI 10.17487/RFC6660, July 2012, 1127 . 1129 [RFC6817] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind, 1130 "Low Extra Delay Background Transport (LEDBAT)", RFC 6817, 1131 DOI 10.17487/RFC6817, December 2012, 1132 . 1134 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 1135 Recommendations Regarding Active Queue Management", 1136 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 1137 . 1139 [RFC8033] Pan, R., Natarajan, P., Baker, F., and G. White, 1140 "Proportional Integral Controller Enhanced (PIE): A 1141 Lightweight Control Scheme to Address the Bufferbloat 1142 Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017, 1143 . 1145 [RFC8290] Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys, 1146 J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler 1147 and Active Queue Management Algorithm", RFC 8290, 1148 DOI 10.17487/RFC8290, January 2018, 1149 . 1151 [RFC8593] Zhu, X., Mena, S., and Z. Sarker, "Video Traffic Models 1152 for RTP Congestion Control Evaluations", RFC 8593, 1153 DOI 10.17487/RFC8593, May 2019, 1154 . 1156 [Zhu-PV13] 1157 Zhu, X. and R. Pan, "NADA: A Unified Congestion Control 1158 Scheme for Low-Latency Interactive Video", in Proc. IEEE 1159 International Packet Video Workshop (PV'13) San Jose, CA, 1160 USA, December 2013. 1162 Appendix A. Network Node Operations 1164 NADA can work with different network queue management schemes and 1165 does not assume any specific network node operation. As an example, 1166 this appendix describes three variants of queue management behavior 1167 at the network node, leading to either implicit or explicit 1168 congestion signals. It needs to be acknowledged that NADA has not 1169 yet been tested with non-probabilistic ECN marking behaviors. 1171 In all three flavors described below, the network queue operates with 1172 the simple first-in-first-out (FIFO) principle. There is no need to 1173 maintain per-flow state. The system can scale easily with a large 1174 number of video flows and at high link capacity. 1176 A.1. Default behavior of drop tail queues 1178 In a conventional network with drop tail or RED queues, congestion is 1179 inferred from the estimation of end-to-end delay and/or packet loss. 1180 Packet drops at the queue are detected at the receiver, and 1181 contributes to the calculation of the aggregated congestion signal 1182 x_curr. No special action is required at network node. 1184 A.2. RED-based ECN marking 1186 In this mode, the network node randomly marks the ECN field in the IP 1187 packet header following the Random Early Detection (RED) algorithm 1188 [RFC7567]. Calculation of the marking probability involves the 1189 following steps: 1191 on packet arrival: 1192 update smoothed queue size q_avg as: 1193 q_avg = w*q + (1-w)*q_avg. 1195 calculate marking probability p as: 1197 / 0, if q < q_lo; 1198 | 1199 | q_avg - q_lo 1200 p= < p_max*--------------, if q_lo <= q < q_hi; 1201 | q_hi - q_lo 1202 | 1203 \ p = 1, if q >= q_hi. 1205 Here, q_lo and q_hi corresponds to the low and high thresholds of 1206 queue occupancy. The maximum marking probability is p_max. 1208 The ECN markings events will contribute to the calculation of an 1209 equivalent delay x_curr at the receiver. No changes are required at 1210 the sender. 1212 A.3. Random Early Marking with Virtual Queues 1214 Advanced network nodes may support random early marking based on a 1215 token bucket algorithm originally designed for Pre-Congestion 1216 Notification (PCN) [RFC6660]. The early congestion notification 1217 (ECN) bit in the IP header of packets are marked randomly. The 1218 marking probability is calculated based on a token-bucket algorithm 1219 originally designed for the Pre-Congestion Notification (PCN) 1220 [RFC6660]. The target link utilization is set as 90%; the marking 1221 probability is designed to grow linearly with the token bucket size 1222 when it varies between 1/3 and 2/3 of the full token bucket limit. 1224 Calculation of the marking probability involves the following steps: 1226 upon packet arrival: 1227 meter packet against token bucket (r,b); 1229 update token level b_tk; 1231 calculate the marking probability as: 1233 / 0, if b-b_tk < b_lo; 1234 | 1235 | b-b_tk-b_lo 1236 p = < p_max* --------------, if b_lo<= b-b_tk =b_hi. 1241 Here, the token bucket lower and upper limits are denoted by b_lo and 1242 b_hi, respectively. The parameter b indicates the size of the token 1243 bucket. The parameter r is chosen to be below capacity, resulting in 1244 slight under-utilization of the link. The maximum marking 1245 probability is p_max. 1247 The ECN markings events will contribute to the calculation of an 1248 equivalent delay x_curr at the receiver. No changes are required at 1249 the sender. The virtual queuing mechanism from the PCN-based marking 1250 algorithm will lead to additional benefits such as zero standing 1251 queues. 1253 Authors' Addresses 1255 Xiaoqing Zhu 1256 Cisco Systems 1257 12515 Research Blvd., Building 4 1258 Austin, TX 78759 1259 USA 1261 Email: xiaoqzhu@cisco.com 1263 Rong Pan * 1264 * Pending affiliation change. 1266 Email: rong.pan@gmail.com 1268 Michael A. Ramalho 1269 Cisco Systems, Inc. 1270 8000 Hawkins Road 1271 Sarasota, FL 34241 1272 USA 1274 Phone: +1 919 476 2038 1275 Email: mar42@cornell.edu 1277 Sergio Mena de la Cruz 1278 Cisco Systems 1279 EPFL, Quartier de l'Innovation, Batiment E 1280 Ecublens, Vaud 1015 1281 Switzerland 1283 Email: semena@cisco.com