idnits 2.17.1 draft-ietf-rmcat-eval-test-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 9 instances of too long lines in the document, the longest one being 4 characters in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 96: '... in this memo SHOULD be used to eval...' RFC 2119 keyword, line 223: '...enerated on the end-to-end path. MUST...' RFC 2119 keyword, line 316: '...These attributes MUST be well defined,...' RFC 2119 keyword, line 321: '...ue of such a set MUST be treated as di...' RFC 2119 keyword, line 335: '... implementers MUST log enough inform...' (4 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 21, 2018) is 2136 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: '300ms' is mentioned on line 1005, but not defined == Missing Reference: '1000ms' is mentioned on line 1005, but not defined == Outdated reference: A later version (-14) exists of draft-ietf-rmcat-eval-criteria-07 == Outdated reference: A later version (-07) exists of draft-ietf-rmcat-video-traffic-model-04 == Outdated reference: A later version (-11) exists of draft-ietf-rmcat-wireless-tests-04 Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Z. Sarker 3 Internet-Draft Ericsson AB 4 Intended status: Informational V. Singh 5 Expires: December 23, 2018 callstats.io 6 X. Zhu 7 M. Ramalho 8 Cisco Systems 9 June 21, 2018 11 Test Cases for Evaluating RMCAT Proposals 12 draft-ietf-rmcat-eval-test-06 14 Abstract 16 The Real-time Transport Protocol (RTP) is used to transmit media in 17 multimedia telephony applications, these applications are typically 18 required to implement congestion control. This document describes 19 the test cases to be used in the performance evaluation of such 20 congestion control algorithms. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at https://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on December 23, 2018. 39 Copyright Notice 41 Copyright (c) 2018 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (https://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 3. Structure of Test cases . . . . . . . . . . . . . . . . . . . 3 59 4. Recommended Evaluation Settings . . . . . . . . . . . . . . . 7 60 4.1. Evaluation metrics . . . . . . . . . . . . . . . . . . . 8 61 4.2. Path characteristics . . . . . . . . . . . . . . . . . . 8 62 4.3. Media source . . . . . . . . . . . . . . . . . . . . . . 9 63 5. Basic Test Cases . . . . . . . . . . . . . . . . . . . . . . 10 64 5.1. Variable Available Capacity with a Single Flow . . . . . 10 65 5.2. Variable Available Capacity with Multiple Flows . . . . . 13 66 5.3. Congested Feedback Link with Bi-directional Media Flows . 14 67 5.4. Competing Media Flows with same Congestion Control 68 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 17 69 5.5. Round Trip Time Fairness . . . . . . . . . . . . . . . . 19 70 5.6. Media Flow Competing with a Long TCP Flow . . . . . . . . 21 71 5.7. Media Flow Competing with Short TCP Flows . . . . . . . . 23 72 5.8. Media Pause and Resume . . . . . . . . . . . . . . . . . 25 73 6. Other potential test cases . . . . . . . . . . . . . . . . . 27 74 6.1. Media Flows with Priority . . . . . . . . . . . . . . . . 27 75 6.2. Explicit Congestion Notification Usage . . . . . . . . . 27 76 6.3. Multiple Bottlenecks . . . . . . . . . . . . . . . . . . 27 77 7. Wireless Access Links . . . . . . . . . . . . . . . . . . . . 29 78 8. Security Considerations . . . . . . . . . . . . . . . . . . . 29 79 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 80 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 30 81 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 30 82 11.1. Normative References . . . . . . . . . . . . . . . . . . 30 83 11.2. Informative References . . . . . . . . . . . . . . . . . 31 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 86 1. Introduction 88 This memo describes a set of test cases for evaluating congestion 89 control algorithm proposals for real-time interactive media. It is 90 based on the guidelines enumerated in [I-D.ietf-rmcat-eval-criteria] 91 and the requirements discussed in [I-D.ietf-rmcat-cc-requirements]. 92 The test cases cover basic usage scenarios and are described using a 93 common structure, which allows for additional test cases to be added 94 to those described herein to accommodate other topologies and/or the 95 modeling of different path characteristics. The described test cases 96 in this memo SHOULD be used to evaluate any proposed congestion 97 control algorithm for real-time interactive media. 99 2. Terminology 101 The terminology defined in RTP [RFC3550], RTP Profile for Audio and 102 Video Conferences with Minimal Control [RFC3551], RTCP Extended 103 Report (XR) [RFC3611], Extended RTP Profile for RTCP-based Feedback 104 (RTP/AVPF) [RFC4585], and Support for Reduced-Size RTCP [RFC5506] 105 apply. 107 3. Structure of Test cases 109 All the test cases in this document follow a basic structure allowing 110 implementers to describe a new test scenario without repeatedly 111 explaining common attributes. The structure includes a general 112 description section that describes the test case and its motivation. 113 Additionally the test case defines a set of attributes that 114 characterize the testbed, for example, the network path between 115 communicating peers and the diverse traffic sources. 117 o Define the test case: 119 * General description: describes the motivation and the goals of 120 the test case. 122 * Expected behavior: describes the desired rate adaptation 123 behavior. 125 * Define a list of metrics to evaluate the desired behavior: this 126 indicates the minimum set of metrics (e.g., link utilization, 127 media sending rate) that a proposed algorithm needs to measure 128 to validate the expected rate adaptation behavior. It should 129 also indicate the time granularity (e.g., averaged over 10ms, 130 100ms, or 1s) for measuring certain metrics. Typical 131 measurement interval is 200ms. 133 o Define testbed topology: every test case needs to define an 134 evaluation testbed topology. Figure 1 shows such an evaluation 135 topology. In this evaluation topology, S1..Sn are traffic 136 sources. These sources generate media traffic and use either 137 congestion control algorithm under investigation. R1..Rn are the 138 corresponding receivers. A test case can have one or more such 139 traffic sources (S) and their corresponding receivers (R). The 140 path from the source to destination is denoted as "forward" and 141 the path from a destination to a source is denoted as "backward". 142 The following basic structure of the test case has been described 143 from the perspective of media generating endpoints attached on the 144 left-hand side of Figure 1. In this setup, the media flows are 145 transported in forward direction and corresponding feedback/ 146 control messages are transported in the backward direction. 147 However, it is also possible to set up the test with media in both 148 forward and backward directions. In that case, unless otherwise 149 specified by the test case, it is expected that the backward path 150 does not introduce any congestion related impairments and has 151 enough capacity to accommodate both media and feedback/control 152 messages. It should be noted that depending on the test cases it 153 is possible to have different path characteristics in either of 154 the directions. 156 +---+ +---+ 157 |S1 |====== \ Forward --> / =======|R1 | 158 +---+ \\ // +---+ 159 \\ // 160 +---+ +-----+ +-----+ +---+ 161 |S2 |=======| A |------------------------------>| B |=======|R2 | 162 +---+ | |<------------------------------| | +---+ 163 +-----+ +-----+ 164 (...) // \\ (...) 165 // <-- Backward \\ 166 +---+ // \\ +---+ 167 |Sn |====== / \ ======|Rn | 168 +---+ +---+ 170 Figure 1: Example of A Testbed Topology 172 In a testbed environment where real equipments are used to create 173 a laboratory, there may exist a significant amount of traffic on 174 portions of the network path between the endpoints that is not 175 desired for the purposes of the tests described in the document. 176 Some of this traffic may be generated by other processes on the 177 endpoints themselves (e.g., discovery protocols) or by other 178 endpoints not presently under test. It is recommended not to 179 route traffic generated by endpoints that are not under test 180 through the test bed and route those traffic generated by the 181 endpoints under test around the bottleneck links specified herein. 183 o Define testbed attributes: 185 * Duration: defines the duration of the test in seconds. 187 * Path characteristics: defines the end-to-end transport level 188 path characteristics of the testbed for a particular test case. 189 Two sets of attributes describe the path characteristics, one 190 for the forward path and the other for the backward path. The 191 path characteristics for a particular path direction is 192 applicable to all the Sources "S" sending traffic on that path. 193 If only one attribute is specified, it is used for both path 194 directions, however, unless specified the reverse path has no 195 capacity restrictions and no path loss. 197 + Path direction: forward or backward. 199 + Bottleneck-link capacity: defines minimum capacity of the 200 end-to-end path 202 + Reference bottleneck capacity: defines a reference value for 203 the bottleneck capacity for test cases with time-varying 204 bottleneck capacities. All bottleneck capacities will be 205 specified as a ratio with respect to the reference capacity 206 value. 208 + One-way propagation delay: describes the end-to-end latency 209 along the path when network queues are empty, i.e., the time 210 it takes for a packet to go from the sender to the receiver 211 without encountering any queuing delay. 213 + Maximum end-to-end jitter: defines the maximum jitter that 214 can be observed along the path. 216 + Bottleneck queue type: for example, Droptail, FQ-CoDel, or 217 PIE. 219 + Bottleneck queue size: defines the size of queue in terms of 220 queuing time when the queue is full (in milliseconds). 222 + Path loss ratio: characterizes the non-congested, additive, 223 losses to be generated on the end-to-end path. MUST 224 describe the loss pattern or loss model used to generate the 225 losses. 227 * Application-related: defines the traffic source behavior for 228 implementing the test case 230 + Media traffic Source: defines the characteristics of the 231 media sources. When using more than one media source, the 232 different attributes are enumerated separately for each 233 different media source. 235 - Media type: Video/Voice 237 - Media flow direction: forward, backward or both. 239 - Number of media sources: defines the total number of 240 media sources 242 - Media codec: Constant Bit Rate (CBR) or Variable Bit Rate 243 (VBR) 245 - Media source behavior: describes the media encoder 246 behavior. It defines the main parameters that affect the 247 adaptation behavior. This may include but is not limited 248 to: 250 o Adaptability: describes the adaptation options. For 251 example, in the case of video it defines the following 252 ranges of adaptation: bit rate, frame rate, video 253 resolution. Similarly, in the case of voice, it 254 defines the range of bit rate adaptation, the sampling 255 rate variation, and the variation in packetization 256 interval. 258 o Output variation : for a VBR encoder it defines the 259 encoder output variation from the average target rate 260 over a particular measurement interval. For example, 261 on average the encoder output may vary between 5% to 262 15% above or below the average target bit rate when 263 measured over a 100 ms time window. The time interval 264 over which the variation is specified must be 265 provided. 267 o Responsiveness to a new bit rate request: the lag in 268 time between a new bit rate request from the 269 congestion control algorithm and actual rate changes 270 in encoder output. Depending on the encoder, this 271 value may be specified in absolute time (e.g. 10ms to 272 1000ms) or other appropriate metric (e.g. next frame 273 interval time). 275 More detailed discussions on expected media source 276 behavior, including those from synthetic video traffic 277 sources, is at [I-D.ietf-rmcat-video-traffic-model]. 279 - Media content: describes the chosen media sequences; For 280 example, test sequences are available at: [xiph-seq] and 281 [HEVC-seq]. 283 - Media timeline: describes the point when the media source 284 is introduced and removed from the testbed. For example, 285 the media source may start transmitting immediately when 286 the test case begins, or after a few seconds. 288 - Startup behavior: the media starts at a defined bit rate, 289 which may be the minimum, maximum bit rate, or a value in 290 between (in Kbps). 292 + Competing traffic source: describes the characteristics of 293 the competing traffic source, the different types of 294 competing flows are enumerated in 295 [I-D.ietf-rmcat-eval-criteria]. 297 - Traffic direction: forward, backward or both. 299 - Type of sources: defines the types of competing traffic 300 sources. Types of competing traffic flows are listed in 301 [I-D.ietf-rmcat-eval-criteria]. For example, the number 302 of TCP flows connected to a web browser, the mean size 303 and distribution of the content downloaded. 305 - Number of sources: defines the total number of competing 306 sources of each media type per traffic direction. 308 - Congestion control: enumerates the congestion control 309 used by each type of competing traffic. 311 - Traffic timeline: describes when the competing traffic 312 starts and ends in the test case. 314 * Additional attributes: describes attributes essential for 315 implementing a test case which are not included in the above 316 structure. These attributes MUST be well defined, so that the 317 other implementers of that particular test case are able to 318 implement it easily. 320 Any attribute can have a set of values (enclosed within "[]"). Each 321 member value of such a set MUST be treated as different value for the 322 same attribute. It is desired to run separate tests for each such 323 attribute value. 325 The test cases described in this document follow the above structure. 327 4. Recommended Evaluation Settings 329 This section describes recommended test case settings and could be 330 overwritten by the respective test cases. 332 4.1. Evaluation metrics 334 To evaluate the performance of the candidate algorithms the 335 implementers MUST log enough information to visualize the following 336 metrics at a fine enough time granularity: 338 1. Flow level: 340 A. End-to-end delay for the congestion controlled media flow. 342 B. Variation in sending bit rate and goodput. Mainly observing 343 the frequency and magnitude of oscillations. 345 C. Packet losses observed at the receiving endpoint. 347 D. Feedback message overhead. 349 E. Convergence time - time to reach steady state for the 350 congestion controlled media flow(s). 352 2. Transport level: 354 A. Bandwidth utilization. 356 B. Queue length (milliseconds at specified path capacity): 358 + average over the length of the session. 360 + 5 and 95 percentile. 362 + median, maximum, minimum. 364 4.2. Path characteristics 366 Each path between a sender and receiver as described in Figure 1 have 367 the following characteristics unless otherwise specified in the test 368 case. 370 o Path direction: forward and backward. 372 o Reference bottleneck capacity: 1Mbps. 374 o One-Way propagation delay: 50ms. Implementers are encouraged to 375 run the experiment with additional propagation delays mentioned in 376 [I-D.ietf-rmcat-eval-criteria] 378 o Maximum end-to-end jitter: 30ms. Jitter models are described in 379 [I-D.ietf-rmcat-eval-criteria] 381 o Bottleneck queue type: Drop tail. Implementers are encouraged to 382 run the experiment with other AQM schemes, such as FQ-CoDel and 383 PIE. 385 o Bottleneck queue size: 300ms. 387 o Path loss ratio: 0%. 389 Examples of additional network parameters are discussed in 390 [I-D.ietf-rmcat-eval-criteria]. 392 For test cases involving time-varying bottleneck capacity, all 393 capacity values are specified as a ratio with respect to a reference 394 capacity value, so as to allow flexible scaling of capacity values 395 along with media source rate range. There exist two different 396 mechanisms for inducing path capacity variation: a) by explicitly 397 modifying the value of physical link capacity; or b) by introducing 398 background non-adaptive UDP traffic with time-varying traffic rate. 399 Implementers are encouraged to run the experiments with both 400 mechanisms for test cases specified in Section 5.1, Section 5.2, and 401 Section 5.3. 403 4.3. Media source 405 Unless otherwise specified, each test case will include one or more 406 media sources as described below. 408 o Media type: Video 410 * Media codec: VBR 412 * Media source behavior: 414 + Adaptability: 416 - Bit rate range: 150 Kbps - 1.5 Mbps. In real-life 417 applications the bit rate range can vary a lot depending 418 on the provided service, for example, the maximum bit 419 rate can be up to 4Mbps. However, for running tests to 420 evaluate the congestion control algorithms it is more 421 important to have a look at how they are reacting to 422 certain amount of bandwidth change. Also it is possible 423 that the media traffic generator used in a particular 424 simulator or testbed is not capable of generating higher 425 bit rate. Hence we have selected a suitable bit rate 426 range typical of consumer-grade video conferencing 427 applications in designing the test case. If a different 428 bit rate range is used in the test cases, then the end- 429 to-end path capacity values will also need to be scaled 430 accordingly. 432 - Frame resolution: 144p - 720p (or 1080p). This 433 resolution range is selected based on the bit rate range. 434 If a different bit rate range is used in the test cases 435 then the frame resolution range also need to be selected 436 suitably. 438 - Frame rate: 10fps - 30fps. This frame rate range is 439 selected based on the bit rate range. If a different bit 440 rate range is used in the test cases then the frame rate 441 range also need to be adjusted suitably. 443 + Variation from target bit rate: +/-5%. Unless otherwise 444 specified in the test case(s), bit rate variation SHOULD be 445 calculated over one (1) second period of time. 447 + Responsiveness to new bit rate request: 100ms 449 * Media content: The media content should represent a typical 450 video conversational scenario with head and shoulder movement. 451 We recommend to use Foreman video sequence. 453 * Media startup behavior: 150Kbps. It should be noted that 454 applications can use smart ways to select an optimal startup 455 bit rate value for a certain network condition. In such cases 456 the candidate proposals MAY show the effectiveness of such 457 smart approach as an additional information for the evaluation 458 process. 460 o Media type: Audio 462 * Media codec: CBR 464 * Media bit rate: 20Kbps 466 5. Basic Test Cases 468 5.1. Variable Available Capacity with a Single Flow 470 In this test case the bottleneck-link capacity between the two 471 endpoints varies over time. This test is designed to measure the 472 responsiveness of the candidate algorithm. This test tries to 473 address the requirements in [I-D.ietf-rmcat-cc-requirements], which 474 requires the algorithm to adapt the flow(s) and provide lower end-to- 475 end latency when there exists: 477 o an intermediate bottleneck 479 o change in available capacity (e.g., due to interface change, 480 routing change, abrupt arrival/departure of background non- 481 adaptive traffic). 483 o maximum media bit rate is greater than link capacity. In this 484 case, the application will attempt to ramp up to its maximum bit 485 rate, since the link capacity is limited to a value lower, the 486 congestion control scheme is expected to stabilize the sending bit 487 rate close to the available bottleneck capacity. 489 It should be noted that the exact variation in available capacity due 490 to any of the above depends on the underlying technologies. Hence, 491 we describe a set of known factors, which may be extended to devise a 492 more specific test case targeting certain behaviors in a certain 493 network environment. 495 Expected behavior: the candidate algorithm is expected to detect the 496 path capacity constraint, converges to the bottleneck link's capacity 497 and adapt the flow to avoid unwanted oscillation when the sending bit 498 rate is approaching the bottleneck link's capacity. The oscillations 499 occur when the media flow(s) attempts to reach its maximum bit rate 500 but overshoots the usage of the available bottleneck capacity then to 501 rectify, it reduces the bit rate and starts to ramp up again. 503 Evaluation metrics : as described in Section 4.1. 505 Testbed topology: One media source S1 is connected to the 506 corresponding R1. The media traffic is transported over the forward 507 path and corresponding feedback/control traffic is transported over 508 the backward path. 510 Forward --> 511 +---+ +-----+ +-----+ +---+ 512 |S1 |=======| A |------------------------------>| B |=======|R1 | 513 +---+ | |<------------------------------| | +---+ 514 +-----+ +-----+ 515 <-- Backward 517 Figure 2: Testbed Topology for Limited Link Capacity 519 Testbed attributes: 521 o Test duration: 100s 523 o Path characteristics: as described in Section 4.2 524 o Application-related: 526 * Media Traffic: 528 + Media type: Video 530 - Media direction: forward. 532 - Number of media sources: one (1) 534 - Media timeline: 536 o Start time: 0s. 538 o End time: 99s. 540 + Media type: Audio 542 - Media direction: forward. 544 - Number of media sources: one (1) 546 - Media timeline: 548 o Start time: 0s. 550 o End time: 99s. 552 * Competing traffic: 554 + Number of sources : zero (0) 556 o Test Specific Information: 558 * One-way propagation delay: [ 50 ms, 100 ms]. on the forward 559 path direction 561 * This test uses bottleneck path capacity variation as listed in 562 Table 1 564 * When using background non-adaptive UDP traffic to induce time- 565 varying bottleneck , the physical path capacity remains at 566 4Mbps and the UDP traffic source rate changes over time as 567 (4-x)Mbps, where x is the bottleneck capacity specified in 568 Table 1 570 +--------------------+--------------+-----------+-------------------+ 571 | Variation pattern | Path | Start | Path capacity | 572 | index | direction | time | ratio | 573 +--------------------+--------------+-----------+-------------------+ 574 | One | Forward | 0s | 1.0 | 575 | Two | Forward | 40s | 2.5 | 576 | Three | Forward | 60s | 0.6 | 577 | Four | Forward | 80s | 1.0 | 578 +--------------------+--------------+-----------+-------------------+ 580 Table 1: Path capacity variation pattern for forward direction 582 5.2. Variable Available Capacity with Multiple Flows 584 This test case is similar to Section 5.1. However in addition this 585 test will also consider persistent network load due to competing 586 traffic. 588 Expected behavior: the candidate algorithm is expected to detect the 589 variation in available capacity and adapt the media stream(s) 590 accordingly. The flows stabilize around their maximum bit rate as 591 the maximum link capacity is large enough to accommodate the flows. 592 When the available capacity drops, the flows adapt by decreasing 593 their sending bit rate, and when congestion disappears, the flows are 594 again expected to ramp up. 596 Evaluation metrics : as described in Section 4.1. 598 Testbed Topology: Two (2) media sources S1 and S2 are connected to 599 their corresponding destinations R1 and R2. The media traffic is 600 transported over the forward path and corresponding feedback/control 601 traffic is transported over the backward path. 603 +---+ +---+ 604 |S1 |===== \ / =======|R1 | 605 +---+ \\ Forward --> // +---+ 606 \\ // 607 +-----+ +-----+ 608 | A |------------------------------>| B | 609 | |<------------------------------| | 610 +-----+ +-----+ 611 // \\ 612 // <-- Backward \\ 613 +---+ // \\ +---+ 614 |S2 |====== / \ ======|R2 | 615 +---+ +---+ 617 Figure 3: Testbed Topology for Variable Available Capacity 619 Testbed attributes: 621 Testbed attributes are similar as described in Section 5.1 except the 622 test specific capacity variation setup. 624 Test Specific Information: This test uses path capacity variation as 625 listed in Table 2 with a corresponding end time of 125 seconds. The 626 reference bottleneck capacity is 2Mbps. When using background non- 627 adaptive UDP traffic to induce time-varying bottleneck for congestion 628 controlled media flows, the physical path capacity is 4Mbps and the 629 UDP traffic source rate changes over time as (4-x)Mbps, where x is 630 the bottleneck capacity specified in Table 2. 632 +--------------------+--------------+-----------+-------------------+ 633 | Variation pattern | Path | Start | Path capacity | 634 | index | direction | time | ratio | 635 +--------------------+--------------+-----------+-------------------+ 636 | One | Forward | 0s | 2.0 | 637 | Two | Forward | 25s | 1.0 | 638 | Three | Forward | 50s | 1.75 | 639 | Four | Forward | 75s | 0.5 | 640 | Five | Forward | 100s | 1.0 | 641 +--------------------+--------------+-----------+-------------------+ 643 Table 2: Path capacity variation pattern for forward direction 645 5.3. Congested Feedback Link with Bi-directional Media Flows 647 Real-time interactive media uses RTP hence it is assumed that RTCP, 648 RTP header extension or such would be used by the congestion control 649 algorithm in the backchannel. Due to asymmetric nature of the link 650 between communicating peers it is possible for a participating peer 651 to not receive such feedback information due to an impaired or 652 congested backchannel (even when the forward channel might not be 653 impaired). This test case is designed to observe the candidate 654 congestion control behavior in such an event. 656 It is expected that the candidate algorithms are able to cope with 657 the lack of feedback information and adapt to minimize the 658 performance degradation of media flows in the forward channel. 660 It should be noted that for this test case: logs are compared with 661 the reference case, i.e, when the backward channel has no 662 impairments. 664 Evaluation metrics : as described in Section 4.1. 666 Testbed topology: One (1) media source S1 is connected to 667 corresponding R1, but both endpoints are additionally receiving and 668 sending data, respectively. The media traffic (S1->R1) is 669 transported over the forward path and corresponding feedback/control 670 traffic is transported over the backward path. Likewise media 671 traffic (S2->R2) is transported over the backward path and 672 corresponding feedback/control traffic is transported over the 673 forward path. 675 +---+ +---+ 676 |S1 |===== \ Forward --> / =======|R1 | 677 +---+ \\ // +---+ 678 \\ // 679 +-----+ +-----+ 680 | A |------------------------------>| B | 681 | |<------------------------------| | 682 +-----+ +-----+ 683 // \\ 684 // <-- Backward \\ 685 +---+ // \\ +---+ 686 |R2 |===== / \ ======|S2 | 687 +---+ +---+ 689 Figure 4: Testbed Topology for Congested Feedback Link 691 Testbed attributes: 693 o Test duration: 100s 695 o Path characteristics: 697 * Reference bottleneck capacity: 1Mbps. 699 o Application-related: 701 * Media Source: 703 + Media type: Video 705 - Media direction: forward and backward 707 - Number of media sources: two (2) 709 - Media timeline: 711 o Start time: 0s. 713 o End time: 99s. 715 + Media type: Audio 717 - Media direction: forward and backward 719 - Number of media sources: two (2) 721 - Media timeline: 723 o Start time: 0s. 725 o End time: 99s. 727 * Competing traffic: 729 + Number of sources : zero (0) 731 o Test Specific Information: this test uses path capacity variations 732 to create congested feedback link. Table 3 lists the variation 733 patterns applied to the forward path and Table 4 lists the 734 variation patterns applied to the backward path. When using 735 background non-adaptive UDP traffic to induce time-varying 736 bottleneck for congestion controlled media flows, the physical 737 path capacity is 4Mbps for both directions and the UDP traffic 738 source rate changes over time as (4-x)Mbps in each direction, 739 where x is the bottleneck capacity specified in Table 4. 741 +--------------------+--------------+-----------+-------------------+ 742 | Variation pattern | Path | Start | Path capacity | 743 | index | direction | time | ratio | 744 +--------------------+--------------+-----------+-------------------+ 745 | One | Forward | 0s | 2.0 | 746 | Two | Forward | 20s | 1.0 | 747 | Three | Forward | 40s | 0.5 | 748 | Four | Forward | 60s | 2.0 | 749 +--------------------+--------------+-----------+-------------------+ 751 Table 3: Path capacity variation pattern for forward direction 753 +--------------------+--------------+-----------+-------------------+ 754 | Variation pattern | Path | Start | Path capacity | 755 | index | direction | time | ratio | 756 +--------------------+--------------+-----------+-------------------+ 757 | One | Backward | 0s | 2.0 | 758 | Two | Backward | 35s | 0.8 | 759 | Three | Backward | 70s | 2.0 | 760 +--------------------+--------------+-----------+-------------------+ 762 Table 4: Path capacity variation pattern for backward direction 764 5.4. Competing Media Flows with same Congestion Control Algorithm 766 In this test case, more than one media flows share the bottleneck 767 link and each of them uses the same congestion control algorithm. 768 This is a typical scenario where a real-time interactive application 769 sends more than one media flow to the same destination and these 770 flows are multiplexed over the same port. In such a scenario it is 771 likely that the flows will be routed via the same path and need to 772 share the available bandwidth amongst themselves. For the sake of 773 simplicity it is assumed that there are no other competing traffic 774 sources in the bottleneck link and that there is sufficient capacity 775 to accommodate all the flows individually. While this appears to be 776 a variant of the test case defined in Section 5.2, it focuses on the 777 capacity sharing aspect of the candidate algorithm. The previous 778 test case, on the other hand, measures adaptability, stability, and 779 responsiveness of the candidate algorithm. 781 Expected behavior: It is expected that the competing flows will 782 converge to an optimum bit rate to accommodate all the flows with 783 minimum possible latency and loss. Specifically, the test introduces 784 three media flows at different time instances, when the second flow 785 appears there should still be room to accommodate another flow on the 786 bottleneck link. Lastly, when the third flow appears the bottleneck 787 link should be saturated. 789 Evaluation metrics : as described in Section 4.1. 791 Testbed topology: Three media sources S1, S2, S3 are connected to R1, 792 R2, R3 respectively. The media traffic is transported over the 793 forward path and corresponding feedback/control traffic is 794 transported over the backward path. 796 +---+ +---+ 797 |S1 |===== \ Forward --> / =======|R1 | 798 +---+ \\ // +---+ 799 \\ // 800 +---+ +-----+ +-----+ +---+ 801 |S2 |=======| A |------------------------------>| B |=======|R2 | 802 +---+ | |<------------------------------| | +---+ 803 +-----+ +-----+ 804 // \\ 805 // <-- Backward \\ 806 +---+ // \\ +---+ 807 |S3 |====== / \ ======|R3 | 808 +---+ +---+ 810 Figure 5: Testbed Topology for Multiple congestion controlled media 811 Flows 813 Testbed attributes: 815 o Test duration: 120s 817 o Path characteristics: 819 * Reference bottleneck capacity: 3.5Mbps 821 * Path capacity ratio: 1.0 823 o Application-related: 825 * Media Source: 827 + Media type: Video 829 - Media direction: forward. 831 - Number of media sources: three (3) 833 - Media timeline: new media flows are added sequentially, 834 at short time intervals. See test specific setup below. 836 + Media type: Audio 838 - Media direction: forward. 840 - Number of media sources: three (3) 842 - Media timeline: new media flows are added sequentially, 843 at short time intervals. See test specific setup below. 845 * Competing traffic: 847 + Number of sources : zero (0) 849 o Test Specific Information: Table 5 defines the media timeline for 850 both media type. 852 +---------+------------+------------+----------+ 853 | Flow ID | Media type | Start time | End time | 854 +---------+------------+------------+----------+ 855 | 1 | Video | 0s | 119s | 856 | 2 | Video | 20s | 119s | 857 | 3 | Video | 40s | 119s | 858 | 4 | Audio | 0s | 119s | 859 | 5 | Audio | 20s | 119s | 860 | 6 | Audio | 40s | 119s | 861 +---------+------------+------------+----------+ 863 Table 5: Media Timeline for Video and Audio media sources 865 5.5. Round Trip Time Fairness 867 In this test case, multiple media flows share the bottleneck link, 868 but the end-to-end path latency for each flow is different. For the 869 sake of simplicity it is assumed that there are no other competing 870 traffic sources in the bottleneck link and that there is sufficient 871 capacity to accommodate all the flows. While this appears to be a 872 variant of test case 5.2, it focuses on the capacity sharing aspect 873 of the candidate algorithm under different RTTs. 875 It is expected that the competing flows will converge to bit rates to 876 accommodate all the flows with minimum possible latency and loss. 877 Specifically, the test introduces five media flows at the same time 878 instance. 880 Evaluation metrics : as described in Section 4.1. 882 Testbed Topology: Five (5) media sources S1,S2,..,S5 are connected to 883 their corresponding media sinks R1,R2,..,R5. The media traffic is 884 transported over the forward path and corresponding feedback/control 885 traffic is transported over the backward path. The topology is the 886 same as in Section 5.4. The end-to-end path delays are: 10ms for 887 S1-R1, 25ms for S2-R2, 50ms for S3-R3, 100ms for S4-R4, and 150ms 888 S5-R5, respectively. 890 Testbed attributes: 892 o Test duration: 300s 894 o Path characteristics: 896 * One-Way propagation delay for each flow: 10ms, 25ms, 50ms, 897 100ms, 150ms. 899 o Application-related: 901 * Media Source: 903 + Media type: Video 905 - Media direction: forward 907 - Number of media sources: five (5) 909 - Media timeline: new media flows are added sequentially, 910 at short time intervals. See test specific setup below. 912 + Media type: Audio 914 - Media direction: forward. 916 - Number of media sources: five (5) 918 - Media timeline: new media flows are added sequentially, 919 at short time intervals. See test specific setup below. 921 * Competing traffic: 923 + Number of sources : zero (0) 925 o Test Specific Information: Table 6 defines the media timeline for 926 both media type. 928 +---------+------------+------------+----------+ 929 | Flow IF | Media type | Start time | End time | 930 +---------+------------+------------+----------+ 931 | 1 | Video | 0s | 299s | 932 | 2 | Video | 10s | 299s | 933 | 3 | Video | 20s | 299s | 934 | 4 | Video | 30s | 299s | 935 | 5 | Video | 40s | 299s | 936 | 6 | Audio | 0 | 299s | 937 | 7 | Audio | 10s | 299s | 938 | 8 | Audio | 20s | 299s | 939 | 9 | Audio | 30s | 299s | 940 | 10 | Audio | 40s | 299s | 941 +---------+------------+------------+----------+ 943 Table 6: Media Timeline for Video and Audio media sources 945 5.6. Media Flow Competing with a Long TCP Flow 947 In this test case, one or more media flows share the bottleneck link 948 with at least one long lived TCP flow. Long lived TCP flows download 949 data throughout the session and are expected to have infinite amount 950 of data to send and receive. This is a scenario where a multimedia 951 application co-exists with a large file download. The test case 952 measures the adaptivity of the candidate algorithm to competing 953 traffic. It addresses the requirement 3 in 954 [I-D.ietf-rmcat-cc-requirements]. 956 Expected behavior: depending on the convergence observed in test case 957 5.1 and 5.2, the candidate algorithm may be able to avoid congestion 958 collapse. In the worst case, the media stream will fall to the 959 minimum media bit rate. 961 Evaluation metrics : following metrics in addition to as described in 962 Section 4.1. 964 1. Flow level: 966 A. TCP throughput. 968 B. Loss for the TCP flow 970 Testbed topology: One (1) media source S1 is connected to the 971 corresponding media sink, R1. In addition, there is a long-live TCP 972 flow sharing the same bottleneck link. The media traffic is 973 transported over the forward path and corresponding feedback/control 974 traffic is transported over the backward path. The TCP traffic goes 975 over the forward path from, S_tcp with acknowledgment packets go over 976 the backward path from, R_tcp. 978 +--+ +--+ 979 |S1|===== \ Forward --> / =======|R1| 980 +--+ \\ // +--+ 981 \\ // 982 +-----+ +-----+ 983 | A |---------------------------->| B | 984 | |<----------------------------| | 985 +-----+ +-----+ 986 // \\ 987 // <-- Backward \\ 988 +-----+ // \\ +-----+ 989 |S_tcp|=== / \ ===|R_tcp| 990 +-----+ +-----+ 992 Figure 6: Testbed Topology for TCP vs congestion controlled media 993 Flows 995 Testbed attributes: 997 o Test duration: 120s 999 o Path characteristics: 1001 * Reference bottleneck capacity: 2Mbps 1003 * Path capacity ratio: 1.0 1005 * Bottleneck queue size: [300ms, 1000ms] 1007 o Application-related: 1009 * Media Source: 1011 + Media type: Video 1013 - Media direction: forward 1015 - Number of media sources: one (1) 1017 - Media timeline: 1019 o Start time: 5s. 1021 o End time: 119s. 1023 + Media type: Audio 1025 - Media direction: forward 1026 - Number of media sources: one (1) 1028 - Media timeline: 1030 o Start time: 5s. 1032 o End time: 119s. 1034 * Additionally, implementers are encouraged to run the experiment 1035 with multiple media sources. 1037 * Competing traffic: 1039 + Number and Types of sources : one (1) and long-lived TCP 1041 + Traffic direction : forward 1043 + Congestion control: default TCP congestion control[RFC5681]. 1045 + Traffic timeline: 1047 - Start time: 0s. 1049 - End time: 119s. 1051 o Test Specific Information: none 1053 5.7. Media Flow Competing with Short TCP Flows 1055 In this test case, one or more congestion controlled media flow 1056 shares the bottleneck link with multiple short-lived TCP flows. 1057 Short-lived TCP flows resemble the on/off pattern observed in the web 1058 traffic, wherein clients (browsers) connect to a server and download 1059 a resource (typically a web page, few images, text files, etc.) using 1060 several TCP connections (up to 4). This scenario shows the 1061 performance of a multimedia application when several browser windows 1062 are active. The test case measures the adaptivity of the candidate 1063 algorithm to competing web traffic, it addresses the requirements 1.E 1064 in [I-D.ietf-rmcat-cc-requirements]. 1066 Depending on the number of short TCP flows, the cross-traffic either 1067 appears as a short burst flow or resembles a long TCP flow. The 1068 intention of this test is to observe the impact of short-term burst 1069 on the behavior of the candidate algorithm. 1071 Evaluation metrics : following metrics in addition to as described in 1072 Section 4.1. 1074 1. Flow level: 1076 A. Variation in the sending rate of the TCP flow. 1078 B. TCP throughput. 1080 Testbed topology: The topology described here is same as the one 1081 described in Figure 6. 1083 Testbed attributes: 1085 o Test duration: 300s 1087 o Path characteristics: 1089 * Reference bottleneck capacity: 2.0Mbps 1091 * Path capacity ratio: 1.0 1093 o Application-related: 1095 * Media source: 1097 + Media type: Video 1099 - Media direction: forward 1101 - Number of media sources: two (2) 1103 - Media timeline: 1105 o Start time: 5s. 1107 o End time: 299s. 1109 + Media type: Audio 1111 - Media direction: forward 1113 - Number of media sources: two (2) 1115 - Media timeline: 1117 o Start time: 5s. 1119 o End time: 299s. 1121 * Competing traffic: 1123 + Number and Types of sources : ten (10), short-lived TCP 1124 flows. 1126 + Traffic direction : forward 1128 + Congestion algorithm: default TCP Congestion control 1129 [RFC5681]. 1131 + Traffic timeline: each short TCP flow is modeled as a 1132 sequence of file downloads interleaved with idle periods. 1133 Not all short TCP flows start at the same time, 2 of them 1134 start in the ON state while rest of the 8 flows start in an 1135 OFF stats. For description of short TCP flow model see test 1136 specific information below. 1138 o Test Specific Information: 1140 * Short-TCP traffic model: The short TCP model to be used in this 1141 test is described in [I-D.ietf-rmcat-eval-criteria]. 1143 5.8. Media Pause and Resume 1145 In this test case, more than one real-time interactive media flows 1146 share the link bandwidth and all flows reach to a steady state by 1147 utilizing the link capacity in an optimum way. At this stage one of 1148 the media flows is paused for a moment. This event will result in 1149 more available bandwidth for the rest of the flows as they are on a 1150 shared link. When the paused media flow resumes it would no longer 1151 have the same bandwidth share on the link. It has to make it's way 1152 through the other existing flows in the link to achieve a fair share 1153 of the link capacity. This test case is important specially for 1154 real-time interactive media which consists of more than one media 1155 flows and can pause/resume media flows at any point of time during 1156 the session. This test case directly addresses the requirement 1157 number 5 in [I-D.ietf-rmcat-cc-requirements]. One can think it as a 1158 variation of test case defined in Section 5.4. However, it is 1159 different as the candidate algorithms can use different strategies to 1160 increase its efficiency, for example in terms of fairness, 1161 convergence time, reduce oscillation etc, by capitalizing the fact 1162 that they have previous information of the link. 1164 Evaluation metrics : following metrics in addition to as described in 1165 Section 4.1. 1167 1. Flow level: 1169 A. Variation in sending bit rate and goodput. Mainly observing 1170 the frequency and magnitude of oscillations. 1172 Testbed Topology: Same as test case defined in Section 5.4 1174 Testbed attributes: The general description of the testbed parameters 1175 are same as Section 5.4 with changes in the test specific setup as 1176 below- 1178 o Other test specific setup: 1180 * Media flow timeline: 1182 + Flow ID: one (1) 1184 + Start time: 0s 1186 + Flow duration: 119s 1188 + Pause time: not required 1190 + Resume time: not required 1192 * Media flow timeline: 1194 + Flow ID: two (2) 1196 + Start time: 0s 1198 + Flow duration: 119s 1200 + Pause time: at 40s 1202 + Resume time: at 60s 1204 * Media flow timeline: 1206 + Flow ID: three (3) 1208 + Start time: 0s 1210 + Flow duration:119s 1212 + Pause time: not required 1214 + Resume time: not required 1216 6. Other potential test cases 1218 It has been noticed that there are other interesting test cases 1219 besides the basic test cases listed above. In many aspects, these 1220 additional test cases can help further evaluation of the candidate 1221 algorithm. They are listed as below. 1223 6.1. Media Flows with Priority 1225 In this test case media flows will have different priority levels. 1226 This will be an extension of Section 5.4 where the same test will be 1227 run with different priority levels imposed on each of the media 1228 flows. For example, the first flow (S1) is assigned a priority of 2 1229 whereas the remaining two flows (S2 and S3) are assigned a priority 1230 of 1. The candidate algorithm MUST reflect the relative priorities 1231 assigned to each media flow. In the previous example, the first flow 1232 (S1) MUST arrive at a steady-state rate approximately twice of that 1233 of the other two flows (S2 and S3). 1235 The candidate algorithm can use a coupled congestion control 1236 mechanism for the bandwidth distribution according to the respective 1237 media flow priority. 1239 6.2. Explicit Congestion Notification Usage 1241 This test case requires to run all the basic test cases with the 1242 availability of Explicit Congestion Notification (ECN) [RFC6679] 1243 feature enabled. The goal of this test is to exhibit that the 1244 candidate algorithms do not fail when ECN signals are available. 1245 With ECN signals enabled the algorithms are expected to perform 1246 better than their delay based variants. 1248 6.3. Multiple Bottlenecks 1250 In this test case one congestion controlled media flow, S1->R2, 1251 traverses a path with multiple bottlenecks. As illustrated in 1252 Figure 7, the first flow (S1->R1) competes with the second congestion 1253 controlled media flow (S2->R2) over the link between A and B which is 1254 close to the sender side; again, that flow (S1->R1) competes with the 1255 third congestion controlled media flow (S3->R3) over the link between 1256 C and D which is close to the receiver side. The goal of this test 1257 is to ensure that the candidate algorithms work properly in the 1258 presence of multiple bottleneck links on the end to end path. 1260 Expected behavior: the candidate algorithm is expected to achieve 1261 full utilization at both bottleneck links without starving any of the 1262 three congestion controlled media flows. 1264 Forward ----> 1266 +---+ +---+ +---+ +---+ 1267 |S2 | |R2 | |S3 | |R3 | 1268 +---+ +---+ +---+ +---+ 1269 | | | | 1270 | | | | 1271 +---+ +-----+ +-----+ +-----+ +-----+ +---+ 1272 |S1 |=======| A |------>| B |----->| C |---->| D |=======|R1 | 1273 +---+ | |<------| |<-----| |<----| | +---+ 1274 +-----+ +-----+ +-----+ +-----+ 1276 1st 2nd 1277 Bottleneck (A->B) Bottleneck (C->D) 1279 <------ Backward 1281 Figure 7: Testbed Topology for Multiple Bottlenecks 1283 Testbed topology: Three media sources S1, S2, and S3 are connected to 1284 respective destinations R1, R2, and R3. For all three flows the 1285 media traffic is transported over the forward path and corresponding 1286 feedback/control traffic is transported over the backward path. 1288 Testbed attributes: 1290 o Test duration: 300s 1292 o Path characteristics: 1294 * Reference bottleneck capacity: 2Mbps. 1296 * Path capacity ratio between A and B: 1.0 1298 * Path capacity ratio between B and C: 4.0. 1300 * Path capacity ratio between C and D: 0.75. 1302 * One-Way propagation delay: 1304 1. Between S1 and R1: 100ms 1306 2. Between S2 and R2: 40ms 1307 3. Between S3 and R3: 40ms 1309 o Application-related: 1311 * Media Source: 1313 + Media type: Video 1315 - Media direction: Forward 1317 - Number of media sources: Three (3) 1319 - Media timeline: 1321 o Start time: 0s. 1323 o End time: 299s. 1325 + Media type: Audio 1327 - Media direction: Forward 1329 - Number of media sources: Three (3) 1331 - Media timeline: 1333 o Start time: 0s. 1335 o End time: 299s. 1337 * Competing traffic: 1339 + Number of sources : Zero (0) 1341 7. Wireless Access Links 1343 Additional wireless network (both cellular network and WiFi network) 1344 specific test cases are defined in [I-D.ietf-rmcat-wireless-tests]. 1346 8. Security Considerations 1348 Security issues have not been discussed in this memo. 1350 9. IANA Considerations 1352 There are no IANA impacts in this memo. 1354 10. Acknowledgements 1356 Much of this document is derived from previous work on congestion 1357 control at the IETF. 1359 The content and concepts within this document are a product of the 1360 discussion carried out in the Design Team. 1362 11. References 1364 11.1. Normative References 1366 [I-D.ietf-rmcat-eval-criteria] 1367 Singh, V., Ott, J., and S. Holmer, "Evaluating Congestion 1368 Control for Interactive Real-time Media", draft-ietf- 1369 rmcat-eval-criteria-07 (work in progress), May 2018. 1371 [I-D.ietf-rmcat-video-traffic-model] 1372 Zhu, X., Cruz, S., and Z. Sarker, "Modeling Video Traffic 1373 Sources for RMCAT Evaluations", draft-ietf-rmcat-video- 1374 traffic-model-04 (work in progress), January 2018. 1376 [I-D.ietf-rmcat-wireless-tests] 1377 Sarker, Z., Johansson, I., Zhu, X., Fu, J., Tan, W., and 1378 M. Ramalho, "Evaluation Test Cases for Interactive Real- 1379 Time Media over Wireless Networks", draft-ietf-rmcat- 1380 wireless-tests-04 (work in progress), May 2017. 1382 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1383 Jacobson, "RTP: A Transport Protocol for Real-Time 1384 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1385 July 2003, . 1387 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 1388 Video Conferences with Minimal Control", STD 65, RFC 3551, 1389 DOI 10.17487/RFC3551, July 2003, 1390 . 1392 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 1393 "RTP Control Protocol Extended Reports (RTCP XR)", 1394 RFC 3611, DOI 10.17487/RFC3611, November 2003, 1395 . 1397 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 1398 "Extended RTP Profile for Real-time Transport Control 1399 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 1400 DOI 10.17487/RFC4585, July 2006, 1401 . 1403 [RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size 1404 Real-Time Transport Control Protocol (RTCP): Opportunities 1405 and Consequences", RFC 5506, DOI 10.17487/RFC5506, April 1406 2009, . 1408 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 1409 and K. Carlberg, "Explicit Congestion Notification (ECN) 1410 for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 1411 2012, . 1413 11.2. Informative References 1415 [HEVC-seq] 1416 HEVC, "Test Sequences", 1417 http://www.netlab.tkk.fi/~varun/test_sequences/ . 1419 [I-D.ietf-rmcat-cc-requirements] 1420 Jesup, R. and Z. Sarker, "Congestion Control Requirements 1421 for Interactive Real-Time Media", draft-ietf-rmcat-cc- 1422 requirements-09 (work in progress), December 2014. 1424 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1425 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1426 . 1428 [xiph-seq] 1429 Xiph.org, "Video Test Media", 1430 http://media.xiph.org/video/derf/ . 1432 Authors' Addresses 1434 Zaheduzzaman Sarker 1435 Ericsson AB 1436 Luleae, SE 977 53 1437 Sweden 1439 Phone: +46 10 717 37 43 1440 Email: zaheduzzaman.sarker@ericsson.com 1442 Varun Singh 1443 Nemu Dialogue Systems Oy 1444 Runeberginkatu 4c A 4 1445 Helsinki 00100 1446 Finland 1448 Email: varun.singh@iki.fi 1449 URI: http://www.callstats.io/ 1450 Xiaoqing Zhu 1451 Cisco Systems 1452 12515 Research Blvd 1453 Austing, TX 78759 1454 USA 1456 Email: xiaoqzhu@cisco.com 1458 Michael A. Ramalho 1459 Cisco Systems, Inc. 1460 6310 Watercrest Way Unit 203 1461 Lakewood Ranch, FL 34202-5211 1462 USA 1464 Phone: +1 919 476 2038 1465 Email: mramalho@cisco.com