idnits 2.17.1 draft-burmeister-avt-rtcp-feedback-sim-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 2 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The "Author's Address" (or "Authors' Addresses") section title is misspelled. == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 2001) is 8197 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '3' is defined on line 1409, but no explicit reference was found in the text == Unused Reference: '8' is defined on line 1427, but no explicit reference was found in the text == Unused Reference: '9' is defined on line 1431, but no explicit reference was found in the text == Outdated reference: A later version (-11) exists of draft-ietf-avt-rtcp-feedback-00 == Outdated reference: A later version (-12) exists of draft-ietf-avt-rtp-new-10 -- Possible downref: Normative reference to a draft: ref. '2' == Outdated reference: A later version (-13) exists of draft-ietf-avt-profile-new-11 -- Possible downref: Normative reference to a draft: ref. '3' -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' -- Possible downref: Non-RFC (?) normative reference: ref. '7' -- Possible downref: Non-RFC (?) normative reference: ref. '8' -- Possible downref: Non-RFC (?) normative reference: ref. '9' Summary: 4 errors (**), 0 flaws (~~), 10 warnings (==), 10 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 C.Burmeister 3 Internet Draft R.Hakenberg 4 draft-burmeister-avt-rtcp-feedback-sim-00.txt A.Miyazaki 5 Expires: April 2002 Matsushita 7 J.Ott 8 University of Bremen TZI 10 N.Sato 11 S.Fukunaga 12 Oki 14 November 2001 16 Extended RTP Profile for RTCP-based Feedback 17 - Results of the Timing Rule Simulations - 19 Status of this Memo 21 This document is an Internet-Draft and is in full conformance 22 with all provisions of Section 10 of RFC2026. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF), its areas, and its working groups. Note that 26 other groups may also distribute working documents as Internet- 27 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other documents 31 at any time. It is inappropriate to use Internet-Drafts as 32 reference material or to cite them other than as "work in progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 39 Abstract 41 This document describes the results we achieved when simulating the 42 timing rules of the Extended RTP Profile for RTCP-based Feedback. 43 Unicast and multicast topologies are considered as well as several 44 protocol and environment configurations. The results show that the 45 timing rules result in better performance regarding feedback delay 46 and still preserve the well accepted RTP rules regarding allowed bit 47 rates for control traffic. 49 Extended RTP Profile for RTCP-based Feedback November 2001 50 - Results of the Timing Rule Simulations - 52 Table of Contents 54 Status of this Memo 55 Abstract 57 1 Introduction 59 2 Conventions used in this document 61 3 Timing rules of the extended RTP profile for RTCP-based feedback 63 4 Simulation Environment 64 4.1 Network Simulator Version 2 65 4.2 RTP Agent 66 4.3 Scenarios 67 4.4 Topologies 69 5 RTCP Bit Rate Measurements 70 5.1 Unicast 71 5.2 Multicast 72 5.3 Summary of the RTCP bit rate measurements 74 6 Feedback Measurements 75 6.1 Unicast 76 6.2 Multicast 77 6.2.1 Shared Losses vs Distributed Losses 78 6.2.2 Sender vs. Receiver 80 7 Investigations on "k" 81 7.1 Feedback Suppression Performance 82 7.2 Loss Report Delay 83 7.3 Summary of "k" investigations 85 8 Investigations on "l" 86 8.1 Feedback Suppression Performance 87 8.2 Loss Report Delay 88 8.3 Summary of "l" investigations 90 9 Applications Using AVPF 91 9.1 NEWPRED Implementation in NS2 92 9.2 Simulation 93 9.2.1. Simulation A - Constant Packet Loss Rate 94 9.2.2. Simulation B - Packet Loss due to Congestion 95 9.3 Summary 97 10 Summary 99 References 100 Authors Addresses 101 Extended RTP Profile for RTCP-based Feedback November 2001 102 - Results of the Timing Rule Simulations - 104 1 Introduction 106 The Real-time Transport Protocol (RTP) is widely used for the 107 transmission of real-time or near real-time media data over the 108 Internet. While it was originally designed to work well for 109 multicast groups in very large scales, its scope is not limited to 110 that. More and more applications use RTP for small multicast groups 111 (e.g. video conferences) or even unicast (e.g. media streaming 112 applications). 114 RTP comes together with its companion protocol Real-time Transport 115 Control Protocol (RTCP), which is used to monitor the transmission 116 of the media data and provide feedback of the reception quality. 117 What is more it can be used for a loosely session control. Having 118 the scope of large multicast groups in mind, the rules when to send 119 feedback were much restricted to avoid feedback explosion or 120 feedback related congestion in the network. RTP and RTCP have proven 121 to work well in the Internet, especially in large multicast groups, 122 which is shown by its tremendous usages today. 124 However the applications that transmit the media data only to small 125 multicast groups or unicast, may benefit from more frequent 126 feedback. The source of the packets might be able to react to 127 changes in the reception quality, which might be due to congestion 128 in the network or other sudden changes. Possible reactions include 129 sending rate adaptation according to a congestion control algorithm 130 or the invocation of error resilience features for the media stream 131 (e.g. retransmissions, reference picture selection, NEWPRED, etc.). 133 As said before, more feedback would be needed to increase the 134 reception quality, but RTP restricts the use of RTCP feedback very 135 much. Hence it was decided to create a new extended RTP profile, 136 which redefines some of the RTCP timing rules, but keeps most of the 137 algorithms for RTP and RTCP, which have proven to work well. The new 138 rules should scale from unicast to multicast, where unicast or small 139 multicast applications have the most gain from it. A detailed 140 description of the new profile and its timing rules can be found in 141 [1]. 143 This document investigates the new algorithms by the means of 144 simulations. We show that the new timing rules scale and behave 145 network friendly. Therefore we first describe roughly the key 146 features of the new RTP profile, which are important for our 147 simulations, in Section 3. After that we describe the environment 148 that is used to conduct the simulations in Section 4. Section 5 149 describes simulation results that show the backwards compatibility 150 to RTP and that the new profile is network friendly in terms of used 151 bit rate for feedback and other control traffic. In Section 6 we 152 show the benefit that applications could get from implementing the 153 new profile. In Section 7 and 8 we show the merit for some special 154 Extended RTP Profile for RTCP-based Feedback November 2001 155 - Results of the Timing Rule Simulations - 157 parameters settings and finally in section 9 we show the performance 158 gain we could get for a special application, namely NEWPRED in 159 MPEG-4. 161 2 Conventions used in this document 163 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 164 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 165 this document are to be interpreted as described in RFC-2119. 167 3 Timing rules of the extended RTP profile for RTCP-based feedback 169 As said above, RTP restricts the usage of RTCP feedback. The main 170 rules that restrict the feedback are as follows: 172 - RTCP messages are sent in compound packets, i.e. every RTCP packet 173 contains at least one sender report (SR) or receiver report (RR) 174 message and a source description (SDES) message. 175 - The RTCP compound packets are sent in time intervals (T_rr), which 176 is computed as a function of the average packet size, the number 177 of senders and receivers in the group and the session bandwidth. 178 (-> 5% of the session bandwidth is used for RTCP messages; this 179 bandwidth is shared between all session members, where the senders 180 might get more than the receivers.) 181 - The minimum interval between two RTCP packets from the same source 182 is 5 seconds. 184 We see that these rules prevent feedback explosion and scale to very 185 large multicast groups. However they do not allow timely feedback at 186 all. While the second rule scales also to small groups or unicast 187 (in this cases the interval might be as small as a few 188 milliseconds), the third rule prevents the receivers from sending 189 feedback in time. 191 The timing rules to send RTCP feedback from the new RTP profile [1] 192 consists of two key components. First the minimum interval of 5 193 seconds is abolished. Second, receivers get once during their (now 194 quite small) RTCP interval the chance to send an RTCP packet 195 "early", i.e. not according to the calculated interval, but 196 virtually immediately. It is important to note that the RTCP 197 interval calculation is still inherited from the original RTP 198 specification. 200 The specification and all the details of the extended timing rules 201 can be found in [1]. We do not want to describe the algorithms here, 202 but rather reference these from the original specification where 203 needed. Therefore we use also the same variable names and 204 abbreviations as in [1]. 206 Extended RTP Profile for RTCP-based Feedback November 2001 207 - Results of the Timing Rule Simulations - 209 4 Simulation Environment 211 This section describes the simulator that was used for the 212 investigations and its key features. The extensions to the 213 simulator, that were necessary are described roughly. 215 4.1 Network Simulator Version 2 217 The simulations were conducted using the network simulator version 2 218 (ns2). ns2 is an open source project, written in a combination of 219 Tool Command Language (TCL) and C++. The scenarios are set-up using 220 TCL. In the scripts it is possible to specify the topologies (nodes 221 and links, bandwidths, queue sizes or error rates for links) and the 222 parameters of the "agents", i.e. protocol configurations. The 223 protocols itself are implemented in C++ in the agents, which are 224 connected to the nodes. A detailed description of ns2 and a 225 downloadable newest version can be found at [4]. 227 4.2 RTP Agent 229 We implemented a new agent, based on RTP/RTCP. RTP packets are sent 230 at a constant packet rate with the correct header sizes. RTCP 231 packets are sent according to the timing rules of [2] and also its 232 algorithms for group membership maintenance are implemented. Sender 233 and receiver reports are sent and the senders use these reports to 234 maintain a RTT estimation to the other group members, as it is 235 described in [2]. 237 Further we extended the agent to support the extended profile [1]. 238 The use of the new timing rules can be turned on and off via 239 parameter settings in TCL. 241 4.3 Scenarios 243 The scenarios that are simulated are defined in TCL scripts. We set- 244 up several different topologies, ranging from unicast with two 245 session members to multicast with up to 25 session members. 246 Depending on the used sending rates and the corresponding link 247 bandwidths congestion losses may occur. In some scenarios, bit 248 errors are inserted on certain links. We simulated groups with 249 RTP/AVP agents, RTP/AVPF agents and mixed groups. 251 The feedback messages are generally NACK messages as defined in [1] 252 and are triggered by packet loss. 254 Extended RTP Profile for RTCP-based Feedback November 2001 255 - Results of the Timing Rule Simulations - 257 4.4 Topologies 259 Mainly four different topologies are simulated to show the key 260 features of the extended profile. However for some specific 261 simulations we used, different topologies, which is then indicated 262 at the description of the simulation results. The main four 263 topologies are named after the number of participating RTP agents, 264 i.e. T-2, T-4, T-8 and T-16, where T-2 is a unicast scenario, T-4 265 contains four agents, etc. The figures below illustrate the main 266 topologies. 267 A5 268 A5 | A6 269 / | / 270 / | /--A7 271 / |/ 272 A2 A2-----A6 A2--A8 273 / / / A9 274 / / / / 275 / / / /---A10 276 A1-----A2 A1-----A3 A1-----A3-----A7 A1------A3< 277 \ \ \ \---A11 278 \ \ \ \ 279 \ \ \ A12 280 A4 A4-----A8 A4--A13 281 |\ 282 | \--A14 283 | \ 284 | A15 285 A16 287 T-2 T-4 T-8 T-16 289 Figure 1: Simulated Topologies. 291 5 RTCP Bit Rate Measurements 293 The new timing rules allow more frequent RTCP feedback for small 294 multicast groups. In large groups the algorithm behaves similar to 295 usual RTP. While it is generally good to have more frequent feedback 296 it cannot be allowed at all to increase the bit rate used for RTCP 297 above a fixed limit, i.e. 5% of the total RTP bandwidth according to 298 RTP. This section shows that with the new timing rules we keep the 299 5% limit for all investigated scenarios, topologies and group sizes. 300 What is more, we show that mixed groups, i.e. some members use AVP 301 some use AVPF, can be allowed and that each session member behaves 302 fair according to its corresponding specification. 304 Extended RTP Profile for RTCP-based Feedback November 2001 305 - Results of the Timing Rule Simulations - 307 5.1 Unicast 309 First we measured the RTCP bandwidth share in the unicast topology 310 T-2. Even for a fixed topology and group size, there are several 311 protocol parameters which are varied to simulate a large range of 312 different scenarios. First we varied the RTP session bandwidth. For 313 large session bandwidths, the allowed RTCP bit rate increases also 314 and thus more RTCP packets can be sent. Second we changed the number 315 of agents that are pure receivers or also senders. This has also 316 some influence on the RTCP feedback, because on the one hand pure 317 receivers do not have an RTT estimation and one the other hand they 318 do not send sender reports. Third we varied the configurations of 319 the agents in that sense that the agents may use the AVP or AVPF. 320 Thereby it is possible that one agent uses AVP and the other AVPF in 321 one RTP session. This is done to test the backwards compatibility. 323 First we consider scenarios where no losses occur. In this case both 324 RTP session members transmit the RTCP compound packets at regular 325 intervals, calculated as T_rr, if they use the AVPF, and use the 326 minimum interval of 5s if they implement the AVP. No early packets 327 are sent, because the need to send feedback is not given. Still it 328 is important to see that not more than 5% of the session bandwidth 329 is used for RTCP and that AVP and AVPF members can co-exist without 330 interference. The results can be found in table 1. 332 | | | | | | Used RTCP Bit Rate | 333 | Session | Send | Rec. | AVP | AVPF | (% of session bw) | 334 |Bandwidth|Agents|Agents|Agents|Agents| A1 | A2 | sum | 335 +---------+------+------+------+------+------+------+------+ 336 | 2 Mbps | 1 | 2 | - | 1,2 | 2.42 | 2.56 | 4.98 | 337 | 2 Mbps | 1,2 | - | - | 1,2 | 2.49 | 2.49 | 4.98 | 338 | 2 Mbps | 1 | 2 | 1 | 1,2 | 0.01 | 2.49 | 2.50 | 339 | 2 Mbps | 1,2 | - | 1 | 1,2 | 0.01 | 2.48 | 2.49 | 340 | 2 Mbps | 1 | 2 | 1,2 | 1,2 | 0.01 | 0.01 | 0.02 | 341 | 2 Mbps | 1,2 | - | 1,2 | 1,2 | 0.01 | 0.01 | 0.02 | 342 |200 kbps | 1 | 2 | - | 1,2 | 2.42 | 2.56 | 4.98 | 343 |200 kbps | 1,2 | - | - | 1,2 | 2.49 | 2.49 | 4.98 | 344 |200 kbps | 1 | 2 | 1 | 1,2 | 0.06 | 2.49 | 2.55 | 345 |200 kbps | 1,2 | - | 1 | 1,2 | 0.08 | 2.50 | 2.58 | 346 |200 kbps | 1 | 2 | 1,2 | 1,2 | 0.06 | 0.06 | 0.12 | 347 |200 kbps | 1,2 | - | 1,2 | 1,2 | 0.08 | 0.08 | 0.16 | 348 | 20 kbps | 1 | 2 | - | 1,2 | 2.44 | 2.54 | 4.98 | 349 | 20 kbps | 1,2 | - | - | 1,2 | 2.50 | 2.51 | 5.01 | 350 | 20 kbps | 1 | 2 | 1 | 1,2 | 0.58 | 2.48 | 3.06 | 351 | 20 kbps | 1,2 | - | 1 | 1,2 | 0.77 | 2.51 | 3.28 | 352 | 20 kbps | 1 | 2 | 1,2 | 1,2 | 0.58 | 0.61 | 1.19 | 353 | 20 kbps | 1,2 | - | 1,2 | 1,2 | 0.77 | 0.79 | 1.58 | 355 Table 1: Unicast simulations without packet loss. 357 Extended RTP Profile for RTCP-based Feedback November 2001 358 - Results of the Timing Rule Simulations - 360 We can see that in configurations, where both Agents use the new 361 timing rules each of them uses about 2.5% of the session bandwidth 362 for RTP, which sums up to 5% of the session bandwidth for both. This 363 is achieved regardless of the agent being a sender or a receiver. In 364 the cases where Agent1 uses AVP and Agent2 AVPF, the total RTCP 365 session bandwidth is decreased. This is due to the fact that Agent1 366 can send RTCP packets only with a minimum interval of 5 seconds. 367 Thus only a small fraction of the session bandwidth is used for its 368 RTCP packets. For a high bit rate session (session bandwidth = 2 369 Mbps) the fraction of the RTCP packets from Agent one is as small as 370 0.01%. For smaller session bandwidths the fraction increases, 371 because the same amount of RTCP data is sent. The bandwidth share 372 that is used by RTCP packets from Agent 2 is not different from what 373 was used, when both Agents implemented the AVPF. Thus the 374 interaction of AVP and AVPF agents is not problematic in these 375 scenarios at all. 377 In our second unicast experiment, we show that the allowed RTCP 378 bandwidth share is not exceeded, even if packet loss occurs. We 379 simulated a constant byte error rate (BYER) on the link. The byte 380 errors are inserted randomly with a uniform distribution. Packets 381 with byte errors are discarded on the link; hence the receiving 382 agents will not see the loss immediately. The agents detect packet 383 loss by a gap in the sequence number. 385 When the agents detect a packet loss, they feel the need to send 386 feedback. In unicast T_dither_max is always zero, hence an early 387 packet can be sent immediately if allow_early is true. If the last 388 packet was already an early one (i.e. allow_early = false), the 389 feedback might be appended to the next regularly scheduled receiver 390 report. The max_feedback_delay parameter (which we set to 1 second 391 in our simulations) determines if that is allowed. 393 The results are shown in table 2, where we can see that there is no 394 difference in the RTCP bandwidth share, whether losses occur or not. 395 This is what we expected, because even though the RTCP packet size 396 grows and early packets are sent, the interval between the packets 397 increases and thus the RTCP bandwidth stays the same. Only the RTCP 398 bandwidth of the Agents that use the AVP increases slightly. This is 399 because the interval between the packets is still 5 seconds, but the 400 packet size increased because of the feedback that is appended. 402 Extended RTP Profile for RTCP-based Feedback November 2001 403 - Results of the Timing Rule Simulations - 405 | | | | | | Used RTCP Bit Rate | 406 | Session | Send | Rec. | AVP | AVPF | (% of session bw) | 407 |Bandwidth|Agents|Agents|Agents|Agents| A1 | A2 | sum | 408 +---------+------+------+------+------+------+------+------+ 409 | 2 Mbps | 1 | 2 | - | 1,2 | 2.42 | 2.56 | 4.98 | 410 | 2 Mbps | 1,2 | - | - | 1,2 | 2.49 | 2.49 | 4.98 | 411 | 2 Mbps | 1 | 2 | 1 | 1,2 | 0.01 | 2.49 | 2.50 | 412 | 2 Mbps | 1,2 | - | 1 | 1,2 | 0.01 | 2.48 | 2.49 | 413 | 2 Mbps | 1 | 2 | 1,2 | 1,2 | 0.01 | 0.02 | 0.03 | 414 | 2 Mbps | 1,2 | - | 1,2 | 1,2 | 0.01 | 0.01 | 0.02 | 415 |200 kbps | 1 | 2 | - | 1,2 | 2.42 | 2.56 | 4.98 | 416 |200 kbps | 1,2 | - | - | 1,2 | 2.50 | 2.49 | 4.99 | 417 |200 kbps | 1 | 2 | 1 | 1,2 | 0.06 | 2.50 | 2.56 | 418 |200 kbps | 1,2 | - | 1 | 1,2 | 0.08 | 2.49 | 2.57 | 419 |200 kbps | 1 | 2 | 1,2 | 1,2 | 0.06 | 0.07 | 0.13 | 420 |200 kbps | 1,2 | - | 1,2 | 1,2 | 0.09 | 0.08 | 0.17 | 421 | 20 kbps | 1 | 2 | - | 1,2 | 2.42 | 2.57 | 4.99 | 422 | 20 kbps | 1,2 | - | - | 1,2 | 2.52 | 2.51 | 5.03 | 423 | 20 kbps | 1 | 2 | 1 | 1,2 | 0.58 | 2.54 | 3.12 | 424 | 20 kbps | 1,2 | - | 1 | 1,2 | 0.83 | 2.43 | 3.26 | 425 | 20 kbps | 1 | 2 | 1,2 | 1,2 | 0.58 | 0.73 | 1.31 | 426 | 20 kbps | 1,2 | - | 1,2 | 1,2 | 0.86 | 0.84 | 1.70 | 428 Table 2: Unicast simulations with packet loss. 430 5.2 Multicast 432 Next we investigated the RTCP bandwidth share in multicast 433 scenarios, i.e. we simulated the topologies T-4, T-8 and T-16 and 434 measured the fraction of the session bandwidth that was used for 435 RTCP packets. Again we considered different situations and protocol 436 configurations (e.g. with or without bit errors, groups with AVP 437 and/or AVPF agents, etc.). For reasons of readability, we present 438 only selected results. For a documentation of all results, see [5]. 440 The simulations of the different topologies in scenarios, where no 441 losses occur, neither through bit errors nor through congestion, 442 show a similar behavior as the unicast scenarios. For all group 443 sizes the maximum used RTCP bit rate share is 5.06% of the session 444 bandwidth in a simulation of 16 session members in a low bit rate 445 scenario (session bandwidth = 20kbps) with several senders. In all 446 other scenarios without losses the used RTCP bit rate share is below 447 that. Thus the requirement, that not more than 5% of the session bit 448 rate should be used for RTCP is fulfilled in reasonable accuracy. 450 Simulations, were bit errors are randomly inserted in RTP and RTCP 451 packets and the corrupted packets are discarded, give the same 452 results. The 5% rule is kept (at maximum 5.07% of the session 453 bandwidth is used for RTCP). 455 Extended RTP Profile for RTCP-based Feedback November 2001 456 - Results of the Timing Rule Simulations - 458 Finally we conducted simulations, where we reduced the link 459 bandwidth and thereby caused congestion related losses. These 460 simulations are different from the previous bit error simulations, 461 in that the losses occur more in bursts and are more correlated, 462 also between different agents. The correlation and burstness of the 463 packet loss is due to the queuing discipline in the routers we 464 simulated; we used simple FIFO queues with a drop-tail strategy to 465 handle congestion. Random Early Detection (RED) queues may enhance 466 the performance, because the burstness of the packet loss might be 467 reduced, however this is not subject of our investigations, but is 468 left for future research. The delay between the agents, which also 469 influence RTP and RTCP packets, is much more variable because of the 470 added queuing delay. Still the used RTCP bit rate share does not 471 increase beyond 5.09% of the session bandwidth. Thus also for these 472 special cases the requirement is fulfilled. 474 5.3 Summary of the RTCP bit rate measurements 476 We have shown that for unicast and reasonable multicast scenarios, 477 feedback explosion does not happen. The requirement that at maximum 478 5% of the session bandwidth is used for RTCP is fulfilled for all 479 investigated scenarios. 481 6 Feedback Measurements 483 In this chapter we describe the results of feedback delay 484 measurements, we conducted in the simulations. Therefore we use two 485 metrics for measuring the performance of the algorithms, these are 486 the mean "waiting time" (MWT) and the number of feedback that is 487 sent, suppressed or not allowed. The waiting time is the time, 488 measured at a certain agent, between the detection of a packet loss 489 and the time when the corresponding feedback is sent. Assuming that 490 the value of the feedback decreases with its delay, we think that 491 the mean waiting time is a good metric to measure the performance 492 gain we could get by using AVPF instead of AVP. 494 The feedback an agent wants to send can be either sent or not sent. 495 If it was not sent, this could be due to the feedback suppression, 496 i.e. another receiver already sent the same feedback or because the 497 feedback was not allowed, i.e. the max_feedback_delay was exceeded. 498 We traced for every detected loss, if the agent sent the 499 corresponding feedback or not and if not, why. The more feedback was 500 not allowed, the worse the performance of the algorithm. Together 501 with the waiting times, this gives us a good hint of the overall 502 performance of the scheme. 504 Extended RTP Profile for RTCP-based Feedback November 2001 505 - Results of the Timing Rule Simulations - 507 6.1 Unicast 509 In the unicast case, the maximum dithering interval T_Dither_max is 510 fixed and set to zero. This is due to the fact that it does not make 511 sense for a unicast receiver to wait for other receivers if they 512 have the same feedback to send. But still feedback can be delayed or 513 might not be permitted to be sent at all. The dithering interval is 514 a parameter for the early packets, but at maximum every second 515 packet can be an early packet. The regularly scheduled packets are 516 spaced according to T_rr, which depends in the unicast case mainly 517 on the session bandwidth. 519 Table 3 shows the mean waiting times (MWT) for some configurations 520 of the unicast topology T-2. The number of feedback packets that are 521 sent or discarded is listed also (feedback sent (sent) or feedback 522 discarded (disc)). We do not list suppressed packets, because for 523 the unicast case feedback suppression does not apply. In the 524 simulations, agent 1 was a sender and agent 2 a pure receiver. We 525 did not vary this, because the only difference in being a sender or 526 pure receiver, is that the sender has an RTT estimation to the 527 receivers. However the RTT estimation is used for the T_Dither_max 528 calculations only in the multicast cases. 530 | | | Feedback Statistics | 531 | Session | | AVP | AVPF | 532 |Bandwidth| PLR | sent |disc| MWT | sent |disc| MWT | 533 +---------+-------+------+----+-------+------+----+-------+ 534 | 2 Mbps | 0.001 | 781 | 0 | 2.604 | 756 | 0 | 0.015 | 535 | 2 Mbps | 0.01 | 7480 | 0 | 2.591 | 7548 | 2 | 0.006 | 536 | 2 Mbps | cong. | 25 | 0 | 2.557 | 1741 | 0 | 0.001 | 537 | 20 kbps | 0.001 | 79 | 0 | 2.472 | 74 | 2 | 0.034 | 538 | 20 kbps | 0.01 | 780 | 0 | 2.605 | 709 | 64 | 0.163 | 539 | 20 kbps | cong. | 780 | 0 | 2.590 | 687 | 70 | 0.162 | 541 Table 3: Feedback Statistics for the unicast simulations. 543 From the table above we see that the mean waiting time can be 544 decreased dramatically by using AVPF instead of AVP. While the 545 waiting times for agents using AVP is always around 2.5 seconds 546 (half the minimum interval) it can be decreased to a few ms for most 547 of the AVPF configurations. 549 In the cases of high session bandwidth normally all feedback is 550 sent. This is because the packet size is quite large (1000byte) and 551 thus per lost packet, more RTCP bandwidth is available. There are 552 only very few exceptions, which are probably due to two packet 553 losses within one RTCP interval, where the first loss was by chance 554 sent quite early. In this case it might be possible that the second 555 feedback is detected after the early packet was sent, but too early 556 Extended RTP Profile for RTCP-based Feedback November 2001 557 - Results of the Timing Rule Simulations - 559 to append it to the next regularly scheduled report, because of the 560 limitation of the max_feedback_delay. This is different for the 561 cases with a small session bandwidth. Here we have a small packet 562 size (100byte) and thus many packets are transmitted, while the RTCP 563 bandwidth share is quite low. T_rr is thus quite large. After an 564 early packet was sent the time to the next regularly scheduled 565 packet can be very high. We saw that in some cases the time was 566 larger than than max_feedback_delay, because in these cases the 567 feedback is not allowed to be sent at all. 569 With a different setting of max_feedback_delay it is possible to 570 have either more feedback that is not allowed and a decreased mean 571 waiting time or more feedback that is sent but an increased waiting 572 time. Thus the parameter should be set with care according to the 573 application's needs. 575 6.2 Multicast 577 In this section we describe some measurements of feedback statistics 578 in the multicast simulations. We picked out certain characteristic 579 and representative results. Therefore we considered the topology T- 580 16. Different scenarios and applications are simulated for this 581 topology. The parameters of the different links are set as follows. 582 The agents A2, A3 and A4 are connected to the middle node of the 583 multicast tree, i.e. agent A1, via high bandwidth and low delay 584 links. The other agents are connected to the nodes 2, 3 and 4 via 585 different link characteristics. The agents connected to node 2 586 represent mobile users. They suffer in certain configurations from a 587 certain byte error rate on their access links and the delays are 588 quite high. The agents that are connected to node 3 have low 589 bandwidth access links, but do not suffer from bit errors. The last 590 agents, that are connected to node 4 have quite high bandwidth and 591 quite low delay. 593 6.2.1 Shared Losses vs Distributed Losses 595 In our first investigation, we wanted to see the influence the loss 596 characteristic on the algorithm's performance, i.e. we wanted to 597 investigate the cases where packet loss occurs for several users 598 simultaneously or totally independently. Therefore we first define 599 agent A1 to be the sender. In the shared-loss-case we insert a 600 constant byte error rate on one of the middle links, i.e. the link 601 between A1 and A2. In the case of distributed losses we inserted the 602 same byte error rate on all links downstream of A2. 604 This scenario is especially interesting, because of the feedback 605 suppression algorithm. When all receivers share the same loss, it is 606 only necessary for one of them to send the loss report. Hence if a 607 member receives feedback with the same content that it has scheduled 608 Extended RTP Profile for RTCP-based Feedback November 2001 609 - Results of the Timing Rule Simulations - 611 to be sent, it suppresses the scheduled feedback. Of course this 612 suppressed feedback does not contribute to the mean waiting times. 613 So we expect reduced waiting times for shared losses, because the 614 probability is high that one of the receivers can send the feedback 615 more or less immediately. The results are shown in the following 616 table. 618 | | Feedback Statistics | 619 | | Shared Losses | Distributed Losses | 620 |Agent|sent|fbsp|disc|sum | MWT |sent|fbsp|disc|sum | MWT | 621 +-----+----+----+----+----+-----+----+----+----+----+-----+ 622 | A2 | 274| 351| 25| 650|0.267| -| -| -| -| -| 623 | A5 | 231| 408| 11| 650|0.243| 619| 2| 32| 653|0.663| 624 | A6 | 234| 407| 9| 650|0.235| 587| 2| 32| 621|0.701| 625 | A7 | 223| 414| 13| 650|0.253| 594| 6| 41| 641|0.658| 626 | A8 | 188| 443| 19| 650|0.235| 596| 1| 32| 629|0.677| 628 Table 4: Feedback statistics for multicast simulations. 630 Table 4 shows the feedback statistics for the simulation of a large 631 group size. All 16 agents of topology T-16 joined the RTP session. 632 However only agent A1 acts as an RTP sender, the other agents are 633 pure receivers. Only 4 or 5 agents suffer from packet loss, i.e. A2, 634 A5, A6, A7 and A8 for the case of shared losses and A5, A6, A7 and 635 A8 in the case of distributed losses. Since the number of session 636 members is the same for both cases, T_rr is also the same on the 637 average. Still the mean waiting times are reduced by more than 50% 638 in the case of shared losses. This proves our assumption that shared 639 losses enhance the performance of the algorithm. 641 The feedback suppression mechanism seems to be working quite fine. 642 Even though some feedback is sent from different receivers (i.e. 643 1150 loss reports are sent in total and only 650 packets were lost, 644 resulting in loss report being received on the average 1.8 times) 645 most of the redundant feedback was suppressed. I.e. 2023 loss 646 reports were suppressed from 3250 individual detected losses, which 647 means that more than 60% of the feedback was actually suppressed. 649 6.2.2 Sender vs. Receiver 651 RTP senders are able to maintain a RTT measurement to all receivers, 652 which send receiver reports. This is done by the means of the ntp 653 timestamp in the sender report and the repetition of this value 654 together with the delay since last sender report value in the 655 receiver report. However RTP session members that do not send RTP 656 packets are not an RTP sender and thus do not send sender reports. 657 Therefore pure receivers do not have an RTT measurement to the 658 senders or other receivers. This fact is considered in AVPF, by 659 giving two possibilities to calculate T_dither_max. 661 Extended RTP Profile for RTCP-based Feedback November 2001 662 - Results of the Timing Rule Simulations - 664 If the RTP member has an RTT measurement to the sender of the packet 665 it wants to provide feedback to, it calculates T_dither_max = k * 666 T_rtt/2 * members, with k = 1. Thus t_dither_max is increased with 667 the number of session members and the RTT. The rational for RTT/2 is 668 that the distance to the sender is a good measure how long to wait 669 at maximum. Other receivers, who are more far away, i.e. have a 670 larger RTT estimation, will detect the packets later and also the 671 feedback from those would arrive later and hence have less value. 672 Thus the nearest receivers get the chance first to send their 673 feedback. Because of the larger distance of the other receivers to 674 the sender, they will probably wait longer (probably, because of the 675 randomness, i.e. we calculate T_dither_max, from which T_dither is 676 picked randomly). While those are waiting, it is likely that they 677 receive the feedback from the receivers that are nearer to the 678 source. With this it is possible to find a good compromise between 679 waiting time and feedback suppression. To let the algorithm scale to 680 large group sizes, the number of session members is included. The 681 number of members is the maximum number of receivers that shared the 682 same loss. The more members are in the session, the higher is the 683 probability that other receivers share the loss and thus the higher 684 is the value of waiting longer, because the probability is increased 685 that feedback suppression will work. If all receivers calculate the 686 same T_dither_max ( i.e. have a similar RTT estimation) and pick a 687 T_dither from this interval randomly with a uniform distribution, it 688 is likely that one feedback is sent within the first RTT interval. 690 In case the RTP session member does not have an RTT measurement, 691 i.e. it is a pure receiver, is calculates T_dither_max = l * T_rr, 692 with l = 0.5. The rational for this is that the receiver, if it has 693 no RTT estimation, does not know at all how long it should wait for 694 other receivers to send feedback. The feedback suppression algorithm 695 would certainly fail, if the time is selected too short. However the 696 waiting time is increased unnecessarily (and thus the value of the 697 feedback is decreased!) in case the time is chosen too long. It 698 would be good to find the optimum time (which is tried to be done 699 with the RTT estimation), but it is not dangerous if the optimum 700 time is not chosen. Decreased feedback value and a failure of the 701 feedback suppression mechanism do not hurt the network stability. We 702 have shown for the cases of distributed losses that the overall 703 bandwidth constraints are kept in any case and thus we could only 704 loose some performance by choosing the wrong time. A good measure 705 for T_dither_max however is the RTCP interval T_rr. This value 706 increases with the number of session members. Also we know that we 707 can send feedback at least every T_rr. Thus increasing T_dither max 708 beyond T_rr would certainly make no sense. So by choosing T_rr/2 we 709 guarantee that at least sometimes (i.e. when a loss is detected in 710 the first half of the interval between two regularly scheduled RTCP 711 packets) we are allowed to send early packets. Because of the 712 randomness of T_dither we still have a good chance to send the early 713 packet in time. 715 Extended RTP Profile for RTCP-based Feedback November 2001 716 - Results of the Timing Rule Simulations - 718 Having said that, we assume that the RTP members who have an RTT 719 measurement would perform better regarding the feedback suppression. 720 We want to show that by simulating the same scenario of the previous 721 section, but enabling all receivers that suffer from packet loss to 722 maintain a RTT measurement. We do this by declaring the 723 corresponding agents to RTP senders. However we do not send RTP 724 packets from this agents, to be comparable to the previous results. 725 The only difference to the previous simulations is that sender 726 reports are sent, which enables the sender to maintain a RTT 727 measurement. 729 | | Feedback Statistics | 730 | | Shared Losses | Distributed Losses | 731 |Agent|sent|fbsp|disc|sum | MWT |sent|fbsp|disc|sum | MWT | 732 +-----+----+----+----+----+-----+----+----+----+----+-----+ 733 | A2 | 582| 43| 7| 632|0.100| -| -| -| -| -| 734 | A5 | 70| 562| 0| 632|0.121| 644| 1| 1| 646|0.576| 735 | A6 | 60| 572| 0| 632|0.114| 638| 5| 1| 644|0.575| 736 | A7 | 73| 559| 0| 632|0.109| 607| 3| 1| 611|0.567| 737 | A8 | 63| 569| 0| 632|0.108| 626| 3| 0| 629|0.589| 739 Table 5: Feedback statistics for multicast simulations, where the 740 agents that suffer from packet loss do have an RTT estimation to the 741 sender. 743 Table 5 shows the results of the simulations. As assumed, we see 744 that the performance regarding the waiting time is increased 745 significantly. In case of shared losses, the mean time is less than 746 half of the mean waiting times of the receivers that do not have a 747 RTT estimation. Also for the case of distributed losses, we see a 748 slight gain in performance, however not as big as for the shared 749 losses. But still we see that the calculation of T-dither_max, using 750 the RTT estimation finds a better tradeoff between waiting time and 751 feedback suppression. The waiting time is reduced and the feedback 752 suppression increased where possible. Thus for both cases, whether 753 feedback suppression is possible or not, the performance is 754 increased. Feedback suppression in the case of shared losses is 755 working much better with a RTT estimation. From 3160 individual 756 detected losses only 848 loss reports are sent. 758 7 Investigations on "k" 760 The parameter k in the formula how to calculate T_Dither_max if an 761 RTT estimation is available has some influence of the performance of 762 the algorithm. Thus we investigated the effect and tried to find an 763 optimum value for k. Therefore we defined a sample scenarios and 764 tried to find an optimum value for k. 766 Extended RTP Profile for RTCP-based Feedback November 2001 767 - Results of the Timing Rule Simulations - 769 We define three representative sample scenarios. We use the topology 770 from the previous section. Most of the agents however contribute 771 only little to the simulations, because we introduced an error rate 772 only on the link between the sender A1 and the agent A2. 774 The first scenario represents cases, where losses are shared between 775 two agents. One agent is located upstream on the path between the 776 other agent and the sender. Therefore agent A2 and agent A5 see the 777 same losses, that are introduce on the link between the sender and 778 agent A2. Agent A6, A7 and A8 do not join the RTP session. From the 779 other agents only agents A3 and A9 join. Both agent A2 and A5 are 780 declared as RTP senders, in order to have an RTT estimation to the 781 sender A1. 783 The second scenario represents also cases, where losses are shared 784 between two agents, but this time the agents are located on 785 different branches of the multicast tree. The delays to the sender 786 are roughly of the same magnitude. Agent A5 and A6 share the same 787 losses. Agents A3 and A9 join the RTP session, but are pure 788 receivers and do not see any losses. 790 Also in the third scenario, the losses re shared between two agents, 791 A5 and A6. The same agents as in the second scenario are active. 792 However the delays of the links are different. The delay of the link 793 between agent A2 and A5 is reduced to 20ms and between A2 and A6 to 794 40ms. Thus the RTT estimations of agents A5 and A6 to the sender are 795 reduced significantly. 797 7.1 Feedback Suppression Performance 799 First we consider the fraction of feedback that the agent An 800 suppresses (Feedback Suppression Rate). An is thereby the agent 801 nearer to the source. The simulation results can be seen from 802 Table 7. In general it can be seen that agent An suppresses more 803 feedback if the differences between the delays to the source are 804 smaller. This is reasonable, because the feedback from other 805 receivers will be faster received in that case. It can also be seen 806 that the feedback suppression rate increases with k. This is due to 807 the fact that T_dither_max increases with k. Thus the agents will 808 wait longer on the average before sending their feedback. By 809 increasing the waiting time for all agents, the time were feedback 810 suppression is possible at all is increased. 812 Extended RTP Profile for RTCP-based Feedback November 2001 813 - Results of the Timing Rule Simulations - 815 | | Feedback Suppression Rate | 816 | k | Scen. 1 | Scen. 2 | Scen. 3 | 817 +------+---------+---------+---------+ 818 | 0.10 | 0.070 | 0.039 | 0.064 | 819 | 0.25 | 0.068 | 0.063 | 0.065 | 820 | 0.50 | 0.062 | 0.114 | 0.124 | 821 | 0.75 | 0.047 | 0.172 | 0.129 | 822 | 1.00 | 0.056 | 0.234 | 0.176 | 823 | 1.25 | 0.056 | 0.282 | 0.233 | 824 | 1.50 | 0.047 | 0.315 | 0.251 | 825 | 1.75 | 0.040 | 0.331 | 0.245 | 826 | 2.00 | 0.048 | 0.297 | 0.284 | 827 | 3.00 | 0.047 | 0.347 | 0.330 | 828 | 4.00 | 0.063 | 0.347 | 0.353 | 830 Table 7: Fraction of feedback that was suppressed at agent An of the 831 total number of feedback the agent wanted to send 833 In Table 8 the results for the feedback suppression of agent Af are 834 depicted. Again we see that the number of feedback suppressions 835 increase with k. Only in scenario 1 the number is more or less 836 constant. However by increasing the waiting times, the probability 837 that the feedback is suppressed is decreased at agent Af. k=1 seems 838 to be a threshold, where the feedback suppression does not change 839 anymore significantly in the given scenarios. This is because for 840 the given parameters, the early packets will not be sent any more, 841 because the next regularly scheduled RTCP packet will we within the 842 T_dither_max interval. 844 | | Feedback Suppression Rate | 845 | k | Scen. 1 | Scen. 2 | Scen. 3 | 846 +------+---------+---------+---------+ 847 | 0.10 | 0.736 | 0.064 | 0.071 | 848 | 0.25 | 0.814 | 0.079 | 0.119 | 849 | 0.50 | 0.859 | 0.162 | 0.239 | 850 | 0.75 | 0.865 | 0.222 | 0.376 | 851 | 1.00 | 0.844 | 0.290 | 0.401 | 852 | 1.25 | 0.850 | 0.338 | 0.429 | 853 | 1.50 | 0.849 | 0.316 | 0.473 | 854 | 1.75 | 0.868 | 0.316 | 0.505 | 855 | 2.00 | 0.843 | 0.376 | 0.487 | 856 | 3.00 | 0.845 | 0.345 | 0.502 | 857 | 4.00 | 0.820 | 0.345 | 0.493 | 859 Table 8 Fraction of feedback that was suppressed at agent Af of the 860 total number of feedback the agent wanted to send 862 In Table 9 the ration of feedback suppression failures is 863 illustrated. In general the observations from the figures above are 864 summarized. The ratio of feedback failures decreases with an 865 Extended RTP Profile for RTCP-based Feedback November 2001 866 - Results of the Timing Rule Simulations - 868 increasing k for the scenarios 2 and 3. In scenario 1 the ratio is 869 hardly influenced at all by k. The simulations show a kind of steady 870 state at k larger two or three, where the rationale for this is that 871 for very large k, T_dither_max becomes equal or more than T_rr and 872 thus no early packets are send any more. The maximum dithering 873 interval is for these cases limited by the next regularly scheduled 874 RR. 876 | |Feedback Suppr. Failure Rate | 877 | k | Scen. 1 | Scen. 2 | Scen. 3 | 878 +------+---------+---------+---------+ 879 | 0.10 | 0.194 | 0.897 | 0.865 | 880 | 0.25 | 0.117 | 0.858 | 0.816 | 881 | 0.50 | 0.079 | 0.725 | 0.638 | 882 | 0.75 | 0.088 | 0.606 | 0.495 | 883 | 1.00 | 0.100 | 0.468 | 0.423 | 884 | 1.25 | 0.094 | 0.381 | 0.338 | 885 | 1.50 | 0.104 | 0.369 | 0.276 | 886 | 1.75 | 0.092 | 0.353 | 0.250 | 887 | 2.00 | 0.110 | 0.328 | 0.229 | 888 | 3.00 | 0.108 | 0.308 | 0.169 | 889 | 4.00 | 0.116 | 0.308 | 0.154 | 891 Table 8: The ratio of feedback suppression failures. 893 Summarizing, it can be said, that the feedback suppression 894 performance is highly dependent on the topology, the parameters and 895 configurations. 897 In general a larger value for k increases the probability that the 898 feedback suppression works, however the performance gain decreases 899 with an increasing k. For a certain threshold, depending on the 900 configuration and environment, an increasing k does not lead to any 901 performance gain any more. 903 7.2 Loss Report Delay 905 In this section we investigate the influence of the parameter k on 906 the loss report delay. Therefore we measured for the three sample 907 scenarios the mean loss report delay as seen by the sender, i.e. the 908 sender calculates for every loss report, it receives for the first 909 time the delay since the corresponding packet was sent. 911 The results are depicted in Table 9. In general it can be said, that 912 the loss report delay increases with k. This is only natural, 913 because T_Dither_max is proportional to k. Thus the agents wait on 914 the average longer to send their early packets. In cases of very 915 large k values, the report delay does not increase significantly any 916 more. In these cases nearly no early packets are sent, because the 917 Extended RTP Profile for RTCP-based Feedback November 2001 918 - Results of the Timing Rule Simulations - 920 next regularly scheduled packet is within the T_dither_max interval. 921 The threshold of k, from which on the delay will not increase, is 922 dependent on the RTT estimation. For increasing RTT values, the 923 threshold decreases. We see that in scenario 1 the threshold lies 924 between k=2 and k=3. For the scenarios with smaller RTT, the 925 threshold is higher. 927 Summarizing it can be said, that the report delay increases with an 928 increasing k. From a certain threshold the increase is not 929 significant, however this threshold is highly dependent on topology 930 and environment parameters. 932 | | Mean Loss Report Delay | 933 | k | Scen. 1 | Scen. 2 | Scen. 3 | 934 +------+---------+---------+---------+ 935 | 0.10 | 0.128 | 0.282 | 0.431 | 936 | 0.25 | 0.135 | 0.266 | 0.430 | 937 | 0.50 | 0.150 | 0.264 | 0.497 | 938 | 0.75 | 0.160 | 0.286 | 0.538 | 939 | 1.00 | 0.194 | 0.305 | 0.613 | 940 | 1.25 | 0.203 | 0.329 | 0.661 | 941 | 1.50 | 0.208 | 0.363 | 0.690 | 942 | 1.75 | 0.209 | 0.387 | 0.739 | 943 | 2.00 | 0.242 | 0.412 | 0.764 | 944 | 3.00 | 0.243 | 0.507 | 0.790 | 945 | 4.00 | 0.287 | 0.568 | 0.790 | 947 Table 9: The mean loss report delay, measured at the sender. 949 7.3 Summary of "k" investigations 951 We have shown by simulations that the parameter k influence the 952 feedback performance. While in general the feedback suppression 953 performance increases with k, the report delay increases also. Hence 954 we need to find a tradeoff, between the amount of feedback that is 955 sent and the delay of the feedback, when it is received at the 956 sender. Since we have shown that the performance curves for the 957 feedback suppression as well as the report delay is highly variable 958 for different topologies and environments, it is not possible to 959 give an optimized parameter value for k. We think that k=1 is a 960 compromise, which should be acceptable for most of our considered 961 cases. At least we guarantee with k=1 that no feedback explosion 962 will occur and thus keep the network stability untouched. 964 Extended RTP Profile for RTCP-based Feedback November 2001 965 - Results of the Timing Rule Simulations - 967 8 Investigations on "l" 969 In this section we want to investigate the influence of the 970 parameter "l" from the T_Dither_max calculation in agents that do 971 not have an RTT estimation to the sender. As we have done in the 972 previous section for the parameter "k", we investigate the feedback 973 suppression performance as well as the report delay for three sample 974 scenarios. For simplicity we use the same scenarios as in the 975 previous section, but this time the all agents beside agent A1 are 976 pure RTP receivers. Thus these agents do not have an RTT estimation 977 to the source. T_Dither_Max is calculated with the other formula, 978 depending only on T_rr and l, which means that all agents should 979 calculate roughly the same T_Dither_Max. 981 8.1 Feedback Suppression Performance 983 The results for the feedback suppression rate of the agent Af that 984 is more far away from the sender, are depicted in Table 10. In 985 general it can be seen that the feedback suppression rate increases 986 with an increasing l. However there is a threshold, depending on the 987 environment, from which the additional gain is not significant any 988 more. 990 | | Feedback Suppression Rate | 991 | l | Scen. 1 | Scen. 2 | Scen. 3 | 992 +------+---------+---------+---------+ 993 | 0.10 | 0.671 | 0.051 | 0.089 | 994 | 0.25 | 0.582 | 0.060 | 0.210 | 995 | 0.50 | 0.524 | 0.114 | 0.361 | 996 | 0.75 | 0.523 | 0.180 | 0.370 | 997 | 1.00 | 0.523 | 0.204 | 0.369 | 998 | 1.25 | 0.506 | 0.187 | 0.372 | 999 | 1.50 | 0.536 | 0.213 | 0.414 | 1000 | 1.75 | 0.526 | 0.215 | 0.424 | 1001 | 2.00 | 0.535 | 0.216 | 0.400 | 1002 | 3.00 | 0.522 | 0.220 | 0.405 | 1003 | 4.00 | 0.522 | 0.220 | 0.405 | 1005 Table 10: Fraction of feedback that was suppressed at agent An of 1006 the total number of feedback the agent wanted to send 1007 Extended RTP Profile for RTCP-based Feedback November 2001 1008 - Results of the Timing Rule Simulations - 1010 Similar results can be seen for the agent that is nearer to the 1011 sender in Table 11. 1013 | | Feedback Suppression Rate | 1014 | l | Scen. 1 | Scen. 2 | Scen. 3 | 1015 +------+---------+---------+---------+ 1016 | 0.10 | 0.056 | 0.056 | 0.090 | 1017 | 0.25 | 0.063 | 0.055 | 0.166 | 1018 | 0.50 | 0.116 | 0.099 | 0.255 | 1019 | 0.75 | 0.141 | 0.141 | 0.312 | 1020 | 1.00 | 0.179 | 0.175 | 0.352 | 1021 | 1.25 | 0.206 | 0.176 | 0.361 | 1022 | 1.50 | 0.193 | 0.193 | 0.337 | 1023 | 1.75 | 0.197 | 0.204 | 0.341 | 1024 | 2.00 | 0.207 | 0.207 | 0.368 | 1025 | 3.00 | 0.196 | 0.203 | 0.359 | 1026 | 4.00 | 0.196 | 0.203 | 0.359 | 1028 Table 11: Fraction of feedback that was suppressed at agent An of 1029 the total number of feedback the agent wanted to send 1031 The rate of feedback suppression failure is depicted in Table 12. 1032 The trend that the additional performance increase is not 1033 significant from a certain threshold, depending on the environment 1034 is here as well visible. 1036 | |Feedback Suppr. Failure Rate | 1037 | l | Scen. 1 | Scen. 2 | Scen. 3 | 1038 +------+---------+---------+---------+ 1039 | 0.10 | 0.273 | 0.893 | 0.822 | 1040 | 0.25 | 0.355 | 0.885 | 0.624 | 1041 | 0.50 | 0.364 | 0.787 | 0.385 | 1042 | 0.75 | 0.334 | 0.679 | 0.318 | 1043 | 1.00 | 0.298 | 0.621 | 0.279 | 1044 | 1.25 | 0.289 | 0.637 | 0.267 | 1045 | 1.50 | 0.274 | 0.595 | 0.249 | 1046 | 1.75 | 0.274 | 0.580 | 0.235 | 1047 | 2.00 | 0.258 | 0.577 | 0.233 | 1048 | 3.00 | 0.282 | 0.577 | 0.236 | 1049 | 4.00 | 0.282 | 0.577 | 0.236 | 1051 Table 12: The ratio of feedback suppression failures. 1053 Summarizing the feedback suppression results it can be said that in 1054 general the feedback suppression performance increases with an 1055 increasing l. However from a certain threshold, depending on 1056 environment parameters such as propagation delays or session 1057 bandwidth, the additional increase is not significant anymore. 1059 Extended RTP Profile for RTCP-based Feedback November 2001 1060 - Results of the Timing Rule Simulations - 1062 8.2 Loss Report Delay 1064 In this section we show the results for the measured report delay 1065 during the simulations of the three sample scenarios. This 1066 measurement is a metric of the performance of the algorithms, 1067 because the value of the feedback for the sender typically decreases 1068 with the delay of its reception. The loss report delay is measured 1069 as the time at the sender between sending a packet and receiving the 1070 first corresponding loss report. 1072 | | Mean Loss Report Delay | 1073 | l | Scen. 1 | Scen. 2 | Scen. 3 | 1074 +------+---------+---------+---------+ 1075 | 0.10 | 0.124 | 0.282 | 0.210 | 1076 | 0.25 | 0.168 | 0.266 | 0.234 | 1077 | 0.50 | 0.243 | 0.264 | 0.284 | 1078 | 0.75 | 0.285 | 0.286 | 0.325 | 1079 | 1.00 | 0.329 | 0.305 | 0.350 | 1080 | 1.25 | 0.351 | 0.329 | 0.370 | 1081 | 1.50 | 0.361 | 0.363 | 0.388 | 1082 | 1.75 | 0.360 | 0.387 | 0.392 | 1083 | 2.00 | 0.367 | 0.412 | 0.400 | 1084 | 3.00 | 0.368 | 0.507 | 0.398 | 1085 | 4.00 | 0.368 | 0.568 | 0.398 | 1087 Table 13: The mean loss report delay, measured at the sender. 1089 As can be seen from Table 13 the delay increases in general with an 1090 increasing value of l. However a similar effect as for the feedback 1091 suppression performance is visible: from a certain threshold, the 1092 additional increase in delay is not significant anymore. The 1093 threshold is environment dependent and seems to be related to the 1094 threshold, where the feedback suppression gain would not increase 1095 anymore. 1097 8.3 Summary of "l" investigations 1099 We have shown that theoretically the performance of the feedback 1100 suppression mechanisms is increasing with an increasing value of l. 1101 The same applies for the report delay, which increases also with an 1102 increasing l. This leads to a threshold where both the performance 1103 and the delay does not increase any further. The threshold is 1104 environment dependent. 1106 So finding an optimum value of l is not possible because it is 1107 always a tradeoff between delay and feedback suppression 1108 performance. With l=0.5 we think that a tradeoff was found that is 1109 acceptable for typical applications and environments. 1111 Extended RTP Profile for RTCP-based Feedback November 2001 1112 - Results of the Timing Rule Simulations - 1114 9 Applications Using AVPF 1116 NEWPRED is one of the error resilience tools, which is defined in 1117 both ISO/IEC MPEG-4 visual part and ITU-T H.263. NEWPRED achieves 1118 fast error recovery using feedback messages. We simulated the 1119 behavior of NEWPRED in the network simulator environment as 1120 described above and measured the waiting time statistics, in order 1121 verify that the extended RTP profile for RTCP-based feedback 1122 (AVPF)[1] is appropriate for the NEWPRED feedback messages. 1123 Simulation results, which present in the following sections, show 1124 that the waiting time is enough small to get the satisfactory 1125 performance of NEWPRED. 1127 9.1 NEWPRED Implementation in NS2 1129 The agent that performs the NEWPRED functionality, called NEWPRED 1130 agent, is different from the RTP agent we described above. Some of 1131 the added features and functionalities are described in the 1132 following points: 1134 Application Feedback 1135 The "Application Layer Feedback Messages" format is used to 1136 transmit the NEWPRED feedback messages. Thereby the NEWPRED 1137 functionality is added to the RTP agent. The NEWPRED agent creates 1138 one NACK message for each lost segment of a video frame, and then 1139 assembles plural number of NACK messages corresponding to the 1140 segments in the same video frame, into one Application Layer 1141 Feedback Message. Although there are two modes, namely NACK mode 1142 and ACK mode in NEWPRED [6][7], only NACK mode is used in these 1143 simulations. 1144 The parameters of NEWPRED agent are as follows: 1145 f: Frame Rate(frames/sec) 1146 seg: Number of segments in one video frame 1147 bw: RTP session bandwidth(kbps) 1149 Generation of NEWPRED's NACK Messages 1150 The NEWPRED agent generates NACK messages when segments are lost. 1151 a. The NEWPRED agent generates plural number of NACK messages per 1152 one video frame when plural number of segments are lost. These 1153 are assembled into one FCI message per video frame. If there is 1154 no lost segment, no message is generated and sent. 1155 b. The length of one NACK message is 4 bytes. Let num be the 1156 number of NACK messages in one video frame(1 <= num <= seg). 1157 Thus, 12+4*num bytes is the size of the low delay RTCP feedback 1158 message. 1160 Measurements 1161 We defined two values to be measured: 1163 Extended RTP Profile for RTCP-based Feedback November 2001 1164 - Results of the Timing Rule Simulations - 1166 - Recovery time 1167 The recovery time is measured as the time between the detection 1168 of a lost segment and reception of a recovered segment. We 1169 measured this "recovery time" for each lost segment. 1170 - Waiting time 1171 The waiting time is the additional delay due to the feedback 1172 limitation of RTP. 1174 Fig.1 depicts the behavior of a NEWPRED agent when a loss occurs. 1175 The recovery time is approximated as follows: 1176 (Recovery time) = (Waiting time) + 1177 (Transmission time for feedback message) + 1178 (Transmission time for media data) 1180 Therefore, the waiting time is derived as follows: 1182 (Waiting time) = (Recovery time) - (Round-trip delay), where 1184 (Round-trip delay ) = (Transmission time for feedback message) + 1185 (Transmission time for media data) 1187 Picture Reference |: Picture Segment 1188 ____________________ %: Lost Segment 1189 /_ _ _ _ \ 1190 v/ \ / \ / \ / \ \ 1191 v \v \v \v \ \ 1192 Sender ---|----|----|----|----|----|---|-------------> 1193 \ \ ^ \ 1194 \ \ / \ 1195 \ \ / \ 1196 \ v / \ 1197 \ x / \ 1198 \ Lost / \ 1199 \ x / \ _____ 1200 v x / NACK v 1201 Receiver ---------------|----%===-%----%----%----|-----> 1202 |-a-| | 1203 |------- b -------| 1205 a: Waiting time 1206 b: Recover time (%: Video segments are lost) 1208 Fig.1: Relation between the measured values at the NEWPRED agent 1209 Extended RTP Profile for RTCP-based Feedback November 2001 1210 - Results of the Timing Rule Simulations - 1212 9.2 Simulation 1214 We conducted two simulations (Simulation A and Simulation B). In 1215 Simulation A, the packets are dropped with a fixed packet loss rate 1216 on a link between two NEWPRED agents. In Simulation B, packet loss 1217 occurs due to congestion from other traffic sources, i.e. ftp 1218 sessions. 1220 9.2.1. Simulation A - Constant Packet Loss Rate 1222 The network topology, used for this simulation is shown in Fig.2. 1224 Link 1 Link 2 Link 3 1225 +--------+ +------+ +------+ +--------+ 1226 | Sender |------|Router|-------|Router|------|Receiver| 1227 +--------+ +------+ +------+ +--------+ 1228 10(msec) x(msec) 10(msec) 1230 Fig2. Network topology that is used for Simulation A 1232 Link1 and link3 are error free, and each link delay is 10 msec. 1233 Packets may get dropped on link2. The packet loss rates (Plr) and 1234 link delay (D) are as follows: 1236 D [ms] = {10, 50, 100, 200, 500} 1237 Plr = {0.005, 0.01, 0.02, 0.03, 0.05, 0.1, 0.2} 1238 Session band width, frame rate and the number of segments are 1239 shown in Table 14 1241 +------------+----------+-------------+-----+ 1242 |Parameter ID| bw(kbps) |f (frame/sec)| seg | 1243 +------------+----------+-------------+-----+ 1244 | 32k-4-3 | 32 | 4 | 3 | 1245 | 32k-5-3 | 32 | 5 | 3 | 1246 | 64k-5-3 | 64 | 5 | 3 | 1247 | 64k-10-3 | 64 | 10 | 3 | 1248 | 128k-10-6 | 128 | 10 | 6 | 1249 | 128k-15-6 | 128 | 15 | 6 | 1250 | 384k-15-6 | 384 | 15 | 6 | 1251 | 384k-30-6 | 384 | 30 | 6 | 1252 | 512k-30-6 | 512 | 30 | 6 | 1253 | 1000k-30-9 | 1000 | 30 | 9 | 1254 | 2000k-30-9 | 2000 | 30 | 9 | 1255 +------------+----------+-------------+-----+ 1256 Extended RTP Profile for RTCP-based Feedback November 2001 1257 - Results of the Timing Rule Simulations - 1259 Table 14: Parameter sets of the NEWPRED agents 1261 Figure3 shows the packet loss rate vs. mean of waiting time. A 1262 plotted line represents a parameter ID ( "[session bandwidth] - 1263 [frame rate] - [the number of segments] - [link2 delay]" ). 1264 E.g. 384k-15-9-100 means the session of 384kbps session bandwidth, 1265 15 frames per second, 9 segments per frame and 100msec link delay. 1267 When the packet loss rate is 5% and the session bandwidth is 32kbps, 1268 the waiting time is around 400msec, which is just allowable for 1269 reasonable NEWPRED performance. 1271 When the packet loss rate is less than 1%, the waiting time is less 1272 than 200msec. In such a case, the NEWPRED allows as much as 200msec 1273 additional link delay. 1275 When the packet loss rate is less than 5% and the session bandwidth 1276 is 64kbps, the waiting time is also less than 200msec. 1278 In 128kbps cases, the result shows that when the packet loss rate is 1279 20%, the waiting time is around 200msec. In cases with more than 1280 512kbps session bandwidth, there is no significant delay. This means 1281 that the waiting time due to the feedback limitation of RTCP is 1282 neglectable for the NEWPRED performance. 1284 +------------------------------------------------------------+ 1285 | | Packet Loss Rate = | 1286 | Bandwidth | 0.005| 0.01 | 0.02 | 0.03 | 0.05 |0.10 |0.20 | 1287 |-----------+------+------+------+------+------+------+------| 1288 | 32k |130- |200- |230- |280- |350- |470- |560- | 1289 | | 180| 250| 320| 390| 430| 610| 780| 1290 | 64k | 80- |100- |120- |150- |180- |210- |290- | 1291 | | 130| 150| 180| 190| 210| 300| 400| 1292 | 128k | 60- | 70- | 90- |110- |130- |170- |190- | 1293 | | 70| 80| 100| 120| 140| 190| 240| 1294 | 384k | 30- | 30- | 30- | 40- | 50- | 50- | 50- | 1295 | | 50| 50| 50| 50| 60| 70| 90| 1296 | 512k | < 50 | < 50 | < 50 | < 50 | < 50 | < 50 | < 60 | 1297 | | | | | | | | | 1298 | 1000k | < 50 | < 50 | < 50 | < 50 | < 50 | < 50 | < 55 | 1299 | | | | | | | | | 1300 | 2000k | < 30 | < 30 | < 30 | < 30 | < 30 | < 35 | < 35 | 1301 +------------------+------+------+------+------+------+------+ 1303 Fig. 3 The result of simulation A 1305 9.2.2. Simulation B - Packet Loss due to Congestion 1306 Extended RTP Profile for RTCP-based Feedback November 2001 1307 - Results of the Timing Rule Simulations - 1309 The configuration of link1, link2, and link3 are the same as in 1310 simulation A except that link2 is also error-free, regarding bit 1311 errors. However in addition, some FTP agents are deployed to 1312 overload link2. See Figure 4 for the simulation topology. 1314 Link1 Link2 Link3 1315 +--------+ +------+ +------+ +--------+ 1316 | Sender |------|Router|-------|Router|------|Receiver| 1317 +--------+ /|+------+ +------+|\ +--------+ 1318 +---+/ | | \+---+ 1319 +-|FTP|+---+ +---+|FTP|-+ 1320 | +---+|FTP| ... |FTP|+---+ | ... 1321 +---+ +---+ +---+ +---+ 1323 FTP Agents FTP Agents 1325 Fig4. Network Topology of Simulation B 1327 The parameters are defined as for Simulation A with the following 1328 values assigned: 1330 D[ms] ={10, 50, 100, 200, 500} 1331 32 FTP agents are deployed at each edge, and totally 64 FTP 1332 agents are active. 1333 The sets of session bandwidth, frame rate, the number of segments 1334 are the same as in Simulation A (Table 14) 1336 We provide the results for the cases of 64 FTP agents, because these 1337 are the cases where packet losses could be detected stable. The 1338 results are similar to the Simulation A except for a constant 1339 additional offset of 50..100ms. This is due to the delay incurred by 1340 the routers buffers. 1342 9.3 Summary of Application Simulations 1344 We have shown that the limitations of RTP AVPF profile do not 1345 generate such high delay to the feedback messages that the 1346 performance of NEWPRED is degraded in the sessions from 32kbps to 1347 2Mbps. We could see that the waiting time increases with a 1348 decreasing session bandwidth and/or an increasing packet loss rate. 1349 Thereby it is not significant what the packet loss caused. 1351 Extended RTP Profile for RTCP-based Feedback November 2001 1352 - Results of the Timing Rule Simulations - 1354 Congestion or constant packet loss rates behave similar. Still we 1355 see that for reasonable conditions and parameters the AVPF is well 1356 suited to support the feedback needed for NEWPRED. 1358 10 Summary 1360 The new RTP profile AVPF was investigated regarding performance and 1361 potential dangers to the network stability. Simulations were 1362 conducted using the network simulator, simulating unicast and 1363 different sized multicast topologies. The results were shown in this 1364 document. 1366 Regarding the network stability, it was important to show that the 1367 new profile does not lead to any feedback explosion, or use more 1368 bandwidth as it is allowed. Thus we measured the bandwidth that was 1369 used for RTCP in relation to the RTP session bandwidth. We have 1370 shown that more or less exactly 5% of the session bandwidth is used 1371 for RTCP, in all considered scenarios. The scenarios included 1372 unicast with and without bit errors, different sized multicast 1373 groups, with and without errors or congestion on the links. Thus we 1374 can say that the new profile behaves network friendly in that sense 1375 that it uses only the allowed bandwidth that was assigned by RTP. 1377 Second we have shown that receivers using the new profile experience 1378 a performance gain. We have shown that especially RTP receiver that 1379 do have an RTT estimation to the sender gain from using the new 1380 profile. But also the other receivers could increase their 1381 performance. This was measured by the delay that the sender sees for 1382 the received feedback. Using the new profile this delay can be 1383 decreased by orders of magnitude. 1385 Third we investigated certain parameters of the new algorithms. We 1386 have shown that there does not exist an optimum value for those. The 1387 influence of the parameters is highly environment specific and a 1388 tradeoff between performance of the feedback suppression algorithm 1389 and the experienced delay has to be found. The values that are given 1390 in the draft seem to be reasonable for most applications and 1391 environments. 1393 Extended RTP Profile for RTCP-based Feedback November 2001 1394 - Results of the Timing Rule Simulations - 1396 References 1398 [1] J.Ott, S.Wenger, S.Fukunaga, N.Sato, K.Yano, A.Miyazaki, 1399 K.Hata, R.Hakenberg, C.Burmeister: Extended RTP Profile for 1400 RTCP-based Feedback, Internet Draft, 1401 draft-ietf-avt-rtcp-feedback-00.txt, Work in Progress, 1402 July 2001. 1404 [2] H.Schulzrinne, S.Casner, R.Frederick, and V.Jacobson: 1405 RTP - A Transport Protocol for Real-time Applications, 1406 Internet Draft, draft-ietf-avt-rtp-new-10.txt, Work in 1407 Progress, July 2001. 1409 [3] H.Schulzrinne, S.Casner: RTP Profile for Audio and Video 1410 Conferences with Minimal Control, Internet Draft, 1411 draft-ietf-avt-profile-new-11.txt, Work in Progress, July 2001. 1413 [4] Network Simulator Version 2 - ns-2, available from 1414 http://www.isi.edu/nsnam/ns 1416 [5] C.Burmeister, T.Klinner: Low Delay Feedback RTCP - Timing Rules 1417 Simulation Results. Technical Report of the Panasonic European 1418 Laboratories, September 2001, available from 1419 http://www.pel.panasonic.de/ietf/docs/SimulationResults-A.pdf 1421 [6] ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology - 1422 Coding of audio-visual objects - Part2: Visual", July 2000. 1424 [7] ITU-T Recommendation, H.263. Video encoding for low bitrate 1425 communication. 1998. 1427 [8] S. Fukunaga, T. Nakai, and H. Inoue, "Error Resilient Video 1428 Coding by Dynamic Replacing of Reference Pictures," IEEE Global 1429 Telecommunications Conference (GLOBECOM), pp.1503-1508, 1996. 1431 [9] Hideaki Kimata, Yasuhiro Tomita, Hiroyuki Yamaguchi, Susumu 1432 Ichinose, and Tadashi Ichikawa, "Receiver-Oriented Real-Time 1433 Error Resilient Video Communication System: Adaptive Recovery 1434 from Error Propagation in Accordance with Memory Size at 1435 Receiver," Electronics and Communications in Japan, Part 1, 1436 vol.84, no.2, pp.8-17, 2001. 1438 Extended RTP Profile for RTCP-based Feedback November 2001 1439 - Results of the Timing Rule Simulations - 1441 Authors Addresses 1443 Carsten Burmeister 1444 Panasonic European Laboratories GmbH 1445 Monzastr. 4c, 63225 Langen, Germany 1446 mailto:burmeister@panasonic.de 1448 Rolf Hakenberg 1449 Panasonic European Laboratories GmbH 1450 Monzastr. 4c, 63225 Langen, Germany 1451 mailto:hakenberg@panasonic.de 1453 Akihiro Miyazaki 1454 Matsushita Electric Industrial Co., Ltd 1455 1006, Kadoma, Kadoma City, Osaka, Japan 1456 mailto :akihiro@isl.mei.co.jp 1458 J�rg Ott 1459 Universit�t Bremen TZI 1460 MZH 5180, Bibliothekstr. 1, 28359 Bremen, Germany 1461 {sip,mailto}:jo@tzi.uni-bremen.de 1463 Noriyuki Sato 1464 Oki Electric Industry Co., Ltd. 1465 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1466 mailto:sato652@oki.co.jp 1468 Shigeru Fukunaga 1469 Oki Electric Industry Co., Ltd. 1470 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1471 mailto:fukunaga444@oki.co.jp