idnits 2.17.1 draft-ietf-avt-rtcp-feedback-00.txt: -(2): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(158): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(411): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(578): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1134): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1295): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1298): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1761): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == There are 11 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 35 longer pages, the longest (page 2) being 62 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Authors' Addresses Section. ** There are 481 instances of too long lines in the document, the longest one being 4 characters in excess of 72. ** The abstract seems to contain references ([2], [10], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. == There are 17 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 2 instances of lines with multicast IPv4 addresses in the document. If these are generic example addresses, they should be changed to use the 233.252.0.x range defined in RFC 5771 ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? RFC 2119 keyword, line 245: '...tiple FB messages MAY be combined in a...' RFC 2119 keyword, line 246: '... packet and they MAY also be sent comb...' RFC 2119 keyword, line 250: '... MUST contain RTCP packets in the order as defined in [1]:...' RFC 2119 keyword, line 252: '... . OPTIONAL encryption prefix tha...' RFC 2119 keyword, line 255: '...ATORY SDES which MUST contain the CNAM...' (84 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 736 has weird spacing: '...r which the "...' == Line 1084 has weird spacing: '... sender knows...' == Line 1510 has weird spacing: '... Mail fukun...' == Line 1517 has weird spacing: '... Mail sato6...' == Line 1530 has weird spacing: '... Mail akihi...' == (3 more instances...) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (January 2002) is 8137 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 1753 looks like a reference -- Missing reference section? '10' on line 1795 looks like a reference -- Missing reference section? '2' on line 1559 looks like a reference -- Missing reference section? '7' on line 1774 looks like a reference -- Missing reference section? '11' on line 1588 looks like a reference -- Missing reference section? '5' on line 1569 looks like a reference -- Missing reference section? '6' on line 1572 looks like a reference -- Missing reference section? '9' on line 1582 looks like a reference -- Missing reference section? '8' on line 1579 looks like a reference -- Missing reference section? '3' on line 1563 looks like a reference -- Missing reference section? '4' on line 1566 looks like a reference -- Missing reference section? '14' on line 1599 looks like a reference -- Missing reference section? '13' on line 1596 looks like a reference -- Missing reference section? '15' on line 1602 looks like a reference -- Missing reference section? '12' on line 1593 looks like a reference -- Missing reference section? '16' on line 1606 looks like a reference -- Missing reference section? '124' on line 1617 looks like a reference Summary: 8 errors (**), 0 flaws (~~), 12 warnings (==), 20 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT J�rg Ott/Universit�t Bremen TZI 3 draft-ietf-avt-rtcp-feedback-00.txt Stephan Wenger/TU Berlin 4 Shigeru Fukunaga/Oki 5 Noriyuki Sato/Oki 6 Koichi Yano/Fast Forward Networks 7 Akihiro Miyazaki/Matsushita 8 Koichi Hata/Matsushita 9 Rolf Hakenberg/Matsushita 10 Carsten Burmeister/Matsushita 12 13 July, 2001 13 Expires January 2002 15 Extended RTP Profile for RTCP-based Feedback 17 Status of this Memo 19 This document is an Internet-Draft and is in full conformance with all 20 provisions of Section 10 of RFC 2026. Internet-Drafts are working 21 documents of the Internet Engineering Task Force (IETF), its areas, and 22 its working groups. Note that other groups may also distribute working 23 documents as Internet-Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six months 26 and may be updated, replaced, or obsoleted by other documents at any 27 time. It is inappropriate to use Internet- Drafts as reference material 28 or to cite them other than as "work in progress." 30 The list of current Internet-Drafts can be accessed at 31 http://www.ietf.org/ietf/1id-abstracts.txt 33 The list of Internet-Draft Shadow Directories can be accessed at 34 http://www.ietf.org/shadow.html. 36 Abstract 38 Real-time media streams are not resilient against packet losses. RTP 39 [1] provides all the necessary mechanisms to restore ordering and 40 timing to properly reproduce a media stream at the recipient. RTP 41 also provides continuous feedback about the overall reception quality 42 from all receivers -- thereby allowing the sender(s) in the mid-term 43 (in the order of several seconds to minutes) to adapt their coding 44 scheme and transmission behavior to the observed network QoS. 45 However, except for a few payload specific mechanisms [10], RTP makes 46 no provision for timely feedback that would allow a sender to repair 47 the media stream immediately: through retransmissions, retro-active 48 FEC, or media-specific mechanisms such as reference picture 49 selection. 51 Generally, real-time transport of media streams across IP networks 52 follows RTP[1] in conjunction with the RTP Profile for Audio and 53 Video Conferences with Minimal Control [2]. This document modifies 54 the profile defined in [2] in two ways: 56 . by providing additional RTCP messages that enable a receiver to 57 convey more precise feedback to a sender and 59 . by adapting the timing algorithm for scheduling RTCP packets in 60 order to allow for occasional timely feedback about events 61 observed by a receiver (such as lost packets). 63 The result is an RTP Profile for Audio and Video Conferences with 64 Minimal Control that allows for more explicit and more immediate 65 receiver feedback but shares all other properties (including all 66 other message types and formats, all code points for codecs, payload 67 formats, scaling capabilities, etc. of [2]). Therefore, this 68 document only specifies the additions and modifications to [2] rather 69 than the repeating the entire specification. 71 1. Introduction 73 Real-time media streams are not resilient against packet losses. RTP 74 [1] provides all the necessary mechanisms to restore ordering and 75 timing present at the sender to properly reproduce a media stream at 76 a recipient. RTP also provides continuous feedback about the overall 77 reception quality from all receivers -- thereby allowing the 78 sender(s) in the mid-term (in the order of several seconds to 79 minutes) to adapt their coding scheme and transmission behavior to 80 the observed network QoS. However, except for a few payload specific 81 mechanisms [10], RTP makes no provision for timely feedback that 82 would allow a sender to repair the media stream immediately: through 83 retransmissions, retro-active FEC, or media-specific mechanisms such 84 as reference picture selection. 86 Current mechanisms available with RTP to improve error resilience 87 include audio redundancy coding [7], video redundancy coding [11], 88 RTP-level FEC [5], and general considerations on more robust media 89 streams transmission [6]. Particularly in small groups, however, 90 virtually all kinds of real-time media streams could benefit from a 91 mechanism that would enable a sender to perform media stream repair - 92 - including but not limited to audio, video, DTMF, and text chat 93 streams. In some cases of networks with acceptable round-trip times 94 but scarce bandwidth, occasional retransmissions may be much 95 preferred over continuous transmission of redundant information. 97 For example, predictive video coding is not loss resilient. Any loss 98 of coded data leads to annoying artifacts not only in the reproduced 99 picture in which the loss occurred, but also in subsequent pictures. 100 Error resilience can be achieved by allocating bits to convey 101 redundant information using source coding based mechanisms or 102 transport based mechanisms. This can be done without the use of any 103 feedback between the decoder(s) and the encoder. Similar 104 consideration apply to protecting e.g. DTMF (and other tones) carried 105 in an RTP stream [9]. 107 Alternatively, where applicable, receivers can inform the sender 108 through a feedback channel about a loss situation, and the sender can 109 react accordingly. This approach provides better media quality and 110 is more efficient with respect to the bandwidth used by the sender to 111 achieve a given media quality. However, using feedback mechanisms is 112 limited to certain application scenarios identified by encoder 113 characteristics, delay constraints, and/or the number of recipients. 115 This document specifies a modified RTP Profile for Audio and Video 116 conferences with minimal control based upon [1] and [2] by means two 117 modifications/additions: 119 To achieve timely feedback the concepts of Immediate Feedback 120 messages and Early RTCP messages as well as algorithms allowing for 121 low delay feedback in small multicast groups (and preventing feedback 122 implosion in large ones) are introduced. Special consideration is 123 given to point-to-point scenarios. 125 In addition, various types of general-purpose feedback messages as 126 well as a format for codec and application-specific feedback 127 information are defined as specific RTCP payloads. 129 1.1 Definitions 131 The definitions from [1] and [2] apply. In addition, the following 132 definitions are used in this document: 134 Early RTCP mode: 135 The mode of operation in which a receiver of a media stream 136 is, statistically, often (but not always) capable of 137 reporting events of interest back to the sender close to 138 their occurrence. In Early RTCP mode, RTCP feedback messages 139 are transmitted according to the timing rules defined in this 140 document. 142 Early RTCP packet: 143 An Early RTCP packet is a packet which is transmitted earlier 144 than would be allowed following the scheduling algorithm of 145 [1], the reason being that an event observed by a receiver. 146 Early RTCP packets may be sent in Immediate feedback and in 147 Early RTCP mode. 149 Event: 150 An observation made by the receiver of a media stream that is 151 (potentially) of interest to the sender -- such as a packet 152 loss or packet reception, frame loss, etc. -- and thus to be 153 reported back to the sender by means of a Feedback message. 155 Feedback (FB) message: 156 An RTCP message as defined in this document used to convey 157 events observed at a receiver -- in addition to long term 158 receiver status information which is carried in RTCP RRs � 159 back to the sender of the media stream. 161 Feedback (FB) threshold: 162 The FB threshold indicates the "borderline" between Immediate 163 Feedback and Early RTCP mode. For a multicast scenario, the 164 FB threshold indicates the maximum group size at which, on 165 average, each receiver is able to report each event back to 166 the sender(s) immediately, i.e. without having to wait for 167 its regularly scheduled RTCP interval. This threshold is 168 highly dependent on network QoS (e.g. packet loss probability 169 and distribution), codec and packetization in use, and 170 application requirements. Hence, no formal definition is 171 presented in this document. 173 Immediate Feedback mode: 174 Mode of operation in which each receiver of a media is, 175 statistically, capable of reporting each event of interest 176 immediately back to the media stream sender. In Immediate 177 Feedback mode, RTCP feedback messages are transmitted 178 according to the timing rules defined in this document. 180 Regular RTCP mode: 181 Mode of operation in which no preferred transmission of 182 feedback messages is allowed. Instead, RTCP messages are 183 sent following the rules of [1] and may contain feedback 184 messages information as defined in this document. 186 Regularly Scheduled RTCP packet: 187 An RTCP packet that is not sent as an Early RTCP packet. 189 1.2 Terminology 191 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 192 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 193 document are to be interpreted as described in RFC 2119 [8] 195 2. RTP and RTCP Packet Formats and Protocol Behavior 197 The rules defined in [2] also apply to this profile except for those 198 rules mentioned in the following: 200 RTCP packet types: 201 Three additional RTCP packet types to convey feedback 202 information are defined in section 4. 204 RTCP report intervals: 205 This memo describes three modes of operation which influence 206 the RTCP report intervals (see section 3.2). In regular 207 RTCP mode, all rules from [1] apply. In both Immediate 208 Feedback and Early RTCP modes the minimal interval of 5 209 seconds between 2 RTCP reports is dropped and the rules 210 specified in section 3 apply if RTCP packets containing 211 feedback messages (defined in section 4) are to be 212 transmitted. 214 The rules set forth in [1] may be overridden by session 215 descriptions specifying different parameters (e.g. for the 216 bandwidth share assigned to RTCP for senders and receivers, 217 respectively. For sessions defined using the Session 218 Description Protocol (SDP) [3], the rules of [4] apply. 220 Congestion control: 221 The same basic rules as detailed in [2] apply. Beyond this, 222 in section 5, further consideration is given to the impact of 223 feedback and a sender's reaction to feedback messages. 225 3. Rules for RTCP Feedback 227 3.1 Compound RTCP Feedback Packets 229 Two components constitute RTCP-based feedback as described in this 230 memo: 232 . Status reports are contained in SR/RR messages and are transmitted 233 at regular intervals as part of compound RTCP packets (which also 234 include SDES and possibly other messages); these status reports 235 provide an overall indication for the recent reception quality of 236 a media stream. 238 . Feedback messages as defined in this document that indicate loss 239 or reception of particular pieces of a media stream (or provide 240 some other form of rather immediate feedback on the data 241 received). Rules for the transmission of feedback messages are 242 newly introduced in this memo. 244 RTCP Feedback (FB) messages are just another RTCP packet type (see 245 section 4). Therefore, multiple FB messages MAY be combined in a 246 single compound RTCP packet and they MAY also be sent combined with 247 other RTCP packets. 249 RTCP packets containing Feedback packets as defined in this document 250 MUST contain RTCP packets in the order as defined in [1]: 252 . OPTIONAL encryption prefix that MUST be present if the RTCP 253 message is to be encrypted. 254 . MANDATORY SR or RR. 255 . MANDATORY SDES which MUST contain the CNAME item; all other SDES 256 items are OPTIONAL. 257 . One or more FB messages. 259 The FB MUST be placed in the compound packet after all RTCP packets 260 defined in [1]. The ordering with respect to other RTCP extensions 261 is not defined. 263 Two types of compound RTCP packets carrying feedback packets are used 264 in this document: 266 a) Minimal compound RTCP feedback packet 268 A minimal compound RTCP feedback packet MUST contain only the 269 mandatory information as listed above: encryption prefix if 270 necessary, exactly one RR or SR, exactly one SDES with only the 271 CNAME item present, and the feedback message(s). This is to 272 minimize the size of the RTCP packet transmitted to convey 273 feedback and thus to maximize the frequency at which feedback can 274 be provided while still adhering to the RTCP bandwidth 275 limitations. 277 This packet format SHOULD be used whenever an RTCP feedback 278 message is sent as part of an Early RTCP packet. 280 b) (Full) compound RTCP feedback packet 282 A (full) compound RTCP feedback packet MAY contain any additional 283 number of RTCP packets (additional RRs, further SDES items, 284 etc.). 286 This packet format MUST be used whenever an RTCP feedback message 287 is sent as part of a regularly scheduled RTCP packet or in 288 Regular RTCP mode. This packet format MAY also be used to send 289 RTCP feedback messages in Immediate Feedback or Early RTCP mode. 291 RTCP packets that do not contain FB messages are referred to as non- 292 FB RTCP packets. 294 3.2 Algorithm Outline 296 FB messages are part of the RTCP control streams and are thus subject 297 to the same bandwidth constraints as other RTCP traffic. This means 298 in particular that it may not be possible to report an event observed 299 at a receiver immediately back to the sender. However, the value of 300 feedback given to a sender typically decreases over time -- in terms 301 of the media quality as perceived by the user at the receiving end 302 and/or the cost required to achieve media stream repair. 304 RTP [1] and the commonly used RTP profile [2] specify rules when 305 compound RTCP packets should be sent. This document modifies those 306 rules in order to allow applications to timely report media loss or 307 reception events to accommodate algorithms that use FB messages and 308 are sensitive to the feedback timing. 310 The modified algorithm can be outlined as follows: Normally, when no 311 FB messages have to be conveyed, compound RTCP packets are sent 312 following the rules of RTP [1] -- except that the 5s minimum interval 313 between RTCP reports is not enforced. If a receiver detects the need 314 for an FB message, the receiver waits for a short, random dithering 315 interval (in case of multicast) and then checks whether it has 316 already seen a corresponding FB message from any other receiver 317 (which it can do with all FB messages that are transmitted via 318 multicast; for unicast sessions, there is no such delay). If this is 319 the case then the receiver refrains from sending the FB message and 320 continues to follow the regular RTCP sending schedule. If the 321 receiver has not yet seen a similar FB message from any other 322 receiver, it checks whether it has recently exceeded its RTCP bit 323 rate budget to transmit another FB message (without waiting for its 324 regularly scheduled RTCP transmission time). Only if this is not the 325 case, it sends the FB message as part of a (minimal) compound RTCP 326 packet. 328 FB messages may also be sent as part of full compound RTCP packets 329 which are interspersed as per [1] in regular intervals. 331 3.3 Modes of Operation 333 RTCP-based feedback may operate in one of three modes (figure 1): 335 a) Immediate feedback mode: the group size is below the FB threshold 336 which gives each receiving party sufficient bandwidth to transmit 337 the feedback traffic for the intended purpose. This means, for 338 each receiver there is enough bandwidth to report each event it is 339 supposed/expected to by means of a virtually "immediate" RTCP 340 feedback packet. 342 The group size threshold is a function of a number of parameters 343 including (but not necessarily limited to) the type of feedback 344 used (e.g. ACK vs. NACK), bandwidth, packet rate, packet loss 345 probability and distribution, media type, codec, and -- again 346 depending on the type of FB used -- the (worst case or observed) 347 frequency of events to report (e.g. frame received, packet lost). 349 A special case of this is the ACK mode (where positive 350 acknowledgements are used to confirm reception of data) which is 351 restricted to point-to-point communications. 353 b) Early RTCP mode: In this mode, the group size and other parameters 354 no longer allow each receiver to react to each event that would be 355 worth (or needed) to report. But feedback can still be given 356 sufficiently often so that it allows the sender to adapt the media 357 stream transmission accordingly and thereby increase the overall 358 reproduced media quality. 360 c) From some group size upwards, it is no longer useful to provide 361 feedback from individual receivers at all -- because of the time 362 scale in which the feedback could be provided and/or because in 363 large groups the sender(s) have no chance to react to individual 364 feedback anymore. 366 As the feedback algorithm described in this memo scales smoothly, 367 there is no need for an agreement on the precise values of the 368 respective "thresholds" within the group. Hence the borders between 369 all these modes are allowed to be fluent. 371 ACK 372 feedback 373 V 374 :<- - - - NACK feedback - - - ->// 375 : 376 : Immediate || 377 : Feedback mode ||Early RTCP mode Regular RTCP mode 378 :<=============>||<=============>//<=================> 379 : || 380 -+---------------||---------------//------------------> group size 381 2 || 382 Application-specific FB Threshold 383 = f(data rate, packet loss, codec, ...) 385 Figure 1: Modes of operation 387 The respective thresholds depend on a number of technical parameters 388 (of the codec, the transport, the feedback used, etc.) but also on 389 the respective application scenarios. Section 3.5 provides some 390 useful hints (but no complete precise calculations) on estimating 391 these thresholds. 393 3.4 Definitions 395 The following pieces of state information need to be maintained 396 (largely taken from [1]): 398 a) Let senders be the number of active senders in the RTP session. 400 b) Let members be the current estimate of the number of receivers 401 in the RTP session. 403 c) Let T_rtt be the maximum round trip time as measured by RTCP 404 (if available to the receiver). Note that this may be asymmetric. 406 d) Let tn and tp be the time for the next (last) scheduled 407 RTCP RR transmission calculated prior to reconsideration. 409 e) Let T_rr be the interval after which, having just sent a regularly 410 scheduled RTCP packet, a receiver would schedule the transmission 411 of its next RTCP packet following the rules of [1]: T_rr = tn � 412 tp. Note that the 5s minimum interval between two report as 413 defined in [1] SHOULD NOT be enforced. 415 f) Let t0 be the time at which an event is detected by a receiver. 417 g) Let T_dither_max be the maximum interval for which an RTCP 418 feedback packet may be additionally delayed (to prevent 419 implosions). 421 h) Let T_max_fb_delay be the upper bound within which feedback to 422 an event needs to be reported back to the sender to be useful at 423 all. 425 i) Let te be the time for which a feedback packet is scheduled. 427 j) Let T_fd be the actual (randomized) delay for the transmission of 428 feedback message in response to an event that a certain packet P 429 caused. 431 k) Let allow_early be a variable that indicates whether a receiver 432 may transmit feedback messages prior to its next regularly 433 scheduled RTCP interval tn. 435 l) Let avg_rtcp_size be the moving average on the RTCP packet size as 436 defined in [1]. 438 The feedback situation for an event to report at a receiver is 439 depicted in figure 2 below. At time t0, such an event (e.g. a packet 440 loss) is detected at the receiver. The receiver decides -- based 441 upon current T_rtt, group size, and other (application-specific) 442 parameters -- that a feedback message needs to be sent back to the 443 sender. 445 To avoid an implosion of immediate feedback packets, the receiver 446 MUST delay the transmission of the compound feedback packet by a 447 random amount T_fd (with the random number evenly distributed in the 448 interval [0, T_dither_max]. Transmission of the compound RTCP packet 449 is then scheduled for te = t0 + T_fd. 451 The T_dither_max parameter is chosen based upon the group size, the 452 RTCP bandwidth constraints, and, if available, the round-trip time. 454 Based upon the parameters influencing T_dither_max and a number of 455 other parameters (such as the type of feedback to be provided) the 456 receiver may determine T_max_fb_delay (as static value or dynamically 457 adjusted) as the upper bound for the feedback information to be 458 useful when it reaches the sender. 460 If a compound RTCP feedback packet is scheduled, the time slot for 461 the next scheduled compound RTCP packet is updated accordingly to a 462 new tn. 464 event to 465 report 466 detected 467 | 468 | RTCP feedback range 469 | (T_max_fb_delay) 470 vXXXXXXXXXXXXXXXXXXXXXXXXXXX ) ) 471 |---+--------+-------------+-----+------------| |--------+---------> 472 | | | | ( ( | 473 | t0 te | 474 tp tn 475 \_______ ________/ 476 \/ 477 T_dither_max 479 Figure 2: Event report and parameters for Early RTCP scheduling 480 3.5 Early RTCP Algorithm 482 Assume an active sender S0 (out of S senders) and a number N of 483 receivers with R being one of these receivers. 485 Assume further that R has verified that using feedback mechanisms is 486 reasonable at the current constellation (which is highly application 487 specific and hence not specified in this memo). 489 Then, receiver R MUST use the following rules for transmitting a 490 Feedback messages as minimal or full compound RTCP packet: 492 Initially, R MUST set allow_early := TRUE. 494 R has transmitted the last RTCP RR packet at tp and has scheduled the 495 next transmission (prior to reconsideration) for tn. 497 At time t0, R detects the need to transmit a feedback message (e.g. 498 because a media "unit" needs to be ACKed or NACKed) and finds that 499 sending the feedback message is useful for the sender. 501 R first checks whether there is still an RTCP feedback packet waiting 502 for transmission. If so, the new feedback message MUST be appended 503 to the packet; the schedule for the waiting RTCP feedback packet MUST 504 remain unchanged. 506 If no RTCP feedback message is already awaiting transmission (as part 507 of an Early RTCP packet), a new (minimal) compound RTCP feedback 508 packet MUST be created and the interval T_dither_max MUST be chosen 509 as follows: 511 i) If the session is a unicast session (group size = 2) then 512 T_dither_max := 0. 514 ii) If the receiver has an RTT estimate to the originator of the 515 media unit to provide feedback about, then 517 T_dither_max := k * T_rtt/2 * members 519 with k=1. 521 iii) If the receiver does not have an RTT estimate to the originator, 522 then 524 T_dither_max := l * T_rr 526 with l=0.5. 528 (Application-specific feedback considerations may make it worthwhile 529 to increase T_dither_max beyond this value. This is up to the 530 discretion of the implementer.) 532 Then, R MUST check whether its next regularly scheduled RTCP packet 533 is within the time bounds for the RTCP FB (t0 + T_dither_max > tn). 534 If so, an Early RTCP packet MUST NOT be scheduled; instead the FB 535 message MUST be stored to be appended to the regular RTCP packet 536 scheduled for tn. 538 Otherwise, R MUST check whether it is allowed to transmit an Early 539 RTCP packet (allow_early == TRUE). 541 If so, R MUST schedule an Early RTCP packet for te := t0 + RND * 542 T_dither_max with the RND function evenly distributed between 0 543 and 1. 545 If, while waiting for te, R receives an RTCP feedback packet R 546 MUST act as follows: 548 1. If R understands the received feedback message's semantics and 549 the message contents is a superset of the feedback R wanted to 550 send then R MUST discard its own feedback message and MUST re- 551 schedule the next regular RTCP message transmission for tn (as 552 calculated before). 554 2. If R understands the received feedback message's semantics and 555 the message contents is not a superset of the feedback R 556 wanted to send then R SHOULD transmit its own feedback message 557 as scheduled. 559 3. If R does not understand the received feedback message's 560 semantics then R MAY send its own feedback message as or Early 561 RTCP packet. Alternatively, R MAY re-schedule the next 562 regular RTCP message transmission for tn (as calculated 563 before) and MAY append the feedback message the now regularly 564 scheduled RTCP message. 566 Refer to section 4 on the comparison of feedback messages and for 567 which feedback messages must be understood by a receiver. 569 Otherwise, when te is reached, R MUST transmit the RTCP packet 570 containing the FB message. R then MUST set allow_early := FALSE 571 and MUST recalculate tn := tp + 2*T_rr. As soon as R sends its 572 next regularly scheduled RTCP RR (at the new tn), it MUST set 573 allow_early := TRUE again. 575 If allow_early == FALSE then R MUST check the time for the next 576 scheduled RR: 578 1. If tn � t0 < T_max_fb_delay (i.e. if, despite late reception, the 579 feedback could still be useful for the sender) then R MAY create 580 an RTCP FB message for transmission along with the RTCP packet at 581 tn. 583 2. Otherwise, R MUST discard the RTCP feedback message. 585 In regular RTCP intervals as specified by [1] (except for the five 586 second minimum), a full compound RTCP packet is sent (which may also 587 contain a feedback message if one has been created according to the 588 above rules and scheduled for transmission along the full compound 589 RTCP message). 591 The E bit in the message header is used upon reception to detect 592 whether this RTCP feedback message was sent as Early RTCP or not. 593 Hence, a feedback message that is sent as an Immediate or Early RTCP 594 packet MUST set the E bit in the message header to "1". Feedback 595 messages piggy-backed on regularly scheduled RTCP packets MUST set 596 the E bit to "0". If a receiver R receives an Early RTCP packet 597 (E=1), then it MAY set allow_early := TRUE. 599 Whenever an RTCP packet is sent or received -- minimal or full 600 compound, early or regularly scheduled -- the avg_rtcp_size variable 601 is updated accordingly (see [1]) and the tn is calculated using the 602 new avg_rtcp_size. 604 3.6 Considerations on the Group Size 606 This section provides guidelines to the group sizes at which the 607 various feedback modes may be used. 609 3.6.1 ACK mode 611 The group size MUST be exactly two participants, i.e. point-to-point 612 communications. Unicast addresses SHOULD be used in the session 613 description. 615 For unidirectional as well as bi-directional communication between 616 two parties, 2.5% of the RTP session bandwidth are available for RTCP 617 traffic from the receivers including feedback. , Assuming that out 618 of ten RTCP packets, nine are sent as minimal compound RTCP packets 619 and one as full compound RTCP packet, at 64kbit/s unidirectional 620 communication scenario, a receiver can report 1.5 events per second 621 back to the sender, at 256kbit/s 6 events and so forth. 623 From 1 Mbit/s upwards, a receiver would be able to acknowledge each 624 individual frame (not packet!) in a 25 fps video stream. 626 ACK strategies should be defined accordingly to work properly with 627 these bandwidth limitations. 629 3.6.2 NACK mode 631 Negative acknowledgements (or similar types of feedback) MUST be 632 used for all groups larger than two. Of course, NACKs MAY be used 633 for point-to-point communications as well. 635 Whether or not the use of Immediate or Early RTCP packets should be 636 considered depends upon a number of parameters including session 637 bandwidth, codec, special type of feedback, number of senders and 638 receivers, among many others. 640 The crucial parameters -- to which virtually all of the above can be 641 reduced -- is the allowed minimal interval between two RTCP reports 642 and the (average) number of events that presumably need reporting per 643 time interval (plus their distribution over time, of course). The 644 minimum interval is derived from the available RTCP bandwidth and the 645 expected average size of an RTCP packet. The number events to report 646 e.g. per second may be derived from the packet loss rate and sender's 647 rate of transmitting packets. From these two values, the allowable 648 group size for the Immediate feedback mode can be calculated. 650 The upper bound for the Early RTCP mode then solely depends on the 651 acceptable quality degradation, i.e. how many events per time 652 interval may go unreported. 654 Example: If a 256kbit/s video with 30 fps is transmitted through a 655 network with an MTU size of some 1500 bytes, then, in most cases, 656 each frame would fit in its own packet leading to a packet rate of 30 657 packets per second. If 5% packet loss occurs in the network (equally 658 distributed, no inter-dependence between receivers), then each 659 receiver will have to report 3 packets lost each two seconds. 660 Assuming a single sender and more then three receivers, this yields 661 3.75% of the RTCP bandwidth allocated to the receivers and thus 662 9.6kbit/s. Assuming further a size of 120 bytes for the average 663 compound RTCP packet allows 10 RTCP packets to be sent per second or 664 20 in two seconds. If every receiver needs to report three packets, 665 this yields a maximum group size of 6-7 receivers if all loss events 666 shall be reported. The rules for transmission of immediate RTCP 667 packets should provide sufficient flexibility for most of this 668 reporting to occur in a timely fashion. 670 Extending this example to determine the upper bound for Early RTCP 671 mode leads to the following considerations: assume that the 672 underlying coding scheme and the application (as well as the tolerant 673 users) allow in the order of one loss without repair per two seconds. 674 Thus the number of packets to be reported by each receiver decreases 675 to two per two seconds second and increases the group size to 10. 676 Assuming further that some number of packet losses are correlated, 677 feedback traffic is further reduced and group sizes of some 12 to 16 678 (maybe even 20) can be reasonably well supported using Early RTCP 679 mode. 681 3.7 Summary of decision steps 683 3.7.1 General Hints 685 Before even considering whether or not to send RTCP feedback 686 information an application has to determine whether this mechanism is 687 applicable: 689 1) An application has to decide whether -- for the current ratio of 690 packet rate with the associated (application-specific) maximum 691 feedback delay and the currently observed round-trip time (if 692 available) -- feedback mechanisms can be applied at all. 694 This decision may obviously be based upon (and dynamically revised 695 following) regular RTCP reception statistics. 697 2) The application has to decide whether -- for a certain observed 698 error rate, assigned bandwidth, frame rate, and group size -- (and 699 which) feedback mechanisms can be applied. 701 Regular RTCP provides valuable input to this step, too. 703 3) If these tests pass, the application has to follow the rules for 704 transmitting Early RTCP packets or regularly scheduled RTCP 705 packets with piggybacked feedback. 707 3.7.2 Session Description Attributes 709 A number of additional SDP parameters MAY be used to describe a 710 session. These are defined as media level attributes. 712 3.7.2.1 Profile identification 714 The AV profile defined in [4] is referred to as "AVP" in the context 715 of e.g. the Session Description Protocol (SDP) [3]. The profile 716 specified in this document is referred to as "AVPF". 718 Feedback information following the modified timing rules as specified 719 in this document MUST NOT be sent for a particular media session 720 unless the profile for this session indicates the use of the "AVPF" 721 profile. 723 Feedback information as part of regularly scheduled compound RTCP 724 packets following the timing rules of [1] and [2] MAY be sent for 725 media sessions for which the "AVP" profile is specified. In this 726 case, however, the receiver providing feedback MUST NOT rely on the 727 sender reacting to the feedback at all. 729 3.7.2.2 RTCP Feedback Capability Attribute 731 A new payload format-specific SDP attribute (for use with "a=fmtp:") 732 is defined to indicate the capability of using RTCP feedback as 733 specified in this document: "rtcp-fb". The "rtcp-fb" attribute MAY 734 only be used as an SDP media attribute and MUST NOT be provided at 735 the session level. The rtcp-fb attribute MUST only be used in media 736 sessions for which the "AVPF" is specified. 738 The rtcp-fb attribute is used to indicate which RTCP feedback 739 messages MAY be used in this media session for the indicated payload 740 type. If several types of feedback are supported, several a=rtcp-fb: 741 lines MUST be used. 743 If no rtcp-fb attribute is specified the RTP receivers SHOULD assume 744 that the RTP senders only support generic NACKs. In addition, the 745 RTP receivers MAY send feedback using other suitable RTCP feedback 746 packets as defined for the respective media type. The RTP receivers 747 MUST NOT rely on the RTP senders reacting to any of the feedback 748 messages. 750 If one or more rtcp-fb attributes are present in a media session 751 description, the RTP receivers for the media session(s) containing 752 the "rtcp-fb" 753 . MUST ignore all rtcp-fb attributes of which they do not fully 754 understand the semantics (i.e. understand the meaning of all 755 values in the a=fmtp:rtcp-fb line); 757 . SHOULD provide feedback information as specified in this document 758 using any of the RTCP feedback packets as specified in one of the 759 rtcp-fb attributes for this media session; and 761 . MUST NOT use other feedback messages than those listed in one of 762 the rtcp-fb attribute lines. 764 RTP senders MUST be prepared to receive any kind of RTCP feedback 765 messages and MUST silently discard all those RTCP feedback messages 766 that they do not understand. 768 The syntax of the rtcp-fb attribute is as follows (the feedback types 769 and optional parameters are all case sensitive): 771 rtcp-fb-syntax = "a=fmtp:" WS "rtcp-fb" WS rtcp-fb-value 773 rtcp-fb-value = "ack" rtcp-fb-param 774 | "nack" rtcp-fb-nack-param 775 | rtcp-fb-id rtcp-fb-param 777 rtcp-fb-id = 1*(alpha-numeric | "-" | "_") 779 rtcp-fb-param = "app" 780 | byte-string 781 | ; empty 783 rtcp-fb-nack-param = "pli" 784 | "sli" 785 | "rpsi" 786 | "app" 787 | byte-string 788 | ; empty 790 The literals of the above grammar have the following semantics: 792 Feedback type "ack": 794 This feedback type indicates that positive acknowledgements for 795 feedback are supported. 797 The feedback type "ack" MUST only be used if the media session 798 is allowed to operate in ACK mode as defined in 3.6.1.2. 800 Parameters may be provided to further distinguish different 801 types of positive acknowledgement feedback. If no parameters 802 are present, the Generic ACK as specified in section 4.1.2 is 803 implied. 805 If the parameter "app" is specified, this indicates the use of 806 application layer feedback. In this case, additional parameters 807 following "app" MAY be used to further differentiate various 808 types of application layer feedback. This document does not 809 define any parameters specific to "app". 811 Further parameters for "ack" MAY be defined in other documents. 813 Feedback type "nack": 815 This feedback type indicates that negative acknowledgements for 816 feedback are supported. 818 The feedback type "nack", without parameters, indicates use of 819 the General NACK feedback format as defined in section 4.2.1. 821 The following three parameters are defined in this document for 822 use with "nack" in conjunction with the media type "video": 824 . "pli" indicates the use of Picture Loss Indication feedback 825 as defined in section 4.3.1. 826 . "sli" indicates the use of Slice Loss Indication feedback as 827 defined in section 4.3.2. 828 . "rpsi" indicates the use of Reference Picture Selection 829 Indication feedback as defined in section 4.3.3. 830 . "app" indicates the use of application layer feedback. 831 Additional parameters after "app" MAY be provided to 832 differentiate different types of application layer feedback. 833 No parameters specific to "app" are defined in this document. 835 Further parameters for "nack" MAY be defined in other documents. 837 Other feedback types : 839 Other documents MAY define additional types of feedback; to keep 840 the grammar extensible for those cases, the rtcp-fb-id is 841 introduced as a placeholder. A new feedback scheme name needs 842 to be unique (and thus has to be registered with IANA). Along 843 with a new name, its semantics, packet formats (if necessary), 844 and rules for its operation need to be specified. 846 Note that it is assumed that more specific information about 847 application layer feedback (as defined in section 4.2.3) will be 848 conveyed as feedback types and parameters defined elsewhere. Hence, 849 no further provision for any types and parameters is made in this 850 document. 852 Further types of feedback as well as further parameters may be 853 defined in other documents. 855 It is up to the recipients whether or not they send feedback 856 information and up to the sender(s) to make use of feedback provided. 858 3.7.2.3 Unicasting 860 If an m= line in the SDP describing a session indicates unicast 861 addresses for a particular media type (and does not operate in multi- 862 unicast mode with all recipients listed explicitly but still 863 addressed via unicast), the RTCP feedback MAY operate in ACK feedback 864 mode. 866 3.7.2.4 RTCP Bandwidth Modifiers 868 The standard RTCP bandwidth assignments as defined in [1] and [2] may 869 be overridden by bandwidth modifiers as specified in [4]: b=RS: 870 and b=RR: MAY be used to assign a different bandwidth (measured 871 in bits per second) to RTP senders and receivers, respectively. The 872 precedence rules of [4] apply to determine the actual bandwidth to be 873 used by senders and receivers. 875 Applications operating knowingly over highly asymmetric links (such 876 as satellite links) SHOULD use this mechanism to reduce the feedback 877 rate for high bandwidth streams to prevent deterministic congestion 878 of the feedback path(s). 880 3.7.2.5 Examples 882 Example 1: The following session description indicates a session made 883 up from an audio and a DTMF for point-to-point communication in which 884 the DTMF stream uses Generic ACKs. This session description could be 885 contained in a SIP INVITE, 200 OK, or ACK message to indicate that 886 its sender is capable of and willing to receive feedback for the DTMF 887 stream it transmits. 889 v=0 890 o=alice 3203093520 3203093520 IN IP4 host.example.com 891 s=Media with feedback 892 t=0 0 893 c=IN IP4 host.example.com 894 m=audio 49170 RTP/AVPF 0 96 895 a=rtpmap:0 PCMU/8000 896 a=rtpmap:96 telephone-event/8000 897 a=fmtp:96 0-16 898 a=fmtp:96 rtcp-fb ack 900 Example 2: The following session description indicates a multicast 901 video-only session (using H.263+) with the video source accepting 902 Generic NACKs and Reference Picture Selection. Such a description 903 may have been conveyed using the Session Announcement Protocol (SAP). 905 v=0 906 o=alice 3203093520 3203093520 IN IP4 host.example.com 907 s=Multicast video with feedback 908 t=3203130148 3203137348 909 m=audio 49170 RTP/AVP 0 910 c=IN IP4 224.2.1.183 911 a=rtpmap:0 PCMU/8000 912 m=video 51372 RTP/AVP 98 913 c=IN IP4 224.2.1.184 914 a=rtpmap:98 H263-1998/90000 915 a=fmtp:98 rtcp-fb nack 916 a=fmtp:98 rtcp-fb nack rpsi 917 4. Format of RTCP Feedback messages 919 This section defines the format of the low delay RTCP feedback 920 messages. These messages classified into three categories as 921 follows: 923 - Transport layer feedback messages 924 - Payload-specific feedback messages 925 - Application layer feedback messages 927 Transport layer feedback messages are intended to transmit general 928 purpose feedback information, i.e. information independent of the 929 particular codec or the application in use. The information is 930 expected to be generated and processed at the transport/RTP layer. 931 Currently, only a general positive acknowledgement (ACK) and negative 932 acknowledgement (NACK) message are defined. 934 Payload-specific feedback messages transport information that is 935 specific to a certain payload and will be generated and acted upon at 936 the codec "layer". This document defines a common header to be used 937 in conjunction with all payload-specific feedback messages. The 938 definition of specific messages is left to either RTP Payload Format 939 specifications or to additional feedback format documents. 941 Application layer feedback messages provide a means to transparently 942 convey feedback from the receiver's to the sender's application. The 943 information contained in such a message is not expected to be acted 944 upon at the transport/RTP or the codec layer. The data to be 945 exchanged between two application instances is usually defined in the 946 application protocol's specification and thus can be identified by 947 the application so that there is no need for additional external 948 information. Hence, this document defines only a common header to be 949 used along with all application layer feedback messages. From a 950 protocol point of view, an application layer feedback message is 951 treated as a special case of a payload-specific feedback message. 953 This document defines two transport layer feedback and three (video) 954 payload-specific feedback messages as well as a container for 955 application layer feedback messages. Additional transport layer and 956 payload specific feedback messages may be defined in other documents 957 and are registered through IANA (see section IANA considerations). 959 The general syntax and semantics for the above RTCP feedback message 960 types is described in the following subsections. 962 4.1 Common Packet Format for Feedback Message 964 All feedback message share a common packet format that is depicted in 965 figure 3: 967 0 1 2 3 968 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 969 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 970 |V=2|P|E| FMT | PT | length | 971 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 972 | SSRC of packet sender | 973 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 974 | SSRC of media source | 975 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 976 : Feedback Control Information (FCI) : 977 : : 979 Figure 3: Common Packet Format for Feedback Messages 981 The various fields V, P, SSRC and length are defined in the RTP 982 specification [2], the respective meaning being summarized below: 984 version (V): 2 bits 985 This field identifies the RTP version. The current version is 2. 987 padding (P): 1 bit 988 If set, the padding bit indicates that the packet contains 989 additional padding octets at the end which are not part of the 990 control information but are included in the length field. 992 Early RTCP (E): 1 bit 993 This bit MUST be set if the packet is sent as an Immediate 994 Feedback or as an Early RTCP packet. 996 Feedback message type (FMT): 4 bits 997 This field identifies the type of the feedback message and is 998 interpreted relative to the RTCP message type (transport, 999 payload-specific, or application feedback). The values for each 1000 of the three feedback types are defined in the respective 1001 sections below. 1003 Payload type (PT): 8 bits 1004 This is the RTCP packet type which identifies the packet as being 1005 an RTCP Feedback Message. Two values are defined (TBA. By IANA): 1007 Name | Value | Brief Description 1008 ----------+-------+-------------------------------------- 1009 RTPFB | 2xx | Transport layer feedback message 1010 PSFB | 2xy | Payload-specific feedback message 1012 Length: 16 bits 1013 The length of this packet in 32-bit words minus one, including 1014 the header and any padding. This is in line with the definition 1015 of the length field used in RTCP sender and receiver reports [3]. 1017 SSRC of packet sender: 32 bits 1018 The synchronization source identifier for the originator of this 1019 packet. 1021 SSRC of media source: 32 bits 1022 The synchronization source identifier of the media source that 1023 this piece of feedback information is related to. 1025 Feedback Control Information (FCI): variable length 1026 The following three sections define which additional information 1027 is included in the feedback message for each type of feedback. 1029 4.2 Transport Layer Feedback Messages 1031 Transport Layer Feedback messages are identified by the value RTPFB 1032 as RTCP message type. 1034 Two general purpose transport layer feedback messages are defined so 1035 far: General ACK and General NACK. They are identified by means of 1036 the FMT parameter as follows: 1038 0: forbidden 1039 1: General NACK 1040 2: General ACK 1041 3-15: reserved 1043 The following two subsections define the packet formats for these 1044 messages. 1046 4.2.1 Generic NACK 1048 The Generic NACK message is identified by PT=RTPFB and FMT=1. 1050 The Generic NACK packet is used to indicate the loss of one or more 1051 RTP packets. The lost packet(s) are identified by the means of a 1052 packet identifier and a bit mask. 1054 The Feedback control information (FCI) field has the following 1055 Syntax (figure 4): 1057 0 1 2 3 1058 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1059 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1060 | PID | BLP | 1061 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1063 Figure 4: Syntax for the Generic NACK message 1065 Packet ID (PID): 16 bits 1066 The PID field is used to specify a lost packet. Typically, the 1067 RTP sequence number is used for PID as the default format, but 1068 RTP Payload Formats may decide to identify a packet differently. 1070 bitmask of following lost packets (BLP): 16 bits 1071 The BLP allows for reporting losses of any of the 16 RTP packets 1072 immediately following the RTP packet indicated by the PID. The 1073 BLP's definition is identical to that given in [10]. Denoting 1074 the BLP's least significant bit as bit 1, and its most 1075 significant bit as bit 16, then bit i of the bit mask is set to 1 1076 if the sender has not received RTP packet number PID+i (modulo 1077 2^16) and the receiver decides this packet is lost; bit i is set 1078 to 0 otherwise. Note that the sender MUST NOT assume that a 1079 receiver has received a packet because its bit mask was set to 0. 1080 For example, the least significant bit of the BLP would be set to 1081 1 if the packet corresponding to the PID and the following packet 1082 have been lost. However, the sender cannot infer that packets 1083 PID+2 through PID+16 have been received simply because bits 2 1084 through 15 of the BLP are 0; all the sender knows is that the 1085 receiver has not reported them as lost at this time. 1087 4.2.2 Generic ACK 1089 The Generic ACK message is identified by PT=RTPFB and FMT=2. 1091 The Generic ACK packet is used to indicate that one or several RTP 1092 packets were received correctly. The received packet(s) are 1093 identified by the means of a packet identifier and a bit mask. 1094 ACKing of a range of consecutive packets is also possible. 1096 The Feedback control information (FCI) field has the following 1097 syntax: 1099 0 1 2 3 1100 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1101 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1102 | PID |R| BLP/#packets | 1103 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1105 Figure 5: Syntax for the Generic ACK message 1107 Packet ID (1st PID): 16 bits 1108 This PID field is used to specify a correctly received packet. 1109 Typically, the RTP sequence number is used for PID as the default 1110 format, but RTP Payload Formats may decide to identify a packet 1111 differently. 1113 Range of ACKs (R): 1 bit 1114 The R-bit indicates that a range of consecutive packets are 1115 received correctly. If R=1 then the PID field specifies the 1116 first packet of that range and the next field (BLP/#packets) will 1117 carry the number of packets being acknowledged. If R=0 then PID 1118 specifies the first packet to be acknowledged and BLP/#packets 1119 provides a bit mask to selectively indicate individual packets 1120 that are acknowledged. 1122 Bit mask of lost packets (BLP)/#packets (PID): 15 bits 1123 The semantics of this field depends on the value of the R-bit. 1125 If R=1, this field is used to identify the number of additional 1126 packets of to be acknowledged: 1127 #packets = - 1128 That is, #packets MUST indicate the number of packet to be ACKed 1129 minus one. In particular, if only a single packet is to be ACKed 1130 and R=1 then #packets MUST be set to 0x0000. 1132 Example: If all packets between and including PIDx=380 and PIDy = 1133 422 have been received, the Generic ACK would contain PID = PIDx 1134 = 380 and #packets = PIDy � PID = 42. In case the PID wraps 1135 around, modulo arithmetic is used to calculate the number of 1136 packets. 1138 If R=0, this field carries a bit mask. The BLP allows for 1139 reporting reception of any of the 15 RTP packets immediately 1140 following the RTP packet indicated by the PID. The BLP's 1141 definition is identical to that given in [10] except that, here, 1142 BLP is only 15 bits wide. Denoting the BLP's least significant 1143 bit as bit 1, and its most significant bit as bit 15, then bit i 1144 of the bitmask is set to 1 if the sender has received RTP packet 1145 number PID+i (modulo 2^16) and the receiver decides to ACK this 1146 packet; bit i is set to 0 otherwise. If only the packet 1147 indicated by PID is to be ACKed and R=0 then BLP MUST be set to 1148 0x0000. 1150 4.3 Payload Specific Feedback Messages 1152 Payload-Specific Feedback Messages are identified by the value PSFB 1153 as RTCP message type. 1155 Three payload-specific feedback messages are defined so far. They 1156 are identified by means of the FMT parameter as follows: 1158 0: forbidden 1159 1: Picture Loss Indication (PLI) 1160 2: Slice Lost Indication (SLI) 1161 3: Reference Picture Selection Indication (RPSI) 1162 4-14: reserved 1163 15: Application layer feedback message 1165 The following subsections define the packet formats for these 1166 messages. 1168 4.3.1 Picture Loss Indication (PLI) 1170 The PLI feedback message is identified by PT=PSFB and FMT=1. 1172 4.3.1.1 Semantics 1174 With the Picture Loss Indication message a decoder informs the 1175 encoder about the loss of one or more full pictures. 1177 4.3.1.2 Message Format 1179 PLI does not require parameters. Therefore, the length field MUST be 1180 2, and there MUST NOT be any Feedback Control Information. 1182 4.3.1.3 Timing Rules 1184 The timing follows the rules outlined in section 3. In systems that 1185 employ both PLI and other types of feedback it may be advisable to 1186 follow the regular RTCP RR timing rules for PLI, since PLI is not as 1187 delay critical as other FB types. 1189 4.3.1.4 Remarks 1191 PLI messages typically trigger the sending of full Intra pictures. 1192 Intra Pictures are several times larger then predicted (Inter) 1193 pictures. Their size is independent of the time they are generated. 1194 In most environments, especially when employing bandwidth-limited 1195 links, the use of an Intra picture implies an allowed delay that is a 1196 significant multitude of the typical frame duration. An example: If 1197 the sending frame rate is 10 fps, and an Intra picture is assumed to 1198 be 10 times as big as an Inter picture (not an unrealistic 1199 assumption, see [14] for details), then a full second of latency has 1200 to be accepted. In such an environment there is no need for a 1201 particular short delay in sending the feedback message. Hence 1202 waiting for the next possible time slot allowed by RTCP timing rules 1203 as per [2] does not have a negative impact on the system performance. 1205 4.3.2 Slice Lost Indication (SLI) 1207 The SLI feedback message is identified by PT=PSFB and FMT=2. 1209 4.3.2.1 Semantics 1211 With the Slice Lost Indication a decoder can inform an encoder that 1212 it was unable to decode one, or several consecutive, macroblocks. 1213 The encoder can take appropriate action in order to re-synchronize 1214 encoder and decoder by means of its choice, typically by sending the 1215 lost macroblocks in Intra mode. This feedback message SHALL NOT be 1216 used for video codecs with non-uniform, dynamically changeable 1217 macroblock sizes such as H.263 with enabled Annex Q. In such a case, 1218 an encoder cannot always identify the corrupted spatial region. 1220 4.3.2.2 Format 1222 When FBT indicates a Slice Lost Indication, then there is one 1223 additional PCI field the content of which is depicted in figure 6. 1224 The length of the feedback message MUST be set to 3. 1226 0 1 2 3 1227 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1228 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1229 | First | Number | TR | 1230 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1232 Figure 6: Syntax of the Slice Lost Indication (SLI) 1234 First: 13 bits 1235 The macroblock (MB) address of the first lost macroblock. The MB 1236 numbering is done such that the macroblock in the upper left 1237 corner of the picture is considered macroblock number 1 and the 1238 number for each macroblock increases from left to right and then 1239 from top to bottom in raster-scan order (such that if there is a 1240 total of N macroblocks in a picture, the bottom right macroblock 1241 is considered macroblock number N). 1243 Number: 13 bits 1244 The number of lost macroblocks, in scan order as discussed above. 1246 TR: 6 bits 1247 The six least significant bits of the Temporal Reference of the 1248 picture. 1250 4.3.2.3 Timing Rules 1252 The efficiency of algorithms using the Slice Lost Indication is 1253 reduced greatly when the Indication is not transmitted in a timely 1254 fashion. Motion compensation propagates corrupted pixels that are 1255 not reported as being corrupted. Therefore, the use of the algorithm 1256 discussed in section 3 is highly recommended. 1258 4.3.2.4 Remarks 1260 The First field of the UCI defines the first macroblock of a picture 1261 as 1 and not, as one could suspect, as 0. This was done to align 1262 this specification with the comparable mechanism available in H.245. 1263 The maximum number of macroblocks in a picture (2**13 or 8192) 1264 corresponds to the maximum picture sizes of the ITU-T and ISO/IEC 1265 video codecs. If future video codecs offer larger picture sizes 1266 and/or smaller macroblock sizes, then an additional feedback message 1267 has to be defined. The six least significant bits of the Temporal 1268 Reference field are deemed to be sufficient to indicate the picture 1269 in which the loss occurred. 1271 Algorithms were reported that keep track of the regions effected by 1272 motion compensation, in order to allow for a transmission of Intra 1273 macroblocks to all those areas, regardless of the timing of the FB 1274 (see H.263 (2000) Appendix I [13]] and [15]. While, when those 1275 algorithms are used, the timing of the FB is less critical then 1276 without, it has to be observed that those algorithms correct large 1277 parts of the picture and, therefore, have to transmit many for bits 1278 in case of delayed FBs. 1280 4.3.3 Reference Picture Selection Indication (RPSI) 1282 The RPSI feedback message is identified by PT=PSFB and FMT=3. 1284 4.3.3.1 Semantics 1286 Modern video coding standards such as MPEG-4 visual version 2 [12] or 1287 H.263 version 2 [13] allow the use of older reference pictures then 1288 the most recent one. Typically, a first-in-first-out queue of 1289 reference pictures is maintained. If an encoder has learned about a 1290 loss of encoder-decoder synchronicity, a known-as-correct reference 1291 picture can be used. As this reference picture is temporally further 1292 away then usual, the resulting predictively coded picture will use 1293 more bits. 1295 Both MPEG-4 and H.263 define a binary format for the �payload� of an 1296 RPSI message that includes information such as the temporal ID of the 1297 damaged picture and the size of the damaged region. This bit string 1298 is typically small �- a couple of dozen bits -�, of variable length, 1299 and self-contained, i.e. contains all information that is necessary 1300 to perform reference picture selection. 1302 Note that both MPEG-4 and H.263 allow the use of RPSI with positive 1303 feedback information as well. That is, all corrected pictures are 1304 reported. Any form of positive feedback MUST NOT be used when in a 1305 multicast environment (reporting positive feedback about individual 1306 reference pictures at RTCP intervals is not expected to be of much 1307 use anyway). For point-to-point communication, positive feedback MAY 1308 be used but, again, the bit rate budget of RTCP feedback will prevent 1309 the use in most scenarios anyway. 1311 4.3.3.2 Format 1313 When FB indicates an RPSI, then the length field is set to the number 1314 of bits of the following bit string that contains the RPS 1315 information. This bit string follows byte aligned in the UCI field. 1316 Bit padding is used to achieve 32-bit word alignment of the UCI 1317 message (and the whole packet). 1319 4.3.3.3 Timing Rules 1321 RPS is even more critical to delay then algorithms using SLI. This 1322 is due to the fact that the older the RPS message is, the more bits 1323 the encoder has to spend to achieve encoder-decoder synchronicity. 1324 See [14] and [15] for some information about the overhead of RPS for 1325 certain bit rate/frame rate/loss rate scenarios. 1327 Therefore, RPS messages should typically be sent as soon as possible, 1328 employing the algorithm of section 3. 1330 4.4 Application Layer Feedback Messages 1332 Payload-Specific Feedback Messages are a special case of payload- 1333 specific messages and identified by PT=PSFB and FMT=15. 1335 These messages are used to transport application defined data 1336 directly from the receiver's to the sender's application. The data 1337 that is transported is not identified by the feedback message. 1338 Therefore the application must be able to identify the messages 1339 payload. 1341 Usually applications define their own set of messages, e.g. NEWPRED 1342 messages in MPEG-4 or feedback messages in H.263/Annex N,U. These 1343 messages do not need any additional information from the RTCP 1344 message. Thus the application message is simply placed into the FCI 1345 field as follows and the length field is set accordingly. 1347 Application Message (FCI): variable length 1348 This field contains the original application message that should 1349 be transported from the receiver to the source. The format is 1350 application dependent. The length of this field is variable. If 1351 the application data is not four-byte aligned, padding must be 1352 added. 1354 As there is no need for additional identification at the RTCP level, 1355 the FMT field is unused and MUST be set to zero: 1357 5. Early Feedback and Congestion Control 1359 In the previous sections, the feedback messages were defined as well 1360 as the timing rules according to which to send these messages. The 1361 way to react to the feedback received depends on the application 1362 using the feedback mechanisms and hence is beyond the scope of this 1363 document. 1365 However, across all applications, there is a common requirement for 1366 (TCP-friendly) congestion control on the media stream as defined in 1367 [1] and [2] when operating in a best-effort network environment. 1369 Low delay feedback supports the use of congestion control algorithms 1370 in two ways: 1372 . The potentially more frequent RTCP messagesallow the sender to 1373 monitor the network state more closely than with regular RTCP 1374 and therefore enable reacting to upcoming congestion in a more 1375 timely fashion. 1377 . The feedback messages themselves may convey additional 1378 information as input to congestion control algorithms and thus 1379 improve reaction over conventional RTCP. (For example, ACK-based 1380 feedback may even allow to construct closed loop algorithms and 1381 NACK-based systems may provide further information on the packet 1382 loss distribution.) 1384 A congestion control algorithm that shares the available bandwidth 1385 fair with competing TCP connections, e.g. TFRC [16], SHOULD be used 1386 to determine the data rate for the media stream (if the low delay RTP 1387 session is transmitted in a best effort environment). 1389 RTCP feedback messages or RTCP SR/RR packets that indicate recent 1390 packet loss MUST NOT lead to a (mid-term) increase in the 1391 transmission data rate and SHOULD lead to a (short-term) decrease of 1392 the transmission data rate. Such messages SHOULD cause the sender to 1393 adjust the transmission data rate to the order of the throughput TCP 1394 would achieve under similar conditions (e.g. using TFRC). 1396 RTCP feedback messages or RTCP SR/RR packets that indicate no recent 1397 packet loss MAY cause the sender to increase the transmission data 1398 rate to roughly the throughput TCP would achieve under similar 1399 conditions (e.g. using TFRC). 1401 6. Security Considerations 1403 RTP packets transporting information with the proposed payload for 1404 mat are subject to the security considerations discussed in the RTP 1405 specification [1]. This implies that confidentiality of the media 1406 streams is achieved by encryption. 1408 If the entire stream (extension data and AU data) is to be secured 1409 and all the participants are expected to have the keys to decode the 1410 entire stream, then the encryption is performed in the usual manner, 1411 and there is no conflict between the two operations (encapsulation 1412 and encryption). 1414 The need for a portion of stream (e.g. extension data) to be 1415 encrypted with a different key, or not to be encrypted, would require 1416 application level signaling protocols to be aware of the usage of 1417 the XT field, and to exchange keys and negotiate their usage on the 1418 media and extension data separately. 1420 7. IANA Considerations 1422 The feedback profile as an extension to the profile for audio-visual 1423 conferences with minimal control needs to be registered: "AVPF". 1425 For the Session Description Protocol, the following "fmtp:" attribute 1426 needs to be registered: "rtcp-fb". 1428 Along with "rtcp-fb", the feedback types "ack" and "nack" need to be 1429 registered. 1431 Along with "nack", the feedback type parameters "sli", "pli", and 1432 "rpsi" need to be registered. 1434 Two RTCP Control Packet Types: for the class of transport layer 1435 feedback messages ("RTPFB") and for the class of payload-specific 1436 feedback messages ("PSFB"). 1438 Within the RTPFB range, three format (FMT) values need to be 1439 registered: 1441 0: forbidden 1442 1: General NACK 1443 2: General ACK 1445 Within the PSFB range, five format (FMT) values need to be 1446 registered: 1448 0: forbidden 1449 1: Picture Loss Indication (PLI) 1450 2: Slice Loss Indication (SLI) 1451 3: Reference Picture Selection Indication (SLI) 1452 15: Application layer feedback (AFB) 1454 8. Acknowledgements 1456 This document is a product of the Audio-Visual Transport (AVT) 1457 Working Group of the IETF. The authors would like to thank Steve 1458 Casner and Colin Perkins for their comments and suggestions as well 1459 as for their responsiveness to numerous questions. 1461 9. Full Copyright Statement 1463 Copyright (C) The Internet Society (2001). All Rights Reserved. 1465 This document and translations of it may be copied and furnished to 1466 others, and derivative works that comment on or otherwise explain it 1467 or assist in its implementation may be prepared, copied, published 1468 and distributed, in whole or in part, without restriction of any 1469 kind, provided that the above copyright notice and this paragraph are 1470 included on all such copies and derivative works. 1472 However, this document itself may not be modified in any way, such as 1473 by removing the copyright notice or references to the Internet Soci- 1474 ety or other Internet organizations, except as needed for the purpose 1475 of developing Internet standards in which case the procedures for 1476 copyrights defined in the Internet Standards process must be fol- 1477 lowed, or as required to translate it into languages other than 1478 English. 1480 The limited permissions granted above are perpetual and will not be 1481 revoked by the Internet Society or its successors or assigns. 1483 This document and the information contained herein is provided on an 1484 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 1485 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 1486 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 1487 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MER- 1488 CHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." 1489 10. Authors' Addresses 1491 J�rg Ott {sip,mailto}:jo@tzi.uni-bremen.de 1492 Universit�t Bremen TZI 1493 MZH 5180 1494 Bibliothekstr. 1 1495 D-28359 Bremen 1496 Germany 1498 Stephan Wenger stewe@cs.tu-berlin.de 1499 TU Berlin 1500 Sekr. FR 6-3 1501 Franklinstr. 28-29 1502 D-10587 Berlin 1503 Germany 1505 Shigeru Fukunaga 1506 Oki Electric Industry Co., Ltd. 1507 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1508 Tel. +81 6 6949 5101 1509 Fax. +81 6 6949 5108 1510 Mail fukunaga444@oki.co.jp 1512 Noriyuki Sato 1513 Oki Electric Industry Co., Ltd. 1514 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1515 Tel. +81 6 6949 5101 1516 Fax. +81 6 6949 5108 1517 Mail sato652@oki.co.jp 1519 Koichi Yano 1520 FastForward Networks, 1521 75 Hawthorne St. #601 1522 San Francisco, CA 94105 1523 Tel. +1.415.430.2500 1525 Akihiro Miyazaki 1526 Matsushita Electric Industrial Co., Ltd 1527 1006, Kadoma, Kadoma City, Osaka, Japan 1528 Tel. +81-6-6900-9192 1529 Fax. +81-6-6900-9193 1530 Mail akihiro@isl.mei.co.jp 1532 Koichi Hata 1533 Matsushita Electric Industrial Co., Ltd 1534 1006, Kadoma, Kadoma City, Osaka, Japan 1535 Tel. +81-6-6900-9192 1536 Fax. +81-6-6900-9193 1537 Mail hata@isl.mei.co.jp 1538 Rolf Hakenberg 1539 Panasonic European Laboratories GmbH 1540 Monzastr. 4c, 63225 Langen, Germany 1541 Tel. +49-(0)6103-766-162 1542 Fax. +49-(0)6103-766-166 1543 Mail hakenberg@panasonic.de 1545 Carsten Burmeister 1546 Panasonic European Laboratories GmbH 1547 Monzastr. 4c, 63225 Langen, Germany 1548 Tel. +49-(0)6103-766-263 1549 Fax. +49-(0)6103-766-166 1550 Mail burmeister@panasonic.de 1552 11. Bibliography 1554 [1] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP - 1555 A Transport Protocol for Real-time Applications," Internet 1556 Draft, draft-ietf-avt-rtp-new-09.txt, Work in Progress, March 1557 2001. 1559 [2] H. Schulzrinne and S. Casner, "RTP Profile for Audio and Video 1560 Conferences with Minimal Control," Internet Draft draft-ietf- 1561 avt-profile-new-10.txt, March 2001. 1563 [3] M. Handley and V. Jacobson, "SDP: Session Description Protocol", 1564 RFC 2327, April 1998. 1566 [4] S. Casner, "SDP Bandwidth Modifiers for RTCP Bandwidth", 1567 Internet Draft draft-ietf-avt-rtcp-bw-03.txt, March 2001. 1569 [5] C. Perkins and O. Hodson, "2354 Options for Repair of Streaming 1570 Media," RFC 2354, June 1998. 1572 [6] J. Rosenberg and H. Schulzrinne, "An RTP Payload Format for 1573 Generic Forward Error Correction,", RFC 2733, December 1999. 1575 [7] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, J.C. 1576 Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP Payload for 1577 Redundant Audio Data," RFC 2198, September 1997. 1579 [8] S. Bradner, "Key words for use in RFCs to Indicate Requirement 1580 Levels," RFC 2119, March 1997. 1582 [9] H. Schulzrinne and S. Petrack, "RTP Payload for DTMF Digits, 1583 Telephony Tones and Telephony Signals," RFC 2833, May 2000. 1585 [10] T. Turletti and C. Huitema, "RTP Payload Format for H.261 Video 1586 Streams, RFC 2032, October 1996. 1588 [11] C. Bormann, L. Cline, G. Deisher, T. Gardos, C. Maciocco, D. 1589 Newell, J. Ott, G. Sullivan, S. Wenger, and C. Zhu, "RTP Payload 1590 Format for the 1998 Version of ITU-T Rec. H.263 Video (H.263+)," 1591 RFC 2429, October 1998. 1593 [12] ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology - 1594 Coding of audio-visual objects - Part2: Visual", July 2000. 1596 [13] ITU-T Recommendation H.263, "Video Coding for Low Bit Rate 1597 Communication," November 2000. 1599 [14] S. Wenger, "Media-aware Protocols -- transport aware Media 1600 Coding," Habilitation thesis, in preparation, 2001. 1602 [15] B. Girod, N. Faerber, "Feedback-based error control for mobile 1603 video transmission," Proceedings IEEE, Vol. 87, No. 10, pp. 1707 1604 � 1723, October, 1999. 1606 [16] M. Handley, J. Padhye, S. Floyd, J. Widmer, "TCP friendly Rate 1607 Control (TFRC): Protocol Specification," Internet Draft, draft- 1608 ietf-tsvwg-02.txt, Work in Progress, May 2001. 1610 Appendix A. Some Background and Motivation (Informative) 1612 A.1 Example: Predictive Video Coding 1614 A.1.1 Video Encoder-decoder synchronicity 1616 Most current video coding schemes for compressed video, such as the 1617 ITU-T H.261 and H.263 and ISO/IEC MPEG[124] employ a mechanism known 1618 as Inter Picture Prediction. Each picture is divided into 1619 macroblocks of uniform size. For each macroblock, one or more 1620 motion vectors may be identified and transmitted. The residual 1621 signal after motion compensation is DCT-transformed, quantized, 1622 entropy coded, and transmitted as well. The encoder reconstructs, 1623 based on this information, a so-called reference picture, which is 1624 used to perform the motion compensation and residual signal coding 1625 steps for the subsequent picture. Since the reference picture is 1626 generated using only such information that is also available at the 1627 decoder, the reference picture is identical to the reconstructed 1628 picture at the decoder. Having identical reference pictures at the 1629 encoder and decoder is referred to as encoder-decoder-synchronicity. 1631 Whenever data is damaged or lost on the way between the encoder and 1632 the decoder, the reconstructed picture at the decoder is no more 1633 identical with the encoder's reference picture -- the encoder-decoder 1634 synchronicity is lost. 1636 Any loss of the encoder-decoder synchronicity results in annoying 1637 artifacts at the decoder. Because the prediction of subsequent 1638 pictures in the decoder is based on a damaged reference picture, the 1639 annoying artifacts are present not only in the picture in which the 1640 loss occurred; they propagate to all subsequent pictures, until, 1641 through source coding based mechanisms, the encoder-decoder 1642 synchronicity is restored. Therefore, the goal of systems employing 1643 predictive video coding in a lossy environment must be to keep the 1644 encoder-decoder synchronicity, or, if this is not possible, to regain 1645 that synchronicity as quickly as possible. 1647 A.1.2. Non-feedback based mechanisms 1649 Avoiding the loss of the encoder-decoder synchronicity corresponds to 1650 avoiding the loss of coded picture data. Such a task can be 1651 performed on the transport layer. In RTP environments, the use of 1652 packet-based FEC is a good example for such a technique. (The use of 1653 TCP or reliable multicast as the transport for media streams would be 1654 an even better one but is inappropriate for low-delay (interactive) 1655 real-time systems.) FEC schemes, interleaving, and other means for 1656 repairing real-time media streams may also add additional delay and 1657 significant bit rate overhead without being able to guarantee 1658 compensation of virtually all packet losses. 1660 Once the encoder-decoder synchronicity is lost, only source coding 1661 oriented mechanisms can help to regain it. One common way is to send 1662 a non-predictively coded picture (known as Intra picture). Intra 1663 pictures have the disadvantage of being several times bigger than 1664 predictively coded pictures (Inter pictures). Therefore, sending 1665 Intra pictures has negative implications both on the bandwidth and 1666 (in bandwidth limited environments) delay. Another way is to use 1667 Intra macroblock refresh. Here, certain parts of the picture (those 1668 affected by a packet loss) are coded non-predictively in order to 1669 resynchronize the encoder and decoder over time. Intra macroblock 1670 refresh has better delay characteristics then full Intra pictures 1671 because the picture size can be kept constant, but is less efficient 1672 in terms of bit rate/distortion than full Intra pictures. More 1673 sophisticated means such as Reference Picture Selection (RPS) are 1674 also available in modern video coding standards. 1676 Systems not employing feedback channels may use any combination of 1677 the mechanisms described above to add error resilience -- at the cost 1678 of added bit rate and, sometimes, added delay. The number of 1679 additional bits spent for error resilience can be adapted using the 1680 long-term packet loss rate information in the RTCP receiver reports. 1681 But, even when using such adaptive means, it is still likely that 1682 systems spend many more bits then theoretically necessary to achieve 1683 error resilience in order to be on the safe side. Plus, as regular 1684 RTCP feedback is aimed at longer terms, reactivity to sudden losses 1685 is limited. In all practical applications today this means that 1686 fewer bits are available for non redundant picture data, and hence 1687 the overall picture quality suffers. 1689 A.1.3 Feedback based systems 1691 Feedback-based systems try to avoid spending too many bits for 1692 redundant information by informing the encoder about a loss situation 1693 at the decoder(s). The encoder can then react accordingly and spend 1694 redundant bits only when needed possibly only for the part of the 1695 picture that was effected by the loss -- thereby reducing the number 1696 of redundant bits and leaving more bits for useful information. As a 1697 result, a higher reproduced picture quality can generally be expected 1698 when feedback channels are available. 1700 Similar to the observations of section 2.1.2, transport and source 1701 coding based mechanisms can be distinguished that react on loss 1702 situations reported by feedback. 1704 Transport based systems employing feedback react media unaware, by 1705 re-transmitting lost packets. TCP is a good example for a protocol 1706 following such a scheme. Transport-based feedback in real-time 1707 and/or multicast environments is a complex matter and subject of a 1708 lot of engineering and research in and outside of the IETF. This 1709 specification is not concerned with pure transport-based feedback. 1711 Source coding based mechanisms may react upon the arrival of a 1712 feedback message indicating a loss situation by adding bits that 1713 restore, or at least make an effort to restore, the encoder-decoder 1714 synchronicity. This process has to be performed by a real-time 1715 encoder. However, schemes were reported, that allow the use of 1716 feedback also for non-real-time encoders by storing multiple 1717 representations of the same data (e.g. Inter and Intra coded), and 1718 dynamically switching between those representations. 1720 Several types of feedback messages, called Feedback Messages or FB 1721 messages, can be defined for such a case. An FB message can be as 1722 simple as a Boolean condition, indicating for example the loss of a 1723 full picture (and, therefore, the need of a full Intra picture 1724 transmission). Other feedback messages may contain more complex 1725 information such as information about the damage of a spatial region 1726 of the picture. A special form consists of a message the format and 1727 semantics of which are not known at the transport level, because they 1728 are defined in the video codec standards. 1730 A.2 Feedback Messages 1732 Most FB messages contain negative acknowledge information, indicating 1733 an erroneous situation at the decoder. In others, the nature of the 1734 acknowledge (positive, negative, or both) is part of the feedback 1735 message itself. When used in multicast environments, positive 1736 acknowledge must not be used. 1738 This document assumes that feedback messages are transmitted using 1739 RTCP packets. RTCP messages from the receivers to the sender cannot 1740 be sent at any possible time, in order to prevent traffic explosion 1741 in case of large multicast groups. Instead, the bit rate for all 1742 RTCP messages of all receivers together has to obey a maximum 1743 fraction of the total RTP session bit rate, yielding a very limited 1744 bit rate budget for a single receiver when having a large multicast 1745 group. This, in turn, leads to an increased average delay when the 1746 size of the receiving multicast group grows. (see section 6 of [1] 1747 for details) 1749 This specification defines an algorithm that adheres to the bit rate 1750 limitations for the feedback channel on the long term, but allows 1751 short-term overdrafting for any receiver (but not all of them 1752 simultaneously). Thus, the algorithm allows for better real-time 1753 performance then the one specified in [1]. Traffic explosion in such 1754 cases in which many receivers identify a picture damage 1755 simultaneously is prevented by dithering. 1757 As this specification assumes a sender that has full control over its 1758 transmission bit rate (e.g. a real-time encoder), there is no scaling 1759 problem on the forward channel. Any reaction to negative feedback 1760 generates additional bits, which have to be conveyed but this is 1761 taken from the sender�s total bit rate budget. The encoder can take 1762 this into account by, for example, changing the encoding mode, packet 1763 size, and so forth. The sender is also free to simply ignore 1764 feedback messages. Adjusting the tradeoff between the reproduced 1765 media quality of all receivers of a multicast group and the amount of 1766 additional repair traffic is a media-dependent, very complex task and 1767 is not covered in this specification. 1769 Finally, frequent RTCP-based feedback messages may provide additional 1770 input to the sender(s)'s congestion control algorithms and thus 1771 improve its reactivity towards network congestion. 1773 Feedback messages as well as sender and receiver behavior are to be 1774 specified in separate documents (such as [7]). Such specifications 1775 need to consider that, frequently, packet loss is an indication of 1776 network congestion and thus define mechanisms for media-specific 1777 congestion control in the presence of feedback as defined in this 1778 memo. 1780 A.3. Applications and Relationships to other Standards 1782 This specification is based on RTCP, which implies its use in an RTP 1783 environment. RTP itself is used in a variety of systems such as in 1784 SIP- or H.323-based multimedia conferencing/telephony, SAP-announced 1785 Mbone conferences, and RTSP-based media streaming. 1787 As for the video codecs, there is currently a small set of standards 1788 that are, for the purpose of this discussion, roughly comparable. 1789 Many mechanisms for regaining encoder-decoder synchronicity are 1790 applicable to all video codecs. Others require certain tools (such 1791 as Reference Picture Selection, aka NEWPRED) that are available only 1792 in certain versions of the standards, and/or optional tools whose use 1793 must be negotiated prior to being used. 1795 A few RTP payload specifications such as RFC 2032 [10] already define 1796 a feedback mechanism for some of the coding algorithms considered in 1797 this specification. An application capable of performing both 1798 schemes MUST use the feedback mechanism defined in this 1799 specification, although, for backward compatibility reasons, it MUST 1800 also be capable to conform to the feedback scheme defined in the 1801 respective RTP payload format, if this is required by that payload 1802 format. 1804 Also, audio, DTMF, and text streams could benefit from more immediate 1805 feedback even though the redundancy payload formats work well for 1806 these media. 1808 All kinds of non-interactive media streams (such as RTSP-controlled 1809 media streaming applications) could benefit significantly as without 1810 interactivity there is more time available for media repair. 1812 A.4 Remarks on the size of the multicast group 1814 This specification prevents traffic explosion on the feedback channel 1815 in a very similar way as RTP does, with the exception of allowing 1816 individual receivers to overdraft their bit rate budget from time to 1817 time. This is necessary in order to allow for low delay, which is 1818 needed by the algorithms reacting to Feedback messages. 1820 This scaling, however, limits the usefulness of this mechanism in 1821 multicast groups from a certain size upwards (where the size 1822 threshold depends on a number of parameters including loss rate, 1823 frame rate, number of packets per frame, and session bandwidth). The 1824 maximum size of the multicast group is soft and also depends on 1825 application requirements and is therefore not specified here. 1826 Considerations on the multicast group sizes are presented in section 1827 3.5.