idnits 2.17.1 draft-ietf-avt-rtcp-feedback-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 12 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 5 instances of lines with multicast IPv4 addresses in the document. If these are generic example addresses, they should be changed to use the 233.252.0.x range defined in RFC 5771 ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? RFC 2119 keyword, line 218: '...tiple FB messages MAY be combined in a...' RFC 2119 keyword, line 219: '... packet and they MAY also be sent comb...' RFC 2119 keyword, line 223: '... document MUST contain RTCP packets in the order as defined in [1]:...' RFC 2119 keyword, line 225: '... o OPTIONAL encryption prefix that M...' RFC 2119 keyword, line 228: '...ATORY SDES which MUST contain the CNAM...' (147 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 1710 has weird spacing: '... These messa...' == Line 1915 has weird spacing: '... Mail fukun...' == Line 1922 has weird spacing: '... Mail sato6...' == Line 1934 has weird spacing: '... Mail akihi...' == Line 1941 has weird spacing: '... Mail hata@...' == (2 more instances...) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (December 2002) is 7803 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 1959 looks like a reference -- Missing reference section? '10' on line 1991 looks like a reference -- Missing reference section? '7' on line 1981 looks like a reference -- Missing reference section? '11' on line 1994 looks like a reference -- Missing reference section? '5' on line 1975 looks like a reference -- Missing reference section? '6' on line 1978 looks like a reference -- Missing reference section? '2' on line 1964 looks like a reference -- Missing reference section? '8' on line 1985 looks like a reference -- Missing reference section? '3' on line 1968 looks like a reference -- Missing reference section? '4' on line 1972 looks like a reference -- Missing reference section? '18' on line 2022 looks like a reference -- Missing reference section? '14' on line 2005 looks like a reference -- Missing reference section? '13' on line 2002 looks like a reference -- Missing reference section? '15' on line 2009 looks like a reference -- Missing reference section? '12' on line 1999 looks like a reference -- Missing reference section? '16' on line 2013 looks like a reference -- Missing reference section? '17' on line 2017 looks like a reference -- Missing reference section? '9' on line 1988 looks like a reference Summary: 5 errors (**), 0 flaws (~~), 10 warnings (==), 21 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT Joerg Ott/Uni Bremen TZI 3 draft-ietf-avt-rtcp-feedback-03.txt Stephan Wenger/TU Berlin 4 Shigeru Fukunaga/Oki 5 Noriyuki Sato/Oki 6 Koichi Yano/Fast Forward Networks 7 Akihiro Miyazaki/Matsushita 8 Koichi Hata/Matsushita 9 Rolf Hakenberg/Matsushita 10 Carsten Burmeister/Matsushita 11 Jose Rey/Matsushita 13 29 June 2002 14 Expires December 2002 16 Extended RTP Profile for RTCP-based Feedback (RTP/AVPF) 18 Status of this Memo 20 This document is an Internet-Draft and is in full conformance with all 21 provisions of Section 10 of RFC 2026. Internet-Drafts are working 22 documents of the Internet Engineering Task Force (IETF), its areas, 23 and its working groups. Note that other groups may also distribute 24 working documents as Internet-Drafts. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet- Drafts as reference 29 material or to cite them other than as "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html. 37 Abstract 39 Real-time media streams that use RTP are not resilient against 40 packet losses. Receivers may use the base mechanisms of RTCP to 41 report packet reception statistics and thus allow a sender to adapt 42 its transmission behavior in the mid-term as sole means for 43 feedback and feedback-based error repair (besides a few codec- 44 specific mechanisms). This document defines an extension to the 45 Audio-visual Profile (AVP) that enables receivers to provide, 46 statistically, more immediate feedback to the senders and thus 47 allow for short-term adaptation and efficient feedback-based repair 48 mechanisms to be implemented. This early feedback profile (AVPF) 49 maintains the AVP bandwidth constraints for RTCP and preserves 50 scalability to large groups. 52 1. Introduction 54 Real-time media streams that use RTP are not resilient against 55 packet losses. RTP [1] provides all the necessary mechanisms to 56 restore ordering and timing present at the sender to properly 57 reproduce a media stream at a recipient. RTP also provides 58 continuous feedback about the overall reception quality from all 59 receivers -- thereby allowing the sender(s) in the mid-term (in the 60 order of several seconds to minutes) to adapt their coding scheme 61 and transmission behavior to the observed network QoS. However, 62 except for a few payload specific mechanisms [10], RTP makes no 63 provision for timely feedback that would allow a sender to repair 64 the media stream immediately: through retransmissions, retro-active 65 FEC control, or media-specific mechanisms for some video codecs, 66 such as reference picture selection. 68 Current mechanisms available with RTP to improve error resilience 69 include audio redundancy coding [7], video redundancy coding [11], 70 RTP-level FEC [5], and general considerations on more robust media 71 streams transmission [6]. These mechanisms may be applied pro- 72 actively (thereby increasing the bandwidth of a given media 73 stream). Alternatively, in sufficiently small groups with small 74 RTTs, the senders may perform repair on-demand, using the above 75 mechanisms and/or media-encoding-specific approaches. Note that 76 "small group" and "sufficiently small RTT" are both highly 77 application dependent. 79 This document specifies a modified RTP Profile for audio and video 80 conferences with minimal control based upon [1] and [2] by means of 81 two modifications/additions: to achieve timely feedback, the 82 concepts of Immediate Feedback messages and Early RTCP messages as 83 well as algorithms allowing for low delay feedback in small 84 multicast groups (and preventing feedback implosion in large ones) 85 are introduced. Special consideration is given to point-to-point 86 scenarios. A small number of general-purpose feedback messages as 87 well as a format for codec and application-specific feedback 88 information are defined as specific RTCP payloads. 90 1.1 Definitions 92 The definitions from [1] and [2] apply. In addition, the following 93 definitions are used in this document: 95 Early RTCP mode: 96 The mode of operation in which a receiver of a media stream 97 is, statistically, often (but not always) capable of 98 reporting events of interest back to the sender close to 99 their occurrence. In Early RTCP mode, RTCP feedback 100 messages are transmitted according to the timing rules 101 defined in this document. 103 Early RTCP packet: 104 An Early RTCP packet is a packet which is transmitted 105 earlier than would be allowed if following the scheduling 106 algorithm of [1], the reason being an "event" observed by a 107 receiver. Early RTCP packets may be sent in Immediate 108 feedback and in Early RTCP mode. 110 Event: 111 An observation made by the receiver of a media stream that 112 is (potentially) of interest to the sender -- such as a 113 packet loss or packet reception, frame loss, etc. -- and 114 thus useful to be reported back to the sender by means of a 115 Feedback message. 117 Feedback (FB) message: 118 An RTCP message as defined in this document is used to 119 convey information about events observed at a receiver -- 120 in addition to long term receiver status information which 121 is carried in RTCP RRs -- back to the sender of the media 122 stream. 124 Feedback (FB) threshold: 125 The FB threshold indicates the transition between Immediate 126 Feedback and Early RTCP mode. For a multicast scenario, 127 the FB threshold indicates the maximum group size at which, 128 on average, each receiver is able to report each event back 129 to the sender(s) immediately, i.e. by means of an Early 130 RTCP packet without having to wait for its regularly 131 scheduled RTCP interval. This threshold is highly 132 dependent on the type of feedback to be provided, network 133 QoS (e.g. packet loss probability and distribution), codec 134 and packetization scheme in use, the session bandwidth, and 135 application requirements. Note that the algorithms do not 136 depend on all senders and receivers agreeing on the same 137 value for this threshold. It is merely intended to provide 138 conceptual guidance to application designers and is not 139 used in any calculations. 141 Immediate Feedback mode: 142 A mode of operation in which each receiver of a media 143 stream is, statistically, capable of reporting each event 144 of interest immediately back to the media stream sender. 146 In Immediate Feedback mode, RTCP feedback messages are 147 transmitted according to the timing rules defined in this 148 document. 150 Regular RTCP mode: 151 Mode of operation in which no preferred transmission of 152 feedback messages is allowed. Instead, RTCP messages are 153 sent following the rules of [1]. Nevertheless, such RTCP 154 messages may contain feedback information as defined in 155 this document. 157 Regularly Scheduled RTCP packet: 158 An RTCP packet that is not sent as an Early RTCP packet. 160 1.2 Terminology 162 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 163 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" 164 in this document are to be interpreted as described in RFC 2119 165 [8] 167 2. RTP and RTCP Packet Formats and Protocol Behavior 169 The rules defined in [2] also apply to this profile except for 170 those rules mentioned in the following: 172 RTCP packet types: 173 Two additional RTCP packet types to convey feedback 174 information are defined in section 6. 176 RTCP report intervals: 177 This memo describes three modes of operation which 178 influence the RTCP report intervals (see section 3.2). In 179 regular RTCP mode, all rules from [1] apply. In both 180 Immediate Feedback and Early RTCP modes the minimal 181 interval of five seconds between two RTCP reports is 182 dropped and the rules specified in section 3 apply if RTCP 183 packets containing feedback messages (defined in section 4) 184 are to be transmitted. 186 The rules set forth in [1] may be overridden by session 187 descriptions specifying different parameters (e.g. for the 188 bandwidth share assigned to RTCP for senders and receivers, 189 respectively). For sessions defined using the Session 190 Description Protocol (SDP) [3], the rules of [4] apply. 192 Congestion control: 193 The same basic rules as detailed in [2] apply. Beyond 194 this, in section 5, further consideration is given to the 195 impact of feedback and a sender's reaction to feedback 196 messages. 198 3. Rules for RTCP Feedback 200 3.1 Compound RTCP Feedback Packets 202 Two components constitute RTCP-based feedback as described in this 203 memo: 205 o Status reports are contained in SR/RR messages and are 206 transmitted at regular intervals as part of compound RTCP 207 packets (which also include SDES and possibly other messages); 208 these status reports provide an overall indication for the 209 recent reception quality of a media stream. 211 o Feedback messages as defined in this document that indicate loss 212 or reception of particular pieces of a media stream (or provide 213 some other form of rather immediate feedback on the data 214 received). Rules for the transmission of feedback messages are 215 newly introduced in this memo. 217 RTCP Feedback (FB) messages are just another RTCP packet type (see 218 section 4). Therefore, multiple FB messages MAY be combined in a 219 single compound RTCP packet and they MAY also be sent combined with 220 other RTCP packets. 222 RTCP packets containing Feedback packets as defined in this 223 document MUST contain RTCP packets in the order as defined in [1]: 225 o OPTIONAL encryption prefix that MUST be present if the RTCP 226 message is to be encrypted. 227 o MANDATORY SR or RR. 228 o MANDATORY SDES which MUST contain the CNAME item; all other SDES 229 items are OPTIONAL. 230 o One or more FB messages. 232 The FB message(s) MUST be placed in the compound packet after RR 233 and SDES RTCP packets defined in [1]. The ordering with respect to 234 other RTCP extensions is not defined. 236 Two types of compound RTCP packets carrying feedback packets are 237 used in this document: 239 a) Minimal compound RTCP feedback packet 241 A minimal compound RTCP feedback packet MUST contain only the 242 mandatory information as listed above: encryption prefix if 243 necessary, exactly one RR or SR, exactly one SDES with only the 244 CNAME item present, and the feedback message(s). This is to 245 minimize the size of the RTCP packet transmitted to convey 246 feedback and thus to maximize the frequency at which feedback 247 can be provided while still adhering to the RTCP bandwidth 248 limitations. 250 This packet format SHOULD be used whenever an RTCP feedback 251 message is sent as part of an Early RTCP packet. 253 b) (Full) compound RTCP feedback packet 255 A (full) compound RTCP feedback packet MAY contain any 256 additional number of RTCP packets (additional RRs, further SDES 257 items, etc.). The above ordering rules MUST be adhered to. 259 This packet format MUST be used whenever an RTCP feedback 260 message is sent as part of a regularly scheduled RTCP packet or 261 in Regular RTCP mode. It MAY also be used to send RTCP 262 feedback messages in Immediate Feedback or Early RTCP mode. 264 RTCP packets that do not contain FB messages are referred to as 265 non-FB RTCP packets. Such packets MUST follow the format rules in 266 [1]. 268 3.2 Algorithm Outline 270 FB messages are part of the RTCP control streams and are thus 271 subject to the same bandwidth constraints as other RTCP traffic. 272 This means in particular that it may not be possible to report an 273 event observed at a receiver immediately back to the sender. 274 However, the value of feedback given to a sender typically 275 decreases over time -- in terms of the media quality as perceived 276 by the user at the receiving end and/or the cost required to 277 achieve media stream repair. 279 RTP [1] and the commonly used RTP profile [2] specify rules when 280 compound RTCP packets should be sent. This document modifies those 281 rules in order to allow applications to timely report events (e.g. 282 loss or reception of media packets) to accommodate algorithms that 283 use FB messages and are sensitive to the feedback timing. 285 The modified RTCP transmission algorithm can be outlined as 286 follows: Normally, when no FB messages have to be conveyed, 287 compound RTCP packets are sent following the rules of RTP [1] -- 288 except that the five second minimum interval between RTCP reports 289 is not enforced and the interval between RTCP reports is only 290 derived from the average RTCP packet size and the RTCP bandwidth 291 share available to the RTP/RTCP entity; in addition, a minimum 292 interval between regular RTCP packets may be enforced. If a 293 receiver detects the need to send an FB message, the receiver waits 294 for a short, random dithering interval (in case of multicast) and 295 then checks whether it has already seen a corresponding FB message 296 from any other receiver (which it can do with all FB messages that 297 are transmitted via multicast; for unicast sessions, there is no 298 such delay). If this is the case then the receiver refrains from 299 sending the FB message and continues to follow the regular RTCP 300 transmission schedule. If the receiver has not yet seen a similar 301 FB message from any other receiver, it checks whether it has 302 recently sent another FB message (without waiting for its regularly 303 scheduled RTCP transmission time). Only if this is not the case, 304 it sends the FB message as part of a (minimal) compound RTCP 305 packet. 307 FB messages may also be sent as part of full compound RTCP packets 308 which are interspersed as per [1] (except for the five second lower 309 bound) in regular intervals. 311 3.3 Modes of Operation 313 RTCP-based feedback may operate in one of three modes (figure 1) as 314 described below. The mode is a hint whether or not a receiver 315 should send early feedback at all and, if so, whether, 316 statistically, all events observed at the receiver can be reported 317 back to the sender in a timely fashion. The current mode of 318 operation is continuously derived independently at each receiver 319 and the receivers do not have to agree on a common mode. 321 a) Immediate feedback mode: the group size is below the FB 322 threshold which gives each receiving party sufficient bandwidth 323 to transmit the RTCP feedback packets for the intended purpose. 324 This means that, for each receiver, there is enough bandwidth 325 to report each event it is supposed/expected to by means of a 326 virtually "immediate" RTCP feedback packet. 328 The group size threshold is a function of a number of 329 parameters including (but not necessarily limited to) the type 330 of feedback used (e.g. ACK vs. NACK), bandwidth, packet rate, 331 packet loss probability and distribution, media type, codec, 332 and -- again depending on the type of FB used -- the (worst 333 case or observed) frequency of events to report (e.g. frame 334 received, packet lost). 336 A special case of this is the ACK mode (where positive 337 acknowledgements are used to confirm reception of data) which 338 is restricted to point-to-point communications. 340 As a rough estimate, let N be the average number of events to 341 be reported per interval T by a receiver, B the RTCP bandwidth 342 fraction for this particular receiver and R the average RTCP 343 packet size, then the receiver operates in Immediate Feedback 344 mode as long as N<=B*T/R. 346 b) Early RTCP mode: In this mode, the group size and other 347 parameters no longer allow each receiver to react to each event 348 that would be worth (or needed) to report. But feedback can 349 still be given sufficiently often so that it allows the sender 350 to adapt the media stream transmission accordingly and thereby 351 increase the overall reproduced media quality. 353 Using the above notation, Early RTCP mode can be roughly 354 characterized by N > B*T/R as "lower bound". An estimate for 355 an upper bound is more difficult. Setting N=1, we obtain for a 356 given R and B the interval T = R/B as average interval between 357 events to be reported. This information can be used as a hint 358 to determine whether or not early transmission of RTCP packets 359 is useful. 361 c) From some group size upwards, it is no longer useful to provide 362 feedback from individual receivers at all -- because of the 363 time scale in which the feedback could be provided and/or 364 because in large groups the sender(s) have no chance to react 365 to individual feedback anymore. 367 No group size threshold can be specified at which this mode 368 starts. 370 As the feedback algorithm described in this memo scales smoothly, 371 there is no need for an agreement among the participants on the 372 precise values of the respective "thresholds" within the group. 373 Hence the borders between all these modes are allowed to be 374 fluent. 376 ACK 377 feedback 378 V 379 :<- - - - NACK feedback - - - ->// 380 : 381 : Immediate || 382 : Feedback mode ||Early RTCP mode Regular RTCP mode 383 :<=============>||<=============>//<=================> 384 : || 385 -+---------------||---------------//------------------> group size 386 2 || 387 Application-specific FB Threshold 388 = f(data rate, packet loss, codec, ...) 390 Figure 1: Modes of operation 392 As stated before, the respective thresholds depend on a number of 393 technical parameters (of the codec, the transport, the type of 394 feedback used, etc.) but also on the respective application 395 scenarios. Section 3.6 provides some useful hints (but no precise 396 calculations) on estimating these thresholds. 398 3.4 Definitions 400 The following pieces of state information need to be maintained per 401 receiver (largely taken from [1]). Note that all variables (except 402 for g) are calculated independently at each receiver and so their 403 local values may differ at any given point in time. 405 a) Let "senders" be the number of active senders in the RTP 406 session. 408 b) Let "members" be the current estimate of the number of receivers 409 in the RTP session. 411 c) Let tn and tp be the time for the next (last) scheduled 412 RTCP RR transmission calculated prior to reconsideration. 414 d) Let Tmin be the minimal interval between RTCP packets as per 415 [1]. Unlike [1], the initial Tmin is set to 1 second, then it 416 is set to 0. 418 e) Let T_rr be the interval after which, having just sent a 419 regularly scheduled RTCP packet, a receiver would schedule the 420 transmission of its next regular RTCP packet following the rules 421 of [1]: T_rr = T (the "calculated interval") with tn = tp + T. 422 Note that a different Tmin is used to compute the "calculated 423 interval T". T_rr refers to the last value of T that has been 424 computed (because of reconsideration or to determine tn). 426 f) Let t0 be the time at which an event that is to be reported is 427 detected by a receiver. 429 g) Let T_dither_max be the maximum interval for which an RTCP 430 feedback packet may be additionally delayed (to prevent 431 implosions). 433 h) Let T_max_fb_delay be the upper bound within which feedback to 434 an event needs to be reported back to the sender to be useful at 435 all. Note that this value is application-specific. 437 i) Let te be the time for which a feedback packet is scheduled. 439 j) Let T_fd be the actual (randomized) delay for the transmission 440 of feedback message in response to an event that a certain 441 packet P caused. 443 k) Let allow_early be a Boolean variable that indicates whether the 444 receiver currently may transmit feedback messages prior to its 445 next regularly scheduled RTCP interval tn. This variable is 446 used to throttle the feedback sent by a single receiver. 447 allow_early is adjusted (set to FALSE) after early feedback 448 transmission and is reset to TRUE as soon as the next regular 449 RTCP transmission has occurred. 451 l) Let avg_rtcp_size be the moving average on the RTCP packet size 452 as defined in [1]. 454 m) Let T_rr_interval be an (optional) minimal interval to be used 455 between regular RTCP packets. If T_rr_interval != 0 then 456 regular RTCP packets will not be scheduled T_rr after the last 457 regular RTCP transmission (at tp+T_rr) but only at least 458 T_rr_interval after the last regular RTCP transmission (later 459 than or at tp+T_rr). T_rr_interval does not affect the 460 calculation of T_rr and tp but may lead to regular RTCP packets 461 being suppressed. T_rr_interval does not affect transmission 462 scheduling for Early RTCP packets. 464 n) Let t_rr_last be the point in time at which the last RTCP packet 465 has been scheduled and sent (i.e. has not been suppressed due to 466 T_rr_interval). 468 NOTE: Providing T_rr_interval as an independent variable is 469 meant to minimize regular feedback (and thus bandwidth 470 consumption) as needed by the application but still allow for 471 more frequent use of Early RTCP packets to provide timely 472 feedback. This goal could not be achieved by reducing the 473 overall RTCP bandwidth as RTCP bandwidth reduction would also 474 impact the Early feedback. 476 The feedback situation for an event to report at a receiver is 477 depicted in figure 2 below. At time t0, such an event (e.g. a 478 packet loss) is detected at the receiver. The receiver decides -- 479 based upon current bandwidth, group size, and other (application- 480 specific) parameters -- that a feedback message needs to be sent 481 back to the sender. 483 To avoid an implosion of immediate feedback packets, the receiver 484 MUST delay the transmission of the RTCP feedback packet by a random 485 amount T_fd (with the random number evenly distributed in the 486 interval [0, T_dither_max]). Transmission of the compound RTCP 487 packet MUST then be scheduled for te = t0 + T_fd. 489 The T_dither_max parameter is derived from the regular RTCP 490 interval (which, in turn, is based upon the group size). 492 For a certain application scenario, a receiver may determine an 493 upper bound for the acceptable local delay of feedback messages: 494 T_max_fb_delay. If an a priori estimation or the actual 495 calculation of T_dither_max indicates that this upper bound MAY be 496 violated (e.g. because T_dither_max > T_max_fb_delay), the receiver 497 MAY decide not to send any feedback at all because the achievable 498 gain is considered insufficient. 500 If an RTCP feedback packet is scheduled, the time slot for the next 501 scheduled (full) compound RTCP packet MUST be updated accordingly 502 to a new tn (which will then be in the order of tn=tp+2*T_rr). 503 This is to ensure that the short term average bandwidth used for 504 RTCP with feedback does not exceed the bandwidth limit that would 505 be used without feedback. 507 event to 508 report 509 detected 510 | 511 | RTCP feedback range 512 | (T_max_fb_delay) 513 vXXXXXXXXXXXXXXXXXXXXXXXXXXX ) ) 514 |---+--------+-------------+-----+------------| |--------+---> 515 | | | | ( ( | 516 | t0 te | 517 tp tn 518 \_______ ________/ 519 \/ 520 T_dither_max 522 Figure 2: Event report and parameters for Early RTCP scheduling 524 3.5 Early RTCP Algorithm 526 Assume an active sender S0 (out of S senders) and a number N of 527 receivers with R being one of these receivers. 529 Assume further that R has verified that using feedback mechanisms 530 is reasonable at the current constellation (which is highly 531 application specific and hence not specified in this memo). 533 Assume that T_rr_interval is 0 ,if no minimal interval between 534 regular RTCP packets is to be enforced, or T_rr_interval is set to 535 some meaningful value, as given by the application. This value 536 then denotes the minimal interval between regular RTCP packets. 538 Then, receiver R MUST use the following rules for transmitting one 539 or more Feedback messages as minimal or full compound RTCP packet: 541 3.5.1 Initialization 543 Initially, R MUST set allow_early = TRUE and t_rr_last = NaN. 545 Furthermore, the initialization of the RTCP variables as per [1] 546 applies except that the initial value for Tmin is 1.0 seconds. 548 3.5.2 Early Feedback Transmission 550 Assume that R has scheduled the last RTCP RR packet for 551 transmission at tp and has scheduled the next transmission 552 (including possible reconsideration) for tn = tp + T_rr. Assume 553 also that the last T_rr_interval-based transmission (if any) has 554 occurred at t_rr_last (if defined). 556 1. At time t0, R detects the need to transmit one or more RTCP 557 feedback messages (e.g. because media "units" needs to be ACKed or 558 NACKed) and finds that sending the feedback information is useful 559 for the sender. 561 2. R first checks whether there is still a compound RTCP feedback 562 packet waiting for transmission (scheduled as early or regular RTCP 563 packet). 565 2.a) If so, the new feedback message MUST be appended to the 566 packet; the scheduling of the waiting RTCP feedback packet 567 MUST remain unchanged. When appending, the feedback 568 information of several RTCP feedback packets SHOULD be merged 569 to produce as few feedback messages as possible. This 570 completes the course of immediate actions to be taken. 572 2.b) If no RTCP feedback message is already awaiting 573 transmission, a new (minimal or full) compound RTCP feedback 574 packet MUST be created and the minimal interval for 575 T_dither_max MUST be chosen as follows: 577 i) If the session is a unicast session (group size = 2) then 578 T_dither_max = 0. 580 ii) If the session is a multicast session with potentially 581 more than two group members then 583 T_dither_max = l * T_rr 585 with l=0.5. 587 The values given above for T_dither_max are minimal values. 588 Application-specific feedback considerations may make it 589 worthwhile to increase T_dither_max beyond this value. This 590 is up to the discretion of the implementer. 592 3. Then, R MUST check whether its next regularly scheduled RTCP 593 packet is within the time bounds for the RTCP FB (t0 + T_dither_max 594 > tn). 596 3.a) If so, an Early RTCP packet MUST NOT be scheduled; 597 instead the FB message(s) MUST be stored to be appended to the 598 regular RTCP packet scheduled for tn. This completes the 599 course of immediate actions to be taken. 601 3.b) Otherwise, the following steps are carried out. 603 4. R MUST check whether it is allowed to transmit an Early RTCP 604 packet (allow_early == TRUE). 606 4.a) If allow_early == FALSE then R MUST check the time for the 607 next scheduled RR: 609 1. If tn - t0 < T_max_fb_delay (i.e. if, despite late 610 reception, the feedback could still be useful for the 611 sender) then R MAY create an RTCP FB message for 612 transmission along with the RTCP packet at tn. 614 2. Otherwise, R MUST discard the RTCP feedback message. 616 This completes the immediate course of actions to be taken. 618 4.b) If allow_early == TRUE then R MUST schedule an Early RTCP 619 packet for te = t0 + RND * T_dither_max with RND being a pseudo 620 random function evenly distributed between 0 and 1. 622 5. If, while waiting for te, R receives RTCP feedback packets 623 contained in one or more (minimal) compound RTCP packets, R MUST 624 act as follows for each of the RTCP feedback messages in the one or 625 more compound RTCP packets received in the interval [t0 - 1s ; te]: 627 5.a) If R understands the received feedback message's semantics 628 and the message contents is a superset of the feedback R wanted 629 to send then R MUST discard its own feedback message and MUST 630 re-schedule the next regular RTCP message transmission for tn 631 (as calculated before). 633 5.b) If R understands the received feedback message's semantics 634 and the message contents is not a superset of the feedback R 635 wanted to send then R SHOULD transmit its own feedback message 636 as scheduled. If there is an overlap between the feedback 637 information to send and the feedback information received, the 638 amount of feedback transmitted is up to R: R MAY send its 639 feedback information unchanged, R MAY as well eliminate any 640 redundancy between its own feedback and the feedback received 641 so far. 643 5.c) If R does not understand the received feedback message's 644 semantics, R MAY send its own feedback message as Early RTCP 645 packet, or R MAY re-schedule the next regular RTCP message 646 transmission for tn (as calculated before) and MAY append the 647 feedback message to the now regularly scheduled RTCP message. 649 Note: With rule #3, receiving unknown feedback packets may not 650 lead to feedback suppression at a particular receiver. As a 651 consequence, a given event may cause M different types of 652 feedback packets (which are all appropriate but not the same 653 and mutually not understood) to be scheduled, and a "large" 654 receiver group may be partitioned into at most M groups. Among 655 members of each of these M groups, feedback suppression will 656 occur following the rules #1 and #2 but no suppression will 657 happen across groups. As a result, O(M) RTCP feedback messages 658 may be received by the sender. Given that these M groups 659 consist of receivers for the same application using the same 660 (set of) codecs in the same RTP session, M is assumed to be 661 small in the general case. Given further that the O(M) 662 feedback packets are randomly distributed over a time interval 663 of T_dither_max, the resulting limited number of extra feedback 664 packets (a) is assumed not to overwhelm the sender and (b) 665 should be conveyed as all contain complementary pieces of 666 information. 668 Refer to section 4 on the comparison of feedback messages and 669 for which feedback messages MUST be understood by a receiver. 671 6. Otherwise, when te is reached, R MUST transmit the RTCP packet 672 containing the FB message. R then MUST set allow_early = FALSE, 673 MUST recalculate tn = tp + 2*T_rr, and MUST set tp to the previous 674 tn. As soon as the newly calculated tn is reached and R sends its 675 next regularly scheduled RTCP RR or suppresses it because of 676 T_rr_interval, it MUST set allow_early = TRUE again. 678 3.5.3 Regular RTCP Transmission 680 In regular intervals full compound RTCP packets MUST be sent. 681 These packets MAY also contain one or more feedback messages. 682 Transmission of regular RTCP packets is scheduled as follows: 684 If T_rr_interval == 0 then the transmission MUST follow the rules 685 as specified by [1] (except for the different Tmin) and MUST adhere 686 to the adjustments of tn specified in section 3.5.2 (i.e. skip one 687 regular transmission if an Early RTCP transmission has occurred). 688 Timer reconsideration takes place when tn is reached as per [1] and 689 the regular RTCP packet is transmitted after timer reconsideration. 690 Whenever a regular RTCP message is sent, allow_early MUST be set to 691 TRUE and tp, tn MUST be updated as per [1]. If this was the first 692 transmission of an RTCP packet, Tmin MUST be set to 0. 694 If T_rr_interval != 0 then the calculation for the transmission 695 times MUST follow the rules as specified in [1] (except for the 696 different Tmin) and MUST adhere to the adjustments of tn specified 697 in section 3.5.2 (i.e. skip one regular transmission if an Early 698 RTCP transmission has occurred). Timer reconsideration takes place 699 when tn is reached as per [1]. After timer reconsideration, the 700 following actions are taken: 702 If no full compound RTCP packet has been sent before (i.e. if 703 t_rr_last == NaN) then a full compound RTCP packet MUST be 704 scheduled. Stored RTCP feedback messages MAY be included in 705 the full compound RTCP packet. t_rr_last MUST be set to tn. 706 Tmin MUST be set to 0. 708 If t_rr_last + T_rr_interval <= tn then a full compound RTCP 709 packet MUST be scheduled. Stored RTCP feedback messages MAY 710 be included in the full compound RTCP packet. t_rr_last MUST 711 be set to tn. 713 If t_rr_last + T_rr_interval > tn and RTCP feedback messages 714 have been stored and are awaiting transmission, an RTCP packet 715 MUST be scheduled. This RTCP packet MAY be a minimal or a 716 full compound RTCP packet (at the discretion of the 717 implementer) and the compound RTCP packet MUST include the 718 stored RTCP feedback message. t_rr_last MUST remain 719 unchanged. 721 Otherwise (if t_rr_last + T_rr_interval > tn but no stored 722 RTCP feedback messages are awaiting transmission), no compound 723 RTCP packet MUST be scheduled. 725 In all the four cases above, allow early MUST be set to TRUE and tp 726 and tn MUST be updated following the rules of [1] except for the 727 five second minimum. 729 3.5.4 Other Considerations 731 Furthermore, if T_rr_interval != 0 then the timeout calculation for 732 RTP/AVPF entities (section 6.3.5 of [1]) MUST be modified to use 733 T_rr_interval instead of Tmin for computing Td. 735 Whenever an RTCP packet is sent or received -- minimal or full 736 compound, early or regularly scheduled -- the avg_rtcp_size 737 variable MUST be updated accordingly (see [1]) and the tn MUST be 738 calculated using the new avg_rtcp_size. 740 3.6 Considerations on the Group Size 742 This section provides some guidelines to the group sizes at which 743 the various feedback modes may be used. 745 3.6.1 ACK mode 747 The group size MUST be exactly two participants, i.e. point-to- 748 point communications. Unicast addresses MUST be used in the 749 session description. 751 For unidirectional as well as bi-directional communication between 752 two parties, 2.5% of the RTP session bandwidth are available for 753 RTCP traffic from the receivers including feedback. For a 64 754 kbit/s stream this yields 1,600 bit/s for RTCP. If we assume an 755 average of 96 bytes (=768 bits) per RTCP packet a receiver can 756 report 2 events per second back to the sender. If acknowledgments 757 for 10 events are collected in each feedback message then 20 events 758 can be acknowledged per second. At 256 kbit/s 8 events could be 759 reported per second; thus the ACKs may be sent in a finer 760 granularity (e.g. only combining three ACKs per RTCP feedback 761 message). 763 From 1 Mbit/s upwards, a receiver would be able to acknowledge each 764 individual frame (not packet!) in a 30 fps video stream. 766 ACK strategies MUST be defined to work properly with these 767 bandwidth limitations. An indication whether or not ACKs are 768 allowed for a session and, if so, which ACK strategy should be 769 used, MAY be conveyed by out-of-band mechanisms, e.g. media- 770 specific attributes in a session description using SDP. 772 3.6.2 NACK mode 774 Negative acknowledgements (or similar types of feedback) MUST be 775 used for all groups larger than two. Of course, NACKs MAY be used 776 for point-to-point communications as well. 778 Whether or not the use of Immediate or Early RTCP packets should be 779 considered depends upon a number of parameters including session 780 bandwidth, codec, special type of feedback, number of senders and 781 receivers, among many others. 783 The crucial parameters -- to which virtually all of the above can 784 be reduced -- is the allowed minimal interval between two RTCP 785 reports and the (average) number of events that presumably need 786 reporting per time interval (plus their distribution over time, of 787 course). The minimum interval can be derived from the available 788 RTCP bandwidth and the expected average size of an RTCP packet. 789 The number of events to report e.g. per second may be derived from 790 the packet loss rate and sender's rate of transmitting packets. 791 From these two values, the allowable group size for the Immediate 792 feedback mode can be calculated. 794 Let N be the average number of events to be reported per 795 interval T by a receiver, B the RTCP bandwidth fraction for 796 this particular receiver and R the average RTCP packet size, 797 then the receiver operates in Immediate Feedback mode is used 798 as long as N<=B*T/R. 800 The upper bound for the Early RTCP mode then solely depends on the 801 acceptable quality degradation, i.e. how many events per time 802 interval may go unreported. 804 Using the above notation, Early RTCP mode can be roughly 805 characterized by N > B*T/R as "lower bound". An estimate for 806 an upper bound is more difficult. Setting N=1, we obtain for a 807 given R and B the interval T = R/B as average interval between 808 events to be reported. This information can be used as a hint 809 to determine whether or not early transmission of RTCP packets 810 is useful. 812 Example: If a 256kbit/s video with 30 fps is transmitted through a 813 network with an MTU size of some 1,500 bytes, then, in most cases, 814 each frame would fit in its own packet leading to a packet rate of 815 30 packets per second. If 5% packet loss occurs in the network 816 (equally distributed, no inter-dependence between receivers), then 817 each receiver will have to report 3 packets lost each two seconds. 818 Assuming a single sender and more than three receivers, this yields 819 3.75% of the RTCP bandwidth allocated to the receivers and thus 820 9.6kbit/s. Assuming further a size of 120 bytes for the average 821 compound RTCP packet allows 10 RTCP packets to be sent per second 822 or 20 in two seconds. If every receiver needs to report three 823 packets, this yields a maximum group size of 6-7 receivers if all 824 loss events shall be reported. The rules for transmission of 825 immediate RTCP packets should provide sufficient flexibility for 826 most of this reporting to occur in a timely fashion. 828 Extending this example to determine the upper bound for Early RTCP 829 mode could lead to the following considerations: assume that the 830 underlying coding scheme and the application (as well as the 831 tolerant users) allow on the order of one loss without repair per 832 two seconds. Thus the number of packets to be reported by each 833 receiver decreases to two per two seconds second and increases the 834 group size to 10. Assuming further that some number of packet 835 losses are correlated, feedback traffic is further reduced and 836 group sizes of some 12 to 16 (maybe even 20) can be reasonably well 837 supported using Early RTCP mode. Note, of course, that all those 838 considerations are based upon statistics and will fail to hold in 839 some cases. 841 3.7 Summary of decision steps 842 3.7.1 General Hints 844 Before even considering whether or not to send RTCP feedback 845 information an application has to determine whether this mechanism 846 is applicable: 848 1) An application has to decide whether -- for the current ratio of 849 packet rate with the associated (application-specific) maximum 850 feedback delay and the currently observed round-trip time (if 851 available) -- feedback mechanisms can be applied at all. 853 This decision may obviously be based upon (and dynamically 854 revised following) regular RTCP reception statistics as well as 855 out-of-band mechanisms. 857 2) The application has to decide -- for a certain observed error 858 rate, assigned bandwidth, frame/packet rate, and group size -- 859 whether (and which) feedback mechanisms can be applied. 861 Regular RTCP provides valuable input to this step, too. 863 3) If these tests pass, the application has to follow the rules for 864 transmitting Early RTCP packets or regularly scheduled RTCP 865 packets with piggybacked feedback. 867 3.7.2 Media Session Attributes 869 Media sessions are typically described using out-of-band mechanisms 870 to convey transport addresses, codec information, etc. between 871 sender(s) and receiver(s). Such a mechanisms consists of a format 872 used to describe a media session and another mechanism for 873 transporting this description. 875 In the IETF, the Session Description Protocol (SDP) is currently 876 used to describe media sessions while protocols such as SIP, SAP, 877 RTSP, and HTTP (among others) are used to convey the descriptions. 879 A media session description format MAY include parameters to 880 indicate that RTCP feedback mechanisms MAY be used (=are supported) 881 in this session and which of the feedback mechanisms MAY be 882 applied. 884 To do so, the profile "AVPF" MUST be indicated instead of "AVP". 885 Further attributes may be defined to show which type(s) of feedback 886 are supported. 888 Section 4 contains the syntax specification to support RTCP 889 feedback with SDP. Similar specifications for other media session 890 description formats are outside the scope of this document. 892 4. SDP Definitions 894 This section defines a number of additional SDP parameters that are 895 used to describe a session. All of these are defined as media 896 level attributes. 898 4.1 Profile identification 900 The AV profile defined in [4] is referred to as "AVP" in the 901 context of e.g. the Session Description Protocol (SDP) [3]. The 902 profile specified in this document is referred to as "AVPF". 904 Feedback information following the modified timing rules as 905 specified in this document MUST NOT be sent for a particular media 906 session unless the profile for this session indicates the use of 907 the "AVPF" profile (exclusively or jointly with other AV profiles). 909 4.2 RTCP Feedback Capability Attribute 911 A new payload format-specific SDP attribute (for use with 912 "a=fmtp:") is defined to indicate the capability of using RTCP 913 feedback as specified in this document: "rtcp-fb". The "rtcp-fb" 914 attribute MUST only be used as an SDP media attribute and MUST NOT 915 be provided at the session level. The "rtcp-fb" attribute MUST 916 only be used in media sessions for which the "AVPF" is specified. 918 The "rtcp-fb" attribute SHOULD be used to indicate which RTCP 919 feedback messages MAY be used in this media session for the 920 indicated payload type. If several types of feedback are 921 supported, several "a=rtcp-fb:" lines MUST be used. 923 If no "rtcp-fb" attribute is specified the RTP receivers SHOULD 924 assume that the RTP senders only support generic NACKs. In 925 addition, the RTP receivers MAY send feedback using other suitable 926 RTCP feedback packets as defined for the respective media type. 927 The RTP receivers MUST NOT rely on the RTP senders reacting to any 928 of the feedback messages. 930 If one or more "rtcp-fb" attributes are present in a media session 931 description, the RTCP receivers for the media session(s) containing 932 the "rtcp-fb" 934 o MUST ignore all "rtcp-fb" attributes of which they do not fully 935 understand the semantics (i.e. where they do not understand the 936 meaning of all values in the a=fmtp:rtcp-fb line); 938 o SHOULD provide feedback information as specified in this 939 document using any of the RTCP feedback packets as specified in 940 one of the "rtcp-fb" attributes for this media session; and 942 o MUST NOT use other feedback messages than those listed in one of 943 the "rtcp-fb" attribute lines. 945 When used in conjunction with the offer/answer model [18], the 946 offerer MAY present a set of these AVPF attributes to its peer. 947 The answerer MUST remove all attributes it does not understand as 948 well as those it does not support in general or does not wish to 949 use in this particular media session. The answerer MUST NOT add 950 feedback parameters to the media description and MUST NOT alter 951 values of such parameters. The answer is binding for the media 952 session and both offerer and answerer MUST only use feedback 953 mechanisms negotiated in this way. 955 RTP senders MUST be prepared to receive any kind of RTCP feedback 956 messages and MUST silently discard all those RTCP feedback messages 957 that they do not understand. 959 The syntax of the "rtcp-fb" attribute is as follows (the feedback 960 types and optional parameters are all case sensitive): 962 (In the following ABNF, SP and CRLF are used as defined in [3].) 963 rtcp-fb-syntax = "a=fmtp:" SP "rtcp-fb" SP rtcp-fb-val 964 CRLF 966 rtcp-fb-val = "ack" rtcp-fb-ack-param 967 | "nack" rtcp-fb-nack-param 968 | "trr-int" SP 1*DIGIT 969 | rtcp-fb-id rtcp-fb-param 971 rtcp-fb-id = 1*(alpha-numeric | "-" | "_") 973 rtcp-fb-param = SP "app" 974 | SP byte-string 975 | ; empty 977 rtcp-fb-ack-param = SP "rpsi" 978 | SP "app" 979 | SP byte-string 980 | ; empty 982 rtcp-fb-nack-param = SP "pli" 983 | SP "sli" 984 | SP "rpsi" 985 | SP "app" 986 | SP byte-string 987 | ; empty 989 The literals of the above grammar have the following semantics: 991 Feedback type "ack": 993 This feedback type indicates that positive acknowledgements 994 for feedback are supported. 996 The feedback type "ack" MUST only be used if the media session 997 is allowed to operate in ACK mode as defined in 3.6.1.2. 999 Parameters MAY be provided to further distinguish different 1000 types of positive acknowledgement feedback. If no parameters 1001 are present, the Generic ACK as specified in section 6.2.2 is 1002 implied. 1004 The parameter "rpsi" indicates the use of Reference Picture 1005 Selection Indication feedback as defined in section 6.3.3. 1007 If the parameter "app" is specified, this indicates the use of 1008 application layer feedback. In this case, additional 1009 parameters following "app" MAY be used to further 1010 differentiate various types of application layer feedback. 1011 This document does not define any parameters specific to 1012 "app". 1014 Further parameters for "ack" MAY be defined in other 1015 documents. 1017 Feedback type "nack": 1019 This feedback type indicates that negative acknowledgements 1020 for feedback are supported. 1022 The feedback type "nack", without parameters, indicates use of 1023 the General NACK feedback format as defined in section 6.2.1. 1025 The following three parameters are defined in this document 1026 for use with "nack" in conjunction with the media type 1027 "video": 1029 o "pli" indicates the use of Picture Loss Indication feedback 1030 as defined in section 6.3.1. 1031 o "sli" indicates the use of Slice Loss Indication feedback 1032 as defined in section 6.3.2. 1033 o "rpsi" indicates the use of Reference Picture Selection 1034 Indication feedback as defined in section 6.3.3. 1036 "app" indicates the use of application layer feedback. 1037 Additional parameters after "app" MAY be provided to 1038 differentiate different types of application layer feedback. 1039 No parameters specific to "app" are defined in this document. 1041 Further parameters for "nack" MAY be defined in other 1042 documents. 1044 Other feedback types : 1046 Other documents MAY define additional types of feedback; to 1047 keep the grammar extensible for those cases, the rtcp-fb-id is 1048 introduced as a placeholder. A new feedback scheme name MUST 1049 to be unique (and thus MUST be registered with IANA). Along 1050 with a new name, its semantics, packet formats (if necessary), 1051 and rules for its operation MUST be specified. 1053 Regular RTCP minimum interval "trr-int": 1055 The attribute "trr-int" is used to specify the minimum 1056 interval T_rr_interval between two regular (full compound) 1057 RTCP packets in milliseconds for this media session. If "trr- 1058 int" is not specified, a default value of 0 is assumed. 1060 Note that it is assumed that more specific information about 1061 application layer feedback (as defined in section 6.4) will be 1062 conveyed as feedback types and parameters defined elsewhere. 1063 Hence, no further provision for any types and parameters is made in 1064 this document. 1066 Further types of feedback as well as further parameters may be 1067 defined in other documents. 1069 It is up to the recipients whether or not they send feedback 1070 information and up to the sender(s) to make use of feedback 1071 provided. 1073 4.3 Unicasting vs. Multicasting 1075 If a media session description indicates unicast addresses for a 1076 particular media type (and does not operate in multi-unicast mode 1077 with all recipients listed explicitly but still addressed via 1078 unicast), the RTCP feedback MAY operate in ACK feedback mode. 1080 If a media session description indicates multicast addresses for a 1081 particular media type or a multi-unicast session, ACK feedback mode 1082 MUST NOT be used. 1084 4.4 RTCP Bandwidth Modifiers 1086 The standard RTCP bandwidth assignments as defined in [1] and [2] 1087 may be overridden by bandwidth modifiers that explicitly define the 1088 maximum RTCP bandwidth. For use with SDP, such modifiers are 1089 specified in [4]: "b=RS:" and "b=RR:" MAY be used to assign 1090 a different bandwidth (measured in bits per second) to RTP senders 1091 and receivers, respectively. The precedence rules of [4] apply to 1092 determine the actual bandwidth to be used by senders and receivers. 1094 Applications operating knowingly over highly asymmetric links (such 1095 as satellite links) SHOULD use this mechanism to reduce the 1096 feedback rate for high bandwidth streams to prevent deterministic 1097 congestion of the feedback path(s). 1099 4.5 Examples 1101 Example 1: The following session description indicates a session 1102 made up from an audio and a DTMF for point-to-point communication 1103 in which the DTMF stream uses Generic ACKs. This session 1104 description could be contained in a SIP INVITE, 200 OK, or ACK 1105 message to indicate that its sender is capable of and willing to 1106 receive feedback for the DTMF stream it transmits. 1108 v=0 1109 o=alice 3203093520 3203093520 IN IP4 host.example.com 1110 s=Media with feedback 1111 t=0 0 1112 c=IN IP4 host.example.com 1113 m=audio 49170 RTP/AVPF 0 96 1114 a=rtpmap:0 PCMU/8000 1115 a=rtpmap:96 telephone-event/8000 1116 a=fmtp:96 0-16 1117 a=fmtp:96 rtcp-fb ack 1119 Example 2: The following session description indicates a multicast 1120 video-only session (using H.263+) with the video source accepting 1121 Generic NACKs and Reference Picture Selection. Such a description 1122 may have been conveyed using the Session Announcement Protocol 1123 (SAP). 1125 v=0 1126 o=alice 3203093520 3203093520 IN IP4 host.example.com 1127 s=Multicast video with feedback 1128 t=3203130148 3203137348 1129 m=audio 49170 RTP/AVP 0 1130 c=IN IP4 224.2.1.183 1131 a=rtpmap:0 PCMU/8000 1132 m=video 51372 RTP/AVPF 98 1133 c=IN IP4 224.2.1.184 1134 a=rtpmap:98 H263-1998/90000 1135 a=fmtp:98 rtcp-fb nack 1136 a=fmtp:98 rtcp-fb nack rpsi 1138 Example 3: The following session description defines the same media 1139 session as example 2 but allows for mixed mode operation of AVP and 1140 AVPF RTP entities (see also next section). Note that both media 1141 descriptions use the same addresses; however, two m= lines are 1142 needed to convey information about both applicable RTP profiles. 1144 v=0 1145 o=alice 3203093520 3203093520 IN IP4 host.example.com 1146 s=Multicast video with feedback 1147 t=3203130148 3203137348 1148 m=audio 49170 RTP/AVP 0 1149 c=IN IP4 224.2.1.183 1150 a=rtpmap:0 PCMU/8000 1151 m=video 51372 RTP/AVP 98 1152 c=IN IP4 224.2.1.184 1153 a=rtpmap:98 H263-1998/90000 1154 m=video 51372 RTP/AVPF 98 1155 c=IN IP4 224.2.1.184 1156 a=rtpmap:98 H263-1998/90000 1157 a=fmtp:98 rtcp-fb nack 1158 a=fmtp:98 rtcp-fb nack rpsi 1160 Note that these two m= lines SHOULD be grouped by some appropriate 1161 mechanisms to indicate that both are alternatives actually 1162 conveying the same contents. A sample mechanism by which this can 1163 be achieved is defined in [14]. 1165 5. Interworking and Co-Existence of AVP and AVPF Entities 1167 The AVPF profile defined in this document is an extension of the 1168 AVP profile as defined in [2]. Both profiles follow the same basic 1169 rules (including the upper bandwidth limit for RTCP and the 1170 bandwidth assignments to senders and receivers). Therefore, 1171 senders and receivers of using either of the two profiles can be 1172 mixed in a single session (see e.g. example 3 in section 4.5). 1174 AVP and AVPF are defined in a way that, from a robustness point of 1175 view, the RTP entities do not need to be aware of entities of the 1176 respective other profile: they will not disturb each other's 1177 functioning. However, the quality of the media presented may 1178 suffer. 1180 The following considerations apply to senders and receivers when 1181 used in a combined session. 1183 o AVP entities (senders and receivers) 1185 AVP senders will receive RTCP feedback packets from AVPF 1186 receivers and ignore these packets. They will see occasional 1187 closer spacing of RTCP messages (e.g. violating the five second 1188 rule) by AVPF entities. As the overall bandwidth constraints 1189 are adhered to by both types of entities, they will still get 1190 their share of the RTCP bandwidth. However, while AVP entities 1191 are bound by the five second rule, depending on the group size 1192 and session bandwidth, AVPF entities may provide more frequent 1193 RTCP reports than AVP ones will. Also, the overall reporting 1194 may decrease slightly as AVPF entities may send bigger compound 1195 RTCP packets (due to the extra RTCP packets). 1197 If T_rr_interval is used as lower bound between regular RTCP 1198 packets, T_rr_interval is sufficiently large (e.g. T_rr_interval 1199 > M*Td as per section 6.3.5 of [1]), and no Early RTCP packets 1200 are sent by AVPF entities, AVP entities MAY accidentally time 1201 out those AVPF group members and hence under-estimate the group 1202 size. Therefore, if AVP entities may be involved in a media 1203 session, T_rr_interval SHOULD NOT be used. 1205 o AVPF senders 1207 AVPF senders will receive feedback information only from AVPF 1208 receivers. If they rely on feedback to provide the target media 1209 quality, the quality achieved for AVP receivers may be sub- 1210 optimal. 1212 o AVPF receivers 1214 AVPF receivers SHOULD send immediate or early RTCP feedback 1215 packets only if all (sending) entities in the media session 1216 support AVPF. AVPF receivers MAY send feedback information as 1217 part of regularly scheduled compound RTCP packets following the 1218 timing rules of [1] and [2] also in media sessions operating in 1219 mixed mode. However, the receiver providing feedback MUST NOT 1220 rely on the sender reacting to the feedback at all. 1222 6. Format of RTCP Feedback Messages 1224 This section defines the format of the low delay RTCP feedback 1225 messages. These messages classified into three categories as 1226 follows: 1228 - Transport layer feedback messages 1229 - Payload-specific feedback messages 1230 - Application layer feedback messages 1231 Transport layer feedback messages are intended to transmit general 1232 purpose feedback information, i.e. information independent of the 1233 particular codec or the application in use. The information is 1234 expected to be generated and processed at the transport/RTP layer. 1235 Currently, only a general positive acknowledgement (ACK) and a 1236 negative acknowledgement (NACK) message are defined. 1238 Payload-specific feedback messages transport information that is 1239 specific to a certain payload type and will be generated and acted 1240 upon at the codec "layer". This document defines a common header 1241 to be used in conjunction with all payload-specific feedback 1242 messages. The definition of specific messages is left to either 1243 RTP payload format specifications or to additional feedback format 1244 documents. 1246 Application layer feedback messages provide a means to 1247 transparently convey feedback from the receiver's to the sender's 1248 application. The information contained in such a message is not 1249 expected to be acted upon at the transport/RTP or the codec layer. 1250 The data to be exchanged between two application instances is 1251 usually defined in the application protocol's specification and 1252 thus can be identified by the application so that there is no need 1253 for additional external information. Hence, this document defines 1254 only a common header to be used along with all application layer 1255 feedback messages. From a protocol point of view, an application 1256 layer feedback message is treated as a special case of a payload- 1257 specific feedback message. 1259 This document defines two transport layer feedback and three 1260 (video) payload-specific feedback messages as well as a single 1261 container for application layer feedback messages. Additional 1262 transport layer and payload specific feedback messages MAY be 1263 defined in other documents and MUST be registered through IANA (see 1264 section IANA considerations). 1266 The general syntax and semantics for the above RTCP feedback 1267 message types are described in the following subsections. 1269 6.1 Common Packet Format for Feedback Message 1271 All feedback message MUST use a common packet format that is 1272 depicted in figure 3: 1274 0 1 2 3 1275 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1276 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1277 |V=2|P|0| FMT | PT | length | 1278 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1279 | SSRC of packet sender | 1280 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1281 | SSRC of media source | 1282 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1283 : Feedback Control Information (FCI) : 1284 : : 1286 Figure 3: Common Packet Format for Feedback Messages 1288 The various fields V, P, SSRC and length are defined in the RTP 1289 specification [2], the respective meaning being summarized below: 1291 version (V): 2 bits 1292 This field identifies the RTP version. The current version is 1293 2. 1295 padding (P): 1 bit 1296 If set, the padding bit indicates that the packet contains 1297 additional padding octets at the end which are not part of the 1298 control information but are included in the length field. 1300 Feedback message type (FMT): 4 bits 1301 This field identifies the type of the feedback message and is 1302 interpreted relative to the RTCP message type (transport, 1303 payload-specific, or application layer feedback). The values 1304 for each of the three feedback types are defined in the 1305 respective sections below. 1307 Payload type (PT): 8 bits 1308 This is the RTCP packet type which identifies the packet as 1309 being an RTCP Feedback Message. Two values are defined (TBA. 1310 by IANA): 1312 Name | Value | Brief Description 1313 ----------+-------+------------------------------------ 1314 RTPFB | 205 | Transport layer feedback message 1315 PSFB | 206 | Payload-specific feedback message 1317 Length: 16 bits 1318 The length of this packet in 32-bit words minus one, including 1319 the header and any padding. This is in line with the 1320 definition of the length field used in RTCP sender and receiver 1321 reports [3]. 1323 SSRC of packet sender: 32 bits 1324 The synchronization source identifier for the originator of 1325 this packet. 1327 SSRC of media source: 32 bits 1328 The synchronization source identifier of the media source that 1329 this piece of feedback information is related to. 1331 Feedback Control Information (FCI): variable length 1332 The following three sections define which additional 1333 information MAY be included in the feedback message for each 1334 type of feedback (further FCI contents MAY be specified in 1335 further documents). Each RTCP feedback packet MUST contain 1336 exactly one FCI field of the types defined in sections 6.2 and 1337 6.3. If multiple FCI fields (even of the same type) need to be 1338 conveyed, then several RTCP feedback packets MUST be generated 1339 and concatenated in the same compound RTCP packet. 1341 6.2 Transport Layer Feedback Messages 1343 Transport Layer Feedback messages are identified by the value RTPFB 1344 as RTCP message type. 1346 Two general purpose transport layer feedback messages are defined 1347 so far: General ACK and General NACK. They are identified by means 1348 of the FMT parameter as follows: 1350 0: reserved 1351 1: Generic NACK 1352 2: Generic ACK 1353 3-15: reserved 1355 The following two subsections define the packet formats for these 1356 messages. 1358 6.2.1 Generic NACK 1360 The Generic NACK message is identified by PT=RTPFB and FMT=1. 1362 The Generic NACK packet is used to indicate the loss of one or more 1363 RTP packets. The lost packet(s) are identified by the means of a 1364 packet identifier and a bit mask. 1366 The Feedback control information (FCI) field has the following 1367 Syntax (figure 4): 1369 0 1 2 3 1370 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1371 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1372 | PID | BLP | 1373 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1375 Figure 4: Syntax for the Generic NACK message 1377 Packet ID (PID): 16 bits 1378 The PID field is used to specify a lost packet. Typically, the 1379 RTP sequence number is used for PID as the default format, but 1380 RTP Payload Formats may decide to identify a packet 1381 differently. 1383 bitmask of following lost packets (BLP): 16 bits 1384 The BLP allows for reporting losses of any of the 16 RTP 1385 packets immediately following the RTP packet indicated by the 1386 PID. The BLP's definition is identical to that given in [10]. 1387 Denoting the BLP's least significant bit as bit 1, and its most 1388 significant bit as bit 16, then bit i of the bit mask is set to 1389 1 if the receiver has not received RTP packet number PID+i 1390 (modulo 2^16) and indicates this packet is lost; bit i is set 1391 to 0 otherwise. Note that the sender MUST NOT assume that a 1392 receiver has received a packet because its bit mask was set to 1393 0. For example, the least significant bit of the BLP would be 1394 set to 1 if the packet corresponding to the PID and the 1395 following packet have been lost. However, the sender cannot 1396 infer that packets PID+2 through PID+16 have been received 1397 simply because bits 2 through 15 of the BLP are 0; all the 1398 sender knows is that the receiver has not reported them as lost 1399 at this time. 1401 6.2.2 Generic ACK 1403 The Generic ACK message is identified by PT=RTPFB and FMT=2. 1405 The Generic ACK packet is used to indicate that one or several RTP 1406 packets were received correctly. The received packet(s) are 1407 identified by the means of a packet identifier and a bit mask. 1408 ACKing of a range of consecutive packets is also possible. 1410 The Feedback control information (FCI) field has the following 1411 syntax: 1413 0 1 2 3 1414 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1415 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1416 | PID |R| BLP/#packets | 1417 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1419 Figure 5: Syntax for the Generic ACK message 1421 Packet ID (1st PID): 16 bits 1422 This PID field is used to specify a correctly received packet. 1423 Typically, the RTP sequence number is used for PID as the 1424 default format, but RTP Payload Formats may decide to identify 1425 a packet differently. 1427 Range of ACKs (R): 1 bit 1428 The R-bit indicates that a range of consecutive packets are 1429 received correctly. If R=1 then the PID field specifies the 1430 first packet of that range and the next field (BLP/#packets) 1431 will carry the number of packets being acknowledged. If R=0 1432 then PID specifies the first packet to be acknowledged and 1433 BLP/#packets provides a bit mask to selectively indicate 1434 individual packets that are acknowledged. 1436 Bit mask of lost packets (BLP)/#packets (PID): 15 bits 1437 The semantics of this field depends on the value of the R-bit. 1439 If R=1, this field is used to identify the number of additional 1440 packets of to be acknowledged: 1442 #packets = - 1444 That is, #packets MUST indicate the number of packet to be 1445 ACKed minus one. In particular, if only a single packet is to 1446 be ACKed and R=1 then #packets MUST be set to 0x0000. 1448 Example: If all packets between and including PIDx = 380 and 1449 PIDy = 422 have been received, the Generic ACK would contain 1450 PID = PIDx = 380 and #packets = PIDy - PID = 42. In case the 1451 PID wraps around, modulo arithmetic is used to calculate the 1452 number of packets. 1454 If R=0, this field carries a bit mask. The BLP allows for 1455 reporting reception of any of the 15 RTP packets immediately 1456 following the RTP packet indicated by the PID. The BLP's 1457 definition is identical to that given in [10] except that, 1458 here, BLP is only 15 bits wide. Denoting the BLP's least 1459 significant bit as bit 1, and its most significant bit as bit 1460 15, then bit i of the bitmask is set to 1 if the receiver has 1461 received RTP packet number PID+i (modulo 2^16) and decides to 1462 ACK this packet; bit i is set to 0 otherwise. If only the 1463 packet indicated by PID is to be ACKed and R=0 then BLP MUST be 1464 set to 0x0000. 1466 6.3 Payload Specific Feedback Messages 1468 Payload-Specific Feedback Messages are identified by the value 1469 PT=PSFB as RTCP message type. 1471 Three payload-specific feedback messages are defined so far plus an 1472 application layer feedback message. They are identified by means 1473 of the FMT parameter as follows: 1475 0: reserved 1476 1: Picture Loss Indication (PLI) 1477 2: Slice Lost Indication (SLI) 1478 3: Reference Picture Selection Indication (RPSI) 1479 4-14: reserved 1480 15: Application layer feedback message 1482 The following subsections define the packet formats for the 1483 payload-specific messages, section 6.4 defines the application 1484 layer feedback message. 1486 6.3.1 Picture Loss Indication (PLI) 1488 The PLI feedback message is identified by PT=PSFB and FMT=1. 1490 6.3.1.1 Semantics 1492 With the Picture Loss Indication message, a decoder informs the 1493 encoder about the loss of an undefined amount of coded video data 1494 belonging to one or more pictures. When used in conjunction with 1495 any video coding scheme that is based on inter-picture prediction, 1496 an encoder that receives a PLI becomes aware that the prediction 1497 chain may be broken. The sender MAY react to a PLI by transmitting 1498 an intra-picture to achieve resynchronization (making effectively 1499 similar to the FIR as defined in [10]); however, the sender MUST 1500 consider congestion control as outlined in section 7 which MAY 1501 restrict its ability to send an intra frame. 1503 Other RTP payload specifications such as RFC 2032 [10] already 1504 define a feedback mechanism for some for certain codecs. An 1505 application supporting both schemes MUST use the feedback mechanism 1506 defined in this specification when sending feedback. For backward 1507 compatibility reasons, such an application SHOULD also be capable 1508 to receive and react to the feedback scheme defined in the 1509 respective RTP payload format, if this is required by that payload 1510 format. 1512 6.3.1.2 Message Format 1514 PLI does not require parameters. Therefore, the length field MUST 1515 be 2, and there MUST NOT be any Feedback Control Information. 1517 6.3.1.3 Timing Rules 1519 The timing follows the rules outlined in section 3. In systems 1520 that employ both PLI and other types of feedback it may be 1521 advisable to follow the regular RTCP RR timing rules for PLI, since 1522 PLI is not as delay critical as other FB types. 1524 6.3.1.4 Remarks 1526 PLI messages typically trigger the sending of full intra pictures. 1527 Intra pictures are several times larger then predicted (inter) 1528 pictures. Their size is independent of the time they are 1529 generated. In most environments, especially when employing 1530 bandwidth-limited links, the use of an intra picture implies an 1531 allowed delay that is a significant multitude of the typical frame 1532 duration. An example: If the sending frame rate is 10 fps, and an 1533 intra picture is assumed to be 10 times as big as an inter picture, 1534 then a full second of latency has to be accepted. In such an 1535 environment there is no need for a particular short delay in 1536 sending the feedback message. Hence waiting for the next possible 1537 time slot allowed by RTCP timing rules as per [2] does not have a 1538 negative impact on the system performance. 1540 6.3.2 Slice Lost Indication (SLI) 1542 The SLI feedback message is identified by PT=PSFB and FMT=2. 1544 6.3.2.1 Semantics 1546 With the Slice Lost Indication a decoder can inform an encoder that 1547 it has detected the loss or corruption of one or several 1548 consecutive macroblock(s) in scan order (see below). This feedback 1549 message MUST NOT be used for video codecs with non-uniform, 1550 dynamically changeable macroblock sizes such as H.263 with enabled 1551 Annex Q. In such a case, an encoder cannot always identify the 1552 corrupted spatial region. 1554 6.3.2.2 Format 1556 The Slice Lost Indication uses one additional PCI field the 1557 content of which is depicted in figure 6. The length of the 1558 feedback message MUST be set to 3. 1560 0 1 2 3 1561 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1562 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1563 | First | Number | PictureID | 1564 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1566 Figure 6: Syntax of the Slice Lost Indication (SLI) 1568 First: 13 bits 1569 The macroblock (MB) address of the first lost macroblock. The 1570 MB numbering is done such that the macroblock in the upper left 1571 corner of the picture is considered macroblock number 1 and the 1572 number for each macroblock increases from left to right and 1573 then from top to bottom in raster-scan order (such that if 1574 there is a total of N macroblocks in a picture, the bottom 1575 right macroblock is considered macroblock number N). 1577 Number: 13 bits 1578 The number of lost macroblocks, in scan order as discussed 1579 above. 1581 PictureID: 6 bits 1582 The six least significant bits of the a codec-specific 1583 identifier that is used to reference the picture in which the 1584 loss of the macroblock (s) has occurred. For many video 1585 codecs, the PictureID is identical to the Temporal Reference.. 1587 6.3.2.3 Timing Rules 1589 The efficiency of algorithms using the Slice Lost Indication is 1590 reduced greatly when the Indication is not transmitted in a timely 1591 fashion. Motion compensation propagates corrupted pixels that are 1592 not reported as being corrupted. Therefore, the use of the 1593 algorithm discussed in section 3 is highly recommended. 1595 6.3.2.4 Remarks 1597 The term Slice is defined and used here in the sense of MPEG-1 -- a 1598 consecutive number of macroblocks in scan order. More recent video 1599 coding standards sometimes have a different understanding of the 1600 term Slice. In H.263 (1998), for example, a concept known as 1601 "rectangular Slice" exist. The loss of one Rectangular Slice may 1602 lead to the necessity of sending more than one SLI in order to 1603 precisely identify the region of lost/damaged MBs. 1605 The first field of the FCI defines the first macroblock of a 1606 picture as 1 and not, as one could suspect, as 0. This was done to 1607 align this specification with the comparable mechanism available in 1608 H.245. The maximum number of macroblocks in a picture (2**13 or 1609 8192) corresponds to the maximum picture sizes of most of the ITU-T 1610 and ISO/IEC video codecs. If future video codecs offer larger 1611 picture sizes and/or smaller macroblock sizes, then an additional 1612 feedback message has to be defined. The six least significant bits 1613 of the Temporal Reference field are deemed to be sufficient to 1614 indicate the picture in which the loss occurred. 1616 The reaction to a SLI is not part of this specification. One 1617 typical way of reacting to a SLI is to use intra refresh for the 1618 affected spatial region. 1620 Algorithms were reported that keep track of the regions affected by 1621 motion compensation, in order to allow for a transmission of Intra 1622 macroblocks to all those areas, regardless of the timing of the FB 1623 (see H.263 (2000) Appendix I [13] and [15]). While, when those 1624 algorithms are used, the timing of the FB is less critical then 1625 without, it has to be observed that those algorithms correct large 1626 parts of the picture and, therefore, have to transmit much higher 1627 data volume in case of delayed FBs. 1629 6.3.3 Reference Picture Selection Indication (RPSI) 1631 The RPSI feedback message is identified by PT=PSFB and FMT=3. 1633 6.3.3.1 Semantics 1635 Modern video coding standards such as MPEG-4 visual version 2 [12] 1636 or H.263 version 2 [13] allow to use older reference pictures than 1637 the most recent one for predictive coding. Typically, a first-in- 1638 first-out queue of reference pictures is maintained. If an encoder 1639 has learned about a loss of encoder-decoder synchronicity, a known- 1640 as-correct reference picture can be used. As this reference picture 1641 is temporally further away then usual, the resulting predictively 1642 coded picture will use more bits. 1644 Both MPEG-4 and H.263 define a binary format for the "payload" of 1645 an RPSI message that includes information such as the temporal ID 1646 of the damaged picture and the size of the damaged region. This 1647 bit string is typically small -- a couple of dozen bits --, of 1648 variable length, and self-contained, i.e. contains all information 1649 that is necessary to perform reference picture selection. 1651 Note that both MPEG-4 and H.263 allow the use of RPSI with positive 1652 feedback information as well. That is, pictures (or Slices) are 1653 reported that were decoded without error. Note that any form of 1654 positive feedback MUST NOT be used when in a multicast environment 1655 (reporting positive feedback about individual reference pictures at 1656 RTCP intervals is not expected to be of much use anyway). 1658 6.3.3.2 Format 1660 The FCI for the RPSI message follows the format depicted in figure 1661 7: 1663 0 1 2 3 1664 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1666 | PB | Native RPSI bit string defined per codec | 1667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1668 | ... | Padding (0)| 1669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1671 Figure 7: Syntax of the Reference Picture Selection Indication 1672 (RPSI) 1674 PB: 8 bits 1675 The number of unused bits required to pad the length of the 1676 RPSI message to a multiple of 32 bits. 1678 Native RPSI bit string: variable length 1679 The RPSI information as natively defined by the video codec. 1681 Padding: #PB bits 1682 A number of bits set to zero to fill up the contents of the 1683 RPSI message to the next 32 bit boundary. The number of 1684 padding bits MUST be indicated by the PB field. 1686 6.3.3.3 Timing Rules 1688 RPS is even more critical to delay then algorithms using SLI. This 1689 is due to the fact that the older the RPS message is, the more bits 1690 the encoder has to spend to re-establish encoder-decoder 1691 synchronicity. See [15] for some information about the overhead of 1692 RPS for certain bit rate/frame rate/loss rate scenarios. 1694 Therefore, RPS messages should typically be sent as soon as 1695 possible, employing the algorithm of section 3. 1697 6.4 Application Layer Feedback Messages 1699 Application Layer Feedback Messages are a special case of payload- 1700 specific messages and identified by PT=PSFB and FMT=15. 1702 These messages are used to transport application defined data 1703 directly from the receiver's to the sender's application. The data 1704 that is transported is not identified by the feedback message. 1705 Therefore, the application MUST be able to identify the messages 1706 payload. 1708 Usually, applications define their own set of messages, e.g. 1709 NEWPRED messages in MPEG-4 or feedback messages in H.263/Annex N, 1710 U. These messages do not need any additional information from the 1711 RTCP message. Thus the application message is simply placed into 1712 the FCI field as follows and the length field is set accordingly. 1714 Application Message (FCI): variable length 1715 This field contains the original application message that 1716 should be transported from the receiver to the source. The 1717 format is application dependent. The length of this field is 1718 variable. If the application data is not byte aligned, padding 1719 bits must be added. Identification of padding bits is up to 1720 the application layer and not defined in this specification. 1722 7. Early Feedback and Congestion Control 1724 In the previous sections, the feedback messages were defined as 1725 well as the timing rules according to which to send these messages. 1726 The way to react to the feedback received depends on the 1727 application using the feedback mechanisms and hence is beyond the 1728 scope of this document. 1730 However, across all applications, there is a common requirement for 1731 (TCP-friendly) congestion control on the media stream as defined in 1732 [1] and [2] when operating in a best-effort network environment. 1734 Low delay feedback supports the use of congestion control 1735 algorithms in two ways: 1737 o The potentially more frequent RTCP messages allow the sender to 1738 monitor the network state more closely than with regular RTCP 1739 and therefore enable reacting to upcoming congestion in a more 1740 timely fashion. 1742 o The feedback messages themselves may convey additional 1743 information as input to congestion control algorithms and thus 1744 improve reaction over conventional RTCP. (For example, ACK-based 1745 feedback may even allow to construct closed loop algorithms and 1746 NACK-based systems may provide further information on the packet 1747 loss distribution.) 1749 A congestion control algorithm that shares the available bandwidth 1750 fair with competing TCP connections, e.g. TFRC [16], SHOULD be used 1751 to determine the data rate for the media stream (if the low delay 1752 RTP session is transmitted in a best effort environment). 1754 RTCP feedback messages or RTCP SR/RR packets that indicate recent 1755 packet loss MUST NOT lead to a (mid-term) increase in the 1756 transmission data rate and SHOULD lead to a (short-term) decrease 1757 of the transmission data rate. Such messages SHOULD cause the 1758 sender to adjust the transmission data rate to the order of the 1759 throughput TCP would achieve under similar conditions (e.g. using 1760 TFRC). 1762 RTCP feedback messages or RTCP SR/RR packets that indicate no 1763 recent packet loss MAY cause the sender to increase the 1764 transmission data rate to roughly the throughput TCP would achieve 1765 under similar conditions (e.g. using TFRC). 1767 8. Security Considerations 1769 RTP packets transporting information with the proposed payload 1770 format are subject to the security considerations discussed in the 1771 RTP specification [1] and in the RTP/AVP profile specification [2]. 1772 This profile does not specify any additional security services. 1774 This profile modifies the timing behavior of RTCP and eliminates 1775 the minimum RTCP interval of five seconds and allows for earlier 1776 feedback to be provided by receivers. Group members of the 1777 associated RTP session (possibly pretending to represent a large 1778 number of entities) may disturb the operation of RTCP by sending 1779 large numbers of RTCP packets thereby reducing the RTCP bandwidth 1780 available for regular RTCP reporting as well as for early feedback 1781 messages. (Note that an entity need not be member of a multicast 1782 group to cause these effects.) 1784 Feedback information may be suppressed if unknown RTCP feedback 1785 packets are received. This introduces the risk of a malicious 1786 group member reducing early feedback by simply transmitting 1787 payload-specific RTCP feedback packets with random contents that 1788 are neither recognized by any receiver (so they will suppress 1789 feedback) nor by the sender (so no repair actions will be taken). 1791 A malicious group member can also report arbitrary high loss rates 1792 in the feedback information to make the sender throttle the data 1793 transmission and increase the amount of redundancy information or 1794 take other action to deal with the pretended packet loss (e.g. send 1795 fewer frames or decrease audio/video quality). This may result in 1796 a degradation of the quality of the reproduced media stream. 1798 Finally, a malicious group member can act as a large number of 1799 group members and thereby obtain an artificially large share of the 1800 early feedback bandwidth and reduce the reactivity of the other 1801 group members -- possibly even causing them to no longer operate in 1802 immediate or early feedback mode and thus undermining the whole 1803 purpose of this profile. 1805 Senders as well as receivers SHOULD behave conservative when 1806 observing strange reporting behavior. For excessive failure 1807 reporting from one or a few receivers, the sender MAY decide to no 1808 longer consider this feedback when adapting its transmission 1809 behavior for the media stream. In any case, senders and receivers 1810 SHOULD still adhere to the maximum RTCP bandwidth but make sure 1811 that they are capable of transmitting at least regularly scheduled 1812 RTCP packets. Senders SHOULD carefully consider how to adjust 1813 their transmission bandwidth when encountering strange reporting 1814 behavior; they MUST NOT increase their transmission bandwidth even 1815 if ignoring suspicious feedback. 1817 Attacks using false RTCP packets (regular as well as early ones) 1818 can be avoided by authenticating all RTCP messages. This can be 1819 achieved by using the AVPF profile together with the Secure RTP 1820 profile as defined in [17]. 1822 9. IANA Considerations 1824 The feedback profile as an extension to the profile for audio- 1825 visual conferences with minimal control needs to be registered: 1826 "RTP/AVPF". 1828 For the Session Description Protocol, the following "fmtp:" 1829 attribute needs to be registered: "rtcp-fb". 1831 Along with "rtcp-fb", the feedback types "ack" and "nack" need to 1832 be registered. Also, the value "trr-int" needs to be registered. 1834 Along with "nack", the feedback type parameters "sli" and "pli" 1835 need to be registered. 1837 Along with "ack" and "nack", the feedback type parameters "rpsi" 1838 and "app" need to be registered. 1840 Two RTCP Control Packet Types: for the class of transport layer 1841 feedback messages ("RTPFB") and for the class of payload-specific 1842 feedback messages ("PSFB"). Section 6 suggests RTPFB=205 and 1843 PSFB=206 to be added to the RTCP registry. 1845 Within the RTPFB range, three format (FMT) values need to be 1846 registered: 1848 1: General NACK 1849 2: General ACK 1851 Within the PSFB range, five format (FMT) values need to be 1852 registered: 1854 1: Picture Loss Indication (PLI) 1855 2: Slice Loss Indication (SLI) 1856 3: Reference Picture Selection Indication (SLI) 1857 15: Application layer feedback (AFB) 1859 10. Acknowledgements 1861 This document is a product of the Audio-Visual Transport (AVT) 1862 Working Group of the IETF. The authors would like to thank Steve 1863 Casner and Colin Perkins for their comments and suggestions as well 1864 as for their responsiveness to numerous questions. 1866 11. Full Copyright Statement 1868 Copyright (C) The Internet Society (2001). All Rights Reserved. 1869 This document and translations of it may be copied and furnished to 1870 others, and derivative works that comment on or otherwise explain 1871 it or assist in its implementation may be prepared, copied, 1872 published and distributed, in whole or in part, without restriction 1873 of any kind, provided that the above copyright notice and this 1874 paragraph are included on all such copies and derivative works. 1876 However, this document itself may not be modified in any way, such 1877 as by removing the copyright notice or references to the Internet 1878 Society or other Internet organizations, except as needed for the 1879 purpose of developing Internet standards in which case the 1880 procedures for copyrights defined in the Internet Standards process 1881 must be followed, or as required to translate it into languages 1882 other than English. 1884 The limited permissions granted above are perpetual and will not be 1885 revoked by the Internet Society or its successors or assigns. 1887 This document and the information contained herein is provided on 1888 an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 1889 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 1890 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1891 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1892 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." 1894 12. Authors' Addresses 1896 Joerg Ott {sip,mailto}:jo@tzi.org 1897 Uni Bremen TZI 1898 MZH 5180 1899 Bibliothekstr. 1 1900 D-28359 Bremen 1901 Germany 1903 Stephan Wenger stewe@cs.tu-berlin.de 1904 TU Berlin 1905 Sekr. FR 6-3 1906 Franklinstr. 28-29 1907 D-10587 Berlin 1908 Germany 1910 Shigeru Fukunaga 1911 Oki Electric Industry Co., Ltd. 1912 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1913 Tel. +81 6 6949 5101 1914 Fax. +81 6 6949 5108 1915 Mail fukunaga444@oki.com 1917 Noriyuki Sato 1918 Oki Electric Industry Co., Ltd. 1919 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1920 Tel. +81 6 6949 5101 1921 Fax. +81 6 6949 5108 1922 Mail sato652@oki.com 1924 Koichi Yano 1925 FastForward Networks, 1926 75 Hawthorne St. #601 1927 San Francisco, CA 94105 1928 Tel. +1.415.430.2500 1929 Akihiro Miyazaki 1930 Matsushita Electric Industrial Co., Ltd 1931 1006, Kadoma, Kadoma City, Osaka, Japan 1932 Tel. +81-6-6900-9192 1933 Fax. +81-6-6900-9193 1934 Mail akihiro@isl.mei.co.jp 1936 Koichi Hata 1937 Matsushita Electric Industrial Co., Ltd 1938 1006, Kadoma, Kadoma City, Osaka, Japan 1939 Tel. +81-6-6900-9192 1940 Fax. +81-6-6900-9193 1941 Mail hata@isl.mei.co.jp 1943 Rolf Hakenberg 1944 Panasonic European Laboratories GmbH 1945 Monzastr. 4c, 63225 Langen, Germany 1946 Tel. +49-(0)6103-766-162 1947 Fax. +49-(0)6103-766-166 1948 Mail hakenberg@panasonic.de 1950 Carsten Burmeister 1951 Panasonic European Laboratories GmbH 1952 Monzastr. 4c, 63225 Langen, Germany 1953 Tel. +49-(0)6103-766-263 1954 Fax. +49-(0)6103-766-166 1955 Mail burmeister@panasonic.de 1957 11. Bibliography 1959 [1] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP 1960 - A Transport Protocol for Real-time Applications," Internet 1961 Draft, draft-ietf-avt-rtp-new-11.txt, Work in Progress, 1962 November 2001. 1964 [2] H. Schulzrinne and S. Casner, "RTP Profile for Audio and Video 1965 Conferences with Minimal Control," Internet Draft draft-ietf- 1966 avt-profile-new-12.txt, November 2001. 1968 [3] M. Handley, V. Jacobson, and Colin Perkins, "SDP: Session 1969 Description Protocol", Internet Draft draft-ietf-mmusic-sdp- 1970 new-10.txt, May 2002. 1972 [4] S. Casner, "SDP Bandwidth Modifiers for RTCP Bandwidth", 1973 Internet Draft draft-ietf-avt-rtcp-bw-05.txt, November 2001. 1975 [5] C. Perkins and O. Hodson, "2354 Options for Repair of 1976 Streaming Media," RFC 2354, June 1998. 1978 [6] J. Rosenberg and H. Schulzrinne, "An RTP Payload Format for 1979 Generic Forward Error Correction,", RFC 2733, December 1999. 1981 [7] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, 1982 J.C. Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP Payload 1983 for Redundant Audio Data," RFC 2198, September 1997. 1985 [8] S. Bradner, "Key words for use in RFCs to Indicate Requirement 1986 Levels," RFC 2119, March 1997. 1988 [9] H. Schulzrinne and S. Petrack, "RTP Payload for DTMF Digits, 1989 Telephony Tones and Telephony Signals," RFC 2833, May 2000. 1991 [10] T. Turletti and C. Huitema, "RTP Payload Format for H.261 1992 Video Streams, RFC 2032, October 1996. 1994 [11] C. Bormann, L. Cline, G. Deisher, T. Gardos, C. Maciocco, D. 1995 Newell, J. Ott, G. Sullivan, S. Wenger, and C. Zhu, "RTP 1996 Payload Format for the 1998 Version of ITU-T Rec. H.263 Video 1997 (H.263+)," RFC 2429, October 1998. 1999 [12] ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology - 2000 Coding of audio-visual objects - Part2: Visual", July 2000. 2002 [13] ITU-T Recommendation H.263, "Video Coding for Low Bit Rate 2003 Communication," November 2000. 2005 [14] G. Camarillo, J. Holler, G. Eriksson, H. Schulzrinne, 2006 "Grouping of media lines in SDP," Internet Draft, draft-ietf- 2007 mmusic-fid-05.txt, Work in Progress, September 2001. 2009 [15] B. Girod, N. Faerber, "Feedback-based error control for mobile 2010 video transmission," Proceedings IEEE, Vol. 87, No. 10, pp. 2011 1707 - 1723, October, 1999. 2013 [16] M. Handley, J. Padhye, S. Floyd, J. Widmer, "TCP friendly Rate 2014 Control (TFRC): Protocol Specification," Internet Draft, 2015 draft-ietf-tsvwg-tfrc-03.txt, Work in Progress, July 2001. 2017 [17] M. Baugher, R. Blom, E. Carrarra, D. McGrew, M. Naslund, K. 2018 Norrman, D. Oran, "The Secure Real-Time Transport Protocol," 2019 Internet Draft, draft-ietf-avt-srtp-04.txt, Work in Progress, 2020 May 2002. 2022 [18] J. Rosenberg and H. Schulzrinne, "An offer/answer model with 2023 SDP," Internet Draft draft-ietf-mmusic-sdp-offer-answer- 2024 02.txt, February 2002.