idnits 2.17.1 draft-ietf-avt-rtcp-feedback-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 12 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 5 instances of lines with multicast IPv4 addresses in the document. If these are generic example addresses, they should be changed to use the 233.252.0.x range defined in RFC 5771 ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? RFC 2119 keyword, line 213: '...tiple FB messages MAY be combined in a...' RFC 2119 keyword, line 214: '... packet and they MAY also be sent comb...' RFC 2119 keyword, line 218: '... document MUST contain RTCP packets in the order as defined in [1]:...' RFC 2119 keyword, line 220: '... o OPTIONAL encryption prefix that M...' RFC 2119 keyword, line 223: '...ATORY SDES which MUST contain the CNAM...' (171 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 473 has weird spacing: '...This is to en...' == Line 930 has weird spacing: '...tribute is de...' == Line 1809 has weird spacing: '... These messa...' == Line 2175 has weird spacing: '... Mail sato6...' == Line 2182 has weird spacing: '... Mail burme...' == (1 more instance...) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (April 2003) is 7681 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 2193 looks like a reference -- Missing reference section? '10' on line 2225 looks like a reference -- Missing reference section? '7' on line 2215 looks like a reference -- Missing reference section? '11' on line 2228 looks like a reference -- Missing reference section? '5' on line 2209 looks like a reference -- Missing reference section? '6' on line 2212 looks like a reference -- Missing reference section? '2' on line 2198 looks like a reference -- Missing reference section? '8' on line 2219 looks like a reference -- Missing reference section? '3' on line 2202 looks like a reference -- Missing reference section? '4' on line 2206 looks like a reference -- Missing reference section? '18' on line 2256 looks like a reference -- Missing reference section? '14' on line 2239 looks like a reference -- Missing reference section? '13' on line 2236 looks like a reference -- Missing reference section? '15' on line 2243 looks like a reference -- Missing reference section? '12' on line 2233 looks like a reference -- Missing reference section? '16' on line 2247 looks like a reference -- Missing reference section? '17' on line 2251 looks like a reference -- Missing reference section? '9' on line 2222 looks like a reference Summary: 5 errors (**), 0 flaws (~~), 10 warnings (==), 21 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT Joerg Ott/Uni Bremen TZI 3 draft-ietf-avt-rtcp-feedback-04.txt Stephan Wenger/TU Berlin 4 Noriyuki Sato/Oki 5 Carsten Burmeister/Matsushita 6 Jose Rey/Matsushita 8 31 October 2002 9 Expires April 2003 11 Extended RTP Profile for RTCP-based Feedback (RTP/AVPF) 13 Status of this Memo 15 This document is an Internet-Draft and is in full conformance with all 16 provisions of Section 10 of RFC 2026. Internet-Drafts are working 17 documents of the Internet Engineering Task Force (IETF), its areas, 18 and its working groups. Note that other groups may also distribute 19 working documents as Internet-Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet- Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 Abstract 34 Real-time media streams that use RTP are not resilient against 35 packet losses. Receivers may use the base mechanisms of RTCP to 36 report packet reception statistics and thus allow a sender to adapt 37 its transmission behavior in the mid-term as sole means for 38 feedback and feedback-based error repair (besides a few codec- 39 specific mechanisms). This document defines an extension to the 40 Audio-visual Profile (AVP) that enables receivers to provide, 41 statistically, more immediate feedback to the senders and thus 42 allow for short-term adaptation and efficient feedback-based repair 43 mechanisms to be implemented. This early feedback profile (AVPF) 44 maintains the AVP bandwidth constraints for RTCP and preserves 45 scalability to large groups. 47 1. Introduction 49 Real-time media streams that use RTP are not resilient against packet 50 losses. RTP [1] provides all the necessary mechanisms to restore 51 ordering and timing present at the sender to properly reproduce a 52 media stream at a recipient. RTP also provides continuous feedback 53 about the overall reception quality from all receivers -- thereby 54 allowing the sender(s) in the mid-term (in the order of several 55 seconds to minutes) to adapt their coding scheme and transmission 56 behavior to the observed network QoS. However, except for a few 57 payload specific mechanisms [10], RTP makes no provision for timely 58 feedback that would allow a sender to repair the media stream 59 immediately: through retransmissions, retro-active FEC control, or 60 media-specific mechanisms for some video codecs, such as reference 61 picture selection. 63 Current mechanisms available with RTP to improve error resilience 64 include audio redundancy coding [7], video redundancy coding [11], 65 RTP-level FEC [5], and general considerations on more robust media 66 streams transmission [6]. These mechanisms may be applied pro- 67 actively (thereby increasing the bandwidth of a given media 68 stream). Alternatively, in sufficiently small groups with small 69 RTTs, the senders may perform repair on-demand, using the above 70 mechanisms and/or media-encoding-specific approaches. Note that 71 "small group" and "sufficiently small RTT" are both highly 72 application dependent. 74 This document specifies a modified RTP Profile for audio and video 75 conferences with minimal control based upon [1] and [2] by means of 76 two modifications/additions: to achieve timely feedback, the 77 concepts of Immediate Feedback messages and Early RTCP messages as 78 well as algorithms allowing for low delay feedback in small 79 multicast groups (and preventing feedback implosion in large ones) 80 are introduced. Special consideration is given to point-to-point 81 scenarios. A small number of general-purpose feedback messages as 82 well as a format for codec and application-specific feedback 83 information are defined as specific RTCP payloads. 85 1.1 Definitions 87 The definitions from [1] and [2] apply. In addition, the following 88 definitions are used in this document: 90 Early RTCP mode: 91 The mode of operation in which a receiver of a media stream 92 is, statistically, often (but not always) capable of 93 reporting events of interest back to the sender close to 94 their occurrence. In Early RTCP mode, RTCP feedback 95 messages are transmitted according to the timing rules 96 defined in this document. 98 Early RTCP packet: 99 An Early RTCP packet is a packet which is transmitted 100 earlier than would be allowed if following the scheduling 101 algorithm of [1], the reason being an "event" observed by a 102 receiver. Early RTCP packets may be sent in Immediate 103 feedback and in Early RTCP mode. 105 Event: 106 An observation made by the receiver of a media stream that 107 is (potentially) of interest to the sender -- such as a 108 packet loss or packet reception, frame loss, etc. -- and 109 thus useful to be reported back to the sender by means of a 110 Feedback message. 112 Feedback (FB) message: 113 An RTCP message as defined in this document is used to 114 convey information about events observed at a receiver -- 115 in addition to long term receiver status information which 116 is carried in RTCP RRs -- back to the sender of the media 117 stream. 119 Feedback (FB) threshold: 120 The FB threshold indicates the transition between Immediate 121 Feedback and Early RTCP mode. For a multicast scenario, 122 the FB threshold indicates the maximum group size at which, 123 on average, each receiver is able to report each event back 124 to the sender(s) immediately, i.e. by means of an Early 125 RTCP packet without having to wait for its regularly 126 scheduled RTCP interval. This threshold is highly 127 dependent on the type of feedback to be provided, network 128 QoS (e.g. packet loss probability and distribution), codec 129 and packetization scheme in use, the session bandwidth, and 130 application requirements. Note that the algorithms do not 131 depend on all senders and receivers agreeing on the same 132 value for this threshold. It is merely intended to provide 133 conceptual guidance to application designers and is not 134 used in any calculations. 136 Immediate Feedback mode: 137 A mode of operation in which each receiver of a media 138 stream is, statistically, capable of reporting each event 139 of interest immediately back to the media stream sender. 140 In Immediate Feedback mode, RTCP feedback messages are 141 transmitted according to the timing rules defined in this 142 document. 144 Regular RTCP mode: 146 Mode of operation in which no preferred transmission of 147 feedback messages is allowed. Instead, RTCP messages are 148 sent following the rules of [1]. Nevertheless, such RTCP 149 messages may contain feedback information as defined in 150 this document. 152 Regularly Scheduled RTCP packet: 153 An RTCP packet that is not sent as an Early RTCP packet. 155 1.2 Terminology 157 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 158 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" 159 in this document are to be interpreted as described in RFC 2119 160 [8] 162 2. RTP and RTCP Packet Formats and Protocol Behavior 164 The rules defined in [2] also apply to this profile except for 165 those rules mentioned in the following: 167 RTCP packet types: 168 Two additional RTCP packet types to convey feedback 169 information are defined in section 6. 171 RTCP report intervals: 172 This memo describes three modes of operation which 173 influence the RTCP report intervals (see section 3.2). In 174 regular RTCP mode, all rules from [1] apply. In both 175 Immediate Feedback and Early RTCP modes the minimal 176 interval of five seconds between two RTCP reports is 177 dropped and the rules specified in section 3 apply if RTCP 178 packets containing feedback messages (defined in section 4) 179 are to be transmitted. 181 The rules set forth in [1] may be overridden by session 182 descriptions specifying different parameters (e.g. for the 183 bandwidth share assigned to RTCP for senders and receivers, 184 respectively). For sessions defined using the Session 185 Description Protocol (SDP) [3], the rules of [4] apply. 187 Congestion control: 188 The same basic rules as detailed in [2] apply. Beyond 189 this, in section 5, further consideration is given to the 190 impact of feedback and a sender's reaction to feedback 191 messages. 193 3. Rules for RTCP Feedback 195 3.1 Compound RTCP Feedback Packets 197 Two components constitute RTCP-based feedback as described in this 198 memo: 200 o Status reports are contained in SR/RR messages and are 201 transmitted at regular intervals as part of compound RTCP 202 packets (which also include SDES and possibly other messages); 203 these status reports provide an overall indication for the 204 recent reception quality of a media stream. 206 o Feedback messages as defined in this document that indicate loss 207 or reception of particular pieces of a media stream (or provide 208 some other form of rather immediate feedback on the data 209 received). Rules for the transmission of feedback messages are 210 newly introduced in this memo. 212 RTCP Feedback (FB) messages are just another RTCP packet type (see 213 section 4). Therefore, multiple FB messages MAY be combined in a 214 single compound RTCP packet and they MAY also be sent combined with 215 other RTCP packets. 217 RTCP packets containing Feedback packets as defined in this 218 document MUST contain RTCP packets in the order as defined in [1]: 220 o OPTIONAL encryption prefix that MUST be present if the RTCP 221 message is to be encrypted. 222 o MANDATORY SR or RR. 223 o MANDATORY SDES which MUST contain the CNAME item; all other SDES 224 items are OPTIONAL. 225 o One or more FB messages. 227 The FB message(s) MUST be placed in the compound packet after RR 228 and SDES RTCP packets defined in [1]. The ordering with respect to 229 other RTCP extensions is not defined. 231 Two types of compound RTCP packets carrying feedback packets are 232 used in this document: 234 a) Minimal compound RTCP feedback packet 236 A minimal compound RTCP feedback packet MUST contain only the 237 mandatory information as listed above: encryption prefix if 238 necessary, exactly one RR or SR, exactly one SDES with only the 239 CNAME item present, and the feedback message(s). This is to 240 minimize the size of the RTCP packet transmitted to convey 241 feedback and thus to maximize the frequency at which feedback 242 can be provided while still adhering to the RTCP bandwidth 243 limitations. 245 This packet format SHOULD be used whenever an RTCP feedback 246 message is sent as part of an Early RTCP packet. 248 b) (Full) compound RTCP feedback packet 250 A (full) compound RTCP feedback packet MAY contain any 251 additional number of RTCP packets (additional RRs, further SDES 252 items, etc.). The above ordering rules MUST be adhered to. 254 This packet format MUST be used whenever an RTCP feedback 255 message is sent as part of a regularly scheduled RTCP packet or 256 in Regular RTCP mode. It MAY also be used to send RTCP 257 feedback messages in Immediate Feedback or Early RTCP mode. 259 RTCP packets that do not contain FB messages are referred to as 260 non-FB RTCP packets. Such packets MUST follow the format rules in 261 [1]. 263 3.2 Algorithm Outline 265 FB messages are part of the RTCP control streams and are thus 266 subject to the same bandwidth constraints as other RTCP traffic. 267 This means in particular that it may not be possible to report an 268 event observed at a receiver immediately back to the sender. 269 However, the value of feedback given to a sender typically 270 decreases over time -- in terms of the media quality as perceived 271 by the user at the receiving end and/or the cost required to 272 achieve media stream repair. 274 RTP [1] and the commonly used RTP profile [2] specify rules when 275 compound RTCP packets should be sent. This document modifies those 276 rules in order to allow applications to timely report events (e.g. 277 loss or reception of media packets) to accommodate algorithms that 278 use FB messages and are sensitive to the feedback timing. 280 The modified RTCP transmission algorithm can be outlined as 281 follows: Normally, when no FB messages have to be conveyed, 282 compound RTCP packets are sent following the rules of RTP [1] -- 283 except that the five second minimum interval between RTCP reports 284 is not enforced and the interval between RTCP reports is only 285 derived from the average RTCP packet size and the RTCP bandwidth 286 share available to the RTP/RTCP entity; in addition, a minimum 287 interval between regular RTCP packets may be enforced. If a 288 receiver detects the need to send an FB message, the receiver waits 289 for a short, random dithering interval (in case of multicast) and 290 then checks whether it has already seen a corresponding FB message 291 from any other receiver (which it can do with all FB messages that 292 are transmitted via multicast; for unicast sessions, there is no 293 such delay). If this is the case then the receiver refrains from 294 sending the FB message and continues to follow the regular RTCP 295 transmission schedule. If the receiver has not yet seen a similar 296 FB message from any other receiver, it checks whether it has 297 recently sent another FB message (without waiting for its regularly 298 scheduled RTCP transmission time). Only if this is not the case, 299 it sends the FB message as part of a (minimal) compound RTCP 300 packet. 302 FB messages may also be sent as part of full compound RTCP packets 303 which are interspersed as per [1] (except for the five second lower 304 bound) in regular intervals. 306 3.3 Modes of Operation 308 RTCP-based feedback may operate in one of three modes (figure 1) as 309 described below. The mode is a hint whether or not a receiver 310 should send early feedback at all and, if so, whether, 311 statistically, all events observed at the receiver can be reported 312 back to the sender in a timely fashion. The current mode of 313 operation is continuously derived independently at each receiver 314 and the receivers do not have to agree on a common mode. 316 a) Immediate feedback mode: the group size is below the FB 317 threshold which gives each receiving party sufficient bandwidth 318 to transmit the RTCP feedback packets for the intended purpose. 319 This means that, for each receiver, there is enough bandwidth 320 to report each event it is supposed/expected to by means of a 321 virtually "immediate" RTCP feedback packet. 323 The group size threshold is a function of a number of 324 parameters including (but not necessarily limited to) the type 325 of feedback used (e.g. ACK vs. NACK), bandwidth, packet rate, 326 packet loss probability and distribution, media type, codec, 327 and -- again depending on the type of FB used -- the (worst 328 case or observed) frequency of events to report (e.g. frame 329 received, packet lost). 331 A special case of this is the ACK mode (where positive 332 acknowledgements are used to confirm reception of data) which 333 is restricted to point-to-point communications. 335 As a rough estimate, let N be the average number of events to 336 be reported per interval T by a receiver, B the RTCP bandwidth 337 fraction for this particular receiver and R the average RTCP 338 packet size, then the receiver operates in Immediate Feedback 339 mode as long as N<=B*T/R. 341 b) Early RTCP mode: In this mode, the group size and other 342 parameters no longer allow each receiver to react to each event 343 that would be worth (or needed) to report. But feedback can 344 still be given sufficiently often so that it allows the sender 345 to adapt the media stream transmission accordingly and thereby 346 increase the overall reproduced media quality. 348 Using the above notation, Early RTCP mode can be roughly 349 characterized by N > B*T/R as "lower bound". An estimate for 350 an upper bound is more difficult. Setting N=1, we obtain for a 351 given R and B the interval T = R/B as average interval between 352 events to be reported. This information can be used as a hint 353 to determine whether or not early transmission of RTCP packets 354 is useful. 356 c) From some group size upwards, it is no longer useful to provide 357 feedback from individual receivers at all -- because of the 358 time scale in which the feedback could be provided and/or 359 because in large groups the sender(s) have no chance to react 360 to individual feedback anymore. 362 No group size threshold can be specified at which this mode 363 starts. 365 As the feedback algorithm described in this memo scales smoothly, 366 there is no need for an agreement among the participants on the 367 precise values of the respective "thresholds" within the group. 368 Hence the borders between all these modes are allowed to be soft. 370 ACK 371 feedback 372 V 373 :<- - - - NACK feedback - - - ->// 374 : 375 : Immediate || 376 : Feedback mode ||Early RTCP mode Regular RTCP mode 377 :<=============>||<=============>//<=================> 378 : || 379 -+---------------||---------------//------------------> group size 380 2 || 381 Application-specific FB Threshold 382 = f(data rate, packet loss, codec, ...) 384 Figure 1: Modes of operation 386 As stated before, the respective thresholds depend on a number of 387 technical parameters (of the codec, the transport, the type of 388 feedback used, etc.) but also on the respective application 389 scenarios. Section 3.6 provides some useful hints (but no precise 390 calculations) on estimating these thresholds. 392 3.4 Definitions 394 The following pieces of state information need to be maintained per 395 receiver (largely taken from [1]). Note that all variables (except 396 for g) are calculated independently at each receiver and so their 397 local values may differ at any given point in time. 399 a) Let "senders" be the number of active senders in the RTP 400 session. 402 b) Let "members" be the current estimate of the number of receivers 403 in the RTP session. 405 c) Let tn and tp be the time for the next (last) scheduled 406 RTCP RR transmission calculated prior to reconsideration. 408 d) Let Tmin be the minimal interval between RTCP packets as per 409 [1]. Unlike [1], the initial Tmin is set to 1 second (to allow 410 for some group size sampling before sending the first RTCP 411 packet), then it is set to 0. 413 e) Let T_rr be the interval after which, having just sent a 414 regularly scheduled RTCP packet, a receiver would schedule the 415 transmission of its next regular RTCP packet following the rules 416 of [1]: T_rr = T (the "calculated interval") with tn = tp + T. 417 Note that Tmin as defined in this specification is used to 418 compute the "calculated interval T". T_rr refers to the last 419 value of T that has been computed (because of reconsideration or 420 to determine tn). 422 f) Let t0 be the time at which an event that is to be reported is 423 detected by a receiver. 425 g) Let T_dither_max be the maximum interval for which an RTCP 426 feedback packet MAY be additionally delayed (to prevent 427 implosions). For point-to-point sessions, T_dither_max is set 428 to 0 and hence no additional delay is introduced. 430 h) Let T_max_fb_delay be the upper bound within which feedback to 431 an event needs to be reported back to the sender to be useful at 432 all. Note that this value is application-specific. 434 i) Let te be the time for which a feedback packet is scheduled. 436 j) Let T_fd be the actual (randomized) delay for the transmission 437 of feedback message in response to an event at time t0. 439 k) Let allow_early be a Boolean variable that indicates whether the 440 receiver currently may transmit feedback messages prior to its 441 next regularly scheduled RTCP interval tn. This variable is 442 used to throttle the feedback sent by a single receiver. 443 allow_early is adjusted (set to FALSE) after early feedback 444 transmission and is reset to TRUE as soon as the next regular 445 RTCP transmission has occurred. 447 l) Let avg_rtcp_size be the moving average on the RTCP packet size 448 as defined in [1]. 450 m) Let T_rr_interval be an (optional) minimal interval to be used 451 between regular RTCP packets. If T_rr_interval != 0 then 452 regular RTCP packets will not be scheduled T_rr after the last 453 regular RTCP transmission (at tp+T_rr) but only at least 454 T_rr_interval after the last regular RTCP transmission (later 455 than or at tp+T_rr). T_rr_interval does not affect the 456 calculation of T_rr and tp but may lead to regular RTCP packets 457 being suppressed. T_rr_interval does not affect transmission 458 scheduling for Early RTCP packets. 460 n) Let t_rr_last be the point in time at which the last RTCP packet 461 has been scheduled and sent (i.e. has not been suppressed due to 462 T_rr_interval). 464 NOTE: Providing T_rr_interval as an independent variable is 465 meant to minimize regular feedback (and thus bandwidth 466 consumption) as needed by the application but still allow for 467 more frequent use of Early RTCP packets to provide timely 468 feedback. This goal could not be achieved by reducing the 469 overall RTCP bandwidth as RTCP bandwidth reduction would also 470 impact the Early feedback. 472 o) Let T_retention be the time window for which past RTCP feedback 473 messages are stored by an AVPF entity. This is to ensure that 474 feedback suppression also works for entities that have received 475 feedback messages from other entities prior to noticing the 476 feedback event itself. T_retention MUST be set to at least 2 477 seconds. 479 The feedback situation for an event to report at a receiver is 480 depicted in figure 2 below. At time t0, such an event (e.g. a 481 packet loss) is detected at the receiver. The receiver decides -- 482 based upon current bandwidth, group size, and other (application- 483 specific) parameters -- that a feedback message needs to be sent 484 back to the sender. 486 To avoid an implosion of immediate feedback packets in multicast 487 sessions, the receiver MUST delay the transmission of the RTCP 488 feedback packet by a random amount T_fd (with the random number 489 evenly distributed in the interval [0, T_dither_max]). 490 Transmission of the compound RTCP packet MUST then be scheduled for 491 te = t0 + T_fd. 493 The T_dither_max parameter is derived from the regular RTCP 494 interval (which, in turn, is based upon the group size). 496 For a certain application scenario, a receiver may determine an 497 upper bound for the acceptable local delay of feedback messages: 498 T_max_fb_delay. If an a priori estimation or the actual 499 calculation of T_dither_max indicates that this upper bound MAY be 500 violated (e.g. because T_dither_max > T_max_fb_delay), the receiver 501 MAY decide not to send any feedback at all because the achievable 502 gain is considered insufficient. 504 If an RTCP feedback packet is scheduled, the time slot for the next 505 scheduled (full) compound RTCP packet MUST be updated accordingly 506 to a new tn (which will then be in the order of tn=tp+2*T_rr). 507 This is to ensure that the short term average bandwidth used for 508 RTCP with feedback does not exceed the bandwidth limit that would 509 be used without feedback. 511 event to 512 report 513 detected 514 | 515 | RTCP feedback range 516 | (T_max_fb_delay) 517 vXXXXXXXXXXXXXXXXXXXXXXXXXXX ) ) 518 |---+--------+-------------+-----+------------| |--------+---> 519 | | | | ( ( | 520 | t0 te | 521 tp tn 522 \_______ ________/ 523 \/ 524 T_dither_max 526 Figure 2: Event report and parameters for Early RTCP scheduling 528 3.5 Early RTCP Algorithm 530 Assume an active sender S0 (out of S senders) and a number N of 531 receivers with R being one of these receivers. 533 Assume further that R has verified that using feedback mechanisms 534 is reasonable at the current constellation (which is highly 535 application specific and hence not specified in this memo). 537 Assume that T_rr_interval is 0, if no minimal interval between 538 regular RTCP packets is to be enforced, or T_rr_interval is set to 539 some meaningful value, as given by the application. This value 540 then denotes the minimal interval between regular RTCP packets. 542 Then, receiver R MUST use the following rules for transmitting one 543 or more Feedback messages as minimal or full compound RTCP packet: 545 3.5.1 Initialization 547 Initially, R MUST set allow_early = TRUE and t_rr_last = NaN. 549 Furthermore, the initialization of the RTCP variables as per [1] 550 applies except that the initial value for Tmin. For a unicast 551 session, the initial Tmin is set to 0. For a multicast session, 552 Tmin is initialized to 1.0 seconds. 554 3.5.2 Early Feedback Transmission 556 Assume that R has scheduled the last RTCP RR packet for 557 transmission at tp and has scheduled the next transmission 558 (including possible reconsideration) for tn = tp + T_rr. Assume 559 also that the last T_rr_interval-based transmission (if any) has 560 occurred at t_rr_last (if defined). 562 1. At time t0, R detects the need to transmit one or more RTCP 563 feedback messages (e.g. because media "units" needs to be ACKed or 564 NACKed) and finds that sending the feedback information is useful 565 for the sender. 567 2. R first checks whether there is already a compound RTCP packet 568 containing an RTCP feedback message scheduled for transmission (as 569 early or regular RTCP packet). 571 2.a) If so, the new feedback message MUST be included in the 572 scheduled packet; the scheduling of the waiting RTCP feedback 573 packet MUST remain unchanged. When doing so, the feedback 574 information of several RTCP feedback packets SHOULD be merged 575 to produce as few feedback messages as possible. This 576 completes the course of immediate actions to be taken. 578 2.b) If no RTCP feedback message is already scheduled for 579 transmission, a new (minimal or full) compound RTCP feedback 580 packet MUST be created and the minimal interval for 581 T_dither_max MUST be chosen as follows: 583 i) If the session is a unicast session (group size = 2) then 584 T_dither_max = 0. 586 ii) If the session is a multicast session with potentially 587 more than two group members then 589 T_dither_max = l * T_rr 591 with l=0.5. 593 The values given above for T_dither_max are minimal values. 594 Application-specific feedback considerations may make it 595 worthwhile to increase T_dither_max beyond this value. This 596 is up to the discretion of the implementer. 598 3. Then, R MUST check whether its next regularly scheduled RTCP 599 packet would be within the time bounds for the RTCP FB (t0 + 600 T_dither_max > tn). 602 3.a) If so, an Early RTCP packet MUST NOT be scheduled; 603 instead the FB message(s) MUST be stored to be appended to the 604 regular RTCP packet scheduled for tn. This completes the 605 course of immediate actions to be taken. 607 3.b) Otherwise, the following steps are carried out. 609 4. R MUST check whether it is allowed to transmit an Early RTCP 610 packet (allow_early == TRUE). 612 4.a) If allow_early == FALSE then R MUST check the time for the 613 next scheduled RR: 615 1. If tn - t0 < T_max_fb_delay (i.e. if, despite late 616 reception, the feedback could still be useful for the 617 sender) then R MAY create an RTCP FB message for 618 transmission along with the RTCP packet at tn. 620 2. Otherwise, R MUST discard the RTCP feedback message. 622 This completes the immediate course of actions to be taken. 624 4.b) If allow_early == TRUE then R MUST schedule an Early RTCP 625 packet for te = t0 + RND * T_dither_max with RND being a pseudo 626 random function evenly distributed between 0 and 1. 628 5. R MUST continuously monitor the received RTCP feedback packets 629 contained in one or more (minimal) compound RTCP packets and keep 630 each of these packets for at least T_retention. When scheduling 631 the transmission of an RTCP feedback message, R MUST check each of 632 the RTCP feedback messages in the one or more compound RTCP packets 633 received in the interval [t0 - T_retention ; te] and act as 634 follows: 636 5.a) If R understands the received feedback message's semantics 637 and the message contents is a superset of the feedback R wanted 638 to send then R MUST discard its own feedback message and MUST 639 re-schedule the next regular RTCP message transmission for tn 640 (as calculated before). 642 5.b) If R understands the received feedback message's semantics 643 and the message contents is not a superset of the feedback R 644 wanted to send then R SHOULD transmit its own feedback message 645 as scheduled. If there is an overlap between the feedback 646 information to send and the feedback information received, the 647 amount of feedback transmitted is up to R: R MAY send its 648 feedback information unchanged, R MAY as well eliminate any 649 redundancy between its own feedback and the feedback received 650 so far. 652 5.c) If R does not understand the received feedback message's 653 semantics, R MAY send its own feedback message as Early RTCP 654 packet, or R MAY re-schedule the next regular RTCP message 655 transmission for tn (as calculated before) and MAY append the 656 feedback message to the now regularly scheduled RTCP message. 658 Note: With rule #3, receiving unknown feedback packets may not 659 lead to feedback suppression at a particular receiver. As a 660 consequence, a given event may cause M different types of 661 feedback packets (which are all appropriate but not the same 662 and mutually not understood) to be scheduled, and a "large" 663 receiver group may be partitioned into at most M groups. Among 664 members of each of these M groups, feedback suppression will 665 occur following the rules #1 and #2 but no suppression will 666 happen across groups. As a result, O(M) RTCP feedback messages 667 may be received by the sender. Given that these M groups 668 consist of receivers for the same application using the same 669 (set of) codecs in the same RTP session, M is assumed to be 670 small in the general case. Given further that the O(M) 671 feedback packets are randomly distributed over a time interval 672 of T_dither_max, the resulting limited number of extra feedback 673 packets (a) is assumed not to overwhelm the sender and (b) 674 should be conveyed as all contain complementary pieces of 675 information. 677 Refer to section 4 on the comparison of feedback messages and 678 for which feedback messages MUST be understood by a receiver. 680 6. Otherwise, when te is reached, R MUST transmit the RTCP packet 681 containing the FB message. R then MUST set allow_early = FALSE, 682 MUST recalculate tn = tp + 2*T_rr, and MUST set tp to the previous 683 tn. As soon as the newly calculated tn is reached and R sends its 684 next regularly scheduled RTCP RR or suppresses it because of 685 T_rr_interval, it MUST set allow_early = TRUE again. 687 3.5.3 Regular RTCP Transmission 689 In regular intervals full compound RTCP packets MUST be sent. 690 These packets MAY also contain one or more feedback messages. 691 Transmission of regular RTCP packets is scheduled as follows: 693 If T_rr_interval == 0 then the transmission MUST follow the rules 694 as specified by [1] (except for the different Tmin) and MUST adhere 695 to the adjustments of tn specified in section 3.5.2 (i.e. skip one 696 regular transmission if an Early RTCP transmission has occurred). 697 Timer reconsideration takes place when tn is reached as per [1] and 698 the regular RTCP packet is transmitted after timer reconsideration. 699 Whenever a regular RTCP message is sent, allow_early MUST be set to 700 TRUE and tp, tn MUST be updated as per [1]. If this was the first 701 transmission of an RTCP packet, Tmin MUST be set to 0. 703 If T_rr_interval != 0 then the calculation for the transmission 704 times MUST follow the rules as specified in [1] (except for the 705 different Tmin) and MUST adhere to the adjustments of tn specified 706 in section 3.5.2 (i.e. skip one regular transmission if an Early 707 RTCP transmission has occurred). Timer reconsideration takes place 708 when tn is reached as per [1]. After timer reconsideration, the 709 following actions are taken: 711 If no full compound RTCP packet has been sent before (i.e. if 712 t_rr_last == NaN) then a full compound RTCP packet MUST be 713 scheduled. Stored RTCP feedback messages MAY be included in 714 the full compound RTCP packet. t_rr_last MUST be set to tn. 715 Tmin MUST be set to 0. 717 Otherwise, an temporary value T_rr_current_interval is 718 calculated as follows: 720 T_rr_current_interval = RND*T_rr_interval 722 with RND being a pseudo random function evenly distributed 723 between 0.5 and 1.5. This dithered value is used for the 724 following alternatives: 726 If t_rr_last + T_rr_current_interval <= tn then a full 727 compound RTCP packet MUST be scheduled. Stored RTCP feedback 728 messages MAY be included in the full compound RTCP packet. 729 t_rr_last MUST be set to tn. 731 If t_rr_last + T_rr_current_interval > tn and RTCP feedback 732 messages have been stored and are awaiting transmission, an 733 RTCP packet MUST be scheduled for transmission at tn. This 734 RTCP packet MAY be a minimal or a full compound RTCP packet 735 (at the discretion of the implementer) and the compound RTCP 736 packet MUST include the stored RTCP feedback message. 737 t_rr_last MUST remain unchanged. 739 Otherwise (if t_rr_last + T_rr_current_interval > tn but no 740 stored RTCP feedback messages are awaiting transmission), no 741 compound RTCP packet MUST be scheduled. 743 In all the four cases above, allow early MUST be set to TRUE and tp 744 and tn MUST be updated following the rules of [1] except for the 745 five second minimum. 747 3.5.4 Other Considerations 749 Furthermore, if T_rr_interval != 0 then the timeout calculation for 750 RTP/AVPF entities (section 6.3.5 of [1]) MUST be modified to use 751 T_rr_interval instead of Tmin for computing Td. 753 Whenever an RTCP packet is sent or received -- minimal or full 754 compound, early or regularly scheduled -- the avg_rtcp_size 755 variable MUST be updated accordingly (see [1]) and subsequent 756 computations of tn MUST use the new avg_rtcp_size. 758 3.6 Considerations on the Group Size 760 This section provides some guidelines to the group sizes at which 761 the various feedback modes may be used. 763 3.6.1 ACK mode 765 The group size MUST be exactly two participants, i.e. point-to- 766 point communications. Unicast addresses MUST be used in the 767 session description. 769 For unidirectional as well as bi-directional communication between 770 two parties, 2.5% of the RTP session bandwidth are available for 771 RTCP traffic from the receivers including feedback. For a 64 772 kbit/s stream this yields 1,600 bit/s for RTCP. If we assume an 773 average of 96 bytes (=768 bits) per RTCP packet a receiver can 774 report 2 events per second back to the sender. If acknowledgments 775 for 10 events are collected in each feedback message then 20 events 776 can be acknowledged per second. At 256 kbit/s 8 events could be 777 reported per second; thus the ACKs may be sent in a finer 778 granularity (e.g. only combining three ACKs per RTCP feedback 779 message). 781 From 1 Mbit/s upwards, a receiver would be able to acknowledge each 782 individual frame (not packet!) in a 30 fps video stream. 784 ACK strategies MUST be defined to work properly with these 785 bandwidth limitations. An indication whether or not ACKs are 786 allowed for a session and, if so, which ACK strategy should be 787 used, MAY be conveyed by out-of-band mechanisms, e.g. media- 788 specific attributes in a session description using SDP. 790 3.6.2 NACK mode 792 Negative acknowledgements (or similar types of feedback) MUST be 793 used for all groups larger than two. Of course, NACKs MAY be used 794 for point-to-point communications as well. 796 Whether or not the use of Immediate or Early RTCP packets should be 797 considered depends upon a number of parameters including session 798 bandwidth, codec, special type of feedback, number of senders and 799 receivers, among many others. 801 The crucial parameters -- to which virtually all of the above can 802 be reduced -- is the allowed minimal interval between two RTCP 803 reports and the (average) number of events that presumably need 804 reporting per time interval (plus their distribution over time, of 805 course). The minimum interval can be derived from the available 806 RTCP bandwidth and the expected average size of an RTCP packet. 807 The number of events to report e.g. per second may be derived from 808 the packet loss rate and sender's rate of transmitting packets. 809 From these two values, the allowable group size for the Immediate 810 feedback mode can be calculated. 812 Let N be the average number of events to be reported per 813 interval T by a receiver, B the RTCP bandwidth fraction for 814 this particular receiver and R the average RTCP packet size, 815 then the receiver operates in Immediate Feedback mode is used 816 as long as N<=B*T/R. 818 The upper bound for the Early RTCP mode then solely depends on the 819 acceptable quality degradation, i.e. how many events per time 820 interval may go unreported. 822 Using the above notation, Early RTCP mode can be roughly 823 characterized by N > B*T/R as "lower bound". An estimate for 824 an upper bound is more difficult. Setting N=1, we obtain for a 825 given R and B the interval T = R/B as average interval between 826 events to be reported. This information can be used as a hint 827 to determine whether or not early transmission of RTCP packets 828 is useful. 830 Example: If a 256kbit/s video with 30 fps is transmitted through a 831 network with an MTU size of some 1,500 bytes, then, in most cases, 832 each frame would fit in its own packet leading to a packet rate of 833 30 packets per second. If 5% packet loss occurs in the network 834 (equally distributed, no inter-dependence between receivers), then 835 each receiver will have to report 3 packets lost each two seconds. 836 Assuming a single sender and more than three receivers, this yields 837 3.75% of the RTCP bandwidth allocated to the receivers and thus 838 9.6kbit/s. Assuming further a size of 120 bytes for the average 839 compound RTCP packet allows 10 RTCP packets to be sent per second 840 or 20 in two seconds. If every receiver needs to report three 841 packets, this yields a maximum group size of 6-7 receivers if all 842 loss events shall be reported. The rules for transmission of 843 immediate RTCP packets should provide sufficient flexibility for 844 most of this reporting to occur in a timely fashion. 846 Extending this example to determine the upper bound for Early RTCP 847 mode could lead to the following considerations: assume that the 848 underlying coding scheme and the application (as well as the 849 tolerant users) allow on the order of one loss without repair per 850 two seconds. Thus the number of packets to be reported by each 851 receiver decreases to two per two seconds second and increases the 852 group size to 10. Assuming further that some number of packet 853 losses are correlated, feedback traffic is further reduced and 854 group sizes of some 12 to 16 (maybe even 20) can be reasonably well 855 supported using Early RTCP mode. Note, of course, that all those 856 considerations are based upon statistics and will fail to hold in 857 some cases. 859 3.7 Summary of decision steps 861 3.7.1 General Hints 863 Before even considering whether or not to send RTCP feedback 864 information an application has to determine whether this mechanism 865 is applicable: 867 1) An application has to decide whether -- for the current ratio of 868 packet rate with the associated (application-specific) maximum 869 feedback delay and the currently observed round-trip time (if 870 available) -- feedback mechanisms can be applied at all. 872 This decision may obviously be based upon (and dynamically 873 revised following) regular RTCP reception statistics as well as 874 out-of-band mechanisms. 876 2) The application has to decide -- for a certain observed error 877 rate, assigned bandwidth, frame/packet rate, and group size -- 878 whether (and which) feedback mechanisms can be applied. 880 Regular RTCP provides valuable input to this step, too. 882 3) If these tests pass, the application has to follow the rules for 883 transmitting Early RTCP packets or regularly scheduled RTCP 884 packets with piggybacked feedback. 886 3.7.2 Media Session Attributes 888 Media sessions are typically described using out-of-band mechanisms 889 to convey transport addresses, codec information, etc. between 890 sender(s) and receiver(s). Such a mechanisms consists of a format 891 used to describe a media session and another mechanism for 892 transporting this description. 894 In the IETF, the Session Description Protocol (SDP) is currently 895 used to describe media sessions while protocols such as SIP, SAP, 896 RTSP, and HTTP (among others) are used to convey the descriptions. 898 A media session description format MAY include parameters to 899 indicate that RTCP feedback mechanisms MAY be used (=are supported) 900 in this session and which of the feedback mechanisms MAY be 901 applied. 903 To do so, the profile "AVPF" MUST be indicated instead of "AVP". 904 Further attributes may be defined to show which type(s) of feedback 905 are supported. 907 Section 4 contains the syntax specification to support RTCP 908 feedback with SDP. Similar specifications for other media session 909 description formats are outside the scope of this document. 911 4. SDP Definitions 913 This section defines a number of additional SDP parameters that are 914 used to describe a session. All of these are defined as media 915 level attributes. 917 4.1 Profile identification 919 The AV profile defined in [4] is referred to as "AVP" in the 920 context of e.g. the Session Description Protocol (SDP) [3]. The 921 profile specified in this document is referred to as "AVPF". 923 Feedback information following the modified timing rules as 924 specified in this document MUST NOT be sent for a particular media 925 session unless the profile for this session indicates the use of 926 the "AVPF" profile (exclusively or jointly with other AV profiles). 928 4.2 RTCP Feedback Capability Attribute 930 A new payload format-specific SDP attribute is defined to indicate 931 the capability of using RTCP feedback as specified in this 932 document: "a=rtcp-fb". The "rtcp-fb" attribute MUST only be used 933 as an SDP media attribute and MUST NOT be provided at the session 934 level. The "rtcp-fb" attribute MUST only be used in media sessions 935 for which the "AVPF" is specified. 937 The "rtcp-fb" attribute SHOULD be used to indicate which RTCP 938 feedback messages MAY be used in this media session for the 939 indicated payload type. A wildcard payload type ("*") MAY be used 940 to indicate that the RTCP feedback attribute applies to all payload 941 types. If several types of feedback are supported and/or the same 942 feedback shall be specified for a subset of the payload types, 943 several "a=rtcp-fb:" lines MUST be used. 945 If no "rtcp-fb" attribute is specified the RTP receivers SHOULD 946 assume that the RTP senders only support generic NACKs. In 947 addition, the RTP receivers MAY send feedback using other suitable 948 RTCP feedback packets as defined for the respective media type. 949 The RTP receivers MUST NOT rely on the RTP senders reacting to any 950 of the feedback messages. 952 If one or more "rtcp-fb" attributes are present in a media session 953 description, the RTCP receivers for the media session(s) containing 954 the "rtcp-fb" 956 o MUST ignore all "rtcp-fb" attributes of which they do not fully 957 understand the semantics (i.e. where they do not understand the 958 meaning of all values in the "a=rtcp-fb" line); 960 o SHOULD provide feedback information as specified in this 961 document using any of the RTCP feedback packets as specified in 962 one of the "rtcp-fb" attributes for this media session; and 964 o MUST NOT use other feedback messages than those listed in one of 965 the "rtcp-fb" attribute lines. 967 When used in conjunction with the offer/answer model [18], the 968 offerer MAY present a set of these AVPF attributes to its peer. 969 The answerer MUST remove all attributes it does not understand as 970 well as those it does not support in general or does not wish to 971 use in this particular media session. The answerer MUST NOT add 972 feedback parameters to the media description and MUST NOT alter 973 values of such parameters. The answer is binding for the media 974 session and both offerer and answerer MUST only use feedback 975 mechanisms negotiated in this way. 977 RTP senders MUST be prepared to receive any kind of RTCP feedback 978 messages and MUST silently discard all those RTCP feedback messages 979 that they do not understand. 981 The syntax of the "rtcp-fb" attribute is as follows (the feedback 982 types and optional parameters are all case sensitive): 984 (In the following ABNF, SP and CRLF are used as defined in [3].) 986 rtcp-fb-syntax = "a=rtcp-fb:" rtcp-fb-pt SP rtcp-fb-val CRLF 988 rtcp-fb-pt = "*" ; wildcard: applies to all formats 989 / fmt ; as defined in SDP spec 991 rtcp-fb-val = "ack" rtcp-fb-ack-param 992 / "nack" rtcp-fb-nack-param 993 / "trr-int" SP 1*DIGIT 994 / rtcp-fb-id rtcp-fb-param 996 rtcp-fb-id = 1*(alpha-numeric | "-" | "_") 998 rtcp-fb-param = SP "app" [SP byte-string] 999 / SP token [SP byte-string] 1000 / ; empty 1002 rtcp-fb-ack-param = SP "rpsi" 1003 / SP "app" [SP byte-string] 1004 / SP token [SP byte-string] 1005 / ; empty 1007 rtcp-fb-nack-param = SP "pli" 1008 / SP "sli" 1009 / SP "rpsi" 1010 / SP "app" [SP byte-string] 1011 / SP token [SP byte-string] 1012 / ; empty 1014 The literals of the above grammar have the following semantics: 1016 Feedback type "ack": 1018 This feedback type indicates that positive acknowledgements 1019 for feedback are supported. 1021 The feedback type "ack" MUST only be used if the media session 1022 is allowed to operate in ACK mode as defined in 3.6.1.2. 1024 Parameters MAY be provided to further distinguish different 1025 types of positive acknowledgement feedback. If no parameters 1026 are present, the Generic ACK as specified in section 6.2.2 is 1027 implied. 1029 The parameter "rpsi" indicates the use of Reference Picture 1030 Selection Indication feedback as defined in section 6.3.3. 1032 If the parameter "app" is specified, this indicates the use of 1033 application layer feedback. In this case, additional 1034 parameters following "app" MAY be used to further 1035 differentiate various types of application layer feedback. 1036 This document does not define any parameters specific to 1037 "app". 1039 Further parameters for "ack" MAY be defined in other 1040 documents. 1042 Feedback type "nack": 1044 This feedback type indicates that negative acknowledgements 1045 for feedback are supported. 1047 The feedback type "nack", without parameters, indicates use of 1048 the General NACK feedback format as defined in section 6.2.1. 1050 The following three parameters are defined in this document 1051 for use with "nack" in conjunction with the media type 1052 "video": 1054 o "pli" indicates the use of Picture Loss Indication feedback 1055 as defined in section 6.3.1. 1056 o "sli" indicates the use of Slice Loss Indication feedback 1057 as defined in section 6.3.2. 1058 o "rpsi" indicates the use of Reference Picture Selection 1059 Indication feedback as defined in section 6.3.3. 1061 "app" indicates the use of application layer feedback. 1062 Additional parameters after "app" MAY be provided to 1063 differentiate different types of application layer feedback. 1064 No parameters specific to "app" are defined in this document. 1066 Further parameters for "nack" MAY be defined in other 1067 documents. 1069 Other feedback types : 1071 Other documents MAY define additional types of feedback; to 1072 keep the grammar extensible for those cases, the rtcp-fb-id is 1073 introduced as a placeholder. A new feedback scheme name MUST 1074 to be unique (and thus MUST be registered with IANA). Along 1075 with a new name, its semantics, packet formats (if necessary), 1076 and rules for its operation MUST be specified. 1078 Regular RTCP minimum interval "trr-int": 1080 The attribute "trr-int" is used to specify the minimum 1081 interval T_rr_interval between two regular (full compound) 1082 RTCP packets in milliseconds for this media session. If "trr- 1083 int" is not specified, a default value of 0 is assumed. 1085 Note that it is assumed that more specific information about 1086 application layer feedback (as defined in section 6.4) will be 1087 conveyed as feedback types and parameters defined elsewhere. 1088 Hence, no further provision for any types and parameters is made in 1089 this document. 1091 Further types of feedback as well as further parameters may be 1092 defined in other documents. 1094 It is up to the recipients whether or not they send feedback 1095 information and up to the sender(s) to make use of feedback 1096 provided. 1098 4.3 Unicasting vs. Multicasting 1100 If a media session description indicates unicast addresses for a 1101 particular media type (and does not operate in multi-unicast mode 1102 with all recipients listed explicitly but still addressed via 1103 unicast), the RTCP feedback MAY operate in ACK feedback mode. 1105 If a media session description indicates multicast addresses for a 1106 particular media type or a multi-unicast session, ACK feedback mode 1107 MUST NOT be used. 1109 4.4 RTCP Bandwidth Modifiers 1111 The standard RTCP bandwidth assignments as defined in [1] and [2] 1112 may be overridden by bandwidth modifiers that explicitly define the 1113 maximum RTCP bandwidth. For use with SDP, such modifiers are 1114 specified in [4]: "b=RS:" and "b=RR:" MAY be used to assign 1115 a different bandwidth (measured in bits per second) to RTP senders 1116 and receivers, respectively. The precedence rules of [4] apply to 1117 determine the actual bandwidth to be used by senders and receivers. 1119 Applications operating knowingly over highly asymmetric links (such 1120 as satellite links) SHOULD use this mechanism to reduce the 1121 feedback rate for high bandwidth streams to prevent deterministic 1122 congestion of the feedback path(s). 1124 4.5 Examples 1126 Example 1: The following session description indicates a session 1127 made up from an audio and a DTMF for point-to-point communication 1128 in which the DTMF stream uses Generic ACKs. This session 1129 description could be contained in a SIP INVITE, 200 OK, or ACK 1130 message to indicate that its sender is capable of and willing to 1131 receive feedback for the DTMF stream it transmits. 1133 v=0 1134 o=alice 3203093520 3203093520 IN IP4 host.example.com 1135 s=Media with feedback 1136 t=0 0 1137 c=IN IP4 host.example.com 1138 m=audio 49170 RTP/AVPF 0 96 1139 a=rtpmap:0 PCMU/8000 1140 a=rtpmap:96 telephone-event/8000 1141 a=fmtp:96 0-16 1142 a=rtcp-fb:96 ack 1144 Example 2: The following session description indicates a multicast 1145 video-only session (using either H.261 or H.263+) with the video 1146 source accepting Generic NACKs for both codecs and Reference 1147 Picture Selection for H.263. Such a description may have been 1148 conveyed using the Session Announcement Protocol (SAP). 1150 v=0 1151 o=alice 3203093520 3203093520 IN IP4 host.example.com 1152 s=Multicast video with feedback 1153 t=3203130148 3203137348 1154 m=audio 49170 RTP/AVP 0 1155 c=IN IP4 224.2.1.183 1156 a=rtpmap:0 PCMU/8000 1157 m=video 51372 RTP/AVPF 98 99 1158 c=IN IP4 224.2.1.184 1159 a=rtpmap:98 H263-1998/90000 1160 a=rtpmap:99 H261/90000 1161 a=rtcp-fb:* nack 1162 a=rtcp-fb:98 nack rpsi 1164 Example 3: The following session description defines the same media 1165 session as example 2 but allows for mixed mode operation of AVP and 1166 AVPF RTP entities (see also next section). Note that both media 1167 descriptions use the same addresses; however, two m= lines are 1168 needed to convey information about both applicable RTP profiles. 1170 v=0 1171 o=alice 3203093520 3203093520 IN IP4 host.example.com 1172 s=Multicast video with feedback 1173 t=3203130148 3203137348 1174 m=audio 49170 RTP/AVP 0 1175 c=IN IP4 224.2.1.183 1176 a=rtpmap:0 PCMU/8000 1177 m=video 51372 RTP/AVP 98 99 1178 c=IN IP4 224.2.1.184 1179 a=rtpmap:98 H263-1998/90000 1180 a=rtpmap:99 H261/90000 1181 m=video 51372 RTP/AVPF 98 99 1182 c=IN IP4 224.2.1.184 1183 a=rtpmap:98 H263-1998/90000 1184 a=rtpmap:99 H261/90000 1185 a=rtcp-fb:* nack 1186 a=rtcp-fb:98 nack rpsi 1188 Note that these two m= lines SHOULD be grouped by some appropriate 1189 mechanisms to indicate that both are alternatives actually 1190 conveying the same contents. A sample mechanism by which this can 1191 be achieved is defined in [14]. 1193 5. Interworking and Co-Existence of AVP and AVPF Entities 1195 The AVPF profile defined in this document is an extension of the 1196 AVP profile as defined in [2]. Both profiles follow the same basic 1197 rules (including the upper bandwidth limit for RTCP and the 1198 bandwidth assignments to senders and receivers). Therefore, 1199 senders and receivers of using either of the two profiles can be 1200 mixed in a single session (see e.g. example 3 in section 4.5). 1202 AVP and AVPF are defined in a way that, from a robustness point of 1203 view, the RTP entities do not need to be aware of entities of the 1204 respective other profile: they will not disturb each other's 1205 functioning. However, the quality of the media presented may 1206 suffer. 1208 The following considerations apply to senders and receivers when 1209 used in a combined session. 1211 o AVP entities (senders and receivers) 1213 AVP senders will receive RTCP feedback packets from AVPF 1214 receivers and ignore these packets. They will see occasional 1215 closer spacing of RTCP messages (e.g. violating the five second 1216 rule) by AVPF entities. As the overall bandwidth constraints 1217 are adhered to by both types of entities, they will still get 1218 their share of the RTCP bandwidth. However, while AVP entities 1219 are bound by the five second rule, depending on the group size 1220 and session bandwidth, AVPF entities may provide more frequent 1221 RTCP reports than AVP ones will. Also, the overall reporting 1222 may decrease slightly as AVPF entities may send bigger compound 1223 RTCP packets (due to the extra RTCP packets). 1225 If T_rr_interval is used as lower bound between regular RTCP 1226 packets, T_rr_interval is sufficiently large (e.g. T_rr_interval 1227 > M*Td as per section 6.3.5 of [1]), and no Early RTCP packets 1228 are sent by AVPF entities, AVP entities MAY accidentally time 1229 out those AVPF group members and hence under-estimate the group 1230 size. Therefore, if AVP entities may be involved in a media 1231 session, T_rr_interval SHOULD NOT be larger than five seconds . 1233 o AVPF senders 1235 AVPF senders will receive feedback information only from AVPF 1236 receivers. If they rely on feedback to provide the target media 1237 quality, the quality achieved for AVP receivers may be sub- 1238 optimal. 1240 o AVPF receivers 1242 AVPF receivers SHOULD send immediate or early RTCP feedback 1243 packets only if all (sending) entities in the media session 1244 support AVPF. AVPF receivers MAY send feedback information as 1245 part of regularly scheduled compound RTCP packets following the 1246 timing rules of [1] and [2] also in media sessions operating in 1247 mixed mode. However, the receiver providing feedback MUST NOT 1248 rely on the sender reacting to the feedback at all. 1250 6. Format of RTCP Feedback Messages 1252 This section defines the format of the low delay RTCP feedback 1253 messages. These messages classified into three categories as 1254 follows: 1256 - Transport layer feedback messages 1257 - Payload-specific feedback messages 1258 - Application layer feedback messages 1260 Transport layer feedback messages are intended to transmit general 1261 purpose feedback information, i.e. information independent of the 1262 particular codec or the application in use. The information is 1263 expected to be generated and processed at the transport/RTP layer. 1265 Currently, only a general positive acknowledgement (ACK) and a 1266 negative acknowledgement (NACK) message are defined. 1268 Payload-specific feedback messages transport information that is 1269 specific to a certain payload type and will be generated and acted 1270 upon at the codec "layer". This document defines a common header 1271 to be used in conjunction with all payload-specific feedback 1272 messages. The definition of specific messages is left to either 1273 RTP payload format specifications or to additional feedback format 1274 documents. 1276 Application layer feedback messages provide a means to 1277 transparently convey feedback from the receiver's to the sender's 1278 application. The information contained in such a message is not 1279 expected to be acted upon at the transport/RTP or the codec layer. 1280 The data to be exchanged between two application instances is 1281 usually defined in the application protocol specification and thus 1282 can be identified by the application so that there is no need for 1283 additional external information. Hence, this document defines only 1284 a common header to be used along with all application layer 1285 feedback messages. From a protocol point of view, an application 1286 layer feedback message is treated as a special case of a payload- 1287 specific feedback message. 1289 NOTE: Proper processing of some feedback messages at the media 1290 sender side may require the sender to know which payload type 1291 the feedback message refers to. Most of the time, this 1292 knowledge can likely be derived from a media stream using only 1293 a single payload type. However, if several codecs are used 1294 simultaneously (e.g. with audio and DTFM) or when codec 1295 changes occur, the payload type information may need to be 1296 conveyed explicitly as part of the feedback message. This 1297 applies to all payload-specific as well as application layer 1298 feedback messages. It is up to the specification of a 1299 feedback message to define how payload type information is 1300 transmitted. 1302 This document defines two transport layer feedback and three 1303 (video) payload-specific feedback messages as well as a single 1304 container for application layer feedback messages. Additional 1305 transport layer and payload specific feedback messages MAY be 1306 defined in other documents and MUST be registered through IANA (see 1307 section IANA considerations). 1309 The general syntax and semantics for the above RTCP feedback 1310 message types are described in the following subsections. 1312 6.1 Common Packet Format for Feedback Message 1313 All feedback message MUST use a common packet format that is 1314 depicted in figure 3: 1316 0 1 2 3 1317 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1318 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1319 |V=2|P| FMT | PT | length | 1320 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1321 | SSRC of packet sender | 1322 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1323 | SSRC of media source | 1324 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1325 : Feedback Control Information (FCI) : 1326 : : 1328 Figure 3: Common Packet Format for Feedback Messages 1330 The various fields V, P, SSRC and length are defined in the RTP 1331 specification [2], the respective meaning being summarized below: 1333 version (V): 2 bits 1334 This field identifies the RTP version. The current version is 1335 2. 1337 padding (P): 1 bit 1338 If set, the padding bit indicates that the packet contains 1339 additional padding octets at the end which are not part of the 1340 control information but are included in the length field. 1342 Feedback message type (FMT): 5 bits 1343 This field identifies the type of the feedback message and is 1344 interpreted relative to the RTCP message type (transport, 1345 payload-specific, or application layer feedback). The values 1346 for each of the three feedback types are defined in the 1347 respective sections below. 1349 Payload type (PT): 8 bits 1350 This is the RTCP packet type which identifies the packet as 1351 being an RTCP Feedback Message. Two values are defined (TBA. 1352 by IANA): 1354 Name | Value | Brief Description 1355 ----------+-------+------------------------------------ 1356 RTPFB | 205 | Transport layer feedback message 1357 PSFB | 206 | Payload-specific feedback message 1359 Length: 16 bits 1360 The length of this packet in 32-bit words minus one, including 1361 the header and any padding. This is in line with the 1362 definition of the length field used in RTCP sender and receiver 1363 reports [3]. 1365 SSRC of packet sender: 32 bits 1366 The synchronization source identifier for the originator of 1367 this packet. 1369 SSRC of media source: 32 bits 1370 The synchronization source identifier of the media source that 1371 this piece of feedback information is related to. 1373 Feedback Control Information (FCI): variable length 1374 The following three sections define which additional 1375 information MAY be included in the feedback message for each 1376 type of feedback (further FCI contents MAY be specified in 1377 further documents). 1379 Each RTCP feedback packet MUST contain at least one feedback 1380 message in the FCI field. Sections 6.2 and 6.3 define for each 1381 FCI type, whether or not multiple feedback messages MAY be 1382 contained in a single FCI field. If multiple feedback messages 1383 are contained in one FCI field, they MUST be of the same type 1384 as. If multiple types of feedback need to be conveyed, then 1385 several RTCP feedback packets MUST be generated and SHOULD 1386 concatenated in the same compound RTCP packet. 1388 6.2 Transport Layer Feedback Messages 1390 Transport Layer Feedback messages are identified by the value RTPFB 1391 as RTCP message type. 1393 Two general purpose transport layer feedback messages are defined 1394 so far: General ACK and General NACK. They are identified by means 1395 of the FMT parameter as follows: 1397 0: unassigned 1398 1: Generic NACK 1399 2: Generic ACK 1400 3-30: unassigned 1401 31: reserved for future expansion of the sequence number space 1403 The following two subsections define the packet formats for these 1404 messages. 1406 6.2.1 Generic NACK 1408 The Generic NACK message is identified by PT=RTPFB and FMT=1. 1410 The FCI field MUST contain at least one and MAY contain more than 1411 one Generic NACK. 1413 The Generic NACK packet is used to indicate the loss of one or more 1414 RTP packets. The lost packet(s) are identified by the means of a 1415 packet identifier and a bit mask. 1417 The Feedback control information (FCI) field has the following 1418 Syntax (figure 4): 1420 0 1 2 3 1421 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1422 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1423 | PID | BLP | 1424 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1426 Figure 4: Syntax for the Generic NACK message 1428 Packet ID (PID): 16 bits 1429 The PID field is used to specify a lost packet. Typically, the 1430 RTP sequence number is used for PID as the default format, but 1431 RTP Payload Formats may decide to identify a packet 1432 differently. 1434 bitmask of following lost packets (BLP): 16 bits 1435 The BLP allows for reporting losses of any of the 16 RTP 1436 packets immediately following the RTP packet indicated by the 1437 PID. The BLP's definition is identical to that given in [10]. 1438 Denoting the BLP's least significant bit as bit 1, and its most 1439 significant bit as bit 16, then bit i of the bit mask is set to 1440 1 if the receiver has not received RTP packet number (PID+i) 1441 (modulo 2^16) and indicates this packet is lost; bit i is set 1442 to 0 otherwise. Note that the sender MUST NOT assume that a 1443 receiver has received a packet because its bit mask was set to 1444 0. For example, the least significant bit of the BLP would be 1445 set to 1 if the packet corresponding to the PID and the 1446 following packet have been lost. However, the sender cannot 1447 infer that packets PID+2 through PID+16 have been received 1448 simply because bits 2 through 15 of the BLP are 0; all the 1449 sender knows is that the receiver has not reported them as lost 1450 at this time. 1452 The length of the feedback message MUST be set to 2+n, with n being 1453 the number of Generic NACKs contained in the FCI field. 1455 The Generic NACK message implicitly references the payload type 1456 through the sequence number(s). 1458 6.2.2 Generic ACK 1460 The Generic ACK message is identified by PT=RTPFB and FMT=2. 1461 The FCI field MUST contain at least one and MAY contain more than 1462 one Generic ACK. 1464 The Generic ACK packet is used to indicate that one or several RTP 1465 packets were received correctly. The received packet(s) are 1466 identified by the means of a packet identifier and a bit mask. 1467 ACKing of a range of consecutive packets is also possible. 1469 The Feedback control information (FCI) field has the following 1470 syntax: 1472 0 1 2 3 1473 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1474 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1475 | PID |R| BLP/#packets | 1476 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1478 Figure 5: Syntax for the Generic ACK message 1480 Packet ID (1st PID): 16 bits 1481 This PID field is used to specify a correctly received packet. 1482 Typically, the RTP sequence number is used for PID as the 1483 default format, but RTP Payload Formats may decide to identify 1484 a packet differently. 1486 Range of ACKs (R): 1 bit 1487 The R-bit indicates that a range of consecutive packets are 1488 received correctly. If R=1 then the PID field specifies the 1489 first packet of that range and the next field (BLP/#packets) 1490 will carry the number of packets being acknowledged. If R=0 1491 then PID specifies the first packet to be acknowledged and 1492 BLP/#packets provides a bit mask to selectively indicate 1493 individual packets that are acknowledged. 1495 Bit mask of lost packets (BLP)/#packets (PID): 15 bits 1496 The semantics of this field depends on the value of the R-bit. 1498 If R=1, this field is used to identify the number of additional 1499 packets of to be acknowledged: 1501 #packets = - 1503 That is, #packets MUST indicate the number of packet to be 1504 ACKed minus one. In particular, if only a single packet is to 1505 be ACKed and R=1 then #packets MUST be set to 0x0000. 1507 Example: If all packets between and including PIDx = 380 and 1508 PIDy = 422 have been received, the Generic ACK would contain 1509 PID = PIDx = 380 and #packets = PIDy - PID = 42. In case the 1510 PID wraps around, modulo arithmetic is used to calculate the 1511 number of packets. 1513 If R=0, this field carries a bit mask. The BLP allows for 1514 reporting reception of any of the 15 RTP packets immediately 1515 following the RTP packet indicated by the PID. The BLP's 1516 definition is identical to that given in [10] except that, 1517 here, BLP is only 15 bits wide. Denoting the BLP's least 1518 significant bit as bit 1, and its most significant bit as bit 1519 15, then bit i of the bitmask is set to 1 if the receiver has 1520 received RTP packet number (PID+i) (modulo 2^16) and decides to 1521 ACK this packet; bit i is set to 0 otherwise. If only the 1522 packet indicated by PID is to be ACKed and R=0 then BLP MUST be 1523 set to 0x0000. 1525 The length of the feedback message MUST be set to 2+n, with n being 1526 the number of Generic ACKs contained in the FCI field. 1528 The Generic ACK message implicitly references the payload type 1529 through the sequence number(s). 1531 6.3 Payload Specific Feedback Messages 1533 Payload-Specific Feedback Messages are identified by the value 1534 PT=PSFB as RTCP message type. 1536 Three payload-specific feedback messages are defined so far plus an 1537 application layer feedback message. They are identified by means 1538 of the FMT parameter as follows: 1540 0: unassigned 1541 1: Picture Loss Indication (PLI) 1542 2: Slice Lost Indication (SLI) 1543 3: Reference Picture Selection Indication (RPSI) 1544 4-14: reserved 1545 15: Application layer feedback message 1546 16-30: unassigned 1547 31: reserved for future expansion of the sequence number space 1549 The following subsections define the packet formats for the 1550 payload-specific messages, section 6.4 defines the application 1551 layer feedback message. 1553 6.3.1 Picture Loss Indication (PLI) 1555 The PLI feedback message is identified by PT=PSFB and FMT=1. 1556 There MUST be exactly one PLI contained in the FCI field. 1558 6.3.1.1 Semantics 1560 With the Picture Loss Indication message, a decoder informs the 1561 encoder about the loss of an undefined amount of coded video data 1562 belonging to one or more pictures. When used in conjunction with 1563 any video coding scheme that is based on inter-picture prediction, 1564 an encoder that receives a PLI becomes aware that the prediction 1565 chain may be broken. The sender MAY react to a PLI by transmitting 1566 an intra-picture to achieve resynchronization (making effectively 1567 similar to the FIR as defined in [10]); however, the sender MUST 1568 consider congestion control as outlined in section 7 which MAY 1569 restrict its ability to send an intra frame. 1571 Other RTP payload specifications such as RFC 2032 [10] already 1572 define a feedback mechanism for some for certain codecs. An 1573 application supporting both schemes MUST use the feedback mechanism 1574 defined in this specification when sending feedback. For backward 1575 compatibility reasons, such an application SHOULD also be capable 1576 to receive and react to the feedback scheme defined in the 1577 respective RTP payload format, if this is required by that payload 1578 format. 1580 6.3.1.2 Message Format 1582 PLI does not require parameters. Therefore, the length field MUST 1583 be 2, and there MUST NOT be any Feedback Control Information. 1585 The semantics of this feedback message is independent of the 1586 payload type. 1588 6.3.1.3 Timing Rules 1590 The timing follows the rules outlined in section 3. In systems 1591 that employ both PLI and other types of feedback it may be 1592 advisable to follow the regular RTCP RR timing rules for PLI, since 1593 PLI is not as delay critical as other FB types. 1595 6.3.1.4 Remarks 1597 PLI messages typically trigger the sending of full intra pictures. 1598 Intra pictures are several times larger then predicted (inter) 1599 pictures. Their size is independent of the time they are 1600 generated. In most environments, especially when employing 1601 bandwidth-limited links, the use of an intra picture implies an 1602 allowed delay that is a significant multitude of the typical frame 1603 duration. An example: If the sending frame rate is 10 fps, and an 1604 intra picture is assumed to be 10 times as big as an inter picture, 1605 then a full second of latency has to be accepted. In such an 1606 environment there is no need for a particular short delay in 1607 sending the feedback message. Hence waiting for the next possible 1608 time slot allowed by RTCP timing rules as per [2] does not have a 1609 negative impact on the system performance. 1611 6.3.2 Slice Lost Indication (SLI) 1613 The SLI feedback message is identified by PT=PSFB and FMT=2. 1614 The FCI field MUST contain at least one and MAY contain more than 1615 one SLI. 1617 6.3.2.1 Semantics 1619 With the Slice Lost Indication a decoder can inform an encoder that 1620 it has detected the loss or corruption of one or several 1621 consecutive macroblock(s) in scan order (see below). This feedback 1622 message MUST NOT be used for video codecs with non-uniform, 1623 dynamically changeable macroblock sizes such as H.263 with enabled 1624 Annex Q. In such a case, an encoder cannot always identify the 1625 corrupted spatial region. 1627 6.3.2.2 Format 1629 The Slice Lost Indication uses one additional PCI field the 1630 content of which is depicted in figure 6. The length of the 1631 feedback message MUST be set to 2+n, with n being the number of 1632 SLIs contained in the FCI field. 1634 0 1 2 3 1635 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1637 |0| Payload Type| MBZ | First | 1638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1639 | Number | PictureId | 1640 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1642 Figure 6: Syntax of the Slice Lost Indication (SLI) 1644 0: 1 bit 1645 MUST be set to '0' upon transmission and MUST be ignored upon 1646 reception. 1648 Payload Type: 7 bits 1649 This field contains the Payload Type of the RTP packet (i.e. 1650 the codec) the SLI message refers to. 1652 MBZ: 8 bits 1653 MUST be set to '0' upon transmission and MUST be ignored upon 1654 reception. 1656 First: 16 bits 1657 The macroblock (MB) address of the first lost macroblock. The 1658 MB numbering is done such that the macroblock in the upper left 1659 corner of the picture is considered macroblock number 1 and the 1660 number for each macroblock increases from left to right and 1661 then from top to bottom in raster-scan order (such that if 1662 there is a total of N macroblocks in a picture, the bottom 1663 right macroblock is considered macroblock number N). 1665 Number: 16 bits 1666 The number of lost macroblocks, in scan order as discussed 1667 above. 1669 PictureID: 16 bits 1670 The six least significant bits of the a codec-specific 1671 identifier that is used to reference the picture in which the 1672 loss of the macroblock (s) has occurred. For many video 1673 codecs, the PictureID is identical to the Temporal Reference. 1675 6.3.2.3 Timing Rules 1677 The efficiency of algorithms using the Slice Lost Indication is 1678 reduced greatly when the Indication is not transmitted in a timely 1679 fashion. Motion compensation propagates corrupted pixels that are 1680 not reported as being corrupted. Therefore, the use of the 1681 algorithm discussed in section 3 is highly recommended. 1683 6.3.2.4 Remarks 1685 The term Slice is defined and used here in the sense of MPEG-1 -- a 1686 consecutive number of macroblocks in scan order. More recent video 1687 coding standards sometimes have a different understanding of the 1688 term Slice. In H.263 (1998), for example, a concept known as 1689 "rectangular Slice" exist. The loss of one Rectangular Slice may 1690 lead to the necessity of sending more than one SLI in order to 1691 precisely identify the region of lost/damaged MBs. 1693 The first field of the FCI defines the first macroblock of a 1694 picture as 1 and not, as one could suspect, as 0. This was done to 1695 align this specification with the comparable mechanism available in 1696 H.245. The maximum number of macroblocks in a picture (2**13 or 1697 8192) corresponds to the maximum picture sizes of most of the ITU-T 1698 and ISO/IEC video codecs. If future video codecs offer larger 1699 picture sizes and/or smaller macroblock sizes, then an additional 1700 feedback message has to be defined. The six least significant bits 1701 of the Temporal Reference field are deemed to be sufficient to 1702 indicate the picture in which the loss occurred. 1704 The reaction to a SLI is not part of this specification. One 1705 typical way of reacting to a SLI is to use intra refresh for the 1706 affected spatial region. 1708 Algorithms were reported that keep track of the regions affected by 1709 motion compensation, in order to allow for a transmission of Intra 1710 macroblocks to all those areas, regardless of the timing of the FB 1711 (see H.263 (2000) Appendix I [13] and [15]). While, when those 1712 algorithms are used, the timing of the FB is less critical then 1713 without, it has to be observed that those algorithms correct large 1714 parts of the picture and, therefore, have to transmit much higher 1715 data volume in case of delayed FBs. 1717 6.3.3 Reference Picture Selection Indication (RPSI) 1719 The RPSI feedback message is identified by PT=PSFB and FMT=3. 1720 There MUST be exactly one RPSI contained in the FCI field. 1722 6.3.3.1 Semantics 1724 Modern video coding standards such as MPEG-4 visual version 2 [12] 1725 or H.263 version 2 [13] allow to use older reference pictures than 1726 the most recent one for predictive coding. Typically, a first-in- 1727 first-out queue of reference pictures is maintained. If an encoder 1728 has learned about a loss of encoder-decoder synchronicity, a known- 1729 as-correct reference picture can be used. As this reference picture 1730 is temporally further away then usual, the resulting predictively 1731 coded picture will use more bits. 1733 Both MPEG-4 and H.263 define a binary format for the "payload" of 1734 an RPSI message that includes information such as the temporal ID 1735 of the damaged picture and the size of the damaged region. This 1736 bit string is typically small -- a couple of dozen bits --, of 1737 variable length, and self-contained, i.e. contains all information 1738 that is necessary to perform reference picture selection. 1740 Note that both MPEG-4 and H.263 allow the use of RPSI with positive 1741 feedback information as well. That is, pictures (or Slices) are 1742 reported that were decoded without error. Note that any form of 1743 positive feedback MUST NOT be used when in a multicast environment 1744 (reporting positive feedback about individual reference pictures at 1745 RTCP intervals is not expected to be of much use anyway). 1747 6.3.3.2 Format 1749 The FCI for the RPSI message follows the format depicted in figure 1750 7: 1752 0 1 2 3 1753 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1755 | PB |0| Payload Type| Native RPSI bit string | 1756 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1757 | as defined per codec ... | Padding (0)| 1758 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1760 Figure 7: Syntax of the Reference Picture Selection Indication 1761 (RPSI) 1763 PB: 8 bits 1764 The number of unused bits required to pad the length of the 1765 RPSI message to a multiple of 32 bits. 1767 0: 1 bit 1768 MUST be set to zero upon transmission and ignored upon 1769 reception. 1771 Payload Type: 8 bits 1772 Indicates the RTP payload type in the context of which the 1773 native RPSI bit string MUST be interpreted. 1775 Native RPSI bit string: variable length 1776 The RPSI information as natively defined by the video codec. 1778 Padding: #PB bits 1779 A number of bits set to zero to fill up the contents of the 1780 RPSI message to the next 32 bit boundary. The number of 1781 padding bits MUST be indicated by the PB field. 1783 6.3.3.3 Timing Rules 1785 RPS is even more critical to delay then algorithms using SLI. This 1786 is due to the fact that the older the RPS message is, the more bits 1787 the encoder has to spend to re-establish encoder-decoder 1788 synchronicity. See [15] for some information about the overhead of 1789 RPS for certain bit rate/frame rate/loss rate scenarios. 1791 Therefore, RPS messages should typically be sent as soon as 1792 possible, employing the algorithm of section 3. 1794 6.4 Application Layer Feedback Messages 1796 Application Layer Feedback Messages are a special case of payload- 1797 specific messages and identified by PT=PSFB and FMT=15. 1798 There MUST be exactly one Application Layer Feedback message 1799 contained in the FCI field. 1801 These messages are used to transport application defined data 1802 directly from the receiver's to the sender's application. The data 1803 that is transported is not identified by the feedback message. 1804 Therefore, the application MUST be able to identify the messages 1805 payload. 1807 Usually, applications define their own set of messages, e.g. 1808 NEWPRED messages in MPEG-4 or feedback messages in H.263/Annex N, 1809 U. These messages do not need any additional information from the 1810 RTCP message. Thus the application message is simply placed into 1811 the FCI field as follows and the length field is set accordingly. 1813 Application Message (FCI): variable length 1814 This field contains the original application message that 1815 should be transported from the receiver to the source. The 1816 format is application dependent. The length of this field is 1817 variable. If the application data is not 32-bit-aligned, 1818 padding bits and bytes must be added. Identification of 1819 padding is up to the application layer and not defined in this 1820 specification. 1822 The application layer feedback message specification MUST define 1823 whether or not the message needs to be interpreted specifically in 1824 the context of a certain codec (identified by the RTP payload 1825 type). If a reference to the payload type is required for proper 1826 processing, the application layer feedback message specification 1827 MUST define a way to communicate the payload type information as 1828 part of the application layer feedback message itself. 1830 7. Early Feedback and Congestion Control 1832 In the previous sections, the feedback messages were defined as 1833 well as the timing rules according to which to send these messages. 1834 The way to react to the feedback received depends on the 1835 application using the feedback mechanisms and hence is beyond the 1836 scope of this document. 1838 However, across all applications, there is a common requirement for 1839 (TCP-friendly) congestion control on the media stream as defined in 1840 [1] and [2] when operating in a best-effort network environment. 1842 Low delay feedback supports the use of congestion control 1843 algorithms in two ways: 1845 o The potentially more frequent RTCP messages allow the sender to 1846 monitor the network state more closely than with regular RTCP 1847 and therefore enable reacting to upcoming congestion in a more 1848 timely fashion. 1850 o The feedback messages themselves may convey additional 1851 information as input to congestion control algorithms and thus 1852 improve reaction over conventional RTCP. (For example, ACK-based 1853 feedback may even allow to construct closed loop algorithms and 1854 NACK-based systems may provide further information on the packet 1855 loss distribution.) 1857 A congestion control algorithm that shares the available bandwidth 1858 fair with competing TCP connections, e.g. TFRC [16], SHOULD be used 1859 to determine the data rate for the media stream (if the low delay 1860 RTP session is transmitted in a best effort environment). 1862 RTCP feedback messages or RTCP SR/RR packets that indicate recent 1863 packet loss MUST NOT lead to a (mid-term) increase in the 1864 transmission data rate and SHOULD lead to a (short-term) decrease 1865 of the transmission data rate. Such messages SHOULD cause the 1866 sender to adjust the transmission data rate to the order of the 1867 throughput TCP would achieve under similar conditions (e.g. using 1868 TFRC). 1870 RTCP feedback messages or RTCP SR/RR packets that indicate no 1871 recent packet loss MAY cause the sender to increase the 1872 transmission data rate to roughly the throughput TCP would achieve 1873 under similar conditions (e.g. using TFRC). 1875 8. Security Considerations 1877 RTP packets transporting information with the proposed payload 1878 format are subject to the security considerations discussed in the 1879 RTP specification [1] and in the RTP/AVP profile specification [2]. 1880 This profile does not specify any additional security services. 1882 This profile modifies the timing behavior of RTCP and eliminates 1883 the minimum RTCP interval of five seconds and allows for earlier 1884 feedback to be provided by receivers. Group members of the 1885 associated RTP session (possibly pretending to represent a large 1886 number of entities) may disturb the operation of RTCP by sending 1887 large numbers of RTCP packets thereby reducing the RTCP bandwidth 1888 available for regular RTCP reporting as well as for early feedback 1889 messages. (Note that an entity need not be member of a multicast 1890 group to cause these effects.) 1892 Feedback information may be suppressed if unknown RTCP feedback 1893 packets are received. This introduces the risk of a malicious 1894 group member reducing early feedback by simply transmitting 1895 payload-specific RTCP feedback packets with random contents that 1896 are neither recognized by any receiver (so they will suppress 1897 feedback) nor by the sender (so no repair actions will be taken). 1899 A malicious group member can also report arbitrary high loss rates 1900 in the feedback information to make the sender throttle the data 1901 transmission and increase the amount of redundancy information or 1902 take other action to deal with the pretended packet loss (e.g. send 1903 fewer frames or decrease audio/video quality). This may result in 1904 a degradation of the quality of the reproduced media stream. 1906 Finally, a malicious group member can act as a large number of 1907 group members and thereby obtain an artificially large share of the 1908 early feedback bandwidth and reduce the reactivity of the other 1909 group members -- possibly even causing them to no longer operate in 1910 immediate or early feedback mode and thus undermining the whole 1911 purpose of this profile. 1913 Senders as well as receivers SHOULD behave conservative when 1914 observing strange reporting behavior. For excessive failure 1915 reporting from one or a few receivers, the sender MAY decide to no 1916 longer consider this feedback when adapting its transmission 1917 behavior for the media stream. In any case, senders and receivers 1918 SHOULD still adhere to the maximum RTCP bandwidth but make sure 1919 that they are capable of transmitting at least regularly scheduled 1920 RTCP packets. Senders SHOULD carefully consider how to adjust 1921 their transmission bandwidth when encountering strange reporting 1922 behavior; they MUST NOT increase their transmission bandwidth even 1923 if ignoring suspicious feedback. 1925 Attacks using false RTCP packets (regular as well as early ones) 1926 can be avoided by authenticating all RTCP messages. This can be 1927 achieved by using the AVPF profile together with the Secure RTP 1928 profile as defined in [17]; as a prerequisite, an appropriate 1929 combination of those two profiles (an "SAVPF") needs to be 1930 specified. 1932 9. IANA Considerations 1934 The following contact information shall be used for all 1935 registrations included here: 1937 Contact: Joerg Ott 1938 mailto:jo@acm.org 1939 tel:+49-421-201-7028 1941 The feedback profile as an extension to the profile for audio- 1942 visual conferences with minimal control needs to be registered for 1943 the Session Description Protocol (specifically the type "proto"): 1944 "RTP/AVPF". 1946 SDP Protocol ("proto"): 1948 Name: RTP/AVPF 1949 Long form: Extended RTP Profile with RTCP-based Feedback 1950 Type of name: proto 1951 Type of attribute: Media level only 1952 Purpose: See this document 1953 Reference: This document 1955 SDP Attribute ("att-field"): 1957 Attribute name: rtcp-fb 1958 Long form: RTCP Feedback parameter 1959 Type of name: att-field 1960 Type of attribute: Media level only 1961 Subject to charset: No 1962 Purpose: See this document 1963 Reference: This document 1964 Values: See this document and registrations below 1966 A new registry needs to be set up for the "rtcp-fb" attribute, with 1967 the following registrations created initially: "ack", "nack", "trr- 1968 int", and "app" as defined in this document. 1970 Initial value registration for the attribute "rtcp-fb" 1972 Value name: ack 1973 Long name: Positive acknowledgement 1974 Reference: This document. 1976 Value name: nack 1977 Long name: Negative Acknowledgement 1978 Reference: This document. 1980 Value name: trr-int 1981 Long name: Minimal receiver report interval 1982 Reference: This document. 1984 Value name: app 1985 Long name: Application-defined paramater 1986 Reference: This document. 1988 Further entries may be registered on a first-come first serve 1989 basis. Each new registration needs to indicate the parameter name 1990 and the syntax of possible additional arguments. For each new 1991 registration, it is mandatory that a permanent, stable, and 1992 publicly accessible document exists that specifies the semantics of 1993 the registered parameter, the syntax and semantics of its 1994 parameters as well as corresponding feedback packet formats (if 1995 needed). The general registration procedures of [3] apply. 1997 For use with both "ack" and "nack", a joint sub-registry needs to 1998 be set up that initially registers the following values: 2000 Initial value registration for the attribute values "ack" and 2001 "nack": 2003 Value name: sli 2004 Long name: Slice Loss Indication 2005 Usable with: nack 2006 Reference: This document. 2008 Value name: pli 2009 Long name: Picture Loss Indication 2010 Usable with: nack 2011 Reference: This document. 2013 Value name: rpsi 2014 Long name: Reference Picture Selection Indication 2015 Usable with: ack, nack 2016 Reference: This document. 2018 Value name: app 2019 Long name: Application layer feedback 2020 Usable with: ack, nack 2021 Reference: This document. 2023 Further entries may be registered on a first-come first serve 2024 basis. Each registrations needs to indicate the parameter name, 2025 the syntax of possible additional arguments, and whether the 2026 parameter is applicable to "ack" or "nack" feedback or both or some 2027 different "rtcp-fb" attribute parameter. For each new 2028 registration, it is mandatory that a permanent, stable, and 2029 publicly accessible document exists that specifies the semantics of 2030 the registered parameter, the syntax and semantics of its 2031 parameters as well as corresponding feedback packet formats (if 2032 needed). The general registration procedures of [3] apply. 2034 Two RTCP Control Packet Types: for the class of transport layer 2035 feedback messages ("RTPFB") and for the class of payload-specific 2036 feedback messages ("PSFB"). Section 6 suggests RTPFB=205 and 2037 PSFB=206 to be added to the RTCP registry. 2039 RTP RTCP Control Packet types (PT): 2041 Name: RTPFB 2042 Long name: Generic RTP Feedback 2043 Value: 205 2044 Reference: This document. 2046 Name: PSFB 2047 Long name: Payload-specific 2048 Value: 206 2049 Reference: This document. 2051 As AVPF defines additional RTCP payload types, the corresponding 2052 "reserved" RTP payload type space (72--76, as defined in [2]), 2053 needs to be expanded accordingly (to cover the range 72--78). 2055 A new sub-registry needs to be set up for the FMT values for both 2056 the RTPFB payload type and the PSFB payload type, with the 2057 following registrations created initially: 2058 Within the RTPFB range, the following three format (FMT) values are 2059 initially registered: 2061 Name: Generic NACK 2062 Long name: Generic negative acknowledgement 2063 Value: 1 2064 Reference: This document. 2066 Name: Generic ACK 2067 Long name: Generic positive acknowledgement 2068 Value: 2 2069 Reference: This document. 2071 Name: Extension 2072 Long name: Reserved for future extensions 2073 Value: 31 2074 Reference: This document. 2076 Within the PSFB range, the following five format (FMT) values are 2077 initially registered: 2079 Name: PLI 2080 Long name: Picture Loss Indication 2081 Value: 1 2082 Reference: This document. 2084 Name: SLI 2085 Long name: Slice Loss Indication 2086 Value: 2 2087 Reference: This document. 2089 Name: RPSI 2090 Long name: Reference Picture Selection Indication 2091 Value: 3 2092 Reference: This document. 2094 Name: AFB 2095 Long name: Application Layer Feedback 2096 Value: 1 2097 Reference: This document. 2099 Name: Extension 2100 Long name: Reserved for future extensions. 2101 Value: 31 2102 Reference: This document. 2104 Further entries may be registered on a first-come first serve 2105 basis. Each registration needs to indicate the FMT value, if there 2106 is a specific feedback message to go into the FCI field, and 2107 whether or not multiple feedback messages may be stacked in a 2108 single FCI field. For each new registration, it is mandatory that 2109 a permanent, stable, and publicly accessible document exists that 2110 specifies the semantics of the registered parameter as well as the 2111 syntax and semantics of the associated feedback message (if any). 2112 The general registration procedures of [3] apply. 2114 10. Acknowledgements 2116 This document is a product of the Audio-Visual Transport (AVT) 2117 Working Group of the IETF. The authors would like to thank Steve 2118 Casner and Colin Perkins for their comments and suggestions as well 2119 as for their responsiveness to numerous questions. The authors 2120 would also like to thank Magnus Westerlund for his review and his 2121 valuable suggestions; Shigeru Fukunaga, Koichi Yano, Akihiro 2122 Miyazaki, and Rolf Hakenberg for the contributions on for feedback 2123 message formats and semantics; and Andreas Buesching for his 2124 feedback based upon implementation and testing. 2126 11. Full Copyright Statement 2128 Copyright (C) The Internet Society (2001). All Rights Reserved. 2129 This document and translations of it may be copied and furnished to 2130 others, and derivative works that comment on or otherwise explain 2131 it or assist in its implementation may be prepared, copied, 2132 published and distributed, in whole or in part, without restriction 2133 of any kind, provided that the above copyright notice and this 2134 paragraph are included on all such copies and derivative works. 2136 However, this document itself may not be modified in any way, such 2137 as by removing the copyright notice or references to the Internet 2138 Society or other Internet organizations, except as needed for the 2139 purpose of developing Internet standards in which case the 2140 procedures for copyrights defined in the Internet Standards process 2141 must be followed, or as required to translate it into languages 2142 other than English. 2144 The limited permissions granted above are perpetual and will not be 2145 revoked by the Internet Society or its successors or assigns. 2147 This document and the information contained herein is provided on 2148 an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 2149 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 2150 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 2151 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 2152 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." 2154 12. Authors' Addresses 2156 Joerg Ott {sip,mailto}:jo@tzi.org 2157 Uni Bremen TZI 2158 MZH 5180 2159 Bibliothekstr. 1 2160 D-28359 Bremen 2161 Germany 2163 Stephan Wenger stewe@cs.tu-berlin.de 2164 TU Berlin 2165 Sekr. FR 6-3 2166 Franklinstr. 28-29 2167 D-10587 Berlin 2168 Germany 2170 Noriyuki Sato 2171 Oki Electric Industry Co., Ltd. 2172 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 2173 Tel. +81 6 6949 5101 2174 Fax. +81 6 6949 5108 2175 Mail sato652@oki.com 2177 Carsten Burmeister 2178 Panasonic European Laboratories GmbH 2179 Monzastr. 4c, 63225 Langen, Germany 2180 Tel. +49-(0)6103-766-263 2181 Fax. +49-(0)6103-766-166 2182 Mail burmeister@panasonic.de 2184 Jose Rey 2185 Panasonic European Laboratories GmbH 2186 Monzastr. 4c, 63225 Langen, Germany 2187 Tel. +49-(0)6103-766-134 2188 Fax. +49-(0)6103-766-166 2189 Mail hakenberg@panasonic.de 2191 11. Bibliography 2193 [1] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP 2194 - A Transport Protocol for Real-time Applications," Internet 2195 Draft, draft-ietf-avt-rtp-new-11.txt, Work in Progress, 2196 November 2001. 2198 [2] H. Schulzrinne and S. Casner, "RTP Profile for Audio and Video 2199 Conferences with Minimal Control," Internet Draft draft-ietf- 2200 avt-profile-new-12.txt, November 2001. 2202 [3] M. Handley, V. Jacobson, and Colin Perkins, "SDP: Session 2203 Description Protocol", Internet Draft draft-ietf-mmusic-sdp- 2204 new-10.txt, May 2002. 2206 [4] S. Casner, "SDP Bandwidth Modifiers for RTCP Bandwidth", 2207 Internet Draft draft-ietf-avt-rtcp-bw-05.txt, November 2001. 2209 [5] C. Perkins and O. Hodson, "2354 Options for Repair of 2210 Streaming Media," RFC 2354, June 1998. 2212 [6] J. Rosenberg and H. Schulzrinne, "An RTP Payload Format for 2213 Generic Forward Error Correction,", RFC 2733, December 1999. 2215 [7] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, 2216 J.C. Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP Payload 2217 for Redundant Audio Data," RFC 2198, September 1997. 2219 [8] S. Bradner, "Key words for use in RFCs to Indicate Requirement 2220 Levels," RFC 2119, March 1997. 2222 [9] H. Schulzrinne and S. Petrack, "RTP Payload for DTMF Digits, 2223 Telephony Tones and Telephony Signals," RFC 2833, May 2000. 2225 [10] T. Turletti and C. Huitema, "RTP Payload Format for H.261 2226 Video Streams, RFC 2032, October 1996. 2228 [11] C. Bormann, L. Cline, G. Deisher, T. Gardos, C. Maciocco, D. 2229 Newell, J. Ott, G. Sullivan, S. Wenger, and C. Zhu, "RTP 2230 Payload Format for the 1998 Version of ITU-T Rec. H.263 Video 2231 (H.263+)," RFC 2429, October 1998. 2233 [12] ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology - 2234 Coding of audio-visual objects - Part2: Visual", July 2000. 2236 [13] ITU-T Recommendation H.263, "Video Coding for Low Bit Rate 2237 Communication," November 2000. 2239 [14] G. Camarillo, J. Holler, G. Eriksson, H. Schulzrinne, 2240 "Grouping of media lines in SDP," Internet Draft, draft-ietf- 2241 mmusic-fid-05.txt, Work in Progress, September 2001. 2243 [15] B. Girod, N. Faerber, "Feedback-based error control for mobile 2244 video transmission," Proceedings IEEE, Vol. 87, No. 10, pp. 2245 1707 - 1723, October, 1999. 2247 [16] M. Handley, J. Padhye, S. Floyd, J. Widmer, "TCP friendly Rate 2248 Control (TFRC): Protocol Specification," Internet Draft, 2249 draft-ietf-tsvwg-tfrc-03.txt, Work in Progress, July 2001. 2251 [17] M. Baugher, R. Blom, E. Carrarra, D. McGrew, M. Naslund, K. 2252 Norrman, D. Oran, "The Secure Real-Time Transport Protocol," 2253 Internet Draft, draft-ietf-avt-srtp-05.txt, Work in Progress, 2254 June 2002. 2256 [18] J. Rosenberg and H. Schulzrinne, "An offer/answer model with 2257 SDP," Internet Draft draft-ietf-mmusic-sdp-offer-answer- 2258 02.txt, February 2002.