idnits 2.17.1 draft-ietf-avt-forward-shifted-red-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document updates RFC4102, but the abstract doesn't seem to mention this, which it should. -- The draft header indicates that this document updates RFC2198, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 459 has weird spacing: '...al mode sw1 ...' (Using the creation date from RFC2198, updated by this document, for RFC5378 checks: 1997-06-25) -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 17, 2011) is 4659 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCXXXX' is mentioned on line 328, but not defined ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCORE Q. Xie 3 Internet-Draft TRG 4 Updates: 2198, 4102 (if approved) June 17, 2011 5 Intended status: Standards Track 6 Expires: December 19, 2011 8 Forward-shifted RTP Redundancy Payload Support 9 draft-ietf-avt-forward-shifted-red-08.txt 11 Abstract 13 This document defines a simple enhancement to support RTP sessions 14 with forward-shifted redundant encodings, i.e., redundant data sent 15 before the corresponding primary data. Forward-shifted redundancy 16 can be used to conceal losses of a large number of consecutive media 17 frames (e.g., consecutive loss of seconds or even tens of seconds of 18 media). 20 Status of this Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on December 19, 2011. 37 Copyright Notice 39 Copyright (c) 2011 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 This document may contain material from IETF Documents or IETF 53 Contributions published or made publicly available before November 54 10, 2008. The person(s) controlling the copyright in some of this 55 material may not have granted the IETF Trust the right to allow 56 modifications of such material outside the IETF Standards Process. 57 Without obtaining an adequate license from the person(s) controlling 58 the copyright in such materials, this document may not be modified 59 outside the IETF Standards Process, and derivative works of it may 60 not be created outside the IETF Standards Process, except to format 61 it for publication as an RFC or to translate it into languages other 62 than English. 64 Table of Contents 66 1. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 67 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 68 2.1. Sending Redundant Data Inband vs. Out-of-band . . . . . . 3 69 3. Allowing Forward-shifted Redundant Data . . . . . . . . . . . 4 70 4. Registration of Media Type "fwdred" . . . . . . . . . . . . . 5 71 5. Mapping Media Type Parameters into SDP . . . . . . . . . . . . 7 72 6. Usage in Offer/Answer . . . . . . . . . . . . . . . . . . . . 7 73 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 74 8. Security Considerations . . . . . . . . . . . . . . . . . . . 8 75 9. Normative References . . . . . . . . . . . . . . . . . . . . . 8 76 Appendix A. Anti-shadow Loss Concealment Using 77 Forward-shifted Redundancy . . . . . . . . . . . . . 9 78 A.1. Sender Side Operations . . . . . . . . . . . . . . . . . . 9 79 A.2. Receiver Side Operations . . . . . . . . . . . . . . . . . 11 80 A.2.1. Normal Mode Operation . . . . . . . . . . . . . . . . 11 81 A.2.2. Anti-shadow Mode Operation . . . . . . . . . . . . . . 12 82 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 13 84 1. Conventions 86 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 87 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 88 document are to be interpreted as described in RFC 2119 [RFC2119]. 90 2. Introduction 92 This document defines a simple enhancement to RFC 2198 [RFC2198] to 93 support RTP sessions with forward-shifted redundant encodings, i.e., 94 redundant data is sent before the corresponding primary data. 96 Forward-shifted redundancy can be used to conceal losses of a large 97 number of consecutive media frames (e.g., consecutive loss of seconds 98 of media). Such capability is highly desirable, especially in 99 wireless mobile communication environments where the radio signal to 100 a mobile wireless media receiver can be temporarily blocked by tall 101 buildings, mountains, tunnels, etc. In other words, the receiver 102 enters into a shadow of the radio coverage. No new data will be 103 received when the receiver is in a shadow. 105 In some extreme cases, the receiver may have to spend seconds or even 106 tens of seconds in a shadow. The traditional backward-shifted 107 redundant encoding scheme (i.e., redundant data is sent after the 108 primary data), as currently supported by RFC 2198 [RFC2198], does not 109 address such consecutive frame losses. 111 In contrast, the forward-shifted redundancy scheme allows one to 112 apply effective anti-shadow loss management at the receiver (as 113 illustrated in Appendix A), thus preventing service interruptions 114 when a mobile receiver runs into such a shadow. 116 Anti-shadow loss concealment described in this document can be 117 readily applied to the streaming of pre-recorded media. Because of 118 the need of generating the forward-shifted anti-shadow redundant 119 stream, to apply anti-shadow loss concealment to the streaming of 120 live media will require the insertion of a delay equal to or greater 121 than the amount of forward-shifting at the source of media. 123 2.1. Sending Redundant Data Inband vs. Out-of-band 125 Regardless of the direction of time shift (e.g., forward-shifting or 126 backward-shifting as in RFC 2198) or the encoding scheme (e.g., 127 Forward Error Correction (FEC), or non-FEC), there is always the 128 option of sending the redundant data and the primary data either in 129 the same RTP session (i.e., inband) or in separate RTP sessions 130 (i.e., out-of-band). There are pros and cons for either approach, as 131 outlined below. 133 Inband Approach: 135 o Pro: A single RTP session is faster to setup and easier to manage. 137 o Pro: A single RTP session presents a simpler problem for NAT/ 138 firewall traversal. 140 o Pro: Less overall overhead - one RTP/UDP/IP overhead. 142 o Con: Lack of flexibility - difficult for middle boxes such as 143 gateways to add/remove the redundant data. 145 o Con: Need more specification - special payload formats need to be 146 defined to carry the redundant data inband. 148 Out-of-band Approach: 150 o Pro: Flexibility - redundant data can be more easily added, 151 removed, or replaced by a middle box such as a gateway. 153 o Pro: Little or none specification - no new payload format is 154 needed. 156 o Con: Multiple RTP sessions may take longer to setup and more 157 complexity to manage. 159 o Con: Multiple RTP sessions NAT/firewall traverse are harder to 160 address. 162 o Con: Bigger overall overhead - more than one RTP/UDP/IP overhead. 164 It is noteworthy that the specification of inband payload formats 165 such as this and RFC 2198 does not preclude a deployment from using 166 the out-of-band approach. Rather, it gives the deployment the choice 167 to use whichever approach deemed most beneficial under a given 168 circumstance. 170 3. Allowing Forward-shifted Redundant Data 172 In RFC 2198, the timestamp offset in the additional header 173 corresponding to a redundant block is defined as a 14 bits unsigned 174 offset of timestamp relative to timestamp given in the RTP header. 175 As stated in RFC 2198: 177 "The use of an unsigned offset implies that redundant data must be 178 sent after the primary data, and is hence a time to be subtracted 179 from the current timestamp to determine the timestamp of the data 180 for which this block is the redundancy." 182 This effectively prevents RFC 2198 from being used to support 183 forward-shifted redundant blocks. 185 In order to support the use of forward-shifted redundant blocks, the 186 media type "fwdred" which allows a parameter, "forwardshift", is 187 introduced for indicating the capability and willingness of using 188 forward-shifted redundancy and the base value of timestamp forward- 189 shifting. The base value of "forwardshift" is an integer equal or 190 greater than '0' in RTP timestamp units. 192 In an RTP session which uses forward-shifted redundant encodings, the 193 timestamp of a redundant block in a received RTP packet is determined 194 as follows: 196 timestamp of redundant block = timestamp in RTP header 197 - timestamp offset in additional header 198 + forward shift base value 200 Note, generally in a forward-shifted session, the timestamp offset in 201 the additional header is set to '0'. 203 The sender MUST NOT change the contents of a packet that appears in a 204 forward shifted stream when it comes time to send it in the main 205 stream. 207 4. Registration of Media Type "fwdred" 209 (The definition is based on media type "red" defined in RFC 2198 210 [RFC2198] and RFC 4102 [RFC4102], with the addition of the 211 "forwardshift" parameter.) 213 Type names: audio, text 215 Subtype names: fwdred 217 Required parameters: 219 rate: as defined in [RFC4102]. 221 pt: as defined in [RFC4102]. 223 forwardshift: An unsigned integer can be specified as value. 225 If this parameter has a value greater than '0', it indicates 226 that the sender of this parameter will use forward shifting 227 with a base value as specified when sending out redundant data. 228 This value is in RTP timestamp units. 230 If this parameter has a value of '0', it indicates that the 231 sender of this parameter will not use forward shifting when 232 sending its redundant data, i.e., the sender will have the same 233 behaviors as defined in RFC 2198. 235 Optional parameters: 237 ptime: as defined in [RFC4102]. 239 maxptime: as defined in [RFC4102]. 241 Encoding considerations: 242 This media type is framed binary data (see RFC 4288, Section 4.8) 243 and is only defined for transfer of RTP redundant data frames 244 specified in RFC 2198. 246 Security considerations: See Section 6 "Security Considerations" of 247 RFC 2198. 249 Interoperability considerations: None. 251 Published specification: 252 RTP redundant data frame format is specified in RFC 2198. 254 Applications that use this media type: 255 It is expected that real-time audio/video and text streaming and 256 conferencing tools applications that want protection against 257 losses of a large number of consecutive frames will be interested 258 in using this type. 260 Additional information: none 262 Person & email address to contact for further information: 263 Qiaobing Xie 265 Intended usage: COMMON 266 Restrictions on usage: 267 This media type depends on RTP framing, and hence is only defined 268 for transfer via RTP (RFC 3550 [RFC3550]). Transfer within other 269 framing protocols is not defined at this time. 271 Author: 272 Qiaobing Xie 274 Change controller: 275 IETF Audio/Video Transport working group delegated from the IESG. 277 5. Mapping Media Type Parameters into SDP 279 The information carried in the media type specification has a 280 specific mapping to fields in the Session Description Protocol (SDP) 281 [RFC4566], which is commonly used to describe RTP sessions. When SDP 282 is used to specify sessions employing the forward-shifted redundant 283 format, the mapping is as follows: 285 o The media type ("audio") goes in SDP "m=" as the media name. 287 o The media subtype ("fwdred") goes in SDP "a=rtpmap" as the 288 encoding name. 290 o The required parameter "forwardshift" goes in the SDP "a=fmtp" 291 attribute by copying it directly from the media type string as 292 "forwardshift=value". 294 Example of usage of indicating forward-shifted (by 5.1 sec) 295 redundancy: 297 m=audio 12345 RTP/AVP 121 0 5 298 a=rtpmap:121 fwdred/8000/1 299 a=fmtp:121 0/5 forwardshift=40800 301 Example of usage of indicating sending redundancy without forward- 302 shifting (equivalent to RFC 2198): 304 m=audio 12345 RTP/AVP 121 0 5 305 a=rtpmap:121 fwdred/8000/1 306 a=fmtp:121 0/5 forwardshift=0 308 6. Usage in Offer/Answer 310 The "forwardshift" SDP parameter specified in this document is 311 declarative, and all reasonable values are expected to be supported. 313 7. IANA Considerations 315 RFC EDITOR'S NOTE: please replace "RFCXXXX" with the number of this 316 specification. 318 This section requests the following IANA actions: 320 o addition of the following assignment in the "Audio Media Types" 321 registry: 323 fwdred [RFCXXXX] 325 o addition of the following assignment in the "Text Media Types" 326 registry: 328 fwdred [RFCXXXX] 330 8. Security Considerations 332 Security considerations discussed in Section 6 of [RFC2198], Section 333 4 of [RFC4856], and Sections 9 and 14 of [RFC3550] apply to this 334 specification. In addition, to prevent denial of service attacks, a 335 receiver SHOULD be prepared to ignore a 'forwardshift' parameter 336 declaration if it considers the offset value in the declaration 337 excessive. In such a case, the receiver SHOULD also ignore the 338 redundant stream in the resultant RTP session. 340 9. Normative References 342 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 343 Requirement Levels", BCP 14, RFC 2119, March 1997. 345 [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., 346 Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- 347 Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, 348 September 1997. 350 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 351 Jacobson, "RTP: A Transport Protocol for Real-Time 352 Applications", RFC 3550, July 2003. 354 [RFC4102] Jones, P., "Registration of the text/red MIME Sub-Type", 355 RFC 4102, June 2005. 357 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 358 Description Protocol", RFC 4566, July 2006. 360 [RFC4856] Casner, S., "Media Type Registration of Payload Formats in 361 the RTP Profile for Audio and Video Conferences", 362 RFC 4856, March 2007. 364 Appendix A. Anti-shadow Loss Concealment Using Forward-shifted 365 Redundancy 367 (Informational) 369 It is not unusual in a wireless mobile communication environment that 370 the radio signal to a mobile wireless media receiver can be 371 temporarily blocked by tall buildings, mountains, tunnels, etc. for a 372 period of time. In other words, the receiver enters into a shadow of 373 the radio coverage. When the receiver is in such a shadow no new 374 data will be received. In some extreme cases, the receiver may have 375 to spend seconds or even tens of seconds in such a shadow. 377 Without special design considerations to compensate the loss of data 378 due to shadowing, a mobile user may experience an unacceptable level 379 of service interruptions. And traditional redundant encoding schemes 380 (including RFC 2198 and most FEC schemes) are known to be ineffective 381 in dealing with such losses of consecutive frames. 383 However, the employment of forward-shifted redundancy, in combination 384 with the anti-shadow loss concealment at the receiver, as described 385 here, can effectively prevent service interruptions due to the effect 386 of shadowing. 388 A.1. Sender Side Operations 390 For anti-shadow loss management, the RTP sender simply adds a 391 forward-shifted redundant stream (called anti-shadow or AS stream) 392 while transmitting the primary media stream. The amount of forward- 393 shifting, which should remain constant for the duration of the 394 session, will determine the maximal length of shadows that can be 395 completely concealed at the receiver, as explained below. 397 Except for the fact that it is forward-shifted relative to the 398 primary stream (i.e., the redundant data is sent ahead of the 399 corresponding primary data), the design decision and trade-offs on 400 the quality, encoding, bandwidth overhead, etc. of the redundant 401 stream is not different from the traditional RTP payload redundant 402 scheme. 404 The following diagram illustrates a segment of the transmission 405 sequence of a forward-shifted redundant RTP session, in which the AS 406 stream is forward-shifted by 155 frames. If, for simplicity here, we 407 assume the value of timestamp is incremented by 1 between two 408 consecutive frames, this forward-shifted redundancy can then be 409 indicated with: 411 forwardshift=155 413 and the setting of timestamp offset to 0 in all the additional 414 headers. This can mean a 3.1 second of forward shifting if each 415 frame represents 20 ms of original media, 417 Primary stream AS stream 419 ... | | 420 v v 421 Pkt k+8 [ 111 ] [ 266 ] 422 | | 423 v v 424 Pkt k+7 [ 110 ] [ 265 ] 425 | | 426 v v 427 ^ Pkt k+6 [ 109 ] [ 264 ] 428 | | | 429 | v v 430 Pkt k+5 [ 108 ] [ 263 ] 431 T | | 432 I v v 433 M Pkt k+4 [ 107 ] [ 262 ] 434 E | | 435 v v 436 Pkt k+3 [ 106 ] [ 261 ] 437 | | 438 v v 439 Pkt k+2 [ 105 ] [ 260 ] 440 | | 441 v v 442 Pkt k+1 [ 104 ] [ 259 ] 443 | | 444 v v 445 Pkt k [ 103 ] [ 258 ] 446 | | 447 v v 449 Transmit first 451 Figure 1. An example of forward-shifted redundant RTP packet 452 transmission. 454 A.2. Receiver Side Operations 456 The anti-shadow receiver is illustrated in the following diagram. 458 +---------+ 459 normal mode sw1 | media | media 460 Primary stream ======================o___o==>| decoder |===> output 461 AS stream ---- +---------+ device 462 | AS mode o 463 | +---------+ | 464 | | anti- | | 465 ------->| shadow |---- 466 | buffer | 467 +---------+ 468 | 469 V 470 expired frames 471 discarded 473 Figure 2. Anti-shadow RTP receiver. 475 The anti-shadow receiver operates between two modes - "normal mode" 476 and "AS mode". When the receiver is not in a shadow (i.e., when it 477 still receives new data), it operates in the normal mode. Otherwise, 478 it operates in the AS mode. 480 A.2.1. Normal Mode Operation 482 In the normal mode, after receiving a new RTP packet that contains 483 the primary data and forward-shifted AS data, the receiver passes the 484 primary data directly to the appropriate media decoder for play-out 485 (a de-jittering buffer may be used before the play-out, but for 486 simplicity we assume none is used here), while the received AS data 487 is stored in an anti-shadow buffer. 489 Moreover, data stored in the anti-shadow buffer will be continuously 490 checked to determine whether it has expired. If a redundant data in 491 the anti-shadow buffer is found to have a timestamp older (i.e., 492 smaller) than that of the last primary frame passed to the media 493 decoder, it will be considered expired and be purged from the anti- 494 shadowing buffer. 496 The following example illustrates the operation of the anti-shadow 497 buffer in normal mode. We use the same assumption as in Figure 1, 498 and assume that the initial timestamp value is 103 when the session 499 starts. 501 Timestamp Timestamp 502 Time being of media in 503 (in ms) played out AS buffer Note 504 ------------------------------------------------------------------ 505 t < 0 -- (buffer empty) 506 ... 507 t=0 103 258 (hold 1 AS frame) 508 t=20 104 258-259 (hold 2 AS frames) 509 t=40 105 258-260 (hold 3 AS frames) 511 ... 512 t=3080 257 258-412 (full, hold 154 AS frames) 513 t=3100 258 259-413 (full, frame 258 purged) 514 t=3120 259 260-414 (full, frame 259 purged) 515 ... 517 t=6240 415 416-570 (always holds 3.08 sec 518 worth of redundant data) 519 ... 521 Figure 3. Example of anti-shadow buffer operation in normal mode. 523 At the beginning of the session (t=0), the anti-shadow buffer will be 524 empty. When the first primary frame is received, the play-out will 525 start immediately, and the first received AS frame is stored in the 526 anti-shadow buffer. And with the arriving of more forward-shifted 527 redundant frames, the anti-shadow buffer will gradually be filled up. 529 For the example shown in Figure 1, after 3.08 seconds (the amount of 530 the forward-shifting minus one frame) from the start of the session, 531 the anti-shadow buffer will be full, holding exactly 3.08 seconds 532 worth of redundant data, with the oldest frame corresponding to t=3.1 533 sec and youngest frame corresponding to t=6.18 sec. 535 And it is not difficult to see that in normal mode because of the 536 continuous purge of expired frames and the addition of new frames, 537 the anti-shadowing buffer will always be full holding the next 538 forward-shift amount of redundant frames. 540 A.2.2. Anti-shadow Mode Operation 542 When the receiver enters a shadow (or any other conditions that 543 prevent the receiver from getting new media data), the receiver 544 switches to the anti-shadow mode, in which it simply continues the 545 play-out from the forward-shifted redundant data stored in the anti- 546 shadow buffer. 548 For the example in Figure 3, if the receiver enters a shadow at 549 t=3120, it can continue the play-out by using the forward-shifted 550 redundant frames (ts=260-414) from the anti-shadow buffer. As far as 551 the receiver can move out of the shadow by t=6240, there will be no 552 service interruption. 554 When the shadow condition ends (meaning new data starts to arrive 555 again), the receiver immediately switches back to normal mode of 556 operation, playing out from newly arrived primary frames. And at the 557 same time, the arrival of new AS frames will start to re-fill the 558 anti-shadow buffer. 560 However, if the duration of the shadow is longer than the amount of 561 forward-shifting, the receiver will run out of media frames from its 562 anti-shadow buffer. At that point, service interruption will occur. 564 Author's Address 566 Qiaobing Xie 567 The Resource Group 568 1700 Pennsylvania Ave. NW 569 Washington, DC 20006 570 US 572 Phone: +1-847-893-0222 573 Email: Qiaobing.Xie@gmail.com