idnits 2.17.1 draft-ietf-avt-forward-shifted-red-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 2) being 65 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == The 'Updates: ' line in the draft header should list only the _numbers_ of the RFCs which will be updated by this document (if approved); it should not include the word 'RFC' in the list. -- The draft header indicates that this document updates RFC4102, but the abstract doesn't seem to mention this, which it should. -- The draft header indicates that this document updates RFC2198, but the abstract doesn't seem to directly say this. It does mention RFC2198 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 415 has weird spacing: '...al mode sw1 ...' == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). (Using the creation date from RFC2198, updated by this document, for RFC5378 checks: 1997-06-25) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (January 21, 2010) is 5209 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Audio Video Transport WG Q. Xie 3 Internet-Draft TRG 4 Updates: RFC 2198, RFC 4102 January 21, 2010 5 (if approved) 6 Intended status: Standards Track 7 Expires: July 25, 2010 9 Forward-shifted RTP Redundancy Payload Support 10 draft-ietf-avt-forward-shifted-red-04.txt 12 Abstract 14 This document defines a simple enhancement to RFC 2198 to support RTP 15 sessions with forward-shifted redundant encodings, i.e., redundant 16 data sent before the corresponding primary data. Forward-shifted 17 redundancy can be used to conceal losses of a large number of 18 consecutive media frames (e.g., consecutive loss of seconds or even 19 tens of seconds of media). 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt. 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html. 42 This Internet-Draft will expire on July 25, 2010. 44 Copyright Notice 46 Copyright (c) 2010 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the BSD License. 59 Table of Contents 61 1. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 62 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2.1. Sending Redundant Data Inband vs. Out-of-band . . . . . . 3 64 3. Allowing Forward-shifted Redundant Data . . . . . . . . . . . 4 65 4. Registration of Media Type "fwdred" . . . . . . . . . . . . . 5 66 5. Mapping Media Type Parameters into SDP . . . . . . . . . . . . 6 67 6. Usage in Offer/Answer . . . . . . . . . . . . . . . . . . . . 7 68 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 69 8. Security Considerations . . . . . . . . . . . . . . . . . . . 7 70 9. Normative References . . . . . . . . . . . . . . . . . . . . . 7 71 Appendix A. Anti-shadow Loss Concealment Using 72 Forward-shifted Redundancy . . . . . . . . . . . . . 8 73 A.1. Sender Side Operations . . . . . . . . . . . . . . . . . . 8 74 A.2. Receiver Side Operations . . . . . . . . . . . . . . . . . 10 75 A.2.1. Normal Mode Operation . . . . . . . . . . . . . . . . 11 76 A.2.2. Anti-shadow Mode Operation . . . . . . . . . . . . . . 12 77 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 13 79 1. Conventions 81 The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, 82 SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL, when 83 they appear in this document, are to be interpreted as described in 84 RFC 2119 [RFC2119]. 86 2. Introduction 88 This document defines a simple enhancement to RFC 2198 [RFC2198] to 89 support RTP sessions with forward-shifted redundant encodings, i.e., 90 redundant data is sent before the corresponding primary data. 92 Forward-shifted redundancy can be used to conceal losses of a large 93 number of consecutive media frames (e.g., consecutive loss of seconds 94 of media). Such capability is highly desirable, especially in 95 wireless mobile communication environments where the radio signal to 96 a mobile wireless media receiver can be temporarily blocked by tall 97 buildings, mountains, tunnels, etc. In other words, the receiver 98 enters into a shadow of the radio coverage. No new data will be 99 received when the receiver is in a shadow. 101 In some extreme cases, the receiver may have to spend seconds or even 102 tens of seconds in a shadow. The traditional backward-shifted 103 redundant encoding scheme (i.e., redundant data is sent after the 104 primary data), as currently supported by RFC 2198 [RFC2198], is known 105 to be ineffective in dealing with such consecutive frame losses. 107 In contrast, the forward-shifted redundancy, when used in combination 108 with the anti-shadow loss management at the receiver (as described in 109 Appendix A), can effectively prevent service interruptions when a 110 mobile receiver runs into such a shadow. 112 2.1. Sending Redundant Data Inband vs. Out-of-band 114 Regardless of the direction of time shift (e.g., forward-shifting or 115 backward-shifting as in RFC 2198) or the encoding scheme (e.g., FEC, 116 or non-FEC), there is always the option of sending the redundant data 117 and the primary data either in the same RTP session (i.e., inband) or 118 in separate RTP sessions (i.e., out-of-band). There are pros and 119 cons for either approach, as outlined below. 121 Inband Approach: 123 o Pro: A single RTP session is faster to setup and easier to manage. 125 o Pro: A single RTP session presents a simpler problem for NAT/ 126 firewall traversal. 128 o Pro: Less overall overhead - one RTP/UDP/IP overhead. 130 o Con: Lack of flexibility - difficult for middle boxes such as 131 gateways to add/remove the redundant data. 133 o Con: Need more specification - special payload formats need to be 134 defined to carry the redundant data inband. 136 Out-of-band Approach: 138 o Pro: Flexibility - redundant data can be more easily added, 139 removed, or replaced by a middle box such as a gateway. 141 o Pro: Little or none specification - no new payload format is 142 needed. 144 o Con: Multiple RTP sessions may take longer to setup and more 145 complexity to manage. 147 o Con: Multiple RTP sessions NAT/firewall traverse are harder to 148 address. 150 o Con: Bigger overall overhead - more than one RTP/UDP/IP overhead. 152 It is noteworthy that the specification of inband payload formats 153 such as this and RFC 2198 does not preclude a deployment from using 154 the out-of-band approach. Rather, it gives the deployment the choose 155 to use whichever approach deemed most beneficiary under a given 156 circumstance. 158 3. Allowing Forward-shifted Redundant Data 160 In RFC 2198, the timestamp offset in the additional header 161 corresponding to a redundant block is defined as a 14 bits unsigned 162 offset of timestamp relative to timestamp given in the RTP header. 163 As stated in RFC 2198: 165 "The use of an unsigned offset implies that redundant data must be 166 sent after the primary data, and is hence a time to be subtracted 167 from the current timestamp to determine the timestamp of the data 168 for which this block is the redundancy." 170 This effectively prevents RFC 2198 from being used to support 171 forward-shifted redundant blocks. 173 In order to support the use of forward-shifted redundant blocks, the 174 media type "fwdred" which allows an optional MIME parameter, 175 "forwardshift", is introduced for indicating the capability and 176 willingness of using forward-shifted redundancy and the base value of 177 timestamp forward-shifting. The base value of "forwardshift" is an 178 integer equal or greater than '0' in RTP timestamp units. 180 In an RTP session which uses forward-shifted redundant encodings, the 181 timestamp of a redundant block in a received RTP packet is determined 182 as follows: 184 timestamp of redundant block = timestamp in RTP header 185 - timestamp offset in additional header 186 + forward shift base value 188 Note, generally in a forward-shifted session, the timestamp offset in 189 the additional header SHOULD be set to '0'. 191 4. Registration of Media Type "fwdred" 193 (The definition is based on media type "red" defined in RFC 2198 194 [RFC2198] and RFC 4102 [RFC4102], with the addition of the optional 195 "forwardshift" parameter.) 197 Type names: audio, text 199 Subtype names: fwdred 201 Required parameters: none 203 Optional parameters: 205 forwardshift: An unsigned integer can be specified as value. 207 If this parameter is present with a value greater than '0', it 208 indicates that the sender of this parameter will use forward 209 shifting with a base value as specified when sending out 210 redundant data. This value is in RTP timestamp units. 212 If this parameter is absent or present with a value of '0', it 213 indicates that the sender of this parameter will not use 214 forward shifting when sending its redundant data, i.e., the 215 sender will have the same behaviors as defined in RFC 2198. 217 Encoding considerations: 218 This media type is framed binary data (see RFC 4288, Section 4.8) 219 and is only defined for transfer of RTP redundant data frames 220 specified in RFC 2198. 222 Security considerations: See Section 6 "Security Considerations" of 223 RFC 2198. 225 Interoperability considerations: None. 227 Published specification: 228 RTP redundant data frame format is specified in RFC 2198. 230 Applications that use this media type: 231 It is expected that real-time audio/video and text streaming and 232 conferencing tools applications that want protection against 233 losses of a large number of consecutive frames will be interested 234 in using this type. 236 Additional information: none 238 Person & email address to contact for further information: 239 Qiaobing Xie 241 Intended usage: COMMON 243 Restrictions on usage: 244 This media type depends on RTP framing, and hence is only defined 245 for transfer via RTP (RFC 3550 [RFC3550]). Transfer within other 246 framing protocols is not defined at this time. 248 Author: 249 Qiaobing Xie 251 Change controller: 252 IETF Audio/Video Transport working group delegated from the IESG. 254 5. Mapping Media Type Parameters into SDP 256 The information carried in the MIME media type specification has a 257 specific mapping to fields in the Session Description Protocol (SDP) 258 [RFC4566], which is commonly used to describe RTP sessions. When SDP 259 is used to specify sessions employing the forward-shifted redundant 260 format, the mapping is as follows: 262 o The media type ("audio") goes in SDP "m=" as the media name. 264 o The media subtype ("fwdred") goes in SDP "a=rtpmap" as the 265 encoding name. 267 o The optional parameter "forwardshift" goes in the SDP "a=fmtp" 268 attribute by copying it directly from the MIME media type string 269 as "forwardshift=value". 271 Example of usage of indicating forward-shifted (by 5.1 sec) 272 redundancy: 274 m=audio 12345 RTP/AVP 121 0 5 275 a=rtpmap:121 fwdred/8000/1 276 a=fmtp:121 0/5 forwardshift=40800 278 Example of usage of indicating sending redundancy without forward- 279 shifting (equivalent to RFC 2198): 281 m=audio 12345 RTP/AVP 121 0 5 282 a=rtpmap:121 fwdred/8000/1 283 a=fmtp:121 0/5 forwardshift=0 285 6. Usage in Offer/Answer 287 The optional "forwardshift" SDP parameter specified in this document 288 is declarative, and all reasonable values are expected to be 289 supported. 291 7. IANA Considerations 293 The registration of the new MIME subtype "fwdred", as described in 294 Section 4, is required. 296 8. Security Considerations 298 See Section 6 "Security Considerations" of RFC 2198 [RFC2198]. In 299 addition, since an excessive forwardshift value can be signalled, as 300 a denial of service, a receive SHOULD be prepared to ignore 301 outrageous values. 303 9. Normative References 305 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 306 Requirement Levels", BCP 14, RFC 2119, March 1997. 308 [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., 309 Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- 310 Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, 311 September 1997. 313 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 314 Jacobson, "RTP: A Transport Protocol for Real-Time 315 Applications", RFC 3550, July 2003. 317 [RFC4102] Jones, P., "Registration of the text/red MIME Sub-Type", 318 RFC 4102, June 2005. 320 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 321 Description Protocol", RFC 4566, July 2006. 323 Appendix A. Anti-shadow Loss Concealment Using Forward-shifted 324 Redundancy 326 It is not unusual in a wireless mobile communication environment that 327 the radio signal to a mobile wireless media receiver can be 328 temporarily blocked by tall buildings, mountains, tunnels, etc. for a 329 period of time. In other words, the receiver enters into a shadow of 330 the radio coverage. When the receiver is in such a shadow no new 331 data will be received. In some extreme cases, the receiver may have 332 to spend seconds or even tens of seconds in such a shadow. 334 Without special design considerations to compensate the loss of data 335 due to shadowing, a mobile user may experience an unacceptable level 336 of service interruptions. And traditional redundant encoding schemes 337 (including RFC 2198 and most FEC schemes) are known to be ineffective 338 in dealing with such losses of consecutive frames. 340 However, the employment of forward-shifted redundancy, in combination 341 with the anti-shadow loss concealment at the receiver, as described 342 here, can effectively prevent service interruptions due to the effect 343 of shadowing. 345 A.1. Sender Side Operations 347 For anti-shadow loss management, the RTP sender simply adds a 348 forward-shifted redundant stream (called anti-shadow or AS stream) 349 while transmitting the primary media stream. The amount of forward- 350 shifting, which should remain constant for the duration of the 351 session, will determine the maximal length of shadows that can be 352 completely concealed at the receiver, as explained below. 354 Except for the fast that it is forward-shifted relative to the 355 primary stream (i.e., the redundant data is sent ahead of the 356 corresponding primary data), the design decision and trade-offs on 357 the quality, encoding, bandwidth overhead, etc. of the redundant 358 stream is not different from the traditional RTP payload redundant 359 scheme. 361 The following diagram illustrates a segment of the transmission 362 sequence of a forward-shifted redundant RTP session, in which the AS 363 stream is forward-shifted by 155 frames. If, for simplicity here, we 364 assume the value of timestamp is incremented by 1 between two 365 consecutive frames, this forward-shifted redundancy can then be 366 indicated with: 368 forwardshift=155 370 and the setting of timestamp offset to 0 in all the additional 371 headers. This can mean a 3.1 second of forward shifting if each 372 frame represents 20 ms of original media, 373 Primary stream AS stream 375 ... | | 376 v v 377 Pkt k+8 [ 111 ] [ 266 ] 378 | | 379 v v 380 Pkt k+7 [ 110 ] [ 265 ] 381 | | 382 v v 383 ^ Pkt k+6 [ 109 ] [ 264 ] 384 | | | 385 | v v 386 Pkt k+5 [ 108 ] [ 263 ] 387 T | | 388 I v v 389 M Pkt k+4 [ 107 ] [ 262 ] 390 E | | 391 v v 392 Pkt k+3 [ 106 ] [ 261 ] 393 | | 394 v v 395 Pkt k+2 [ 105 ] [ 260 ] 396 | | 397 v v 398 Pkt k+1 [ 104 ] [ 259 ] 399 | | 400 v v 401 Pkt k [ 103 ] [ 258 ] 402 | | 403 v v 405 Transmit first 407 Figure 1. An example of forward-shifted redundant RTP packet 408 transmission. 410 A.2. Receiver Side Operations 412 The anti-shadow receiver is illustrated in the following diagram. 414 +---------+ 415 normal mode sw1 | media | media 416 Primary stream ======================o___o==>| decoder |===> output 417 AS stream ---- +---------+ device 418 | AS mode o 419 | +---------+ | 420 | | anti- | | 421 ------->| shadow |---- 422 | buffer | 423 +---------+ 424 | 425 V 426 expired frames 427 discarded 429 Figure 2. Anti-shadow RTP receiver. 431 The anti-shadow receiver operates between two modes - "normal mode" 432 and "AS mode". When the receiver is not in a shadow (i.e., when it 433 still receives new data), it operates in the normal mode. Otherwise, 434 it operates in the AS mode. 436 A.2.1. Normal Mode Operation 438 In the normal mode, after receiving a new RTP packet that contains 439 the primary data and forward-shifted AS data, the receiver passes the 440 primary data directly to the appropriate media decoder for play-out 441 (a de-jittering buffer may be used before the play-out, but for 442 simplicity we assume none is used here), while the received AS data 443 is stored in an anti-shadow buffer. 445 Moreover, data stored in the anti-shadow buffer will be continuously 446 checked to determine whether it has expired. If a redundant data in 447 the anti-shadow buffer is found to have a timestamp older (i.e., 448 smaller) than that of the last primary frame passed to the media 449 decoder, it will be considered expired and be purged from the anti- 450 shadowing buffer. 452 The following example illustrates the operation of the anti-shadow 453 buffer in normal mode. We use the same assumption as in Figure 1, 454 and assume that the initial timestamp value is 103 when the session 455 starts. 457 Timestamp Timestamp 458 Time being of media in 459 (in ms) played out AS buffer Note 460 ------------------------------------------------------------------ 461 t < 0 -- (buffer empty) 462 ... 463 t=0 103 258 (hold 1 AS frame) 464 t=20 104 258-259 (hold 2 AS frames) 465 t=40 105 258-260 (hold 3 AS frames) 467 ... 468 t=3080 257 258-412 (full, hold 154 AS frames) 469 t=3100 258 259-413 (full, frame 258 purged) 470 t=3120 259 260-414 (full, frame 259 purged) 471 ... 473 t=6240 415 416-570 (always holds 3.08 sec 474 worth of redundant data) 475 ... 477 Figure 3. Example of anti-shadow buffer operation in normal mode. 479 At the beginning of the session (t=0), the anti-shadow buffer will be 480 empty. When the first primary frame is received, the play-out will 481 start immediately, and the first received AS frame is stored in the 482 anti-shadow buffer. And with the arriving of more forward-shifted 483 redundant frames, the anti-shadow buffer will gradually be filled up. 485 For the example shown in Figure 1, after 3.08 seconds (the amount of 486 the forward-shifting minus one frame) from the start of the session, 487 the anti-shadow buffer will be full, holding exactly 3.08 seconds 488 worth of redundant data, with the oldest frame corresponding to t=3.1 489 sec and youngest frame corresponding to t=6.18 sec. 491 And it is not difficult to see that in normal mode because of the 492 continuous purge of expired frames and the addition of new frames, 493 the anti-shadowing buffer will always be full holding the next 494 forward-shift amount of redundant frames. 496 A.2.2. Anti-shadow Mode Operation 498 When the receiver enters a shadow (or any other conditions that 499 prevent the receiver from getting new media data), the receiver 500 switches to the anti-shadow mode, in which it simply continues the 501 play-out from the forward-shifted redundant data stored in the anti- 502 shadow buffer. 504 For the example in Figure 3, if the receiver enters a shadow at 505 t=3120, it can continue the play-out by using the forward-shifted 506 redundant frames (ts=260-414) from the anti-shadow buffer. As far as 507 the receiver can move out of the shadow by t=6240, there will be no 508 service interruption. 510 When the shadow condition ends (meaning new data starts to arrive 511 again), the receiver immediately switches back to normal mode of 512 operation, playing out from newly arrived primary frames. And at the 513 same time, the arrival of new AS frames will start to re-fill the 514 anti-shadow buffer. 516 However, if the duration of the shadow is longer than the amount of 517 forward-shifting, the receiver will run out of media frames from its 518 anti-shadow buffer. At that point, service interruption will occur. 520 Anti-shadow loss concealment described above can be readily applied 521 to the streaming of pre-recorded media. Because of the need of 522 generating the forward-shifted anti-shadow redundant stream, to apply 523 anti-shadow loss concealment to the streaming of live media will 524 require the insertion of a delay equal to or greater than the amount 525 of forward-shifting at the source of media. 527 Author's Address 529 Qiaobing Xie 530 The Resource Group 531 1700 Pennsylvania Ave. NW 532 Washington, DC 20006 533 US 535 Phone: +1-314-420-8248 536 Email: Qiaobing.Xie@gmail.com