idnits 2.17.1 draft-westin-payload-vp8-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document is more than 15 pages and seems to lack a Table of Contents. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 94: '... : OPTIONAL RTP padding ...' RFC 2119 keyword, line 98: '... in the sequel. OPTIONAL RTP padding ...' RFC 2119 keyword, line 120: '...for future use. MUST be equal to zero...' RFC 2119 keyword, line 159: '.... The PictureID SHALL start on a rand...' RFC 2119 keyword, line 160: '...aximum ID. Leading zero bytes MUST be...' (18 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 602 has weird spacing: '...or lost gol...' -- The document date (February 8, 2011) is 4824 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 2327 (ref. '7') (Obsoleted by RFC 4566) Summary: 4 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Westin 3 Internet-Draft H. Lundin 4 Intended status: Experimental M. Glover 5 Expires: August 12, 2011 J. Uberti 6 F. Galligan 7 Google 8 February 8, 2011 10 Proposal for the IETF on "RTP Payload Format for VP8 Video" 11 draft-westin-payload-vp8-00 13 Abstract 15 This memo describes an RTP Payload format for the VP8 video codec. 16 The payload format has wide applicability, as it supports 17 applications from low bit-rate peer-to-peer usage, to high bit-rate 18 Video conferences. 20 Status of this Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on August 12, 2011. 37 Copyright Notice 39 Copyright (c) 2011 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 1. Introduction 54 An encoded VP8 frame can be divided into two or more partitions, as 55 described in [1]. The first partition (prediction or mode) contains 56 prediction mode parameters and motion vectors for all macroblocks. 57 The remaining partitions all contain the DCT/WHT coefficients for the 58 residuals. The first partition is decodable without the remaining 59 residual partitions. The subsequent partitions may be useful even if 60 some part of the frame is lost. The format specification is 61 described in Section 2. Section 3 illustrates how VP8 can be 62 combined with uneven level FEC protection. Section 4 describes a 63 method to acknowledge receipt of reference frames using RTCP 64 techniques is described. Both these examples serve as motivation for 65 two of the fields included in the payload format: the "1st partition 66 size" and "PictureID" fields. 68 2. Payload Format 70 The general RTP payload format for VP8 is depicted below. 72 0 1 2 3 73 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 74 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 75 |V=2|P|X| CC |M| PT | sequence number | 76 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 77 | timestamp | 78 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 79 | synchronization source (SSRC) identifier | 80 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 81 | contributing source (CSRC) identifiers | 82 | .... | 83 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 84 | VP8 payload descriptor (integer #bytes) | 85 : : 86 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 87 | : VP8 payload header (3 octets) | 88 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 89 | VP8 pyld hdr : | 90 +-+-+-+-+-+-+-+-+ | 91 : Bytes 4..N of VP8 payload : 92 | | 93 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 94 | : OPTIONAL RTP padding | 95 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 97 The VP8 payload descriptor and VP8 payload header will be described 98 in the sequel. OPTIONAL RTP padding MUST NOT be included unless the 99 marker bit is set. 101 Figure 1 103 2.1. VP8 Payload Descriptor 105 The first bytes after the RTP header are the VP8 payload descriptor, 106 with the following structure. 108 0 1 2 3 109 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 110 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 111 | RSV |I|N|FI |B| PictureID (integer #bytes) | 112 +-+-+-+-+-+-+-+-+ | 113 : : 114 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 115 | : (VP8 data or VP8 payload header; byte aligned)| 116 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 118 Figure 2 120 RSV: Bits reserved for future use. MUST be equal to zero and MUST 121 be ignored by the receiver. 123 I: PictureID present. When set to one, a PictureID is provided 124 after the first byte of the payload descriptor. When set to zero, 125 the PictureID is omitted, and the one-byte payload descriptor is 126 immediately followed by the VP8 payload. 128 N: Non-reference frame. When set to one, the frame can be 129 discarded without affecting any other future or past frames. 131 FI: Fragmentation information field. This field contains 132 information about the fragmentation of VP8 payloads carried in the 133 RTP packet. The four different values are listed below. 135 * 00 The RTP packet contains no fragmented VP8 partitions. The 136 payload is one or several complete partitions. 138 * 01 The RTP packet contains the first part of a fragmented 139 partition. The fragment must be placed in its own RTP packet. 141 * 10 The RTP packet contains a fragment that is neither the first 142 nor the last part of a fragmented partition. The fragment must 143 be placed in its own RTP packet. 145 * 11 The RTP packet contains the last part of a fragmented 146 partition. The fragment must be placed in its own RTP packet. 148 B: Beginning VP8 frame. When set to 1 this signals that a new VP8 149 frame starts in this RTP packet. 151 PictureID: Multiple of 8 bits. This is a running index of the 152 frames. The field is present only if the I bit is equal to one. 153 The most significant bit of each byte is an extension flag. The 7 154 following bits carry (parts of) the PictureID. If the extension 155 flag is one, the PictureID continues in the next byte. If the 156 extension flag is zero, the 7 remaining bits are the last (and 157 least significant) bits in the PictureID. The sender may choose 158 any number of bytes, smaller or equal to 9 bytes as the maximum 159 PictureID. The PictureID SHALL start on a random number, and MUST 160 wrap after reaching the maximum ID. Leading zero bytes MUST be 161 supressed. 163 2.2. VP8 Payload Header 165 The first three bytes of an encoded VP8 frame are uncompressed, and 166 co-serve as payload header in this RTP format. Note that the header 167 is present only in packets which have the B bit equal to one in the 168 payload descriptor. Subsequent packets for the same frame do not 169 carry the payload header. 171 0 1 2 3 172 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 173 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 174 |P| VER |H| 1st partition size | | 175 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 176 | | 177 : Bytes 4..N of VP8 payload : 178 | | 179 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 180 | : OPTIONAL RTP padding | 181 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 183 Figure 3 185 P: Inverse key frame flag. When set to 0 the current frame is a 186 key frame. When set to 1 the current frame is an interframe. 187 Defined in [1] 189 VER: A version number as defined in [1]. 191 H: Show frame bit as defined in [1]. 193 1st partition size: 19 bits. A field containing the size of the 194 first data partition in bytes, as defined in [1]. 196 2.3. Aggregated and Fragmented Payloads 198 An encoded VP8 frame can be divided into two or more partitions, as 199 described in Section 1. The fragmentation information described in 200 Section 2.1 MUST be used to signal if any fragmentation is applied. 201 Aggregation of encoded partitions is done without explicit signaling. 202 Partitions MUST be aggregated in decoding order. An aggregation MUST 203 have exactly one payload descriptor. Aggregated partitions MUST 204 represent parts of one and the same video frame. Consequently, an 205 aggregated packet will have one or no payload header, depending on 206 whether the aggregate contains the first partition of a frame or not, 207 respectively. Note that the length of the first partition can always 208 be obtained from the first partition size parameter in the VP8 209 payload header. Fragments of encoded partitions MUST NOT be 210 aggregated. 212 2.4. Examples of VP8 RTP Stream 214 A few examples of how the VP8 RTP payload can be used are included 215 below. 217 2.4.1. Key frame in a single RTP packet 219 Marker bit = 1. I = 1. B = 1. PictureID = 17 = 0001001 binary. P 220 = 0. 222 0 1 2 3 223 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 224 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 225 | RTP Header M=1 | 226 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 227 |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|0: VER :1: 1st partition | 228 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 229 | size = L | | 230 +-+-+-+-+-+-+-+-+ | 231 | | 232 : Bytes 4..L of first VP8 partition : 233 | | 234 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 235 | | | 236 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 237 | | 238 : Remaining VP8 partitions : 239 | | 240 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 241 | : OPTIONAL RTP padding | 242 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 244 2.4.2. VP8 interframe in a single RTP packet; no PictureID 246 Marker bit = 1. I = 0. B = 1. P = 1. 248 0 1 2 3 249 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 250 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 251 | RTP Header M=1 | 252 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 253 |0 0 0 0 0 0 0 1|1: VER :1: 1st partition size = L | 254 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 255 | | 256 : Bytes 4..L of first VP8 partition : 257 | | 258 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 259 | | | 260 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 261 | | 262 : Remaining VP8 partitions : 263 | | 264 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 265 | : OPTIONAL RTP padding | 266 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 268 2.4.3. VP8 partitions in separate RTP packets 270 First RTP packet; marker bit = 0. I = 1. B = 1. PictureID = 17. 272 0 1 2 3 273 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 274 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 275 | RTP Header M=0 | 276 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 277 |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|1: VER :1: 1st partition | 278 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 279 | size = L | | 280 +-+-+-+-+-+-+-+-+ | 281 | | 282 : Bytes 4..L of first VP8 partition : 283 | | 284 | | 285 | | 286 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 288 Second RTP packet; marker bit = 1. B = 0. 290 0 1 2 3 291 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 292 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 293 | RTP Header M=1 | 294 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 295 |0 0 0 1 0 0 0 0:0 0 0 0 1 0 0 1| | 296 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 297 | | 298 : Remaining VP8 partitions : 299 | | 300 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 301 | : OPTIONAL RTP padding | 302 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 2.4.4. VP8 frame fragmented across RTP packets 306 First RTP packet; marker bit = 0. I = 1. FI = 00. B = 1. 308 0 1 2 3 309 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 310 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 311 | RTP Header M=0 | 312 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 313 |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|1: VER :1: 1st partition | 314 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 315 | size = L | | 316 +-+-+-+-+-+-+-+-+ | 317 | | 318 : Bytes 4..L of first VP8 partition : 319 | | 320 | | 321 | | 322 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 324 Second RTP packet; marker bit = 0. FI = 01. B = 0. 326 0 1 2 3 327 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 328 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 329 | RTP Header M=0 | 330 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 331 |0 0 0 1 0 0 1 0:0 0 0 0 1 0 0 1| | 332 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 333 | | 334 : First fragment of second VP8 partition : 335 | | 336 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 338 Third RTP packet; marker bit = 0. FI = 10. B = 0. 340 0 1 2 3 341 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 342 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 343 | RTP Header M=0 | 344 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 345 |0 0 0 1 0 1 0 0:0 0 0 0 1 0 0 1| | 346 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 347 | | 348 : Middle fragment of second VP8 partition : 349 | | 350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 352 Last RTP packet; marker bit = 1. FI = 11. B = 0. 354 0 1 2 3 355 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 357 | RTP Header M=1 | 358 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 359 |0 0 0 1 0 1 1 0:0 0 0 0 1 0 0 1| | 360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 361 | | 362 : Last fragment of second VP8 partition : 363 | | 364 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 365 | : OPTIONAL RTP padding | 366 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 368 2.4.5. VP8 frame with long PictureID 370 PictureID = 4711 = 01001001100111 binary (first 7 bits: 0100100, last 371 7 bits: 1100111). 373 0 1 2 3 374 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 375 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 376 | RTP Header M=1 | 377 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 378 |0 0 0 1 0 0 0 1:1 0 1 0 0 1 0 0 0 1 1 0 0 1 1 1|1: VER :1: 1st | 379 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 380 | partition size = L | | 381 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 382 | | 383 : Bytes 4..N of first VP8 frame : 384 | | 385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 387 3. Using VP8 with Uneven Level FEC Protection 389 RFC 5109 [2] specifies a payload format for generic forward error 390 correction (FEC) for RTP packets. (An errata with critical changes 391 was also published.) One salient feature of RFC 5109 is that it 392 provides "uneven level protection", ULP, which enables FEC protection 393 of parts of an RTP packet. Specifically, the first part of an RTP 394 packet can be given a stronger protection than the remaining part. 395 The special case is where only the first part of the RTP packet is 396 protected. The length of the protected part (actually the length of 397 each protection level) is chosen and changed dynamically during a 398 session. The concept of ULP FEC fits well with the VP8 video format. 399 The first partition of an encoded VP8 frame consists of context 400 variables and prediction parameters (mode and vectors), while the 401 subsequent partitions contain encoded residual information. For a 402 decoder, the residual information is not useful without the first 403 partition. However, a decoder could successfully use the information 404 in the first partition to provide good packet loss concealment, even 405 if the subsequent partitions are lost. The conclusion is that the 406 first partition deserves a higher protection factor than the 407 remaining data. By including the "first partition size" parameter in 408 the VP8 payload header(Section 2.2), the application, or even a media 409 aware network element, can apply the ULP FEC to the VP8 payloads, 410 since it can readily identify and obtain the length of the first 411 partition to which (a stronger) protection should be granted. RFC 412 5109 suggests two methods for multiplexing the media data and the FEC 413 data: using the RED payload type (RFC 2198, [3]) and using separate 414 sessions. The example below is based on the RED method, although the 415 above RTP format for VP8 does not preclude any method. Since we are 416 only interested in protecting the first partition of the VP8 payload, 417 it is in some cases possible to obtain increased robustness for that 418 part even without FEC signaling. If the first partition is isolated 419 in a separate RTP packet, this packet can simply be sent twice (exact 420 replica of RTP header and payload). 422 3.1. Example: Using VP8 with Uneven Level FEC Protection 424 The following terminology is used in the examples below: 426 o RTP: RTP header, 12 octets. 427 (Special rules apply if the header is extended; see [2].) 429 o RED: Redundancy payload header, 1 or 4 octets. 431 o VP8pd: VP8 payload descriptor, 1 or more octets (see Section 2.1). 433 o VP8ph: VP8 payload header, 3 octets (see Section 2.2)). 435 o VP8first: First partition of a VP8 frame, length given in VP8ph. 437 o VP8second: Second (or later) partition of a VP8 frame. 439 o FEC: FEC header for FEC packets, 10 octets (see [2]). 441 o ULP: FEC level header for FEC packets, 2 or 4 octets (see [2]). 443 3.1.1. VP8 first partition isolated 445 The first partition of the VP8 frame is sent in its own RTP packet, 446 which can easily be duplicated for increased robustness. This case 447 provides a 50% protection factor (ratio of number of FEC packet to 448 the sum of FEC and protected packets; sending two duplicates provides 449 a 67% protection, and so on). 451 +-----+-------+------------------+ 452 Packet A: | RTP | VP8pd | VP8ph : VP8first | 453 +-----+-------+------------------+ 455 +-----+-------+------------------------+ 456 Packet B: | RTP | VP8pd | VP8second | 457 +-----+-------+------------------------+ 459 +-----+-------+------------------+ 460 Duplicate A: | RTP | VP8pd | VP8ph : VP8first | 461 +-----+-------+------------------+ 463 3.1.2. VP8 first partition split 465 The first partition of the VP8 frame can be split across two (or 466 more) RTP packets. It must be done if the first partition is larger 467 than the MTU, but can also be done to facilitate protection factors 468 lower than 50%. The FEC header is calculated from the RTP headers of 469 packets A and B, while the FEC payload FEC(A,B) is constructed from 470 the VP8 payload descriptor, payload header and first partition, as 471 indicated in the figure below. 473 +-----+-----+-------+------------------+ 474 A: | RTP | RED | VP8pd | VP8ph : VP8first | 475 +-----+-----+-------+------------------+ 476 [*****] [**************************] 477 +-----+-----+-------+------------------+ 478 B: | RTP | RED | VP8pd | VP8first cont'd | 479 +-----+-----+-------+------------------+ 480 [*****] [**************************] 481 \ \ 482 ---------- --------- 483 \ \ 484 [*****] [**************************] 485 +-----+-----+-----+-----+--------------------------+ 486 FEC1: | RTP | RED | FEC | ULP | FEC(A,B) | 487 +-----+-----+-----+-----+--------------------------+ 489 +-----+-----+-------+---------------------------+ 490 C: | RTP | RED | VP8pd | VP8 second | 491 +-----+-----+-------+---------------------------+ 493 The split of the first partition is preferably done such that the 494 payloads used to calculate FEC(A,B) are close to equal size. 496 3.1.3. VP8 partitions aggregated 498 In the case when the first partition is sent in the same packet as 499 one or more subsequent partitions, the level protection can be 500 applied to facilitate a bit- conservative protection for only the 501 first partition. 503 +-----+-----+-------+------------------+--------------------+ 504 A: | RTP | RED | VP8pd | VP8ph : VP8first | VP8second | 505 +-----+-----+-------+------------------+--------------------+ 506 [*****] [**************************] 507 \ \ 508 ---------- --------- 509 \ \ 510 [*****] [**************************] 511 +-----+-----+-----+-----+--------------------------+ 512 FEC1: | RTP | RED | FEC | ULP | Duplicate | 513 +-----+-----+-----+-----+--------------------------+ 515 +-----+-----+-------+---------------------------+ 516 C: | RTP | RED | VP8pd | VP8third | 517 +-----+-----+-------+---------------------------+ 519 4. Using VP8 with RPSI and SLI Feedback 521 The VP8 payload descriptor defined in Section 2.1 above contains an 522 optional PictureID parameter. This parameter is included mainly to 523 enable use of reference picture selection index (RPSI) and slice loss 524 indication (SLI), both defined in RFC 4585 [4]. 526 4.1. RPSI 528 The reference picture selection index is a payload-specific feedback 529 message defined within the RTCP-based feedback format. The RPSI 530 message is generated by a receiver and can be used in two ways. 531 Either it can signal a preferred reference picture when a loss has 532 been detected by the decoder -- preferably then a reference that the 533 decoder knows is perfect -- or, it can be used as positive feedback 534 information to acknowledge correct decoding of certain reference 535 pictures. The positive feedback method is useful for VP8 used as 536 unicast. The use of RPSI for VP8 is preferably combined with a 537 special update pattern of the codec's two special reference frames -- 538 the golden frame and the altref frame -- in which they are updated in 539 an alternating leapfrog fashion. When a receiver has received and 540 correctly decoded a golden or altref frame, and that frame had a 541 PictureID in the payload descriptor, the receiver can acknowledge 542 this simply by sending an RPSI message back to the sender. The 543 message body (i.e., the "native RPSI bit string" in RFC 4585 [4]) is 544 simply the PictureID of the received frame. 546 4.2. SLI 548 The slice loss indication is another payload-specific feedback 549 message defined within the RTCP-based feedback format. The SLI 550 message is generated by the receiver when a loss or corruption is 551 detected in a frame. The format of the SLI message is as follows 552 [4]: 554 0 1 2 3 555 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 556 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 557 | First | Number | PictureID | 558 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 560 Figure 4 562 Here, First is the macroblock address (in scan order) of the first 563 lost block and Number is the number of lost blocks. PictureID is the 564 six least significant bits of the codec-specific picture identifier 565 in which the loss or corruption has occurred. For VP8, this codec- 566 specific identifier is naturally the PictureID of the current frame, 567 as read from the payload descriptor. If the payload descriptor of 568 the current frame does not have a PictureID, the receiver MAY send 569 the last received PictureID+1 in the SLI message. The receiver MAY 570 set the First parameter to 0, and the Number parameter to the total 571 number of macroblocks per frame, even though only parts of the frame 572 is corrupted. When the sender receives an SLI message, it can make 573 use of the knowledge from the latest received RPSI message. Knowing 574 that the last golden or altref frame was successfully received, it 575 can encode the next frame with reference to that established 576 reference. 578 4.3. Example 580 The use of RSPI and SLI is best illustrated in an example. In this 581 example, the encoder may not update the altref frame until the last 582 sent golden frame has been acknowledged with an RPSI message. If an 583 update is not received within some time, a new golden frame update is 584 sent instead. Once the new golden frame is established and 585 acknowledge, the same rule applies when updating the altref frame. 587 Event Sender Receiver Established 588 reference 589 +----+--------------------+--------------------------+------------+ 590 1000 Send golden frame 591 PictureID = 0 592 Receive and decode 593 golden frame 595 1001 Receive RPSI(0) 596 Send RPSI(0) golden 598 ... (sending regular frames) 600 1100 Send altref frame 601 PictureID = 100 602 Altref corrupted or lost golden 604 1101 Receive SLI(100) 605 Send SLI(100) golden 607 1102 Send frame with 608 reference to golden 609 Receive and decode frame 610 (decoder state restored) golden 612 ... (sending regular frames) 614 1200 Send altref frame 615 PictureID = 200 616 Receive and decode 617 altref frame golden 619 1201 Receive RPSI(200) 620 Send RPSI(200) altref 622 ... (sending regular frames) 624 1300 Send golden frame 625 PictureID = 300 626 Receive and decode 627 golden frame altref 629 1301 RPSI lost 630 Send RPSI(300) altref 632 1400 Send golden frame 633 PictureID = 400 634 Receive and decode 635 golden frame altref 637 1401 Receive RPSI(400) 638 Send RPSI(400) golden 639 +----+--------------------+--------------------------+------------+ 641 Note that the scheme is robust to loss of the feedback messages. If 642 the RPSI is lost, the sender will try to update the golden (or 643 altref) again after a while, without releasing the established 644 reference. Also, if an SLI is lost, the receiver can keep sending 645 SLI messages at any interval, as long as the picture is corrupted. 647 5. Payload Format Parameters 649 This section specifies the parameters that MAY be used to select 650 optional features of the payload format and certain features of the 651 bitstream. 653 5.1. MIME Registration 655 The receiver MUST ignore any unspecified parameter. 657 Media Type name: video 659 Media subtype name: VP8 661 Required parameters: none 663 Security considerations: 664 - See Section 6 of RFC xxxx. 666 5.2. SDP Parameters 668 The receiver MUST ignore any parameter unspecified in this memo. 670 5.2.1. Mapping of MIME Parameters to SDP 672 The MIME media type video/VP8 string is mapped to fields in the 673 Session Description Protocol (SDP) [7] as follows: 675 o The media name in the "m=" line of SDP MUST be video. 677 o The encoding name in the "a=rtpmap" line of SDP MUST be VP8 (the 678 MIME subtype). 680 o The clock rate in the "a=rtpmap" line MUST be 90000. 682 o The OPTIONAL parameter "version", if included, MUST be in the 683 a=fmtp SDP field. This parameter matches the VP8 bitstream 684 version. 686 5.3. Example 688 An example of media representation in SDP is as follows: 690 m=video 49170 RTP/AVP 98 691 a=rtpmap:98 VP8/90000 692 a=fmtp:98 version=0 694 6. Security Considerations 696 RTP packets using the payload format defined in this specification 697 are subject to the security considerations discussed in the RTP 698 specification [5]. This implies that confidentiality of the media 699 streams is achieved by encryption; one example of this is SRTP [6]. 701 7. IANA Considerations 703 The IANA is requested to register the following values: 704 - MIME registration as described in Section 5.1. 706 8. References 708 [1] Google, Inc., "VP8 Data Format and Decoding Guide", July 2010, 709 . 711 [2] Perkins, C., Kouvelas, U., Hodson, O., Hardman, V., Handley, M., 712 Bolot, J., Vega-Garcia, A., and S. Fosse-Parisis, "RTP Payload 713 Format for Generic Forward Error Correction", RFC 5109, STD 1, 714 December 2007. 716 [3] Li, A., "RTP Payload for Redundant Audio Data", RFC 2198, STD 1, 717 September 1997. 719 [4] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 720 "Extended RTP Profile for Real-time Transport Control Protocol 721 (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, STD 1, July 2006. 723 [5] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, 724 "RTP: A Transport Protocol for Real-Time Applications", 725 RFC 3550, STD 64, July 2003. 727 [6] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 728 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 729 RFC 3711, STD 1, March 2004. 731 [7] Handley, M. and V. Jacobson, "SDP: Session Description 732 Protocol", RFC 2327, STD 1, April 1998. 734 Authors' Addresses 736 Patrik Westin 737 Google, Inc. 738 Kungsbron 2 739 Stockholm, 11122 740 Sweden 742 Email: patrik.westin@gmail.com 744 Henrik Lundin 745 Google, Inc. 747 Michael Glover 748 Google, Inc. 750 Justin Uberti 751 Google, Inc. 753 Frank Galligan 754 Google, Inc.