idnits 2.17.1 draft-westin-payload-vp8-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document is more than 15 pages and seems to lack a Table of Contents. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 523 has weird spacing: '...or lost gol...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: An encoded VP8 frame can be divided into two or more partitions, as described in Section 1. The fragmentation information described in Section 4.1 MUST be used to signal if any fragmentation is applied. Aggregation of encoded partitions is done without explicit signaling. Partitions MUST be aggregated in decoding order. Two fragments from different partitions MUST not be agregated into the same packet. An aggregation MUST have exactly one payload descriptor. Aggregated partitions MUST represent parts of one and the same video frame. Consequently, an aggregated packet will have one or no payload header, depending on whether the aggregate contains the first partition of a frame or not, respectively. Note that the length of the first partition can always be obtained from the first partition size parameter in the VP8 payload header. Fragments of encoded partitions MUST NOT be aggregated. -- The document date (March 8, 2011) is 4797 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: 'RFC3550' on line 577 ** Obsolete normative reference: RFC 2327 (ref. '6') (Obsoleted by RFC 4566) ** Obsolete normative reference: RFC 4288 (ref. '7') (Obsoleted by RFC 6838) Summary: 4 errors (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Westin 3 Internet-Draft H. Lundin 4 Intended status: Experimental M. Glover 5 Expires: September 9, 2011 J. Uberti 6 F. Galligan 7 Google 8 March 8, 2011 10 Proposal for the IETF on "RTP Payload Format for VP8 Video" 11 draft-westin-payload-vp8-02 13 Abstract 15 This memo describes an RTP payload format for the VP8 video codec. 16 The payload format has wide applicability, as it supports 17 applications from low bit-rate peer-to-peer usage, to high bit-rate 18 video conferences. 20 Status of this Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on September 9, 2011. 37 Copyright Notice 39 Copyright (c) 2011 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 1. Introduction 54 This memo describes an RTP payload specification applicable to the 55 transmission of video streams encoded using the VP8 video codec [1]. 56 The format described in this document can be used both in peer-to- 57 peer and video conferencing applications. 59 The VP8 codec uses three different reference frames for interframe 60 prediction: the previous frame, the golden frame, and the altref 61 frame. The payload specification in this memo has elements that 62 enable advanced use of the reference frames, e.g., for improved loss 63 robustness. 65 Another property of the VP8 codec is that it applies data 66 partitioning to the encoded data. Thus, an encoded VP8 frame can be 67 divided into two or more partitions, as described in "VP8 Data Format 68 and Decoding Guide" [1]. The first partition (prediction or mode) 69 contains prediction mode parameters and motion vectors for all 70 macroblocks. The remaining partitions all contain the DCT/WHT 71 coefficients for the residuals. The first partition is decodable 72 without the remaining residual partitions. The subsequent partitions 73 may be useful even if some part of the frame is lost. This memo 74 allows the partitions to be sent in the same RTP packet. 75 Nevertheless, it may be beneficial for decoder error-concealment to 76 use separate packets for the two partition types, even though it is 77 not mandatory according to this specification. 79 The format specification is described in Section 4. Section 5 80 describes a method to acknowledge receipt of reference frames using 81 RTCP techniques is described. Both these examples serve as 82 motivation for two of the fields included in the payload format: the 83 "1st partition size" and "PictureID" fields. 85 2. Conventions, Definitions and Acronyms 87 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 88 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 89 document are to be interpreted as described in RFC 2119 [2]. 91 3. Media Format Background 93 VP8 is based on decomposition of frames into square subblocks of 94 pixels, prediction of such subblocks using previously constructed 95 blocks, and adjustment of such predictions (as well as synthesis of 96 unpredicted blocks) using a discrete cosine transform (hereafter 97 abbreviated as DCT). In one special case, however, VP8 uses a 98 "Walsh-Hadamard" (hereafter abbreviated as WHT) transform instead of 99 a DCT. An encoded VP8 frame is divided into two or more partitions, 100 as described in [1]. The first partition (prediction or mode) 101 contains prediction mode parameters and motion vectors for all 102 macroblocks. The remaining partitions all contain the DCT/WHT 103 coefficients for the residuals. 105 4. Payload Format 107 The general RTP payload format for VP8 is depicted below. 109 0 1 2 3 110 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 111 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 112 |V=2|P|X| CC |M| PT | sequence number | 113 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 114 | timestamp | 115 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 116 | synchronization source (SSRC) identifier | 117 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 118 | contributing source (CSRC) identifiers | 119 | .... | 120 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 121 | VP8 payload descriptor (integer #bytes) | 122 : : 123 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 124 | : VP8 payload header (3 octets) | 125 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 126 | VP8 pyld hdr : | 127 +-+-+-+-+-+-+-+-+ | 128 : Bytes 4..N of VP8 payload : 129 | | 130 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 131 | : OPTIONAL RTP padding | 132 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 134 The VP8 payload descriptor and VP8 payload header will be described 135 in the sequel. OPTIONAL RTP padding MUST NOT be included unless the 136 P bit is set. 138 Figure 1 140 Marker bit: The marker bit indicates the last packet part of an 141 frame. This enables an decoder to finish decoding the picture, 142 where it otherwise may need to wait for the next packet to 143 explicitly know that. 145 Timestamp: The RTP timestamp indicate the time when the frame was 146 sampled at a clock rate of 90KHz. 148 Sequence number: The sequence number are monotonically increasing 149 and set as packets are sent. 151 The remaining RTP header fields are used as specified in RFC 3550 152 [4]. 154 4.1. VP8 Payload Descriptor 156 The first bytes after the RTP header are the VP8 payload descriptor, 157 with the following structure. 159 0 1 2 3 160 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 161 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 162 | RSV |I|N|FI |B| PictureID (integer #bytes) | 163 +-+-+-+-+-+-+-+-+ | 164 : : 165 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 166 | : (VP8 data or VP8 payload header; byte aligned)| 167 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 169 Figure 2 171 RSV: Bits reserved for future use. MUST be set to zero and MUST 172 be ignored by the receiver. 174 I: PictureID present. When set to one, a PictureID is provided 175 after the first byte of the payload descriptor. When set to zero, 176 the PictureID is omitted, and the one-byte payload descriptor is 177 immediately followed by the VP8 payload. 179 N: Non-reference frame. When set to one, the frame can be 180 discarded without affecting any other future or past frames. 182 FI: Fragmentation information field. This field contains 183 information about the fragmentation of VP8 payloads carried in the 184 RTP packet. The four different values are listed below. 186 * 00 The RTP packet contains no fragmented VP8 partitions. The 187 payload is one or several complete partitions. 189 * 01 The RTP packet contains the first part of a fragmented 190 partition. The fragment must be placed in its own RTP packet. 192 * 10 The RTP packet contains a fragment that is neither the first 193 nor the last part of a fragmented partition. The fragment must 194 be placed in its own RTP packet. 196 * 11 The RTP packet contains the last part of a fragmented 197 partition. The fragment must be placed in its own RTP packet. 199 B: Beginning VP8 frame. When set to 1 this signals that a new VP8 200 frame starts in this RTP packet. 202 PictureID: Multiple of 8 bits. This is a running index of the 203 frames. The field is present only if the I bit is equal to one. 204 The most significant bit of each byte is an extension flag. The 7 205 following bits carry (parts of) the PictureID. If the extension 206 flag is one, the PictureID continues in the next byte. If the 207 extension flag is zero, the 7 remaining bits are the last (and 208 least significant) bits in the PictureID. The sender may choose 209 any number of bytes, smaller or equal to 9 bytes as the maximum 210 PictureID. The PictureID SHALL start on a random number, and MUST 211 wrap after reaching the maximum ID. Leading zero bytes MUST be 212 supressed. 214 4.2. VP8 Payload Header 216 The first three bytes of an encoded VP8 frame are refered to as an 217 "uncompressed data chunk" in [1], and co-serve as payload header in 218 this RTP format. Note that the header is present only in packets 219 which have the B bit equal to one in the payload descriptor. 220 Subsequent packets for the same frame do not carry the payload 221 header. 223 0 1 2 3 224 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 225 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 226 |P| VER |H| 1st partition size | | 227 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 228 | | 229 : Bytes 4..N of VP8 payload : 230 | | 231 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 232 | : OPTIONAL RTP padding | 233 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 235 Figure 3 237 P: Inverse key frame flag. When set to 0 the current frame is a 238 key frame. When set to 1 the current frame is an interframe. 239 Defined in [1] 241 VER: A version number as defined in [1]. 243 H: Show frame bit as defined in [1]. 245 1st partition size: 19 bits. A field containing the size of the 246 first data partition in bytes, as defined in [1]. 248 4.3. Aggregated and Fragmented Payloads 250 An encoded VP8 frame can be divided into two or more partitions, as 251 described in Section 1. The fragmentation information described in 252 Section 4.1 MUST be used to signal if any fragmentation is applied. 253 Aggregation of encoded partitions is done without explicit signaling. 254 Partitions MUST be aggregated in decoding order. Two fragments from 255 different partitions MUST not be agregated into the same packet. An 256 aggregation MUST have exactly one payload descriptor. Aggregated 257 partitions MUST represent parts of one and the same video frame. 258 Consequently, an aggregated packet will have one or no payload 259 header, depending on whether the aggregate contains the first 260 partition of a frame or not, respectively. Note that the length of 261 the first partition can always be obtained from the first partition 262 size parameter in the VP8 payload header. Fragments of encoded 263 partitions MUST NOT be aggregated. 265 4.4. Examples of VP8 RTP Stream 267 A few examples of how the VP8 RTP payload can be used are included 268 below. 270 4.4.1. Key frame in a single RTP packet 272 Marker bit = 1. I = 1. B = 1. PictureID = 17 = 0001001 binary. P 273 = 0. 275 0 1 2 3 276 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 277 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 278 | RTP Header M=1 | 279 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 280 |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|0: VER :1: 1st partition | 281 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 282 | size = L | | 283 +-+-+-+-+-+-+-+-+ | 284 | | 285 : Bytes 4..L of first VP8 partition : 286 | | 287 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 288 | | | 289 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 290 | | 291 : Remaining VP8 partitions : 292 | | 293 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 294 | : OPTIONAL RTP padding | 295 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 297 4.4.2. VP8 interframe in a single RTP packet; no PictureID 299 Marker bit = 1. I = 0. B = 1. P = 1. 301 0 1 2 3 302 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 | RTP Header M=1 | 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 306 |0 0 0 0 0 0 0 1|1: VER :1: 1st partition size = L | 307 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 308 | | 309 : Bytes 4..L of first VP8 partition : 310 | | 311 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 312 | | | 313 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 314 | | 315 : Remaining VP8 partitions : 316 | | 317 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 318 | : OPTIONAL RTP padding | 319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 321 4.4.3. VP8 partitions in separate RTP packets 323 First RTP packet; marker bit = 0. I = 1. B = 1. PictureID = 17. 325 0 1 2 3 326 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 327 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 328 | RTP Header M=0 | 329 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 330 |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|1: VER :1: 1st partition | 331 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 332 | size = L | | 333 +-+-+-+-+-+-+-+-+ | 334 | | 335 : Bytes 4..L of first VP8 partition : 336 | | 337 | | 338 | | 339 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 341 Second RTP packet; marker bit = 1. B = 0. 343 0 1 2 3 344 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 345 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 346 | RTP Header M=1 | 347 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 348 |0 0 0 1 0 0 0 0:0 0 0 0 1 0 0 1| | 349 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 350 | | 351 : Remaining VP8 partitions : 352 | | 353 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 354 | : OPTIONAL RTP padding | 355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 357 4.4.4. VP8 frame fragmented across RTP packets 359 First RTP packet; marker bit = 0. I = 1. FI = 00. B = 1. 361 0 1 2 3 362 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 363 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 364 | RTP Header M=0 | 365 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 366 |0 0 0 1 0 0 0 1:0 0 0 0 1 0 0 1|1: VER :1: 1st partition | 367 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 368 | size = L | | 369 +-+-+-+-+-+-+-+-+ | 370 | | 371 : Bytes 4..L of first VP8 partition : 372 | | 373 | | 374 | | 375 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 377 Second RTP packet; marker bit = 0. FI = 01. B = 0. 379 0 1 2 3 380 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 381 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 382 | RTP Header M=0 | 383 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 384 |0 0 0 1 0 0 1 0:0 0 0 0 1 0 0 1| | 385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 386 | | 387 : First fragment of second VP8 partition : 388 | | 389 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 391 Third RTP packet; marker bit = 0. FI = 10. B = 0. 393 0 1 2 3 394 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 395 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 396 | RTP Header M=0 | 397 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 398 |0 0 0 1 0 1 0 0:0 0 0 0 1 0 0 1| | 399 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 400 | | 401 : Middle fragment of second VP8 partition : 402 | | 403 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 405 Last RTP packet; marker bit = 1. FI = 11. B = 0. 407 0 1 2 3 408 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 409 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 410 | RTP Header M=1 | 411 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 412 |0 0 0 1 0 1 1 0:0 0 0 0 1 0 0 1| | 413 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 414 | | 415 : Last fragment of second VP8 partition : 416 | | 417 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 418 | : OPTIONAL RTP padding | 419 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 421 4.4.5. VP8 frame with long PictureID 423 PictureID = 4711 = 01001001100111 binary (first 7 bits: 0100100, last 424 7 bits: 1100111). 426 0 1 2 3 427 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 428 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 429 | RTP Header M=1 | 430 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 431 |0 0 0 1 0 0 0 1:1 0 1 0 0 1 0 0 0 1 1 0 0 1 1 1|1: VER :1: 1st | 432 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 433 | partition size = L | | 434 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 435 | | 436 : Bytes 4..N of first VP8 frame : 437 | | 438 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 440 5. Using VP8 with RPSI and SLI Feedback 442 The VP8 payload descriptor defined in Section 4.1 above contains an 443 optional PictureID parameter. This parameter is included mainly to 444 enable use of reference picture selection index (RPSI) and slice loss 445 indication (SLI), both defined in RFC 4585 [3]. 447 5.1. RPSI 449 The reference picture selection index is a payload-specific feedback 450 message defined within the RTCP-based feedback format. The RPSI 451 message is generated by a receiver and can be used in two ways. 452 Either it can signal a preferred reference picture when a loss has 453 been detected by the decoder -- preferably then a reference that the 454 decoder knows is perfect -- or, it can be used as positive feedback 455 information to acknowledge correct decoding of certain reference 456 pictures. The positive feedback method is useful for VP8 used as 457 unicast. The use of RPSI for VP8 is preferably combined with a 458 special update pattern of the codec's two special reference frames -- 459 the golden frame and the altref frame -- in which they are updated in 460 an alternating leapfrog fashion. When a receiver has received and 461 correctly decoded a golden or altref frame, and that frame had a 462 PictureID in the payload descriptor, the receiver can acknowledge 463 this simply by sending an RPSI message back to the sender. The 464 message body (i.e., the "native RPSI bit string" in RFC 4585 [3]) is 465 simply the PictureID of the received frame. 467 5.2. SLI 469 The slice loss indication is another payload-specific feedback 470 message defined within the RTCP-based feedback format. The SLI 471 message is generated by the receiver when a loss or corruption is 472 detected in a frame. The format of the SLI message is as follows 473 [3]: 475 0 1 2 3 476 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 | First | Number | PictureID | 479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 481 Figure 4 483 Here, First is the macroblock address (in scan order) of the first 484 lost block and Number is the number of lost blocks. PictureID is the 485 six least significant bits of the codec-specific picture identifier 486 in which the loss or corruption has occurred. For VP8, this codec- 487 specific identifier is naturally the PictureID of the current frame, 488 as read from the payload descriptor. If the payload descriptor of 489 the current frame does not have a PictureID, the receiver MAY send 490 the last received PictureID+1 in the SLI message. The receiver MAY 491 set the First parameter to 0, and the Number parameter to the total 492 number of macroblocks per frame, even though only parts of the frame 493 is corrupted. When the sender receives an SLI message, it can make 494 use of the knowledge from the latest received RPSI message. Knowing 495 that the last golden or altref frame was successfully received, it 496 can encode the next frame with reference to that established 497 reference. 499 5.3. Example 501 The use of RSPI and SLI is best illustrated in an example. In this 502 example, the encoder may not update the altref frame until the last 503 sent golden frame has been acknowledged with an RPSI message. If an 504 update is not received within some time, a new golden frame update is 505 sent instead. Once the new golden frame is established and 506 acknowledge, the same rule applies when updating the altref frame. 508 Event Sender Receiver Established 509 reference 510 +----+--------------------+--------------------------+------------+ 511 1000 Send golden frame 512 PictureID = 0 513 Receive and decode 514 golden frame 516 1001 Send RPSI(0) 517 1002 Receive RPSI(0) golden 519 ... (sending regular frames) 521 1100 Send altref frame 522 PictureID = 100 523 Altref corrupted or lost golden 525 1101 Send SLI(100) golden 526 1102 Receive SLI(100) 528 1103 Send frame with 529 reference to golden 530 Receive and decode frame 531 (decoder state restored) golden 533 ... (sending regular frames) 535 1200 Send altref frame 536 PictureID = 200 537 Receive and decode 538 altref frame golden 540 1201 Send RPSI(200) 541 1202 Receive RPSI(200) altref 543 ... (sending regular frames) 545 1300 Send golden frame 546 PictureID = 300 547 Receive and decode 548 golden frame altref 550 1301 Send RPSI(300) altref 551 1302 RPSI lost 553 1400 Send golden frame 554 PictureID = 400 555 Receive and decode 556 golden frame altref 558 1401 Send RPSI(400) 559 1402 Receive RPSI(400) golden 560 +----+--------------------+--------------------------+------------+ 562 Note that the scheme is robust to loss of the feedback messages. If 563 the RPSI is lost, the sender will try to update the golden (or 564 altref) again after a while, without releasing the established 565 reference. Also, if an SLI is lost, the receiver can keep sending 566 SLI messages at any interval, as long as the picture is corrupted. 568 6. Payload Format Parameters 570 This section specifies the parameters that MAY be used to select 571 optional features of the payload format and certain features of the 572 bitstream. 574 6.1. Restrictions on usage 576 This media type depends on RTP framing, and hence is only defined for 577 transfer via RTP [RFC3550] [4].Transport within other framing 578 protocols is not defined at this time. 580 6.2. Media Type Registration 582 This registration is done using the template defined in RFC 4288 [7] 583 and following RFC 4855 [8]. 585 Type name: video 587 Subtype name: VP8 589 Required parameters: none 591 Optional parameters: none 593 Encoding considerations: 594 This media type is framed and contains binary data; see Section 595 4.8 of [7]. 597 Security considerations: See Section 7 of RFC xxxx. 598 [RFC Editor: Upon publication as an RFC, please replace "XXXX" 599 with the number assigned to this document and remove this note.] 601 Interoperability considerations: None. 603 Published specification: VP8 bitstream format [1] and RFC XXXX. 604 [RFC Editor: Upon publication as an RFC, please replace "XXXX" 605 with the number assigned to this document and remove this note.] 607 Applications which use this media type: 608 For example: Video over IP, video conferencing. 610 Additional information: None. 612 Person & email address to contact for further information: 613 Patrik Westin, patrik.westin@gmail.com 615 Intended usage: COMMON 617 Restrictions on usage: 618 This media type depends on RTP framing, and hence is only defined 619 for transfer via RTP [4]. 621 Author: Patrik Westin, patrik.westin@gmail.com 623 Change controller: 624 IETF Payload Working Group delegated from the IESG. 626 6.3. SDP Parameters 628 The receiver MUST ignore any parameter unspecified in this memo. 630 6.3.1. Mapping of MIME Parameters to SDP 632 The MIME media type video/VP8 string is mapped to fields in the 633 Session Description Protocol (SDP) [6] as follows: 635 o The media name in the "m=" line of SDP MUST be video. 637 o The encoding name in the "a=rtpmap" line of SDP MUST be VP8 (the 638 MIME subtype). 640 o The clock rate in the "a=rtpmap" line MUST be 90000. 642 o The OPTIONAL parameter "version", if included, MUST be in the 643 a=fmtp SDP field. This parameter matches the VP8 bitstream 644 version. 646 6.4. Example 648 An example of media representation in SDP is as follows: 650 m=video 49170 RTP/AVPF 98 651 a=rtpmap:98 VP8/90000 652 a=fmtp:98 version=0 654 7. Security Considerations 656 RTP packets using the payload format defined in this specification 657 are subject to the security considerations discussed in the RTP 658 specification [4], and in any applicable RTP profile. The main 659 security considerations for the RTP packet carrying the RTP payload 660 format defined within this memo are confidentiality, integrity and 661 source authenticity. Confidentiality is achieved by encryption of 662 the RTP payload. Integrity of the RTP packets through suitable 663 cryptographic integrity protection mechanism. Cryptographic system 664 may also allow the authentication of the source of the payload. A 665 suitable security mechanism for this RTP payload format should 666 provide confidentiality, integrity protection and at least source 667 authentication capable of determining if an RTP packet is from a 668 member of the RTP session or not. Note that the appropriate 669 mechanism to provide security to RTP and payloads following this memo 670 may vary. It is dependent on the application, the transport, and the 671 signalling protocol employed. Therefore a single mechanism is not 672 sufficient, although if suitable the usage of SRTP [5] is 673 recommended. This RTP payload format and its media decoder do not 674 exhibit any significant non-uniformity in the receiver-side 675 computational complexity for packet processing, and thus are unlikely 676 to pose a denial-of-service threat due to the receipt of pathological 677 data. Nor does the RTP payload format contain any active content. 679 8. Congestion Control 681 Congestion control for RTP SHALL be used in accordance with RFC 3550 682 [4], and with any applicable RTP profile; e.g., RFC 3551 [9]. The 683 congestions control mechanism can, in a real-time encoding scenario, 684 adapt the transmission rate by instructing the encoder to encode at a 685 certain target rate. Media aware network elements MAY use the 686 information in the VP8 payload descriptor in Section 4.1 to identify 687 non-reference frames and discard them in order to reduce network 688 congestion. 690 9. IANA Considerations 692 The IANA is requested to register the following values: 693 - Media type registration as described in Section 6.2. 695 10. References 697 [1] Google, Inc., "VP8 Data Format and Decoding Guide", July 2010, 698 . 700 [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement 701 Levels", RFC 2119, STD 1, December 1997. 703 [3] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 704 "Extended RTP Profile for Real-time Transport Control Protocol 705 (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, STD 1, July 2006. 707 [4] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, 708 "RTP: A Transport Protocol for Real-Time Applications", 709 RFC 3550, STD 64, July 2003. 711 [5] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 712 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 713 RFC 3711, STD 1, March 2004. 715 [6] Handley, M. and V. Jacobson, "SDP: Session Description 716 Protocol", RFC 2327, STD 1, April 1998. 718 [7] Freed, N. and J. Klensin, "Media Type Specifications and 719 Registrations Procedures", RFC 4288, BCP 13, December 2005. 721 [8] Casner, S., "Media Type Registration of RTP Payload Formats", 722 RFC 4855, February 2007. 724 [9] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video 725 Conferences with Minimal Control", RFC 3551, STD 65, July 2003. 727 Authors' Addresses 729 Patrik Westin 730 Google, Inc. 731 Kungsbron 2 732 Stockholm, 11122 733 Sweden 735 Email: patrik.westin@gmail.com 737 Henrik F Lundin 738 Google, Inc. 739 Kungsbron 2 740 Stockholm, 11122 741 Sweden 743 Email: hlundin@google.com 745 Michael Glover 746 Google, Inc. 748 Justin Uberti 749 Google, Inc. 751 Frank Galligan 752 Google, Inc.