idnits 2.17.1 draft-ietf-payload-vp8-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 24, 2012) is 4466 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 2327 (Obsoleted by RFC 4566) ** Obsolete normative reference: RFC 4288 (Obsoleted by RFC 6838) ** Downref: Normative reference to an Informational RFC: RFC 6386 Summary: 4 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Payload Working Group P. Westin 3 Internet-Draft H. Lundin 4 Intended status: Standards Track M. Glover 5 Expires: July 27, 2012 J. Uberti 6 F. Galligan 7 Google 8 January 24, 2012 10 RTP Payload Format for VP8 Video 11 draft-ietf-payload-vp8-03 13 Abstract 15 This memo describes an RTP payload format for the VP8 video codec. 16 The payload format has wide applicability, as it supports 17 applications from low bit-rate peer-to-peer usage, to high bit-rate 18 video conferences. 20 Status of this Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on July 27, 2012. 37 Copyright Notice 39 Copyright (c) 2012 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 4 56 3. Media Format Description . . . . . . . . . . . . . . . . . . . 5 57 4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . . 6 58 4.1. RTP Header Usage . . . . . . . . . . . . . . . . . . . . . 6 59 4.2. VP8 Payload Descriptor . . . . . . . . . . . . . . . . . . 7 60 4.3. VP8 Payload Header . . . . . . . . . . . . . . . . . . . . 10 61 4.4. Aggregated and Fragmented Payloads . . . . . . . . . . . . 11 62 4.5. Examples of VP8 RTP Stream . . . . . . . . . . . . . . . . 13 63 4.5.1. Key frame in a single RTP packet . . . . . . . . . . . 13 64 4.5.2. VP8 interframe in a single RTP packet; no PictureID . 13 65 4.5.3. VP8 partitions in separate RTP packets . . . . . . . . 14 66 4.5.4. VP8 frame fragmented across RTP packets . . . . . . . 15 67 4.5.5. VP8 frame with long PictureID . . . . . . . . . . . . 17 68 5. Using VP8 with RPSI and SLI Feedback . . . . . . . . . . . . . 18 69 5.1. RPSI . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 70 5.2. SLI . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 71 5.3. Example . . . . . . . . . . . . . . . . . . . . . . . . . 19 72 6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 22 73 6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 22 74 6.2. SDP Parameters . . . . . . . . . . . . . . . . . . . . . . 23 75 6.2.1. Mapping of MIME Parameters to SDP . . . . . . . . . . 23 76 6.2.2. Offer/Answer Considerations . . . . . . . . . . . . . 23 77 7. Security Considerations . . . . . . . . . . . . . . . . . . . 24 78 8. Congestion Control . . . . . . . . . . . . . . . . . . . . . . 25 79 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 80 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 27 81 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 83 1. Introduction 85 This memo describes an RTP payload specification applicable to the 86 transmission of video streams encoded using the VP8 video codec 87 [RFC6386]. The format described in this document can be used both in 88 peer-to-peer and video conferencing applications. 90 VP8 is based on decomposition of frames into square sub-blocks of 91 pixels, prediction of such sub-blocks using previously constructed 92 blocks, and adjustment of such predictions (as well as synthesis of 93 unpredicted blocks) using a discrete cosine transform (hereafter 94 abbreviated as DCT). In one special case, however, VP8 uses a 95 "Walsh-Hadamard" (hereafter abbreviated as WHT) transform instead of 96 a DCT. An encoded VP8 frame is divided into two or more partitions, 97 as described in [RFC6386]. The first partition (prediction or mode) 98 contains prediction mode parameters and motion vectors for all 99 macroblocks. The remaining partitions all contain the quantized DCT/ 100 WHT coefficients for the residuals. There can be 1, 2, 4, or 8 DCT/ 101 WHT partitions per frame, depending on encoder settings. 103 In summary, the payload format described in this document enables a 104 number of features in VP8, including: 106 o Taking partition boundaries into consideration, to improve loss 107 robustness and facilitate efficient packet loss concealment at the 108 decoder. 110 o Temporal scalability. 112 o Advanced use of reference frames to enable efficient error 113 recovery. 115 o Marking of frames that have no impact on the decoding of any other 116 frame, so that these non-reference frames can be discarded in a 117 server or media-aware network element if needed. 119 2. Conventions, Definitions and Acronyms 121 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 122 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 123 document are to be interpreted as described in [RFC2119]. 125 3. Media Format Description 127 The VP8 codec uses three different reference frames for interframe 128 prediction: the previous frame, the golden frame, and the altref 129 frame. The payload specification in this memo has elements that 130 enable advanced use of the reference frames, e.g., for improved loss 131 robustness. 133 One specific use case of the three reference frame types is temporal 134 scalability. By setting up the reference hierarchy in the 135 appropriate way, up to five temporal layers can be encoded. (How to 136 set up the reference hierarchy for temporal scalability is not within 137 the scope of this memo.) 139 Another property of the VP8 codec is that it applies data 140 partitioning to the encoded data. Thus, an encoded VP8 frame can be 141 divided into two or more partitions, as described in "VP8 Data Format 142 and Decoding Guide" [RFC6386]. The first partition (prediction or 143 mode) contains prediction mode parameters and motion vectors for all 144 macroblocks. The remaining partitions all contain the transform 145 coefficients for the residuals. The first partition is decodable 146 without the remaining residual partitions. The subsequent partitions 147 may be useful even if some part of the frame is lost. This memo 148 allows the partitions to be sent separately or in the same RTP 149 packet. It may be beneficial for decoder error-concealment to send 150 the partitions in different packets, even though it is not mandatory 151 according to this specification. 153 The format specification is described in Section 4. In Section 5, a 154 method to acknowledge receipt of reference frames using RTCP 155 techniques is described. 157 The payload partitioning and the acknowledging method both serve as 158 motivation for three of the fields included in the payload format: 159 the "PartID", "1st partition size" and "PictureID" fields. The 160 ability to encode a temporally scalable stream motivates the 161 "TL0PICIDX" and "TID" fields. 163 4. Payload Format 165 This section describes how the encoded VP8 bitstream is encapsulated 166 in RTP. Usage of RTP/AVPF [RFC4585] is recommended. 168 4.1. RTP Header Usage 170 The general RTP payload format for VP8 is depicted below. 172 0 1 2 3 173 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 174 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 175 |V=2|P|X| CC |M| PT | sequence number | 176 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 177 | timestamp | 178 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 179 | synchronization source (SSRC) identifier | 180 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 181 | contributing source (CSRC) identifiers | 182 | .... | 183 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 184 | VP8 payload descriptor (integer #bytes) | 185 : : 186 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 187 | : VP8 payload header (3 octets) | 188 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 189 | VP8 pyld hdr : | 190 +-+-+-+-+-+-+-+-+ | 191 : Bytes 4..N of VP8 payload : 192 | | 193 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 194 | : OPTIONAL RTP padding | 195 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 197 The VP8 payload descriptor and VP8 payload header will be described 198 in the sequel. OPTIONAL RTP padding MUST NOT be included unless the 199 P bit is set. 201 Figure 1 203 Marker bit (M): Set for the very last packet of each encoded frame 204 in line with the normal use of the M bit in video formats. This 205 enables a decoder to finish decoding the picture, where it 206 otherwise may need to wait for the next packet to explicitly know 207 that the frame is complete. 209 Timestamp: The RTP timestamp indicates the time when the frame was 210 sampled at a clock rate of 90 kHz. 212 Sequence number: The sequence numbers are monotonically increasing 213 and set as packets are sent. 215 The remaining RTP header fields are used as specified in 216 [RFC3550]. 218 4.2. VP8 Payload Descriptor 220 The first octets after the RTP header are the VP8 payload descriptor, 221 with the following structure. 223 0 1 2 3 4 5 6 7 224 +-+-+-+-+-+-+-+-+ 225 |X|R|N|S|PartID | (REQUIRED) 226 +-+-+-+-+-+-+-+-+ 227 X: |I|L|T|K| RSV | (OPTIONAL) 228 +-+-+-+-+-+-+-+-+ 229 I: | PictureID | (OPTIONAL) 230 +-+-+-+-+-+-+-+-+ 231 L: | TL0PICIDX | (OPTIONAL) 232 +-+-+-+-+-+-+-+-+ 233 T/K: |TID|Y| KEYIDX | (OPTIONAL) 234 +-+-+-+-+-+-+-+-+ 236 Figure 2 238 X: Extended control bits present. When set to one, the extension 239 octet MUST be provided immediately after the mandatory first 240 octet. If the bit is zero, all optional fields MUST be omitted. 242 R: Bit reserved for future use. MUST be set to zero and MUST be 243 ignored by the receiver. 245 N: Non-reference frame. When set to one, the frame can be discarded 246 without affecting any other future or past frames. If the 247 reference status of the frame is unknown, this bit SHOULD be set 248 to zero to avoid discarding frames needed for reference. 250 Informative note: This document does not describe how to 251 determine if an encoded frame is non-reference. The reference 252 status of an encoded frame is preferably provided from the 253 encoder implementation. 255 S: Start of VP8 partition. SHOULD be set to 1 when the first payload 256 octet of the RTP packet is the beginning of a new VP8 partition, 257 and MUST NOT be 1 otherwise. The S bit MUST be set to 1 for the 258 first packet of each encoded frame. 260 PartID: Partition index. Denotes which VP8 partition the first 261 payload octet of the packet belongs to. The first VP8 partition 262 (containing modes and motion vectors) MUST be labeled with PartID 263 = 0. PartID SHOULD be incremented for each subsequent partition, 264 but MAY be kept at 0 for all packets. PartID MUST NOT be larger 265 than 8. If more than one packet in an encoded frame contains the 266 same PartID, the S bit MUST NOT be set for any other packet than 267 the first packet with that PartID. 269 When the X bit is set to 1 in the first octet, the OPTIONAL extension 270 bit field MUST be present in the second octet. If the X bit is 0, 271 the extension bit field MUST NOT be present, and all bits below MUST 272 be implicitly interpreted as 0. 274 I: PictureID present. When set to one, the OPTIONAL PictureID MUST 275 be present after the extension bit field and specified as below. 276 Otherwise, PictureID MUST NOT be present. 278 L: TL0PICIDX present. When set to one, the OPTIONAL TL0PICIDX MUST 279 be present and specified as below, and the T bit MUST be set to 1. 280 Otherwise, TL0PICIDX MUST NOT be present. 282 T: TID present. When set to one, the OPTIONAL TID/KEYIDX octet MUST 283 be present. The TID|Y part of the octet MUST be specified as 284 below. If K (below) is set to one but T is set to zero, the TID/ 285 KEYIDX octet MUST be present, but the TID|Y field MUST be ignored. 286 If neither T nor K is set to one, the TID/KEYIDX octet MUST NOT be 287 present. 289 K: KEYIDX present. When set to one, the OPTIONAL TID/KEYIDX octet 290 MUST be present. The KEYIDX part of the octet MUST be specified 291 as below. If T (above) is set to one but K is set to zero, the 292 TID/KEYIDX octet MUST be present, but the KEYIDX field MUST be 293 ignored. If neither T nor K is set to one, the TID/KEYIDX octet 294 MUST NOT be present. 296 RSV: Bits reserved for future use. MUST be set to zero and MUST be 297 ignored by the receiver. 299 After the extension bit field follow the extension data fields that 300 are enabled. 302 PictureID: 8 or 16 bits. This is a running index of the frames. 303 The field MUST be present if the I bit is equal to one. The most 304 significant bit of the first octet is an extension flag. The 7 305 following bits carry (parts of) the PictureID. If the extension 306 flag is one, the PictureID continues in the next octet forming a 307 15 bit index, where the 8 bits in the second octet are the least 308 significant bits of the PictureID. If the extension flag is zero, 309 there is no extension, and the PictureID is the 7 remaining bits 310 of the first (and only) octet. The sender may choose 7 or 15 bits 311 index. The PictureID SHOULD start on a random number, and MUST 312 wrap after reaching the maximum ID. 314 TL0PICIDX: 8 bits temporal level zero index. The field MUST be 315 present if the L bit is equal to 1, and MUST NOT be present 316 otherwise. TL0PICIDX is a running index for the temporal base 317 layer frames, i.e., the frames with TID set to 0. If TID is 318 larger than 0, TL0PICIDX indicates which base layer frame the 319 current image depends on. TL0PICIDX MUST be incremented when TID 320 is 0. The index SHOULD start on a random number, and MUST restart 321 at 0 after reaching the maximum number 255. 323 TID: 2 bits temporal layer index. The TID/KEYIDX octet MUST be 324 present when either the T bit or the K bit or both are equal to 1, 325 and MUST NOT be present otherwise. The TID field MUST be ignored 326 by the receiver when the T bit is set equal to 0. The TID field 327 indicates which temporal layer the packet represents. The lowest 328 layer, i.e., the base layer, MUST have TID set to 0. Higher 329 layers SHOULD increment the TID according to their position in the 330 layer hierarchy. 332 Y: 1 layer sync bit. The TID/KEYIDX octet MUST be present when 333 either the T bit or the K bit or both are equal to 1, and MUST NOT 334 be present otherwise. The Y bit MUST be ignored by the receiver 335 when the T bit is set equal to 0. The Y bit SHOULD be set to 1 if 336 the current frame depends only on the base layer (TID = 0) frame 337 with TL0PICIDX equal to that of the current frame. The Y bit MUST 338 be set to 0 if the current frame depends any other frame than the 339 base layer (TID = 0) frame with TL0PICIDX equal to that of the 340 current frame. 342 Informative note: This document does not describe how to 343 determine the dependence status for a frame; this information 344 is preferably provided from the encoder implementation. In the 345 case of unknown status, the Y bit can safely be set to 0. 347 KEYIDX: 5 bits temporal key frame index. The TID/KEYIDX octet MUST 348 be present when either the T bit or the K bit or both are equal to 349 1, and MUST NOT be present otherwise. The KEYIDX field MUST be 350 ignored by the receiver when the K bit is set equal to 0. The 351 KEYIDX field is a running index for key frames. KEYIDX SHOULD 352 start on a random number, and MUST restart at 0 after reaching the 353 maximum number 31. When in use, the KEYIDX SHOULD be present for 354 both key frames and interframes. The sender MUST increment KEYIDX 355 for key frames which convey parameter updates critical to the 356 interpretation of subsequent frames, and SHOULD leave the KEYIDX 357 unchanged for key frames that do not contain these critical 358 updates. A receiver SHOULD NOT decode an interframe if it has not 359 received and decoded a key frame with the same KEYIDX after the 360 last KEYIDX wrap-around. 362 Informative note: This document does not describe how to 363 determine if a key frame updates critical parameters; this 364 information is preferably provided from the encoder 365 implementation. A sender that does not have this information 366 may either omit the KEYIDX field (set K equal to 0), or 367 increment the KEYIDX on every key frame. The benefit with the 368 latter is that any key frame loss will be detected by the 369 receiver, which can signal for re-transmission or request a new 370 key frame. 372 4.3. VP8 Payload Header 374 The first three octets of an encoded VP8 frame are referred to as an 375 "uncompressed data chunk" in [RFC6386], and co-serve as payload 376 header in this RTP format. The codec bitstream format specifies two 377 different variants of the uncompressed data chunk: a 3 octet version 378 for interframes and a 10 octet version for key frames. The first 3 379 octets are common to both variants. In the case of a key frame the 380 remaining 7 octets are considered to be part of the remaining payload 381 in this RTP format. Note that the header is present only in packets 382 which have the S bit equal to one and the PartID equal to zero in the 383 payload descriptor. Subsequent packets for the same frame do not 384 carry the payload header. 386 0 1 2 3 4 5 6 7 387 +-+-+-+-+-+-+-+-+ 388 |Size0|H| VER |P| 389 +-+-+-+-+-+-+-+-+ 390 | Size1 | 391 +-+-+-+-+-+-+-+-+ 392 | Size2 | 393 +-+-+-+-+-+-+-+-+ 394 | Bytes 4..N of | 395 | VP8 payload | 396 : : 397 +-+-+-+-+-+-+-+-+ 398 | OPTIONAL RTP | 399 | padding | 400 : : 401 +-+-+-+-+-+-+-+-+ 403 Figure 3 405 H: Show frame bit as defined in [RFC6386]. 407 VER: A version number as defined in [RFC6386]. 409 P: Inverse key frame flag. When set to 0 the current frame is a key 410 frame. When set to 1 the current frame is an interframe. Defined 411 in [RFC6386] 413 SizeN: The size of the first partition size in bytes is calculated 414 from the 19 bits in Size0, Size1, and Size2 as 1stPartitionSize = 415 Size0 + 8 * Size1 + 2048 * Size2. [RFC6386]. 417 4.4. Aggregated and Fragmented Payloads 419 An encoded VP8 frame can be divided into two or more partitions, as 420 described in Section 1. One packet can contain a fragment of a 421 partition, a complete partition, or an aggregate of fragments and 422 partitions. In the preferred use case, the S bit and PartID fields 423 described in Section 4.2 should be used to indicate what the packet 424 contains. The PartID field should indicate which partition the first 425 octet of the payload belongs to, and the S bit indicates that the 426 packet starts on a new partition. Aggregation of encoded partitions 427 is done without explicit signaling. Partitions MUST be aggregated in 428 decoding order. Two fragments from different partitions MAY be 429 aggregated into the same packet. An aggregation MUST have exactly 430 one payload descriptor. Aggregated partitions MUST represent parts 431 of one and the same video frame. Consequently, an aggregated packet 432 will have one or no payload header, depending on whether the 433 aggregate contains the beginning of the first partition of a frame or 434 not, respectively. Note that the length of the first partition can 435 always be obtained from the first partition size parameter in the VP8 436 payload header. 438 The VP8 bitstream format [RFC6386] specifies that if multiple DCT/WHT 439 partitions are produced, the location of each partition start is 440 found at the end of the first (prediction/mode) partition. In this 441 RTP payload specification, the location offsets are considered to be 442 part of the first partition. 444 It is OPTIONAL for a packetizer implementing this RTP specification 445 to pay attention to the partition boundaries within an encoded frame. 446 If packetization of a frame is done without considering the partition 447 boundaries, the PartID field MAY be set to zero for all packets, and 448 the S bit MUST NOT be set to one for any other packet than the first. 450 4.5. Examples of VP8 RTP Stream 452 A few examples of how the VP8 RTP payload can be used are included 453 below. 455 4.5.1. Key frame in a single RTP packet 457 0 1 2 3 4 5 6 7 458 +-+-+-+-+-+-+-+-+ 459 | RTP header | 460 | M = 1 | 461 +-+-+-+-+-+-+-+-+ 462 |1|0|0|1|0 0 0 0| X = 1; S = 1; PartID = 0 463 +-+-+-+-+-+-+-+-+ 464 |1|0|0|0|0 0 0 0| I = 1 465 +-+-+-+-+-+-+-+-+ 466 |0 0 0 0 1 0 0 1| PictureID = 17 467 +-+-+-+-+-+-+-+-+ 468 |Size0|1| VER |0| P = 0 469 +-+-+-+-+-+-+-+-+ 470 | Size1 | 471 +-+-+-+-+-+-+-+-+ 472 | Size2 | 473 +-+-+-+-+-+-+-+-+ 474 | VP8 payload | 475 +-+-+-+-+-+-+-+-+ 477 4.5.2. VP8 interframe in a single RTP packet; no PictureID 479 0 1 2 3 4 5 6 7 480 +-+-+-+-+-+-+-+-+ 481 | RTP header | 482 | M = 1 | 483 +-+-+-+-+-+-+-+-+ 484 |0|0|0|1|0 0 0 0| X = 0; S = 1; PartID = 0 485 +-+-+-+-+-+-+-+-+ 486 |Size0|1| VER |1| P = 1 487 +-+-+-+-+-+-+-+-+ 488 | Size1 | 489 +-+-+-+-+-+-+-+-+ 490 | Size2 | 491 +-+-+-+-+-+-+-+-+ 492 | VP8 payload | 493 +-+-+-+-+-+-+-+-+ 495 4.5.3. VP8 partitions in separate RTP packets 497 First RTP packet; complete first partition. 499 0 1 2 3 4 5 6 7 500 +-+-+-+-+-+-+-+-+ 501 | RTP header | 502 | M = 0 | 503 +-+-+-+-+-+-+-+-+ 504 |1|0|0|1|0 0 0 0| X = 1; S = 1; PartID = 0 505 +-+-+-+-+-+-+-+-+ 506 |1|0|0|0|0 0 0 0| I = 1 507 +-+-+-+-+-+-+-+-+ 508 |0 0 0 0 1 0 0 1| PictureID = 17 509 +-+-+-+-+-+-+-+-+ 510 |Size0|1| VER |1| P = 1 511 +-+-+-+-+-+-+-+-+ 512 | Size1 | 513 +-+-+-+-+-+-+-+-+ 514 | Size2 | 515 +-+-+-+-+-+-+-+-+ 516 | Bytes 4..L of | 517 | first VP8 | 518 | partition | 519 : : 520 +-+-+-+-+-+-+-+-+ 522 Second RTP packet; complete second partition. 524 0 1 2 3 4 5 6 7 525 +-+-+-+-+-+-+-+-+ 526 | RTP header | 527 | M = 1 | 528 +-+-+-+-+-+-+-+-+ 529 |1|0|0|1|0 0 0 1| X = 1; S = 1; PartID = 1 530 +-+-+-+-+-+-+-+-+ 531 |1|0|0|0|0 0 0 0| I = 1 532 +-+-+-+-+-+-+-+-+ 533 |0 0 0 0 1 0 0 1| PictureID = 17 534 +-+-+-+-+-+-+-+-+ 535 | Remaining VP8 | 536 | partitions | 537 : : 538 +-+-+-+-+-+-+-+-+ 540 4.5.4. VP8 frame fragmented across RTP packets 542 First RTP packet; complete first partition. 544 0 1 2 3 4 5 6 7 545 +-+-+-+-+-+-+-+-+ 546 | RTP header | 547 | M = 0 | 548 +-+-+-+-+-+-+-+-+ 549 |1|0|0|1|0 0 0 0| X = 1; S = 1; PartID = 0 550 +-+-+-+-+-+-+-+-+ 551 |1|0|0|0|0 0 0 0| I = 1 552 +-+-+-+-+-+-+-+-+ 553 |0 0 0 0 1 0 0 1| PictureID = 17 554 +-+-+-+-+-+-+-+-+ 555 |Size0|1| VER |1| P = 1 556 +-+-+-+-+-+-+-+-+ 557 | Size1 | 558 +-+-+-+-+-+-+-+-+ 559 | Size2 | 560 +-+-+-+-+-+-+-+-+ 561 | Complete | 562 | first | 563 | partition | 564 : : 565 +-+-+-+-+-+-+-+-+ 567 Second RTP packet; first fragment of second partition. 569 0 1 2 3 4 5 6 7 570 +-+-+-+-+-+-+-+-+ 571 | RTP header | 572 | M = 0 | 573 +-+-+-+-+-+-+-+-+ 574 |1|0|0|1|0 0 0 1| X = 1; S = 1; PartID = 1 575 +-+-+-+-+-+-+-+-+ 576 |1|0|0|0|0 0 0 0| I = 1 577 +-+-+-+-+-+-+-+-+ 578 |0 0 0 0 1 0 0 1| PictureID = 17 579 +-+-+-+-+-+-+-+-+ 580 | First fragment| 581 | of second | 582 | partition | 583 : : 584 +-+-+-+-+-+-+-+-+ 586 Third RTP packet; second fragment of second partition. 588 0 1 2 3 4 5 6 7 589 +-+-+-+-+-+-+-+-+ 590 | RTP header | 591 | M = 0 | 592 +-+-+-+-+-+-+-+-+ 593 |1|0|0|0|0 0 0 1| X = 1; S = 0; PartID = 1 594 +-+-+-+-+-+-+-+-+ 595 |1|0|0|0|0 0 0 0| I = 1 596 +-+-+-+-+-+-+-+-+ 597 |0 0 0 0 1 0 0 1| PictureID = 17 598 +-+-+-+-+-+-+-+-+ 599 | Mid fragment | 600 | of second | 601 | partition | 602 : : 603 +-+-+-+-+-+-+-+-+ 605 Fourth RTP packet; last fragment of second partition. 607 0 1 2 3 4 5 6 7 608 +-+-+-+-+-+-+-+-+ 609 | RTP header | 610 | M = 1 | 611 +-+-+-+-+-+-+-+-+ 612 |1|0|0|0|0 0 0 1| X = 1; S = 0; PartID = 1 613 +-+-+-+-+-+-+-+-+ 614 |1|0|0|0|0 0 0 0| I = 1 615 +-+-+-+-+-+-+-+-+ 616 |0 0 0 0 1 0 0 1| PictureID = 17 617 +-+-+-+-+-+-+-+-+ 618 | Last fragment | 619 | of second | 620 | partition | 621 : : 622 +-+-+-+-+-+-+-+-+ 624 4.5.5. VP8 frame with long PictureID 626 PictureID = 4711 = 001001001100111 binary (first 7 bits: 0010010, 627 last 8 bits: 01100111). 629 0 1 2 3 4 5 6 7 630 +-+-+-+-+-+-+-+-+ 631 | RTP header | 632 | M = 1 | 633 +-+-+-+-+-+-+-+-+ 634 |1|0|0|1|0 0 0 0| X = 1; S = 1; PartID = 0 635 +-+-+-+-+-+-+-+-+ 636 |1|0|0|0|0 0 0 0| I = 1; 637 +-+-+-+-+-+-+-+-+ 638 |1 0 0 1 0 0 1 0| Long PictureID flag = 1 639 |0 1 1 0 0 1 1 1| PictureID = 4711 640 +-+-+-+-+-+-+-+-+ 641 |Size0|1| VER |1| 642 +-+-+-+-+-+-+-+-+ 643 | Size1 | 644 +-+-+-+-+-+-+-+-+ 645 | Size2 | 646 +-+-+-+-+-+-+-+-+ 647 | Bytes 4..N of | 648 | VP8 payload | 649 : : 650 +-+-+-+-+-+-+-+-+ 652 5. Using VP8 with RPSI and SLI Feedback 654 The VP8 payload descriptor defined in Section 4.2 above contains an 655 optional PictureID parameter. This parameter is included mainly to 656 enable use of reference picture selection index (RPSI) and slice loss 657 indication (SLI), both defined in [RFC4585]. 659 5.1. RPSI 661 The reference picture selection index is a payload-specific feedback 662 message defined within the RTCP-based feedback format. The RPSI 663 message is generated by a receiver and can be used in two ways. 664 Either it can signal a preferred reference picture when a loss has 665 been detected by the decoder -- preferably then a reference that the 666 decoder knows is perfect -- or, it can be used as positive feedback 667 information to acknowledge correct decoding of certain reference 668 pictures. The positive feedback method is useful for VP8 used as 669 unicast. The use of RPSI for VP8 is preferably combined with a 670 special update pattern of the codec's two special reference frames -- 671 the golden frame and the altref frame -- in which they are updated in 672 an alternating leapfrog fashion. When a receiver has received and 673 correctly decoded a golden or altref frame, and that frame had a 674 PictureID in the payload descriptor, the receiver can acknowledge 675 this simply by sending an RPSI message back to the sender. The 676 message body (i.e., the "native RPSI bit string" in [RFC4585]) is 677 simply the PictureID of the received frame. 679 5.2. SLI 681 The slice loss indication is another payload-specific feedback 682 message defined within the RTCP-based feedback format. The SLI 683 message is generated by the receiver when a loss or corruption is 684 detected in a frame. The format of the SLI message is as follows 685 [RFC4585]: 687 0 1 2 3 688 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 689 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 690 | First | Number | PictureID | 691 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 693 Figure 4 695 Here, First is the macroblock address (in scan order) of the first 696 lost block and Number is the number of lost blocks. PictureID is the 697 six least significant bits of the codec-specific picture identifier 698 in which the loss or corruption has occurred. For VP8, this codec- 699 specific identifier is naturally the PictureID of the current frame, 700 as read from the payload descriptor. If the payload descriptor of 701 the current frame does not have a PictureID, the receiver MAY send 702 the last received PictureID+1 in the SLI message. The receiver MAY 703 set the First parameter to 0, and the Number parameter to the total 704 number of macroblocks per frame, even though only parts of the frame 705 is corrupted. When the sender receives an SLI message, it can make 706 use of the knowledge from the latest received RPSI message. Knowing 707 that the last golden or altref frame was successfully received, it 708 can encode the next frame with reference to that established 709 reference. 711 5.3. Example 713 The use of RPSI and SLI is best illustrated in an example. In this 714 example, the encoder may not update the altref frame until the last 715 sent golden frame has been acknowledged with an RPSI message. If an 716 update is not received within some time, a new golden frame update is 717 sent instead. Once the new golden frame is established and 718 acknowledge, the same rule applies when updating the altref frame. 720 +-------+-------------------+-------------------------+-------------+ 721 | Event | Sender | Receiver | Established | 722 | | | | reference | 723 +-------+-------------------+-------------------------+-------------+ 724 | 1000 | Send golden frame | | | 725 | | PictureID = 0 | | | 726 | | | | | 727 | | | Receive and decode | | 728 | | | golden frame | | 729 | | | | | 730 | 1001 | | Send RPSI(0) | | 731 | | | | | 732 | 1002 | Receive RPSI(0) | | golden | 733 | | | | | 734 | ... | (sending regular | | | 735 | | frames) | | | 736 | | | | | 737 | 1100 | Send altref frame | | | 738 | | PictureID = 100 | | | 739 | | | | | 740 | | | Altref corrupted or | golden | 741 | | | lost | | 742 | | | | | 743 | 1101 | | Send SLI(100) | golden | 744 | | | | | 745 | 1102 | Receive SLI(100) | | | 746 | | | | | 747 | 1103 | Send frame with | | | 748 | | reference to | | | 749 | | golden | | | 750 | | | | | 751 | | | Receive and decode | golden | 752 | | | frame (decoder state | | 753 | | | restored) | | 754 | | | | | 755 | ... | (sending regular | | | 756 | | frames) | | | 757 | | | | | 758 | 1200 | Send altref frame | | | 759 | | PictureID = 200 | | | 760 | | | | | 761 | | | Receive and decode | golden | 762 | | | altref frame | | 763 | | | | | 764 | 1201 | | Send RPSI(200) | | 765 | | | | | 766 | 1202 | Receive RPSI(200) | | altref | 767 | | | | | 768 | ... | (sending regular | | | 769 | | frames) | | | 770 | | | | | 771 | 1300 | Send golden frame | | | 772 | | PictureID = 300 | | | 773 | | | | | 774 | | | Receive and decode | altref | 775 | | | golden frame | | 776 | | | | | 777 | 1301 | | Send RPSI(300) | altref | 778 | | | | | 779 | 1302 | RPSI lost | | | 780 | | | | | 781 | 1400 | Send golden frame | | | 782 | | PictureID = 400 | | | 783 | | | | | 784 | | | Receive and decode | altref | 785 | | | golden frame | | 786 | | | | | 787 | 1401 | | Send RPSI(400) | | 788 | | | | | 789 | 1402 | Receive RPSI(400) | | golden | 790 +-------+-------------------+-------------------------+-------------+ 792 Table 1: Exemple signaling between sender and receiver 794 Note that the scheme is robust to loss of the feedback messages. If 795 the RPSI is lost, the sender will try to update the golden (or 796 altref) again after a while, without releasing the established 797 reference. Also, if an SLI is lost, the receiver can keep sending 798 SLI messages at any interval, as long as the picture is corrupted. 800 6. Payload Format Parameters 802 This payload format has no parameters. 804 6.1. Media Type Definition 806 This registration is done using the template defined in [RFC4288] and 807 following [RFC4855]. 809 Type name: video 811 Subtype name: VP8 813 Required parameters: none 815 Optional parameters: none 817 Encoding considerations: 818 This media type is framed in RTP and contains binary data; see 819 Section 4.8 of [RFC4288]. 821 Security considerations: See Section 7 of RFC xxxx. 822 [RFC Editor: Upon publication as an RFC, please replace "XXXX" 823 with the number assigned to this document and remove this note.] 825 Interoperability considerations: None. 827 Published specification: VP8 bitstream format [RFC6386] and RFC 828 XXXX. 829 [RFC Editor: Upon publication as an RFC, please replace "XXXX" 830 with the number assigned to this document and remove this note.] 832 Applications which use this media type: 833 For example: Video over IP, video conferencing. 835 Additional information: None. 837 Person & email address to contact for further information: 838 Patrik Westin, patrik.westin@gmail.com 840 Intended usage: COMMON 842 Restrictions on usage: 843 This media type depends on RTP framing, and hence is only defined 844 for transfer via RTP [RFC3550]. 846 Author: Patrik Westin, patrik.westin@gmail.com 848 Change controller: 849 IETF Payload Working Group delegated from the IESG. 851 6.2. SDP Parameters 853 The receiver MUST ignore any parameter unspecified in this memo. 855 6.2.1. Mapping of MIME Parameters to SDP 857 The MIME media type video/VP8 string is mapped to fields in the 858 Session Description Protocol (SDP) [RFC2327] as follows: 860 o The media name in the "m=" line of SDP MUST be video. 862 o The encoding name in the "a=rtpmap" line of SDP MUST be VP8 (the 863 MIME subtype). 865 o The clock rate in the "a=rtpmap" line MUST be 90000. 867 6.2.1.1. Example 869 An example of media representation in SDP is as follows: 871 m=video 49170 RTP/AVPF 98 872 a=rtpmap:98 VP8/90000 874 6.2.2. Offer/Answer Considerations 876 This payload format has no parameters wherefore there are no offer/ 877 answer considerations. 879 7. Security Considerations 881 RTP packets using the payload format defined in this specification 882 are subject to the security considerations discussed in the RTP 883 specification [RFC3550], and in any applicable RTP profile. The main 884 security considerations for the RTP packet carrying the RTP payload 885 format defined within this memo are confidentiality, integrity and 886 source authenticity. Confidentiality is achieved by encryption of 887 the RTP payload. Integrity of the RTP packets through suitable 888 cryptographic integrity protection mechanism. Cryptographic system 889 may also allow the authentication of the source of the payload. A 890 suitable security mechanism for this RTP payload format should 891 provide confidentiality, integrity protection and at least source 892 authentication capable of determining if an RTP packet is from a 893 member of the RTP session or not. Note that the appropriate 894 mechanism to provide security to RTP and payloads following this memo 895 may vary. It is dependent on the application, the transport, and the 896 signaling protocol employed. Therefore a single mechanism is not 897 sufficient, although if suitable the usage of SRTP [RFC3711] is 898 recommended. This RTP payload format and its media decoder do not 899 exhibit any significant non-uniformity in the receiver-side 900 computational complexity for packet processing, and thus are unlikely 901 to pose a denial-of-service threat due to the receipt of pathological 902 data. Nor does the RTP payload format contain any active content. 904 8. Congestion Control 906 Congestion control for RTP SHALL be used in accordance with RFC 3550 907 [RFC3550], and with any applicable RTP profile; e.g., RFC 3551 908 [RFC3551]. The congestion control mechanism can, in a real-time 909 encoding scenario, adapt the transmission rate by instructing the 910 encoder to encode at a certain target rate. Media aware network 911 elements MAY use the information in the VP8 payload descriptor in 912 Section 4.2 to identify non-reference frames and discard them in 913 order to reduce network congestion. 915 9. IANA Considerations 917 The IANA is requested to register the following values: 918 - Media type registration as described in Section 6.1. 920 10. References 922 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 923 Requirement Levels", BCP 14, RFC 2119, March 1997. 925 [RFC2327] Handley, M. and V. Jacobson, "SDP: Session Description 926 Protocol", RFC 2327, April 1998. 928 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 929 Jacobson, "RTP: A Transport Protocol for Real-Time 930 Applications", STD 64, RFC 3550, July 2003. 932 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 933 Video Conferences with Minimal Control", STD 65, RFC 3551, 934 July 2003. 936 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 937 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 938 RFC 3711, March 2004. 940 [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and 941 Registration Procedures", BCP 13, RFC 4288, December 2005. 943 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 944 "Extended RTP Profile for Real-time Transport Control 945 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 946 July 2006. 948 [RFC4855] Casner, S., "Media Type Registration of RTP Payload 949 Formats", RFC 4855, February 2007. 951 [RFC6386] Bankoski, J., Koleszar, J., Quillio, L., Salonen, J., 952 Wilkins, P., and Y. Xu, "VP8 Data Format and Decoding 953 Guide", RFC 6386, November 2011. 955 Authors' Addresses 957 Patrik Westin 958 Google, Inc. 959 Kungsbron 2 960 Stockholm, 11122 961 Sweden 963 Email: patrik.westin@gmail.com 965 Henrik F Lundin 966 Google, Inc. 967 Kungsbron 2 968 Stockholm, 11122 969 Sweden 971 Email: hlundin@google.com 973 Michael Glover 974 Google, Inc. 976 Justin Uberti 977 Google, Inc. 979 Frank Galligan 980 Google, Inc.