idnits 2.17.1 draft-ietf-avt-mpeg-new-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-03-19) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (December 1997) is 9591 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. '1' -- Possible downref: Non-RFC (?) normative reference: ref. '2' ** Obsolete normative reference: RFC 1889 (ref. '3') (Obsoleted by RFC 3550) ** Obsolete normative reference: RFC 1890 (ref. '4') (Obsoleted by RFC 3551) Summary: 10 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Audio/Video Transport Working Group 2 INTERNET-DRAFT D. Hoffman 3 draft-ietf-avt-mpeg-new-01.txt G. Fernando 4 Sun Microsystems, Inc. 5 V. Goyal 6 Precept Software, Inc. 7 M. Reha Civanlar 8 AT&T Labs - Research 9 June 1997 10 Expires December 1997 12 RTP Payload Format for MPEG1/MPEG2 Video 14 Status of this Memo 16 This document is an Internet-Draft. Internet-Drafts are working 17 documents of the Internet Engineering Task Force (IETF), its areas, 18 and its working groups. Note that other groups may also distribute 19 working documents as Internet-Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 To learn the current status of any Internet-Draft, please check the 27 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 28 Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), 29 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 30 ftp.isi.edu (US West Coast). 32 Distribution of this document is unlimited. 34 Abstract 36 This memo describes a packetization scheme for MPEG video and audio 37 streams. The scheme proposed can be used to transport such a video 38 or audio flow over the transport protocols supported by RTP. Two 39 approaches are described. The first is designed to support maximum 40 interoperability with MPEG System environments. The second is 41 designed to provide maximum compatibility with other RTP-encapsulated 42 media streams and future conference control work of the IETF. 44 This memo is a revision of RFC 2038, an Internet standards track pro- 45 tocol, prepared for publication as a revised RFC. In this revision, 46 the packet loss resilience mechanisms in Section 3.4 were extended to 47 include additional picture header information required for MPEG2. 49 1. Introduction 51 ISO/IEC JTC1/SC29 WG11 (also referred to as the MPEG committee) has 52 defined the MPEG1 standard (ISO/IEC 11172)[1] and the MPEG2 standard 53 (ISO/IEC 13818)[2]. This memo describes a packetization scheme to 54 transport MPEG video and audio streams using the Real-time Transport 55 Protocol (RTP), version 2 [3, 4]. 57 The MPEG1 specification is defined in three parts: System, Video and 58 Audio. It is designed primarily for CD-ROM-based applications, and 59 is optimized for approximately 1.5 Mbits/sec combined data rates. The 60 video and audio portions of the specification describe the basic for- 61 mat of the video or audio stream. These formats define the Elemen- 62 tary Streams (ES). The MPEG1 System specification defines an encap- 63 sulation of the ES that contains Presentation Time Stamps (PTS), 64 Decoding Time Stamps and System Clock references, and performs multi- 65 plexing of MPEG1 compressed video and audio ES's with user data. 67 The MPEG2 specification is structured in a similar way. However, it 68 hasn't been restricted only to CD-ROM applications. The MPEG2 System 69 specification defines two system stream formats: the MPEG2 Transport 70 Stream (MTS) and the MPEG2 Program Stream (MPS). The MTS is tailored 71 for communicating or storing one or more programs of MPEG2 compressed 72 data and also other data in relatively error-prone environments. The 73 MPS is tailored for relatively error-free environments. 75 We seek to achieve interoperability among 4 types of end-systems in 76 the following specification. The 4 types are: 78 1. Transmitting Interworking Unit (TIU) 80 Receives MPEG information from a native MTS system for 81 distribution over packet networks using a native RTP-based 82 system layer (such as an IP-based internetwork). Examples: 83 real-time encoder, MTS satellite link to Internet, video 84 server with MTS-encoded source material. 86 2. Receiving Interworking Unit (RIU) 88 Receives MPEG information in real time from an RTP-based 89 network for forwarding to a native MTS environment. 90 Examples: Internet-based video server to MTS-based cable 91 distribution plant. 93 3. Transmitting Internet End-System (TAES) 95 Transmits MPEG information generated or stored within the 96 internet end-system itself, or received from internet-based 97 computer networks. Example: video server. 99 4. Receiving Internet End-System (RAES) 101 Receives MPEG information over an RTP-based internet for 102 consumption at the internet end-system or forwarding to 103 traditional computer network. Example: desktop PC or 104 workstation viewing training video. 106 Each of the 2 types of transmitters must work with each of the 2 107 types of receivers. Because it is probable that the TAES, and cer- 108 tain that the RAES, will be based on existing and planned internet- 109 connected computers, it is highly desirable for the interoperable 110 protocol to be based on RTP. 112 Because of the range of applications that might employ MPEG streams, 113 we propose to define two payload formats. 115 Much interest in the MPEG community is in the use of one of the MPEG 116 System encodings, and hence, in Section 2 we propose encapsulations 117 of MPEG1 System streams and MPEG2 Transport and Program Streams with 118 RTP. This profile supports the full semantics of MPEG System and 119 offers basic interoperability among all four end-system types. 121 When operating only among internet-based end-systems (i.e., TAES and 122 RAES) a payload format that provides greater compatibility with the 123 Internet architecture is desired, deferring some of the system issues 124 to other protocols being defined in the Internet community (such as 125 the MMUSIC WG). In Section 3 we propose an encapsulation of 126 compressed video and audio data (referred to in MPEG documentation as 127 "Elementary Streams" (ES)) complying with either MPEG1 or MPEG2. 128 Here, neither of the System standards of MPEG1 or MPEG2 are utilized. 129 The ES's are directly encapsulated with RTP. 131 Throughout this specification, we make extensive use of MPEG termi- 132 nology. The reader should consult the primary MPEG references for 133 definitive descriptions of this terminology. 135 2. Encapsulation of MPEG System and Transport Streams 137 Each RTP packet will contain a timestamp derived from the sender's 138 90KHz clock reference. This clock is synchronized to the system 139 stream Program Clock Reference (PCR) or System Clock Reference (SCR) 140 and represents the target transmission time of the first byte of the 141 packet payload. The RTP timestamp will not be passed to the MPEG 142 decoder. This use of the timestamp is somewhat different than nor- 143 mally is the case in RTP, in that it is not considered to be the 144 media display or presentation timestamp. The primary purposes of the 145 RTP timestamp will be to estimate and reduce any network-induced 146 jitter and to synchronize relative time drift between the transmitter 147 and receiver. 149 For MPEG2 Transport Streams the RTP payload will contain an integral 150 number of MPEG transport packets. To avoid end system inefficien- 151 cies, data from multiple small MTS packets (normally fixed in size at 152 188 bytes) are aggregated into a single RTP packet. The number of 153 transport packets contained is computed by dividing RTP payload 154 length by the length of an MTS packet (188). 156 For MPEG2 Program streams and MPEG1 system streams there are no pack- 157 etization restrictions; these streams are treated as a packetized 158 stream of bytes. 160 2.1 RTP header usage 162 The RTP header fields are used as follows: 164 Payload Type: Distinct payload types should be assigned for 165 MPEG1 System Streams, MPEG2 Program Streams and MPEG2 166 Transport Streams. See [4] for payload type assignments. 168 M bit: Set to 1 whenever the timestamp is discontinuous 169 (such as might happen when a sender switches from one data 170 source to another). This allows the receiver and any 171 intervening RTP mixers or translators that are synchronizing 172 to the flow to ignore the difference between this timestamp 173 and any previous timestamp in their clock phase detectors. 175 timestamp: 32 bit 90K Hz timestamp representing the target 176 transmission time for the first byte of the packet. 178 3. Encapsulation of MPEG Elementary Streams 180 The following ES types may be encapsulated directly in RTP: 182 (a) MPEG1 Video (ISO/IEC 11172-2) 183 (b) MPEG2 Video (ISO/IEC 13818-2) 184 (c) MPEG1 Audio (ISO/IEC 11172-3) 185 (d) MPEG2 Audio (ISO/IEC 13818-3) 187 A distinct RTP payload type is assigned to MPEG1/MPEG2 Video and 188 MPEG1/MPEG2 Audio, respectively. Further indication as to whether the 189 data is MPEG1 or MPEG2 need not be provided in the RTP or MPEG- 190 specific headers of this encapsulation, as this information is avail- 191 able in the ES headers. 193 Presentation Time Stamps (PTS) of 32 bits with an accuracy of 90 kHz 194 shall be carried in the fixed RTP header. All packets that make up a 195 audio or video frame shall have the same time stamp. 197 3.1 MPEG Video elementary streams 199 MPEG1 Video can be distinguished from MPEG2 Video at the video 200 sequence header, i.e. for MPEG2 Video a sequence_header() is followed 201 by sequence_extension(). The particular profile and level of MPEG2 202 Video (MAIN_Profile@MAIN_Level, HIGH_Profile@HIGH_Level, etc) are 203 determined by the profile_and_level_indicator field of the 204 sequence_extension header of MPEG2 Video. 206 The MPEG bit-stream semantics were designed for relatively error-free 207 environments, and there is significant amount of dependency (both 208 temporal and spatial) within the stream such that loss of some data 209 make other uncorrupted data useless. The format as defined in this 210 encapsulation uses application layer framing information plus addi- 211 tional information in the RTP stream-specific header to allow for 212 certain recovery mechanisms. Appendix 1 suggests several recovery 213 strategies based on the properties of this encapsulation. 215 Since MPEG pictures can be large, they will normally be fragmented 216 into packets of size less than a typical LAN/WAN MTU. The following 217 fragmentation rules apply: 219 1. The MPEG Video_Sequence_Header, when present, will always 220 be at the beginning of an RTP payload. 221 2. An MPEG GOP_header, when present, will always be at the 222 beginning of the RTP payload, or will follow a 223 Video_Sequence_Header. 224 3. An MPEG Picture_Header, when present, will always be at the 225 beginning of a RTP payload, or will follow a GOP_header. 227 Each ES header must be completely contained within the packet. Con- 228 sequently, a minimum RTP payload size of 261 bytes must be supported 229 to contain the largest single header defined in the ES (that is, the 230 extension_data() header containing the quant_matrix_extension()). 231 Otherwise, there are no restrictions on where headers may appear 232 within packet payloads. 234 In MPEG, each picture is made up of one or more "slices," and a slice 235 is intended to be the unit of recovery from data loss or corruption. 236 An MPEG-compliant decoder will normally advance to the beginning of 237 next slice whenever an error is encountered in the stream. MPEG 238 slice begin and end bits are provided in the encapsulation header to 239 facilitate this. 241 The beginning of a slice must either be the first data in a packet 242 (after any MPEG ES headers) or must follow after some integral number 243 of slices in a packet. This requirement insures that the beginning 244 of the next slice after one with a missing packet can be found 245 without requiring that the receiver scan the packet contents. Slices 246 may be fragmented across packets as long as all the above rules are 247 met. 249 An implementation based on this encapsulation assumes that the 250 Video_Sequence_Header is repeated periodically in the MPEG bit- 251 stream. In practice (though not required by MPEG standard) this is 252 used to allow channel switching and to receive and start decoding a 253 continuously relayed MPEG bit-stream at arbitrary points in the media 254 stream. It is suggested that when playing back from an MPEG stream 255 from a file format (where the Video_Sequence_Header may only be 256 represented at the beginning of the stream) that the first 257 Video_Sequence_Header (preceded by an end-of-stream indicator) be 258 saved by the packetizer for periodic injection in to the network 259 stream. 261 3.2 MPEG Audio elementary streams 263 MPEG1 Audio can be distinguished from MPEG2 Audio from the MPEG 264 ancillary_data() header. For either MPEG1 or MPEG2 Audio, distinct 265 Presentation Time Stamps may be present for frames which correspond 266 to either 384 samples for Layer-I, or 1152 samples for Layer-II or 267 Layer-III. The actual number of bytes required to represent this 268 number of samples will vary depending on the encoder parameters. 270 Multiple audio frames may be encapsulated within one RTP packet. In 271 this case, an integral number of audio frames must be contained 272 within the packet and the fragmentation header defined in Section 3.5 273 shall be set to 0. 275 Also, if relatively short packets are to be used, one frame may be so 276 large that it may straddle multiple RTP packets. For example, for 277 Layer-II MPEG audio sampled at a rate of 44.1 KHz each frame would 278 represent a time slot of 26.1 msec. At this sampling rate if the 279 compressed bit-rate is 384 kbits/sec (i.e. 48 kBytes/sec) then the 280 average audio frame size would be 1.25 KBytes. If packets were to be 281 500 Bytes long, then each audio frame would straddle 3 RTP packets. 282 The audio fragmentation indicator header (See Section 3.5) shall be 283 present for an MPEG1/2 Audio payload type to provide for this frag- 284 mentation. 286 3.3 RTP Fixed Header for MPEG ES encapsulation 288 The RTP header fields are used as follows: 290 Payload Type: Distinct payload types should be assigned 291 for video elementary streams and audio elementary streams. 292 See [4] for payload type assignments. 294 M bit: For video, set to 1 on packet containing MPEG frame 295 end code, 0 otherwise. For audio, set to 1 on first packet 296 of a "talk-spurt," 0 otherwise. 298 PT: MPEG video or audio stream ID. 300 timestamp: 32-bit 90K Hz timestamp representing presentation 301 time of MPEG picture or audio frame. Same for all packets 302 that make up a picture or audio frame. May not be 303 monotonically increasing in video stream if B pictures 304 present in stream. For packets that contain only a video 305 sequence and/or GOP header, the timestamp is that of the 306 subsequent picture. 308 3.4 MPEG Video-specific header 310 This header shall be attached to each RTP packet after the RTP fixed 311 header. 313 0 1 2 3 314 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 315 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 316 | MBZ |T| TR | |N|S|B|E| P | | BFC | | FFC | 317 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 318 AN FBV FFV 320 MBZ: Unused. Must be set to zero in current 321 specification. This space is reserved for future use. 323 T: MPEG-2 (Two) specific header extension present (1 bit). 324 Set to 1 when the MPEG-2 video-specific header extension 325 (see Section 3.4.1) follows this header. This extension 326 may be needed for improved error resilience; however, 327 its inclusion in an RTP packet is optional. (See Appendix 328 1.) 330 TR: Temporal-Reference (10 bits). The temporal reference of 331 the current picture within the current GOP. This value 332 ranges from 0-1023 and is constant for all RTP packets of a 333 given picture. 335 AN: Active N bit for error resilience (1 bit). Set to 1 when 336 the following bit (N) is used to signal changes in the 337 picture header information for MPEG-2 payloads. It must be 338 set to 0 for MPEG-1 payloads or when N bit is not used. 340 N: New picture header (1 bit). Used for MPEG-2 payloads when 341 the previous bit (AN) is set to 1. Otherwise, it must 342 be set to zero. Set to 1 when the information contained in 343 the previously transmitted Picture Headers can't be used to 344 reconstruct a header for the current picture. This happens 345 when the current picture is encoded using a different 346 set of parameters than the previous pictures of the same 347 type. The N bit must be constant for all RTP packets that 348 belong to the same picture so that receipt of any packet 349 from a picture allows detecting whether information 350 necessary for reconstruction was contained in that picture 351 (N = 1) or a previous one (N = 0). 353 S: Sequence-header-present (1 bit). Normally 0 and set to 1 at 354 the occurrence of each MPEG sequence header. Used to 355 detect presence of sequence header in RTP packet. 357 B: Beginning-of-slice (BS) (1 bit). Set when the start of the 358 packet payload is a slice start code, or when a slice start 359 code is preceded only by one or more of a 360 Video_Sequence_Header, GOP_header and/or Picture_Header. 362 E: End-of-slice (ES) (1 bit). Set when the last byte of the 363 payload is the end of an MPEG slice. 365 P: Picture-Type (3 bits). I (1), P (2), B (3) or D (4). This 366 value is constant for each RTP packet of a given picture. 367 Value 000B is forbidden and 101B - 111B are reserved to 368 support future extensions to the MPEG ES specification. 370 FBV: full_pel_backward_vector 371 BFC: backward_f_code 372 FFV: full_pel_forward_vector 373 FFC: forward_f_code 374 Obtained from the most recent picture header, and are 375 constant for each RTP packet of a given picture. For I 376 frames none of these values are present in the picture 377 header and they must be set to zero in the RTP header. 378 For P frames only the last two values are present and 379 FBV and BFC must be set to zero in the RTP header. For 380 B frames all the four values are present. 382 3.4.1 MPEG-2 Video-specific header extension 384 0 1 2 3 385 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 386 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 387 |X|E|f_[0,0]|f_[0,1]|f_[1,0]|f_[1,1]| DC| PS|T|P|C|Q|V|A|R|H|G|D| 388 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 390 X: Unused (1 bit). Must be set to zero in current 391 specification. This space is reserved for future use. 393 E: Extensions present (1 bit). If set to 1, this header 394 extension, including the composite display extension 395 when D = 1, will be followed by one or more of the 396 following extensions: quant matrix extension, picture 397 display extension, picture temporal scalable extension, 398 picture spatial scalable extension and copyright extension. 400 The first byte of these extensions data gives the length of 401 the extensions in 32 bit words including the length field 402 itself. Zero padding bytes are used at the end if required 403 to align the extensions to 32 bit boundary. 405 Since they may not be vital in decoding of a picture, the 406 inclusion of any one of these extensions in an RTP packet 407 is optional even when the MPEG-2 video-specific header 408 extension is included in the packet (T = 1). (See Appendix 409 1.) If present, they should be copied from the 410 corresponding extensions following the most recent MPEG-2 411 picture coding extension and they remain constant for each 412 RTP packet of a given picture. 414 The extension start code (32 bits) and the extension start 415 code ID (4 bits) are included. Therefore the extensions are 416 self identifying. 418 f_[0,0]: forward horizontal f_code (4 bits) 419 f_[0,1]: forward vertical f_code (4 bits) 420 f_[1,0]: backward horizontal f_code (4 bits) 421 f_[1,1]: backward vertical f_code (4 bits) 422 DC: intra_DC_precision (2 bits) 423 PS: picture_structure (2 bits) 424 T: top_field_first (1 bit) 425 P: frame_predicted_frame_dct (1 bit) 426 C: concealment_motion_vectors (1 bit) 427 Q: q_scale type (1 bit) 428 V: intra_vlc_format (1 bit) 429 A: alternate scan (1 bit) 430 R: repeat_first_field (1 bit) 431 H: chroma_420_type (1 bit) 432 G: progressive frame (1 bit) 433 D: composite_display_flag (1 bit). If set to 1, next 32 bits 434 following this one contains 12 zeros followed by 20 bits 435 of composite display information. 437 These values are copied from the most recent picture 438 coding extension and are constant for each RTP packet 439 of a given picture. Their meanings are as explained in 440 the MPEG-2 standard. 442 3.5 MPEG Audio-specific header 444 This header shall be attached to each RTP packet at the start of the 445 payload and after any RTP headers for an MPEG1/2 Audio payload type. 447 0 1 2 3 448 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 | MBZ | Frag_offset | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 453 Frag_offset: Byte offset into the audio frame for the data 454 in this packet. 456 Appendix 1. Error Recovery and Resynchronization Strategies. 458 The following error recovery and resynchronization strategies are 459 intended to be guidelines only. A compliant receiver is free to 460 employ alternative (or no) strategies. 462 When initially decoding an RTP-encapsulated MPEG Elementary Stream, 463 the receiver may discard all packets until the Sequence-header- 464 present bit is set to 1. At this point, sufficient state information 465 is contained in the stream to allow processing by an MPEG decoder. 467 Loss of packets containing the GOP_header and/or Picture_Header are 468 detected by an unexpected change in the Temporal-Reference and 469 Picture-Type values. Consider the following example GOP sequence: 471 In display order: 0B 1B 2I 3B 4B 5P 6B 7B 8P GOP_HDR 0B ... 472 In stream order: 2I 0B 1B 5P 3B 4B 8P 6B 7B GOP_HDR 2I ... 474 Consider also two counters: 476 ref_pic_temp (Reference Picture (I,P) Temporal Reference) 477 dep_pic_temp (Dependent Picture (B) Temporal Reference) 479 At each GOP beginning, set these counters to the temporal reference 480 value of the corresponding picture type. For our example GOP 481 sequence, ref_pic_temp = 2 and dep_pic_temp = 0. Keep incrementing 482 BOTH counters by unity with each following picture. Ref_pic_temp 483 should match the temporal references of the I and P frames, and 484 dep_pic_temp should match the temporal references of the B frames. 486 dep_pic_temp: - 0 1 2 3 4 5 6 7 8 9 487 In stream order: 2I 0B 1B 5P 3B 4B 8P 6B 7B GOP_H 2I 0B 1B ... 488 ref_pic_temp: 2 3 4 5 6 7 8 9 10 ^ 11 489 -------------------------- | ^ 490 Match Drop | 491 Mismatch 492 in ref_pic_temp 494 The loss of a GOP header can be detected by matching the appropriate 495 counter (based on picture type) to the temporal reference value. A 496 mismatch indicates a lost GOP header. If desired, a GOP header can be 497 re-constructed using a "null" time_code, repeating the closed_gop 498 flag from previous GOP headers, and setting the broken_link flag to 499 1. 501 The loss of a Picture_Header can also be detected by a mismatch in 502 the Temporal Reference contained in the RTP packet from the appropri- 503 ate dep_pic_temp or ref_pic_temp counters at the receiver. For MPEG-1 504 payloads, after scanning to the next Beginning-of-slice the 505 Picture_Header is reconstructed from the P, TR, FBV, BFC, FFV and FFC 506 contained in that packet, and from stream-dependent default values. 508 For MPEG-2, additional information is needed for the reconstruction. 509 This information is provided by the MPEG-2 video specific header 510 extension contained in that packet if the T bit is set to 1, or the 511 Picture Header for the current picture may be available from previous 512 packets belonging to the same picture. The transmitter's strategy for 513 inclusion of the MPEG-2 video specific header extension may depend 514 upon a number of factors. This header may not be needed when: 516 1. the information has been transmitted a sufficient number of 517 times in previous packets to assure reception with the desired 518 probability, or 520 2. the information is transmitted over a separate reliable 521 channel, or 523 3. expected loss rates are low enough that missed frames are 524 not a concern, or 526 4. conserving bandwidth is more important than error resilience, 527 etc. 529 If T=1 and E=0, there may be extensions present in the original video 530 bitstream that are not included in the current packet. The 531 transmitter may choose not to include extensions in a packet when 532 they are not necessary for decoding or if one of the cases listed 533 above for not including the MPEG-2 video specific header extension in 534 a packet applies only to the extension data. 536 If N=0, then the Picture Header from a previous picture of the same 537 type (I,P or B) may be used so long as at least one packet has been 538 received for every intervening picture of the same type and that the 539 N bit was 0 for each of those pictures. This may involve: 541 1. Saving the relevant picture header information that can be 542 obtained from the MPEG-2 video specific header extension or 543 directly from the video bitstream for each picture type, 545 2. Keeping validity indicators for this saved information based 546 on the received N bits and lost packets, and, 548 3. Updating the data whenever a packet with N=1 is received. 550 If the necessary information is not available from any of these 551 sources, data deletion until a new picture start code is advised. 553 Any time an RTP packet is lost (as indicated by a gap in the RTP 554 sequence number), the receiver may discard all packets until the 555 Beginning-of-slice bit is set. At this point, sufficient state 556 information is contained in the stream to allow processing by an MPEG 557 decoder starting at the next slice boundary (possibly after recon- 558 struction of the GOP_header and/or Picture_Header as described 559 above). 561 References 563 [1] ISO/IEC International Standard 11172; "Coding of moving pictures 564 and associated audio for digital storage media up to about 1,5 565 Mbits/s", November 1993. 567 [2] ISO/IEC International Standard 13818; "Generic coding of moving 568 pictures and associated audio information", November 1994. 570 [3] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson, 571 "RTP: A Transport Protocol for Real-Time Applications", 572 RFC 1889, January 1996. 574 [4] H. Schulzrinne, "RTP Profile for Audio and Video Conferences 575 with Minimal Control", RFC 1890, January 1996. 577 Authors' Addresses 579 Gerard Fernando 580 Sun Microsystems, Inc. 581 Mail-stop UMPK14-305 582 2550 Garcia Avenue 583 Mountain View, California 94043-1100 584 USA 586 Phone: +1 415-786-6373 587 EMail: gerard.fernando@eng.sun.com 589 Vivek Goyal 590 Precept Software, Inc. 591 1072 Arastradero Rd, 592 Palo Alto, CA 94304 593 USA 595 Phone: +1 415-845-5200 596 EMail: goyal@precept.com 598 Don Hoffman 599 Sun Microsystems, Inc. 600 Mail-stop UMPK14-305 601 2550 Garcia Avenue 602 Mountain View, California 94043-1100 603 USA 605 Phone: +1 503-297-1580 606 EMail: don.hoffman@eng.sun.com 608 M. Reha Civanlar 609 AT&T Labs - Research 610 100 Schulz Drive, 3-213 611 Red Bank, NJ 07701-7033 612 USA 614 Phone: +1 732-345-3305 615 EMail: civanlar@research.att.com