idnits 2.17.1 draft-ietf-payload-rfc3016bis-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to contain a disclaimer for pre-RFC5378 work, but was first submitted on or after 10 November 2008. The disclaimer is usually necessary only for documents that revise or obsolete older RFCs, and that take significant amounts of text from those RFCs. If you can contact all authors of the source material and they are willing to grant the BCP78 rights to the IETF Trust, you can and should remove the disclaimer. Otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 12, 2011) is 4607 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCxxxx' is mentioned on line 1026, but not defined -- Possible downref: Non-RFC (?) normative reference: ref. '14496-2' -- Possible downref: Non-RFC (?) normative reference: ref. '14496-3' -- Possible downref: Non-RFC (?) normative reference: ref. '23003-1' ** Obsolete normative reference: RFC 4288 (Obsoleted by RFC 6838) ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) -- Obsolete informational reference (is this intentional?): RFC 3016 (Obsoleted by RFC 6416) -- Obsolete informational reference (is this intentional?): RFC 5246 (Obsoleted by RFC 8446) Summary: 2 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Audio/Video Transport Payloads M. Schmidt 3 Internet-Draft Dolby Laboratories 4 Obsoletes: 3016 (if approved) F. de Bont 5 Intended status: Standards Track Philips Electronics 6 Expires: March 15, 2012 S. Doehla 7 Fraunhofer IIS 8 Jaehwan. Kim 9 LG Electronics Inc. 10 September 12, 2011 12 RTP Payload Format for MPEG-4 Audio/Visual Streams 13 draft-ietf-payload-rfc3016bis-03.txt 15 Abstract 17 This document describes Real-Time Transport Protocol (RTP) payload 18 formats for carrying each of MPEG-4 Audio and MPEG-4 Visual 19 bitstreams without using MPEG-4 Systems. It is a revision of RFC 20 3016 and is needed because of some misalignments between RFC 3016 and 21 the 3GPP PSS specification regarding the RTP payload format for 22 MPEG-4 Audio. 24 For the purpose of directly mapping MPEG-4 Audio/Visual bitstreams 25 onto RTP packets, this document provides specifications for the use 26 of RTP header fields and also specifies fragmentation rules. It also 27 provides specifications for Media Type registration and the use of 28 Session Description Protocol (SDP). The audio payload format 29 described in this document has some limitations related to the 30 signaling of audio codec parameters for the required multiplexing 31 format. Therefore, for new system designs should be utilize RFC 3640 32 which does not have these restrictions. Nevertheless, this revision 33 of RFC 3016 is provided to update and complete the specification, and 34 to enable interopeable implementations. 36 This document obsoletes RFC 3016. It contains a summary of changes 37 from RFC 3016 and discussed backward compatibility to RFC 3016. 39 Status of this Memo 41 This Internet-Draft is submitted in full conformance with the 42 provisions of BCP 78 and BCP 79. 44 Internet-Drafts are working documents of the Internet Engineering 45 Task Force (IETF). Note that other groups may also distribute 46 working documents as Internet-Drafts. The list of current Internet- 47 Drafts is at http://datatracker.ietf.org/drafts/current/. 49 Internet-Drafts are draft documents valid for a maximum of six months 50 and may be updated, replaced, or obsoleted by other documents at any 51 time. It is inappropriate to use Internet-Drafts as reference 52 material or to cite them other than as "work in progress." 54 This Internet-Draft will expire on March 15, 2012. 56 Copyright Notice 58 Copyright (c) 2011 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (http://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with respect 66 to this document. Code Components extracted from this document must 67 include Simplified BSD License text as described in Section 4.e of 68 the Trust Legal Provisions and are provided without warranty as 69 described in the Simplified BSD License. 71 This document may contain material from IETF Documents or IETF 72 Contributions published or made publicly available before November 73 10, 2008. The person(s) controlling the copyright in some of this 74 material may not have granted the IETF Trust the right to allow 75 modifications of such material outside the IETF Standards Process. 76 Without obtaining an adequate license from the person(s) controlling 77 the copyright in such materials, this document may not be modified 78 outside the IETF Standards Process, and derivative works of it may 79 not be created outside the IETF Standards Process, except to format 80 it for publication as an RFC or to translate it into languages other 81 than English. 83 Table of Contents 85 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 86 1.1. MPEG-4 Visual RTP Payload Format . . . . . . . . . . . . . 4 87 1.2. MPEG-4 Audio RTP Payload Format . . . . . . . . . . . . . 5 88 1.3. Interoperability with RFC 3016 . . . . . . . . . . . . . . 5 89 1.4. Relation with RFC 3640 . . . . . . . . . . . . . . . . . . 6 90 2. Definitions and Abbreviations . . . . . . . . . . . . . . . . 6 91 3. Clarifications on specifying codec configurations for 92 MPEG-4 Audio . . . . . . . . . . . . . . . . . . . . . . . . . 7 93 4. LATM Restrictions for RTP Packetization of MPEG-4 Audio 94 Bitstreams . . . . . . . . . . . . . . . . . . . . . . . . . . 7 95 5. RTP Packetization of MPEG-4 Visual Bitstreams . . . . . . . . 8 96 5.1. Use of RTP Header Fields for MPEG-4 Visual . . . . . . . . 9 97 5.2. Fragmentation of MPEG-4 Visual Bitstream . . . . . . . . . 9 98 5.3. Examples of Packetized MPEG-4 Visual Bitstream . . . . . . 11 99 6. RTP Packetization of MPEG-4 Audio Bitstreams . . . . . . . . . 14 100 6.1. RTP Packet Format . . . . . . . . . . . . . . . . . . . . 14 101 6.2. Use of RTP Header Fields for MPEG-4 Audio . . . . . . . . 15 102 6.3. Fragmentation of MPEG-4 Audio Bitstream . . . . . . . . . 16 103 7. Media Type Registration for MPEG-4 Audio/Visual Streams . . . 16 104 7.1. Media Type Registration for MPEG-4 Visual . . . . . . . . 16 105 7.2. Mapping to SDP for MPEG-4 Visual . . . . . . . . . . . . . 18 106 7.2.1. Declarative SDP Usage for MPEG-4 Visual . . . . . . . 19 107 7.3. Media Type Registration for MPEG-4 Audio . . . . . . . . . 19 108 7.4. Mapping to SDP for MPEG-4 Audio . . . . . . . . . . . . . 23 109 7.4.1. Declarative SDP Usage for MPEG-4 Audio . . . . . . . . 23 110 7.4.1.1. Example: In-band Configuration . . . . . . . . . . 24 111 7.4.1.2. Example: 6kb/s CELP . . . . . . . . . . . . . . . 24 112 7.4.1.3. Example: 64 kb/s AAC LC Stereo . . . . . . . . . . 24 113 7.4.1.4. Example: Use of the SBR-enabled Parameter . . . . 25 114 7.4.1.5. Example: Hierarchical Signaling of SBR . . . . . . 25 115 7.4.1.6. Example: HE AAC v2 Signaling . . . . . . . . . . . 26 116 7.4.1.7. Example: Hierarchical Signaling of PS . . . . . . 26 117 7.4.1.8. Example: MPEG Surround . . . . . . . . . . . . . . 26 118 7.4.1.9. Example: MPEG Surround with Extended SDP 119 Parameters . . . . . . . . . . . . . . . . . . . . 27 120 7.4.1.10. Example: MPEG Surround with Single Layer 121 Configuration . . . . . . . . . . . . . . . . . . 27 122 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28 123 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 28 124 10. Security Considerations . . . . . . . . . . . . . . . . . . . 28 125 11. Differences to RFC 3016 . . . . . . . . . . . . . . . . . . . 29 126 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 30 127 12.1. Normative References . . . . . . . . . . . . . . . . . . . 30 128 12.2. Informative References . . . . . . . . . . . . . . . . . . 31 129 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 32 131 1. Introduction 133 The RTP payload formats described in this document specify how MPEG-4 134 Audio [14496-3] and MPEG-4 Visual streams [14496-2] are to be 135 fragmented and mapped directly onto RTP packets. 137 These RTP payload formats enable transport of MPEG-4 Audio/Visual 138 streams without using the synchronization and stream management 139 functionality of MPEG-4 Systems [14496-1]. Such RTP payload formats 140 will be used in systems that have intrinsic stream management 141 functionality and thus require no such functionality from MPEG-4 142 Systems. H.323 [H323] terminals are an example of such systems, 143 where MPEG-4 Audio/Visual streams are not managed by MPEG-4 Systems 144 Object Descriptors but by H.245 [H245]. The streams are directly 145 mapped onto RTP packets without using the MPEG-4 Systems Sync Layer. 146 Other examples are Session Initiation Protocol (SIP) [RFC3261] and 147 RTSP where Media Type and SDP are used. Media Type and SDP usages of 148 the RTP payload formats described in this document are defined to 149 directly specify the attribute of Audio/Visual streams (e.g., media 150 type, packetization format and codec configuration) without using 151 MPEG-4 Systems. The obvious benefit is that these MPEG-4 Audio/ 152 Visual RTP payload formats can be handled in an unified way together 153 with those formats defined for non-MPEG-4 codecs. The disadvantage 154 is that interoperability with environments using MPEG-4 Systems may 155 be difficult, hence, other payload formats may be better suited to 156 those applications. 158 The semantics of RTP headers in such cases need to be clearly 159 defined, including the association with MPEG-4 Audio/Visual data 160 elements. In addition, it is beneficial to define the fragmentation 161 rules of RTP packets for MPEG-4 Video streams so as to enhance error 162 resiliency by utilizing the error resiliency tools provided inside 163 the MPEG-4 Video stream. 165 1.1. MPEG-4 Visual RTP Payload Format 167 MPEG-4 Visual is a visual coding standard with many features: high 168 coding efficiency; high error resiliency; multiple, arbitrary shape 169 object-based coding; etc. [14496-2]. It covers a wide range of 170 bitrates from scores of Kbps to several Mbps. It also covers a wide 171 variety of networks, ranging from those guaranteed to be almost 172 error-free to mobile networks with high error rates. 174 With respect to the fragmentation rules for an MPEG-4 Visual 175 bitstream defined in this document, since MPEG-4 Visual is used for a 176 wide variety of networks, it is desirable not to apply too much 177 restriction on fragmentation, and a fragmentation rule such as "a 178 single video packet shall always be mapped on a single RTP packet" 179 may be inappropriate. On the other hand, careless, media unaware 180 fragmentation may cause degradation in error resiliency and bandwidth 181 efficiency. The fragmentation rules described in this document are 182 flexible but manage to define the minimum rules for preventing 183 meaningless fragmentation while utilizing the error resiliency 184 functionalities of MPEG-4 Visual. 186 The fragmentation rule "Different Video Object Planes (VOPs) SHOULD 187 be fragmented into different RTP packets" is made so that the RTP 188 timestamp uniquely indicates the VOP time framing. On the other 189 hand, MPEG-4 video may generate VOPs of very small size, in cases 190 with an empty VOP (vop_coded=0) containing only VOP header or an 191 arbitrary shaped VOP with a small number of coding blocks. To reduce 192 the overhead for such cases, the fragmentation rule permits 193 concatenating multiple VOPs in an RTP packet. (See fragmentation 194 rule (4) in Section 5.2 and marker bit and timestamp in Section 5.1.) 196 While the additional media specific RTP header defined for such video 197 coding tools as H.261 [H261] or MPEG-1/2 is effective in helping to 198 recover picture headers corrupted by packet losses, MPEG-4 Visual has 199 already error resiliency functionalities for recovering corrupt 200 headers, and these can be used on RTP/IP networks as well as on other 201 networks (H.223/mobile, MPEG-2/TS, etc.). Therefore, no extra RTP 202 header fields are defined in this MPEG-4 Visual RTP payload format. 204 1.2. MPEG-4 Audio RTP Payload Format 206 MPEG-4 Audio is an audio standard that integrates many different 207 types of audio coding tools. Low-overhead MPEG-4 Audio Transport 208 Multiplex (LATM) manages the sequences of audio data with relatively 209 small overhead. In audio-only applications, then, it is desirable 210 for LATM-based MPEG-4 Audio bitstreams to be directly mapped onto RTP 211 packets without using MPEG-4 Systems. 213 For MPEG-4 Audio coding tools, as is true for other audio coders, if 214 the payload is a single audio frame, packet loss will not impair the 215 decodability of adjacent packets. Therefore, the additional media 216 specific header for recovering errors will not be required for MPEG-4 217 Audio. Existing RTP protection mechanisms, such as Generic Forward 218 Error Correction [RFC5109] and Redundant Audio Data [RFC2198], MAY be 219 applied to improve error resiliency. 221 1.3. Interoperability with RFC 3016 223 This specification is not backwards compatible with [RFC3016] as a 224 binary incompatible LATM version is mandated. Existing 225 implementations of RFC 3016 that use a recent LATM version may 226 already comply to this specification and must be considered as not 227 RFC 3016 compliant. The 3GPP PSS service [3GPP] is such an example 228 as a more recent LATM version is mandated in the 3GPP PSS 229 specification. Existing implementations that use the LATM version as 230 specified in RFC3016 MUST be updated to comply with this 231 specification. 233 1.4. Relation with RFC 3640 235 In this document a payload format for the transport of MPEG-4 236 Elementary Streams is specified. For MPEG-4 Audio streams "out of 237 band" signaling is defined such that a receiver is not obliged to 238 decode the payload data to determine the audio codec and its 239 configuration. The signaling capabilities specified in this document 240 are less explicit than those defined in [RFC3640]. But, the use of 241 the MPEG-4 LATM in various transmission standards justifies its right 242 to exist, see also Section 1.2. 244 2. Definitions and Abbreviations 246 This document makes use of terms, specified in [14496-2], [14496-3], 247 and [23003-1]. In addition, the following terms are used in this 248 document and have specific meaning within the context of this 249 document. 251 Abbreviations: 253 AAC: Advanced Audio Coding 255 ASC: AudioSpecificConfig 257 HE AAC: High Efficiency AAC 259 LATM: Low-overhead MPEG-4 Audio Transport Multiplex 261 PS: Parametric Stereo 263 SBR: Spectral Band Replication 265 VOP: Video Object Plane 267 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 268 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 269 document are to be interpreted as described in [RFC2119]. 271 3. Clarifications on specifying codec configurations for MPEG-4 Audio 273 For MPEG-4 Audio [14496-3] streams the decoder output configuration 274 can differ from the core codec configuration depending of use of the 275 SBR and PS tools. 277 The core codec sampling rate is the default audio codec sampling 278 rate. When SBR is used, typically the double value of the core codec 279 sampling rate will be regarded as the definitive sampling rate (i.e., 280 the decoder's output sampling rate) 282 Note: The exception is downsampled SBR mode in which case the SBR 283 sampling rate and core codec sampling rate are identical. 285 The core codec channel configuration is the default audio codec 286 channel configuration. When PS is used, the core codec channel 287 configuration indicates one channel (i.e., mono) whereas the 288 definitive channel configuration is two channels (i.e. stereo). When 289 MPEG Surround is used, the definitive channel configuration depends 290 on the output of the MPEG Surround decoder. 292 4. LATM Restrictions for RTP Packetization of MPEG-4 Audio Bitstreams 294 While LATM has several multiplexing features as follows; 296 o Carrying configuration information with audio data, 298 o Concatenation of multiple audio frames in one audio stream, 300 o Multiplexing multiple objects (programs), 302 o Multiplexing scalable layers, 304 in RTP transmission there is no need for the last two features. 305 Therefore, these two features MUST NOT be used in applications based 306 on RTP packetization specified by this document. Since LATM has been 307 developed for only natural audio coding tools, i.e., not for 308 synthesis tools, it seems difficult to transmit Structured Audio (SA) 309 data and Text to Speech Interface (TTSI) data by LATM. Therefore, SA 310 data and TTSI data MUST NOT be transported by the RTP packetization 311 in this document. 313 For transmission of scalable streams, audio data of each layer SHOULD 314 be packetized onto different RTP streams allowing for the different 315 layers to be treated differently at the IP level, for example via 316 some means of differentiated service. On the other hand, all 317 configuration data of the scalable streams are contained in one LATM 318 configuration data "StreamMuxConfig" and every scalable layer shares 319 the StreamMuxConfig. The mapping between each layer and its 320 configuration data is achieved by LATM header information attached to 321 the audio data. In order to indicate the dependency information of 322 the scalable streams, the signaling mechanism as specified in 323 [RFC5583] SHOULD be used (see Section 6.2). 325 5. RTP Packetization of MPEG-4 Visual Bitstreams 327 This section specifies RTP packetization rules for MPEG-4 Visual 328 content. An MPEG-4 Visual bitstream is mapped directly onto RTP 329 packets without the addition of extra header fields or any removal of 330 Visual syntax elements. The Combined Configuration/Elementary stream 331 mode MUST be used so that configuration information will be carried 332 to the same RTP port as the elementary stream. (see 6.2.1 "Start 333 codes" of [14496-2]) The configuration information MAY additionally 334 be specified by some out-of-band means. If needed by systems using 335 Media Type parameters and SDP parameters, "e.g., SIP and RTSP", the 336 optional parameter "config" MUST be used to specify the configuration 337 information (see Section 7.1 and Section 7.2). 339 When the short video header mode is used, the RTP payload format for 340 H.263 SHOULD be used (the format defined in [RFC4629] is RECOMMENDED, 341 but the [RFC4628] format MAY be used for compatibility with older 342 implementations). 344 0 1 2 3 345 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 346 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 347 |V=2|P|X| CC |M| PT | sequence number | RTP 348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 349 | timestamp | Header 350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 351 | synchronization source (SSRC) identifier | 352 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 353 | contributing source (CSRC) identifiers | 354 | .... | 355 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 356 | | RTP 357 | MPEG-4 Visual stream (byte aligned) | Pay- 358 | | load 359 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 360 | :...OPTIONAL RTP padding | 361 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 363 Figure 1 - An RTP packet for MPEG-4 Visual stream 365 5.1. Use of RTP Header Fields for MPEG-4 Visual 367 Payload Type (PT): The assignment of an RTP payload type for this 368 packet format is outside the scope of this document, and will not be 369 specified here. It is expected that the RTP profile for a particular 370 class of applications will assign a payload type for this encoding, 371 or if that is not done then a payload type in the dynamic range SHALL 372 be chosen by means of an out-of-band signaling protocol (e.g., H.245, 373 SIP, etc). 375 Extension (X) bit: Defined by the RTP profile used. 377 Sequence Number: Incremented by one for each RTP data packet sent, 378 starting, for security reasons, with a random initial value. 380 Marker (M) bit: The marker bit is set to one to indicate the last RTP 381 packet (or only RTP packet) of a VOP. When multiple VOPs are carried 382 in the same RTP packet, the marker bit is set to one. 384 Timestamp: The timestamp indicates the sampling instance of the VOP 385 contained in the RTP packet. A constant offset, which is random, is 386 added for security reasons. 388 o When multiple VOPs are carried in the same RTP packet, the 389 timestamp indicates the earliest of the VOP times within the VOPs 390 carried in the RTP packet. Timestamp information of the rest of 391 the VOPs are derived from the timestamp fields in the VOP header 392 (modulo_time_base and vop_time_increment). 394 o If the RTP packet contains only configuration information and/or 395 Group_of_VideoObjectPlane() fields, the timestamp of the next VOP 396 in the coding order is used. 398 o If the RTP packet contains only visual_object_sequence_end_code 399 information, the timestamp of the immediately preceding VOP in the 400 coding order is used. 402 The resolution of the timestamp is set to its default value of 90kHz, 403 unless specified by an out-of-band means (e.g., SDP parameter or 404 Media Type parameter as defined in Section 7). 406 Other header fields are used as described in [RFC3550]. 408 5.2. Fragmentation of MPEG-4 Visual Bitstream 410 A fragmented MPEG-4 Visual bitstream is mapped directly onto the RTP 411 payload without any addition of extra header fields or any removal of 412 Visual syntax elements. The Combined Configuration/Elementary 413 streams mode is used. The following rules apply for the 414 fragmentation. 416 In the following, header means one of the following: 418 o Configuration information (Visual Object Sequence Header, Visual 419 Object Header and Video Object Layer Header) 421 o visual_object_sequence_end_code 423 o The header of the entry point function for an elementary stream 424 (Group_of_VideoObjectPlane() or the header of VideoObjectPlane(), 425 video_plane_with_short_header(), MeshObject() or FaceObject()) 427 o The video packet header (video_packet_header() excluding 428 next_resync_marker()) 430 o The header of gob_layer() 432 o See 6.2.1 "Start codes" of [14496-2] for the definition of the 433 configuration information and the entry point functions. 435 (1) Configuration information and Group_of_VideoObjectPlane() fields 436 SHALL be placed at the beginning of the RTP payload (just after the 437 RTP header) or just after the header of the syntactically upper layer 438 function. 440 (2) If one or more headers exist in the RTP payload, the RTP payload 441 SHALL begin with the header of the syntactically highest function. 442 Note: The visual_object_sequence_end_code is regarded as the lowest 443 function. 445 (3) A header SHALL NOT be split into a plurality of RTP packets. 447 (4) Different VOPs SHOULD be fragmented into different RTP packets so 448 that one RTP packet consists of the data bytes associated with a 449 unique VOP time instance (that is indicated in the timestamp field in 450 the RTP packet header), with the exception that multiple consecutive 451 VOPs MAY be carried within one RTP packet in the decoding order if 452 the size of the VOPs is small. 454 Note: When multiple VOPs are carried in one RTP payload, the 455 timestamp of the VOPs after the first one may be calculated by the 456 decoder. This operation is necessary only for RTP packets in which 457 the marker bit equals to one and the beginning of RTP payload 458 corresponds to a start code. (See timestamp and marker bit in 459 Section 5.1.) 460 (5) It is RECOMMENDED that a single video packet is sent as a single 461 RTP packet. The size of a video packet SHOULD be adjusted in such a 462 way that the resulting RTP packet is not larger than the path-MTU. 463 If the video packet is disabled by the coder configuration (by 464 setting resync_marker_disable in the VOL header to 1), or in coding 465 tools where the video packet is not supported, a VOP MAY be split at 466 arbitrary byte-positions. 468 The video packet starts with the VOP header or the video packet 469 header, followed by motion_shape_texture(), and ends with 470 next_resync_marker() or next_start_code(). 472 5.3. Examples of Packetized MPEG-4 Visual Bitstream 474 Figure 2 shows examples of RTP packets generated based on the 475 criteria described in Section 5.2 477 (a) is an example of the first RTP packet or the random access point 478 of an MPEG-4 Visual bitstream containing the configuration 479 information. According to criterion (1), the Visual Object Sequence 480 Header(VS header) is placed at the beginning of the RTP payload, 481 preceding the Visual Object Header and the Video Object Layer 482 Header(VO header, VOL header). Since the fragmentation rule defined 483 in Section 5.2 guarantees that the configuration information, 484 starting with visual_object_sequence_start_code, is always placed at 485 the beginning of the RTP payload, RTP receivers can detect the random 486 access point by checking if the first 32-bit field of the RTP payload 487 is visual_object_sequence_start_code. 489 (b) is another example of the RTP packet containing the configuration 490 information. It differs from example (a) in that the RTP packet also 491 contains a VOP header and a Video Packet in the VOP following the 492 configuration information. Since the length of the configuration 493 information is relatively short (typically scores of bytes) and an 494 RTP packet containing only the configuration information may thus 495 increase the overhead, the configuration information and the 496 immediately following VOP can be packetized into a single RTP packet. 498 (c) is an example of an RTP packet that contains 499 Group_of_VideoObjectPlane(GOV). Following criterion (1), the GOV is 500 placed at the beginning of the RTP payload. It would be a waste of 501 RTP/IP header overhead to generate an RTP packet containing only a 502 GOV whose length is 7 bytes. Therefore, (a part of) the following 503 VOP can be placed in the same RTP packet as shown in (c). 505 (d) is an example of the case where one video packet is packetized 506 into one RTP packet. When the packet-loss rate of the underlying 507 network is high, this kind of packetization is recommended. Even 508 when the RTP packet containing the VOP header is discarded by a 509 packet loss, the other RTP packets can be decoded by using the 510 HEC(Header Extension Code) information in the video packet header. 511 No extra RTP header field is necessary. 513 (e) is an example of the case where more than one video packet is 514 packetized into one RTP packet. This kind of packetization is 515 effective to save the overhead of RTP/IP headers when the bit-rate of 516 the underlying network is low. However, it will decrease the packet- 517 loss resiliency because multiple video packets are discarded by a 518 single RTP packet loss. The optimal number of video packets in an 519 RTP packet and the length of the RTP packet can be determined 520 considering the packet-loss rate and the bit-rate of the underlying 521 network. 523 (f) is an example of the case when the video packet is disabled by 524 setting resync_marker_disable in the VOL header to 1. In this case, 525 a VOP may be split into a plurality of RTP packets at arbitrary byte- 526 positions. For example, it is possible to split a VOP into fixed- 527 length packets. This kind of coder configuration and RTP packet 528 fragmentation may be used when the underlying network is guaranteed 529 to be error-free. 531 Figure 3 shows examples of RTP packets prohibited by the criteria of 532 Section 5.2. 534 Fragmentation of a header into multiple RTP packets, as in (a), will 535 not only increase the overhead of RTP/IP headers but also decrease 536 the error resiliency. Therefore, it is prohibited by the criterion 537 (3). 539 When concatenating more than one video packets into an RTP packet, 540 VOP header or video_packet_header() are not allowed to be placed in 541 the middle of the RTP payload. The packetization as in (b) is not 542 allowed by criterion (2) due to the aspect of the error resiliency. 543 Comparing this example with Figure 2(d), although two video packets 544 are mapped onto two RTP packets in both cases, the packet-loss 545 resiliency is not identical. Namely, if the second RTP packet is 546 lost, both video packets 1 and 2 are lost in the case of Figure 3(b) 547 whereas only video packet 2 is lost in the case of Figure 2(d). 549 +------+------+------+------+ 550 (a) | RTP | VS | VO | VOL | 551 |header|header|header|header| 552 +------+------+------+------+ 554 +------+------+------+------+------+------------+ 555 (b) | RTP | VS | VO | VOL | VOP |Video Packet| 556 |header|header|header|header|header| | 557 +------+------+------+------+------+------------+ 559 +------+-----+------------------+ 560 (c) | RTP | GOV |Video Object Plane| 561 |header| | | 562 +------+-----+------------------+ 564 +------+------+------------+ +------+------+------------+ 565 (d) | RTP | VOP |Video Packet| | RTP | VP |Video Packet| 566 |header|header| (1) | |header|header| (2) | 567 +------+------+------------+ +------+------+------------+ 569 +------+------+------------+------+------------+------+------------+ 570 (e) | RTP | VP |Video Packet| VP |Video Packet| VP |Video Packet| 571 |header|header| (1) |header| (2) |header| (3) | 572 +------+------+------------+------+------------+------+------------+ 574 +------+------+------------+ +------+------------+ 575 (f) | RTP | VOP |VOP fragment| | RTP |VOP fragment| 576 |header|header| (1) | |header| (2) | ___ 577 +------+------+------------+ +------+------------+ 579 Figure 2 - Examples of RTP packetized MPEG-4 Visual bitstream 581 +------+-------------+ +------+------------+------------+ 582 (a) | RTP |First half of| | RTP |Last half of|Video Packet| 583 |header| VP header | |header| VP header | | 584 +------+-------------+ +------+------------+------------+ 586 +------+------+----------+ +------+---------+------+------------+ 587 (b) | RTP | VOP |First half| | RTP |Last half| VP |Video Packet| 588 |header|header| of VP(1) | |header| of VP(1)|header| (2) | 589 +------+------+----------+ +------+---------+------+------------+ 591 Figure 3 - Examples of prohibited RTP packetization for MPEG-4 Visual 592 bitstream 594 6. RTP Packetization of MPEG-4 Audio Bitstreams 596 This section specifies RTP packetization rules for MPEG-4 Audio 597 bitstreams. MPEG-4 Audio streams MUST be formatted LATM (Low- 598 overhead MPEG-4 Audio Transport Multiplex) [14496-3] streams, and the 599 LATM-based streams are then mapped onto RTP packets as described in 600 the sections below. 602 6.1. RTP Packet Format 604 LATM-based streams consist of a sequence of audioMuxElements that 605 include one or more PayloadMux elements which carry the audio frames. 606 A complete audioMuxElement or a part of one SHALL be mapped directly 607 onto an RTP payload without any removal of audioMuxElement syntax 608 elements (see Figure 4). The first byte of each audioMuxElement 609 SHALL be located at the first payload location in an RTP packet. 611 0 1 2 3 612 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 613 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 614 |V=2|P|X| CC |M| PT | sequence number |RTP 615 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 616 | timestamp |Header 617 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 618 | synchronization source (SSRC) identifier | 619 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 620 | contributing source (CSRC) identifiers | 621 | .... | 622 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 623 | |RTP 624 : audioMuxElement (byte aligned) :Payload 625 | | 626 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 627 | :...OPTIONAL RTP padding | 628 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 630 Figure 4 - An RTP packet for MPEG-4 Audio 632 In order to decode the audioMuxElement, the following 633 muxConfigPresent information is required to be indicated by out-of- 634 band means. When SDP is utilized for this indication, the Media Type 635 parameter "cpresent" corresponds to the muxConfigPresent information 636 (see Section 7.3). The following restrictions apply: 638 o In the out-of-band configuration case the number of PayloadMux 639 elements contained in each audioMuxElement can only be set once. 640 If more than one PayloadMux element is contained in each 641 audioMuxElement, special care is required to ensure that the last 642 RTP packet remains decodable. 644 o To construct the audioMuxElement in the in-band configuration 645 case, non octet aligned configuration data is inserted immediately 646 before the one or more PayloadMux elements. Since the generation 647 of RTP payloads with non octet aligned data is not possible with 648 RTP hint tracks, as defined by the MP4 file format [14496-12] 649 [14496-14], this document does not support RTP hint tracks for the 650 in-band configuration case. 652 muxConfigPresent: If this value is set to 1 (in-band mode), the 653 audioMuxElement SHALL include an indication bit "useSameStreamMux" 654 and MAY include the configuration information for audio compression 655 "StreamMuxConfig". The useSameStreamMux bit indicates whether the 656 StreamMuxConfig element in the previous frame is applied in the 657 current frame. If the useSameStreamMux bit indicates to use the 658 StreamMuxConfig from the previous frame, but if the previous frame 659 has been lost, the current frame may not be decodable. Therefore, in 660 case of in-band mode, the StreamMuxConfig element SHOULD be 661 transmitted repeatedly depending on the network condition. On the 662 other hand, if muxConfigPresent is set to 0 (out-band mode), the 663 StreamMuxConfig element is required to be transmitted by an out-of- 664 band means. In case of SDP, Media Type parameter "config" is 665 utilized (see Section 7.3). 667 6.2. Use of RTP Header Fields for MPEG-4 Audio 669 Payload Type (PT): The assignment of an RTP payload type for this 670 packet format is outside the scope of this document, and will only be 671 restricted here. It is expected that the RTP profile for a 672 particular class of applications will assign a payload type for this 673 encoding, or if that is not done then a payload type in the dynamic 674 range shall be chosen by means of an out-of-band signaling protocol 675 (e.g., H.245, SIP, etc). In the dynamic assignment of RTP payload 676 types for scalable streams, the server SHALL assign a different value 677 to each layer. The dependency relationships between the enhanced 678 layer and the base layer MUST be signaled as specified in [RFC5583]. 679 An example of the use of such signaling for scalable audio streams 680 can be found in [RFC5691]. 682 Marker (M) bit: The marker bit indicates audioMuxElement boundaries. 683 It is set to one to indicate that the RTP packet contains a complete 684 audioMuxElement or the last fragment of an audioMuxElement. 686 Timestamp: The timestamp indicates the sampling instance of the first 687 audio frame contained in the RTP packet. Timestamps are RECOMMENDED 688 to start at a random value for security reasons. 690 Unless specified by an out-of-band means, the resolution of the 691 timestamp is set to its default value of 90 kHz. 693 Sequence Number: Incremented by one for each RTP packet sent, 694 starting, for security reasons, with a random value. 696 Other header fields are used as described in [RFC3550]. 698 6.3. Fragmentation of MPEG-4 Audio Bitstream 700 It is RECOMMENDED to put one audioMuxElement in each RTP packet. If 701 the size of an audioMuxElement can be kept small enough that the size 702 of the RTP packet containing it does not exceed the size of the path- 703 MTU, this will be no problem. If it cannot, the audioMuxElement 704 SHALL be fragmented and spread across multiple packets. 706 7. Media Type Registration for MPEG-4 Audio/Visual Streams 708 The following sections describe the Media Type registrations for 709 MPEG-4 Audio/Visual streams, which are registered in accordance with 710 [RFC4855] and uses the template of [RFC4288]. Media Type 711 registration and SDP usage for the MPEG-4 Visual stream are described 712 in Section 7.1 and Section 7.2, respectively, while Media Type 713 registration and SDP usage for MPEG-4 Audio stream are described in 714 Section 7.3 and Section 7.4, respectively. 716 7.1. Media Type Registration for MPEG-4 Visual 718 The receiver MUST ignore any unspecified parameter, to ensure that 719 additional parameters can be added in any future revision of this 720 specification. 722 Type name: video 724 Subtype name: MP4V-ES 726 Required parameters: none 728 Optional parameters: 730 rate: This parameter is used only for RTP transport. It indicates 731 the resolution of the timestamp field in the RTP header. If this 732 parameter is not specified, its default value of 90000 (90kHz) is 733 used. 735 profile-level-id: A decimal representation of MPEG-4 Visual 736 Profile and Level indication value (profile_and_level_indication) 737 defined in Table G-1 of [14496-2]. This parameter MAY be used in 738 the capability exchange or session setup procedure to indicate 739 MPEG-4 Visual Profile and Level combination of which the MPEG-4 740 Visual codec is capable. If this parameter is not specified by 741 the procedure, its default value of 1 (Simple Profile/Level 1) is 742 used. 744 config: This parameter SHALL be used to indicate the configuration 745 of the corresponding MPEG-4 Visual bitstream. It SHALL NOT be 746 used to indicate the codec capability in the capability exchange 747 procedure. It is a hexadecimal representation of an octet string 748 that expresses the MPEG-4 Visual configuration information, as 749 defined in subclause 6.2.1 Start codes of [14496-2]. The 750 configuration information is mapped onto the octet string in an 751 MSB-first basis. The first bit of the configuration information 752 SHALL be located at the MSB of the first octet. The configuration 753 information indicated by this parameter SHALL be the same as the 754 configuration information in the corresponding MPEG-4 Visual 755 stream, except for first_half_vbv_occupancy and 756 latter_half_vbv_occupancy, if exist, which may vary in the 757 repeated configuration information inside an MPEG-4 Visual stream 758 (See 6.2.1 Start codes of [14496-2]). 760 Published specification: 762 The specifications for MPEG-4 Visual streams are presented in 763 [14496-2]. The RTP payload format is described in [RFCxxxx]. 765 Encoding considerations: 767 Video bitstreams MUST be generated according to MPEG-4 Visual 768 specifications [14496-2]. A video bitstream is binary data and 769 MUST be encoded for non-binary transport (for Email, the Base64 770 encoding is sufficient). This type is also defined for transfer 771 via RTP. The RTP packets MUST be packetized according to the 772 MPEG-4 Visual RTP payload format defined in [RFCxxxx]. 774 Security considerations: 776 See Section 10 of [RFCxxxx]. 778 Interoperability considerations: 780 MPEG-4 Visual provides a large and rich set of tools for the 781 coding of visual objects. For effective implementation of the 782 standard, subsets of the MPEG-4 Visual tool sets have been 783 provided for use in specific applications. These subsets, called 784 'Profiles', limit the size of the tool set a decoder is required 785 to implement. In order to restrict computational complexity, one 786 or more Levels are set for each Profile. A Profile@Level 787 combination allows: 789 * a codec builder to implement only the subset of the standard he 790 needs, while maintaining interworking with other MPEG-4 devices 791 included in the same combination, and 793 * checking whether MPEG-4 devices comply with the standard 794 ('conformance testing'). 796 The visual stream SHALL be compliant with the MPEG-4 Visual 797 Profile@Level specified by the parameter "profile-level-id". 798 Interoperability between a sender and a receiver may be achieved 799 by specifying the parameter "profile-level-id", or by arranging a 800 capability exchange/announcement procedure for this parameter. 802 Applications which use this Media Type: 804 Audio and visual streaming and conferencing tools 806 Additional information: none 808 Person and email address to contact for further information: 810 See Authors' Address section at the end of [RFCxxxx]. 812 Intended usage: COMMON 814 Author: 816 See Authors' Address section at the end of [RFCxxxx]. 818 Change controller: 820 IETF Audio/Video Transport Payloads working group delegated from 821 the IESG. 823 7.2. Mapping to SDP for MPEG-4 Visual 825 The Media Type video/MP4V-ES string is mapped to fields in the 826 Session Description Protocol (SDP) [RFC4566], as follows: 828 o The Media Type (video) goes in SDP "m=" as the media name. 830 o The Media subtype (MP4V-ES) goes in SDP "a=rtpmap" as the encoding 831 name. 833 o The optional parameter "rate" goes in "a=rtpmap" as the clock 834 rate. 836 o The optional parameter "profile-level-id" and "config" go in the 837 "a=fmtp" line to indicate the coder capability and configuration, 838 respectively. These parameters are expressed as a string, in the 839 form of as a semicolon separated list of parameter=value pairs. 841 Example usages for the profile-level-id parameter are: 842 1 : MPEG-4 Visual Simple Profile/Level 1 843 34 : MPEG-4 Visual Core Profile/Level 2 844 145: MPEG-4 Visual Advanced Real Time Simple Profile/Level 1 846 7.2.1. Declarative SDP Usage for MPEG-4 Visual 848 The following are some examples of media representation in SDP: 850 Simple Profile/Level 1, rate=90000(90kHz), "profile-level-id" and 851 "config" are present in "a=fmtp" line: 852 m=video 49170/2 RTP/AVP 98 853 a=rtpmap:98 MP4V-ES/90000 854 a=fmtp:98 profile-level-id=1;config=000001B001000001B50900000100000001 855 20008440FA282C2090A21F 857 Core Profile/Level 2, rate=90000(90kHz), "profile-level-id" is present 858 in "a=fmtp" line: 859 m=video 49170/2 RTP/AVP 98 860 a=rtpmap:98 MP4V-ES/90000 861 a=fmtp:98 profile-level-id=34 863 Advance Real Time Simple Profile/Level 1, rate=90000(90kHz), 864 "profile-level-id" is present in "a=fmtp" line: 865 m=video 49170/2 RTP/AVP 98 866 a=rtpmap:98 MP4V-ES/90000 867 a=fmtp:98 profile-level-id=145 869 7.3. Media Type Registration for MPEG-4 Audio 871 The receiver MUST ignore any unspecified parameter, to ensure that 872 additional parameters can be added in any future revision of this 873 specification. 875 Type name: audio 877 Subtype name: MP4A-LATM 879 Required parameters: 881 rate: the rate parameter indicates the RTP time stamp clock rate. 882 The default value is 90000. Other rates MAY be indicated only if 883 they are set to the same value as the audio sampling rate (number 884 of samples per second). 886 In the presence of SBR, the sampling rates for the core en-/ 887 decoder and the SBR tool are different in most cases. This 888 parameter SHALL therefore NOT be considered as the definitive 889 sampling rate. If this parameter is used, the server must follow 890 the rules below: 892 * When the presence of SBR is not explicitly signaled by the 893 optional SDP parameters such as object parameter, profile- 894 level-id or config string, this parameter SHALL be set to the 895 core codec sampling rate. 897 * When the presence of SBR is explicitly signaled by the optional 898 SDP parameters such as object parameter, profile-level-id or 899 config string this parameter SHALL be set to the SBR sampling 900 rate. 902 NOTE: The optional parameter SBR-enabled in SDP a=fmtp is useful 903 for implicit HE AAC / HE AAC v2 signaling. But the SBR-enabled 904 parameter can also be used in the case of explicit HE AAC / HE AAC 905 v2 signaling. Therefore, its existence itself is not the criteria 906 to determine whether HE AAC / HE AAC v2 signaling is explicit or 907 not. 909 Optional parameters: 911 profile-level-id: a decimal representation of MPEG-4 Audio Profile 912 Level indication value defined in [14496-3]. This parameter 913 indicates which MPEG-4 Audio tool subsets the decoder is capable 914 of using. If this parameter is not specified in the capability 915 exchange or session setup procedure, its default value of 30 916 (Natural Audio Profile/Level 1) is used. 918 MPS-profile-level-id: a decimal representation of the MPEG 919 Surround Profile Level indication as defined in [14496-3]. This 920 parameter indicates the support of the MPEG Surround profile and 921 level by the decoder to be capable to decode the stream. 923 object: a decimal representation of the MPEG-4 Audio Object Type 924 value defined in [14496-3]. This parameter specifies the tool to 925 be used by the decoder. It CAN be used to limit the capability 926 within the specified "profile-level-id". 928 bitrate: the data rate for the audio bit stream. 930 cpresent: a boolean parameter indicates whether audio payload 931 configuration data has been multiplexed into an RTP payload (see 932 Section 6.1). A 0 indicates the configuration data has not been 933 multiplexed into an RTP payload and in this case the "config" 934 parameter MUST be present, a 1 indicates that it has. The default 935 if the parameter is omitted is 1. If this parameter is set to 1 936 and the "config" parameter is present, the multiplexed 937 configuration data and the value of the "config" parameter SHALL 938 be consistent. 940 config: a hexadecimal representation of an octet string that 941 expresses the audio payload configuration data "StreamMuxConfig", 942 as defined in [14496-3]. Configuration data is mapped onto the 943 octet string in an MSB-first basis. The first bit of the 944 configuration data SHALL be located at the MSB of the first octet. 945 In the last octet, zero-padding bits, if necessary, SHALL follow 946 the configuration data. Senders MUST set the StreamMuxConfig 947 elements taraBufferFullness and latmBufferFullness to their 948 largest respective value, indicating that buffer fullness measures 949 are not used in SDP. Receivers MUST ignore the value of these two 950 elements contained in the config parameter. 952 MPS-asc: a hexadecimal representation of an octet string that 953 expresses audio payload configuration data "AudioSpecificConfig", 954 as defined in [14496-3]. If this parameter is not present the 955 relevant signaling is performed by other means (e.g. in-band or 956 contained in the config string). 958 The same mapping rules as for the config parameter apply. 960 ptime: duration of each packet in milliseconds. 962 SBR-enabled: a boolean parameter which indicates whether SBR-data 963 can be expected in the RTP-payload of a stream. This parameter is 964 relevant for an SBR-capable decoder if the presence of SBR can not 965 be detected from an out-of-band decoder configuration (e.g. 966 contained in the config string). 968 If this parameter is set to 0, a decoder MAY expect that SBR is 969 not used. If this parameter is set to 1, a decoder CAN upsample 970 the audio data with the SBR tool, regardless whether SBR data is 971 present in the stream or not. 973 If the presence of SBR can not be detected from out-of-band 974 configuration and the SBR-enabled parameter is not present, the 975 parameter defaults to 1 for an SBR-capable decoder. If the 976 resulting output sampling rate or the computational complexity is 977 not supported, the SBR tool can be disabled or run in downsampled 978 mode. 980 The timestamp resolution at RTP layer is determined by the rate 981 parameter. 983 Published specification: 985 Encoding specifications are provided in [14496-3]. The RTP 986 payload format specification is described in [RFCxxxx]. 988 Encoding considerations: 990 This type is only defined for transfer via RTP. 992 Security considerations: 994 See Section 10 of [RFCxxxx]. 996 Interoperability considerations: 998 MPEG-4 Audio provides a large and rich set of tools for the coding 999 of audio objects. For effective implementation of the standard, 1000 subsets of the MPEG-4 Audio tool sets similar to those used in 1001 MPEG-4 Visual have been provided (see Section 7.1). 1003 The audio stream SHALL be compliant with the MPEG-4 Audio Profile@ 1004 Level specified by the parameters "profile-level-id" and "MPS- 1005 profile-level-id". Interoperability between a sender and a 1006 receiver may be achieved by specifying the parameters "profile- 1007 level-id" and "MPS-profile-level-id", or by arranging in the 1008 capability exchange procedure to set this parameter mutually to 1009 the same value. Furthermore, the "object" parameter can be used 1010 to limit the capability within the specified Profile@Level in 1011 capability exchange. 1013 Applications which use this media type: 1015 Audio and video streaming and conferencing tools. 1017 Additional information: none 1019 Personal and email address to contact for further information: 1021 See Authors' Address section at the end of [RFCxxxx]. 1023 Intended usage: COMMON 1024 Author: 1026 See Authors' Address section at the end of [RFCxxxx]. 1028 Change controller: 1030 IETF Audio/Video Transport Payloads working group delegated from 1031 the IESG. 1033 7.4. Mapping to SDP for MPEG-4 Audio 1035 The Media Type audio/MP4A-LATM string is mapped to fields in the 1036 Session Description Protocol (SDP) [RFC4566], as follows: 1038 o The Media Type (audio) goes in SDP "m=" as the media name. 1040 o The Media subtype (MP4A-LATM) goes in SDP "a=rtpmap" as the 1041 encoding name. 1043 o The required parameter "rate" goes in "a=rtpmap" as the clock 1044 rate. 1046 o The optional parameter "ptime" goes in SDP "a=ptime" attribute. 1048 o The optional parameters "profile-level-id", "MPS-profile-level-id" 1049 and "object" goes in the "a=fmtp" line to indicate the coder 1050 capability. 1052 The following are some examples of the profile-level-id value: 1053 1 : Main Audio Profile Level 1 1054 9 : Speech Audio Profile Level 1 1055 15: High Quality Audio Profile Level 2 1056 30: Natural Audio Profile Level 1 1057 44: High Efficiency AAC Profile Level 2 1058 48: High Efficiency AAC v2 Profile Level 2 1059 55: Baseline MPEG Surround Profile (see ISO/IEC 23003-1) Level 3 1061 The optional payload-format-specific parameters "bitrate", 1062 "cpresent", "config", "MPS-asc" and "SBR-enabled" go also in the 1063 "a=fmtp" line. These parameters are expressed as a string, in the 1064 form of as a semicolon separated list of parameter=value pairs. 1066 7.4.1. Declarative SDP Usage for MPEG-4 Audio 1068 The following sections contain some examples of the media 1069 representation in SDP. 1071 Note that the a=fmtp line in some of the examples has been wrapped to 1072 fit the page; they would comprise a single line in the SDP file. 1074 7.4.1.1. Example: In-band Configuration 1076 In this example the audio configuration data appears in the RTP 1077 payload exclusively (i.e., the MPEG-4 audio configuration is known 1078 when a StreamMuxConfig element appears within the RTP payload). 1080 m=audio 49230 RTP/AVP 96 1081 a=rtpmap:96 MP4A-LATM/90000 1082 a=fmtp:96 object=2; cpresent=1 1084 The "clock rate" is set to 90kHz. This is the default value and the 1085 real audio sampling rate is known when the audio configuration data 1086 is received. 1088 7.4.1.2. Example: 6kb/s CELP 1090 6 kb/s CELP bitstreams (with an audio sampling rate of 8 kHz) 1092 m=audio 49230 RTP/AVP 96 1093 a=rtpmap:96 MP4A-LATM/8000 1094 a=fmtp:96 profile-level-id=9; object=8; cpresent=0; 1095 config=40008B18388380 1096 a=ptime:20 1098 In this example audio configuration data is not multiplexed into the 1099 RTP payload and is described only in SDP. Furthermore, the "clock 1100 rate" is set to the audio sampling rate. 1102 7.4.1.3. Example: 64 kb/s AAC LC Stereo 1104 64 kb/s AAC LC stereo bitstream (with an audio sampling rate of 24 1105 kHz) 1107 m=audio 49230 RTP/AVP 96 1108 a=rtpmap:96 MP4A-LATM/24000/2 1109 a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0; 1110 object=2; config=400026203fc0 1112 In this example audio configuration data is not multiplexed into the 1113 RTP payload and is described only in SDP. Furthermore, the "clock 1114 rate" is set to the audio sampling rate. 1116 In this example, the presence of SBR can not be determined by the SDP 1117 parameter set. The clock rate represents the core codec sampling 1118 rate. An SBR enabled decoder can use the SBR tool to upsample the 1119 audio data if complexity and resulting output sampling rate permits. 1121 7.4.1.4. Example: Use of the SBR-enabled Parameter 1123 These two examples are identical to the example above with the 1124 exception of the SBR-enabled parameter. The presence of SBR is not 1125 signaled by the SDP parameters object, profile-level-id and config, 1126 but instead the SBR-enabled parameter is present. The rate parameter 1127 and the StreamMuxConfig contain the core codec sampling rate. 1129 Example with "SBR-enabled=0", definitive and core codec sampling rate 1130 24kHz: 1132 m=audio 49230 RTP/AVP 96 1133 a=rtpmap:96 MP4A-LATM/24000/2 1134 a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0; 1135 SBR-enabled=0; config=400026203fc0 1137 Example with "SBR-enabled=1", core codec sampling rate 24kHz, 1138 definitive and SBR sampling rate 48kHz: 1140 m=audio 49230 RTP/AVP 96 1141 a=rtpmap:96 MP4A-LATM/24000/2 1142 a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0; 1143 SBR-enabled=1; config=400026203fc0 1145 In this example, the clock rate is still 24000 and this information 1146 is used for RTP timestamp calculation. The value of 24000 is used to 1147 support old AAC decoders. This makes the decoder supporting only AAC 1148 understand the HE AAC coded data, although only plain AAC is 1149 supported. A HE AAC decoder is able to generate output data with the 1150 SBR sampling rate. 1152 7.4.1.5. Example: Hierarchical Signaling of SBR 1154 When the presence of SBR is explicitly signaled by the SDP parameters 1155 object, profile-level-id or the config string as in the example 1156 below, the StreamMuxConfig contains both the core codec sampling rate 1157 and the SBR sampling rate. 1159 m=audio 49230 RTP/AVP 96 1160 a=rtpmap:96 MP4A-LATM/48000/2 1161 a=fmtp:96 profile-level-id=44; bitrate=64000; cpresent=0; 1162 config=40005623101fe0; SBR-enabled=1 1164 This config string uses the explicit signaling mode 2.A (hierarchical 1165 signaling; See [14496-3]. This means that the AOT(Audio Object Type) 1166 is SBR(5) and SFI(Sampling Frequency Index) is 6(24000 Hz) which 1167 refers to the underlying core codec sampling frequency. CC(Channel 1168 Configuration) is stereo(2), and the ESFI(Extension Sampling 1169 Frequency Index)=3 (48000) is referring to the sampling frequency of 1170 the extension tool(SBR). 1172 7.4.1.6. Example: HE AAC v2 Signaling 1174 HE AAC v2 decoders are required to always produce a stereo signal 1175 from a mono signal. Hence, there is no parameter necessary to signal 1176 the presence of PS. 1178 Example with "SBR-enabled=1" and 1 channel signaled in the a=rtpmap 1179 line and within the config parameter. Core codec sampling rate is 1180 24kHz, definitive and SBR sampling rate is 48kHz. Core codec channel 1181 configuration is mono, PS channel configuration is stereo. 1183 m=audio 49230 RTP/AVP 110 1184 a=rtpmap:110 MP4A-LATM/24000/1 1185 a=fmtp:110 profile-level-id=15; object=2; cpresent=0; 1186 config=400026103fc0; SBR-enabled=1 1188 7.4.1.7. Example: Hierarchical Signaling of PS 1190 Example: 48khz stereo audio input: 1192 m=audio 49230 RTP/AVP 110 1193 a=rtpmap:110 MP4A-LATM/48000/2 1194 a=fmtp:110 profile-level-id=48; cpresent=0; config=4001d613101fe0 1196 The config parameter indicates explicit hierarchical signaling of PS 1197 and SBR. This configuration method is not supported by legacy AAC an 1198 HE AAC decoders and these are therefore unable to decode the the 1199 coded data. 1201 7.4.1.8. Example: MPEG Surround 1203 The following examples show how MPEG Surround configuration data can 1204 be signaled using SDP. The configuration is carried within the 1205 config string in the first example by using two different layers. 1206 The general parameters in this example are: AudioMuxVersion=1; 1207 allStreamsSameTimeFraming=1; numSubFrames=0; numProgram=0; 1208 numLayer=1. The first layer describes the HE AAC payload and signals 1209 the following parameters: ascLen=25; audioObjectType=2 (AAC LC); 1210 extensionAudioObjectType=5 (SBR); samplingFrequencyIndex=6 (24kHz); 1211 extensionSamplingFrequencyIndex=3 (48kHz); channelConfiguration=2 1212 (2.0 channels). The second layer describes the MPEG surround payload 1213 and specifies the following parameters: ascLen=110; 1214 AudioObjectType=30 (MPEG Surround); samplingFrequencyIndex=3 (48kHz); 1215 channelConfiguration=6 (5.1 channels); sacPayloadEmbedding=1; 1216 SpatialSpecificConfig=(48 kHz; 32 slots; 525 tree; ResCoding=1; 1217 ResBands=[7,7,7,7]). 1219 In this example the signaling is carried by using two different LATM 1220 layers. The MPEG surround payload is carried together with the AAC 1221 payload in a single layer as indicated by the sacPayloadEmbedding 1222 Flag. 1224 m=audio 49230 RTP/AVP 96 1225 a=rtpmap:96 MP4A-LATM/48000 1226 a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0; 1227 SBR-enabled=1; 1228 config=8FF8004192B11880FF0DDE3699F2408C00536C02313CF3CE0FF0 1230 7.4.1.9. Example: MPEG Surround with Extended SDP Parameters 1232 The following example is an extension of the configuration given 1233 above by the MPEG Surround specific parameters. The MPS-asc 1234 parameter specifies the MPEG Surround Baseline Profile at Level 3 1235 (PLI55) and the MPS-asc string contains the hexadecimal 1236 representation of the MPEG Surround ASC [audioObjectType=30 (MPEG 1237 Surround); samplingFrequencyIndex=0x3 (48kHz); channelConfiguration=6 1238 (5.1 channels); sacPayloadEmbedding=1; SpatialSpecificConfig=(48 kHz; 1239 32 slots; 525 tree; ResCoding=1; ResBands=[0,13,13,13])]. 1241 m=audio 49230 RTP/AVP 96 1242 a=rtpmap:96 MP4A-LATM/48000 1243 a=fmtp:96 profile-level-id=44; bitrate=64000; cpresent=0; 1244 config=40005623101fe0; MPS-profile-level-id=55; 1245 MPS-asc=F1B4CF920442029B501185B6DA00; 1247 7.4.1.10. Example: MPEG Surround with Single Layer Configuration 1249 The following example shows how MPEG Surround configuration data can 1250 be signaled using the SDP config parameter. The configuration is 1251 carried within the config string using a single layer. The general 1252 parameters in this example are: AudioMuxVersion=1; 1253 allStreamsSameTimeFraming=1; numSubFrames=0; numProgram=0; 1254 numLayer=0. The single layer describes the combination of HE AAC and 1255 MPEG Surround payload and signals the following parameters: 1256 ascLen=101; audioObjectType=2 (AAC LC); extensionAudioObjectType=5 1257 (SBR); samplingFrequencyIndex=7 (22.05kHz); 1258 extensionSamplingFrequencyIndex=7 (44.1kHz); channelConfiguration=2 1259 (2.0 channels). A backward compatible extension according to 1260 [14496-3/Amd.1] signals the presence of MPEG surround payload data 1261 and specifies the following parameters: SpatialSpecificConfig=(44.1 1262 kHz; 32 slots; 525 tree; ResCoding=0). 1264 In this example the signaling is carried by using a single LATM 1265 layer. The MPEG surround payload is carried together with the HE AAC 1266 payload in a single layer. 1268 m=audio 49230 RTP/AVP 96 1269 a=rtpmap:96 MP4A-LATM/44100 1270 a=fmtp:96 profile-level-id=44; bitrate=64000; cpresent=0; 1271 SBR-enabled=1; config=8FF8000652B920876A83A1F440884053620FF0; 1272 MPS-profile-level-id=55 1274 8. IANA Considerations 1276 This document updates the media subtypes "MP4A-LATM" and "MP4V-ES" 1277 from RFC 3016. The new registrations are in Section 7.1 and 1278 Section 7.3 of this document. 1280 9. Acknowledgements 1282 The authors would like to thank Yoshihiro Kikuchi, Yoshinori Matsui, 1283 Toshiyuki Nomura, Shigeru Fukunaga and Hideaki Kimata for their work 1284 on RFC 3016, and Ali Begen, Keith Drage, Roni Even and Qin Wu for 1285 their valuable input and comments on this document. 1287 10. Security Considerations 1289 RTP packets using the payload format defined in this specification 1290 are subject to the security considerations discussed in the RTP 1291 specification [RFC3550], and in any applicable RTP profile. The main 1292 security considerations for the RTP packet carrying the RTP payload 1293 format defined within this document are confidentiality, integrity, 1294 and source authenticity. Confidentiality is achieved by encryption 1295 of the RTP payload, and integrity of the RTP packets through a 1296 suitable cryptographic integrity protection mechanism. A 1297 cryptographic system may also allow the authentication of the source 1298 of the payload. A suitable security mechanism for this RTP payload 1299 format should provide confidentiality, integrity protection, and at 1300 least source authentication capable of determining whether or not an 1301 RTP packet is from a member of the RTP session. 1303 Note that most MPEG-4 codecs define an extension mechanism to 1304 transmit extra data within a stream that is gracefully skipped by 1305 decoders that do not support this extra data. This covert channel 1306 may be used to transmit unwanted data in an otherwise valid stream. 1308 The appropriate mechanism to provide security to RTP and payloads 1309 following this may vary. It is dependent on the application, the 1310 transport, and the signaling protocol employed. Therefore, a single 1311 mechanism is not sufficient, although if suitable, the usage of the 1312 Secure Real-time Transport Protocol (SRTP) [RFC3711] is recommended. 1313 Other mechanisms that may be used are IPsec [RFC4301] and Transport 1314 Layer Security (TLS) [RFC5246] (e.g., for RTP over TCP), but other 1315 alternatives may also exist. 1317 This RTP payload format and its media decoder do not exhibit any 1318 significant non-uniformity in the receiver-side computational 1319 complexity for packet processing, and thus are unlikely to pose a 1320 denial-of-service threat due to the receipt of pathological data. 1321 The complete MPEG-4 system allows for transport of a wide range of 1322 content, including Java applets (MPEG-J) and scripts. Since this 1323 payload format is restricted to audio and video streams, it is not 1324 possible to transport such active content in this format. 1326 11. Differences to RFC 3016 1328 The RTP payload format for MPEG-4 Audio as specified in RFC 3016 is 1329 used by the 3GPP PSS service [3GPP]. However, there are some 1330 misalignments between RFC 3016 and the 3GPP PSS specification that 1331 are addressed by this update: 1333 o The audio payload format (LATM) referenced in this document is the 1334 newer format specified in [14496-3], which is binary compatible to 1335 the format used in [3GPP]. This newer format is not binary 1336 compatible with the LATM referenced in RFC 3016, which is 1337 specified in [14496-3:1999/Amd.1:2000]. 1339 o The audio signaling format (StreamMuxConfig) referenced in this 1340 document is binary compatible to the format used in [3GPP]. The 1341 StreamMuxConfig element has also been revised by MPEG since RFC 1342 3016. 1344 o The use of an audio parameter "SBR-enabled" is now defined in this 1345 document, which is used by 3GPP implementations [3GPP]. RFC 3016 1346 does not define this parameter. 1348 o The rate parameter is defined unambiguously in this document for 1349 the case of presence of SBR (Spectral Band Replication). In RFC 1350 3016 the definition of the rate parameter is ambiguous. 1352 o The number of audio channels parameter is defined unambiguously in 1353 this document for the case of presence of PS (Parametric Stereo). 1354 In RFC 3016 PS is not defined yet. 1356 Furthermore some comments have been addressed and signaling support 1357 for MPEG surround [23003-1] was added. 1359 Below a summary of the changes in requirements by this update: 1361 o In the dynamic assignment of RTP payload types for scalable MPEG-4 1362 Audio streams, the server SHALL assign a different value to each 1363 layer. 1365 o The dependency relationships between the enhanced layer and the 1366 base layer for scalable MPEG-4 Audio streams MUST be signaled as 1367 specified in [RFC5583]. 1369 o If the size of an audioMuxElement is so large that the size of the 1370 RTP packet containing it does exceed the size of the path-MTU, the 1371 audioMuxElement SHALL be fragmented and spread across multiple 1372 packets. 1374 o The receiver MUST ignore any unspecified parameter, to ensure that 1375 additional parameters can be added in any future revision of this 1376 specification. 1378 12. References 1380 12.1. Normative References 1382 [14496-2] MPEG, "ISO/IEC International Standard 14496-2 - Coding of 1383 audio-visual objects, Part 2: Visual", 2003. 1385 [14496-3] MPEG, "ISO/IEC International Standard 14496-3 - Coding of 1386 audio-visual objects, Part 3 Audio", 2009. 1388 [14496-3/Amd.1] 1389 MPEG, "ISO/IEC International Standard 14496-3 - Coding of 1390 audio-visual objects, Part 3: Audio, Amendment 1: HD-AAC 1391 profile and MPEG Surround signaling", 2009. 1393 [23003-1] MPEG, "ISO/IEC International Standard 23003-1 - MPEG 1394 Surround (MPEG D)", 2007. 1396 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1397 Requirement Levels", BCP 14, RFC 2119, March 1997. 1399 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1400 Jacobson, "RTP: A Transport Protocol for Real-Time 1401 Applications", STD 64, RFC 3550, July 2003. 1403 [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and 1404 Registration Procedures", BCP 13, RFC 4288, December 2005. 1406 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 1407 Description Protocol", RFC 4566, July 2006. 1409 [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. 1410 Even, "RTP Payload Format for ITU-T Rec", RFC 4629, 1411 January 2007. 1413 [RFC4855] Casner, S., "Media Type Registration of RTP Payload 1414 Formats", RFC 4855, February 2007. 1416 [RFC5583] Schierl, T. and S. Wenger, "Signaling Media Decoding 1417 Dependency in the Session Description Protocol (SDP)", 1418 RFC 5583, July 2009. 1420 12.2. Informative References 1422 [14496-1] MPEG, "ISO/IEC International Standard 14496-1 - Coding of 1423 audio-visual objects, Part 1 Systems", 2004. 1425 [14496-12] 1426 MPEG, "ISO/IEC International Standard 14496-12 - Coding of 1427 audio-visual objects, Part 12 ISO base media file format". 1429 [14496-14] 1430 MPEG, "ISO/IEC International Standard 14496-14 - Coding of 1431 audio-visual objects, Part 12 MP4 file format". 1433 [14496-3:1999/Amd.1:2000] 1434 MPEG, "ISO/IEC International Standard 14496-3 - Coding of 1435 audio-visual objects, Part 3 Audio, Amendment 1: Audio 1436 extensions", 2000. 1438 [3GPP] 3GPP, "3rd Generation Partnership Project; Technical 1439 Specification Group Services and System Aspects; 1440 Transparent end-to-end Packet-switched Streaming Service 1441 (PSS); Protocols and codecs (Release 9)", 3GPP TS 26.234 1442 V9.5.0, December 2010. 1444 [H245] ITU, "International Telecommunications Union, "CONTROL 1445 PROTOCOL FOR MULTIMEDIA COMMUNICATION", ITU Recommendation 1446 H.245, December 2009", 2009. 1448 [H261] ITU, "International Telecommunications Union, "Video codec 1449 for audiovisual services at p x 64 kbit/s", ITU 1450 Recommendation H.261, March 1993", 1993. 1452 [H323] ITU, "International Telecommunications Union, "Packet- 1453 based multimedia communications systems", ITU 1454 Recommendation H.323, December 2009", 2009. 1456 [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., 1457 Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- 1458 Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, 1459 September 1997. 1461 [RFC3016] Kikuchi, Y., Nomura, T., Fukunaga, S., Matsui, Y., and H. 1462 Kimata, "RTP Payload Format for MPEG-4 Audio/Visual 1463 Streams", RFC 3016, November 2000. 1465 [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, 1466 A., Peterson, J., Sparks, R., Handley, M., and E. 1467 Schooler, "SIP: Session Initiation Protocol", RFC 3261, 1468 June 2002. 1470 [RFC3640] van der Meer, J., Mackie, D., Swaminathan, V., Singer, D., 1471 and P. Gentric, "RTP Payload Format for Transport of 1472 MPEG-4 Elementary Streams", RFC 3640, November 2003. 1474 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 1475 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 1476 RFC 3711, March 2004. 1478 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1479 Internet Protocol", RFC 4301, December 2005. 1481 [RFC4628] Even, R., "RTP Payload Format for H.263 Moving RFC 2190 to 1482 Historic Status", RFC 4628, January 2007. 1484 [RFC5109] Li, A., "RTP Payload Format for Generic Forward Error 1485 Correction", RFC 5109, December 2007. 1487 [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security 1488 (TLS) Protocol Version 1.2", RFC 5246, August 2008. 1490 [RFC5691] de Bont, F., Doehla, S., Schmidt, M., and R. 1491 Sperschneider, "RTP Payload Format for Elementary Streams 1492 with MPEG Surround Multi-Channel Audio", RFC 5691, 1493 October 2009. 1495 Authors' Addresses 1497 Malte Schmidt 1498 Dolby Laboratories 1499 Deutschherrnstr. 15-19 1500 90537 Nuernberg, 1501 DE 1503 Phone: +49 911 928 91 42 1504 Email: malte.schmidt@dolby.com 1506 Frans de Bont 1507 Philips Electronics 1508 High Tech Campus 5 1509 5656 AE Eindhoven, 1510 NL 1512 Phone: +31 40 2740234 1513 Email: frans.de.bont@philips.com 1515 Stefan Doehla 1516 Fraunhofer IIS 1517 Am Wolfmantel 33 1518 91058 Erlangen, 1519 DE 1521 Phone: +49 9131 776 6042 1522 Email: stefan.doehla@iis.fraunhofer.de 1524 Jaehwan Kim 1525 LG Electronics Inc. 1526 221, Yangjae-dong, Seocho-gu 1527 Seoul 137-130, 1528 Korea 1530 Phone: +82 10 6225 0619 1531 Email: kjh1905m@naver.com