Internet Engineering Task Force Audio/Video Transport Working Group INTERNET DRAFT Balabanian-Nortel draft-balabanian-rtp-mpeg4-dmif-00.txt September 16,1998 Expires: March 20,1999 The Role of DMIF in Support of RTP MPEG-4 Payloads STATUS OF THIS MEMO This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or made obsolete by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress''. To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. ABSTRACT This draft technical proposal describes how RTP carrying MPEG-4 payloads interacts with the MPEG-4 Sync layer through the MPEG (Delivery Multimedia Integration Framework) DMIF. Single or multiple MPEG-4 streams can be carried over one RTP session. MPEG-4 information essential for the efficient packing and unpacking of MPEG-4 streams into/from RTP is identified. The DMIF end-to-end signaling protocol is applied to identify the MPEG-4 RTP payload types and ensure stack compatibility at both sender and receiver locations. DMIF also interprets the RTCP reports by comparing its statistics to the requested MPEG-4 media based QoS. If the statistics fail to meet the requested QoS then action is taken to either continue with the impaired performance, upgrade the network service class, scale down the stream or delete the stream. This action is apart from scalability using the stream back-channel flow control which may be present between an encoder and its decoder. This is an update of the draft-ietf-avt-rtp-mpeg4-dmif-00. It reflects the latest MPEG-4 specs. In addition some clarifications are included and an open issues section is established based on feedback received. Balabanian [Page 1] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 1 Introduction MPEG-4 is a recent standard from ISO/IEC for the coding of natural and synthetic audio-visual data in the form of audiovisual objects that are arranged into an audiovisual scene by means of a scene description [1] [2][3][4]. This draft technical proposal specifies how DMIF is used to facilitate the generation and consumption of the RTP MPEG-4 payloads [5][6]. The MPEG-4 standards are versioned. Each version beyond V1 represents a backward compatible extension. MPEG-4 V1 is targeted to become ISO International Standard on December 1998 and each subsequent version will be displaced approximately by a year. MPEG-4 V2 is targeted for February 2000. DMIF is the part 6 of the MPEG-4 standard. Where indicated, parts of this draft technical proposal will impact on MPEG V2 International Standard targeted for February 2000. This draft technical proposal provides a solution for discussion in IETF AVT and ISO/IEC MPEG technical communities in order to identify issues in using of MPEG-4/DMIF with RTP and incorporate the results. This would lead to the finalization of the specification on RTP use of MPEG-4 with DMIF. 1.1 Overview of MPEG-4 End-System Architecture Figure 1 below shows the general architecture of MPEG-4 terminals. The Compression Layer processes individual audio-visual media streams without regard to delivery technologies. The compression schemes in MPEG-4 achieve efficient encoding over a wide range from few Kbps to multiple Mbps. The MPEG-4 compression schemes are defined in the ISO/IEC specifications 14496-2 and -3 [2][3]. The media content at this layer is organized in Elementary Streams. The MPEG-4 Systems specification, ISO/IEC 14496-1 [1], defines the concepts needed to describe the relations between Elementary Streams in a way that allows to create distributed, yet integrated, content presentations and to synchronize the streams. This part of the specification is both media unaware and delivery technology unaware. The Delivery Layer in MPEG-4 consists of the Delivery Multimedia Integration Framework defined in ISO/IEC 14496-6 [4]. This layer is media unaware but delivery technology aware. It provides transparent access to and delivery of content irrespective of the technologies used. The interface between the Sync Layer and DMIF is called DMIF Application Interface (DAI). It offers content location independent procedures for establishing MPEG-4 sessions and access to transport channels. DMIF is primarily an integration framework. It provides a default DMIF signaling (DS) protocol which corresponds to DMIF Network Interface (DNI)primitives, see Figure 2. DS is used to complement the lack functionality in underlying control protocols in order to keep the Balabanian [Page 2] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 integrity of the DMIF framework. media aware +-----------------------------------------+ delivery unaware | COMPRESSION LAYER | 14496-2 Visual | streams from few Kbps to multi-Mbps | 14496-3 Audio +-----------------------------------------+ Elementary Stream =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3DInterface (ESI) +-------------------------------------------+ media and | SYSTEMS SYNC LAYER | delivery unaware | manages elementary streams, their synch- | 14496-1 Systems | ronization and hierarchical relations | +-------------------------------------------+ DMIF Application =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3DInterface (DAI) +-------------------------------------------+ delivery aware | DELIVERY LAYER | media unaware |provides transparent access to and delivery| 14496-6 DMIF | of content irrespective of delivery | | technology | +-------------------------------------------+ Figure 1: General MPEG-4 terminal architecture 1.2 The DMIF Model DMIF as an integration framework uses a uniform procedure at the DAI interface to access the MPEG-4 content irrespective whether the content is broadcast, stored on a local file or obtained through interaction with a remote end-system. The model is shown in Figure 2 below. The specific instance of interest in this memorandum is the interaction with a remote end-system. For this case DMIF uses internal (informative) DMIF-Network Interface(DNI)primitives to map the controls obtained from the application through DAI into the various signaling systems appropriate to the various networks. The default end-to-end DMIF signaling (DS)which corresponds to DNI is specified in DMIF V1 [4]. Balabanian [Page 3] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 ! +---+ +-----------+ +-----------+ +-----------+ ! | | |Local DMIF | |Remote DMIF| |Remote App.| +-----+ ! | D | |for Brd'cst| |(locally | |(locally | Brd'cst | | ! | M | | | | emulated) | | emulated) |<-----source |Local| ! | I | +-----------+ +-----------+ +-----------+ | | ! | F | +-----------+ +-----------+ +-----------+ |App | ! | | |Local DMIF | |Remote DMIF| |Remote App.| File | | ! | F | |for Local | |(locally | |(locally |<-----source | | ! | i | | Files | | emulated) | | emulated) | | | ! | l | +-----------+ +-----------+ +-----------+ | | ! | t | +-----------+ ! +---+ / ---- \ | | ! | e | |Local DMIF | ! |Sig| | --- ---- | +-----+ ! | r | |for Remote | ! |map|<->( Network ) |Outside the ! | | | Service | ! | | | ---- ----- |scope of DMIF ! +---+ +-----------+ ! +---+ | ----- | DAI DNI | ^ | \_____|_________/ | | | +---+!+------+!+------+ | |Sig|!|Remote|!|Remote| +>|map|!| DMIF |!| App. | | |!|(real)|!|(real)| +---+!+------+!+------+ DNI DAI Figure 2: The DMIF model covers broadcast, local file storage and remote service with a uniform procedure for application transparency 1.3 Mapping between MPEG-4/DMIF and RTP Figure 3 below draws the correspondence between RTP and MPEG-4/DMIF. It is noted that DAI signaling allows the establishment of an MPEG-4 Service e.g., Video on Demand, the request of channels to carry MPEG-4 Elementary Streams for that service and the reading of Elementary Stream data when received. The control flows for various scenarios are defined in [4] Balabanian [Page 4] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 RTP MPEG-4/DMIF +-----------------------------+ +---------------------------+ / DATA TRANSPORT CONTROL \ / DATA TRANSPORT \ (RTP) (RTCP) | (Elementary Stream) Audio, Video SR, RR, SDES . Audio, Video Simulated Data BYE, APP | Simulated Data . ^ ^ | ^ CONTROL (Scene, Object I | . I Descriptor(OD), Play/ I | | I Pause, Intellectual I v . I Property Management) I Corresponds to | I ^ I DAI SIGNALING . I | I | I | I . I / I | I / I . I / | I / In addition to . I / SIGNALING Audio, Video | I/ (Service, Channel, Simulated Data . I Data) to include Scene | I ^ and OD streams . =3D=3D=3D=3DI=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D|=3D=3D=3D DAI | I | . I v | Elementary Streams . in SL-PDU fragments Figure 3: Drawing some correspondence between MPEG-4/DMIF and RTP The MPEG-4 stream packets passed across the DAI are formatted in Sync Layer PDUs (SL-PDU). The SL-PDUs can be mapped to RTP-PDUs as follows [6]: RTP-PDU 1:1 SL-PDU RTP-PDU 1:N SL-PDU RTP-PDU N:1 SL-PDU The selection for a particular MPEG-4 stream from the above choices is based on a number of factors including the size of the SL-PDU compared to the RTP-PDU MTU size[6]. The first choice uses MPEG-4 single stream RTP payload type. The second case uses MPEG-4 RTP FlexMux payload type. The last choice also uses MPEG-4 FlexMux RTP payload type. It occurs when MPEG-4 Sync Layer is not able to adjust the MPEG-4 SL-PDU lengths to be within the path MTU. Since MPEG-4 FlexMux is optional, any other equivalent scheme could be used. Some muxing schemes under consideration now for RTP are provided in [8][9][10]. Balabanian [Page 5] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 2 Operation of the RTP MPEG-4 payloads with DMIF This section covers the conceptual operation of the MPEG-4/DMIF with RTP. The DAI primitives shall be used to set up the MPEG-4 session[4]. When the RTP is used an originating (or a destination) DMIF entity could = be used to start the RTP session with its corresponding RTP data transport (carrying one or more MPEG-4 Elementary Streams) and RTCP for control. RTCP packets use the same transport media over which the RTP data packets are sent. 2.1 Using MPEG-4 single stream mapping into RTP session The AVT WG encourages the initial experimentaion on MPEG-4 payloads using a single MPEG_4 stream per RTP session. This is in contrast to the mode of multiple streams per RTP session specified in section 2.2. | | v +---+ SIGNALING |SL | (MTU + SLConfigDescriptor) +---+ | ^ MPEG ------- MPEG -------- | | =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D|=3D=3D=3DDAI=3D=3D=3D v v +------------------------------------+ +---------------+ | Regenerate SL-PDUs |<------| DMIF | +------------------------------------+ . | instance | MPEG-4 | Time | Sequence| . +---------------+ Payloads | Stamps | Numbers| ALConfig ^ ^ v v v Descriptor | | +-----------------------+ v | | RTP Coder | +-----------+| +-----------------------+ |RTCP Codec || I +-----------+| I ^ | I | | I +-----------------------+ v I | ~~~~~~~~DNI IETF ------ (DMIF Signaling V v Protocol) | MPEG ------ | v Figure 4: Conceptual view of the sender operating with MPEG-4 single stream RTP Payload type Balabanian [Page 6] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 Figures 4 and 5 show the operation at the sender and receiver ends respectively. In case of a single MPEG-4 stream payload type, the SLConfigDescriptor is being received at the sender side and being used both at the sender and the receiver for efficient packing and unpacking of the streams into and from the RTP transport [6]. The horizontal lines across the flows indicate the standard to which the flow conforms to. It is noted that while the streams above the DAI conform to MPEG, the network streams consist of a combination of IETF RTP/RTCP and MPEG Default DMIF signaling protocol. ^ | | +---+ SIGNALING |SL | (MTU + ALConfigDescriptor) +---+ ^ ^ | | MPEG ------- MPEG -------- | | =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D|=3D=3D=3DDAI=3D=3D=3D | v +------------------------------------+ +---------------+ | Regenerate SL-PDUs |<------| DMIF | +------------------------------------+ . | | MPEG-4 ^ Time ^ Sequence^ . +---------------+ Payloads | Stamps | Numbers| SLConfig ^ ^ | | | Descriptor | | +-----------------------+ v | | RTP Parser | +-----------+| +-----------------------+ |RTCP Parser|| ^ +-----------+| I ^ | I | | I +-----------------------+ | I | | IETF ------ | I | ~~~~~~~~DNI I v (Default DMIF Signaling Protocol) | MPEG ------ | v Figure 5: Conceptual view of the receiver operating with MPEG-4 single stream RTP Payload type Balabanian [Page 7] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 2.2 Using MPEG-4 multiple stream mapping into RTP session Although using the method of MPEG-4 FlexMux to carry multiple MPEG-4 streams over an RTP session is technically sound, the AVT WG is in the process of examining methods of generic muxing of RTP streams which in effect will achieve the same end of MPEG-4 FlexMux but without unpredictable side effects on RTP[8][9][10]. Figure 6 (sender) and 7 (receiver) show that in the case of MPEG-4 FlexMux RTP payload type, information for the MTU and SLConfig is not required. The FlexMux decoder however needs the MuxCode information which is generated at the sending end by the FlexMux muxing code and passed to the receiver through the DMIF signaling (DS). DS is an out of band point-to-point protocol to MPEG-4 media streams. It complements RTCP. Multicast DMIF signaling is for DMIF V2 or later consideration. Audio, Video, Simulated Data, Scene, ODs | | | | | | | | v v v v +---+ +---+ +---+ +---+ |SL | |SL | |SL |. . . . . . |SL | +---+ +---+ +---+ +---+ SIGNALING | | | | ^ | | | | | =3D=3D|=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D|=3D=3D=3DDAI=3D=3D=3D v v v v v +--------------------------------------+ +---------------+ | FlexMux |------>| DMIF | +--------------------------------------+ . | instance | MPEG-4 | . +---------------+ Payloads | . ^ ^ v MuxCode | | +-----------------------+ Descriptor v | | RTP Coder | +-----------+| +-----------------------+ |RTCP Codec || I +-----------+| I ^ | I | | I +----------------------------+ | I | ~~~~~~~~DNI IETF ------ (Default DMIF Signaling I | Protocol) V v | MPEG ------ | v Figure 6: Conceptual view of the sender operating with MPEG-4 FlexMux RTP Payload type Balabanian [Page 8] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 Audio, Video, Simulated Data, Scene, ODs ^ ^ ^ ^ | | | | | | | | +---+ +---+ +---+ +---+ |SL | |SL | |SL |. . . . . . |SL | SIGNALING +---+ +---+ +---+ +---+ ^ | | | | | ---------------------- MPEG ----------- MPEG ------ =3D=3D|=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D|=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D|=3D=3D=3DDAI=3D=3D=3D | | | | v +--------------------------------------+ +---------------+ | FlexDemux |<------| DMIF | +--------------------------------------+ . | instance | MPEG-4 | . +---------------+ Payloads | . ^ ^ v MuxCode | | +-----------------------+ Descriptor v | | RTP Coder | +-----------+| +-----------------------+ |RTCP Codec || ^ +-----------+| I ^ | I | | I +----------------------------+ | I | ~~~~~~~~DNI IETF ------ (Default DMIF Signaling I | Protocol) I v | MPEG ------ | v Figure 7: Conceptual view of the receiver operating with MPEG-4 FlexMux RTP Payload type 3 RTCP Sender and Receiver Reports RTP receivers provide reception quality feedback using RTCP report packets which may take one of two forms depending upon whether or not the receiver is also a sender. These reports shall be used by DMIF in the case of MPEG-4/DMIF-RTP to readjust the demand put on the network based on a predefined policy which may involve a decision to be made by the user. The sender report packet consists of three sections, possibly followed by a fourth profile-specific extension section if defined (none has been specified so far for MPEG-4 RTP payloads). Balabanian [Page 9] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 The third section contains zero or more reception report blocks depending on the number of other sources heard by this sender since the last report. Each reception report block conveys statistics on the reception of RTP packets from a single synchronization source. Annex A provides the DMIF per channel (MPEG-4 elementary stream) "media based" QoS. This is adjusted for the RTP stream in both single and multiple stream MPEG-4 mappings. The remainder of this section attempts to match the "delivered" RTP stream performance as measured by the receiver reports to the expected performance calculated using the "media based" QoS. SSRC_n (source identifier): -------------------------- The SSRC identifier of the source to which the information in this reception report block pertains. This SSRC may either relate to an MPEG-4 single or FlexMux RTP payload session. fraction lost: ------------- The fraction of RTP data packets from source SSRC_n lost since the previous SR or RR packet was sent, expressed as a fixed point number. This type of loss is normally compensated by the decoder through mechanisms such as concealment. The fraction lost is compared to the LOSS_PROB in Annex A. It is important that the duration over which this metric is measured is 1 sec to correspond to the same duration used to express the LOSS_PROB. If the statistics consistently exceeds the LOSS_PROB then the policy enforcer is brought into action. As a result the load on the RTP stream is reduced. interarrival jitter: ------------------- An estimate of the statistical variance of the RTP data packet interarrival time, measured in timestamp units and expressed as an unsigned integer. The jitter calculation in RTCP is based on the variation of consecutive interarrival times: If Si is the RTP timestamp from packet i, and Ri is the time of arrival in RTP timestamp units for packet i, then for two packets i and j, D may be expressed as D(i,j)=3D(Rj-Ri)-(Sj-Si)=3D(Rj-Sj)-(Ri-Si) The interarrival jitter is calculated continuously as each data packet i is received from source SSRC_n, using this difference D for that packet and the previous packet i-1 in order of arrival (not necessarily in sequence), according to the formula J=3DJ+(|D(i-1,i)|-J)/16 Balabanian [Page 10] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 Whenever a reception report is issued, the current value of J is sampled. The synchronization between streams in MPEG-4 does not rely on the jitter value e.g., for lip sync. This function is carried out at the level of the Sync layer based on composition time stamps of the respective streams. In MPEG-4/DMIF V2 the jitter is used to ensure that the receive decoder buffers are not exceeded. It is noted that the RTCP interarrival jitter is intended as a comparison measure between streams or at different times rather than as an absolute measure. therefore the formula below is based on a specific method of managing the dejitter buffer. Assuming that the operation adjusts so that the pointer for decoding the stream is in the middle of the Dejitter_Buffer. The Buffer can accept an amount of burst or deficiency not exceeding twice the value of J x Maximum Rate where: Maximum Rate =3D MAX_RTP_SIZE* MAX_RTP_RATE Therefore, J must be <=3D to .5*Dejitter_Buffer/( MAX_RTP_SIZE* MAX_RTP_RATE) If this value is exceeded consistently for some time then the QoS policy enforcer is brought into action. As a result the load on the RTP stream is reduced. Round trip Delay: ---------------- This delay is calculated by measuring the time sending a sender report and receiving the associated receiver report and subtracting the delay it took to send the receiver report at the receiver. Delay must be <=3D 2*MAX_DELAY converted to seconds from microseconds Average Delay over 1 minute <=3D 2*AVG_DELAY converted to seconds from microseconds If either of the above values is exceeded consistently for some time then the QoS policy enforcer is brought into action. As a result the load on the RTP stream is reduced. 4. SDES When a DA_ChannelAdd() is requested by the application, DMIF decides whether to initiate a new RTP stream or use one of the existing ones with a FlexMuxed payload type. Only in the case if a new RTP stream is decided the SDES RTCP packet will be sent. This will coincide with the DS_TransMuxSetup() sent on DMIF Signaling [7]. Balabanian [Page 11] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 5. BYE: Goodbye RTCP packet When a single stream is used in the case of the MPEG-4 single stream RTP = payload, a BYE packet is sent along with the DS_ChannelDelete using DMIF-DMIF signaling[4]. At the receiver either a BYE packet or DS_ChannelDelete signal will cause DAI to pass DA_ChannelDelete to the application. When a FlexMux stream is used, the BYE packet is generated when no longer any MPEG-4 streams are carried on the RTP session. This means that DS_ChannelDeletes have already been sent for all the channels carried on the RTP session and the application has been notified by DA_ChannelDelete(s) across the DAI. A BYE message is followed by a DS_TransMuxDelete which at the reception will allow both the sending and receiving DMIF sides to reuse the RTP/IP ports[4]. 6. Open Issues: 1- Multicast operation and the inclusion of RTP mixers i.e., aggregation of the streams and adjustment of their QoSs from receivers up to the senders. 2- The inclusion of RTP Payload Format for User Multiplexing [8][9][10] Balabanian [Page 12] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 A Derivation of the RTP QoS using MPEG-4/DMIF Media Based QoS The media based QoS and associated priority are important considerations in MPEG-4 [1][4] since they are used as a base for decision making on how to transport the streams over a network. The following table provides the media QoS specified by the content provider irrespective of the network used for the transport of the media. No QoS values will be available in DMIF V2 The only parameters being specified in V1 are for characterizing the traffic load of the stream (the last 3 rows in the table below). The values expressed in the table below relate to the MPEG-4 Access Units (AU), these are the smallest data entities to which timing information can be attributed. The AUs form the payload of the SL-PDUs and may undergo segmentation by the Sync Layer. For example the maximum SL-PDU size of an MPEG-4 stream can never be larger than the one that corresponds to the maximum AU size. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ | Media | Meaning of the | | QoS_QualifierTag | ES Media QoS | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ | MAX_DELAY |Maximum delay per AU (microseconds) | | (DMIF V2) |measured over 1 sec. | +---------------------+-------------------------------------+ | AVG_DELAY |Average delay per AU allowed | | (DMIF V2) |(microseconds) measured over 1 min | +---------------------+-------------------------------------+ | Dejitter Buffer | Bytes reserved for the removal of | | (DMIF V2) | transport jitter from the steam | +---------------------+-------------------------------------+ | LOSS_PROB |Probability of loss of any single AU | | (DMIF V2) |(Fraction (0.00 - 1.00) over 1 sec. | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | MAX_AU_SIZE |Maximum size of an AU (Bytes) | | (DMIF V1) | | +---------------------+-------------------------------------+ | MAX_AU_RATE |Maximum arrival rate of an AUs | | (DMIF V1) |(AU-PDU/second) | +---------------------+-------------------------------------+ | AVG_RTP_SIZE |Average size of AUs (Bytes) | | (DMIF V1) | | +---------------------+-------------------------------------+ Balabanian [Page 13] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 A.1 The case RTP-PDU 1:1 (or N:1) SL-PDU The table below shows the stream QoS for the case when a single ES is mapped into the RTP PDU. Only the traffic load parameters (the last 3 rows in the table below) are specified in DMIF V1 targeted to be an ISO/IEC International Standard in December 1998. Normally an RTP-PDU would carry an SL-PDU with a complete AU. In rare cases the SL-PDU would segment the AU in order to for its size to correspond the the RTP-PDU MTU (IP size) as dictated by the underlying network. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ | RTP Stream | Derivation from the | | QoS_QualifierTag | ES Media transport-QoS | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ | MAX_DELAY of RTP PDU|Maximum delay per AU (microseconds) | | (DMIF V2) |measured over 1 sec. | +---------------------+-------------------------------------+ | AVG_DELAY of RTP PDU|Average delay per AU allowed | | (DMIF V2) |(microseconds) measured over 1 min | +---------------------+-------------------------------------+ | Dejitter Buffer | Adjusted for the overhead of the | | for the RTP stream | RTP PDU | | (DMIF V2) | | +---------------------+-------------------------------------+ | LOSS_PROB of RTP PDU|Probability of loss of any single AU | | (DMIF V2) |(Fraction (0.00 - 1.00) over 1 sec. | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | MAX_RTP_SIZE |Maximum size of an AU (Bytes) | | (DMIF V1) |Plus AL-PDU and RTP overhead | +---------------------+-------------------------------------+ | MAX_RTP_RATE |Maximum arrival rate of AUs | | (DMIF V1) |(RTP-PDU/second) | +---------------------+-------------------------------------+ | AVG_RTP_SIZE |Average size of AUs (Bytes) | | (DMIF V1) |Plus AL-PDU and RTP overhead | +---------------------+-------------------------------------+ Balabanian [Page 14] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 A.2 The case RTP-PDU 1:N SL-PDU The table below shows the stream QoS for the case when multiple ES are aggregated into the RTP PDU. Only the traffic load parameters (the last 3 rows in the table below) are specified in DMIF V1 targeted to be an ISO/IEC International Standard in December 1998. In most cases this method will be used when the AU is <<256 bytes. Each SL-PDU therefore will carry a complete AU. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ | RTP Stream | Derivation from the | | QoS_QualifierTag | ES Media QoS | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+ | MAX_DELAY of RTP PDU|Least Maximum delay per AU from | | (DMIF V2) |among the N AL-PDUs(microseconds) | | |measured over 1 sec. | +---------------------+-------------------------------------+ | AVG_DELAY of RTP PDU|Average delay per AU allowed | | (DMIF V2) |(microseconds) measured over 1 min. | +---------------------+-------------------------------------+ | Dejitter Buffer | Total of dejitter buffers adjusted | | for the RTP stream | for the overhead of the RTP PDU | | (DMIF V2) | | +---------------------+-------------------------------------+ | LOSS_PROB of RTP PDU|Least Probability of loss of any | | (DMIF V2) |single AU from the N AL-PDUs | | |(Fraction (0.00 - 1.00) over 1 sec. | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ | MAX_RTP_SIZE |Sum of the MAX_AU_SIZEs of from | | (DMIF V1) |each of the N AL-PDUs Plus AL-PDU | | |and RTP overhead | +---------------------+-------------------------------------+ | MAX_RTP_RATE |Highest MAX_AU_RATE of AUs from each | | (DMIF V1) |of the N AL-PDUs (RTP-PDU/second) | +---------------------+-------------------------------------+ | AVG_RTP_SIZE |Sum of Average size of AUs from | | (DMIF V1) |each of the N AL-PDUs Plus | | |AL-PDU and RTP overhead (Bytes) | +---------------------+-------------------------------------+ Note all the MPEG-4 streams chosen for aggregation over an RTP stream belong to the same stream priority level identified by the Sync Layer. B Authors' Addresses Vahe Balabanian Nortel P.O.Box 3511, St. C Ottawa, Ontario Canada K1Y 4H7 Email: balabani@nortel.ca Balabanian [Page 15] Internet Draft Role of DMIF with RTP MPEG-4 Payloads September 16,1998 B Bibliography [1] ISO/IEC 14496-1 FCD "MPEG-4 Systems" Oct. 1998, obtained from the MPEG Home Page http://drogo.cselt.it/mpeg/ [2] ISO/IEC 14496-2 FCD "MPEG-4 Visual" Oct. 1998, obtained from the MPEG Home Page http://drogo.cselt.it/mpeg/ [3] ISO/IEC 14496-3 FCD "MPEG-4 Audio" Oct. 1998, obtained from the MPEG Home Page http://drogo.cselt.it/mpeg/ [4] ISO/IEC 14496-6 CD "DMIF" Oct. 1998, obtained from the MPEG Home Page http://drogo.cselt.it/mpeg/ [5] Schulzrinne, Casner, Frederick, Jacobson 'RTP: A Transport Protocol for Real Time Applications' RFC 1889,Internet Engineering Task Force, Jan. 1996. [6] Carsten, Balabanian, Basso, Civanlar, Hoffman, Speer, Schulzrinne, 'RTP payload format for MPEG-4 Elementary Streams' draft-ietf-avt-rtp-mpeg4, Internet Engineering Task Force, March 1998. [7] Balabanian, 'The Use of MPEG-4/DMIF and RSVP with Integrated Services' draft-balabanian-intserv-dmif, Internet Engineering Task Force, August 1998. [8] J.Rosenberg, H.Schulzrinne 'An RTP Payload Format for User Multiplexing' draft-ietf-avt-aggregation, Internet Engineering Task Force, May 1998. [9] Keiko Tanigawa, Tohru Hoshi, Koji Tsukada 'An RTP Simple Multiplexing Transfer Method for Internet Telephony Gateway' draft-tanigawa-rtp-multiplex-00, Internet Engineering Task Force, June 16, 1998. [10] B. Subbiah, S. Sengodan 'User Multiplexing in RTP payload between IP Telephony Gateways' draft-ietf-avt-mux-rtp-00, Internet Engineering Task Force, Aug 31, 1998. Internet Engineering Task Force Audio/Video Transport Working Group INTERNET DRAFT Balabanian-Nortel draft-balabanian-rtp-mpeg4-dmif-00.txt September 16, 1998 Expires: March 20, 1999 Balabanian [Page 16]