[AVT] SVC payload draft - decoding order recovery for the NI-C and NI-TC modes

["Ye-Kui.Wang at nokia.com" <Ye-Kui.Wang at nokia.com>]

Folks,

We just had a design team conference call discussing the SVC payload
draft
(http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-svc-13.txt). One
topic discussed was about the decoding order recovery process for the
NI-C and NI-TC modes (specified in section 7.1.2 in
draft-ietf-avt-rtp-svc-13.txt). In this process, a remultiplexing buffer
is used. The remultiplexing buffer takes as input the RTP packets output
from the RTP-level reception process for each of the RTP sessions
involved in the multi-session transmission. The decoding order recovery
process then operates on the remultiplexing buffer. 

The topic discussed was why the ecoding order recovery process does not
operate directly one NAL units from the RTP-level reception buffer of
each RTP session, similarly as in the decoding order recovery process
for the NI-T and NI-TC modes (specified in section 7.1.1 in
draft-ietf-avt-rtp-svc-13.txt), and what is the usefulness of the
related parameters like sprop-mst-interleave-depth and
sprop-remux-buf-req. I have explained the use cases during the
conference call, and was requested to send an email to this mailing with
the explanation, in order to have an open discussion.

So here are the use cases and the use of the parameters related to the
remultiplexing buffer. 

Use case 1: robust scheduling. The sender transmits more important data
in the lower layers conveyed in lower RTP sessions earlier than less
important data in the higher layers conveyed in higher RTP sessions,
thus allowing a better chance to receive more important data, e.g. by
allowing retransmission of the more important data. As a result, the
receiver needs to use extra buffer for higher layer data before starting
the decoding order recovery process. The parameters define the minimum
size of the buffer required. 

Use case 2: constant bitrate transmission. In many application
scenarios, e.g. DVB-H, the pipes for conveying data from different RTP
sessions are of constant bandwidth. However, the instant video bitrates
for different layers are by nature variable over time. This is more true
in SVC. For example, for a video bitstream containing regular random
access points, which are required in multicast and broadcast
applications to enable new comers to tune in and program channel
switching, IDR pictures are typically encoded at the random access
points. For the base layer, which is H.264/AVC compatible, IDR pictures
are true intra pictures. For the enhancement layers, IDR pictures can
still be encoded using inter-layer prediction hence the coded size is
way smaller than the base layer intra picture at the same access unit
(it is common that an intra coded picture is 10 time larger as an inter
coded picture). Therefore, the bitrate of an enhancement layer can be
much smoother than the bitrate of the base layer. As a result, to let
the data pass the pipes, the transmitter has to transmit data of the
base layer after the IDR picture much later than the data of an
enhancement layer at the same access unit. Similarly as above, the
receiver needs to use extra buffer for enhancement layer data before
starting the decoding order recovery process. The parameters define the
minimum size of the buffer required. 

Use case 3: use of NI-MTAP packets. The newly specified NI-MTAP packet
allows aggregation of data from multiple access units in a
non-interleaved manner within a session. However, when the use of
NI-MTAPs is not aligned among all the RTP sessions (e.g. this is always
the case when the base RTP session is RFC 3984 compatible - which is a
requirement per the payload draft), the receiver needs to use extra
buffer for the RTP session that does not use NI-MTAP or that uses
NI-MTAP but aggregating data from less access units before starting the
decoding order recovery process. The parameters define the minimum size
of the buffer required.
 
BR,YK
_______________________________________________
Audio/Video Transport Working Group
avt at ietf.org
https://www.ietf.org/mailman/listinfo/avt