RE: [AVT] Post WG last call edits on the H.264 payload format: Technical change

[miska.hannuksela at nokia.com]

Magnus, all,

sprop-max-don-diff cannot be used to control de-interleaving buffering in the use case described in the Internet Draft (section 5.5):

       Live broadcast is interrupted 
       by pre-encoded content such as commercials from time to time.  
       The first intra picture of a pre-encoded clip is transmitted in 
       advance to ensure that it is readily available in the receiver.  
       At the time of transmitting the first intra picture, the 
       originator does not exactly know how many NAL units are going to 
       be encoded before the first intra picture of the pre-encoded 
       clip follows in decoding order.  Thus, the values of DON for the 
       NAL units of the first intra picture of the pre-encoded clip 
       have to be estimated at the time of transmitting them and gaps 
       in values of DON may occur. 

In this example, the first intra picture of the pre-encoded commercial must be given a large DON value to make sure that it is not decoded too early. The value of DON for this first intra picture must also be such that it does not cause pictures to be flushed from the de-interleaving buffer. Thus, the use of sprop-interleaving-depth is required in this use case.

sprop-max-don-diff makes the de-interleaving process vulnerable to transmission losses in some cases. For example, if the DONs of the transmitted stream are ..., 100, 101, 102, 200, 103, 104, 105, ..., and sprop-max-don-diff is e.g. 50, NAL unit with DON 200 is supposed to flush the de-interleaving buffer. However, if that NAL unit is lost during transmission the de-interleaving buffer keeps filling in and may overflow.


sprop-init-buf-time is useful at least in the following cases:
- The de-interleaving buffer can also be used to smooth out variations in transmission scheduling. Video decoder input buffers are not guaranteed to handle non-constant input bitrate (or to be more specific: other rate of input than specified in the hypothetical reference decoder of the video coding standard). Thus, to be on the safe side, if a server applies any transmission scheduling other than constant bitrate, it should make sure that the receiver has a buffer, e.g. de-interleaving buffer, before the decoder. sprop-max-don-diff cannot be used to control the initial buffering before decoder (as data unit may be sent in decoding order).
- As demonstrated above, sprop-max-don-diff cannot be used in some use cases. Then, sprop-init-buf-time and sprop-interleaving-depth become the only parameters to control the initial buffering.
- The file containing the stream also contains a hint track which always includes a transmission schedule. If a streaming server does not modify the transmission schedule, the value of sprop-init-buf-time is valid. Otherwise (the server applies fine-grain rate control / transmission scheduling), sprop-init-buf-time gives an approximate of initial buffering for the receiver.

Moreover, sprop-init-buf-time is clearly specified and optional to use (in sender and receiver). It gives the receiver a possibility for shorter initial buffering in some use cases than what can be achieved with sprop-max-don-diff. Therefore, I don't see any harm of keeping the text regarding sprop-init-buf-time unchanged.


> Thus one can ask the questions:
> 1. Is it necessary to know what memory or space requirements 
> are put on 
> de-interleaving buffer?

Yes, on devices having fairly constrained amount of memory it is important to allocate as small amount of memory as possible. On the other hand, dynamic memory allocation during the de-interleaving process may not be always possible.

> 2. Is the most appropriate way to express the buffer size in 
> number of 
> bytes or NALUs?

Both options have merits and shortcomings. 

sprop-deint-buf-req is specified for a specific implementation of a deinterleaving buffer (described in the informative part of the Internet Draft). There can be implementations where its value is irrelevant and it cannot be used for de-interleaving buffer allocation. On the other hand, sprop-deint-buf-req gives an exact amount of memory to allocate, if the receiver is implemented as described in the informative part of the Internet Draft.

sprop-interleaving-depth is specified based on the characteristics of the packet stream. It can be used in all implementations for de-interleaving buffer size determination, but concluding the required maximum buffer size in bytes accurately is impossible. sprop-interleaving-depth is also necessary to control the buffering process, as sprop-max-don-diff may not always be sufficient (as described above).


Based on the motivatins above, I propose keeping the Internet Draft unchanged.

Best Regards,
Miska Hannuksela
Nokia

_______________________________________________
Audio/Video Transport Working Group
avt at ietf.org
https://www1.ietf.org/mailman/listinfo/avt