Re: [AVT] alignment of layers issue

[Magnus Westerlund <magnus.westerlund at ericsson.com>]

Hi,

See inline

Thomas Schierl skrev:
>> Thus there are some consideration that clock skew updates should not be
>> done to often and maybe some implementation tips would be to hold off
>> adjusting a multi-layered until SR packets with updates has been
>> received for all layers. This do work as clock skew fixing is only
>> adjusting its relation to other media streams, not the internal process.
>>   
> 
> But dealing with the updates seems to become really complicated. What
> would this mean? You would have to inform the receiver, that an update
> was going on in the binding of RTP and NTP. This would mean the receiver
> would have to get all "updated" RTCP reports in the different sessions
> first, before it can make use of it. So, another extra signaling?
> 

No, a receiver can detect when the reference point by comparing data
from previous SR packet and the present if there has been a change. So
in the case of multi-layer the receiver continues to use the old
synchronization information until it has received changes on all media
streams.

> 
> Anyway, let us assume for the time being, we can make this work with a
> relatively complex implementation in the sender as well as in the receiver.
> 

This is not complex, it is standard RTCP that you anyway need to
implement, with the exception of handling of changes that is new for
multi-layer.

> I still ask myself and I guess also other people do so, why not doing
> this little change towards "RTP timestamp alignment" for data of the
> same layered sample in the multiple sessions? It's so easy and the only
> real argument against it was the security issue which I guess was
> discussed earlier. With this, I further assume we can keep the SSRC
> alignment of the sessions in the session multiplex and we can also make
> sure, that using the same timescale in the different sessions is ok.
> 

The reason is that you anyway need to do it. You can't handle this in a
non-layered codec that is synched with something else without these
mechanism, so what is the difference between synching audio and video
with synching within a single media between the layers.

> The thing what I would wish to have is a simple and robust mechanism,
> where I do not have to tweak timestamps and where I do not have to have
> extra signaling and where I do not have to care about lost or late
> packets and where I just can take two numerical values from the packets
> and know whether this is of the same sample or not. No rounding problems
> at all. The mechanism is there and it comes for free!

I agree that the mechanism is in place and exist, it is called RTCP SR!
> 
> Remember for multi session transmission, the alignment of data is not
> required after decoding as for lip-sync, it is already required before
> you can start decoding.  And, there are not so many implementations
> which gets the synchronization right. For lip-sync it is just an
> quality-of-implementation issue. For data alignment of layered codecs,
> it becomes an correctness issue for the implementation.
> 

I do understand why if one has a single media that it appears that
aligning the timestamps solves all the problems. Well that isn't the
truth, as you anyway need to ensure that you bind the correct SSRC in
each session through CNAME to the right original source. So if you want
this to be possible to use you will require also SSRC alignment across
all sessions.

Yes, this has been done before, however it doesn't come for free and has
a latent problem if you have sources that aren't using the layered codec
and don't think about these restrictions. So by accepting this as a
general rule will result in restriction to the RTP model.

So I do want to question if we really should do this restriction which
only provides a benefit in the single media case. As soon you have more
than one media or streams that needs synchronization you anyway need to
use RTCP.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: magnus.westerlund at ericsson.com
----------------------------------------------------------------------
_______________________________________________
Audio/Video Transport Working Group
avt at ietf.org
https://www.ietf.org/mailman/listinfo/avt