[AVT] Clock Skew update issue for layered codecs

Hi,

I wanted to start a separate thread on this particular issue with
multi-layered codecs.

Lets assume that we have 3 layers produced by the media codec from a
single media source (microphone). We have some clock skew on the audio
card sampling clock that will need to be communicated at least now and
then to keep this under control.

The transmission of RTCP SR will happen at regular intervals in each
layer but the jittering will make this transmission into a random
process on the short time scale.

In the below figure each x is an RTCP packet, the y is when the RTCP SR
information has been updated for the clock skew.

          123456789ABCDE
A ---x----x----y--y----y

B -x----x--x----y---y---

C --x--x---x--x---y----y

So the big question here is when a receiver can take the clock skew
compensation into consideration and apply it to the actual playout?

A time 1 all layers have the same synch information. At time 6 layer A
receives the clock skew update in the RTCP SR. At this point it could
apply it to this layer. However, to avoid misaligning the audio frames
the compensation needs to happen at all layers. To me it appears that
can be done in two ways either wait until you have seen it in all layers
or do it as soon as you see it in one layer. And the first doesn't work
as one can't be certain that all layers will provide the same update to
all layers because of the following situation:

          123456789ABCDE
A ---x----x----y--z----z

B -x----x--x----y---z---

C --x--x---x--x---z----z

So in this case the C layer never transmit an RTCP packet with the y
clock skew correction because a new one is needed an indicated by z. As
this can continue forever there is no way waiting until it has been seen
in all layers. Thus one must use the latest adjustment.

If one is relying on the individual layers RTCP SR to align the audio
frames in each layer one suddenly have a issue. Does an RTCP SR packet
in one layer adjust the clock skew or provide an accurate point of
aligning the frames? The only way of getting out of this bind I think
would be to rely very carefully on the provided clocks and really check
if the timestamp values align perfectly. And in cases the clock skew is
exactly one frame length one needs to adjust the compensation to not align.

Frankly, I think aligning the timestamp values between the layers to
simplify the clock skew compensation is the only reasonable solution.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: magnus.westerlund@ericsson.com
----------------------------------------------------------------------
_______________________________________________
Audio/Video Transport Working Group
avt@ietf.org
https://www.ietf.org/mailman/listinfo/avt