[AVT] Carrying SMPTE time-codes in RTP streams, discussion email

[Dave Singer <singer at apple.com>]

Here are some thoughts on carrying SMPTE time-code information in RTP.

First a brief background on SMPTE time-codes.

SMPTE time-codes count frames.  There are two common forms of 
display:  either a simple counter, or what looks like a normal clock 
value (hh:mm:ss.frame).  When the frame rate is truly integer, then 
this can be a normal clock value, in that seconds tick by at the same 
rate as the seconds we know and love.

However, NTSC video infamously runs slightly slower than 30 
frames/second.  Some people call it 29.97 (which isn't quite right) 
and some say that a frame takes 1001 ticks of a 30000 tick/second 
clock (which is closer).  Be that as it may, SMPTE time codes count 
30 of these frames and deem that to make a second.

This causes a SMPTE time-code display to 'run slow' compared to 
real-time.  To ameliorate this, a format called drop-frame is used. 
Some of the frame numbers are skipped, so that the counter 
periodically 'catches up' (so some time-code-seconds actually only 
have 28 or 29 frames in them).


So, what we desire is a system that allows us to associate a SMPTE 
time-code with some media.  Since in RTP all media has a clock 
already, we can leverage that fact.  If we treat the media as having 
'runs' of time in which the time-code is simply counting up, then the 
time-code anywhere within a run can be calculated if you know
1. the RTP timestamp of the start of the run
2. the RTP timestamp where you want to know the time-code
3. the time-code of the start of the run
4. the counting rate and other parameters of the time-code.

My proposal is that we put periodic mappings between (1) and (3) into 
RTCP packets, and provide (4) the 'setup' information out-of-band, 
for example in SDP.

The setup information includes:
  (the timescale of the RTP stream, already provided)
  the duration, in that timescale, of a single frame-count in the 
'frames' portion of the time-code
  the number of those frames that make a time-code-second
  the following booleans:
    is-NTSC-drop-frame:  should the usual 'left out numbers' of 
drop-frame be applied or not?
    wrap-at-24-hours:  should the hours portion wrap from 23 to 0, or 
keep counting up?
    allow-negative-time-codes:  are negative time-codes used in this stream?
    display-time-code-as-counter:  should the display be an integer 
frame-count, or hh:mm:ss.fr format?
    time-code-displayed:  is it intended that this time-code be 
displayed somehow?

For example, if associated with a video track with the common 
time-scale of 90000, then frame-duration of 3003 and 
frames-per-tc-second of 30 would yield a 'normal' SMPTE time-code for 
NTSC video.  Similarly values of 3750 and 24 yield a time-code for 24 
fps film content, and so on.

Now, we put into an RTCP APP packet (or a new RTCP packet), a mapping 
between an RTP time-stamp value and the time-code.  The RTP timestamp 
and the that time-code are 32 bits;  the time-code is either a signed 
counter value (if we're in counter format), or it is the format
  hours(8) -- 0 to 255
  sign(1) -- 1 for negative, 0 for positive
  minutes(7) -- 0 to 59
  seconds(8) -- 0 to 59
  frames(8) -- 0 to (frames-per-tc-second - 1)

This establishes the time-code for all RTP times greater than or 
equal to the one given, until a subsequent APP packet reestablishes 
the mapping.  It's unfortunate that the sign is in the middle, but 
that allows the hours to use the full range, and the minutes don't 
need to.

* * *

It might be argued that we could set the initial mapping also in the 
SDP, since RTCP packets might get lost.  But this means that the SDP 
now has to have knowledge of the RTP random offset (nasty);  and if 
one puts this APP packet into all sender reports, it's probably good 
enough.  Then if you don't have time-codes, you don't have 
audio-video-sync either.

This associates the time-code with a particular media stream.  An 
alternative would be to make it an RTP stream in its own right;  but 
the data rate is so low, this seems egregious.  And by packing it 
inline, we can do this backwards-compatible for gateways etc. that 
already handle dual-stream.

Comments?

--
David Singer
Apple Computer/QuickTime

_______________________________________________
Audio/Video Transport Working Group
avt at ietf.org
https://www1.ietf.org/mailman/listinfo/avt