[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [AVT] Submission and request for feedback on draft-valin-celt-rtp-profile-00.txt

Randell Jesup wrote:

Noted. We missed that when changing some SHOULDs for MUSTs. We hesitated a
lot in deciding what's a SHOULD and what's a MUST. Basically, CELT's
advantage is that it can operate with almost any sampling rate, frame size
or bit-rate. The disadvantage is that unless we specify "baseline
requirements", we might end up with several implementations that are unable
to inter-operate. Also, feel free to suggest a better baseline if you think
we didn't select the right one.

Is this intended to be used as a "speech" codec at all; as an alternative
to iLBC/G.729/G.722.x/etc?  If so, then support for 8KHz and/or 16KHz may
be important to mandate for interoperability reasons.
CELT is (so far at least) not intended for lower sampling rates like 8 kHz or 16 kHz and doesn't operate in the same space as the codecs above or Speex. It's closer to codecs such as AAC-LD, G.722.1C, G.719, and ULD, though only ULD has a delay as short as CELT's. 
You DON'T want to be
mixing sample rates if multiple codecs are accepted.  (In theory it can be
done, but in practice it would be risky, especially in the face of packet
loss.)  For example, if you have this:

Random example (I probably have the G722 media type wrong):
m=audio 4321 RTP/AVP 0 97 98
a=rtpmap:0 PCMU/8000
a=rtpmap:97 G722/16000
a=rtpmap:98 CELT/48000

To quote from an earlier AVT email I wrote on this subject on 3 Dec 2007:

    Subject: Re: [AVT] I-D ACTION:draft-ietf-avt-rtcpxr-audio-01.txt
    [big SNIP]
    This means the timestamp rate can change at any point, on a
    packet-by-packet basis.  It's even theoretically allowable to alternate
    G711 and G722 packets.  Totally odd and non-useful, but it illustrates
    the point.  More realistic is a change from one to the other half-way
    through an RTCP monitoring period.
So you mean that you'd need to maintain a coherent timestamp despite changing between codecs that have different sampling rates? I wasn't aware of that, so I guess it's something that needs to be addressed. Maybe just by saying that different sampling rates SHOULD NOT be used with the same m= 


The wording there is very lawyer-ese, and really addressing how it's
affected by a multicast or conference setting. 
If the b=AS is at the m= level of the SDP (not above all the m='s), then it
only applies to that one media stream.  However, exactly what b=AS *means*
is very fuzzy.  b=AS is not a codec parameter; it's a stream parameter, and
it's also a reception parameter, not a "I plan to send" parameter.
There's been a lot of discussion about b=TIAS (RFC 3890) as a better way to
specify bandwidth.  Note that b=AS INCLUDES RTP/UDP/IP overhead, and thus
implicitly is dependent on packet rate.
I've seen many G.711 implementations using b=AS:64, so I thought we could use it in a way that excludes the overhead.
More to the point, b=AS is just one way to specify bandwidth. 
Another huge blocker for using b=AS (or b=anything) in this way: what if
another codec also offered in this stream wanted to re-use b=AS as well?
And what if the preferred bitrates (or max-bitrate) for each codec was
different?

How much do you *need* to specify bandwidth here?  Realize that most
devices don't have a good idea what receive bandwidth is even theoretically
available, let alone practically.  Most configuration is done at the sender
end, or by explicit choice of codec and bitrate.

I suggest reviewing how other multiple-bitrate codecs like G.722.x handle
this (AMR-WB, etc).

Also, isn't CELT true variable-bitrate?  If so, the bitrate to use
(initially?) might be very different than the "maximum" bitrate.
CELT can change bit-rate at any time, but so far it only changes based on what the senter wants to use, i.e. to adapt to congestion. Even if used with b=AS:, I was thinking that it would be more like a max bit-rate anyway. 


Well, I didn't see that to be a problem considering that one would probably
want the same packetization time for any codec. Do you see a case where you
wouldn't want that? I'm not not quite sure how people use the ptime in
practice and how much it is followed.


Sure.  You might prefer 10ms, which G.711 and some others support, but iLBC
only supports 20 and 30 ms frames, and thus only multiples of those for the
actual packetization time. You can still specify a ptime of 10 when using
iLBC. 
ptime is merely a "I would prefer to receive" parameter.  Do not rely on it
for anything.  The actual packetization time does NOT have to be the same
in each direction, or even the same from one RTP packet to the next.
(packet 1 could have 1 frame and packet 2 could have 10). 
iLBC is really negotiating the framesize, not the packetization time.
Well, I was thinking of using the ptime just for a preference. You specify the frame size and ptime helps decide how many frames get sent per packet. 
Ok, but this really isn't part of the codec.  Doesn't hurt to tell people.
(I'm assuming that the SDP spec for maxptime allows ignoring it (somewhat)
- if it doesn't, then you have to reject that payload.)


As far as I understand, maxptime is a SHOULD in the rfc, so I thought we'd
just mention its interpretation wrt CELT.


Ok, then say something like "per [RFC 4566], if the maximum is lower..."

OK, will do that.

OK, so maybe we should add a BNF. As for the examples, I think they're
adequate, but let me know if that's not the case.


BNF may not be *needed*, but if there's anything at all complex it's handy to
avoiding mistakes.

Noted.

The fundamental issue here is that one needs to know the frame size to be
able to initialise the decoder. Just like a codec like iLBC had two modes
(for 20 ms and 30 ms frame), CELT has a *very large* number of modes: one
for each combination of frame size and sampling rate. So the idea was that
one side offers a list of frame sizes and the other side responds with the
one it likes best and both sides use that. There is no way to decode media
without knowing the frame size. Any idea what's the best way to handle
that?


a) use different payloads.  Clear, easy, wastes space in SDP
b) include framesize in the bitstream in *every* packet.  Clear, easy,
   no-fuss, wastes some bandwidth all the time.  And isn't this required
   anyways if there's more than one channel?
c) ignore media until you get an answer with a clear selection.  Clear,
   easy, could be a major loss of media on a delayed answer.
d) require media sent (at least until acknowledgment of an answer) be
   clearly distinguishable - i.e. do not vary framesize from the offered
   values until an ACK, and do not allow packet sizes that are common
   multiples of offered framesizes.
For example, if you offer (say) 20 and 30, do not send a packet containing 60, 120, etc. You can send 20, 30, 40, 80, 90, 100, since 
   they can't be mis-understood.
   Complex, doesn't waste bits, artificial constraints, no adaptation to
   congestion/etc until ACK.

OK, I'll need to give this a bit more thought.


We're also considering having a "configuration" packet to be sent at the
beginning of any stream and that includes even more mode-specific data to
increase flexibility. However, we haven't found a good way of doing that
yet (wrt loss of the configuration packet). Any thought on that?


Yes: you can't assume 0-loss.  Also, if possible, the bitstream should be
decodable without the out-of-band channel information.  Video people have
struggled with this with sprop-parameter-sets in H.264 (RFC 3984).
Downsize will be bandwidth used.  You can amortize the overhead by sending
the config packets only periodically, but you'll probably need to send
them reasonably often to allow mid-stream join (think conferences).
Yes, I'm aware of the loss problem and I'm not sure how to handle that. I'll have a look at the RFC you mention. 
I don't see a reason to make it different from other codecs considering
that CELT can handle about any ptime. I wouldn't mind doing it though if
there's a use case for it.


Then specify a value, and let CELT round up (or down) as needed.
It might be nice to show how this might interact with alternative payloads
that might want multiples of 10.

I guess the rounding would just be a bit more. WIll add an example.

Actually, we're not yet sure the reduced overhead is worth the trouble of
adding another layout. Any thought?


Probably not.  How big a saving is it?  This mandates (effectively) CBR.
Too bad there's no way in-stream to know when you've finished decoding
the channel (stop token or the like).  Then you don't need channel
sizes at all (though perhaps you might want them).

Well, the stop token would waste about the same space as the size value...


Well, we have the same fundamental issue everywhere. If we don't know what
frame size was selected, we cannot decode anything. So we always need the
answer. The only way I can see to go around that would be to use a
different payload type for every parameter combination, but I think that
would be ugly.


You really *need* to handle the media-before-answer case somehow, even if
it's separate payloads.
Hadn't realised it was an issue. Need to give more thought into this. Other suggestions welcome. 
That would be (I assume) really 1 byte per channel per frame, not 1 byte
per frame.


Well, technically you can increase the bit-rate by one byte for only one
channel...


I meant that the change rate is one byte per channel per frame (max).  This
implies the frame-size rate of change could be anywhere in the range of
plus/minus the number of channels (in bytes).

I assume the decoder would handle large frame gaps with corresponding large
changes in framesize
Basically, any channel of any frame can have a different size and change arbitrarily from one frame to the next. You can have one channel jumping from 64 to 128 kbps while the other goes from 96 to 48 kbps. That's why there's one byte per channel per frame used for the size. 

Cheers,

   Jean-Marc