Hi Roni, Here is my understanding regarding MCU switching of the video bitstream for aFrom avt-bounces at ietf.org Mon Jul 28 04:03:09 2008 Return-Path: <avt-bounces at ietf.org> X-Original-To: avt-archive at optimus.ietf.org Delivered-To: ietfarch-avt-archive at core3.amsl.com Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9DB223A6813; Mon, 28 Jul 2008 04:03:09 -0700 (PDT) X-Original-To: avt at core3.amsl.com Delivered-To: avt at core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BECCA3A6813 for <avt at core3.amsl.com>; Mon, 28 Jul 2008 04:03:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.449 X-Spam-Level: X-Spam-Status: No, score=-6.449 tagged_above=-999 required=5 tests=[AWL=0.149, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ih0yVx1kTWbQ for <avt at core3.amsl.com>; Mon, 28 Jul 2008 04:03:06 -0700 (PDT) Received: from mgw-mx03.nokia.com (smtp.nokia.com [192.100.122.230]) by core3.amsl.com (Postfix) with ESMTP id 1C1103A67E6 for <avt at ietf.org>; Mon, 28 Jul 2008 04:03:05 -0700 (PDT) Received: from esebh107.NOE.Nokia.com (esebh107.ntc.nokia.com [172.21.143.143]) by mgw-mx03.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id m6SB2q8n019489; Mon, 28 Jul 2008 14:03:12 +0300 Received: from vaebh103.NOE.Nokia.com ([10.160.244.24]) by esebh107.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 28 Jul 2008 14:02:52 +0300 Received: from vaebe101.NOE.Nokia.com ([10.160.244.11]) by vaebh103.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 28 Jul 2008 14:02:52 +0300 x-mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Mon, 28 Jul 2008 14:02:50 +0300 Message-ID: <44C96BEE548AC8429828A37623150347D240BE at vaebe101.NOE.Nokia.com> In-Reply-To: <144ED8561CE90C41A3E5908EDECE315C05CBBEE9 at IsrExch01.israel.polycom.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [AVT] SDP offer/answer for the 3984bis and SVC payload drafts Thread-Index: AcjrUeMxZJ0ODZVcSEeC9mcryNrq0gAdxkuAATR/SiAReferences: <686BCEA3-5128-4CAB-B712-0E16E3895C50 at vidyo.com><44C96BEE548AC8429828A37623150347C47F61 at vaebe101.NOE.Nokia.com><ybur6a0z8tv.fsf at jesup.eng.wgate.com><44C96BEE548AC8429828A37623150347C47FAD at vaebe101.NOE.Nokia.com><ybur69zog7f.fsf at jesup.eng.wgate.com><44C96BEE548AC8429828A37623150347C47FBA at vaebe101.NOE.Nokia.com><ybumykbl1ne.fsf at jesup.eng.wgate.com><144ED8561CE90C41A3E5908EDECE315C05CBBE6B at IsrExch01.israel.polycom.com> <ybuiquzkwpk.fsf at jesup.eng.wgate.com> <144ED8561CE90C41A3E5908EDECE315C05CBBEE9 at IsrExch01.israel.polycom.com> From: <Ye-Kui.Wang at nokia.com> To: <roni.even at polycom.co.il>, <rjesup at wgate.com> X-OriginalArrivalTime: 28 Jul 2008 11:02:52.0180 (UTC) FILETIME=[7ADD8D40:01C8F0A1] X-Nokia-AV: Clean Cc: avt at ietf.org Subject: Re: [AVT] SDP offer/answer for the 3984bis and SVC payload drafts X-BeenThere: avt at ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Audio/Video Transport Working Group <avt.ietf.org> List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/avt>, <mailto:avt-request at ietf.org?subject=unsubscribe> List-Archive: <http://www.ietf.org/pipermail/avt> List-Post: <mailto:avt at ietf.org> List-Help: <mailto:avt-request at ietf.org?subject=help> List-Subscribe: <https://www.ietf.org/mailman/listinfo/avt>, <mailto:avt-request at ietf.org?subject=subscribe> Content-Type: multipart/mixed; boundary="==============72737984==" Sender: avt-bounces at ietf.org Errors-To: avt-bounces at ietf.org This is a multi-part message in MIME format.
Hi Roni,
Here is my understanding regarding MCU switching
of the video bitstream for a receiver between different sources. This may well
have been discussed earlier - please forgive me it that is the case. Let's use
Fig. 7 in RFC 5117 for example.
Shortcut name: Topo-Video-switch-MCU
+---+ +------------+ +---+
| A |------| Multipoint |------| B |
+---+ | Control | +---+
| Unit |
+---+ | (MCU) | +---+
| C |------| |------| D |
+---+ +------------+ +---+
Figure 7 - Point to Multipoint Using a Video Switceo Switching MCU
Assume that each endpoint may receive the video bitstream from one of any other endpoints, switched by the MCU, and out-of-band transmission of parameter sets is used.
In the negotiation process, the MCU carries out separate SDP offer/answer processes with A, B, C and D. Even though there is no source specific informtation in SDP messages during the SDP offer/answer processes, the MCU should still be able to know from whom it has received and SDP (I guess SIP, e.g example, should provide this funcationality). Therefore, the MCU stores the different copies of SDP, including parameter sets, from different endpoints and marks them with the assoicated endpoints.
The answer to the following question decides whether it is possible to do out-out-band transmission of parameter sets (without caring of ID value conflict across endpoints) with the above topology: Is it possible for one endpoint (e.g. endpoint B) to know which other endpoint (e.g. endpoint A) an SDP parameter (e.g. sprop-parameter-sets) is from, per existing standards? For example, if the endpoints use different payload types, then the answer is yes. Is there any other means (e.g. through SIP) to know this (with existing standards)?
If the answer is yes to the first question in the above paragraph, then each endpoint stores all different copies of parameter sets associated with different endpoint points. For example, endpoint A stores all copies of parameter sets associated with A, B, C and D, respectively. The set of parameter sets assoicated with A is used for encoding by endpoint A. The set of parameter sets assoicated with B is used to decode incoming bitstreams orignated from B. As source switching can only happen at IDR picture positions, the receiver implementation of A can simply feed all the parameter sets assoicated with B to the decocer of A at the first access unit a switching from any other endpoint to endpoint B occurs.
At least from the video coding standard point of view, I see no problem with the above process. Please comment what is the answer to the important question above.
If the answer is clearly no, then we need to find out another way to improve this aspect in RFC 3984.
BR, YK
>-----Original Message-----
>From: ext Even, Roni [mailto:roni.even at polycom.co.il]
>Sent:
Tuesday, July 22, 2008 10:08 AM
>To: Randell Jesup
>Cc: Wang Ye-Kui
(Nokia-NRC/Tampere); avt at ietf.org
>Subject: RE: [AVT] SDP offer/answer for
the 3984bis and SVC
>payload drafts
>
>Randell,
>The
problem is that the endpoint has signaling connection only
>to the MCU,
and if he has to forward the sprop parameter sets
>in the signaling there
is no relation to the SSRC.
>Example, the MCU establish connections with
three EPs and get
>their sprop parameter sets in the offer, all of them
use same
>numbers. Now what does it send back in the answer (Change
the
>numbers? Add relation to SSRC? All this is not defined).
Does
>it mean that before doing video switch the MCU needs to do
a
>re-invite to send new parameter
sets.
>
>Roni
>
>> -----Original
Message-----
>> From: Randell Jesup [mailto:rjesup at wgate.com]
>> Sent:
Monday, July 21, 2008 7:51 PM
>> To: Even, Roni
>> Cc:
Ye-Kui.Wang at nokia.com; avt at ietf.org
>> Subject: Re: [AVT] SDP
offer/answer for the 3984bis and SVC payload
>>
drafts
>>
>> "Even, Roni" hing MCU
Assume that each endpoint may receive the video bitstream from one of any other endpoints, switched by the MCU, and out-of-band transmission of parameter sets is used.
In the negotiation process, the MCU carries out separate SDP offer/answer processes with A, B, C and D. Even though there is no source specific informtation in SDP messages during the SDP offer/answer processes, the MCU should still be able to know from whom it has received and SDP (I guess SIP, e.g example, should provide this funcationality). Therefore, the MCU stores the different copies of SDP, including parameter sets, from different endpoints and marks them with the assoicated endpoints.
The answer to the following question decides whether it is possible to do out-out-band transmission of parameter sets (without caring of ID value conflict across endpoints) with the above topology: Is it possible for one endpoint (e.g. endpoint B) to know which other endpoint (e.g. endpoint A) an SDP parameter (e.g. sprop-parameter-sets) is from, per existing standards? For example, if the endpoints use different payload types, then the answer is yes. Is there any other means (e.g. through SIP) to know this (with existing standards)?
If the answer is yes to the first question in the above paragraph, then each endpoint stores all different copies of parameter sets associated with different endpoint points. For example, endpoint A stores all copies of parameter sets associated with A, B, C and D, respectively. The set of parameter sets assoicated with A is used for encoding by endpoint A. The set of parameter sets assoicated with B is used to decode incoming bitstreams orignated from B. As source switching can only happen at IDR picture positions, the receiver implementation of A can simply feed all the parameter sets assoicated with B to the decocer of A at the first access unit a switching from any other endpoint to endpoint B occurs.
At least from the video coding standard point of view, I see no problem with the above process. Please comment what is the answer to the important question above.
If the answer is clearly no, then we need to find out another way to improve this aspect in RFC 3984.
BR, YK
>-----Original Message-----
>From: ext Even, Roni [mailto:roni.even at polycom.co.il]
>Sent:
Tuesday, July 22, 2008 10:08 AM
>To: Randell Jesup
>Cc: Wang Ye-Kui
(Nokia-NRC/Tampere); avt at ietf.org
>Subject: RE: [AVT] SDP offer/answer for
the 3984bis and SVC
>payload drafts
>
>Randell,
>The
problem is that the endpoint has signaling connection only
>to the MCU,
and if he has to forward the sprop parameter sets
>in the signaling there
is no relation to the SSRC.
>Example, the MCU establish connections with
three EPs and get
>their sprop parameter sets in the offer, all of them
use same
>numbers. Now what does it send back in the answer (Change
the
>numbers? Add relation to SSRC? All this is not defined).
Does
>it mean that before doing video switch the MCU needs to do
a
>re-invite to send new parameter
sets.
>
>Roni
>
>> -----Original
Message-----
>> From: Randell Jesup [mailto:rjesup at wgate.com]
>> Sent:
Monday, July 21, 2008 7:51 PM
>> To: Even, Roni
>> Cc:
Ye-Kui.Wang at nokia.com; avt at ietf.org
>> Subject: Re: [AVT] SDP
offer/answer for the 3984bis and SVC payload
>>
drafts
>>
>> "Even, Roni" <roni<roni.even at polycom.co.il>
writes:
>> >One of the issues with the parameter set has to do with
multipoint
>> voice
>> >activated video switch. In this
scenario the MCU switch the video
>> >source. Since the switch may
occur when the receivers do not have a
>> >synchronization point
(Both IDR picture and parameter sets) in H.323
>> we
>> >do
not send parameter sets out of band and require that the fast
>>
update
>> >will cause the sending of the IDR picture and the
parameter sets.
>> >Currently in my view RFC 3984 is broken in this
case. There is no
>> >requirement to send the parameter sets for
fast update.
>>
>> In this case, the MCU is acting as a video
multiplexer - forwarding
>but
>> not
>> modifying the
streams. Recipients should see the SSRC (and
>>
source/SDES)
>> change on a switch by the MCU. If the new source
can have
>conflicting
>> parameter sets in what it sends, then
the recipient MUST NOT try to
>> decode the new video source until it
has the correct paramsets
>> installed.
>> This
>>
may be why they suggest avoiding potential conflicting
paramsets.
>>
>> Note that in-band doesn't help here (since
any in-band packet can be
>> lost, even with FEC and retransmission,
etc) unless the
>decoder throws
>> away all parameter sets on any
source change, and goes black/static
>> until it gets a set of
paramsets and an IDR. If the paramsets are
>> *known* to be
compatible (which is probably the point of
>> paramater-add=0), then it
can start decoding immediately (if it
>> doesn't mind odd
"predator-mode" results), and any IDR (without
>> paramsets) or other
form of refresh will fix the video.
>>
>> >Currently all
conference participants may use the same
>parameter sets
>>
>numbers which will make it difficult on the receiver to
keep
>parameter
>> >sets with the same numbers without knowing
how to correlate
>them to a
>> >specific source, this can be
addressed by having the MCU assign the
>> >parameter sets numbers or
by relating the parameter sets in the SDP
>to
>> the
>>
>SSRC or doing like H.241 requiring the sending of parameter
>sets
with
>> IDR
>> >on fast update (In this case video
conferencing will not need the
>> sprop
>>
>support)
>>
>> If we define lack of parameter sets to mean
in-band is required for
>> both sides (and live with possible loss),
then this use
>case/topology
>> could use that (by having the
MCU/multiplexer use SDP's without
>> sprop-parameter-sets). In
the case of a "dumb" multiplexer, which
>> isn't modifying or
re-encoding the streams, it's hard to support
>> anything else, unless
the MCU insists on a parameter-set (via
>> parameter-add=0, and using
re-INVITE if the initial invite came from
>> the
endpoint).
>>
>> What about other topologies? My
apologies; I didn't really
>follow the
>> topology draft during
the discussion of CCM.
>>
>> --
>> Randell Jesup,
Worldgate (developers of the Ojo videophone),
>ex-Amiga
>> OS
team rjesup at wgate.com "The fetters imposed on liberty at
>home
have
>> ever been forged out of the weapons provided for
defence
>against real,
>> pretended, or imaginary
dangers
>from
>> abroad."
>>
&.even at polycom.co.il>
writes:
>> >One of the issues with the parameter set has to do with
multipoint
>> voice
>> >activated video switch. In this
scenario the MCU switch the video
>> >source. Since the switch may
occur when the receivers do not have a
>> >synchronization point
(Both IDR picture and parameter sets) in H.323
>> we
>> >do
not send parameter sets out of band and require that the fast
>>
update
>> >will cause the sending of the IDR picture and the
parameter sets.
>> >Currently in my view RFC 3984 is broken in this
case. There is no
>> >requirement to send the parameter sets for
fast update.
>>
>> In this case, the MCU is acting as a video
multiplexer - forwarding
>but
>> not
>> modifying the
streams. Recipients should see the SSRC (and
>>
source/SDES)
>> change on a switch by the MCU. If the new source
can have
>conflicting
>> parameter sets in what it sends, then
the recipient MUST NOT try to
>> decode the new video source until it
has the correct paramsets
>> installed.
>> This
>>
may be why they suggest avoiding potential conflicting
paramsets.
>>
>> Note that in-band doesn't help here (since
any in-band packet can be
>> lost, even with FEC and retransmission,
etc) unless the
>decoder throws
>> away all parameter sets on any
source change, and goes black/static
>> until it gets a set of
paramsets and an IDR. If the paramsets are
>> *known* to be
compatible (which is probably the point of
>> paramater-add=0), then it
can start decoding immediately (if it
>> doesn't mind odd
"predator-mode" results), and any IDR (without
>> paramsets) or other
form of refresh will fix the video.
>>
>> >Currently all
conference participants may use the same
>parameter sets
>>
>numbers which will make it difficult on the receiver to
keep
>parameter
>> >sets with the same numbers without knowing
how to correlate
>them to a
>> >specific source, this can be
addressed by having the MCU assign the
>> >parameter sets numbers or
by relating the parameter sets in the SDP
>to
>> the
>>
>SSRC or doing like H.241 requiring the sending of parameter
>sets
with
>> IDR
>> >on fast update (In this case video
conferencing will not need the
>> sprop
>>
>support)
>>
>> If we define lack of parameter sets to mean
in-band is required for
>> both sides (and live with possible loss),
then this use
>case/topology
>> could use that (by having the
MCU/multiplexer use SDP's without
>> sprop-parameter-sets). In
the case of a "dumb" multiplexer, which
>> isn't modifying or
re-encoding the streams, it's hard to support
>> anything else, unless
the MCU insists on a parameter-set (via
>> parameter-add=0, and using
re-INVITE if the initial invite came from
>> the
endpoint).
>>
>> What about other topologies? My
apologies; I didn't really
>follow the
>> topology draft during
the discussion of CCM.
>>
>> --
>> Randell Jesup,
Worldgate (developers of the Ojo videophone),
>ex-Amiga
>> OS
team rjesup at wgate.com "The fetters imposed on liberty at
>home
have
>> ever been forged out of the weapons provided for
defence
>against real,
>> pretended, or imaginary
dangers
>from
>> abroad."
>>
&nbnbsp; - James Madison, 4th US president
(1751-1836)
>
>
_______________________________________________ Audio/Video Transport Working Group avt at ietf.org https://www.ietf.org/mailman/listinfo/avt