RE: [AVT] Suggestion for how to handlederived/computedmetricindraft-avt-rtcpxr-*[Scanned]

["Ravi Raviraj" <ravi.raviraj at psytechnics.com>]

Roni,
When the EP reports MOS values, it should identify exactly which
standard it used to generate the MOS value and any variation within it.
In this way there won't be any confusion.
As in any other standard, when an EP implementer claims compliance to a
specific standard, they will(should) know what they are talking about.

My draft
https://datatracker.ietf.org/drafts/draft-raviraj-sipping-endpoint-mos/
shows the number of variations one could have with perceptual MOS. I
will bring a draft next time to identify the standards we should
identify. 

Ravi.


-----Original Message-----
From: Even, Roni [mailto:roni.even at polycom.co.il] 
Sent: 26 July 2007 21:03
To: Alan Clark; David R Oran; AVT WG
Subject: RE: [AVT] Suggestion for how to
handlederived/computedmetricindraft-avt-rtcpxr-*[Scanned]

Alan,
This will be helpful.
Still my concern is that EP implementers will not know what to calculate
since their product may use different types of codecs.
Roni

> -----Original Message-----
> From: Alan Clark [mailto:alan.d.clark at telchemy.com]
> Sent: Thursday, July 26, 2007 10:59 PM
> To: Even, Roni; 'David R Oran'; 'AVT WG'
> Subject: RE: [AVT] Suggestion for how to handle
> derived/computedmetricindraft-avt-rtcpxr-*
> 
> 
> Roni
> 
> Certainly there is a difference between voice and audio codecs. In the
> case
> of voice codecs (both narrowband and wideband) the methods for
objectively
> measuring the quality of the call and for taking codec characteristics
> into
> account is well understood - there are admittedly several approaches
> however
> there are existing standards that address means for calculating
objective
> estimates of MOS and even for conformance testing of algorithms that
do
> this.
> 
> The approach used for voice quality is fairly straightforward.  There
are
> several parameters that are defined on a codec specific basis -
notably Ie
> and Bpl, where Ie identifies the quality degradation due to the
encoder
> and
> Bpl the robustness of the PLC algorithm against packet loss/ discard.
The
> endpoint incorporates a small table of these values for each voice
codec
> that it supports.  Based on the codec selected for the call - the
> appropriate parameters are used.  Some other parameters, for example
> signal
> level, noise level, echo, delay, are less codec dependant.  A good
source
> of
> information on how to do this using G.107 is ITU-T Recommendation
G.799.1,
> which is a VoIP trunking gateway standard that incorporates a clear
> definition of how to compute the R factors and MOS scores required by
RTCP
> XR.
> 
> Methods for measuring audio quality are less heavily standardized at
this
> time, although there are ITU standards such as G.1070 (for IP
> videoconferencing) that incorporate voice, video and multimedia
quality
> metrics and ITU-R BS-1387 (which is a PESQ-like algorithm for audio).
> There
> is ongoing work within the industry related to IPTV audio stream
> measurement
> that is occurring in at least three different standards organizations.
> 
> I'd be happy to put together an informational document that outlines
> existing approaches to objective voice/ audio/ video performance
> measurement
> and reviews some of the current industry activities - if that would be
> useful?
> 
> Regards
> 
> Alan
> 
> 
> 
> 
> 
> -----Original Message-----
> From: Even, Roni [mailto:roni.even at polycom.co.il]
> Sent: Thursday, July 26, 2007 3:30 PM
> To: Alan Clark; David R Oran; AVT WG
> Subject: RE: [AVT] Suggestion for how to handle
> derived/computedmetricindraft-avt-rtcpxr-*
> 
> Alan,
> What bothers me is that in my view many people building audio and
video
> EPs
> will use codecs that come from the DSP manufacturer or from a open
source
> library. It will be difficult to them to choose the right metrics. I
am
> not
> the biggest expert on quality assessments but I think that there is a
> difference between the current voice codecs (3, 7Khz) and the new
audio
> ones
> (14 and 20Khz) which are now used in video conferencing and are coming
to
> VOIP applications in term of the quality assessment.
> 
> Now how do you expect a non-educated EP developer to know what is the
> right
> way to calculate the quality. How will he calculate quality if his
system
> may use G.711 in one call and AAC-LD in a second call. It needs either
to
> be
> very easy to understand or just ask him for raw data.
> 
> Roni
> 
> > -----Original Message-----
> > From: Alan Clark [mailto:alan.d.clark at telchemy.com]
> > Sent: Tuesday, July 24, 2007 1:08 AM
> > To: 'David R Oran'; 'AVT WG'
> > Subject: RE: [AVT] Suggestion for how to handle derived/computed
> > metricindraft-avt-rtcpxr-*
> >
> > Dave
> >
> > I agree with the general tenor of your thoughts.  I also agree that
we
> > should not mandate that endpoints have to implement any particular
set
> of
> > derived metrics - these should be optional and should be implemented
> based
> > on their usefulness.
> >
> > For cases in which there is a fairly well defined metric class (e.g.
> MOS
> > for
> > speech) then I suggest that we do define a separate draft for that
> with a
> > registry of algorithms that could be (optionally) used - it is
easier
> to
> > update a registry with information that a new version of ITU-T G.xxx
> is
> > available than it would be to generate a new draft.
> >
> > Regards
> >
> > Alan
> >
> >
> > -----Original Message-----
> > From: David R Oran [mailto:oran at cisco.com]
> > Sent: Monday, July 23, 2007 5:51 PM
> > To: AVT WG
> > Subject: [AVT] Suggestion for how to handle derived/computed metric
> > indraft-avt-rtcpxr-*
> >
> > At the Mic during the AVT session I sketched a vague idea for how to
> > resolve the conflict between the desire on the part of many folks to
> > only
> carry
> > raw/primitive instrumented measurements in RTCP-XR, and other
equally
> > vocal, who argued for carrying the derived/computed metrics as well.
> >
> > I won't repeat the pro/con arguments here. Let's assume we want to
> carry
> > both.
> >
> > The chairs asked me to write up my vague thoughts as the beginnings
of
> a
> > concrete proposal.
> >
> > Some particular technical difficulties with the derived metrics that
> makes
> > them different from the primitive measurements is:
> > - there are competing standards for how to compute them
> > - the results are not generally aggregatable, comparable, or self-
> > describing.
> > - the set of interesting derived metrics is not small and seems to
be
> > growing, so it will be difficult to nail these down for stable RTCP
> > reporting.
> >
> > This makes it problematic to mix primitive and derived metrics in
the
> same
> > metric blocks in RTCP.
> >
> > So, my suggestion is to do the following.
> >
> > - A standards body who defines one or more consistent derived
metrics
> may
> > create (by bringing a draft to IETF) a separate metric block for
those
> > metrics.
> > - The block and its contained derived/computed metrics would be
> completely
> > self-describing in terms of exactly what version of what standard
> defines
> > the block.
> > - The definition of the metric would explicit contain the input
> variables,
> > in order to ensure that those inputs are separately available as
> primitive
> > metrics in the RTCP-XR. This increases the probability that any
metric
> > that is computable by the endpoint is also computable by a central
> > element receiving and processing RTCP-XR reports.
> > - If there are competing/overlapping blocks, an endpoint need not do
> all
> > of
> > them and there would be no confusion about exactly which (e.g."video
> MOS")
> > calculation was done.
> >
> > As an aside, I would rather not see further updates to the existing
> drafts
> > until we resolve this issue, since this is in fact major surgery to
> the
> > documents that is best done before polishing the text and individual
> > metric definitions.
> >
> > Dave Oran.
> >
> > _______________________________________________
> > Audio/Video Transport Working Group
> > avt at ietf.org
> > https://www1.ietf.org/mailman/listinfo/avt
> >
> >
> > _______________________________________________
> > Audio/Video Transport Working Group
> > avt at ietf.org
> > https://www1.ietf.org/mailman/listinfo/avt
> 
> _______________________________________________
> Audio/Video Transport Working Group
> avt at ietf.org
> https://www1.ietf.org/mailman/listinfo/avt


_______________________________________________
Audio/Video Transport Working Group
avt at ietf.org
https://www1.ietf.org/mailman/listinfo/avt

This email and attachment are confidential. If you have received this message in error, please notify the sender immediately and delete the message from your computer. Psytechnics Ltd has taken steps to ensure that this email is virus free but the recipient should take necessary steps to make certain.

Company Name : Psytechnics Ltd
Registered Number : 4110724
Registered Office : Fraser House, 23 Museum St, Ipswich, Suffolk, IP1 1HN, UK
Registration : England and Wales


_______________________________________________
Audio/Video Transport Working Group
avt at ietf.org
https://www1.ietf.org/mailman/listinfo/avt