[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[ippm] Comments on draft-ietf-ippm-framework-compagg-04.txt
Al
The draft has some good
definitions of metrics - however to make it a little more generic and hence
capable of wider application it may be useful to make some minor
changes.
(i) Composite Metric definition
- 1
There are many cases in the
industry where the term Composite Metric is used. One common case is where
a metric is constructed from several dissimilar metrics - for example a simple
case is the combined loss/discard rate in RFC3611, which is not the
sum of two packet loss rates but the sum of a packet loss rate with a
jitter buffer packet discard rate. There are many more complex cases in
common usage. The text at the present seems to be more oriented towards
composition of identical metrics - either in space or time.
Recommendation: add text to
state that composite metrics may be constructed from either identically defined
metrics or from dissimilar metrics.
(ii) An
"Index"
There are common cases when an
artifical index is created - often from a complex function of some
metrics. The value range of the index typically has no relationship to any
observable parameter - and is selected for convenience, the function use to
calculate the index is then determined. Obvious telecom examples are MOS
scores and R factors.
Recommendation:
Add
3.n Index
An Index is a composite metric
for which the output value range has been selected for convenience or clarity,
and the behavior of which is selected to support ease of understanding.
The composition function for an index is often developed after the index
range and index behavior have been determined. Examples include MOS scores
and R factors.
(iii) Models
and metrics
Many metrics are either
implicitly or explicitly the result of assuming a model of the process being
measured, and then estimating the parameters of the model. Simple
examples include "average packet loss rate", which seems to assume
independent loss probability, RFC3550's PPDV, which seems to assume alternately
early (or on-time) and late packets, and consecutive loss
metrics. IMHO it is better to explicitly state what model is being
used, as that brings out the assumptions being made and allows their
validity to be questioned. For example, one of the most common means
of reporting performance is to measure the average of some value during a time
interval - in many cases this is done simply because it is a common thing to do
and the process of analyzing the behavior of the parameter
being measured is skipped.
Models can also be very useful
in understanding system behavior and identifying particularly useful
metrics. I've used the term "intermediate model" to mean a model that is
(a) representative of some key aspect of performance for an/the application that
is using the service/ system being measured and (b) which is capable of
accepting as input data extracted from the service/ system being measured.
In some senses an intermediate model is a composition function - however the
function has been designed to be "tuned' for some class of applications.
One example of this is the 4-state Markov Model used in RFC3611, which captures
information on sparse bursts of lost or discarded packets, another is the MAPDV2
jitter metric, which is based on a simplified representation of jitter buffer
behavior.
Recommendation:
Add to Section 5. text to
indicate that the model (statistical) of the process being measured and the
validity of this assumption should be explicitly stated
Add;
Section 4.n Composition
Functions based on Intermediate Models
An Intermediate Model is a
model that is representative of some key aspect of performance for a class of
applications and which is capable of accepting
_______________________________________________
ippm mailing list
ippm at ietf.org
https://www1.ietf.org/mailman/listinfo/ippm