Re: [ippm] Comments on draft-ietf-ippm-framework-compagg-04.txt [2]

[Al Morton <acmorton at att.com>]

Hi Alan,

Thanks for your comments on the Framework draft.

I agree with most of your proposals, but the very last one
stretches the scope a bit too far, I think.  Others may want to
comment, too.  See below.

Al

At 12:38 PM 7/28/2007, Alan Clark wrote:
> Outlook decided to
> send the email before I'd finished it!!

That's why it's called MS LookOut in some circles... :-)

Points (i) and (ii) seemed reasonable to me.

> ...
> (iii) Models and metrics
> Many metrics are either implicitly or explicitly the result of assuming a
> model of the process being measured, and then estimating the parameters
> of the model.  Simple examples include "average packet loss
> rate", which seems to assume independent loss probability, RFC3550's
> PPDV, which seems to assume alternately early (or on-time) and late
> packets, and consecutive loss metrics.  IMHO it is better to
> explicitly state what model is being used, as that brings out the
> assumptions being made and allows their validity to be questioned. 
> For example, one of the most common means of reporting performance is to
> measure the average of some value during a time interval - in many cases
> this is done simply because it is a common thing to do and the process of
> analyzing the behavior of the parameter being measured is
> skipped.   

I agree, and our framework RFC 2330, section 12 has some relevant
guidance, too.

>  
> Models can also be very useful in understanding system
> behavior and identifying particularly useful metrics.  I've used the
> term "intermediate model" to mean a model that is (a)
> representative of some key aspect of performance for an/the application
> that is using the service/ system being measured and (b) which is capable
> of accepting as input data extracted from the service/ system being
> measured.  In some senses an intermediate model is a composition
> function - however the function has been designed to be "tuned' for
> some class of applications.  One example of this is the 4-state
> Markov Model used in RFC3611, which captures information on sparse bursts
> of lost or discarded packets, another is the MAPDV2 jitter metric, which
> is based on a simplified representation of jitter buffer
> behavior.

Hmmm.  If I understand correctly, this is where one or more
complete
path metrics (in the terms of framework draft) would be combined to
derive
another metric that can be used to estimate performance at some
higher
layer.  

>  
> Recommendation:
>  
> Add to Section 5. text to indicate that the model
> (statistical) of the process being measured and the validity of this
> assumption should be explicitly stated

OK

>  
> Add;
>  
> Section 4.n Composition Functions based on Intermediate
> Models
> An Intermediate Model is a composite function that is representative of
> some key aspect of performance for a class of applications and which is
> capable of accepting as input metrics from the system being
> measured.  
>  
This seems different from the rest of the
metrics envisioned
in this framework, where we compose an estimate of the "whole"
using 
measurements of the "parts" (e.g., sub-path metrics are
combined to 
estimate the complete path metric in Spatial Composition).

How do other folks feel about this?  
Are Intermediate Models in the scope of the Composition/Aggregation
Framework?





_______________________________________________
ippm mailing list
ippm at ietf.org 
https://www1.ietf.org/mailman/listinfo/ippm