Re: [ippm] draft-duffield-ippm-burst-loss-metrics-01

["DUFFIELD, NICHOLAS G (NICK)" <duffield at research.att.com>]

Yaakov,

> -----Original Message-----
> From: ippm-bounces at ietf.org [mailto:ippm-bounces at ietf.org] On Behalf 
> Of Yaakov Stein
> Sent: Sunday, November 01, 2009 9:36 AM
> To: IETF IPPM WG
> Subject: [ippm] draft-duffield-ippm-burst-loss-metrics-01
> 
> I read the aforementioned draft with interest, as I believe that it is 
> important to have a way of quantifying the burstiness of packet loss.
> 
> I must admit to having read it too quickly, as I am traveling and 
> wanted to get some comments in before the upcoming meeting.

Thanks for finding the time to review the draft before the upcoming IETF meeting. Please see responses inline below:

> 
> 
> 
> While I believe that the metrics defined are a good first step, I must 
> admit to feeling unsatisfied with them.
> 
> I simply am not sure that the concept of burstiness has been captured 
> sufficiently well.
> 
> 
> 
> Yes, the duration of a packet burst and the frequency of bursts are 
> nice to have,
> 
> but first we need to quantify how bursty the loss is - if it is not 
> bursty then these two metrics are meaningless.

We are not trying to reproduce some other definitions of burstiness, e.g., those based on moments (or cumulants), such as the index of dispersion of counts. 

The metrics of duration and frequency of loss episodes directly characterize the (loss) experience of traffic flows, and can be related to application requirements and SLAs. They are transparent, easy to implement, do not depend on a reference model, and are meaningful whether or not the underlying loss patterns are bursty. 

(As an aside, one could formulate a test for the presence or absence of burstiness based on the metric values, according to whether they are together consistent with Poissonian loss).

> 
> For this purpose the draft proposes loss-pair-counts and 
> bi-packet-loss- ratio,

The "end products" of this draft are the metrics in Section 6:

1. Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio (the packet loss rate) 

2. Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration (the loss episode duration) 

3. Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency (the frequency at which loss episode start)

> 
> but loss-pair-counts are too raw
> 

For exposition, we organized some of the intermediate quantities as what we termed proto-metrics in Section 5, in order to avoid repeated definitions in the end product metrics of Section 6. But these could be rolled up together in implementations. We can add a statement that the proto-metrics themselves need not be reported.

> and bi-packet-loss-ratio doesn't seem to
> describe the right thing.

Bi-packet-loss-ratio is the average loss as reported using the metric methodology. Although it is not the central new metric of this effort (it doesn't inform on burstiness) we believe it is still useful to have the ability to report average loss in the burst loss measurement framework, since it makes the burst-loss metrics standalone for the measurement of average loss.

> 
> 
> Another minor gripe I have with the methodology is the introduction of
> time.
> 
> I think of burstiness in terms of packet (transmit) sequence number.
> 
> Is it really necessary to introduce a time scale ? It seems to complicate
> things.
> 

If people want to know the temporal duration and frequency of loss episodes, then time is useful.

> 
> 
> What did I expect as a metric ? Well, I have become used to the use of
> Gilbert-Elliott models,
> 
> where there are two states - loss and non-loss, with easily measured
> probabilities of transitions

As mentioned above, the metrics here are model independent. On the other hand, the metric values that would be obtained if applied to a Gilbert-Elliot loss process can be related to the parameters of the model. We can detail this in the draft if that would be of interest.

> 
> between the states, and probabilities of loss or not in either state.
> These probabilities seem to
> 
> capture well the subjective idea of burstiness, but it takes a while to
> get used to them.
> 
> 
> 
> Intuitively, after defining the probability of loss, we need to describe
> the higher moments or cumulants.
> 
> So a simple derived metric would be the probability of consecutive packets
> being lost divided by the PLR^2,

The metrics in the draft in fact work directly with the probabilities of the loss or transmission for two consecutive packets. In particular, the loss-pair-counts, to which you earlier referred, are, when divided by the number of probe pairs, the measured probabilities of these pair events.

> 
> the probability of three consecutive packets lost divided by PLR^3, etc.

In principle, higher order moments could be used to obtain a more detailed statistical picture of the burstiness. However, higher order moments are subject to greater statistical estimation error, and are in any case not needed to answer the basic question as we see it: how frequent and how long are loss episodes? This last point also applies to the "distant packets" matter following.

> 
> As a complement one could do the probability of two randomly chosen
> distant packets both being lost
> 
> divided by PLR^2, etc.
> 
> 
> 
> Can someone map the metrics described in the draft to these ideas ?
> 

In summary, we can detail the connection to specific models in future versions on the draft.


> 
> 
> Y(J)S
> 
> 

Nick