Re: [ippm] [M-Lab-Steering] Testing TCP Throughput Capacity in Operator Networks

[Matt Mathis <matt.mathis at gmail.com>]

Barry,

NPAD uses essentially the same algorithm as your proposed test.
While it works quite well as a diagnostic, it doesn't work so well as
a metric, because the results can't be well calibrated or modeled.

I have been thinking about this problem for a very long time (It was
my goal when I was the founding co-chair of the IPPM BOF, at IETF 35).
Following my experiences developing NPAD (which was yet another
iteration on the problem) have come to the conclusion that part of the
problem is that we have been asking the wrong question all these
years.

TCP performance is to unstable and unpredictable to be a good metric
of anything.  A better metric would be a packet loss test at high data
rate (e.g. 80% of the link capacity), that asks a pass-fail question
(is the loss rate lower than X).  With the proper parameters the test
guarantees a lower bound on ideal TCP performance, and at the same
time supports an algebra:  if the separate sections of a path each
pass the metric, then you can predict if the concatenated sections do
as well.

For an SLA the most useful metric would actually be the statistics of
the test: what fraction of the time does it pass for what fraction of
the endpoints?

I will be at Hiroshima - will you?

Thanks,
--MM--
-------------------------------------------
Matt Mathis      http://staff.psc.edu/mathis
Work:412.268.3319   Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.



On Tue, Oct 6, 2009 at 6:49 PM, Stephen Stuart <sstuart at google.com> wrote:
> Hi, Barry. I saw your post on the IPPM list:
>
>    http://www.ietf.org/mail-archive/web/ippm/current/msg02176.html
>
> and I think it would be worth your while to have a look at the tools
> used in Measurement Lab - http://measurementlab.net/ - NPAD and NDT in
> particular. I'm cc'ing the rest of the steering committee so that they
> can see the pointer to your IPPM post and so that you can follow up
> with any questions regarding the platform, tools, test methodology,
> etc.
>
> Thanks,
> Stephen