[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ippm] [CCAMP] [Fwd: IPPM expert review request]



Hi Reinhard,

I appreciate your response. I am glad that we resolved all other concerns :).
Please see one last comment below.


[RS8]
How to compare metrics that have been generated by two different Poisson
processes, i.e. e.g. two different arrival rates?
[sunwq]
No way. Do we need to do that anyway?

[RS8a]
Pls also see my comments above.
If you cannot compare two sets of measurements how will you then arrive at
an objective study of the network components under test?
[sunwq]
Maybe I misunderstood you again here. The Poisson arrival rate is an
approximation of real traffic load. Of course it is worthwhile to find out
how DUT/SUT performs when traffic load changes. But what else do you expect by comparing two data sets from different traffic load? Did I miss something

here?

[RS]
I was thinking along the lines of say starting with a reasonably large
lambda_m, get the metric sample for that and then subsequently e.g.
shortening lambda_m, comparing the resulting metric tuples. I was looking
for a guideline on how to possibly compare these resulting tuples.
As this is a stochastic process even shorter lambda_m values can still
generate longer interarrival times (surely with lower probability but still
possible). Makes sense?

Larger lambda_m represents higher arrival rate hence will result in shorter average inter-arrival delay. I guess what you want to say is that shorter lambda_m may generate shorter inter-arrival times. Yes it is true. But once we decided to get a ``sample", instead of a singleton value, then very likely we are interested in the system performance on average (and of course, in some other statistics of the sample), possibly averaged over a large number of singleton values. In this case each singleton value does not have much individual significance. This also can be explained by the fact the the processing of signaling messages on each NE are themselves random processes. When you generate the requests with constant inter-arrival time, you may still get singleton values that vary drastically. So it does not make sense to compare any two singleton values. However, once averaged over a large number of singleton values, the result is meaningful.

And I believe the same thing happens in IP performance measurement. For example, in RFC 2681, Poisson process is suggested but no guideline of how to compare two samples is provided. It think it makes sense that the documents provides methodologies on how to perform a measurement, and leaves the task of interpreting the results to the testers, or the owners of DUT/SUTs.

Hope this helps!

Best,
Weiqiang