Hi all,
Thanks to Henk and Al, the authors have now updated the LSP DPPM draft.
http://www.ietf.org/internet-drafts/draft-ietf-ccamp-lsp-dppm-02.txt
The two notable changes are:
1) We recovered the *sample definition* part which was removed from older versions.
We feel the existing IPPM framework (by defining a singleton metric followed by definition of samples of singleton values) is the most convenient and reasonable way for us to cover the bound issue raised by Henk and Al. The arrival parameters can also be placed comfortablely in this sectioin.
2) Two experts from Carriers joined us: Dr. Tomohiro Otani (KDDI) and Dr. Ruiquan Jing (CT).
Inputs from carriers has always been one major driving force for this work. The close involvement of carriers will possibly guides us to define more metrics, and accelerate the adoption of the defined metrics in testing and operational environments.
Below, I try to summarize the received comments and the changes made accordingly. Please let us know if you have further comments.
-Comments from Henk
3.4. Units.
I don't think this is quite correct: you start to measure at T1 and
wait for the setup to complete. However, it doesn't, so at some point
you become bored and stop the measurement. You report undefined.
However, one then has to report in the metrics parameters when the
measurement will be stopped. This value should be added to the
metrics parameters. (Note: this will apply to more metrics).
Response: Yes this has been addressed in the sample definition sections, section 8-12.
-------------------------------------
4.3. Metrics Parameters.
In IPPM speak, Lamba refers to a rate but it also specifies a distribution
for the interval between probe packets. For example, poisson or periodic.
The draft should mention which distribution to use.
Response：We mentioned the use of poisson process in sections like 4.5/4.6/4.7. For sample definition, we mentioned it before actual definition begins.
However, despite the poisson process, periodic process may also be used in the sampling process. The question is that do we need a new parameter for the choice of different arrival model? We would like to have your comments.
----------------------------
12.1. Statistics.
I think you better report the 5th or 10th percentile of the distributions,
rather than the minimum (and the same for the maximum). This excludes
data points where something went wrong in the measurement process.
Response: This section is now numbered 14 in the new version. We give a generalized definition of percentile in 14.3. Hope it works.
=====
-Comments from Al
As I've said in the past, this draft is an excellent start,
and because it adopts the metric template used in bmwg and ippm,
it's an easy read. Thanks for doing that part.
Thanks. It feels good to be native. : -)
----------------------------------------------
There is a taxonomy of possible outcomes for set-up protocols.
Each set-up attempt will have one of the following outcomes:
1. Success
2. Failure
3. Incorrect Set-up (wrong destination)
The same basic outcomes apply to release attempts.
In the current text, the set-up time metrics cover the
Successful outcome (they could be called Successful *.* Setup Time).
.....For more nit-picking background, read
http://tools.ietf.org/id/draft-morton-ippm-reporting-metrics-05.txt
So, I propose to add metrics to cover the Failure outcome,
defined as a count of failures=Undefined setup times divided by the
total attempts. ...
Adding metrics to cover the Incorrect setup/release outcome
would complete the taxonomy. ...
Response: Thanks for the detailed explanations and for pointing us to the draft. We totally agree. We have added a new section, section 13 to discuss this (below).
Any more comments on that?
13. Discussion for unsuccessful setup/release cases
As has been mentioned earlier, LSP setup/release may fail due to
various reasons. For example, setup/release may fail when the
control plane is overburdened or when there is resource shortage in
one of the intermediat nodes. Since the setup/release failure may
have significant impact on network operation, it is worthwhile to
report each failure cases, so that appropriate operations can be
performed to check the possible implementation,configuration or other
deficiency.
Although not commonly seen, an LSP setup/release attemp may be
falsely carried out. for example, the setup/release request may be
targed to a wrong egress node. Although faulty results may have
totally different implications to the control plane, if compared with
failure cases, for the purpose of performance evaluation, it is still
reasonable to treat such results as unsuccessful cases. Thus the
unsuccessful cases include both failure and incorrect cases.
Once a sample of a particular metric, e.g, single unidirectional LSP
setup delay, is obtained, we can deduce the unsuccessful cases by
sorting out from the sample the <time, delay> pairs with undefined
delay value.
---------------------------
3.3 Metric Parameters
o ID0, the ingress LSR ID
o ID1, the egress LSR ID
You should use these ID#s in the definition of the metric.
o T, a time
I've found that the two word description of a parameter is
not always sufficient for me. I suggest:
o T, a time when the setup is attempted
I suggest to add a parameter, for clarity:
o WAITTIME, the maximum waiting time for successful setup
and mention this parameter in the sections where discussed,
like 3.5 and 3.6.
Response: These have been addressed.
---------------------------------------------
13. Discussion
The control-plane-only nature of these metrics should be clear(er)
in the Introduction/Scope.
Response: We change the introduction part into:
This draft defines performance metrics and methodologies that can be
used to depict the dynamic LSP provisioning performance of GMPLS
networks, **more specifically, performance of the signaling protocol**.
Hope it is clearer.
--------------------------------------------------------------------
How important is it to have data-plane verification of the
apparent control-plane success/failure/incorrect performance?
Response: Yes, it is important. But as the introduction of the verification may involve complex and totally different methodologies, we'd rather address it in (future) separate documents. Anyhow, the performance of the signaling itself is still an important metric to have. And depending on implementations, the overall time of an successful setup/release can sometimes be deduced from (or approximated by) this metric.
----------------------------------
Our question:
With the addition of sample definition, do we still need the multiple LSP setup delay metric(in Section 4,6) and Samples of Multiple LSPs Setup Delay(in section 9,11)? We'd like to have your comments.
Best regards,
Authors
_______________________________________________ ippm mailing list ippm at ietf.org https://www.ietf.org/mailman/listinfo/ippm