[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [ippm] [CCAMP] [Fwd: IPPM expert review request]
Hi Reinhard,
Thanks for responding. Please see inline.
Thanks for sending this part. See below for my responses.
[RS7]
Is there any help or suggestion on how to actually compute a usable
Lambda_m?
[sunwq]
No. Depending on the performance of the implementation, the lambda_m can
be
very different, even under the same topology. A proper value of Lambda_m
can
only be tuned in the testing.
[RS7a]
I am a bit at loss here.
It was my understanding that you use a metric to measure certain network
quantities, e.g. throughput, set up delays, etc.
In a second step you'd use these measured metrics to determine whether the
systems under test meet your previously expected or defined quality
parameters. If not, you'd make changes to the components or network
design,
measure again, and compare the previous set of data to the one after the
change and see whether you have improved and are moving into the right
direction.
You seem to be coming from the other end, i.e. tweak the input parameters
to
suit your system under test.
[sunwq]
OK, I get your point here.
What you said is true. I had thought by saying ``a usable lambda_m", you are
saying that a lambda_m that is *best achievable* by the DUT/SUT.
If you are expecting certain performance from the DUT/SUT, then very likely
you will have in mind an "expected" lambda_m, won't you?
The methodology here does not suggest how to select lambda_m. This in fact
can provide flexibility. Either the test will use an expected lambda_m, or
tune the lambda_m that reflects the best system performance, depending on
the purpose of the test. Does this make sense?
[RS8]
How to compare metrics that have been generated by two different Poisson
processes, i.e. e.g. two different arrival rates?
[sunwq]
No way. Do we need to do that anyway?
[RS8a]
Pls also see my comments above.
If you cannot compare two sets of measurements how will you then arrive at
an objective study of the network components under test?
[sunwq]
Maybe I misunderstood you again here. The Poisson arrival rate is an
approximation of real traffic load. Of course it is worthwhile to find out
how DUT/SUT performs when traffic load changes. But what else do you expect
by comparing two data sets from different traffic load? Did I miss something
here?
[RS9]
These sample tuples are dependent on the arrival rate Lambda_m. How to
deduct this input parameter from a resulting sample?
For instance, if two result sample sets are given, how can we be sure that
they can be compared?
[sunwq]
My understanding of the sample is that it is a sequence of singleton
values
obtained under a set of parameters. The parameters, such as Lambda_m, are
not visible in the sample. However, when reporting the sample, the
parameters must also be reported.
[RS9a]
OK. Noted and understood.
[sunwq]
Great, thanks.
[RS10]
A superposition of two different Poisson processes is being used here.
How do you map the resulting RESV messages to the correct process, i.e.
how
to differentiate e.g. a late RESV message resulting from third PATH
message
in fourth invocation from an early RESV message solicited by first PATH
message in fourth invocation?
[sunwq]
RESV messages can be differentiated by either MSG ID or non-overlapping
LSP
ID, or combined. The RSVP-TE protocol does that automatically. So I
believe
this is an RSVP-TE implementation issue, rather than a testing methodology
issue.
[RS10a]
OK. Noted and understood.
[sunwq]
Great, thanks.
[RS11]
????
[sunwq]
Well, the full text is like: "The percentile of Metric is defined as:
given
a metric and a percent X between 0% and 100%, the Xth percentile of all
the
dT values in the sample." Would it help a bit if sentence is replaced by
the
full text above?
Please be noted that this text is inherited from the IPPM docs (See RFC
2681, page 16, 4.1).
[RS11a]
The way it was written in the doc it was just a fragment of a sentence.
Unfortunately RFC 2681, page 16, 4.1. also does not give a proper
definition, as it too is just a fragmented sentence.
RFC 2330, p.26, 11.3 gives a better explanation, as:
"We will use the term 'percentile' to refer to the smallest value of x for
which the empirical distribution function is greater or equal a given
percentage"
[sunwq]
Well, to tell the truth I was troubled by this too. Later I realized that
one reason that might lead to such a definition is that the authors of 2681
had assumed that ``percentile" is a ``well known" term, and does not
inheriting its definition from RFC 2330. And quite surprisingly wikipedia
says "there is no standard definition of percentile." But wikipedia also
says ``however all definitions yield similar results when the number of
observations is large."
I will replace the original example with the following to avoid such a
dilemma (of conflicting calculations in two RFCs).
============original text===============
Example: suppose we take a sample and the results are: Stream1 = <
<T1, 100 msec>, <T2, 110 msec>, <T3, undefined>, <T4, 90 msec>, <T5,
500 msec> >
Then the 50th percentile would be 110 msec, since 90 msec and 100
msec are smaller, and 110 and 500 msec are larger (undefined values
are not counted in).
============end of original text========
============new text===============
Example: suppose we take a sample and the results are: Stream1 = <
<T1, 100 msec>, <T2, 110 msec>, <T3, undefined>, <T4, 90 msec>, <T5,
500 msec>, <T6, 350msec> >
Then the 60th percentile would be 110 msec, since 90 msec and 100
msec are smaller, and 110, 350 and 500 msec are larger (undefined values
are not counted in).
A more detailed definition of percentile may be found in RFC 2330.
============end of new text========
[RS12]
As stipulated in RFC2330 the EDF is defined as a function F(x) which for
any
x gives the fractional proportion of the total measurements that were <=
x.
(see RFC2330 11.3 p26)
Hence F(90)=0.25, F(100)=0.5,F(110)=0.75, F(500)=1 and therefore the 50th
percentile is defined as 100 as opposed to 110.
[sunwq]
Yes you are right with regard to RFC 2330. But we have used exactly the
same
example as in RFC 2681 (page 16, section 4.1). The good thing is that the
difference of the two calculations is very small when the number of
measurement is large. Change needed?
[RS12a]
As outlined above RFC 2330 as opposed to RFC2681 gives a proper definition
of percentile. Therefore we should stick to RFC2330 and change
accordingly.
It should be noted though, that also RFC2330 defines percentile
differently
than it is done in the statistical literature, but we can live with this,
as
it makes the definition easier to apply to measurement samples.
Good point, thanks for explaining. I hope the change above is satisfactory
for you.
Thanks and best regards,
Weiqiang
[------------------End of new comments---------------------]
And I realized that the points you have raised covered the main part of
the
text. We would expect your further comments regarding our responses.
Otherwise we will assume that you are happy with it and will proceed to
change the text based on the revision you sent in your previous email :)
Thank you again for your time in reviewing the draft so carefully.
Best regards,
Weiqiang
--
Weiqiang Sun
Shanghai Jiao Tong University
http://front.sjtu.edu.cn/~sunwq/
--------------------------------------------------
From: "Rschrage" <rschrage at schrageconsult.net>
Sent: Wednesday, August 19, 2009 9:57 PM
To: "'Weiqiang Sun'" <sunwq at MIT.EDU>; "'Lou Berger'" <lberger at labn.net>;
"'Henk Uijterwaal'" <henk at ripe.net>
Cc: <ccamp at ietf.org>; <zhangguoying at mail.ritt.com.cn>; "'IETF IPPM WG'"
<ippm at ietf.org>
Subject: Re: [CCAMP] [ippm] [Fwd: IPPM expert review request]
Hi all,
pls find attached a further updated edition of previous revised doc
draft-ietf-ccamp-lsp-dppm-06.
I believe this is as far as we can go for the moment before a mutual
satisfactory understanding has been reached and a new draft has been
issued
by the authors.
Again, pls feel free to comment.
Many thanks.
Brgds.
Reinhard Schrage
Tel: +49 (0) 5137 909540
Mobile: +49 (0) 172 26.36.046
reinhard at schrageconsult.com
-----Original Message-----
From: ippm-bounces at ietf.org [mailto:ippm-bounces at ietf.org] On Behalf Of
Weiqiang Sun
Sent: Mittwoch, 19. August 2009 09:28
To: Rschrage; 'Lou Berger'; 'Henk Uijterwaal'
Cc: ccamp at ietf.org; zhangguoying at mail.ritt.com.cn; 'BRUNGARD, DEBORAH A,
ATTLABS'; 'IETF IPPM WG'
Subject: Re: [ippm] [Fwd: IPPM expert review request]
More responses below:
RS4.
Should be more specific as to why this metric is no simple function of
single uni-directional LSP Setup Delay metric, i.e. why can it not be
deduced from single LSP Setup Delay?
[sunwq]
This point is raised regarding the sentence "The time needed to setup a
large number of LSPs during a short time period can not be deduced by
single
LSP setup delay."
One reason is that when a large number of LSPs in being setup during a
short
period of time, the control plane may be crowded or even overwhelmed by
the
large number of signaling and routing messages. This may significantly
slow
down the processing of a particular signaling message, which will results
in
longer LSP provisioning delays.
Are you suggesting that we add similar explanations to the document? We'd
love to do that if you feel it is necessary.
RS5.
The current definition only addresses multiple uni-directional LSP setups
between two nodes ID0 and ID1. What about the the case when ID0 is to
setup
multiple uni-directional LSP setups to different egress nodes ID1, ID2,
ID3,.,IDN?
[sunwq]
Yes, it is a useful case. Likewise you would expect a case to establish a
varied number of (instead of one) LSPs to a set of destinations. Our
suggestion is that this case is approached by integrating the defined
methodologies of single/multiple LSPs provisioning delay, in the form of
a
particular testing case, rather than a formal testing methodology. This
is
in fact a trade-off between complexity and accuracy.
Best regards,
Weiqiang
--
Weiqiang Sun
Shanghai Jiao Tong University
http://front.sjtu.edu.cn/~sunwq/
--------------------------------------------------
From: "Weiqiang Sun" <sunwq at mit.edu>
Sent: Wednesday, August 19, 2009 2:48 PM
To: "Rschrage" <rschrage at schrageconsult.net>; "'Lou Berger'"
<lberger at labn.net>; "'Henk Uijterwaal'" <henk at ripe.net>
Cc: <zhangguoying at mail.ritt.com.cn>; "'BRUNGARD, DEBORAH A, ATTLABS'"
<dbrungard at att.com>; "'IETF IPPM WG'" <ippm at ietf.org>; <ccamp at ietf.org>
Subject: Re: [ippm] [Fwd: IPPM expert review request]
Hi Reinhard,
Many thanks for doing the careful review. We appreciate your many
wording
suggestions. We will go through these one by one and adopt those we feel
appropriate. In this email I will not list all the comments since these
will not lead to major technical changes.
Below I try to respond to the points you raised in the revision.
RS1.
This would imply that the metric may produce a range of several values
at
one time with the minimum value to be taken from that range.
I believe what the authors are trying to say is, that the LSP Setup
Delay
Metric provides a lower bound on all possible LSP Delay Metrics of
actually instantiated LSPs, or in other words: an actual LSP Delay
Metric
cannot get smaller than an LSP Setup Delay Metric.
[sunwq]
This point is raised regarding the motivation of the sigleton definition
of Single Uni-directional LSP Setup Delay (ie. 4.1, para 2). We are
trying
to say that the minimum value of this metric reflects the (likely)
single
lsp setup delay when the control plane is lightly loaded. The formal
definition of the minimum value is in "14.1 The minimum of metric"
In fact we have inherited this from the IPPM documents (see RFC 2681,
page
2 section 1.1 bullet 3).
RS2.
How? Should there be a methodology be defined for this?
[sunwq]
This point is raised regarding the sentense in our methodologies
sections - "Make sure that the network has enough resource to set up the
requested LSP."
We have assumed that test personnel should have adequate expertise in
allocating enough resources for the testing. The allocation of resources
for the testing purpose can be very test specific and we believe it is
quite outside the scope of this document.
RS3.
Too vague? As both timestamps are to be taken on the same ingress node,
they should be taken at the same application/network level.
[sunwq]
This point is raised regarding the sentense "If the corresponding RESV
message arrives within a reasonable period of time, take the timestamp
(T2) as soon as possible upon receipt of the message."
Yes, ideally the timestamp should be taken immediately after the
reception
of the RESV message. The wording "as soon as possible" is used to allow
for some flexibility when RESV message needs to be propagated to certain
modules where timestamp-taking is most appropriate.
And again, this text is inherited from the IPPM documents (see RFC 2681
page 7, bullet 3).
Thanks again and looking forward to your further comments and revisions.
Weiqiang
--
Weiqiang Sun
Shanghai Jiao Tong University
http://front.sjtu.edu.cn/~sunwq/
--------------------------------------------------
From: "Rschrage" <rschrage at schrageconsult.net>
Sent: Tuesday, August 18, 2009 8:23 PM
To: "'Lou Berger'" <lberger at labn.net>; "'Henk Uijterwaal'"
<henk at ripe.net>
Cc: <zhangguoying at mail.ritt.com.cn>; "'BRUNGARD, DEBORAH A, ATTLABS'"
<dbrungard at att.com>; <sunwq at mit.edu>; "'IETF IPPM WG'" <ippm at ietf.org>;
<ccamp at ietf.org>
Subject: RE: [ippm] [Fwd: IPPM expert review request]
Hello,
sorry for late response-
Nevertheless here is a first edition of my revision of doc
draft-ietf-ccamp-lsp-dppm-06.
The doc has been saved in Word 2003 format to easily track changes.
I am using Kaspersky Anti Virus 2010 software so I trust there should
be
no
unpleasant surprises when downloading the attached word doc.
I thought it would be advisable to first address the suggested comments
and
questions upto and including the uni-directional LSP setup delay metric
and
after that to continue with the bi-directional and sample definitions
as
covered in the doc.
Please let me have your thoughts on this.
Many thanks.
Reinhard Schrage
Tel: +49 (0) 5137 909540
Mobile: +49 (0) 172 26.36.046
reinhard at schrageconsult.com
-----Original Message-----
From: ippm-bounces at ietf.org [mailto:ippm-bounces at ietf.org] On Behalf Of
Lou
Berger
Sent: Dienstag, 28. Juli 2009 14:59
To: Henk Uijterwaal
Cc: zhangguoying at mail.ritt.com.cn; BRUNGARD, DEBORAH A, ATTLABS;
sunwq at mit.edu; IETF IPPM WG
Subject: Re: [ippm] [Fwd: IPPM expert review request]
Hank/Reinhard,
Thank you very much for undertaking this review. Please cc
ccamp at ietf.org on any comments you may have.
Lou
On 7/28/2009 6:47 AM, Henk Uijterwaal wrote:
Lou, others,
We received the request for a review of a document currently under
discussion in the CCAMP WG, please see below. Is there anybody who
has time to do this review in the near future?
Reinhard Schrage (rschrage at schrageconsult.net) has voluntered to do
this.
Reinhard: please post anything you find to both the list and the
authors.
And thank you for doing this.
Henk
Matt & Henk
-------- Original Message --------
Subject: IPPM expert review request
Date: Fri, 24 Jul 2009 15:57:41 -0400
From: Lou Berger <lberger at labn.net>
To: ippm-chairs at tools.ietf.org
CC: Brungard, Deborah A, ALABS <dbrungard at att.com>, sunwq at mit.edu,
zhangguoying <zhangguoying at mail.ritt.com.cn>
Hi,
We, the CCAMP WG chairs, would like to request that the IPPM WG
review
a draft that is progressing through the CCAMP WG. This work applies
IPPM approaches to GMPLS. The document we'd like reviewed is
available at:
http://tools.ietf.org/html/draft-ietf-ccamp-lsp-dppm-06
Is this acceptable? Can you undertake this review? Alternatively,
we
can just last call the document in your WG (it has already passed
CCAMP
WG LC).
Thank you,
Lou (and Deborah)
_______________________________________________
ippm mailing list
ippm at ietf.org
https://www.ietf.org/mailman/listinfo/ippm
_______________________________________________
ippm mailing list
ippm at ietf.org
https://www.ietf.org/mailman/listinfo/ippm
_______________________________________________
CCAMP mailing list
CCAMP at ietf.org
https://www.ietf.org/mailman/listinfo/ccamp