Re: [bmwg] WGLC: draft-ietf-bmwg-protection term-06 and meth-05

I-D Title(s): Benchmarking Terminology for Protection Performance

Filename(s): draft-ietf-bmwg-protection-meth-05.txt

Reviewer Name: Vrushi Gurudevappa

Date: July 20, 2009

Please organize your comments in the following categories below.

Review Summary:

This draft provides all the scenarios and measurement techniques needed to successfully benchmark FastReroute Protection.

Overall:

* Does/Do the draft(s) provide clear identification of the

scope of work? E.g., is the class of device, system, or

service being characterized clearly articulated.

Yes.

* If a terminology memo, are the measurement areas clearly

defined or otherwise cited? Is the working set of

supporting terminology sufficient and correct? To your

knowledge, are the areas of the memo that may conflict

with other bodies of work? Are there any measurements or

terminology that are superfluous? Are any missing?

n/a

* If a methodology memo, does the methodology AND its

corresponding terminology adequately define a benchmarking

solution for its application area? Do the methodologies present

sufficient detail for the experimental control of the benchmarks?

Yes they do.

* If neither a terminology or methodology, does the offered

memo offer complementary information important to the use

or application of the related benchmarking solution?

n/a

* Do you feel there are undocumented limitations or caveats to

the benchmarking solution being proposed? If so, please

describe.

Not aware of any

* Does the memo attempt to define acceptance criteria for

any of the benchmark areas?

Technical Content: (Accuracy, Completeness of coverage)

Are definitions accurate? Is the terminology offered relevant?

Definitions are accurate and terminology is relevant

To your knowledge, are there technical areas that are erroneous?

Are there questionable technical areas that need to be re-examined

or otherwise scrutinized.

Not to my knowledge.

Does the solution adequately address IPv6?

n/a

Do you feel the memo(s) being offered are technically mature enough

for advancement to informational RFC?

Yes, I do

Clarity and Utility:

If you had a need, would you utilize the benchmarking solutions

advocated by this and its related memos? If not, why?

Yes, I would and in fact I have done it in the past year.

Conformance to BMWG principles: (see BMWG charter) http://www.ietf.cnri.reston.va.us/html.charters/bmwg-charter.html

Do you have confidence that the benchmarks, as explicitly

defined, will yield consistent results if repeated on the

same device (DUT/SUT), multiple times for a given test condition.

If not, cite benchmark(s) and issue(s).

Absolutely yes.

Do you have confidence that the benchmarks, if executed for a

given test condition, utilizing the documented methodology

on multiple test infrastructure (e.g., test equipment), would

yield correct and consistent results on the same DUT/SUT?

(Said differently, are the benchmark's methodology written

with enough exacting detail, that benchmark implementation

differences do not yield a difference in the measured quantities?)

If not, cite benchmark(s) and issue(s).

Yes of course.

Do you feel that the benchmarks form a basis of comparison between

implementations of quantity being characterized? (I.e., are the

benchmarks suitable for comparing solutions from different vendors.)

I do feel the benchmarks for a good basis for comparison between implementations.

Thanks,

-Vrushi