|
I-D Title(s): Benchmarking
Terminology for Protection Performance Filename(s):
draft-ietf-bmwg-protection-meth-05.txt Reviewer Name: Vrushi
Gurudevappa Date: July 20, 2009 Please organize your
comments in the following categories below. Review Summary: This draft provides all the
scenarios and measurement techniques needed to successfully benchmark
FastReroute Protection. Overall: * Does/Do the draft(s)
provide clear identification of the scope of work? E.g., is
the class of device, system, or service being
characterized clearly articulated. Yes. * If a terminology memo,
are the measurement areas clearly defined or otherwise
cited? Is the working set of supporting terminology
sufficient and correct? To your knowledge, are the
areas of the memo that may conflict with other bodies of
work? Are there any measurements or terminology that are
superfluous? Are any missing? n/a * If a methodology memo,
does the methodology AND its corresponding
terminology adequately define a benchmarking solution for its
application area? Do the methodologies present sufficient detail for
the experimental control of the benchmarks? Yes they do. * If neither a
terminology or methodology, does the offered memo offer
complementary information important to the use or application of the
related benchmarking solution? n/a * Do you feel there are
undocumented limitations or caveats to the benchmarking
solution being proposed? If so, please describe. Not aware of any * Does the memo attempt
to define acceptance criteria for any of the benchmark
areas? No Technical Content:
(Accuracy, Completeness of coverage) Are definitions accurate?
Is the terminology offered relevant? Definitions are accurate and
terminology is relevant To your knowledge, are
there technical areas that are erroneous? Are there questionable
technical areas that need to be re-examined or otherwise scrutinized. Not to my knowledge. Does the solution
adequately address IPv6? n/a Do you feel the memo(s)
being offered are technically mature enough for advancement to
informational RFC? Yes, I do Clarity and Utility: If you had a need, would
you utilize the benchmarking solutions advocated by this and its
related memos? If not, why? Yes, I would and in fact I
have done it in the past year. Conformance to BMWG
principles: (see BMWG charter) http://www.ietf.cnri.reston.va.us/html.charters/bmwg-charter.html Do you have confidence
that the benchmarks, as explicitly defined, will yield
consistent results if repeated on the same device (DUT/SUT),
multiple times for a given test condition. If not, cite benchmark(s)
and issue(s). Absolutely yes. Do you have confidence
that the benchmarks, if executed for a given test condition,
utilizing the documented methodology on multiple test
infrastructure (e.g., test equipment), would yield correct and
consistent results on the same DUT/SUT? (Said differently, are the
benchmark's methodology written with enough exacting
detail, that benchmark implementation differences do not yield a
difference in the measured quantities?) If not, cite benchmark(s)
and issue(s). Yes of course. Do you feel that the
benchmarks form a basis of comparison between implementations of
quantity being characterized? (I.e., are the benchmarks suitable for
comparing solutions from different vendors.) I do feel the benchmarks for
a good basis for comparison between implementations. Thanks, -Vrushi |