Re: [bmwg] WGLC: draft-ietf-bmwg-protection term-06 and meth-05

[Arun Gandhi <gandhi at juniper.net>]

I-D Title(s): Benchmarking Terminology for Protection Performance
Filename(s): draft-ietf-bmwg-protection-meth-05.txt
Reviewer Name: Arun Gandhi
Date: July 23, 2009
 
Please organize your comments in the following categories below.

Review Summary: 
---------------
This is significantly useful and good work.


Overall: 
--------
* Does/Do the draft(s) provide clear identification of the scope of 
  work? E.g., is the class of device, system, or service being 
  characterized clearly articulated.

  Yes, the scope of the work is clearly defined.

* If a terminology memo, are the measurement areas clearly defined 
  or otherwise cited? Is the working set of supporting terminology 
  sufficient and correct?  To your knowledge, are the areas of the 
  memo that may conflict with other bodies of work? Are there any 
  measurements or terminology that are superfluous? Are any missing?

  Not Applicable.

* If a methodology memo, does the methodology AND its corresponding 
  terminology adequately define a benchmarking solution for its 
  application area? Do the methodologies present sufficient detail 
  for the experimental control of the benchmarks?

  Yes.

* If neither a terminology or methodology, does the offered memo 
  offer complementary information important to the use or application 
  of the related benchmarking solution?

  Not Applicable.

* Do you feel there are undocumented limitations or caveats to the
  benchmarking solution being proposed? If so, please describe.
 
  No.

* Does the memo attempt to define acceptance criteria for any of
  the benchmark areas?

  No.


Technical Content:  (Accuracy, Completeness of coverage)
--------------------------------------------------------
* Are definitions accurate? Is the terminology offered relevant?

  Yes. 

* To your knowledge, are there technical areas that are erroneous?
  Are there questionable technical areas that need to be re-examined
  or otherwise scrutinized.

  No.

* Does the solution adequately address IPv6?

  Not Applicable.

* Do you feel the memo(s) being offered are technically mature 
  enough for advancement to informational RFC?

  Yes.


Clarity and Utility:
--------------------
* If you had a need, would you utilize the benchmarking solutions
  advocated by this and its related memos? If not, why?

  Yes.


Conformance to BMWG principles: (see BMWG charter)
  http://www.ietf.cnri.reston.va.us/html.charters/bmwg-charter.html

* Do you have confidence that the benchmarks, as explicitly defined,
  will yield consistent results if repeated on the same device (DUT/
  SUT), multiple times for a given test condition. If not, cite
  benchmark(s) and issue(s).
 
  Yes ("under same conditions").

* Do you have confidence that the benchmarks,if executed for a given
  test condition, utilizing the documented methodology on multiple
  test infrastructure (e.g.,test equipment), would yield correct and
  consistent results on the same DUT/SUT? (Said differently, are the
  benchmark's methodology written with enough exacting detail, that
  benchmark implementation differences do not yield a difference in
  the measured quantities?) If not, cite benchmark(s) and issue(s).

  Yes.

* Do you feel that the benchmarks form a basis of comparison between
  implementations of quantity being characterized? (i.e., are the
  benchmarks suitable for comparing solutions from different vendors)

  Yes.

* If not, cite benchmarks and issues.

  Not Applicable.

* For those benchmarks cited above, do you feel that the benchmarks,
  as specified, have universal applicability for the given behavior
  being characterized? (i.e., benchmarks might not form a basis for
  cross-vendor comparison, can be used universally in a different
  role.)

  I do believe so.

Thanks,
Arun
____________________________________________________________________

From: bmwg-bounces at ietf.org [mailto:bmwg-bounces at ietf.org] On Behalf Of Al Morton
Sent: Thursday, July 02, 2009 2:25 PM
To: bmwg at ietf.org
Subject: [bmwg] WGLC: draft-ietf-bmwg-protection term-06 and meth-05

BMWG,

This message begins a Last call on the Sub-IP Protection terms and methods.

http://tools.ietf.org/wg/bmwg/draft-ietf-bmwg-protection-term 
http://tools.ietf.org/wg/bmwg/draft-ietf-bmwg-protection-meth 

The Last Call with end on July 31, 2009.

We've been discussing this topic in BMWG almost as long as I been chairman. Two original authors of the terms had to leave us, but others stepped-up and have brought this work to a good state of completeness.

In fact, I've recently shared our terms draft with some colleagues in the MEF who were quite impressed.

Please weigh-in on whether or not these Internet-Drafts should be given to the Area Directors and IESG for consideration and publication as an Informational RFCs. Send your comments to this list or acmorton at att.com.

Al

bmwg chair