[PMOL] feedback on draft-morton-perf-metrics-framework-02

[Benoit Claise <bclaise at cisco.com>]

Dear all,

Here is my feedback after reading the draft.

- First of all, as mentioned during the WG meeting, the following 3 items seem to be a structure for the draft.

   There are at least three phases in the development of performance
   standards.  They are:

   1.  Definition of a Performance Metric and its units of measure

   2.  Specification of a Method of Measurement

   3.  Specification of the Reporting Format

I understand it's not the case, so the focus of the document compared to these 3 points should be explained
- you mention "Performance Metrics Entity" in the abstract. If capitalized, doesn't it imply a definition?
The draft only mentions:

   This memo refers to a Performance Metrics Entity, or PM Entity, which
   may in future be a working group or directorate or a combination of
   these two.

You might to mention what its goal is.
- " (ii) the correlation between the metric and the quality of service / experience delivered to the user (person or other application)"
Ideally the metric is the quality of service/experience
- In  section 3.4.2,  an example of the Service Level Contract/Service Level Agreement would seem ideal, as the SLA must be clearly, including the way of metering it.
- 
- 3.4.4. Metric Definition Template

Name
Description
Measurement
Method
Units of measurement
Measurement or Sampling Interval
Use and Applications
Implementation
Reporting Model

Which ones MUST/SHOULD be present? All of them?
- Section 3.7 "Reporting protocols typically introduce some limitations and assumptions with regard to the definition of a metric."
After discussing with Alan after the meeting, I understand what you want to say. However, I think it should be expanded.
- Section 3.12. I don't understand why we have the following excerpt in there:

   The key areas of variation between different metric users include:

   o  Suitability of passive measurements of live traffic, or active
      measurements using dedicated traffic

   o  Measurement in laboratory environment, or on a network of deployed
      devices

   o  Accuracy of the results

   o  Access to measurement points and configuration information

   o  Measurement topology (point-to-point, point-to-multipoint)

   o  Scale of the measurement system

   o  Measurements conducted on-demand, or continuously

   o  Required reporting formats

   ------------------- end of Al's sections, but more in 4.2 -----

Editorial:
- you might want to give a reference to RAQMON
- section 3.1 " Metrics are intended for use in measuring the performance of an application, network or service." while section 3.2 mentions "A metric is a measure of an observable behavior of an application, protocol or other system."
- realign

   (i) Description of metric The description should explain (ii) Metric
   use and applications (iii) Implementation (iv) Reporting Model

Regards, Benoit.

_______________________________________________
PMOL mailing list
PMOL at ietf.org
https://www.ietf.org/mailman/listinfo/pmol