Performance Metrics for Other Layers WG (pmol) Tuesday, March 11, 2008 1300-1500 Afternoon Session I Salon J OPS Co-Chairs: Al Morton <acmorton@att.com> Alan Clark <alan@telchemy.com> PMOL met with 25 people present and 1 participating remotely. These minutes were prepared by Al Morton, based on notes from Ralf Wolter (as official note taker) and additional notes supplied by Matt Zekauskas and Benoit Claise. 1. Brief overview of the PMOL status (Chairs - Al) http://www.ietf.org/html.charters/pmol-charter.html The agenda was agreed as published, and Al Morton gave the short WG Status. The SIP performance metrics draft has been adopted as a WG draft following discussion on the mailing list. The WG appear on track to meet its milestones: the first is June submission of the SIP metrics to the IESG. Since the SIP draft is a "proof of concept" for the framework and guidelines draft, it was agreed that there should be an informative reference to the framework as "work in progress". Also some text in the SIP metrics Introduction could indicate that this draft is consistent with the framework at its current state of development (because the framework will be finalized a few months later). The framework draft is not yet a WG draft, but it had undergone additional development to address the concerns raised at the last meeting. The plan was to revisit that status again after the draft was presented. 2. SIP Performance Metrics Draft (Daryl Malas) http://www.ietf.org/internet-drafts/draft-ietf-pmol-sip-perf-metrics-00.txt There were many good suggestions from SIPPING folks adopted in this version of the draft. The meeting participants raised additional comments, including the need to quantify time interval accuracy or error estimates, and also time of day accuracy (to anchor *when* each interval was measured). The "TB" and "TS" terminology should be replaced with something more typical (T1 and T2, or other). There were many other suggestions, including: + The time interval and time of day accuracies that a measurement device can achieve should be specified along with the results. + The draft should identify sources of time error, clock jumps are a big problem. + The set of times corresponding to a set of measurements should be reported. Daryl asked for specific text suggestions to cover the points above. There was agreement to expand the discussion in the Framework and Guidelines draft to cover this "time accuracy" guidance, but also to reference the IPPM framework RFC 2330 and not re-invent the wheel. Some specific points to cover are: + guidance for relative (interval) and absolute time measurement, + single device vs. multiple device time measurement and need for synchronization, + when and how much time accuracy is required, etc. Also, it was agreed that the statistical calculations will be removed throughout the draft. The leading Considerations section will describe the statistical summarization that users may perform, but the draft will define only the fundamental metrics (or what the IPPM has called singleton metrics). 3. Metrics Framework Draft (Alan) http://www.ietf.org/internet-drafts/draft-morton-perf-metrics-framework-02.txt The framework and guidelines draft has had extensive work to address comments at the last meeting. Almost all sections now have straw-man text. There was a discussion of the template for the elements of a metric. It may be useful to make a distinction between metrics that are defined in an abstract/ideal space, and those that are defined with practical measurement limitations taken into account. Comments suggested to take add new elements to the template, specifically "Error Estimate" and "Dependencies" sections were agreed to be useful (and anything you need to report on ti understand the results). There was a brief discussion of "wire time" and its implications on practical measurements, with agreement to add some more text on this. A suggestion to reference RFC 2026 in the approval process section was agreed. Further, there were valuable suggestions for items to cover in the Security and IANA sections, specifically that they should give guidance on what to include in the context of performance metric specification. There was a suggestion to be more specific about the reporting protocol. ipfix is one of many possible choices, and there was agreement to discuss this in more detail in the draft (but without being too specific). About 8 people had read the draft, and there was even stronger support (12 people) to take-up this draft as a working group item (to be ratified on the pmol list). In general, this was a much more productive meeting than the first, as there was little confusion about the charter.