IP Performance Metrics WG (ippm) Monday, March 10, 2008, 15:20--17:20 ==================================== This session was chaired by Henk Uijterwaal and Matt Zekauskas. Joel Sommers and Al Morton scribed the meeting, and their notes were edited into these minutes by the Chairs. AGENDA: 1. Administrivia 2. Status of Drafts and Milestones 3. Composition drafts 4. Multimetrics draft 5. Reporting draft 5a. Group draft/Martin Swany 5b. Individual Submission/Al Morton 6. Delay Variation AS 7. Possible Future Work for the group 7a. SLA monitoring (Nick Duffield/Joel Sommers) 7b. TWAMP extensions (Al Morton) 7c. Comparing Metrics (Henk and various others) 7d. Futures Discussion 8. Any Other Business 1. Administrivia Agenda bashing, Scribe, Minutes, Blue Sheets Henk opened the meeting. There were no changes to the agenda. Joel Sommers and Al Morton agreed to take notes. 2. Status of Drafts and Milestones Henk then went on to talk about the status of the drafts and milestones. The Network Capacity Definitions draft was published as RFC 5136. TWAMP is in WGLC. The traceroutes draft has GEN Art issues, which are being resolved. The Duplication draft will go to WGLC after the meeting. The revised milestones show everything finishing this year... any new work? 3. Composition drafts draft-ietf-ippm-framework-compagg-06 draft-ietf-ippm-spatial-composition-06 -- Al Morton Al discussed overall framework, which is an umbrella document with common definitions and concepts. There are three related drafts, in synch currently at version 06: the spatial composition draft (Al asked for more folks to read spatial drafts), the spatial aggregation draft, and the temporal aggregation draft. See the slides for how the comments since the last meeting have been reflected in the draft. The text appears to be stable, and if the group agree, then more focus on spatial composition drafts can be done. Steve Konish said that one thing he noticed was that some terms in draft are used prior to definition, then they're defined. For the spatial composition draft, text has been been changed related to ground truth (again, see the slides for details). Please read and comment, things are looking stable. An issue with multicast was brought up by S. Konish; Al thought we had resolved to just focus on unicast, and will double-check. Kaynam Hedayat volunteered to read and comment on the draft. Henk said he would probe the other folks that had volunteered again; Loki has commented on the framework but not on the spatial composition draft. An issue with temporal draft was noted relating to long term reporting when short-term measurements are being taken, and how to resolve the two. There is an ongoing need for temporal aggregation draft. Al felt that the framework is ready for last call, even though the specific aggregation drafts need more work. 4. Multimetrics draft draft-ietf-ippm-multimetrics-06 -- Emile Stephan Question: Do we split the draft into multicast and spatial? Emile presented pros and cons Emile presented changes made to the multimetrics draft. There have been editorial updates, but the main issue is with path stability. If the path changes, the metric will likely change too. So this case is now covered in the methodology and the discussion. Emile went through an example (see slides). Emile also discussed how loss at an intermediate point affects the spatial metrics, and the relationship of this draft with IPDV. One question was whether the draft should be split into multicast and spatial metrics; Emile presented pros and cons. His preferred approach is to leave the document as-is. Al Morton agreed, and said that people should also look at the minutes of the last meeting. Emile asked for a working group last call on the document. Henk agreed that the document has been largely stable, and said he would issue a last call in a couple weeks. 5. Reporting draft 5a. Group draft draft-ietf-ippm-reporting-01 (currently expired) -- Martin Swany Martin presented the reporting draft, which defines presentation of the metrics to end users, using ones that are easy to compute and understand. The initial set of metrics: delay, loss, jitter, duplication, reordering. Through discussion, the set has become: median delay, loss ratio, delay spread (difference between 25th/75h percentile), duplication, and reordering. Martin said that there was a new version of the draft available today and sent to the list; it will get posted officially after the IETF. This version includes clarifications to the metric names, and report data at the end of the time interval, rather than the middle. Open issues - should we revisit the selection of the population for delay in a larger context? Leave it as is? For example, not presenting measurements from network down periods. And there are some references that are missing and need to be added. Emile asked kind of user might be interested in these metrics, and when? Martin thought casual users might be interested; Skype already presents these (without any explanation); idea is that users want to know whether network is working well or poorly. Would these metrics be reported separately or together? Idea is that they are given together. Emile thought that some kind of index may be necessary so that a user can understand what is "good" versus "poor" performance. Otherwise the user doesn't know how to interpret. Emile - the risk is that this info is presented as something like SLA data. Should index or the specific values be presented? Martin - this is only supposed to be descriptive, not an SLA, and users are already doing this. [The chairs suggest looking at the draft and also past meeting minutes for a thorough discussion of the intended targets.] Kaynam said that these kinds of metrics are specific to a given application. What happens when other apps come along? E.g., videoconference. Matt Z -- that's not the direction this is intended to go. Kaynam's concern is that metrics for applications are defined elsewhere, so we should be careful, and re-read the draft to make sure that is clear. Steve Konish ask about throughput. He was surprised not to see it listed. Martin - throughput is a higher-level sort of thing, application (and operating system) specific. For that, we'd probably need some kind of calibration in order to think about reporting it. Matt Z noted that the group has tried to come up with a throughput or bulk transport metric, and has not succeeded to date. Look at the bulk transport capacity (BTC) framework draft, and please comment if you have an idea. 5b. Individual Submission draft-morton-ippm-reporting-metrics-04 -- Al Morton This draft talks about the different points of view of consumers of metrics, and offers guidance on reporting long-term measurements. The new version of draft addresses a Steve Konish comment, along with "processing forks" (see slides). It discussed the various recommendations for what should be measured (as well as what shouldn't be reported). There were not many people that have read the drafts since IETF-70; Al asked folks to read drafts and supply feedback. 6. Delay Variation AS draft-ietf-ippm-delay-var-as-00 -- Al Morton Al Morton presented the delay variation applicability statement draft, which is now a working group document. See the slides for presentation details. The comparisons of delay variation metrics haven't changed significantly since the last personal submission version. Al asks the group to consider if there are any other tasks that need to be added to the task description section. One part of the draft that deserves more study is the use of IPDV to detect path changes. The group should also review the measurement considerations section. Fred Baker commented to the authors that not all delay variation is due to queuing, and that there needed to be a bit more discussion of that in the draft. There were no questions during the meeting, but the group should read and comment on this draft. 7. Possible Future Work for the group 7a. Accurate and Efficient SLA Compliance Monitoring -- Joel Sommers Joel presented some joint work with Nick Duffield and others that presents strategies to efficiently monitor loss, delay and delay variation. This work is also described in a paper in SIGCOMM 2007, and may have some intellectual property associated with it. The strategy includes superimposed streams to serve multiple metrics efficiently at the same time ("multiobjective probing"). There is also a new metric on delay variation -- distortion. The slides have details on the paper and a pointer to code. Kaynam wanted to know how much more accurate the strategy was. The "backup slides" shows the advantages. Simulations show it to be particularly better for loss measurements. Matt asked about the IPR, what are the terms? Joel said he isn't sure because he's not directly involved in this, but he assumes that if IPR is eventually claimed it would have RAND terms. Ideally, the authors are interested in working with the group to revisit some of the sampling recommendations. 7b. TWAMP extensions draft-morton-ippm-more-twamp-00 -- Al Morton Al Morton presented a security extension to TWAMP, and also a packet padding extension that someone else suggested. See the slides for details. He first noted that he thought the wording of the sentence that says the OWAMP-Test protocol inheriting the security mode of OWAMP-Control should include a MUST. The security extension was to allow encrypted control with unauthenticated test streams. Thus, he proposes a registry for the security modes, and the extension that would allow the test stream to be unauthenticated. The second extension was to allow a specification of packet padding, and a directive to copy a specified number of bytes to the reflected packet. So, this really allows opaque data to be added to the reflected stream. There was a comment from Jabber that reflected packet padding might lead to abuse, or exploitation of code errors. Perhaps authenticated mode should be required for the reflect padding extension. 7c. Comparing Metrics -- Henk Uijterwaal Henk reprised a talk that was originally presented at PAM 2000. It compared the Advanced Network & Services Surveyor implementation of the one-way delay metrics with the RIPE Test-Traffic implementation. Overlaying scatterplots of the delays initially showed some differences, but these were accounted for by the difference in packet sizes (which Les Cottrell of SLAC figured out). So, one thing you need to do is ensure the test streams are equivalent, or account for the differences. So, the question for the group is how to make this sort of analysis more generic. Is that a good future avenue of work? Al Morton asked if the analysis uncovered anything ambiguous in the specification that needs to be revised. The observed difference could be explained by differences in packet size. But does that point to something more fundamental in the loss or delay specification? This is what the objective should be, not to ensure the values are exactly equivalent. 7d. Futures Discussion Henk noted that in addition to things discussed today, and other things that people might bring to the list, metrics for passive measurement is another potential area, although one clearly not in our charter today. Al asked if this was for the IP layer, and Henk said that was what he has heard in the past. Henk invited people that have ideas to post to the mailing list. Matt noted that a May 08 milestone is to develop new charter text. Lars asked for rough show of hands for what people are interested in, in order to gauge interest. There was interest in all the topics, at about the 5-person per item level. Lars noted that some prioritization was necessary, and that we needed to clear the current milestones first. 8. Any Other Business Al Morton brought up an invitation for folks interested in a ITU-T experts group on packet performance, hosted by ATT in April (now past). (You can see the invite PDF, posted as a slide.) With that, the meeting was closed.