IP Performance Metrics WG (ippm)

This session was chaired by Henk Uijterwaal and Matt Zekauskas. Vinayak Hedge and Al Morton scribed the meeting, and their notes were edited into these minutes by the Chairs.

AGENDA:

1. Administrativia (Chairs)
2. Status of drafts and milestones (Chairs)
3. Testing Standards Track Metrics
1. Group metrics testing draft (Ruediger Geib)
2. Individual lab testing metrics (Al Morton)
4. TCP Throughput Testing Methodology (Reinhard Schrage)
5. Reporting Metrics
1. Short-Term Reporting Metrics [Reporting Metrics to Users] (Zekauskas for Martin Swany)
2. Long-Term Reporting Metrics (Morton)
6. Loss Episode Metrics (Morton for Nick Duffield)
7. Round-trip Loss Metrics (Morton)
8. Requirements for IP Multicast Performance Monitoring
9. Metrics Registry
   1. Introduction (Henk)
   2. Obsolete the registry (Morton)
   3. Modify the registry (Emile Stefan)
10. AOB

1. Administrativia (Chairs)

Ruediger was delayed getting to the meeting, so we skipped the Testing Standards Track Metrics, which moved to after the Round-Trip Loss Metrics presentation (hence out of order below).

4. TCP Throughput Testing Methodology

draft-ietf-ippm-tcp-throughput-tm-04.txt

Rienhard presented the key issues from Anaheim in the view of the authors, and what has been done. He said that they were not concentrating on metrics but methodology. They addressed the use of RFC 2544, guidelines for hardware performance of the testers, diagnostic help, and experimentation to validate the methodology. The majority of the presentation presented results of lab testing. See the slides for details.

Lars Eggert asked if this methodology was envisioned using off the shelf hardware. Rienhard said no; Lars thought that was good because tracking retransmitted data cannot be done without custom kernels. Al asked how you reconcile TCP segments and bytes; the equation was in bytes, but the example in segments (which is simpler to implement). And what happens if segments are retransmitted multiple times?

Lars asked if only one-way transfers were measured? What if transfer rates are asymmetric? (Residential equipment is often asymmetric.) The target is a corporate/managed network, which would be symmetric. Vinayak asked how do different application-layer protocols affect the throughput in an non-idle network.

Rudiger Geib asked if this is for physical or logical networks? If it's for either, what happens if others are sharing equipment or lines? Again, it was emphasized that this was for a managed network, so if it is logical, the different logical networks should not affect each other.

Lars thought that the assumptions should be made explicit: not for residential networks, or paths need to be stable. We should pin down all the parameters and test conditions in which the test was conducted; we need to identify all the parameters that have influence on the results.

Al ended with a couple of quick comments. First, on RFC 2544. RFC 2544 is listed as an example now, not the only qualification test, but that needs more clarification. It is out of scope for in-service network measurements. Second, while you say that diagnostic information is provided, Al thought scope of the document was diagnostic measurements.

5. Reporting Metrics

5.1 Short-Term Reporting Metrics (Reporting Metrics to Users)

draft-ietf-ippm-reporting-05.txt

Matt presented the changes that Martin Swany made in the latest version of the draft, largely based on the observation that the packet reordering definition did not match the IPPM metric referenced (see slides for details). Corrected prose was added, along with an additional reference to RFC 4737 (the reordering document).

Since the definition changed (although the intent did not), there will be a short working group last call of the draft. Please read and add any additional comments.

5.2 Long-Term Reporting Metrics (Different Points of View)

draft-ietf-ippm-reporting-metrics-03.txt

Al gave a summary of the draft's recommendations, and the two points of view that the recommendations are based upon. He then summarized the changes in the -02 and -03 versions (see slides for details).

There are two questions on page 5 of the slides, which look in particular at variation with respect to utilization and available capacity. Al would like discussion on these questions on the mailing list (there were no comments during the meeting), and more people to read the draft and comment.

6. Loss Episode Metrics

draft-ietf-ippm-loss-episode-metrics-00.txt

There is IPR associated with this draft ( http://datatracker.ietf.org/ipr/1354/); the statement was updated to reflect the new draft name. Changes since the last IETF were to make this a working group draft and update the IPR statement. Al then went on to present answers to questions posed at the last meeting (relation to the Gilbert model) and on the mailing list (why sample), none of which required changes to the draft (see slides for details). The authors would like more people to read and comment on the draft.

Dave McDysan started a discussion on the merits of loss episode metrics. If losses occur, then in some sense it's too late if the network has loss objectives. Can traffic characterization and latency prediction help you detect when lossy conditions might occur, and offer opportunities to do traffic engineering or otherwise react? Yaakov Stein noted that as a timing person, when doing accurate timing recovery, you can find congestion before loss. (He also pointed the group to the tictoc WG for more detail.) Lars noted that's is the premise behind delay-based congestion control. Al thought about applying error correction as a reaction to loss.

Viniak noted that to an application, not all packets are equal. For example I versus P frames in MPEG, or for HTTP responses the header may be more important than the content. (Yaakov noted H.264 might be another example.) Thus it might be hard to correlate this with user perceptions. Al didn't believe that it was taken into account currently with packet-pair sampling.

7. Round-trip Loss Metrics

draft-morton-ippm-rt-loss-00.txt

Al gave a short presentation describing the motivation and basically what was in the draft (see slides for details). He felt this metric would complete the basic IPPM set, and also "everyone does it" -- for example, that's what ping would show. In addition, he claimed round-trip loss is what really matters to an application, and if conditional round-trip delay was presented, round-trip loss needed to be presented as well as a companion (which is parallel with the argument about presenting conditional one-way delay and loss together).

Yaakov pondered what round-trip loss meant -- there is no physical analog to "two-way loss". Packets are only lost at one point, either upstream or downstream. Al again reiterated that his view is that an application doesn't care which direction. Yaakov later noted that he understood this, but still round-trip delay is useful because of the difficulties in synchronizing clocks but there is no such problem if there is loss.

Dave noted that this is in fact what most customers test, and it could be used as a trigger for a more detailed one-way diagnostic.

3. Testing Standards Track Metrics

draft-ietf-ippm-metrictest-00.txt

The main changes to the draft were clarifications generated when a student tried to implement the draft. Preliminary results of the student's work was then presented (see slides for details). Ruediger wondered if there were others that wanted to get together to try out the methodology. Please read and comment on the draft. One issue is how much lab testing might be incorporated (see next subtopic on lab testing for more).

Yaakov noted that pseudowires use RFC 4723 boxes on both sides, which might be another tunneling option. Dave McDysan noted that old devices that tunnel L2TP have performance that depends on throughput, which might introduce unintended dependencies (and errors), and that is an important consideration in live, repeatable network testing.

Lars asked about using RFC 2119 language. Something like MUST show in lab, SHOULD verify on Internet. Or if an impairment is infrequent, then MAY use lab equipment. Lars also issued a request to please try this out; two implementations is enough to advance a metric, and he'd like to see at least one metric move up while he is still AD.

3.2 Individual draft - Lab Testing Metrics

draft-morton-ippm-advance-metrics-01.txt

Yaakov noted that we are not testing interoperability this way, successful reproduction of results is what we are after. Lars said that the test plan should be available before sharing results -- the working group can decide what equivalent results are before tests are conducted.

8. Requirements for IP Multicast Performance Monitoring

draft-bipi-mboned-ip-multicast-pm-requirement-01.txt

Alberto presented work that had been previously developed in the context of the mboned working group. Two solutions drafts have also been developed. However, the techniques are not specific to multicast, and could be applied to general traffic. Hence this presentation to ippm. See the slides for details on the requirements and pointers to the solutions. A key design consideration, though, is "in-line performance measurement" -- deriving measurements from passively viewing user traffic, not by injecting probe traffic.

IPPM is currently only chartered to consider active measurement. Lars noted that he has seen proposals for passive measurement, but they are usually specific to an application. The IPPM framework is general, and we would need a similar general passive measurement framework so results are comparable. This requires work; how do we generalize the results from a measurement on a specific application stream? We don't know. Existing methods may not work, and we need to discuss this on the list.

Yaakov asked about the relationship to Metro Ethernet Forum work. There are well developed SLA measurements for layer two in the MEF. This triggered a discussion. How much of this is multicast? IP multicast is different from layer 2 multicast. There were references to MEF6, MEF1, MEF2 and MEF 10.2 - service metrics (but these are based on OAM packets; triggers which cause counters to be read in switches).

Time was short, so discussion of consideration of a passive measurement framework will be taken to the list.

9. Metrics Registry

9.1 Introduction

Henk introduced the issue -- the Metrics Registry was introduced in RFC 4148, but has apparently seen little use. One anecdote is that it took 8 months to discover an error in a registry update that a parser would have found easily. Maintaining the registry requires work, both from document authors and IANA. Henk presented some potential options (Withdraw, Expand, Replace) and issues to consider. See the slides for a more detailed description. Goal: decide what to do with this registry by the Beijing meeting.

9.2 RFC 4148 and the IPPM Metrics Registry are Obsolete

draft-morton-ippm-rfc4148-obsolete-01.txt

Al presented the case for reclassifying RFC 4148 as historic. In his view, the IPPM metrics do not lend themselves to registration; there are many parameters, and the metrics need to be evaluated in context. If RFC 4148 was classified as historic, IANA would "withdraw the registry", which simply means that there would be no new registrations. Any existing uses could continue (but this draft argues that the registry is apparently unused). See the slides for the complete argument.

Yaakov asked if there was any usage or a report as to why it was unused.

9.3 IPPM Metrics Registry Extension

draft-stephan-ippm-registry-ext-00.txt

Emile presented on the current registry, it's limits, and possible extensions based on a "metric flavor" concept. In Emile's view there are two uses, for functional specifications or "RFPs" or references in MIB modules. See the slides for details.

Emile also noted that there is a need for long term metrics exchange in cooperative projects related to the ALTO working group (end to end metrics from applications, and per-network measurement from ISPs).

Lars noted that ALTO metrics are not comparable, as ALTO has to take in consideration network maps. The IPPM metrics and ALTO metrics may not be the same as the network and the measurement conditions are different.

Al asked if there was consensus on withdrawing the existing registry, as he saw it in Emile's slides. Emile worried that withdrawal would affect any existing use, but Al reiterated that historic status would maintain the existing definitions, but not allow new definitions.

The working group was over time, and Henk said we would have to take this to the list. There must be a user community and application for the registry, otherwise we should not do it.

10. AOB

There was no time remaining, so the Chairs closed the meeting.