2.7.4 IP Performance Metrics (ippm)

NOTE: This charter is a snapshot of the 49th IETF Meeting in San Diego, California. It may now be out-of-date. Last Modified: 09-Nov-00

Chair(s):

Merike Kaeo <kaeo@merike.com>
Matthew Zekauskas <matt@advanced.org>

Transport Area Director(s):

Scott Bradner <sob@harvard.edu>
Allison Mankin <mankin@east.isi.edu>

Transport Area Advisor:

Allison Mankin <mankin@east.isi.edu>

Mailing Lists:

General Discussion:ippm@advanced.org
To Subscribe: ippm-request@advanced.org
Archive: http://www.advanced.org/IPPM/archive/

Description of Working Group:

The IPPM WG will develop a set of standard metrics that can be applied to the quality, performance, and reliability of Internet data delivery services. These metrics will be designed such that they can be performed by network operators, end users, or independent testing groups. It is important that the metrics not represent a value judgement (i.e. define "good" and "bad"), but rather provide unbiased quantitative measures of performance.

Functions peripheral to Internet data delivery services, such as NOC/NIC services, are beyond the scope of this working group.

The IPPM WG will define specific metrics, cultivate technology for the accurate measurement and documentation of these metrics, and promote the sharing of effective tools and procedures for measuring these metrics. It will also offer a forum for sharing information about the implementation and application of these metrics, but actual implementations and applications are understood to be beyond the scope of this working group.

Goals and Milestones:

Oct 97

  

Submit drafts of standard metrics for connectivity and treno-bulk-throughput.

Nov 97

  

Submit a framework document describing terms and notions used in the IPPM effort, and the creation of metrics by the working group to IESG for publication as an Informational RFC.

Feb 98

  

Submit documents on delay and loss to IESG for publication as Informational RFCs.

Apr 98

  

Submit a document on connectivity to IESG for publication as an Informational RFC.

Sep 98

  

Submit a document on bulk-throughput to IESG for publication as an Informational RFC.

Internet-Drafts:

Request For Comments:

RFC

Status

Title

RFC2330

 

Framework for IP Performance Metrics

RFC2678

E

IPPM Metrics for Measuring Connectivity

RFC2679

PS

A One-way Delay Metric for IPPM

RFC2680

PS

A One-way Packet Loss Metric for IPPM

RFC2681

PS

A Round-trip Delay Metric for IPPM

Current Meeting Report

Minutes of the IP Performance Metrics WG (ippm)
Tuesday, 12 December 2000, 17:00-18:00
======================================

The meeting was chaired by the working group chairs, Matt Zekauskas and Merike Kaeo. Thanks to Paul Love who took notes during the meeting.

AGENDA

1. Introduction of new co-chair & agenda bashing
2. Discuss the new charter and milestones
3. Update on Bulk Transfer Capacity
4. Update on Network Performance Measurement for Periodic Streams
5. One-way delay protocol document discussion
6. Discussion of any issues with IPDV or loss patterns

IETF home page: http://www.ietf.org/html.charters/ippm-charter.html
IPPM home page: http://www.advanced.org/IPPM/

1. Introduction of new co-chair & agenda bashing
-- Matt Zekauskas, Advanced Network & Services

The meeting was called to order by Matt Zekauskas who introduced the new co-chair and reviewed the meeting agenda.

2. Discussion of new charter and milestones
-- Matt Zekauskas

A discussion on the new proposed charter and new milestones was led by Matt who noted that what's on the web page is 3 years old and stale. Matt read/editorialized the new text which was published to list. The following points were made:

- Draft on protocols will be a priority in upcoming weeks
- The Framework, RFC 2330, was reviewed by the Chairs and looks to still be OK. Therefore it is pulled from milestone list and people were asked to post to the list if they thought otherwise.
- Need a BCP on one-way delay & loss.
- Need statistician(s) to help with comparisons & validity of metrics. Henk Uijterwaal commented that Les Cottrell and a SLAC colleague have done some work on this (see the Proceedings of the The First Passive and Active Measurement Workshop (PAM 2000), Hamilton, NZ, April 2000).
- Longer-term milestones, where nobody has signed up, will be deleted if no one steps forward. These include:

Mar 01 Develop draft for comparison and validity of metrics
Jun 01 Develop draft for passive application of metrics
Nov 01 Develop draft for BCP to apply existing metrics to diffserv
Nov 01 Develop draft for BCP to apply existing metrics to multicast

Comments from floor on milestones:

Will Leland noted that IPPM also needs to do work on MIB(s) for metrics. Bob Cole stated there is work being done on a MIB to instrument inflow. RMONMIB WG is doing work on additional, related MIBs so perhaps no additional work is needed.

Matt Zekauskas commented that IPPM would like to work closely with that effort, and is also trying to work closely with the Traffic Engineering WG.

[After the meeting two people approached the chairs saying they were statisticians and were interested in pursuing the comparison draft. The chairs will follow this up with them (one from Telcordia, one from CAIDA).]

3. Update on Bulk Transfer Capacity -- Mark Allman, NASA Glenn

Mark used one slide to say that the authors think the framework is done and the question was posed to the group as to what they thought. It was asked if this was a draft or a framework? The answer was that it's a framework to define BTC that tries to clear up which things need to be nailed down by implementation. In January, one particular implementation will be published as a draft (see milestone list). There was agreement in those present who have read that it should go out.

4. Update on Network Performance Measurement for Periodic Streams
-- Vilho Raisanen, Nokia

This draft is now in 3rd revision with changes from 2nd being relatively minor. There was a comment that it does not address statistics - this it is just a building block. It can also produce results on packet loss.

There were comments made with respect to implementations, principally that having an implementation would be desirable before advancing the draft. Vilho reported on a draft of an implementation report. It is draft-raisanen-ippm-npmps-results-00.txt; a PDF version is available. This implementation draft describes application of the metric. Also, packet integrity checks will be added into the draft to account for packet errors.

There was a comment related to the figures from the implementation draft. Vilho described that the delays shown in the figures were not one-way, but estimated by taking half of round-trip delay. The two delay distributions in the second figure were taken from very different networks: one a private network and the other a public Internet, hence differences shown.

Matt Zekauskas asked if including what a Poisson stream would have show for same networks would be useful? Merike asked if comparison data between Poisson-type tests and the stream-type tests could be included.

5. One-way delay protocol document discussion
-- Stanislav Shalunov, Internet2/UCAID

Stanislav reviewed the protocol's goals and stated that this protocol was designed to be useful in a wide range of scenarios. He addressed some comments from the list including making padding pseudo-random and arbitrary. He also clarified that the Test protocol can be used without the Control protocol.

In the ensuing discussion, Matt Mathis commented that the word 'stealth' should not be used because of some potentially bad connotations due to the way the word is used elsewhere. Also, he added that for the test packet you could create an opaque cookie and hide the test packet in a more generalized TCP packet so there was no need to recreate packet structure. All you have to do is specify length and offset for the test packet. This would also make the protocol flexible for new uses.

Ruediger Geib noted that a version number should be included for backwards compatibility, especially for the control protocol. He then asked how the protocol handles responses that are unexpected and suggested that the draft be more specific under which conditions packets are discarded. The built-in resiliency needs to be specified in the draft for implementers to all select the same mechanism. Ruediger also initiated debate on how start and stop messages were used. It was decided to take the issue onto the mailing list.

Bob Cole commented that cell delay variation should be added. Matt Zekauskas would like any results (of cell delay variation studies, in particular on using periodic streams) published on list. Henk suggested extending test packets to allow more information than just time stamps. Henk will supply samples. Also, should the two protocols (Test and Control) be in one or two separate documents? Bob Cole thought that if in one document, then it would be easier to talk about how they were used in various situations.

Stanislav said he would summarize his interpretation of what was needed to the list, and that others should comment.

7. Discussion of any issues with IPDV or loss patterns

Phil Chimento, the IPDV author, thought IPDV was nearly done, but recently there has been lots of traffic on the list. He went over the list of comments made to the list and promised to have a new draft by 1 Feb 2001.

Al Morton commented that specification of delay variation is simplified if packet size doesn't change. Phil thought this was added to one of the earlier versions but if not, he will add to the new draft.

Slides

Agenda
Status of the IPDV Draft
A One-way Delay Measurement Protocol
IETF 49 Update
NPMPS-Style Measurement Results