IPPM, Thu 1300, IETF-86.
Minutes from this session are primarily a combination of notes by Nevil Brownlee and Al Morton
13:10 draft-ietf-ippm-rate-problem-02 +
10m A. Morton
Samita Chakrabarti: Allow symmetrical size Al Morton: just use it, nothing stops you.
Brian Trammel - need use case
Steve, make it BTC Problem statement, and
this is for Out-of service testing only (No! not necessary to limit.)
10m A. Morton
13:30 draft-morton-ippm-2679-bis-01 +
10m A. Morton
Matt Mathis asks whether the need to do a 2330bis instead of an update,
and whether the this impacts standards track advancement?
- Want to do this for 2680 (1-way loss)
Benoit Claise: clarification - All? the ippm metrics?
- Matt Mathis: “It helps users know that different implementations got the same results"
- Ready for WGLC now
Al Morton: Specs to advance now - 2679 and 2680
- slight changes to both RFCs
- WGLC now (as in current charter)
Matt Mathis: Could this challenge other docs now in process?
Live with what we have now.
Chair: discuss on Friday
5m A. Morton
Al Morton: Advanced Stream and Sampling Framework for IPPM
Has had detailed discussion on the list
Chair: maybe we want to open 2330 a little wider
13:45 draft-bagnulo-ippm-new-registry-00 +
20m M. Bagnulo
Mike Bugenhagen - Essentially, what is a registry?
a read-only database of services and assigned values.
You can' fix compliance or inaccuracy with a registry,
but you can't even start the dialogue without it.
Jeurgen Schoenwalder - There will always be parts of the registry that are not used,
and this approach may not work (?)
Matt Mathis - Make sure you have enough parameters to sufficiently
specify the test
Benoit Claise- RFC6390 - Has the template for metrics,
should be incorporated here.
Also, he preferred the non-independent template from an implementor
point of view, (but this is the first person to speak for this type).
Marcello Bagnulo: Registry for IPPM Metrics
Metrics can be hard to identify, RFCs get updated by new RFCs
Mike Bugenhagen: What kind of registry? IANA
will need lots of environment codepoints
could be hard for vendors to implement/support
need to know it target host doesn't support a particular measurement
Lots of databases out there already, how do you know whether
the target couldn't run a particular test?
repeats: have to know target couldn't run test
What parameters does registry specify? test types
Vendors need to know "what test do we have to run to get _this_ metric?"
Benoit Claise (jabber) as router vendor
Trying to reduce number of metrics for vendors
Want a template for all of them (pmol)
30m M. Mathis
First - why is BTC so hard?
Mike Ramalho - there's still heisenberg affect: ratio of test to cross
traffic must be low.
Matt Mathis: Why is BTC so hard?
Are you suggesting UDP for capacity?
No. Use TCP but let the applicaton control its rate
Agrees, but can't see how to do this. Controlled-TCP traffic
still needs to be a minority of total traffic
Matt Mathis: Model Based Metrics
A better way to do BTC
Al Morton: Sequential Probability Ratio Test (SPRT) (part of Matt's slides)
This test assumes the loss distributions are stable over time
15m K. Ko
Al: G.1021 has a similar model
Ken: Looks good.
Kostas: How real-world is this test what is the topology? Wireless?
Ken: It can be anything, this was a fixed broadband access.
Mike Ramalho: DASH or HTTP can be very bursty. Ken: model works better in more interesting cases.
ITU-T did a model very like this .21
More about testing method?
Very short TCP test at short intervals, polled off actual networks
Model is independent of topology
Need to repllicate these tests, video stream tend to be bursty
10m K. Pentikousis
Friday 15 March, 12:30 - 13:30, Caribbean 6
IV. Rechartering Discussion
12:30 Welcome, review of Thursday discussion
10m Chairs (B. Cerveny, B. Trammell)
12:40 Presentation of Draft Charter and Discussion
50m Chairs (B. Cerveny, B. Trammell) + Open Mike
Any Other Business (time permitting)
Minutes from this session are taken from notes posted in the draft IPPM charter discussed during the session. Text in italics consists of charter text discussed and updated during the meeting.
[begin charter discussion text]
The IP Performance Metrics (IPPM) Working Group develops and maintains standard metrics that can be applied to the quality, performance, and reliability of Internet data delivery services and applications. It also develops and maintains protocols for the measurement of these metrics. These metrics are designed such that they can be performed by network operators, end users, or independent testing groups. Metrics developed by the IPPM WG are intended to provide unbiased quantitative performance measurements and not a value judgement, such as “good” or “bad” performance.
[Al Morton: tests against specific targets? Make sure we’re not straying into “good” and “bad” by making pass/fail indeterminate]
[Matt Mathis: a pass/fail is a different kind of metric]
[Al Morton: A specific target would be a provider with an advertised rate, and the metrics giving a different rate, evaluating whether that advertised rate was actually present]
[Michael B: You should be able to say whether a test achieved a measurement, and did that measurement fall in the specified range]
[Matt Mathis: given that lmap is not chartered, this is defining this group’s space wrt what theirs will be]
[Al Morton: The scopes can be coordinated]
The IPPM WG has produced documents that define specific metrics and procedures for accurately measuring and documenting these metrics. The working group will continue advancing these metrics along the standards track, using the guidelines stated in RFC 6576.
[Should we stop maintaining the metrics? No!]
[Lars Eggert: Do you want to use the old stuff for the new system, or not?]
[To the extent that’s possible]
[Lars Eggert: Certainly, but it needs to be stated why the old metrics will not suffice]
[Matt: the charter probably does not need to say anything about automation]
[Automation, no, automatability yes for 2330bis]
The WG will develop the minimum number of new metrics and models needed to more accurately characterize the network path(s) under test and/or the behavior of transport and application layer protocols on these path(s). Additional methods will be defined for the composition and calibration of these metrics, as well as active, passive and hybrid measurement methods for these metrics, and their applicability, as appropriate. The WG may update its core framework RFC 2330 as necessary to accommodate these activities.
[Greg: There’s a suggestion of active by default]
[To date we’ve done active methods]
[Lars Eggert: we’ve struggled with passive. Active seems easier, because you control the test traffic. One question is, passive would be nice but if you don’t have as good a measure, is it worth it?]
[Chair agrees as IC: the best you can do often is to estimate error]
[Lars Eggert: When active measurements are problematic, is when they absorb resources from the users.]
[Or when the measurement changes the value]
[Active is no longer research, passive is still]
[Lars Eggert: In cases where an active measurement would cause undue impact on the user or distort the measurement...]
[paragraph attracted quite a bit of discussion]
[Examples tend to be read as a list of things that are in or out of scope, so we’d like to avoid them]
[Characterize? Connotations of value judgement? Not really, but more basic english might be better]
The WG has produced protocols for communication among test equipment to enable the measurement of the one- and two-way metrics (OWAMP and TWAMP respectively). OWAMP and TWAMP will be advanced along the standards track. The WG will further develop and improve these protocols.
[Al Morton: The protocols we have written don’t implement the metrics, they provide timestamps etc...]
[Are new active measurement protocols in scope?]
[“new measurement protocols such as...”]
[We had a MIB proposal, which was dropped because there was insufficient interest]
[Matt Mathis: although, asking the question ‘is the link idle’ has to be done passively, which may be an SNMP question]
[Samita Chakrabarti: This was a MIB for exporting measured data, rather than controlling the protocols]
[That seems in scope, definitely]
[Any pass/fail has to be replicable, and therefore requires to include the test parameters]
[Matt Mathis: We may be overusing the word MIB and really mean Schema]
The equivalency of metric definitions is crucial to accurate, comparable and reproducible measurement. The WG will define and maintain a registry of metric definitions to this end. The WG encourages work which compares IPPM metrics with metrics developed elsewhere, particularly in regards to equivalency, similarity and functionality. The WG also encourages work which improves equivalency of IPPM protocols with other protocols for measuring similar metrics.
[Al Morton: we should use a more process-based term for interoperability]
[Phil: does the LMAP reference path stuff from yesterday fit in here?]
[Al Morton: Yes]
[Isn’t that an lmap thing? It’s kind of in the boundary]
[Lars: Do we really see this happening here?]
[That’s what we’re encouraging it, but we don’t set it as a goal]
[Samita: note-taker missed comment (Samita thought comment was about "a need for a usecase document for usage of
existing measurement protocols and the metrics.")]
The IPPM WG seeks cooperation with other appropriate standards bodies and forums (such as ATIS IIF, ITU-T SG 12 and 15, MEF, BBF, etc.) to promote consistent approaches and metrics. Within the IETF process, IPPM metrics definitions will be subject to as rigorous a scrutiny for usefulness, clarity, and accuracy as other protocol standards. The IPPM WG will interact with other areas of IETF activity whose scope intersects with the requirement of these specific metrics. The WG will, on request, provide input to other IETF working groups on the use and implementation of these metrics.
[Al Morton: we can lose SG 13]
[Matt Mathis: IEEE?]
[Such as might be read exclusive]
[Drafts discussion notes]
[Matt Mathis: Minor process issue, would like to know well before Berlin if we need a -00 for model-based-metrics]
[Consensus call should be soon]
[Al Morton: the reference path was presented]
[But there’s no document... should be on the list of drafts too]
[Matt Mathis: lmap is about access only, whereas ref path is about the whole rest of the net]
[end charter discussion text]