DRAFT DRAFT DRAFT DRAFT DRAFT IP Performance Metrics WG (ippm) Wednesday, July 30, 2008, 13:00--15:00 ====================================== This session was chaired by Henk Uijterwaal and Matt Zekauskas, and Joe Kopena was the scribe. AGENDA: 1. Administrativia (Chairs, 5') 2. Status of Drafts and Milestones (Chairs, 10') a) TWAMP b) MultiMetrics draft c) Traceroute draft 3. Composition drafts (Al Morton, 15') 4. Reporting IP Performance Metrics to Users (Stanislav Shalunov, 15') http://www.ietf.org/internet-drafts/draft-ietf-ippm-reporting-02.txt 5. Delay variation AS (Al Morton, 15') 6. TWAMP extensions. (3x5') = Mixed Mode Extension for TWAMP (Morton and Hedayat) http://tools.ietf.org/id/draft-morton-ippm-more-twamp-02.txt = TWAMP Reflect Padding Feature (Morton and Ciavattone) http://tools.ietf.org/id/draft-morton-ippm-twamp-reflect-padding-00.txt = Individual Session Control Feature for TWAMP (Morton) http://tools.ietf.org/id/draft-morton-ippm-twamp-session-cntrl-00.txt 7. Duplication Metric. (Henk Uijterwaal, 15') 8. Liaison Statement from ITU-T SG 12 (Al Morton, 10') Liaison statement on the Development of a New Performance Method of Measurement for Bandwidth Available in Real Time -BART https://datatracker.ietf.org/liaison/460/ 9. AOB [ Status of Drafts and Milestones ] Duplication Metric in WGLC, one notable issue left (discussed later) Traceroute status, slide correction: It's actually in IESG review TWAMP draft security issues were under discussion, essentially resolved, other comments addressed, looks ready to go (Al) Multimetrics draft WGLC ended, no issues raised - Some sections would really benefit from another pass on the language, but this is not a major blocker + Several sentences are difficult to understand, possibly because they've seen a lot of review and need final massaging + Authors could use concrete directions on what needs fixing + Al Morton: Will go through and address readability issues, but won't be able to so until September + Henk Uijterwaal: Doesn't sound like comments are more than language issues, so draft seems to be in good place. No real rush, fine if this isn't resolved until September. Everyone should consult the IPPM++ BOF notes closely, and a number of issues from there will likely be showing up on the mailing list [ Composition Drafts (Al Morton) ] Two drafts in progress under this topic - Composition Framework: Umbrella draft for all composition drafts - Spatial Composition Metrics: Path metrics Framework has had good readership, good feedback, several iterations Spatial composition has not had much readership or commenting - Though some useful notes on other sources of error were provided - Did not get comments from people who agreed in Chicago to review - This is holding up the framework draft + Want at least one concrete draft first, to check the framework - Group members need to read and comment for draft to go forward! [ Reporting IP Performance Metrics to Users (Stanislav Shalunov) ] Status: On charter, draft was written, lapsed, has been resurrected - Most issues resolved, a few outstanding ones Target: Simple, concrete, entry draft to guide application developers - IPPM has produced many metrics, richly parameterized, but it is difficult for application developers to choose & utilize correctly - Application types of particular interest: VOIP, games, IM, etc + Many report metrics to users (such as delay & loss), but they're ad hoc and not robust measures, making them less useful Five metrics described in draft: Median delay, loss ratio, delay spread, duplication, reordering - Three of these were renamed since the initial draft - Key overview points, to be discussed: + In median delay, loss is considered to be +infinity. If more than half of all packets are lost, delay is +infinity. That is somewhat troubling, but if that's the case, should probably be in a "network down" mode anyway. + Loss ratio is limited to packets timing out in 2 seconds. Almost any application that may care about this does not care about any packets arriving later than that (voip, games, etc), and limit allows app to report metric quickly. + Delay spread, commonly referred to as jitter, defined as statistics for 75th and 25th percentile. If percentiles are too far from the middle, they're too noisy (i.e. 95th, 99th percentile are very unstable). Beyond that, only special value of these two percentiles are that statisticians commonly use them and they're reasonably far from both the ends and the middle. + Duplication defined to not exceed 100% even if more packets are received than sent; does this fit with Duplication Metric writeup? + For reordering, simplest of options in reordering draft was pulled out & used. This needs referencing. Draft needs references to be put in Do we need to block on duplication draft? - Stanislav Shalunov: Have not checked on status of that draft Recommend excluding "network down" periods? - Easy way: If delay spread is +infinite, don't report other metrics - These metrics are short-term, so this may not be a big deal + But if this was to be used for long term intervals then this issue would need to be resolved + Al: Add "short term" to the title, to further emphasize? What statements should draft make about sliding window (continuous last N display) or periodic interface updates? - Stas: Latter clearly a less desirable user experience, but open to discussion about other opinions; many app users in this area are used to continuously updated metrics, albeit metrics used in current practice are too noisy to be always easy to interpret, compare - Al: How often are results updated? Not sure we want to tie into that necessarily. For example, results updated once every ten seconds, then that number should probably jump - Dave Dyson (?): Sliding window relates to sample period; if timeouts are 2 seconds, that's probably reasonable for a reporting period. - Matt XXX: Why do we need to specify or recommend this at all? + Stas: App developers must be able to read this draft, implement, and incorporate into their software without much difficulty or consultation of other documents Discussion on sampling windows & reporting periods (following previous) - Dave: Y.15.41 has content on networks being down, could be used for ideas on reporting periods, etc. I.e., if delay comes up infinite after a few seconds, users will decide that's an unusable network/connection and try again, much like on a cell phone. - Al: ITU-T specification Y.15.40 defines an availability metric for IP: Network is down after 5 minute fixed window with packet loss threshold of 75%. Not very applicable to short term use cases as focused on here. However, with increasing emphasis on IP for realtime use, there has been interest in revisiting that definition. - Stas: IPPM has not in the past defined things like acceptable performance levels, i.e. to indicate "Network Down." No need to start here, especially as it may create conflicting definitions with ITU-T, and may get really complex. It's also unnecessary---if the loss indicator hits 100% then that's a pretty clear indicator of the status of the network, for all users. Discussion about percentiles defined for reporting Delay Spread - Very hard to agree on one set of values to cover all apps + 25% and 75% are common in general statistics, help reduce noise + But may not be relevant, e.g. performance at 90% quartile and above are frequently the relevant statistics for voip, video apps - Have to strike a balance between usability and robustness - Dave: Should look at what telecom providers use. Also, maybe go back to using the term "jitter" as it is more familiar to users? + Joe Kopena: One argument for changing away from "jitter" was that its overloaded in use with many slightly varying definitions, and group wanted to highlight the precise definition being used here. - Stas: If the upper quartile is too high, it essentially becomes a loss metric, i.e. at 99%, if you lose just 1% of the packets then the delay goes to +infinity - Despite difficulty choosing set values, not desirable to allow developers to choose + Want to have a non-parameterized set of metrics so that apps use the same metrics and results are comparable + In that case, shouldn't allow them to change the loss timeout period [ Delay Variation Applicability Statement (Al Morton) ] Lots of comments in last half of 2007, reached working group status - Most comments addressed, incorporated - Several comments on other sources of delay, issues w/ loss + Notes made in the draft - Serious comments made on sections 5, 6, 8, as to their possibly not being in-scope for IPPM or appropriate for an RFC + Authors disagree, feel they collect useful material in one place for the first time, particularly useful for someone new to measurement/IPPM specs. * Al: This draft has high potential to be a lead-in draft to users new to measurement and metrics. + Lars Eggert: Need to reduce text about the process or the group, and focus on the documents and RFCs. - Otherwise, looks about set for WGLC, all open issues closed? + Henk: Will put on list of documents to be last called. [ Mixed Mode Extension for TWAMP (Al Morton) ] 2008: The year of TWAMP! - Lots of people implementing and reading, talking about new features and highlight issues end necessary clarifications OWAMP security modes must match between test and control sessions - Valuable to be able to split the modes and run a secure control protocol but unauthenticated test stream + Control stream fairly small, test stream load perhaps fairly high * Stas: Whether in software or hardware, costs of doing authenticated mode is ridiculously low---checksums on packets are more expensive. * Al: Still hearing from people that mixed mode is something people want, to reduce potential load. - New security mode 8 allows unauthenticated test protocol, authenticated control protocol + Previously asked for 3 modes, but reality is only needed one mode as encrypted control protocol implies authenticated control mode Very simple extension, adds one new mode using existing features - Major other effect here: establishing a mode bit registry + Needed by several other TWAMP feature extensions - Looking to move this forward as a stand-alone working group draft Discussion about TWAMP security - Lars: Formerly there were some strong security concerns about OWAMP, but allowed to go through. Building TWAMP on OWAMP raised those concerns again. Are extensions like this going to raise more concerns? What can be done to make security people happier? - Al: Going to have to ask security people what their concerns are. - Henk, Stas: Do not have records of reservations from security people [ TWAMP Reflect Padding Feature (Al Morton) ] Basic idea: Add optional capability to copy X octets from sent packet padding back to the response - Rationale: Senders right now can't put anything in packet to use for adding functionality, e.g. identifying flows. Adding two additional mode bit extensions, one for reflect padding capability and another for reflect and operate on padding bit - Latter would need some sort of equipment registry + Thinking about putting equipment ID after first two octets of MBZ Might make it mandatory for sender to put more padding in packet as the response is a much larger packet Other idea: Mandate capability to run tests using equal size packets Will take to mailing list for feedback [ Individual Session Control Feature for TWAMP (Al Morton) ] Currently start and stop commands apply to all accepted test sessions - Why not add session identifiers to enable individual controls? Need to add another mode registry to add this - Hence the interest in establishing that in mixed mode security draft Will take to mailing list for feedback [ Duplication Metric (Henk Uijterwaal) ] AD review raised a couple nits, some other issues Big issue: When is a duplicate a duplicate? - Case 1: Sender sends 1 packet, duplicated along way, 2 copies arrive - Case 2: Sender sends 2 identical packets intentionally - Former is clearly duplicating and second is not Current draft assumes packets may be uniquely identified - I.e., in active measurement, packets are going to differentiate by sequence number, etc. in the test information fields + Draft reminds developers to ensure generated packets are unique - For passive measurements this is an issue, no uniqueness guarantees - Can't even rely on IP header identifier being unique, broken on several platforms, sometimes intentionally misused - Hashing and digest techniques won't help if no header or data fields differ, which is possible - Could note this only works if environment produces unique packets - Lars: Focusing draft on active measurements may resolve these issues + Henk: Sounds good + Matt: Passive measurement somewhat out of scope, so should be fine [ Liaison Statement from ITU-T SG 12 (Al Morton) ] Received statement from SG-12, considering taking up some work on measurement of Bandwidth Available in Realtime - Available at https://datatracker.ietf.org/liaison/460/ - No impending need to respond to this, deadline for comments isn't until way out in March 2009 at next SG 12 meeting - Have pointed authors to existing IPPM work in this area + Also discussed previous work on G.CHIRP on assessing download times, which was not followed up on * Of interest, planned to apply models to assess performance, e.g. perception models - People should take a look, put comments out on mailing list Wrote up a draft for multicast performance metrics at last SG-12 meeting - Looked at IGMP, developed metrics on session join times, etc - Similar set of metrics to multimetrics draft, described differently - Measuring multicast performance is rising in interest due to IP TV Some work on Ethernet performance measurements also conducted