Benchmarking Methodology WG (bmwg) TUESDAY, March 21, 2006 1300-1500 Afternoon Session I (Miro) ===================================== CHAIRS: Kevin Dubray <kdubray@juniper.net> Al Morton <acmorton@att.com> MEETING MINUTES/REPORT: Diego Dugatkin graciously agreed to be drafted as minute taker, so his detailed notes, along with Kevin's Jabber Log, http://www.ietf.org/meetings/ietf-logs/bmwg@rooms.jabber.ietf.org/2006-03-21.html were edited into the minutes by the Co-Chairs. About 22 people attended the BMWG session. Many were new to BMWG, and about half of the people attending for the first time had already joined the mailing list. AGENDA: See https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=65 for Slides 1. Extended Working Group Status (Chairs) Network-layer Traffic Control Mechanisms Terms (dsmterm) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-dsmterm-12.txt Submitted to ADs for publication, WG chair shepherding form needs to be distributed on the list Hash and Stuffing Draft (1st WGLC completed, cross-area review sought) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-hash-stuffing-05.txt Benchmarking Terminology for Resource Reservation Capable Routers (WGLC completed without comment - submitted for publication) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-benchres-term-07.txt The latest status of all WG drafts can be viewed here: http://tools.ietf.org/wg/bmwg/ 2. IGP Data plane convergence benchmark I-Ds (Scott Poretsky) WGLC completed with positive comments. http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-meth-10.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-term-10.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-app-10.txt Comments from the WGLC and the subsequent cross-area review (conducted by Sue Hares) have been addressed in the current drafts. The group discussed the reasons to avoid adding an equation describing the processes that contribute to the overall convergence time observed in the dataplane. One such reason is that most of the processes are not externally observable. There will be a short WGLC on -10 versions of these drafts, since there have been changes and 2 new versions since the October 2005 WGLC. 3. Techniques for Benchmarking Core Router Accelerated Life Testing.(Poretsky) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-term-08.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-meth-04.txt There was some progress against Action Items from the last meeting and reflected in the revised terminology draft. Scott is still working on the comments from IETF-64 in the methodology. Several people indicated a preference to progress the terminology and methodology drafts together as a package (there is precedent to do this in other convergence work, and it makes a more straight-forward task for the IESG). Al called for hums to sample opinion to progress these drafts separate/together/don't care, and the group clearly supported "progress together", so that's the path we'll take. 4. Network-layer Traffic Control Mechanisms Methodology (Poretsky) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-dsmmeth-01.txt This was the WG's first opportunity to review this Draft. Scott described the contents in some detail, and asked the WG to provide suggestions for additional test cases, methodologies, application notes, etc. There were several suggestions offered, including: o testing virtual interfaces (maybe in a future methodology) o matching packet size with their usual queue assignments, is this worth pursuing? o there are multiple types of congestion, not just link congestion o impact of enabling different QoS features on forwarding rates o impact of packet classification could be added (cost of marking traffic) There's room for additional authors on this draft, so if you have good ideas for this area, contact Scott. 5. Terms and Methods for Benchmarking IPsec Devices (Merike Kaeo) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-term-08.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-meth-01.txt The Terminology Draft is well-baked, still evolving the methodology. Merike will forward the drafts to IPsec folks for review. There were questions on IPv6 and IPsec extension headers (no need to address them), and methods in paragraph text, rather than in numbered lists (no need to change these, as long as they are unambigous). WG review and feedback is needed! Please read and comment. Merike promises a draft on IKEv2 in time for the next meeting. 6. Milestone Status (Chairs) We're behind on all our milestones, but apparently this is a widespread problem across almost all working groups (according to Bert Wijnen). The proposed milestones on slide 6 of the Co-chair's deck were discussed and agreed by the attendees. Bert also offered that a way to keep drafts on track is to set intermediate dates for review, and get people to sign-up to complete reviews according to the detailed schedule. We may try this in the future. 7. New Work Proposals (Chairs) (I) Sub-IP Protection Mechanisms - (Samir Vapiwala) a. work item proposal We have drafts that implement a harmonized proposal, but there needs to be enthusiastic support to get this off the ground. b. a list of potential non-bmwg reviewers has been identified. c. updates to existing drafts: http://www.ietf.org/internet-drafts/draft-poretsky-protection-term-01.txt http://www.ietf.org/internet-drafts/draft-poretsky-mpls-protection-meth-05.txt d. new, related draft - how does this fit with the proposal? http://tools.ietf.org/wg/bmwg/draft-vapiwala-bmwg-frr-failover-meth-00.txt The chairs presented a recap of discussions of this work item over the last 4 years. To summarize, interest has reached a peak, with new members joining to author and contribute comments on the list. Samir presented the plan to combine the two methodology drafts, making a cogent proposal once again. A large number of attendees had read the draft, and there was clear support for making the drafts a work item, with no objections. The next step is prepare a work proposal for this item and circulate on the list for interest and commitment to review the drafts. (II) IPv6 Benchmarking Methodology - (Ahmed Hamza) This proposal is to provide additional benchmarking guidelines which in conjunction with RFC 2544 will lead to a more complete and realistic evaluation of the IPv6 performance of network elements. Related Draft: http://tools.ietf.org/wg/bmwg/draft-popoviciu-bmwg-ipv6benchmarking-00.txt Ahmed made a case for this new work, citing several IPv6 specific features that are beyond RFC 2544's coverage that are the goals for this effort. The authors shared their draft with Scott Bradner (BMWG chairman emeritus), and he provided comments (which may be shared with the list now). There was a question whether the existing terminology (RFC 1242) would be sufficient, and it was thought to be, unless this effort is expanded into a general maintenance update of RFC 2544. There was a discussion on the list from Jan-Mar 2003 which the authors should review. Kevin Dubray stated that if there is anything wrong with the existing definitions, this is the opportunity to correct them. There were also points raised on: o Is a full /32 address range really needed by BMWG? two people thought not. o Extension headers, are they all needed? Hop-by-hop are not included now. o Tests with extension headers might only test one type at a time. o New methods check the CPU and memory usage, are these necessary? o IMIX - this could be a rathole, the editors agreed to remove it. There was definitely support for this work in the room, but the authors were asked to consider the terminology-related issues raised, and come back to the group with a solid proposal that either includes revising the terminology, adding new terminology, or just a methodology. 8. Kevin Dubray's Final presentation as BMWG Co-chair. After ten years of service to the IETF and BMWG, this was Kevin Dubray's final meeting as Co-chair. For a view of the group's accomplishments over that period, see Kevin's talk (informally titled, "10 years, So What? Lots!!!") http://www3.ietf.org/proceedings/06mar/slides/bmwg-0.ppt at the end of our agenda slide deck. 9. General comments and Action Items (arising during the meeting) Publishing Drafts in the Interim period between meetings is a "good thing", as it avoids the draft landslide at the deadline and potential readers are more likely to find the draft. While reading and commenting is good, implementation experience is "gold". Participants are encouraged to try-out these new methods and share their notes on the list (as Jerry Perser did recently). Since IETF-64, there have been two instances where company standards organizations reigned-in participants after they made a contribution. Participants are reminded that they contribute to the IETF as individuals, and not as representatives of the companies in their e-mail addresses. 1. dsmterm shepherding form to the bmwg-list. (done) 2. Initiate cross-area review on hash and stuffing draft. (done) 3. Initiate WGLC on IGP-Dataplane drafts. (done) 4. Revised Milestones to ADs/secretariat. (done) 5. New work proposal on Protection Benchmarking to the list.