Benchmarking Methodology WG (bmwg) Meeting Notes THURSDAY, November 11, 2010 1300-1500 Afternoon Session I Room: Jade 1 CHAIR(s): Al Morton <acmorton@att.com> [Draft December 14, 2010] This report was prepared by Al Morton, based on detailed notes provided by Sarah Banks as official note taker. Mike Hamilton monitored the Jabber room, enabling comments to be displayed and read during the meeting. Mike also provided a point-to-point Skype chat with Ron Bonica, our AD advisor, who was unable to travel. Over 15 people attended in person, with two people participating remotely. The meeting required 1:35 of the 2 hour timeslot, as one presentation was cancelled at the last minute due to conflict with the IDR working group. This report is divided in two parts: and executive summary with action items, and detailed minutes of the meeting (from Sarah Banks). Summary of the BMWG Session at IETF-79 -------------------------------------- Re-chartering was successful and a welcome outcome for all the well-developed work proposals - all are now on the Charter. Fred Baker presented a new proposal on Dual-Stack Device Benchmarking, principally in the area of connection set-up time with the Happy Eyeballs constraints in mind. There was good discussion that helped the group understand the scope of this effort - ideally observed at Layer 4 (Transport). Expect a Publication Request the RESET draft, following WG Consensus and completion of the Shepherding Questionnaire. The BGP Convergence work has a new draft, with Rajiv Papneja and Sue Hares leading the effort. There was a brief presentation of Data Center Bridging work and Flow Information Export Work. One comment during Re-chartering asked if we could include TRILL RBridges, and we will. Progress was made on the Content-Aware Device benchmarking. A new Appendix begins to address the problem of describing applications-level traffic for the tests. There was good feedback and discussion on the IMIX Genome Proposal, and additional possibilities for specification will be included. Also, there was discussion of the need for an RFC 2544 Applicability Statement, since this Lab-only RFC has been widely deployed for Live network testing. AD-feedback was to make the statement very strong, "Use of RFC 2544 on Live Networks Considered Harmful". There was feedback on the Power benchmarking proposal to coordinate on this topic with the Energy Management Working Group Chairs, to be sure there is no overlapping work in this area. There was also agreement on the need for an RFC 2544 Applicability Statement, even with a stronger title including the sentiment "use outside scope, such as on live networks, considered harmful." BMWG may request an Interim meeting before IETF-80, to progress the new work on the Charter. This would be a virtual meeting, in the February Time-frame. ACTION Items: ------------------ Publication Request the RESET draft, following WG Consensus. - DONE Call to adopt draft-novak-.-ipfix as a WG draft to satisfy milestone. - DONE Coordinate on Power Benchmarking with the Energy Management WG Chairs. - in progress Revised versions of IPsec drafts needed (addressing IESG review). Revised versions of IGP-Dataplane drafts needed (addressing IESG review). WG comments on key issues (see the detailed minutes). Detailed Minutes of the BMWG Session at IETF-79 ----------------------------------------------- BMWG - Thursday November 11, 2011 13:00-15:00 IPR statement made 0. WG Status & Milestones BMWG Activity (Slide 4 in Al's deck) It would be good to bring protection draft out of expired status IPSec - Expired: lots of work could be lost here, no inputs lately BMWG protection/switching approved, waiting behind ISG for references No new RFCs Charter update complete We have a supplementary BMWG page for people who would like to join in our work: http://home.comcast.net/acmacm/BMWG Slide 5: activity. Lists drafts which BMWG could consider making WG drafts to complete new charter items. LDP - expired draft, we have a chartered item for this, need to get it going. Slide 7: we have a standard paragraph that we use for security considerations, that we should be using in the drafts submitted. This ensures readers understand that we're working on lab networks, not live traffic networks Slide 8: current milestones Slide 9: work proposal summary (3 items, all presented below) Agenda: 1. Fred Baker, Eyeball Happiness Addresses draft from an IPv6 operations perspective. Gregory Cauchie (FT) - clarification on Fred's idea. Purpose is to test and determine behaviour on the ability to open sessions/connectivity between devices. Ron Bonica: We're currently testing TCP behaviour, Q: do we want to drive this as an application test or TCP based test? A: appears that transport would be a good place to consolidate across many applications. Al will sign up to help, David Newman might help (per Fred Baker) 2. Basic BGP convergence Skipped over initially, Sue not in room (had a conflict with IDR WG). 3. Content-aware network devices Part of the charter of BMWG, finally. What do we do now? Items needed: Terminology draft Methodology draft Call for volunteers, Gregory Cauchie, France Telecom Outstanding issues: do we define a test case for specific traffic profiles AND leave a way to update them? Sarah's comment: leave it open to the user to record what he used, and use the methodology from the draft to test it. 4. Data center bridging devices Covered by Al, quickly. This item received some comments during re-chartering, and we added a point (in the charter) to investigate implications of TRILL/Rbridges, if any. 5. IP Flow accounting and export Jan Novak couldn't attend in person. Al to cover. Ask working group about using -05 as the version to work on as a WG item. Jan can make it to Prague/next meeting. Hope to wrap this up by then. 6. IMIX Genome Congratulations to Al on his first BMWG working group draft! Sarah: What happens when you test with a packet size not listed on slide 3 Al: Then you'd use frequency of a size, in the table form listed in the draft. Aamer Akter: Thanks for taking it on, it's been a hard time defining what IMIX is, how do you change it, etc. WRT draft, use slide 3 as a suggestion instead of a requirement? Al: it's a non goal to require a particular IMIX, this is a standardization of a reporting method. Aamer: All the old IMIX's are out of date, as cited in the references, etc. Maybe the draft could give advice on how to develop a custom IMIX based on the measurements they make. Aamer will help Al draft the text/paragraph to cover this. 3 ways to specify: standard sizes + genome (both size and sequence are specified) Frequency of a size If you want to do option 1 with your own sizes, then you write them out and record them. Sarah: If option 3 is a super set of option 1, why not just skip to option 3? Al: because having a standard set of sizes, would give the industry a common capability that would be more likely to be reproducible from test bed to test bed. Sarah nodded. Al: Would this help in context of the content-aware draft? Greg, Sarah and Mike have to think about it. 7. Power usage while networking Al presenting Aamer: comments. Vishwas's concern was that there were already groups that had worked on measuring this, wasn't sure. Al: Need to coordinate this a bit with other orgs Defined new metrics, similar to TEEER but more comprehensive. He's computing NECR and NEPI. Action item: look at ITU-T spec (un-named), and check with EMAN chairs. Dan Romescanu: Comment: It's very early work, it's the start of a long and interesting road. Question: Do we have any kind of testing of behaviour of devices under power constraints requirement? Al: what do you mean by operation of device in a constrained power env? Like a brown out? Dan: first I get an indication of power levels, then when falling under a threshold I get a beep, some then die graciously and others die suddenly. Question is, when they start dying, is there a last gasp or is there a mode of power savings that will make a critical part of the functionality useful, and so on. Can we consider this topic in the draft as well? Aamer: doesn't have the answer to the q but does know that on certain classes of routers we see the power consumption increases. In data center there's a lot of work going on in this area, for eg, VM's move when there's a certain level of power or temperature exceeded. AOB 2 items: 1. A lot of authors couldn't make it to Beijing; proposes to have a virtual interim meeting in the February time frame, helping to move things along and deal with new charter items. 2. RFC 2544 is getting dragged through the live network testing mud everywhere. Maybe it's time to have a 1 page applicability statement that brings 2544 and connects it with our charter. It also makes it clear to people that we have a BMWG private address space - meant to use in the lab in the v4/v6 world. Dan: Al should be less nice and less civilized, and we should use this document and call it "using RFC 2544 on live networks is harmful". Al: Declaring WG consensus on reset draft after a silent 2nd WGLC, Publication Request will follow shortly. Ended 20 minutes early - Basically the time for the BGP topic/material that was not presented. EOT