Benchmarking Methodology (bmwg)

This Working Group did not meet

NOTE: This charter is a snapshot of the 59th IETF Meeting in Seoul, Korea. It may now be out-of-date.

Last Modified: 2004-02-12

Chair(s):
Kevin Dubray <kdubray@juniper.net>
Al Morton <acmorton@att.com>
Operations and Management Area Director(s):
Bert Wijnen <bwijnen@lucent.com>
David Kessens <david.kessens@nokia.com>
Operations and Management Area Advisor:
David Kessens <david.kessens@nokia.com>
Mailing Lists:
General Discussion: bmwg@ietf.org
To Subscribe: bmwg-request@ietf.org
In Body: subscribe your_email_address
Archive: ftp://ftp.ietf.org/ietf-mail-archive/bmwg/
Description of Working Group:
The major goal of the Benchmarking Methodology Working Group is to make a series of recommendations concerning the measurement of the performance characteristics of various internetworking technologies; further, these recommendations may focus on the systems or services that are built from these technologies.

Each recommendation will describe the class of equipment, system, or service being addressed; discuss the performance characteristics that are pertinent to that class; clearly identify a set of metrics that aid in the description of those characteristics; specify the methodologies required to collect said metrics; and lastly, present the requirements for the common, unambiguous reporting of benchmarking results.

To better distinguish the BMWG from other measurement initiatives in the IETF, the scope of the BMWG is limited to technology characterization using simulated stimuli in a laboratory environment. Said differently, the BMWG does not attempt to produce benchmarks for live, operational networks. Moreover, the benchmarks produced by this WG shall strive to be vendor independent or otherwise have universal applicability to a given technology class.

Because the demands of a particular technology may vary from deployment to deployment, a specific non-goal of the Working Group is to define acceptance criteria or performance requirements.

An ongoing task is to provide a forum for discussion regarding the advancement of measurements designed to provide insight on the operation internetworking technologies.

Goals and Milestones:
Done  Expand the current Ethernet switch benchmarking methodology draft to define the metrics and methodologies particular to the general class of connectionless, LAN switches.
Done  Edit the LAN switch draft to reflect the input from BMWG. Issue a new version of document for comment. If appropriate, ascertain consensus on whether to recommend the draft for consideration as an RFC.
Done  Take controversial components of multicast draft to mailing list for discussion. Incorporate changes to draft and reissue appropriately.
Done  Submit workplan for initiating work on Benchmarking Methodology for LAN Switching Devices.
Done  Submit workplan for continuing work on the Terminology for Cell/Call Benchmarking draft.
Done  Submit initial draft of Benchmarking Methodology for LAN Switches.
Done  Submit Terminology for IP Multicast Benchmarking draft for AD Review.
Done  Submit Benchmarking Terminology for Firewall Performance for AD review
Done  Progress ATM benchmarking terminology draft to AD review.
Done  Submit Benchmarking Methodology for LAN Switching Devices draft for AD review.
Done  Submit first draft of Firewall Benchmarking Methodology.
Done  First Draft of Terminology for FIB related Router Performance Benchmarking.
Done  First Draft of Router Benchmarking Framework
Done  Progress Frame Relay benchmarking terminology draft to AD review.
Done  Methodology for ATM Benchmarking for AD review.
Done  Terminology for ATM ABR Benchmarking for AD review.
Done  Terminology for FIB related Router Performance Benchmarking to AD review.
Done  Firewall Benchmarking Methodology to AD Review
Done  First Draft of Methodology for FIB related Router Performance Benchmarking.
Done  First draft Net Traffic Control Benchmarking Methodology.
Done  Methodology for IP Multicast Benchmarking to AD Review.
Mar 03  Resource Reservation Benchmarking Terminology to AD Review
Done  First I-D on IPsec Device Benchmarking Terminology
Apr 03  Net Traffic Control Benchmarking Terminology to AD Review
Apr 03  Methodology for FIB related Router Performance Benchmarking to AD review.
Apr 03  EGP Convergence Benchmarking Terminology to AD Review
Jul 03  Resource Reservation Benchmarking Methodology to AD Review
Jul 03  Basic BGP Convergence Benchmarking Methodology to AD Review.
Dec 03  Net Traffic Control Benchmarking Methodology to AD Review.
Dec 03  IPsec Device Benchmarking Terminology to AD Review
Internet-Drafts:
  • - draft-ietf-bmwg-mcastm-14.txt
  • - draft-ietf-bmwg-dsmterm-09.txt
  • - draft-ietf-bmwg-benchres-term-04.txt
  • - draft-ietf-bmwg-conterm-05.txt
  • - draft-ietf-bmwg-ospfconv-term-07.txt
  • - draft-ietf-bmwg-ospfconv-intraarea-07.txt
  • - draft-ietf-bmwg-ospfconv-applicability-04.txt
  • - draft-ietf-bmwg-ipsec-term-03.txt
  • - draft-ietf-bmwg-igp-dataplane-conv-meth-02.txt
  • - draft-ietf-bmwg-igp-dataplane-conv-term-02.txt
  • - draft-ietf-bmwg-igp-dataplane-conv-app-02.txt
  • - draft-ietf-bmwg-acc-bench-term-02.txt
  • - draft-ietf-bmwg-acc-bench-framework-00.txt
    • Request For Comments:
    RFCStatusTitle
    RFC1242 I Benchmarking Terminology for Network Interconnection Devices
    RFC1944 I Benchmarking Methodology for Network Interconnect Devices
    RFC2285 I Benchmarking Terminology for LAN Switching Devices
    RFC2432 I Terminology for IP Multicast Benchmarking
    RFC2544 I Benchmarking Methodology for Network Interconnect Devices
    RFC2647 I Benchmarking Terminology for Firewall Performance
    RFC2761 I Terminology for ATM Benchmarking
    RFC2889 I Benchmarking Methodology for LAN Switching Devices
    RFC3116 I Methodology for ATM Benchmarking
    RFC3133 I Terminology for Frame Relay Benchmarking
    RFC3134 I Terminology for ATM ABR Benchmarking
    RFC3222 I Terminology for Forwarding Information Base (FIB) based Router Performance
    RFC3511 I Benchmarking Methodology for Firewall Performance

    Current Meeting Report

    None received.

    Slides

    None received.