Benchmarking Methodology (BMWG)

Charter 

Chair(s)

      Jim McQuaid:  mcquaid@wg.com

Mailing Lists

      General Discussion:  bmwg@harvard.edu
      To Subscribe:        bmwg-request@harvard.edu
      Archive: 

Description of Working Group

     The major goal of the Benchmarking Methodology Working Group is
     to make a series of recommendations concerning the measurement
     of the performance characteristics of different classes of
     network equipment and software services.

     Each recommendation will describe the class of equipment or
     service, discuss the performance characteristics that are
     pertinent to that class, specify a suite of performance
     benchmarks that test the described characteristics, as well as
     specify the requirements for common reporting of benchmark
     results.

     Classes of network equipment can be broken down into two broad
     categories.  The first deals with stand-alone network devices
     such as routers, bridges, repeaters, and LAN wiring
     concentrators.  The second category includes host-dependent
     equipment and services, such as network interfaces or TCP/IP
     implementations.

     Once benchmarking methodologies for stand-alone devices have
     matured sufficiently, the group plans to focus on methodologies
     for testing system-wide performance, including issues such as
     the responsiveness of routing algorithms to topology changes.

Goals and Milestones

TBD      Once the community has had time to comment on the definitions
         of devices and performance criteria, a second document will be
         issued.  This document will make specific recommendations
         regarding the suite of benchmark performance tests for each of
         the defined classes of network devices.

Done     The document will also define various classes of stand-alone
         network devices such as repeaters, bridges, routers, and LAN
         wiring concentrators as well as detail the relative importance
         of various performance criteria within each class.

Done     Issue a document that provides a common set of definitions for
         performance criteria, such as latency and throughput.


Request For Comments


RFC 1242   ``Benchmarking Terminology for Network Interconnection
         Devices''