Network Working Group INTERNET-DRAFT Expires in: December 2003 Scott Poretsky Avici Systems Shankar Rao Qwest Communications Ray Piatt Cable and Wireless June 2003 Terminology for Benchmarking Core Router Software Accelerated Life Testing Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. ABSTRACT This terminology document provides the terms to be used for benchmarking router software under accelerated stress conditions. A framework is defined to configure routing protocols, security policies, traffic forwarding, and management. Conditions to produce instability and accelerate operational conditions are also defined. Benchmarks for evaluating a router subjected to the accelerated life test are introduced. The DUT configuration and accelerated stress conditions emulate those of Internet Core routers. Table of Contents 1. Introduction ............................................... 2 Poretsky, Rao, Piatt [Page 1] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing 2. Existing definitions ....................................... 3 3. Term definitions............................................ 3 3.1 Configuration Modules..................................... 3 3.1.1 Control Plane Configuration Module.................... 4 3.1.2 Data Plane Configuration Module....................... 5 3.1.3 Management Configuration Module....................... 6 3.1.4 Security Configuration Module......................... 7 3.2 Configuration Sets...................................... 7 3.2.1 Control Plane Configuration Sets...................... 7 3.2.1.1 Routing Protocol Configuration Set.................. 7 3.2.1.2 Multicast Protocol Configuration Set................ 8 3.2.1.3 MPLS Protocol Configuration Set..................... 9 3.2.2 Data Plane Configuration Set.......................... 9 3.2.3 Management Configuration Sets.........................10 3.2.3.1 User Access Configuration Set.......................10 3.2.3.2 SNMP Configuration Set..............................10 3.2.3.3 Logging/Debug Configuration Set.....................11 3.2.3.4 Packet Statistics Collector Configuration...........11 3.2.4 Security Configuration Set............................11 3.2.4.1 ACL Configuration Set.................................12 3.3 Instability Conditions..................................12 3.3.1 Interface Shutdown Cycling Rate.......................12 3.3.2 BGP Route Flap Rate...................................13 3.3.3 IGP Route Flap Rate...................................13 3.3.4 Route Convergence Due to Better Next-Hop..............14 3.3.5 LSP Reroute Rate......................................14 3.3.6 Intended Test Duration................................15 3.4 Evaluation Benchmarks...................................15 3.4.1 Run-Time without Error................................15 3.4.2 Run-Time without Control Plane Error..................16 3.4.3 Run-Time without Data Plane Error.....................16 3.4.4 Run-Time without Management Plane Error...............17 3.4.5 Run-Time without Security Plane Error.................17 4. Security Considerations.....................................18 5. References..................................................18 6. Author's Address............................................18 7. Full Copyright Statement....................................19 Appendix 1 - White Box Benchmarks..............................19 1. Introduction Router testing benchmarks have consistently been made in a monolithic fashion in which a single protocol or behavior is measured in an isolated environment. It is important to know the limits for a router/switch's (hereby referred to as Router) behavior for each protocol, however this does not produce a reliable benchmark of the router's behavior in a deployed network. Routers in an operational network are simultaneously configured with multiple protocols and security policies while forwarding traffic and being managed. To accurately benchmark a router for deployment it is necessary to test that router in operational conditions by simultaneously configuring the network protocols and security policies, sourcing traffic, and managing the router. It is helpful Poretsky, Rao, Piatt [Page 2] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing to accelerate these network operational conditions so that the router under test can be benchmarked with faster test duration. Testing a router in accelerated network conditions is known as Accelerated Life Testing. White Box benchmarks are defined in Appendix 1. Accelerated Life Testing of Routers provides the following benefits: 1. Evaluation of multiple protocols enabled simultaneously as configured in deployed networks 2. Evaluation of System and Software Stability 3. Evaluation of Manageability under stressful conditions 4. Identification of Software Coding bugs such as: a. Memory Leaks b. Suboptimal CPU Utilization c. Coding Logic These evaluations are externally observable to the DUT as control plane or data plane errors. Configuration and operational conditions vary for Core, Peering, Aggregate, and Edge Routers. This document focuses on defining the terminology for Accelerated Life Testing of Core Routers. Terminology for configuring network conditions, accelerating operational parameters, accelerating network instability, and evaluating results are provided. 2. Existing definitions RFC 1242 "Benchmarking Terminology for Network Interconnect Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching Devices" should be consulted before attempting to make use of this document. For the sake of clarity and continuity this RFC adopts the template for definitions set out in Section 2 of RFC 1242. Definitions are indexed and grouped together in sections for ease of reference. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Term definitions 3.1 Configuration Modules Definition: The features and protocols enabled for the Accelerated Life Test and conditions created during the test. Discussion: Features and Protocols are divided into four configuration modules as shown in Figure 1. These are the Control Plane, Data Plane, Management Plane, and Security Plane. Poretsky, Rao, Piatt [Page 3] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Definitions for each of these modules are provided in the following sections. ___________ ___________ | Control | | Management| | Plane |___ ___| Plane | | Module | | | | Module | ----------- | | ----------- \/ \/ ___________ ___________ | Security | | |<-----------| Plane | | DUT | | Module | |--->| |<---| ----------- | ----------- | | | | ___________ | | | Data | | |--->| Plane |<---| | Module | ----------- Figure 1. Configuration Modules Measurement units: N/A Issues: None See Also: Control Plane Configuration Module Data Plane Configuration Module Management Configuration Module Security Configuration Module 3.1.1 Control Plane Configuration Module Definition: The control protocols enabled for the Accelerated Life Test. Discussion: Control Plane Configuration Module is divided into three Configuration Sets as shown in Figure 2. These are Routing Protocol, Multicast, and MPLS. These can be enabled or disabled for a benchmark test. For example, a network operator that operates a network with BGP and an IGP without Multicast and MPLS protocols can execute a benchmark test with the Routing Protocol Configuration Set enabled and the Multicast and MPLS Configuration Sets disabled. Configuration Sets are further explained in the next section. Measurement units: Enabled/Disabled Issues:None Poretsky, Rao, Piatt [Page 4] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing ____________ ____________ ____________ | Routing | | Multicast | | MPLS | | Protocol |___ | Protocol | __| Protocol | | Config Set | | | Config Set | | | Config Set | ------------ | ------------ | ------------ | | | | | | | \/ | | ___________ | | | | | |------->| DUT |<------| | | ----------- Figure 2. Control Plane Configuration Module See Also: Routing Protocol Configuration Set Multicast Protocol Configuration Set MPLS Protocol Configuration Set 3.1.2 Data Plane Configuration Module Definition: The data traffic and interfaces enabled for the Accelerated Life Test. Discussion: Accelerated Life Testing includes protocols, as defined by the Control Plane Configuration Module, and data forwarding, as defined by the Data Plane Configuration Module. As shown in Figure 3, the Data Plane Configuration Module has a corresponding Traffic Profile Configuration Set. ___________ | Traffic | | Profile |___ | Config Set| | ----------- | \/ ____________ | | | DUT | | | ------------ Measurement Units: Enabled/Disabled See Also: Traffic Profile Configuration Set System Configuration Set Poretsky, Rao, Piatt [Page 5] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing 3.1.3 Management Configuration Module Definition: The router management features enabled for the Accelerated Life Test. Discussion: A key component of the Accelerated Life Test is the Management Configuration Module to assess manageability of the router under stress. The Management Configuration Module can be divided into User Access Configuration Set, SNMP Configuration Set, Logging/Debug Configuration Set, and Packet Statistics Collector Configuration Set as shown in Figure 4. ____________ ____________ ____________ | User | | | | Logging/ | | Access | | SNMP | __| Debug | | Config Set | | Config Set | | | Config Set | ------------ ------------ | ------------ | | | | | | | \/ | | ___________ | | | | | |------->| DUT |<-----| | | ----------- | | \/ ___________ | Packet | | Statistics| | Collector | | Config Set| ----------- Figure 4. Management Configuration Module Measurement units: Enabled/Disabled Issues: None See Also: User Access Configuration Set SNMP Configuration Set Logging/Debug Configuration Set Packet Statistics Collector Configuration Set Poretsky, Rao, Piatt [Page 6] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing 3.1.4 Security Configuration Module Definition: Security features enabled for the Accelerated Life Test. Discussion: Accelerated Life Testing of Core Router can include configuration of Security features. The Security Configuration Module includes the ACL Configuration Set, Secure Protocol Configuration Set, and User Login Configuration Set as shown in Figure 5. ____________ ____________ ____________ | | | Secure | | User | | ACL | | Protocol | __| Login | | Config Set | | Config Set | | | Config Set | ------------ ------------ | ------------ | | | | | | | \/ | | ___________ | | | | | |-------> | DUT |<--------| | | ----------- Figure 5. Security Configuration Module Measurement units: Enabled/Disabled Issues: None See Also: ACL Configuration Set Secure Protocol Configuration Set Password Login Configuration Set 3.2 Configuration Sets 3.2.1 Control Plane Configuration Sets 3.2.1.1 Routing Protocol Configuration Set Definition: Table of Routing Protocols configuration for Accelerated Life Test. Discussion: The Routing Protocol Configuration Set is one of the Control Plane Configuration Sets BGP, MBGP, ISIS, OSPF are the routing protocols that can be enabled or disabled in any combination. The specific protocol parameters used to Poretsky, Rao, Piatt [Page 7] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing establish the test conditions are also provided in the table. Measurement units: PARAMETER UNITS BGP Enabled/Disabled Number of EBGP Peers Peers Number of IBGP Peers Peers Number of BGP Route Instances Routes Number of BGP Installed Routes Routes MBGP Enabled/Disabled Number of MBGP Route Instances Routes Number of MBGP Installed Routes Routes ISIS Enabled/Disabled ISIS-TE Enabled/Disabled Number of ISIS Adjacencies Adjacencies Number of ISIS Routes Routes Number of Nodes per Area Nodes OSPF Enabled/Disabled OSPF-TE Enabled/Disabled Number of OSPF Adjacencies Adjacencies Number of OSPF Routes Routes Number of Nodes per Area Nodes Issues: None See Also: Control Plane Configuration Module Multicast Protocol Configuration Set MPLS Protocol Configuration Set 3.2.1.2 Multicast Protocol Configuration Set Definition: Table of Multicast Protocols configuration for Accelerated Life Test. Discussion: The Multicast Protocol Configuration Set is one of the Control Plane Configuration Sets PIM-SM and MSDP are the routing protocols that can be enabled or disabled in any combination. The specific protocol parameters used to establish the test conditions are also provided in the table. Measurement units: PARAMETER UNITS PIM-SM Enabled/Disabled RP Enabled/Disabled Number of Multicast Groups Groups Poretsky, Rao, Piatt [Page 8] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing MSDP Enabled/Disabled Issues: None See Also: Control Plane Configuration Module Routing Protocol Configuration Set MPLS Protocol Configuration Set 3.2.1.3 MPLS Protocol Configuration Set Definition: Table of MPLS Protocols configuration for Accelerated Life Test. Discussion: The MPLS Protocol Configuration Set is one of the Control Plane Configuration Sets MPLS-TE (RSVP-TE, ISIS-TE, OSPF-TE) and LDP are the routing protocols that can be enabled or disabled in any combination. The specific protocol parameters used to establish the test conditions are also provided in the table. Measurement units: PARAMETER UNITS MPLS-TE Number of Ingress Tunnels Tunnels Number of Mid-Point Tunnels Tunnels Number of Egress Tunnels Tunnels LDP Number of Sessions Sessions Number of FECs FECs Issues: None See Also: Control Plane Configuration Module Routing Protocol Configuration Set Multicast Protocol Configuration Set 3.2.2 Data Plane Configuration Sets 3.2.2.1 Data Plane Configuration Set Definition: Table of Forwarded Traffic configuration for Accelerated Life Test. Discussion: Measurement units: Poretsky, Rao, Piatt [Page 9] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing PARAMETER UNITS Traffic Forwarding Enabled/Disabled Packet Size Byte Size Distribution Number of Flows Flows Offered Load bps (or pps) Aggregate Number of Interfaces Interfaces Issues: None See Also: Data Plane Configuration Module 3.2.3 Management Configuration Sets 3.2.3.1 User Access Configuration Set Definition: Table of User Access methods during Accelerated Life Test. Discussion: Measurement units: PARAMETER UNITS Telnet Rate Sessions/Hour FTP Rate Sessions/Hour Concurrent Telnet Sessions Sessions Concurrent FTP Session Sessions SSH Enabled/Disabled RADIUS Enabled/Disabled TACACS Enabled/Disabled Issues: None See Also: Management Configuration Module SNMP Configuration Set Logging/Debug Configuration Set Packet Statistics Collector Configuration Set 3.2.3.2 SNMP Configuration Set Definition: Table of SNMP Configuration for the Accelerated Life Test. Discussion: Measurement units: SNMP GET Rate SNMP Gets/minute Issues: None Poretsky, Rao, Piatt [Page 10] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing See Also: Management Configuration Module User Access Configuration Set Logging/Debug Configuration Set Packet Statistics Collector Configuration Set 3.2.3.3 Logging/Debug Configuration Set Definition: Table of Logging and Debug configuration for the Accelerated Life Test. Discussion: Measurement units: Logging Enabled/Disabled Protocol Debug Enabled/Disabled Issues: None See Also: Management Configuration Module User Access Configuration Set SNMP Configuration Set Packet Statistics Collector Configuration Set 3.2.3.4 Packet Statistics Collector Configuration Set Definition: Table of Packet Statistics Collection for the Accelerated Life Test. Discussion: Measurement units: Packet Statistics Collector Enabled/Disabled Sampling Rate X:1 packets Issues: None See Also: Management Configuration Module User Access Configuration Set SNMP Configuration Set Logging/Debug Configuration Set 3.2.4 Security Configuration Sets Poretsky, Rao, Piatt [Page 11] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing 3.2.4.1 ACL Configuration Set Definition: Table of ACL configuration for the Accelerated Life Test Discussion: Measurement units: Access-Control Lists Enabled/Disabled Number of ACLs ACLs ACL Depth ACEs Issues: None See Also: Security Configuration Module 3.3 Instability Conditions Definition: Test conditions that occur during the Accelerated Life Test that are typical of instability in an operational network. These conditions occur at a much greater rate during the test. Discussion: Configuration Modules and Configuration Sets establish the configuration of the DUT for the Accelerated Life Test. Instability Conditions are events that occur during the Accelerated Life Test to stress the router. Measurement units: N/A Issues: None See Also: Interface Shutdown Cycling Rate BGP Route Flap Rate IGP Route Flap Rate Route Convergence Due to Better Next-Hop LSP Reroute Rate Target Run Time 3.3.1 Interface Shutdown Rate Definition: The rate at which physical interfaces are shutdown on the DUT. Discussion: Poretsky, Rao, Piatt [Page 12] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing This instability condition is an administrative shutdown or remote loss of carrier for every interface of the DUT. This produces great instability on the DUT due to continuous protocol session loss and convergence. Measurement units: number of interface shutdowns per minute Issues: None See Also: Instability Conditions Route Flap Rate Target Run Time 3.3.2 BGP Route Flap Rate Definition: The rate at which BGP routes installed in the route table are flapped. Discussion: This instability condition defines the rate of repeated installation, withdrawal,and installation of routes installed in the route table. This produces great instability on the DUT due to continuous changes to the forwarding table. Measurement units: routes flapped per second Issues: None See Also: Instability Conditions Interface Shutdown Cycling Rate Target Run Time 3.3.3 IGP Route Flap Rate Definition: The rate at which IGP routes installed in the route table are flapped. Discussion: This instability condition defines the rate of repeated installation, withdrawal,and installation of routes installed in the route table. This produces great instability on the DUT due to continuous changes to the forwarding table. Poretsky, Rao, Piatt [Page 13] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Measurement units: routes flapped per second Issues: None See Also: Instability Conditions Interface Shutdown Cycling Rate Target Run Time 3.3.4 Route Convergence Due to Better Next-Hop Definition: The rate at which routes in the FIB are replaced by a route instance in the RIB with a different next-hop becoming more preferred. Discussion: The route changes are due to attribute and TLV changes. RIB contains routes matching those in the FIB. Convergence due to better path can occur for BGP and IGP routes. Measurement units: more preferred routes per second Issues: None See Also: Instability Conditions Interface Shutdown Cycling Rate Target Run Time 3.3.5 LSP Reroute Rate Definition: The rate at which established LSPs experience a failure condition and are forced to reroute. Discussion: This instability condition defines the rate of repeated LSP reroutes. This produces great instability on the DUT due to continuous tunnel path calculations and changes to the forwarding table. Rerouting can be performed with Headend Reroute, Standby LSP, or Fast Reroute. Measurement units: routes flapped per second Issues: None Poretsky, Rao, Piatt [Page 14] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing See Also: Instability Conditions Interface Shutdown Cycling Rate Target Run Time 3.3.6 Intended Test Duration Definition: The maximum targetted run time for Accelerated Life Test. Discussion: This is an instability condition because the protocol sessions and scaling values must be maintained for this specified time with interfaceshutdowns and route flapping. The test may be halted upon reaching the Intended Test Duration. Measurement units: Hours Issues: None See Also: Instability Conditions Interface Shutdown Cycling Rate Route Flap Rate 3.4 Evaluation Benchmarks 3.4.1 Run-Time without Error Definition: The run-time for the Accelerated Life Test without the DUT exhibiting an error in the control plane, data plane, management plane, or security plane. Discussion: For a successful test, the Run-Time Without Error will equal the Intended Test Duration. In the event of any failure, the Run-Time Without Error is less than the Intended Test Duration and will equal the maximum of the Run-Time without Control Plane Error, Run-Time without Data Plane Error, Run-Time without Management Plane Error, and Run-Time without Security Plane Error. Measurement units: Hours and Minutes Issues: None See Also: Poretsky, Rao, Piatt [Page 15] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Intended Test Duration Run-Time without Control Plane Error Run-Time without Data Plane Error Run-Time without Management Plane Error Run-Time without Security Plane Error 3.4.2 Run-Time without Control Plane Error Definition: The run-time for the Accelerated Life Test without the DUT exhibiting an error in the control plane to Routing, Multicast, and MPLS Protocol. Discussion: For a successful test, the Run-Time Without Control Plane Error will equal the Run-Time without Error. In the event of a Control Plane failure, the Run-Time Without Control Plane Error is less than the Run-Time without Error. Measurement units: Hours and Minutes Issues: None See Also: Intended Test Duration Run-Time without Error Run-Time without Data Plane Error Run-Time without Management Plane Error Run-Time without Security Plane Error 3.4.3 Run-Time without Data Plane Error Definition: The run-time for the Accelerated Life Test without the DUT exhibiting an error in the Data Plane for traffic forwarding or DUT interfaces. Discussion: For a successful test, the Run-Time Without Data Plane Error will equal the Run-Time without Error. In the event of a Data Plane failure, the Run-Time Without Data Plane Error is less than the Run-Time without Error. Measurement units: Hours and Minutes Issues: None See Also: Poretsky, Rao, Piatt [Page 16] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Intended Test Duration Run-Time without Error Run-Time without Control Plane Error Run-Time without Management Plane Error Run-Time without Security Plane Error 3.4.4 Run-Time without Management Plane Error Definition: The run-time for the Accelerated Life Test without the DUT exhibiting an error in the Management Plane to User Access, SNMP, or Logging/Debug. Discussion: For a successful test, the Run-Time Without Management Plane Error will equal the Run-Time without Error. In the event of a Management Plane failure, the Run-Time Without Management Plane Error is less than the Run-Time without Error. Measurement units: Hours and Minutes Issues: None See Also: Intended Test Duration Run-Time without Error Run-Time without Control Plane Error Run-Time without Data Plane Error Run-Time without Security Plane Error 3.4.5 Run-Time without Security Plane Error Definition: The run-time for the Accelerated Life Test without the DUT exhibiting an error in the Security Plane to ACLs. Discussion: For a successful test, the Run-Time Without Security Plane Error will equal the Run-Time without Error. In the event of a Security Plane failure, the Run-Time Without Control Plane Error is less than the Run-Time without Error. Measurement units: Hours and Minutes Issues: None See Also: Intended Test Duration Poretsky, Rao, Piatt [Page 17] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Run-Time without Error Run-Time without Control Plane Error Run-Time without Data Plane Error Run-Time without Management Plane Error 4. Security Considerations Documents of this type do not directly effect the security of the Internet or of corporate networks as long as benchmarking is not performed on devices or systems connected to operating networks. 5. References [1] Bradner, S., Editor, "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991. [2] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, June 1998. [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999. [4] "Core Router Evaluation for Higher Availability", Scott Poretsky, NANOG 25, June 8, 2002, Toronto, CA. [5] "Router Stress Testing to Validate Readiness for Network Deployment", Scott Poretsky, IEEE CQR 2003. 6. Author's Address Scott Poretsky Avici Systems 101 Billerica Avenue N. Billerica, MA 01862 USA Phone: + 1 978 964 2287 EMail: sporetsky@avici.com Shankar Rao Qwest Communications Denver, CO USA Phone: + 1 303 437 6643 Email: srao@qwest.net Ray Piatt Cable and Wireless 11700 Plaza America Drive Reston, VA 20190 USA Poretsky, Rao, Piatt [Page 18] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Phone: + 1 703 292 2113 Email: rpiatt@cw.net 7. Full Copyright Statement Copyright (C) The Internet Society (1998). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Appendix 1. White Box Benchmarking Terminology Appendix 1.1 Minimum Available Memory Definition: Minimum DUT Available Memory during the duration of the Accelerated Life Test. Discussion: It is necessary to monitor DUT memory to measure this benchmark. Measurement units: bytes Issues: None See Also: Maximum CPU Utilization Poretsky, Rao, Piatt [Page 19] INTERNET-DRAFT Terminology for Benchmarking Software Core June 2003 Router Software Accelerated Life Testing Appendix 1.2 Maximum CPU Utilization Definition: Maximum DUT CPU utilization during the duration of the Accelerated Life Test. Discussion: It is necessary to monitor DUT CPU Utilization to measure this benchmark. Measurement units: % Issues: None See Also: Minimum Available Memory Poretsky, Rao, Piatt [Page 20]