idnits 2.17.1 draft-ietf-bmwg-acc-bench-term-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard == It seems as if not all pages are separated by form feeds - found 0 form feeds but 20 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. (A line matching the expected section header was found, but with an unexpected indentation: ' 1. Introduction' ) ** The document seems to lack a Security Considerations section. (A line matching the expected section header was found, but with an unexpected indentation: ' 4. Security Considerations' ) ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack an Authors' Addresses Section. ** There are 482 instances of lines with control characters in the document. ** The abstract seems to contain references ([2], [3], [4], [5], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (December 2003) is 7437 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 929 looks like a reference -- Missing reference section? '2' on line 932 looks like a reference -- Missing reference section? '3' on line 935 looks like a reference -- Missing reference section? '4' on line 938 looks like a reference -- Missing reference section? '5' on line 941 looks like a reference Summary: 7 errors (**), 0 flaws (~~), 4 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group 2 INTERNET-DRAFT 3 Expires in: December 2003 4 Scott Poretsky 5 Avici Systems 7 Shankar Rao 8 Qwest Communications 10 Ray Piatt 11 Cable and Wireless 13 June 2003 15 Terminology for Benchmarking 16 Core Router Software Accelerated Life Testing 18 20 Status of this Memo 22 This document is an Internet-Draft and is in full conformance with 23 all provisions of Section 10 of RFC2026. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF), its areas, and its working groups. Note that 27 other groups may also distribute working documents as Internet- 28 Drafts. 30 Internet-Drafts are draft documents valid for a maximum of six 31 months and may be updated, replaced, or obsoleted by other 32 documents at any time. It is inappropriate to use Internet-Drafts 33 as reference material or to cite them other than as "work in 34 progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html. 42 ABSTRACT 43 This terminology document provides the terms to be used for 44 benchmarking router software under accelerated stress conditions. 45 A framework is defined to configure routing protocols, security 46 policies, traffic forwarding, and management. Conditions to 47 produce instability and accelerate operational conditions are 48 also defined. Benchmarks for evaluating a router subjected to 49 the accelerated life test are introduced. The DUT configuration 50 and accelerated stress conditions emulate those of Internet 51 Core routers. 53 Table of Contents 54 1. Introduction ............................................... 2 55 Router Software Accelerated Life Testing 56 2. Existing definitions ....................................... 3 57 3. Term definitions............................................ 3 58 3.1 Configuration Modules..................................... 3 59 3.1.1 Control Plane Configuration Module.................... 4 60 3.1.2 Data Plane Configuration Module....................... 5 61 3.1.3 Management Configuration Module....................... 6 62 3.1.4 Security Configuration Module......................... 7 63 3.2 Configuration Sets...................................... 7 64 3.2.1 Control Plane Configuration Sets...................... 7 65 3.2.1.1 Routing Protocol Configuration Set.................. 7 66 3.2.1.2 Multicast Protocol Configuration Set................ 8 67 3.2.1.3 MPLS Protocol Configuration Set..................... 9 68 3.2.2 Data Plane Configuration Set.......................... 9 69 3.2.3 Management Configuration Sets.........................10 70 3.2.3.1 User Access Configuration Set.......................10 71 3.2.3.2 SNMP Configuration Set..............................10 72 3.2.3.3 Logging/Debug Configuration Set.....................11 73 3.2.3.4 Packet Statistics Collector Configuration...........11 74 3.2.4 Security Configuration Set............................11 75 3.2.4.1 ACL Configuration Set.................................12 76 3.3 Instability Conditions..................................12 77 3.3.1 Interface Shutdown Cycling Rate.......................12 78 3.3.2 BGP Route Flap Rate...................................13 79 3.3.3 IGP Route Flap Rate...................................13 80 3.3.4 Route Convergence Due to Better Next-Hop..............14 81 3.3.5 LSP Reroute Rate......................................14 82 3.3.6 Intended Test Duration................................15 83 3.4 Evaluation Benchmarks...................................15 84 3.4.1 Run-Time without Error................................15 85 3.4.2 Run-Time without Control Plane Error..................16 86 3.4.3 Run-Time without Data Plane Error.....................16 87 3.4.4 Run-Time without Management Plane Error...............17 88 3.4.5 Run-Time without Security Plane Error.................17 89 4. Security Considerations.....................................18 90 5. References..................................................18 91 6. Author's Address............................................18 92 7. Full Copyright Statement....................................19 93 Appendix 1 - White Box Benchmarks..............................19 95 1. Introduction 97 Router testing benchmarks have consistently been made in a 98 monolithic fashion in which a single protocol or behavior is 99 measured in an isolated environment. It is important to know the 100 limits for a router/switch's (hereby referred to as Router) behavior 101 for each protocol, however this does not produce a reliable benchmark 102 of the router's behavior in a deployed network. Routers in an 103 operational network are simultaneously configured with multiple 104 protocols and security policies while forwarding traffic and being 105 managed. To accurately benchmark a router for deployment it is 106 necessary to test that router in operational conditions by 107 simultaneously configuring the network protocols and security 108 policies, sourcing traffic, and managing the router. It is helpful 109 Router Software Accelerated Life Testing 111 to accelerate these network operational conditions so that the 112 router under test can be benchmarked with faster test duration. 113 Testing a router in accelerated network conditions is known as 114 Accelerated Life Testing. White Box benchmarks are defined in 115 Appendix 1. 116 Accelerated Life Testing of Routers provides the following benefits: 117 1. Evaluation of multiple protocols enabled simultaneously as 118 configured in deployed networks 119 2. Evaluation of System and Software Stability 120 3. Evaluation of Manageability under stressful conditions 121 4. Identification of Software Coding bugs such as: 122 a. Memory Leaks 123 b. Suboptimal CPU Utilization 124 c. Coding Logic 126 These evaluations are externally observable to the DUT as control 127 plane or data plane errors. 129 Configuration and operational conditions vary for Core, Peering, 130 Aggregate, and Edge Routers. This document focuses on defining 131 the terminology for Accelerated Life Testing of Core Routers. 132 Terminology for configuring network conditions, accelerating 133 operational parameters, accelerating network instability, and 134 evaluating results are provided. 136 2. Existing definitions 138 RFC 1242 "Benchmarking Terminology for Network Interconnect 139 Devices" and RFC 2285 "Benchmarking Terminology for LAN Switching 140 Devices" should be consulted before attempting to make use of this 141 document. 143 For the sake of clarity and continuity this RFC adopts the template 144 for definitions set out in Section 2 of RFC 1242. Definitions are 145 indexed and grouped together in sections for ease of reference. 147 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 148 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 149 this document are to be interpreted as described in RFC 2119. 151 3. Term definitions 153 3.1 Configuration Modules 155 Definition: 156 The features and protocols enabled for the Accelerated Life 157 Test and conditions created during the test. 159 Discussion: 160 Features and Protocols are divided into four configuration 161 modules as shown in Figure 1. These are the Control Plane, 162 Data Plane, Management Plane, and Security Plane. 164 Router Software Accelerated Life Testing 166 Definitions for each of these modules are provided in the 167 following sections. 168 ___________ ___________ 169 | Control | | Management| 170 | Plane |___ ___| Plane | 171 | Module | | | | Module | 172 ----------- | | ----------- 173 \/ \/ ___________ 174 ___________ | Security | 175 | |<-----------| Plane | 176 | DUT | | Module | 177 |--->| |<---| ----------- 178 | ----------- | 179 | | 180 | ___________ | 181 | | Data | | 182 |--->| Plane |<---| 183 | Module | 184 ----------- 186 Figure 1. Configuration Modules 188 Measurement units: N/A 190 Issues: None 192 See Also: 193 Control Plane Configuration Module 194 Data Plane Configuration Module 195 Management Configuration Module 196 Security Configuration Module 198 3.1.1 Control Plane Configuration Module 200 Definition: 201 The control protocols enabled for the Accelerated Life Test. 203 Discussion: 204 Control Plane Configuration Module is divided into three 205 Configuration Sets as shown in Figure 2. These are Routing 206 Protocol, Multicast, and MPLS. These can be enabled or 207 disabled for a benchmark test. For example, a network operator 208 that operates a network with BGP and an IGP without Multicast 209 and MPLS protocols can execute a benchmark test with the 210 Routing Protocol Configuration Set enabled and the Multicast 211 and MPLS Configuration Sets disabled. Configuration Sets are 212 further explained in the next section. 214 Measurement units: Enabled/Disabled 216 Issues:None 217 Router Software Accelerated Life Testing 219 ____________ ____________ ____________ 220 | Routing | | Multicast | | MPLS | 221 | Protocol |___ | Protocol | __| Protocol | 222 | Config Set | | | Config Set | | | Config Set | 223 ------------ | ------------ | ------------ 224 | | | 225 | | | 226 | \/ | 227 | ___________ | 228 | | | | 229 |------->| DUT |<------| 230 | | 231 ----------- 232 Figure 2. Control Plane Configuration Module 234 See Also: 235 Routing Protocol Configuration Set 236 Multicast Protocol Configuration Set 237 MPLS Protocol Configuration Set 239 3.1.2 Data Plane Configuration Module 241 Definition: 242 The data traffic and interfaces enabled for the Accelerated 243 Life Test. 245 Discussion: 246 Accelerated Life Testing includes protocols, as defined by the 247 Control Plane Configuration Module, and data forwarding, as 248 defined by the Data Plane Configuration Module. As shown in 249 Figure 3, the Data Plane Configuration Module has a 250 corresponding Traffic Profile Configuration Set. 252 ___________ 253 | Traffic | 254 | Profile |___ 255 | Config Set| | 256 ----------- | 257 \/ 258 ____________ 259 | | 260 | DUT | 261 | | 262 ------------ 264 Measurement Units: 265 Enabled/Disabled 267 See Also: 268 Traffic Profile Configuration Set 269 System Configuration Set 270 Router Software Accelerated Life Testing 272 3.1.3 Management Configuration Module 274 Definition: 275 The router management features enabled for the 276 Accelerated Life Test. 278 Discussion: 279 A key component of the Accelerated Life Test is the 280 Management Configuration Module to assess manageability 281 of the router under stress. The Management Configuration 282 Module can be divided into User Access Configuration Set, 283 SNMP Configuration Set, Logging/Debug Configuration Set, 284 and Packet Statistics Collector Configuration Set as 285 shown in Figure 4. 287 ____________ ____________ ____________ 288 | User | | | | Logging/ | 289 | Access | | SNMP | __| Debug | 290 | Config Set | | Config Set | | | Config Set | 291 ------------ ------------ | ------------ 292 | | | 293 | | | 294 | \/ | 295 | ___________ | 296 | | | | 297 |------->| DUT |<-----| 298 | | 299 ----------- 300 | 301 | 302 \/ 303 ___________ 304 | Packet | 305 | Statistics| 306 | Collector | 307 | Config Set| 308 ----------- 310 Figure 4. Management Configuration Module 312 Measurement units: 313 Enabled/Disabled 315 Issues: 316 None 318 See Also: 319 User Access Configuration Set 320 SNMP Configuration Set 321 Logging/Debug Configuration Set 322 Packet Statistics Collector Configuration Set 323 Router Software Accelerated Life Testing 325 3.1.4 Security Configuration Module 327 Definition: 328 Security features enabled for the Accelerated Life Test. 330 Discussion: 331 Accelerated Life Testing of Core Router can include 332 configuration of Security features. The Security 333 Configuration Module includes the ACL Configuration 334 Set, Secure Protocol Configuration Set, and User Login 335 Configuration Set as shown in Figure 5. 337 ____________ ____________ ____________ 338 | | | Secure | | User | 339 | ACL | | Protocol | __| Login | 340 | Config Set | | Config Set | | | Config Set | 341 ------------ ------------ | ------------ 342 | | | 343 | | | 344 | \/ | 345 | ___________ | 346 | | | | 347 |-------> | DUT |<--------| 348 | | 349 ----------- 350 Figure 5. Security Configuration Module 352 Measurement units: 353 Enabled/Disabled 355 Issues: 356 None 358 See Also: 359 ACL Configuration Set 360 Secure Protocol Configuration Set 361 Password Login Configuration Set 363 3.2 Configuration Sets 365 3.2.1 Control Plane Configuration Sets 366 3.2.1.1 Routing Protocol Configuration Set 368 Definition: 369 Table of Routing Protocols configuration for Accelerated 370 Life Test. 372 Discussion: 373 The Routing Protocol Configuration Set is one of the 374 Control Plane Configuration Sets BGP, MBGP, ISIS, OSPF are 375 the routing protocols that can be enabled or disabled in 376 any combination. The specific protocol parameters used to 377 Router Software Accelerated Life Testing 379 establish the test conditions are also provided in the table. 381 Measurement units: 382 PARAMETER UNITS 383 BGP Enabled/Disabled 384 Number of EBGP Peers Peers 385 Number of IBGP Peers Peers 386 Number of BGP Route Instances Routes 387 Number of BGP Installed Routes Routes 389 MBGP Enabled/Disabled 390 Number of MBGP Route Instances Routes 391 Number of MBGP Installed Routes Routes 393 ISIS Enabled/Disabled 394 ISIS-TE Enabled/Disabled 395 Number of ISIS Adjacencies Adjacencies 396 Number of ISIS Routes Routes 397 Number of Nodes per Area Nodes 399 OSPF Enabled/Disabled 400 OSPF-TE Enabled/Disabled 401 Number of OSPF Adjacencies Adjacencies 402 Number of OSPF Routes Routes 403 Number of Nodes per Area Nodes 405 Issues: 406 None 408 See Also: 409 Control Plane Configuration Module 410 Multicast Protocol Configuration Set 411 MPLS Protocol Configuration Set 413 3.2.1.2 Multicast Protocol Configuration Set 415 Definition: 416 Table of Multicast Protocols configuration for Accelerated 417 Life Test. 419 Discussion: 420 The Multicast Protocol Configuration Set is one of the 421 Control Plane Configuration Sets PIM-SM and MSDP are 422 the routing protocols that can be enabled or disabled in 423 any combination. The specific protocol parameters used to 424 establish the test conditions are also provided in the table. 426 Measurement units: 427 PARAMETER UNITS 428 PIM-SM Enabled/Disabled 429 RP Enabled/Disabled 430 Number of Multicast Groups Groups 431 Router Software Accelerated Life Testing 433 MSDP Enabled/Disabled 435 Issues: 436 None 438 See Also: 439 Control Plane Configuration Module 440 Routing Protocol Configuration Set 441 MPLS Protocol Configuration Set 443 3.2.1.3 MPLS Protocol Configuration Set 445 Definition: 446 Table of MPLS Protocols configuration for Accelerated Life Test. 448 Discussion: 449 The MPLS Protocol Configuration Set is one of the Control Plane 450 Configuration Sets MPLS-TE (RSVP-TE, ISIS-TE, OSPF-TE) and LDP 451 are the routing protocols that can be enabled or disabled in any 452 combination. The specific protocol parameters used to establish 453 the test conditions are also provided in the table. 455 Measurement units: 456 PARAMETER UNITS 457 MPLS-TE 458 Number of Ingress Tunnels Tunnels 459 Number of Mid-Point Tunnels Tunnels 460 Number of Egress Tunnels Tunnels 462 LDP 463 Number of Sessions Sessions 464 Number of FECs FECs 466 Issues: 467 None 469 See Also: 470 Control Plane Configuration Module 471 Routing Protocol Configuration Set 472 Multicast Protocol Configuration Set 474 3.2.2 Data Plane Configuration Sets 476 3.2.2.1 Data Plane Configuration Set 478 Definition: 479 Table of Forwarded Traffic configuration for Accelerated Life 480 Test. 482 Discussion: 484 Measurement units: 486 Router Software Accelerated Life Testing 488 PARAMETER UNITS 489 Traffic Forwarding Enabled/Disabled 490 Packet Size Byte Size Distribution 491 Number of Flows Flows 492 Offered Load bps (or pps) Aggregate 493 Number of Interfaces Interfaces 495 Issues: 496 None 498 See Also: 499 Data Plane Configuration Module 501 3.2.3 Management Configuration Sets 502 3.2.3.1 User Access Configuration Set 504 Definition: 505 Table of User Access methods during Accelerated Life Test. 507 Discussion: 509 Measurement units: 510 PARAMETER UNITS 511 Telnet Rate Sessions/Hour 512 FTP Rate Sessions/Hour 513 Concurrent Telnet Sessions Sessions 514 Concurrent FTP Session Sessions 515 SSH Enabled/Disabled 516 RADIUS Enabled/Disabled 517 TACACS Enabled/Disabled 519 Issues: 520 None 522 See Also: 523 Management Configuration Module 524 SNMP Configuration Set 525 Logging/Debug Configuration Set 526 Packet Statistics Collector Configuration Set 528 3.2.3.2 SNMP Configuration Set 530 Definition: 531 Table of SNMP Configuration for the Accelerated Life Test. 533 Discussion: 535 Measurement units: 536 SNMP GET Rate SNMP Gets/minute 538 Issues: 539 None 540 Router Software Accelerated Life Testing 542 See Also: 543 Management Configuration Module 544 User Access Configuration Set 545 Logging/Debug Configuration Set 546 Packet Statistics Collector Configuration Set 548 3.2.3.3 Logging/Debug Configuration Set 550 Definition: 551 Table of Logging and Debug configuration for the 552 Accelerated Life Test. 554 Discussion: 556 Measurement units: 557 Logging Enabled/Disabled 558 Protocol Debug Enabled/Disabled 560 Issues: 561 None 563 See Also: 564 Management Configuration Module 565 User Access Configuration Set 566 SNMP Configuration Set 567 Packet Statistics Collector Configuration Set 569 3.2.3.4 Packet Statistics Collector Configuration Set 571 Definition: 572 Table of Packet Statistics Collection for the 573 Accelerated Life Test. 575 Discussion: 577 Measurement units: 578 Packet Statistics Collector Enabled/Disabled 579 Sampling Rate X:1 packets 581 Issues: 582 None 584 See Also: 585 Management Configuration Module 586 User Access Configuration Set 587 SNMP Configuration Set 588 Logging/Debug Configuration Set 590 3.2.4 Security Configuration Sets 591 Router Software Accelerated Life Testing 593 3.2.4.1 ACL Configuration Set 595 Definition: 596 Table of ACL configuration for the Accelerated Life Test 598 Discussion: 600 Measurement units: 601 Access-Control Lists Enabled/Disabled 602 Number of ACLs ACLs 603 ACL Depth ACEs 605 Issues: 606 None 608 See Also: 609 Security Configuration Module 611 3.3 Instability Conditions 613 Definition: 614 Test conditions that occur during the Accelerated Life Test 615 that are typical of instability in an operational network. 616 These conditions occur at a much greater rate during the 617 test. 619 Discussion: 620 Configuration Modules and Configuration Sets establish the 621 configuration of the DUT for the Accelerated Life Test. 622 Instability Conditions are events that occur during the 623 Accelerated Life Test to stress the router. 625 Measurement units: 626 N/A 628 Issues: 629 None 631 See Also: 632 Interface Shutdown Cycling Rate 633 BGP Route Flap Rate 634 IGP Route Flap Rate 635 Route Convergence Due to Better Next-Hop 636 LSP Reroute Rate 637 Target Run Time 639 3.3.1 Interface Shutdown Rate 641 Definition: 642 The rate at which physical interfaces are shutdown on the DUT. 644 Discussion: 646 Router Software Accelerated Life Testing 647 This instability condition is an administrative shutdown or remote 648 loss of carrier for every interface of the DUT. This produces great 649 instability on the DUT due to continuous protocol session loss and 650 convergence. 652 Measurement units: 653 number of interface shutdowns per minute 655 Issues: 656 None 658 See Also: 659 Instability Conditions 660 Route Flap Rate 661 Target Run Time 663 3.3.2 BGP Route Flap Rate 665 Definition: 666 The rate at which BGP routes installed in the route table are 667 flapped. 669 Discussion: 670 This instability condition defines the rate of repeated 671 installation, withdrawal,and installation of routes installed in 672 the route table. This produces great instability on the DUT due 673 to continuous changes to the forwarding table. 675 Measurement units: 676 routes flapped per second 678 Issues: 679 None 681 See Also: 682 Instability Conditions 683 Interface Shutdown Cycling Rate 684 Target Run Time 686 3.3.3 IGP Route Flap Rate 688 Definition: 689 The rate at which IGP routes installed in the route table are 690 flapped. 692 Discussion: 693 This instability condition defines the rate of repeated 694 installation, withdrawal,and installation of routes installed in 695 the route table. This produces great instability on the DUT due 696 to continuous changes to the forwarding table. 698 Router Software Accelerated Life Testing 699 Measurement units: 700 routes flapped per second 702 Issues: 703 None 705 See Also: 706 Instability Conditions 707 Interface Shutdown Cycling Rate 708 Target Run Time 710 3.3.4 Route Convergence Due to Better Next-Hop 712 Definition: 713 The rate at which routes in the FIB are replaced by a route 714 instance in the RIB with a different next-hop becoming more 715 preferred. 717 Discussion: 718 The route changes are due to attribute and TLV changes. RIB 719 contains routes matching those in the FIB. Convergence due 720 to better path can occur for BGP and IGP routes. 722 Measurement units: 723 more preferred routes per second 725 Issues: 726 None 728 See Also: 729 Instability Conditions 730 Interface Shutdown Cycling Rate 731 Target Run Time 733 3.3.5 LSP Reroute Rate 735 Definition: 736 The rate at which established LSPs experience a failure condition 737 and are forced to reroute. 739 Discussion: 740 This instability condition defines the rate of repeated LSP 741 reroutes. This produces great instability on the DUT due to 742 continuous tunnel path calculations and changes to the 743 forwarding table. Rerouting can be performed with Headend 744 Reroute, Standby LSP, or Fast Reroute. 746 Measurement units: 747 routes flapped per second 749 Issues: 750 None 751 Router Software Accelerated Life Testing 753 See Also: 754 Instability Conditions 755 Interface Shutdown Cycling Rate 756 Target Run Time 758 3.3.6 Intended Test Duration 760 Definition: 761 The maximum targetted run time for Accelerated Life Test. 763 Discussion: 764 This is an instability condition because the protocol sessions 765 and scaling values must be maintained for this specified time with 766 interfaceshutdowns and route flapping. The test may be halted 767 upon reaching the Intended Test Duration. 769 Measurement units: 770 Hours 772 Issues: 773 None 775 See Also: 776 Instability Conditions 777 Interface Shutdown Cycling Rate 778 Route Flap Rate 780 3.4 Evaluation Benchmarks 782 3.4.1 Run-Time without Error 784 Definition: 785 The run-time for the Accelerated Life Test without the DUT 786 exhibiting an error in the control plane, data plane, management 787 plane, or security plane. 789 Discussion: 790 For a successful test, the Run-Time Without Error will equal the 791 Intended Test Duration. In the event of any failure, the Run-Time 792 Without Error is less than the Intended Test Duration and will equal 793 the maximum of the Run-Time without Control Plane Error, Run-Time 794 without Data Plane Error, Run-Time without Management Plane Error, 795 and Run-Time without Security Plane Error. 797 Measurement units: 798 Hours and Minutes 800 Issues: 801 None 803 See Also: 805 Router Software Accelerated Life Testing 807 Intended Test Duration 808 Run-Time without Control Plane Error 809 Run-Time without Data Plane Error 810 Run-Time without Management Plane Error 811 Run-Time without Security Plane Error 813 3.4.2 Run-Time without Control Plane Error 815 Definition: 816 The run-time for the Accelerated Life Test without the DUT 817 exhibiting an error in the control plane to Routing, Multicast, 818 and MPLS Protocol. 820 Discussion: 821 For a successful test, the Run-Time Without Control Plane Error 822 will equal the Run-Time without Error. In the event of a Control 823 Plane failure, the Run-Time Without Control Plane Error is less 824 than the Run-Time without Error. 826 Measurement units: 827 Hours and Minutes 829 Issues: 830 None 832 See Also: 833 Intended Test Duration 834 Run-Time without Error 835 Run-Time without Data Plane Error 836 Run-Time without Management Plane Error 837 Run-Time without Security Plane Error 839 3.4.3 Run-Time without Data Plane Error 841 Definition: 842 The run-time for the Accelerated Life Test without the DUT 843 exhibiting an error in the Data Plane for traffic forwarding 844 or DUT interfaces. 846 Discussion: 847 For a successful test, the Run-Time Without Data Plane Error 848 will equal the Run-Time without Error. In the event of a Data 849 Plane failure, the Run-Time Without Data Plane Error is less 850 than the Run-Time without Error. 852 Measurement units: 853 Hours and Minutes 855 Issues: 856 None 858 See Also: 860 Router Software Accelerated Life Testing 862 Intended Test Duration 863 Run-Time without Error 864 Run-Time without Control Plane Error 865 Run-Time without Management Plane Error 866 Run-Time without Security Plane Error 868 3.4.4 Run-Time without Management Plane Error 870 Definition: 871 The run-time for the Accelerated Life Test without the DUT 872 exhibiting an error in the Management Plane to User Access, 873 SNMP, or Logging/Debug. 875 Discussion: 876 For a successful test, the Run-Time Without Management Plane 877 Error will equal the Run-Time without Error. In the event of 878 a Management Plane failure, the Run-Time Without Management 879 Plane Error is less than the Run-Time without Error. 881 Measurement units: 882 Hours and Minutes 884 Issues: 885 None 887 See Also: 888 Intended Test Duration 889 Run-Time without Error 890 Run-Time without Control Plane Error 891 Run-Time without Data Plane Error 892 Run-Time without Security Plane Error 894 3.4.5 Run-Time without Security Plane Error 896 Definition: 897 The run-time for the Accelerated Life Test without the DUT 898 exhibiting an error in the Security Plane to ACLs. 900 Discussion: 901 For a successful test, the Run-Time Without Security Plane 902 Error will equal the Run-Time without Error. In the event 903 of a Security Plane failure, the Run-Time Without Control 904 Plane Error is less than the Run-Time without Error. 906 Measurement units: 907 Hours and Minutes 909 Issues: 910 None 912 See Also: 913 Intended Test Duration 914 Router Software Accelerated Life Testing 916 Run-Time without Error 917 Run-Time without Control Plane Error 918 Run-Time without Data Plane Error 919 Run-Time without Management Plane Error 921 4. Security Considerations 922 Documents of this type do not directly effect the security of 923 the Internet or of corporate networks as long as benchmarking 924 is not performed on devices or systems connected to operating 925 networks. 927 5. References 929 [1] Bradner, S., Editor, "Benchmarking Terminology for Network 930 Interconnection Devices", RFC 1242, July 1991. 932 [2] Mandeville, R., "Benchmarking Terminology for LAN Switching 933 Devices", RFC 2285, June 1998. 935 [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for 936 Network Interconnect Devices", RFC 2544, March 1999. 938 [4] "Core Router Evaluation for Higher Availability", Scott 939 Poretsky, NANOG 25, June 8, 2002, Toronto, CA. 941 [5] "Router Stress Testing to Validate Readiness for Network 942 Deployment", Scott Poretsky, IEEE CQR 2003. 944 6. Author's Address 946 Scott Poretsky 947 Avici Systems 948 101 Billerica Avenue 949 N. Billerica, MA 01862 950 USA 952 Phone: + 1 978 964 2287 953 EMail: sporetsky@avici.com 955 Shankar Rao 956 Qwest Communications 957 Denver, CO 958 USA 960 Phone: + 1 303 437 6643 961 Email: srao@qwest.net 963 Ray Piatt 964 Cable and Wireless 965 11700 Plaza America Drive 966 Reston, VA 20190 967 USA 968 Router Software Accelerated Life Testing 970 Phone: + 1 703 292 2113 971 Email: rpiatt@cw.net 973 7. Full Copyright Statement 975 Copyright (C) The Internet Society (1998). All Rights 976 Reserved. 978 This document and translations of it may be copied and 979 furnished to others, and derivative works that comment on or 980 otherwise explain it or assist in its implementation may be 981 prepared, copied, published and distributed, in whole or in 982 part, without restriction of any kind, provided that the above 983 copyright notice and this paragraph are included on all such 984 copies and derivative works. However, this document itself may 985 not be modified in any way, such as by removing the copyright 986 notice or references to the Internet Society or other Internet 987 organizations, except as needed for the purpose of developing 988 Internet standards in which case the procedures for copyrights 989 defined in the Internet Standards process must be followed, or 990 as required to translate it into languages other than English. 992 The limited permissions granted above are perpetual and will 993 not be revoked by the Internet Society or its successors or 994 assigns. This document and the information contained herein is 995 provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE 996 INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, 997 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY 998 THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY 999 RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 1000 FOR A PARTICULAR PURPOSE. 1002 Appendix 1. White Box Benchmarking Terminology 1003 Appendix 1.1 Minimum Available Memory 1005 Definition: 1006 Minimum DUT Available Memory during the duration of the 1007 Accelerated Life Test. 1009 Discussion: 1010 It is necessary to monitor DUT memory to measure this 1011 benchmark. 1013 Measurement units: 1014 bytes 1016 Issues: 1017 None 1019 See Also: 1020 Maximum CPU Utilization 1021 Router Software Accelerated Life Testing 1023 Appendix 1.2 Maximum CPU Utilization 1025 Definition: 1026 Maximum DUT CPU utilization during the duration of the 1027 Accelerated Life Test. 1029 Discussion: 1030 It is necessary to monitor DUT CPU Utilization to measure 1031 this benchmark. 1033 Measurement units: 1034 % 1036 Issues: 1037 None 1039 See Also: 1040 Minimum Available Memory