idnits 2.17.1 draft-ietf-bmwg-reset-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC2544, updated by this document, for RFC5378 checks: 1999-03-01) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 1, 2010) is 5109 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Benchmarking Methodology WG Rajiv Asati 2 Internet Draft Cisco 3 Updates: 2544 (if approved) Carlos Pignataro 4 Intended status: Informational Cisco 5 Expires: November 2010 Fernando Calabria 6 Cisco 7 Cesar Olvera 8 Consulintel 10 May 1, 2010 12 Device Reset Characterization 13 draft-ietf-bmwg-reset-00 15 Abstract 17 An operational forwarding device may need to be re-started 18 (automatically or manually) for a variety of reasons, an event that 19 we call a "reset" in this document. Since there may be an 20 interruption in the forwarding operation during a reset, it is 21 useful to know how long a device takes to begin forwarding packets 22 again. 24 This document specifies a methodology for characterizing reset 25 during benchmarking of forwarding devices, and provides clarity and 26 consistency in reset test procedures beyond what's specified in 27 RFC2544. It therefore updates RFC2544. 29 Status of this Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF), its areas, and its working groups. Note that 36 other groups may also distribute working documents as Internet- 37 Drafts. 39 Internet-Drafts are draft documents valid for a maximum of six 40 months and may be updated, replaced, or obsoleted by other documents 41 at any time. It is inappropriate to use Internet-Drafts as 42 reference material or to cite them other than as "work in progress." 43 The list of current Internet-Drafts can be accessed at 44 http://www.ietf.org/ietf/1id-abstracts.txt 46 The list of Internet-Draft Shadow Directories can be accessed at 47 http://www.ietf.org/shadow.html 49 This Internet-Draft will expire on November 1, 2010. 51 Copyright Notice 53 Copyright (c) 2010 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with 61 respect to this document. Code Components extracted from this 62 document must include Simplified BSD License text as described in 63 Section 4.e of the Trust Legal Provisions and are provided without 64 warranty as described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction...................................................4 69 1.1. Scope.....................................................5 70 2. Key Words to Reflect Requirements..............................5 71 3. Reset Test.....................................................5 72 3.1. Hardware Reset............................................6 73 3.1.1. Routing Processor (RP) / Routing Engine reset........6 74 3.1.1.1. RP Failure for a single-RP device (mandatory)...6 75 3.1.1.2. RP Failure for a multiple-RP device (optional)..8 76 3.1.2. Line Card (LC) Removal and Insertion (mandatory)....10 77 3.2. Software Reset...........................................12 78 3.2.1. Operating System (OS) reset (mandatory).............13 79 3.2.2. Process reset (optional)............................14 80 3.3. Power interruption.......................................16 81 3.3.1. Power Interruption (mandatory)......................17 82 4. Security Considerations.......................................19 83 5. IANA Considerations...........................................19 84 6. Acknowledgments...............................................19 85 7. References....................................................20 86 7.1. Normative References.....................................20 87 7.2. Informative References...................................20 88 Authors' Addresses...............................................21 90 1. Introduction 92 An operational forwarding device (or one of its components) may need 93 to be re-started for a variety of reasons, an event that we call a 94 "reset" in this draft. Since there may be an interruption in the 95 forwarding operation during a reset, it is useful to know how long a 96 device takes to begin forwarding packets again. 98 However, the answer to this question is no longer simple and 99 straight-forward as the modern forwarding devices employ many 100 hardware advancements (distributed forwarding, etc.) and software 101 advancements (graceful restart, etc.) that influence the recovery 102 time after the reset. 104 In order to provide consistent and fairness while benchmarking a set 105 of different DUTs, the Network tester / Operator MUST (a) use 106 identical control and data plane information during testing, (b) 107 document & report any factors that may influence the overall time 108 after reset / convergence. 110 Some of these factors follow: 112 1. Type of reset - Hardware (line-card crash, etc.) vs. Software 113 (protocol reset, process crash, etc.) or even complete power 114 failures 116 2. Manual vs. Automatic reset 118 3. Scheduled vs. non-scheduled reset 120 4. Local vs. Remote reset 122 5. Scale - Number of line cards present vs. in-use 124 6. Scale - Number of physical and logical interfaces 126 7. Scale - Number of routing protocol instances 128 8. Scale - Number of Routing Table entries 130 9. Scale - Number of Route Processors available 132 10. Performance - Redundancy strategy deployed for route 133 processors and line cards 135 11. Performance - Interface encapsulation as well as achievable 136 NDR (non-dropping rate) 138 12. Any other internal or external factor that may influence 139 recovery time after a hardware or software reset 141 This document specifies a methodology for characterizing reset 142 during benchmarking of forwarding devices, and provides clarity and 143 consistency in reset procedures beyond what's specified in 144 [RFC2544]. These procedures may be used by other benchmarking 145 documents such as [RFC2544], [RFC5180], [RFC5695], etc. 147 This document updates Section 26.6 of [RFC2544]. 149 1.1. Scope 151 This document focuses on only the reset criterion of benchmarking, 152 and presumes that it would be beneficial to [RFC2544], [RFC5180], 153 [RFC5695], and other BMWG benchmarking efforts. 155 2. Key Words to Reflect Requirements 157 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 158 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 159 document are to be interpreted as described in BCP 14, RFC 2119 160 [RFC2119]. RFC 2119 defines the use of these key words to help make 161 the intent of standards track documents as clear as possible. While 162 this document uses these keywords, this document is not a standards 163 track document. 165 3. Reset Test 167 This section contains the description of the tests that are related 168 to the characterization of DUTs (Device Under Test) / SUTs (System 169 Under Test) speed to recover from a reset. There are three types of 170 reset considered in this document: 172 1. Hardware resets 174 2. Software resets 176 3. Power interruption 178 Different types of reset have potentially different impact on the 179 forwarding behavior of the device. As an example, a software reset 180 (of a routing process) might not result in forwarding interruption, 181 whereas a hardware reset (of a line card) most likely will. 183 Section 3.1 describes various hardware resets, whereas Section 3.2 184 describes various software resets. Additionally, Section 3.3 185 describes power interruption tests. These sections define and 186 characterize these resets. 188 Additionally, since device specific implementations may vary for 189 hardware and software type resets, it is desirable to classify each 190 test case as "MUST" or "optional". 192 3.1. Hardware Reset 194 A test designed to characterize the time it takes a DUT to recover 195 from the hardware reset. 197 A "hardware reset" generally involves the re-initialization of one 198 or more physical components in the DUT, but not the entire DUT. 200 A hardware reset is executed by the operator for example by physical 201 removal of a physical component, by pressing on a "reset" button for 202 the component, or could even be triggered from the command line 203 interface. 205 For routers that do not contain separate Routing Processor and Line 206 Card modules, the hardware reset tests are not performed since they 207 are not relevant; instead, the power interruption tests MUST be 208 performed (see Section 3.3) in these cases. 210 3.1.1. Routing Processor (RP) / Routing Engine reset 212 The Routing Processor (RP) is the DUT module that is primarily 213 concerned with Control Plane functions. 215 3.1.1.1. RP Failure for a single-RP device (mandatory) 217 Objective 219 To characterize the speed at which a DUT recovers from a Route 220 processor hardware reset in a single RP environment. 222 Procedure 223 First, ensure that the RP is in a permanent state to which it will 224 return to after the reset, by performing some or all of the 225 following operational tasks: save the current DUT configuration, 226 specify boot parameters, ensure the appropriate software files are 227 available, or perform additional Operating System or hardware 228 related task. 230 Second, ensure that the DUT is able to forward the traffic for at 231 least 15 seconds before any test activities are performed. The 232 traffic should use the minimum frame size possible on the media 233 used in the testing and rate should be sufficient for the DUT to 234 attain the maximum forwarding throughput. This enables a finer 235 granularity in the recovery time measurement. 237 Modern traffic generators support the feature of transmitting 238 packets while ignoring the link status. The operator / tester MUST 239 ensure that this feature is enabled. 241 Third, perform the Route Processor (RP) hardware reset at this 242 point. This entails for example physically removing the RP to 243 later re-insert it, or triggering a hardware reset by other means 244 (e.g., command line interface, physical switch, etc.) 246 Finally, the characterization is completed by measuring the frame 247 loss and recovery time from the moment the RP is re-initialized or 248 reinserted. 250 Reporting format 252 The reset results are reported in a simple statement including the 253 frame loss and recovery times. 255 For each test case, it is RECOMMENDED that the following 256 parameters be reported in these units: 258 Parameter Units or Examples 260 Throughput Frames per second and bits per 262 second 264 Loss Frames 266 Time Seconds, with sufficient resolution 267 to convey meaningful info 269 Protocol IPv4, IPv6, MPLS, etc. 271 Frame Size Octets 273 Port Media Ethernet, GigE (Gigabit Ethernet), 275 POS (Packet over SONET), etc. 277 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 279 Interface Encap. Ethernet, Ethernet VLAN, 281 PPP, HDLC, etc. 283 Additionally, the DUT and test bed provisioning, configuration, 284 and deployed methodologies that may influence the overall recovery 285 time MUST be listed. (Refer to the additional factors listed in 286 Section 1). 288 The reporting of results MUST regard repeatability considerations 289 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 290 trials and report average results. 292 3.1.1.2. RP Failure for a multiple-RP device (optional) 294 Objective 296 To characterize the speed at which a "secondary" Route Processor 297 (sometimes referred to as "backup" RP) of a DUT becomes active 298 after a "primary" (or "active") Route Processor hardware reset. 299 This process is often referred to as "RP Switchover". The 300 characterization in this test should be done for the default DUT 301 behavior as well as a DUT's non-default configuration that 302 minimizes frame loss. 304 Procedure 306 This test characterizes "RP Switchover". Many implementations 307 allow for optimized switchover capabilities that minimize the 308 downtime during the actual switchover. This test consists of two 309 sub-cases from a switchover characteristics standpoint: First, a 310 default behavior (with no switchover-specific configurations); and 311 second, a non-default behavior with switchover configuration to 312 minimize frame loss. Therefore, the procedures hereby described 313 are executed twice, and reported separately. 315 First, ensure that the RPs are in a permanent state such that the 316 secondary will be activated to the same state as the active is, by 317 performing some or all of the following operational tasks: save 318 the current DUT configuration, specify boot parameters, ensure the 319 appropriate software files are available, or perform additional 320 Operating System or hardware related task. 322 Second, ensure that the DUT is able to forward the traffic for at 323 least 15 seconds before any test activities are performed. The 324 traffic should use the minimum frame size possible on the media 325 used in the testing and rate should be sufficient for the DUT to 326 attain the maximum forwarding throughput. This enables a finer 327 granularity in the recovery time measurement. 329 Modern traffic generators support the feature of transmitting 330 packets while ignoring the link status. The operator / tester MUST 331 ensure that this feature is enabled. 333 Third, perform the primary Route Processor (RP) hardware reset at 334 this point. This entails for example physically removing the RP, 335 or triggering a hardware reset by other means (e.g., command line 336 interface, physical switch, etc.) Is up to the Operator to decide 337 if the active RP needs to be re-inserted after a grace period or 338 not. 340 Finally, the characterization is completed by measuring the 341 complete frame loss and recovery time from the moment the active 342 RP is hardware-reset. 344 Reporting format 346 The reset results are reported twice, one for the default 347 switchover behavior and the other for the non-default one. For 348 each, the report consists of a simple statement including the 349 frame loss and recovery times, as well as any specific redundancy 350 scheme in place. 352 For each test case, it is RECOMMENDED that the following 353 parameters be reported in these units: 355 Parameter Units or Examples 357 Throughput Frames per second and bits per 359 second 361 Loss Frames 363 Time Seconds, with sufficient resolution 365 to convey meaningful info 367 Protocol IPv4, IPv6, MPLS, etc. 369 Frame Size Octets 371 Port Media Ethernet, GigE (Gigabit Ethernet), 373 POS (Packet over SONET), etc. 375 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 377 Interface Encap. Ethernet, Ethernet VLAN, 379 PPP, HDLC, etc. 381 Additionally, the DUT and test bed provisioning, configuration, 382 and deployed methodologies that may influence the overall recovery 383 time MUST be listed. (Refer to the additional factors listed in 384 Section 1). 386 The reporting of results MUST regard repeatability considerations 387 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 388 trials and report average results. 390 3.1.2. Line Card (LC) Removal and Insertion (mandatory) 392 The Line Card (LC) is the DUT component that is responsible with 393 packet forwarding. 395 Objective 397 To characterize the speed at which a DUT recovers from a Line Card 398 removal and insertion event. 400 Procedure 402 For this test, the Line Card that is being hardware-reset MUST be 403 on the forwarding path and all destinations MUST be directly 404 connected. 406 First, complete some or all of the following operational tasks: 407 save the current DUT configuration, specify boot parameters, 408 ensure the appropriate software files are available, or perform 409 additional Operating System or hardware related task. 411 Second, ensure that the DUT is able to forward the traffic for at 412 least 15 seconds before any test activities are performed. The 413 traffic should use the minimum frame size possible on the media 414 used in the testing and rate should be sufficient for the DUT to 415 attain the maximum forwarding throughput. This enables a finer 416 granularity in the recovery time measurement. 418 Modern traffic generators support the feature of transmitting 419 packets while ignoring the link status. The operator / tester MUST 420 ensure that this feature is enabled. 422 Third, perform the Line Card (LC) hardware reset at this point. 423 This entails for example physically removing the LC to later re- 424 insert it, or triggering a hardware reset by other means (e.g., 425 command line interface (CLI), physical switch, etc.). However, 426 most accurate results will be obtained using the CLI or a physical 427 switch, and therefore these are RECOMMENDED. Otherwise, the time 428 spend trying to physically seat the LC will get mixed into the 429 results. 431 Finally, the characterization is completed by measuring the frame 432 loss and recovery time from the moment the LC is reinitialized or 433 reinserted. 435 Reporting Format 437 The reset results are reported in a simple statement including the 438 frame loss and recovery times. 440 For each test case, it is RECOMMENDED that the following 441 parameters be reported in these units: 443 Parameter Units or Examples 445 Throughput Frames per second and bits per 447 second 449 Loss Frames 451 Time Seconds, with sufficient resolution 453 to convey meaningful info 455 Protocol IPv4, IPv6, MPLS, etc. 457 Frame Size Octets 459 Port Media Ethernet, GigE (Gigabit Ethernet), 461 POS (Packet over SONET), etc. 463 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 465 Interface Encap. Ethernet, Ethernet VLAN, 467 PPP, HDLC, etc. 469 Additionally, the DUT and test bed provisioning, configuration, 470 and deployed methodologies that may influence the overall recovery 471 time MUST be listed. (Refer to the additional factors listed in 472 Section 1). 474 The reporting of results MUST regard repeatability considerations 475 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 476 trials and report average results. 478 3.2. Software Reset 480 To characterize the speed at which a DUT recovers from the software 481 reset. 483 In contrast to a "hardware reset", a "software reset" involves only 484 the re-initialization of the execution, data structures, and partial 485 state within the software running on the DUT module(s). 487 A software reset is initiated for example from the DUT's Command 488 Line Interface (CLI). 490 3.2.1. Operating System (OS) reset (mandatory) 492 Objective 494 To characterize the speed at which a DUT recovers from an 495 Operating System (OS) software reset. 497 Procedure 499 First, complete some or all of the following operational tasks: 500 save the current DUT configuration, specify software boot 501 parameters, ensure the appropriate software files are available, 502 or perform additional Operating System task. 504 Second, ensure that the DUT is able to forward the traffic for at 505 least 15 seconds before any test activities are performed. The 506 traffic should use the minimum frame size possible on the media 507 used in the testing and rate should be sufficient for the DUT to 508 attain the maximum forwarding throughput. This enables a finer 509 granularity in the recovery time measurement. 511 Modern traffic generators support the feature of transmitting 512 packets while ignoring the link status. The operator / tester MUST 513 ensure that this feature is enabled. 515 Third, trigger an Operating System re-initialization in the DUT, 516 by operational means such as use of the DUT's Command Line 517 Interface (CLI) or other management interface. 519 Finally, the characterization is completed by measuring the 520 complete frame loss and recovery time from the moment the reset 521 instruction was given until the Operating System finished the 522 reload and re-initialization (inferred by the re-establishing of 523 traffic). 525 Reporting format 527 The reset results are reported in a simple statement including the 528 frame loss and recovery times. 530 For each test case, it is RECOMMENDED that the following 531 parameters be reported in these units: 533 Parameter Units or Examples 535 Throughput Frames per second and bits per 537 second 539 Loss Frames 541 Time Seconds, with sufficient resolution 543 to convey meaningful info 545 Protocol IPv4, IPv6, MPLS, etc. 547 Frame Size Octets 549 Port Media Ethernet, GigE (Gigabit Ethernet), 551 POS (Packet over SONET), etc. 553 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 555 Interface Encap. Ethernet, Ethernet VLAN, 557 PPP, HDLC, etc. 559 Additionally, the DUT and test bed provisioning, configuration, 560 and deployed methodologies that may influence the overall recovery 561 time MUST be listed. (Refer to the additional factors listed in 562 Section 1). 564 The reporting of results MUST regard repeatability considerations 565 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 566 trials and report average results. 568 3.2.2. Process reset (optional) 570 Objective 571 To characterize the speed at which a DUT recovers from a software 572 process reset. 574 Such speed may depend upon the number and types of process running 575 in the DUT and which ones are tested. Different implementations of 576 forwarding devices include various common processes. A process 577 reset should be performed only in the processes most relevant to 578 the tester. 580 Procedure 582 First, complete some or all of the following operational tasks: 583 save the current DUT configuration, specify software parameters or 584 environmental variables, or perform additional Operating System 585 task. 587 Second, ensure that the DUT is able to forward the traffic for at 588 least 15 seconds before any test activities are performed. The 589 traffic should use the minimum frame size possible on the media 590 used in the testing and rate should be sufficient for the DUT to 591 attain the maximum forwarding throughput. This enables a finer 592 granularity in the recovery time measurement. 594 Modern traffic generators support the feature of transmitting 595 packets while ignoring the link status. The operator / tester MUST 596 ensure that this feature is enabled. 598 Third, trigger a process reset for each process running in the DUT 599 and considered for testing from a management interface (e.g., by 600 means of the Command Line Interface (CLI), etc.) 602 Finally, the characterization for each individual process is 603 completed by measuring the complete frame loss and recovery time 604 from the moment the reset instruction was given until the 605 Operating System finished the reload and re-initialization 606 (inferred by the re-establishing of traffic). 608 Reporting format 610 The reset results are reported in a simple statement including the 611 frame loss and recovery times for each process running in the DUT 612 and tested. Given the implementation nature of this test, details 613 of the actual process tested should be included along with the 614 statement. 616 For each test case, it is RECOMMENDED that the following 617 parameters be reported in these units: 619 Parameter Units or Examples 621 Throughput Frames per second and bits per 623 second 625 Loss Frames 627 Time Seconds, with sufficient resolution 629 to convey meaningful info 631 Protocol IPv4, IPv6, MPLS, etc. 633 Frame Size Octets 635 Port Media Ethernet, GigE (Gigabit Ethernet), 637 POS (Packet over SONET), etc. 639 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 641 Interface Encap. Ethernet, Ethernet VLAN, 643 PPP, HDLC, etc. 645 Additionally, the DUT and test bed provisioning, configuration, 646 and deployed methodologies that may influence the overall recovery 647 time MUST be listed. (Refer to the additional factors listed in 648 Section 1). 650 The reporting of results MUST regard repeatability considerations 651 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 652 trials and report average results. 654 3.3. Power interruption 656 "Power interruption" refers to the complete loss of power on the 657 DUT. It can be viewed as a special case of a hardware reset, 658 triggered by the loss of the power supply to the DUT or its 659 components, and is characterized by the re-initialization of all 660 hardware and software in the DUT. 662 3.3.1. Power Interruption (mandatory) 664 Objective 666 To characterize the speed at which a DUT recovers from a complete 667 loss of electric power or complete power interruption. This test 668 simulates a complete power failure or outage, and should be 669 indicative of the DUT/SUTs behavior during such event. 671 Procedure 673 First, ensure that the entire DUT is at a permanent state to which 674 it will return to after the power interruption, by performing some 675 or all of the following operational tasks: save the current DUT 676 configuration, specify boot parameters, ensure the appropriate 677 software files are available, or perform additional Operating 678 System or hardware related task. 680 Second, ensure that the DUT is able to forward the traffic for at 681 least 15 seconds before any test activities are performed. The 682 traffic should use the minimum frame size possible on the media 683 used in the testing and rate should be sufficient for the DUT to 684 attain the maximum forwarding throughput. This enables a finer 685 granularity in the recovery time measurement. 687 Modern traffic generators support the feature of transmitting 688 packets while ignoring the link status. The operator / tester MUST 689 ensure that this feature is enabled. 691 Third, interrupt the power (AC or DC) that feeds the corresponding 692 DUTs power supplies at this point. This entails for example 693 physically removing the power supplies in the DUT to later re- 694 insert them, or simply disconnecting or switching off their power 695 feeds (AC or DC as applicable). The actual power interruption 696 should last at least 15 seconds. 698 Finally, the characterization is completed by measuring the frame 699 loss and recovery time from the moment the power is restored or 700 the power supplies reinserted in the DUT. 702 Reporting format 704 The reset results are reported in a simple statement including the 705 frame loss and recovery times. 707 For each test case, it is RECOMMENDED that the following 708 parameters be reported in these units: 710 Parameter Units or Examples 712 Throughput Frames per second and bits per 714 second 716 Loss Frames 718 Time Seconds, with sufficient resolution 720 to convey meaningful info 722 Protocol IPv4, IPv6, MPLS, etc. 724 Frame Size Octets 726 Port Media Ethernet, GigE (Gigabit Ethernet), 728 POS (Packet over SONET), etc. 730 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 732 Interface Encap. Ethernet, Ethernet VLAN, 734 PPP, HDLC, etc. 736 Additionally, the DUT and test bed provisioning, configuration, 737 and deployed methodologies that may influence the overall recovery 738 time MUST be listed. (Refer to the additional factors listed in 739 Section 1). 741 The reporting of results MUST regard repeatability considerations 742 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 743 trials and report average results. 745 4. Security Considerations 747 Benchmarking activities, as described in this memo, are limited to 748 technology characterization using controlled stimuli in a laboratory 749 environment, with dedicated address space and the constraints 750 specified in the sections above. 752 The benchmarking network topology will be an independent test setup 753 and MUST NOT be connected to devices that may forward the test 754 traffic into a production network or misroute traffic to the test 755 management network. 757 Furthermore, benchmarking is performed on a "black-box" basis, 758 relying solely on measurements observable external to the DUT/SUT. 760 Special capabilities SHOULD NOT exist in the DUT/SUT specifically 761 for benchmarking purposes. Any implications for network security 762 arising from the DUT/SUT SHOULD be identical in the lab and in 763 production networks. 765 There are no specific security considerations within the scope of 766 this document. 768 5. IANA Considerations 770 There is no IANA consideration for this document. 772 6. Acknowledgments 774 The authors would like to thank Ron Bonica, who motivated us to 775 write this document. The authors would also like to thank Al Morton, 776 Andrew Yourtchenko, David Newman, and John E Dawson for providing 777 thorough review, useful suggestions, and valuable input. 779 This document was prepared using 2-Word-v2.0.template.dot. 781 7. References 783 7.1. Normative References 785 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 786 Requirement Levels", BCP 14, RFC 2119, March 1997. 788 [RFC2544] Bradner, S. and McQuaid, J., "Benchmarking Methodology for 789 Network Interconnect Devices", RFC 2544, March 1999. 791 7.2. Informative References 793 [RFC5180] Popoviciu, C., et al, "IPv6 Benchmarking Methodology for 794 Network Interconnect Devices", RFC 5180, May 2008. 796 [RFC5695] Akhter, A., Asati, R., and C. Pignataro, "MPLS Forwarding 797 Benchmarking Methodology for IP Flows", RFC 5695, November 798 2009. 800 Authors' Addresses 802 Rajiv Asati 803 Cisco Systems 804 7025-6 Kit Creek Road 805 RTP, NC 27709 806 USA 808 Email: rajiva@cisco.com 810 Carlos Pignataro 811 Cisco Systems 812 7200-12 Kit Creek Road 813 RTP, NC 27709 814 USA 816 Email: cpignata@cisco.com 818 Fernando Calabria 819 Cisco Systems 820 7200-12 Kit Creek Road 821 RTP, NC 27709 822 USA 824 Email: fcalabri@cisco.com 826 Cesar Olvera 827 Consulintel 828 Joaquin Turina, 2 829 Pozuelo de Alarcon, Madrid, E-28224 830 Spain 832 Phone: +34 91 151 81 99 833 Email: cesar.olvera@consulintel.es