idnits 2.17.1 draft-ietf-bmwg-reset-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC2544, updated by this document, for RFC5378 checks: 1999-03-01) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (December 18, 2010) is 4871 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-meth-21 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Benchmarking Methodology WG Rajiv Asati 2 Internet Draft Cisco 3 Updates: 2544 (if approved) Carlos Pignataro 4 Intended status: Informational Cisco 5 Expires: June 2011 Fernando Calabria 6 Cisco 7 Cesar Olvera 8 Consulintel 10 December 18, 2010 12 Device Reset Characterization 13 draft-ietf-bmwg-reset-05 15 Abstract 17 An operational forwarding device may need to be re-started 18 (automatically or manually) for a variety of reasons, an event that 19 we call a "reset" in this document. Since there may be an 20 interruption in the forwarding operation during a reset, it is 21 useful to know how long a device takes to resume the forwarding 22 operation. 24 This document specifies a methodology for characterizing reset (and 25 recovery time) during benchmarking of forwarding devices, and 26 provides clarity and consistency in reset test procedures beyond 27 what's specified in RFC2544. It therefore updates RFC2544. 29 Status of this Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF), its areas, and its working groups. Note that 36 other groups may also distribute working documents as Internet- 37 Drafts. 39 Internet-Drafts are draft documents valid for a maximum of six 40 months and may be updated, replaced, or obsoleted by other documents 41 at any time. It is inappropriate to use Internet-Drafts as 42 reference material or to cite them other than as "work in progress." 43 The list of current Internet-Drafts can be accessed at 44 http://www.ietf.org/ietf/1id-abstracts.txt 46 The list of Internet-Draft Shadow Directories can be accessed at 47 http://www.ietf.org/shadow.html 49 This Internet-Draft will expire on June 18, 2011. 51 Copyright Notice 53 Copyright (c) 2010 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with 61 respect to this document. Code Components extracted from this 62 document must include Simplified BSD License text as described in 63 Section 4.e of the Trust Legal Provisions and are provided without 64 warranty as described in the Simplified BSD License. 66 Table of Contents 68 1. Key Words to Reflect Requirements..............................4 69 2. Introduction...................................................4 70 2.1. Scope.....................................................4 71 2.2. Reset Characterization....................................5 72 2.3. Recovery Time Measurement Methods.........................6 73 2.4. Reporting Format..........................................7 74 3. Test Requirements..............................................8 75 4. Reset Test.....................................................9 76 4.1. Hardware Reset Test......................................10 77 4.1.1. Routing Processor (RP) / Routing Engine Reset.......10 78 4.1.1.1. RP Reset for a single-RP device (REQUIRED).....10 79 4.1.1.2. RP Switchover for a multiple-RP device (OPTIONAL) 80 ........................................................11 81 4.1.2. Line Card (LC) Removal and Insertion (REQUIRED).....13 82 4.2. Software Reset Test......................................13 83 4.2.1. Operating System (OS) Reset (REQUIRED)..............14 84 4.2.2. Process Reset (OPTIONAL)............................14 85 4.3. Power Interruption Test..................................15 86 4.3.1. Power Interruption (REQUIRED).......................16 87 5. Security Considerations.......................................17 88 6. IANA Considerations...........................................17 89 7. Acknowledgments...............................................17 90 8. References....................................................18 91 8.1. Normative References.....................................18 92 8.2. Informative References...................................18 93 Authors' Addresses...............................................19 95 1. Key Words to Reflect Requirements 97 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 98 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 99 document are to be interpreted as described in BCP 14, RFC 2119 100 [RFC2119]. RFC 2119 defines the use of these key words to help make 101 the intent of standards track documents as clear as possible. While 102 this document uses these keywords, this document is not a standards 103 track document. 105 2. Introduction 107 An operational forwarding device (or one of its components) may need 108 to be re-started for a variety of reasons, an event that we call a 109 "reset" in this document. Since there may be an interruption in the 110 forwarding operation during a reset, it is useful to know how long a 111 device takes to resume the forwarding operation. In other words, it 112 is desired to know the duration of the recovery time following the 113 reset. 115 However, the answer to this question is no longer simple and 116 straight-forward as the modern forwarding devices employ many 117 hardware advancements (distributed forwarding, etc.) and software 118 advancements (graceful restart, etc.) that influence the recovery 119 time after the reset. 121 2.1. Scope 123 This document specifies a methodology for characterizing reset (and 124 recovery time) during benchmarking of forwarding devices, and 125 provides clarity and consistency in reset procedures beyond what is 126 specified in [RFC2544]. Software upgrades involve additional 127 benchmarking complexities and are outside the scope of this 128 document. 130 These procedures may be used by other benchmarking documents such as 131 [RFC2544], [RFC5180], [RFC5695], etc., and is expected that other 132 protocol-specific benchmarking documents would reference this 133 document for reset recovery time characterization. Specific Routing 134 Information Base (RIB) and Forwarding Information Base (FIB) scaling 135 considerations are outside the scope of this document and can be 136 quite complex to characterize. However, other documents can 137 characterize specific dynamic protocols scaling and interactions and 138 leverage and augment the reset tests defined in this document. 140 This document updates Section 26.6 of [RFC2544]. 142 This document focuses only on the reset criterion of benchmarking, 143 and presumes that it would be beneficial to [RFC5180], [RFC5695], 144 and other IETF Benchmarking Methodology Working Group (BMWG) 145 efforts. 147 2.2. Reset Characterization 149 Definition 151 Reset Time is the total time when a device is determined to be out 152 of operation, and includes the time to perform the reset and the 153 time to recover from it. 155 Discussion 157 During a period of time after a reset or power up, network devices 158 may not accept and forward frames. The duration of this period of 159 forwarding unavailability can be useful in evaluating devices. In 160 addition, some network devices require some form of reset when 161 specific setup variables are modified. If the reset period were 162 long it might discourage network managers from modifying these 163 variables on production networks. 165 The events characterized in this document are entire reset events. 166 That is, the recovery period measured includes the time to perform 167 the reset and the time to recover from it. Some reset events will 168 be atomic (such as pressing a reset button) while others (such as 169 power cycling) may comprise multiple actions with a recognized 170 interval between them. In both cases, the duration considered is 171 from the start of the event until full recovery of forwarding 172 after the completion of the reset events. 174 Measurement units 176 Time in milliseconds, providing sufficient resolution to 177 distinguish between different trials and different 178 implementations. See Section 2.4. 180 Issues 181 There are various types of Reset: Hardware resets, software 182 resets, and power interruption. See Section 4. 184 2.3. Recovery Time Measurement Methods 186 The 'recovery time' is the time during which the traffic forwarding 187 is temporarily interrupted following a reset event. Strictly 188 speaking, this is the time over which one or more frames are lost. 189 This definition is similar to that of 'Loss of connectivity period' 190 defined in [IGPConv] section 4. 192 There are two accepted methods to measure the 'recovery time': 194 1. Frame-Loss Method - This method requires test tool capability 195 to monitor the number of lost frames. In this method, the 196 offered stream rate (frames per second) must be known. The 197 recovery time is calculated per the below equation: 199 Frames_lost (packets) 200 Recovery_time = ------------------------------------- 201 Offered_rate (packets per second) 203 2. Time-Stamp Method - This method requires test tool capability 204 to timestamp each frame. In this method, the test tool 205 timestamps each transmitted frame and monitors the received 206 frame's timestamp. During the test, the test tool would record 207 the timestamp (Timestamp A) of the frame that was last 208 received prior to the reset interruption and the timestamp 209 (Timestamp B) of the first frame after the interruption 210 stopped. The difference between Timestamp B and Timestamp A is 211 the recovery time. 213 The tester / operator MAY use either method for recovery time 214 measurement depending on the test tool capability. However, the 215 Frame-loss method SHOULD be used if the test tool is capable of (a) 216 counting the number of lost frames per stream, and (b) transmitting 217 test frame despite the physical link status, whereas Time-stamp 218 method SHOULD be used if the test tool is capable of (a) time- 219 stamping each frame, (b) monitoring received frame's timestamp, and 220 (c) transmitting frames only if the physical link status is UP. That 221 is, specific test tool capabilities may dictate which method to use. 223 If the test tool supports both methods based on its capabilities, 224 the tester / operator SHOULD use the one that provides more 225 accuracy. 227 2.4. Reporting Format 229 All reset results are reported in a simple statement including the 230 frame loss (if measured) and recovery times. 232 For each test case, it is RECOMMENDED that the following parameters 233 be reported in these units: 235 Parameter Units or Examples 237 ----------------------------------------------------------------- 239 Throughput Frames per second and bits per 240 second 242 Loss (average) Frames 244 Recovery Time (average) Milliseconds 246 Number of trials Integer count 248 Protocol IPv4, IPv6, MPLS, etc. 250 Frame Size Octets 252 Port Media Ethernet, GigE (Gigabit Ethernet), 253 POS (Packet over SONET), etc. 255 Port Speed 10 Gbps, 1 Gbps, 100 Mbps, etc. 257 Interface Encap. Ethernet, Ethernet VLAN, 258 PPP, HDLC, etc. 260 For mixed protocol environments, frames SHOULD be distributed 261 between all the different protocols. The distribution MAY 262 approximate the network conditions of deployment. In all cases the 263 details of the mixed protocol distribution MUST be included in the 264 reporting. 266 Additionally, the DUT (Device Under Test) / SUT (System Under Test) 267 and test bed provisioning, port and Line Card arrangement, 268 configuration, and deployed methodologies that may influence the 269 overall recovery time MUST be listed. (Refer to the additional 270 factors listed in Section 3). 272 The reporting of results MUST regard repeatability considerations 273 from Section 4 of [RFC2544]. It is RECOMMENDED to perform multiple 274 trials and report average results. 276 3. Test Requirements 278 Tests SHOULD first be performed such that the forwarding state re- 279 establishment is independent from an external source (i.e., using 280 static address resolution, routing and forwarding configuration, and 281 not dynamic protocols). However tests MAY subsequently be performed 282 using dynamic protocols that the forwarding state depends on (e.g., 283 dynamic Interior Gateway Protocols (IGP), ARP, PPP Control 284 Protocols, etc.) The considerations in this Section apply. 286 In order to provide consistence and fairness while benchmarking a 287 set of different DUTs, the Network tester / operator MUST (a) use 288 identical control and data plane information during testing, (b) 289 document & report any factors that may influence the overall time 290 after reset / convergence. 292 Some of these factors include: 294 1. Type of reset - Hardware (line-card crash, etc.) vs. Software 295 (protocol reset, process crash, etc.) or even complete power 296 failures 298 2. Manual vs. Automatic reset 300 3. Scheduled vs. non-scheduled reset 302 4. Local vs. Remote reset 304 5. Scale - Number of line cards present vs. in-use 306 6. Scale - Number of physical and logical interfaces 308 7. Scale - Number of routing protocol instances 310 8. Scale - Number of Routing Table entries 311 9. Scale - Number of Route Processors available 313 10. Performance - Redundancy strategy deployed for route 314 processors and line cards 316 11. Performance - Interface encapsulation as well as achievable 317 Throughput [RFC2544] 319 12. Any other internal or external factor that may influence 320 recovery time after a hardware or software reset 322 The recovery time is one of the key characterization results 323 reported after each test run. While the recovery time during a reset 324 test event may be zero, there may still be effects on traffic, such 325 as transient delay variation or increased latency. However, that is 326 not covered and deemed outside the scope of this document. In this 327 case, only "no loss" is reported. 329 4. Reset Test 331 This section contains the description of the tests that are related 332 to the characterization of the time needed for DUTs (Device Under 333 Test) / SUTs (System Under Test) to recover from a reset. There are 334 three types of reset considered in this document: 336 1. Hardware resets 338 2. Software resets 340 3. Power interruption 342 Different types of reset have potentially different impact on the 343 forwarding behavior of the device. As an example, a software reset 344 (of a routing process) might not result in forwarding interruption, 345 whereas a hardware reset (of a line card) most likely will. 347 Section 4.1 describes various hardware resets, whereas Section 4.2 348 describes various software resets. Additionally, Section 4.3 349 describes power interruption tests. These sections define and 350 characterize these resets. 352 Additionally, since device specific implementations may vary for 353 hardware and software type resets, it is desirable to classify each 354 test case as "REQUIRED" or "OPTIONAL". 356 4.1. Hardware Reset Test 358 A test designed to characterize the time it takes a DUT to recover 359 from the hardware reset. 361 A "hardware reset" generally involves the re-initialization of one 362 or more physical components in the DUT, but not the entire DUT. 364 A hardware reset is executed by the operator for example by physical 365 removal of a hardware component, by pressing on a "reset" button for 366 the component, or could even be triggered from the command line 367 interface. 369 Reset procedures that do not require the physical removal and 370 insertion of a hardware component are RECOMMENDED. These include 371 using the Command Line Interface (CLI) or a physical switch or 372 button. If such procedures cannot be performed (e.g., for lack of 373 platform support, or because the corresponding Test Case calls for 374 them), human operation time SHOULD be minimized across different 375 platforms and Test Cases as much as possible, and variation in human 376 operator time SHOULD also be minimized across different vendors 377 products as much as practical, by having the same person perform the 378 operation, and by practicing the operation. 380 For routers that do not contain separate Routing Processor and Line 381 Card modules, the hardware reset tests are not performed since they 382 are not relevant; instead, the power interruption tests MUST be 383 performed (see Section 4.3) in these cases. 385 4.1.1. Routing Processor (RP) / Routing Engine Reset 387 The Routing Processor (RP) is the DUT module that is primarily 388 concerned with Control Plane functions. 390 4.1.1.1. RP Reset for a single-RP device (REQUIRED) 392 Objective 394 To characterize time needed for a DUT to recover from a Route 395 processor hardware reset in a single RP environment. 397 Procedure 399 First, ensure that the RP is in a permanent state to which it will 400 return to after the reset, by performing some or all of the 401 following operational tasks: save the current DUT configuration, 402 specify boot parameters, ensure the appropriate software files are 403 available, or perform additional Operating System or hardware 404 related task. 406 Second, ensure that the DUT is able to forward the traffic for at 407 least 15 seconds before any test activities are performed. The 408 traffic should use the minimum frame size possible on the media 409 used in the testing and rate should be sufficient for the DUT to 410 attain the maximum forwarding throughput. This enables a finer 411 granularity in the recovery time measurement. 413 Third, perform the Route Processor (RP) hardware reset at this 414 point. This entails for example physically removing the RP to 415 later re-insert it, or triggering a hardware reset by other means 416 (e.g., command line interface, physical switch, etc.) 418 Finally, the characterization is completed by recording the frame 419 loss or time stamps (as reported by the test tool) and calculating 420 the recovery time (as defined in Section 2.3). 422 Reporting format 424 The reporting format is defined in Section 2.4. 426 4.1.1.2. RP Switchover for a multiple-RP device (OPTIONAL) 428 Objective 430 To characterize time needed for "secondary" Route Processor 431 (sometimes referred to as "backup" RP) of a DUT to become active 432 after a "primary" (or "active") Route Processor hardware reset. 433 This process is often referred to as "RP Switchover". The 434 characterization in this test should be done for the default DUT 435 behavior as well as a DUT's non-default configuration that 436 minimizes frame loss, if exists. 438 Procedure 440 This test characterizes "RP Switchover". Many implementations 441 allow for optimized switchover capabilities that minimize the 442 downtime during the actual switchover. This test consists of two 443 sub-cases from a switchover characteristics standpoint: First, a 444 default behavior (with no switchover-specific configurations); and 445 potentially second, a non-default behavior with switchover 446 configuration to minimize frame loss. Therefore, the procedures 447 hereby described are executed twice, and reported separately. 449 First, ensure that the RPs are in a permanent state such that the 450 secondary will be activated to the same state as the active is, by 451 performing some or all of the following operational tasks: save 452 the current DUT configuration, specify boot parameters, ensure the 453 appropriate software files are available, or perform additional 454 Operating System or hardware related task. 456 Second, ensure that the DUT is able to forward the traffic for at 457 least 15 seconds before any test activities are performed. The 458 traffic should use the minimum frame size possible on the media 459 used in the testing and rate should be sufficient for the DUT to 460 attain the maximum forwarding throughput. This enables a finer 461 granularity in the recovery time measurement. 463 Third, perform the primary Route Processor (RP) hardware reset at 464 this point. This entails for example physically removing the RP, 465 or triggering a hardware reset by other means (e.g., command line 466 interface, physical switch, etc.) It is up to the operator to 467 decide if the primary RP needs to be re-inserted after a grace 468 period or not. 470 Finally, the characterization is completed by recording the frame 471 loss or time stamps (as reported by the test tool) and calculating 472 the recovery time (as defined in Section 2.3). 474 Reporting format 476 The reset results are potentially reported twice, one for the 477 default switchover behavior (i.e., the DUT without any switchover- 478 specific enhanced configuration) and the other for the switchover- 479 specific behavior if it exists (i.e., the DUT configured for 480 optimized switchover capabilities that minimize the downtime 481 during the actual switchover). 483 The reporting format is defined in Section 2.4, and also includes 484 any specific redundancy scheme in place. 486 4.1.2. Line Card (LC) Removal and Insertion (REQUIRED) 488 The Line Card (LC) is the DUT component that is responsible with 489 packet forwarding. 491 Objective 493 To characterize time needed for a DUT to recover from a Line Card 494 removal and insertion event. 496 Procedure 498 For this test, the Line Card that is being hardware-reset MUST be 499 on the forwarding path and all destinations MUST be directly 500 connected. 502 First, complete some or all of the following operational tasks: 503 save the current DUT configuration, specify boot parameters, 504 ensure the appropriate software files are available, or perform 505 additional Operating System or hardware related task. 507 Second, ensure that the DUT is able to forward the traffic for at 508 least 15 seconds before any test activities are performed. The 509 traffic should use the minimum frame size possible on the media 510 used in the testing and rate should be sufficient for the DUT to 511 attain the maximum forwarding throughput. This enables a finer 512 granularity in the recovery time measurement. 514 Third, perform the Line Card (LC) hardware reset at this point. 515 This entails for example physically removing the LC to later re- 516 insert it, or triggering a hardware reset by other means (e.g., 517 CLI, physical switch, etc.). 519 Finally, the characterization is completed by recording the frame 520 loss or time stamps (as reported by the test tool) and calculating 521 the recovery time (as defined in Section 2.3). 523 Reporting Format 525 The reporting format is defined in Section 2.4. 527 4.2. Software Reset Test 529 To characterize time needed for a DUT to recover from the software 530 reset. 532 In contrast to a "hardware reset", a "software reset" involves only 533 the re-initialization of the execution, data structures, and partial 534 state within the software running on the DUT module(s). 536 A software reset is initiated for example from the DUT's CLI. 538 4.2.1. Operating System (OS) Reset (REQUIRED) 540 Objective 542 To characterize time needed for a DUT to recover from an Operating 543 System (OS) software reset. 545 Procedure 547 First, complete some or all of the following operational tasks: 548 save the current DUT configuration, specify software boot 549 parameters, ensure the appropriate software files are available, 550 or perform additional Operating System task. 552 Second, ensure that the DUT is able to forward the traffic for at 553 least 15 seconds before any test activities are performed. The 554 traffic should use the minimum frame size possible on the media 555 used in the testing and rate should be sufficient for the DUT to 556 attain the maximum forwarding throughput. This enables a finer 557 granularity in the recovery time measurement. 559 Third, trigger an Operating System re-initialization in the DUT, 560 by operational means such as use of the DUT's CLI or other 561 management interface. 563 Finally, the characterization is completed by recording the frame 564 loss or time stamps (as reported by the test tool) and calculating 565 the recovery time (as defined in Section 2.3). 567 Reporting format 569 The reporting format is defined in Section 2.4. 571 4.2.2. Process Reset (OPTIONAL) 573 Objective 574 To characterize time needed for a DUT to recover from a software 575 process reset. 577 Such time period may depend upon the number and types of process 578 running in the DUT and which ones are tested. Different 579 implementations of forwarding devices include various common 580 processes. A process reset should be performed only in the 581 processes most relevant to the tester and most impactful to 582 forwarding. 584 Procedure 586 First, complete some or all of the following operational tasks: 587 save the current DUT configuration, specify software parameters or 588 environmental variables, or perform additional Operating System 589 task. 591 Second, ensure that the DUT is able to forward the traffic for at 592 least 15 seconds before any test activities are performed. The 593 traffic should use the minimum frame size possible on the media 594 used in the testing and rate should be sufficient for the DUT to 595 attain the maximum forwarding throughput. This enables a finer 596 granularity in the recovery time measurement. 598 Third, trigger a process reset for each process running in the DUT 599 and considered for testing from a management interface (e.g., by 600 means of the CLI, etc.) 602 Finally, the characterization is completed by recording the frame 603 loss or time stamps (as reported by the test tool) and calculating 604 the recovery time (as defined in Section 2.3). 606 Reporting format 608 The reporting format is defined in Section 2.4, and is used for 609 each process running in the DUT and tested. Given the 610 implementation nature of this test, details of the actual process 611 tested should be included along with the statement. 613 4.3. Power Interruption Test 615 "Power interruption" refers to the complete loss of power on the 616 DUT. It can be viewed as a special case of a hardware reset, 617 triggered by the loss of the power supply to the DUT or its 618 components, and is characterized by the re-initialization of all 619 hardware and software in the DUT. 621 4.3.1. Power Interruption (REQUIRED) 623 Objective 625 To characterize time needed for a DUT to recover from a complete 626 loss of electric power or complete power interruption. This test 627 simulates a complete power failure or outage, and should be 628 indicative of the DUT/SUTs behavior during such event. 630 Procedure 632 First, ensure that the entire DUT is at a permanent state to which 633 it will return to after the power interruption, by performing some 634 or all of the following operational tasks: save the current DUT 635 configuration, specify boot parameters, ensure the appropriate 636 software files are available, or perform additional Operating 637 System or hardware related task. 639 Second, ensure that the DUT is able to forward the traffic for at 640 least 15 seconds before any test activities are performed. The 641 traffic should use the minimum frame size possible on the media 642 used in the testing and rate should be sufficient for the DUT to 643 attain the maximum forwarding throughput. This enables a finer 644 granularity in the recovery time measurement. 646 Third, interrupt the power (AC or DC) that feeds the corresponding 647 DUTs power supplies at this point. This entails for example 648 physically removing the power supplies in the DUT to later re- 649 insert them, or simply disconnecting or switching off their power 650 feeds (AC or DC as applicable). The actual power interruption 651 should last at least 15 seconds. 653 Finally, the characterization is completed by recording the frame 654 loss or time stamps (as reported by the test tool) and calculating 655 the recovery time (as defined in Section 2.3). 657 For easier comparison with other testing, the 15 seconds are 658 removed from the reported recovery time. 660 Reporting format 662 The reporting format is defined in Section 2.4. 664 5. Security Considerations 666 Benchmarking activities, as described in this memo, are limited to 667 technology characterization using controlled stimuli in a laboratory 668 environment, with dedicated address space and the constraints 669 specified in the sections above. 671 The benchmarking network topology will be an independent test setup 672 and MUST NOT be connected to devices that may forward the test 673 traffic into a production network or misroute traffic to the test 674 management network. 676 Furthermore, benchmarking is performed on a "black-box" basis, 677 relying solely on measurements observable external to the DUT/SUT. 679 Special capabilities SHOULD NOT exist in the DUT/SUT specifically 680 for benchmarking purposes. Any implications for network security 681 arising from the DUT/SUT SHOULD be identical in the lab and in 682 production networks. 684 There are no specific security considerations within the scope of 685 this document. 687 6. IANA Considerations 689 There is no IANA consideration for this document. 691 7. Acknowledgments 693 The authors would like to thank Ron Bonica, who motivated us to 694 write this document. The authors would also like to thank Al Morton, 695 Andrew Yourtchenko, David Newman, John E. Dawson, Timmons C. Player, 696 Jan Novak, Steve Maxwell, Ilya Varlashkin, and Sarah Banks for 697 providing thorough review, useful suggestions, and valuable input. 699 This document was prepared using 2-Word-v2.0.template.dot. 701 8. References 703 8.1. Normative References 705 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 706 Requirement Levels", BCP 14, RFC 2119, March 1997. 708 [RFC2544] Bradner, S. and McQuaid, J., "Benchmarking Methodology for 709 Network Interconnect Devices", RFC 2544, March 1999. 711 8.2. Informative References 713 [IGPConv] Poretsky, S., Imhoff, B., and K. Michielsen, "Benchmarking 714 Methodology for Link-State IGP Data Plane Route 715 Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-21 716 (work in progress), May 2010. 718 [RFC5180] Popoviciu, C., et al, "IPv6 Benchmarking Methodology for 719 Network Interconnect Devices", RFC 5180, May 2008. 721 [RFC5695] Akhter, A., Asati, R., and C. Pignataro, "MPLS Forwarding 722 Benchmarking Methodology for IP Flows", RFC 5695, November 723 2009. 725 Authors' Addresses 727 Rajiv Asati 728 Cisco Systems 729 7025-6 Kit Creek Road 730 RTP, NC 27709 731 USA 733 Email: rajiva@cisco.com 735 Carlos Pignataro 736 Cisco Systems 737 7200-12 Kit Creek Road 738 RTP, NC 27709 739 USA 741 Email: cpignata@cisco.com 743 Fernando Calabria 744 Cisco Systems 745 7200-12 Kit Creek Road 746 RTP, NC 27709 747 USA 749 Email: fcalabri@cisco.com 751 Cesar Olvera 752 Consulintel 753 Joaquin Turina, 2 754 Pozuelo de Alarcon, Madrid, E-28224 755 Spain 757 Email: cesar.olvera@consulintel.es