idnits 2.17.1 draft-ietf-bmwg-acc-bench-meth-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3667, Section 5.1 on line 20. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 742. ** The document claims conformance with section 10 of RFC 2026, but uses some RFC 3978/3979 boilerplate. As RFC 3978/3979 replaces section 10 of RFC 2026, you should not claim conformance with it if you have changed to using RFC 3978/3979 boilerplate. ** The document seems to lack an RFC 3978 Section 5.1 IPR Disclosure Acknowledgement -- however, there's a paragraph with a matching beginning. Boilerplate error? ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** The document seems to lack an RFC 3978 Section 5.5 (updated by RFC 4748) Disclaimer -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack an RFC 3979 Section 5, para. 1 IPR Disclosure Acknowledgement -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack an RFC 3979 Section 5, para. 3 IPR Disclosure Invitation -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document uses RFC 3667 boilerplate or RFC 3978-like boilerplate instead of verbatim RFC 3978 boilerplate. After 6 May 2005, submission of drafts without verbatim RFC 3978 boilerplate is not accepted. The following non-3978 patterns matched text found in the document. That text should be removed or replaced: By submitting this Internet-Draft, I certify that any applicable patent or other IPR claims of which I am aware have been disclosed, or will be disclosed, and any of which I become aware will be disclosed, in accordance with RFC 3668. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard == It seems as if not all pages are separated by form feeds - found 0 form feeds but 16 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. (A line matching the expected section header was found, but with an unexpected indentation: ' 1. Introduction' ) ** The document seems to lack a Security Considerations section. (A line matching the expected section header was found, but with an unexpected indentation: ' 5. Security Considerations' ) ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack an Authors' Addresses Section. ** There are 38 instances of too long lines in the document, the longest one being 13 characters in excess of 72. ** There are 218 instances of lines with control characters in the document. ** The abstract seems to contain references ([2], [RFC3871], [3], [CONVMETH], [4], [IEEECQR], [NANOG25], [5], [Br97], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 2005) is 6765 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '4' on line 684 looks like a reference -- Missing reference section? 'Br97' on line 132 looks like a reference -- Missing reference section? '5' on line 688 looks like a reference -- Missing reference section? '1' on line 675 looks like a reference -- Missing reference section? '2' on line 678 looks like a reference -- Missing reference section? '3' on line 681 looks like a reference -- Missing reference section? 'RFC3871' on line 694 looks like a reference -- Missing reference section? 'NANOG25' on line 698 looks like a reference -- Missing reference section? 'IEEECQR' on line 701 looks like a reference -- Missing reference section? 'CONVMETH' on line 704 looks like a reference Summary: 15 errors (**), 0 flaws (~~), 3 warnings (==), 14 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group 3 INTERNET-DRAFT 4 Expires in: October 2005 5 Scott Poretsky 6 Quarry Technologies 8 Shankar Rao 9 Qwest Communications 11 February 2005 13 Methodology for Accelerated Stress Benchmarking 14 16 Intellectual Property Rights (IPR) statement: 17 By submitting this Internet-Draft, I certify that any applicable 18 patent or other IPR claims of which I am aware have been disclosed, or 19 will be disclosed, and any of which I become aware will be disclosed, 20 in accordance with RFC 3668. 22 Status of this Memo 24 This document is an Internet-Draft and is in full conformance with 25 all provisions of Section 10 of RFC2026. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as Internet- 30 Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six 33 months and may be updated, replaced, or obsoleted by other 34 documents at any time. It is inappropriate to use Internet-Drafts 35 as reference material or to cite them other than as "work in 36 progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/ietf/1id-abstracts.txt 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html. 44 ABSTRACT 45 Routers in an operational network are simultaneously configured with 46 multiple protocols and security policies while forwarding traffic and 47 being managed. To accurately benchmark a router for deployment it is 48 necessary that the router be tested in these simultaneous 49 operational conditions, which is known as Stress Testing. This 50 document provides the Methodology for performing Stress Benchmarking 51 of networking devices. Descriptions of Test Topology, Benchmarks and 52 Reporting Format are provided in addition to procedures for 53 conducting various test cases. The methodology is to be used with 54 the companion terminology document [4]. 56 Table of Contents 57 1. Introduction ............................................... 2 58 2. Existing definitions ....................................... 3 59 3. Test Setup.................................................. 3 60 3.1 Test Topologies............................................ 3 61 3.2 Test Considerations........................................ 3 62 3.3 Reporting Format........................................... 4 63 3.3.1 Configuration Sets....................................... 5 64 3.3.2 Instability Conditions................................... 6 65 3.3.3 Benchmarks............................................... 6 66 4. Test Cases.................................................. 7 67 4.1 Failed Primary EBGP Peer................................... 7 68 4.2 Establish New EBGP Peer.................................... 8 69 4.3 BGP Route Explosion........................................ 8 70 4.4 BGP Policy Configuration................................... 9 71 4.5 Persistent BGP Flapping.................................... 9 72 4.6 BGP Route Flap Dampening...................................10 73 4.7 Nested Convergence Events..................................11 74 4.8 Restart Under Load.........................................12 75 4.9 Destination Control Processor..............................12 76 4.10 Destination Control Processor with Rate-Limiting..........13 77 4.11 Destination Interfaces....................................13 78 4.12 DoS Attack................................................14 79 5. Security Considerations.....................................14 80 6. Normative References........................................14 81 7. Normative References........................................15 82 8. Author's Address............................................15 84 1. Introduction 85 Router testing benchmarks have consistently been made in a 86 monolithic fashion wherein a single protocol or behavior is 87 measured in an isolated environment. It is important to know the 88 limits for a networking device's behavior for each protocol in isolation, 89 however this does not produce a reliable benchmark of the device's 90 behavior in an operational network. 92 Routers in an operational network are simultaneously configured with 93 multiple protocols and security policies while forwarding traffic 94 and being managed. To accurately benchmark a router for deployment 95 it is necessary to test that router in operational conditions by 96 simultaneously configuring and scaling network protocols and security 97 policies, forwarding traffic, and managing the device. It is helpful 98 to accelerate these network operational conditions with Instability 99 Conditions [4] so that the networking devices are stress tested. 101 This document provides the Methodology for performing Stress 102 Benchmarking of networking devices. Descriptions of Test Topology, 103 Benchmarks and Reporting Format are provided in addition to 104 procedures for conducting various test cases. The methodology is 105 to be used with the companion terminology document [4]. 107 Stress Testing of networking devices provides the following benefits: 108 1. Evaluation of multiple protocols enabled simultaneously as 109 configured in deployed networks 110 2. Evaluation of System and Software Stability 111 3. Evaluation of Manageability under stressful conditions 112 4. Identification of Buffer Overflow conditions 113 5. Identification of Software Coding bugs such as: 114 a. Memory Leaks 115 b. Suboptimal CPU Utilization 116 c. Coding Logic 118 These benefits produce significant advantages for network operations: 119 1. Increased stability of routers and protocols 120 2. Hardened routers to DoS attacks 121 3. Verified manageability under stress 122 4. Planning router resources for growth and scale 124 2. Existing definitions 125 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 126 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 127 document are to be interpreted as described in BCP 14, RFC 2119 128 [Br97]. RFC 2119 defines the use of these key words to help make the 129 intent of standards track documents as clear as possible. While this 130 document uses these keywords, this document is not a standards track 131 document. 133 Terms related to Accelerated Stress Benchmarking are defined in [4]. 135 3. Test Setup 136 3.1 Test Topologies 137 Figure 1 shows the physical configuration to be used for the 138 methodologies provided in this document. The number of interfaces 139 between the tester and DUT will scale depending upon the number of 140 control protocol sessions and traffic forwarding interfaces. A 141 separate device may be required to externally manage the device in 142 the case that the test equipment does not support such 143 functionality. Figure 2 shows the logical configuration for the 144 stress test methodologies. Each plane may be emulated by single or 145 multiple test equipment. 147 3.2 Test Considerations 148 The Accelerated Stress Benchmarking test can be applied in 149 service provider test environments to benchmark DUTs under 150 stress in an environment that is reflective of an operational 151 network. A particular Configuration Set is defined and the 152 DUT is benchmarked using this configuration set and the 153 Instability Conditions. Varying Configuration Sets and/or 154 Instability Conditions applied in an iterative fashion can 155 provide an accurate characterization of the DUT 156 to help determine future network deployments. 158 ___________ 159 | DUT | 160 ___|Management | 161 | | | 162 | ----------- 163 \/ 164 ___________ 165 | | 166 | DUT | 167 |--->| |<---| 168 xN | ----------- | xN 169 interfaces | | interfaces 170 | ___________ | 171 | | | | 172 |--->| Tester |<---| 173 | | 174 ----------- 176 Figure 1. Physical Configuration 178 ___________ ___________ 179 | Control | | Management| 180 | Plane |___ ___| Plane | 181 | | | | | | 182 ----------- | | ----------- 183 \/ \/ ___________ 184 ___________ | Security | 185 | |<-----------| Plane | 186 | DUT | | | 187 |--->| |<---| ----------- 188 | ----------- | 189 | | 190 | ___________ | 191 | | Data | | 192 |--->| Plane |<---| 193 | | 194 ----------- 196 Figure 2. Logical Configuration 198 3.3 Reporting Format 200 Each methodology requires reporting of information for test 201 repeatability when benchmarking the same or different devices. 202 The information that are the Configuration Sets, Instability 203 Conditions, and Benchmarks, as defined in [4]. Example 204 reporting formats for each are provided below. 206 3.3.1 Configuration Sets 207 Example Routing Protocol Configuration Set- 208 PARAMETER UNITS 209 BGP Enabled/Disabled 210 Number of EBGP Peers Peers 211 Number of IBGP Peers Peers 212 Number of BGP Route Instances Routes 213 Number of BGP Installed Routes Routes 215 MBGP Enabled/Disabled 216 Number of MBGP Route Instances Routes 217 Number of MBGP Installed Routes Routes 219 IGP Enabled/Disabled 220 IGP-TE Enabled/Disabled 221 Number of IGP Adjacencies Adjacencies 222 Number of IGP Routes Routes 223 Number of Nodes per Area Nodes 225 Example MPLS Protocol Configuration Set- 226 PARAMETER UNITS 227 MPLS-TE 228 Number of Ingress Tunnels Tunnels 229 Number of Mid-Point Tunnels Tunnels 230 Number of Egress Tunnels Tunnels 232 LDP 233 Number of Sessions Sessions 234 Number of FECs FECs 236 Example Multicast Protocol Configuration Set- 237 PARAMETER UNITS 238 PIM-SM Enabled/Disabled 239 RP Enabled/Disabled 240 Number of Multicast Groups Groups 241 MSDP Enabled/Disabled 243 Example Data Plane Configuration Set- 244 PARAMETER UNITS 245 Traffic Forwarding Enabled/Disabled 246 Aggregate Offered Load bps (or pps) 247 Number of Ingress Interfaces number 248 Number of Egress Interfaces number 250 TRAFFIC PROFILE 251 Packet Size(s) bytes 252 Packet Rate(interface) array of packets per second 253 Number of Flows number 254 Encapsulation(flow) array of encapsulation type 255 Management Configuration Set- 256 PARAMETER UNITS 257 SNMP GET Rate SNMP Gets/minute 258 Logging Enabled/Disabled 259 Protocol Debug Enabled/Disabled 260 Telnet Rate Sessions/Hour 261 FTP Rate Sessions/Hour 262 Concurrent Telnet Sessions Sessions 263 Concurrent FTP Session Sessions 264 Packet Statistics Collector Enabled/Disabled 265 Statistics Sampling Rate X:1 packets 267 Security Configuration Set - 268 PARAMETER UNITS 269 Packet Filters Enabled/Disabled 270 Number of Filters For-Me number 271 Number of Filter Rules For-Me number 272 Number of Traffic Filters number 273 Number of Traffic Filter Rules number 274 SSH Enabled/Disabled 275 Number of simultaneous SSH sessions number 276 RADIUS Enabled/Disabled 277 TACACS Enabled/Disabled 279 3.3.2 Instability Conditions 280 PARAMETER UNITS 281 Interface Shutdown Cycling Rate interfaces per minute 282 BGP Session Flap Rate sessions per minute 283 BGP Route Flap Rate routes per minutes 284 IGP Route Flap Rate routes per minutes 285 LSP Reroute Rate LSP per minute 286 Overloaded Links number 287 Amount Links Overloaded % of bandwidth 288 FTP Rate Mb/minute 289 IPsec Session Loss sessions per minute 290 Filter Policy Changes policies per hour 291 SSH Session Restart SSH sessions per hour 292 Telnet Session Restart Telnet session per hour 294 3.3.3 Benchmarks 295 PARAMETER UNITS PHASE 296 Stable Aggregate Forwarding Rate pps Startup 297 Stable Latency seconds Startup 298 Stable Session Count sessions Startup 299 Unstable Aggregate Forwarding Rate pps Instability 300 Degraded Aggregate Forwarding Rate pps Instability 301 Ave. Degraded Aggregate Forwarding Rate pps Instability 302 Unstable Latency seconds Instability 303 Unstable Uncontrolled Sessions Lost sessions Instability 304 Recovered Aggregate Forwarding Rate pps Recovery 305 Recovered Latency seconds Recovery 306 Recovery Time seconds Recovery 307 Recovered Uncontrolled Sessions Lost sessions Recovery 308 It is RECOMMENDED that Aggregate Forwarding Rates, Latencies, and 309 Session Losses be measured at one-second intervals. These same 310 benchmarks can also be used as Variability Benchmarks reported as 311 the differences between the Benchmarks for multiple iterations 312 with the same DUT. For the purpose of the Variability Benchmarks, 313 A more complete characterization of the DUT would be to apply 314 multiple test iterations for the same Configuration Sets and 315 Instability Conditions, measure the Variability Benchmarks, and 316 then vary the Configuration Set and/or Instability Conditions. 318 4. Test Cases 320 4.1 Failed Primary EBGP Peer 322 Objective 323 The purpose of this test is to benchmark the performance 324 of the DUT during stress conditions when losing an EBGP 325 Peer from which most FIB routes have been learned. 327 Procedure 328 1. Report Configuration Set 329 2. Begin Startup Conditions with the DUT 330 3. Establish Configuration Sets with the DUT 331 4. Report benchmarks (for stability) 332 5. Apply Instability Conditions 333 6. Remove link to EBGP peer with most FIB routes. This SHOULD 334 be achieved by losing physical layer connectivity with 335 a local fiber pull. Loss of the peering session SHOULD 336 cause the DUT to withdraw 100,000 or greater routes. 337 7. Report benchmarks (for instability) 338 8. Stop applying all Instability Conditions 339 9. Report benchmarks (for recovery) 340 10. Optional - Change Configuration Set and/or Instability 341 Conditions for next iteration 343 Results 344 It is expected that there will be significant packet loss 345 until the DUT converges from the lost EBGP link. Other DUT 346 operation should be stable without session loss or sustained 347 packet loss. Recovery time should not be infinite. 349 4.2 Establish New EBGP Peer 351 Objective 352 The purpose of this test is to benchmark the performance 353 of the DUT during stress conditions when establishing a 354 new EBGP Peer from which routes are learned. 356 Procedure 357 1. Report Configuration Set 358 2. Begin Startup Conditions with the DUT 359 3. Establish Configuration Sets with the DUT 360 4. Report benchmarks (for stability) 361 5. Apply Instability Conditions 362 6. Configure a new EBGP peering session at DUT and peering router. 363 Physical and Data Link Layer connectivity SHOULD already exist 364 to perform this step. Establishment of the peering session 365 MUST result in the DUT learning 100,000 or greater routes from 366 the BGP peer and advertising 100,000 or greater routes to 367 the BGP peer 368 7. Report benchmarks (for instability) 369 8. Stop applying all Instability Conditions 370 9. Report benchmarks (for recovery) 371 10. Optional - Change Configuration Set and/or Instability 372 Conditions for next iteration 374 Results 375 It is expected that there will be zero packet loss as the DUT 376 learns the new routes. Other DUT operation should be stable 377 without session loss or sustained packet loss. 379 4.3 BGP Route Explosion 381 Objective 382 The purpose of this test is to benchmark the performance 383 of the DUT during stress conditions when there is BGP Route 384 Explosion experienced in the network. 386 Procedure 387 1. Report Configuration Set 388 2. Begin Startup Conditions with the DUT 389 3. Establish Configuration Sets with the DUT 390 4. Report benchmarks (for stability) 391 5. Apply Instability Conditions 392 6. Advertise 1M BGP routes to the DUT from a single EBGP 393 neighbor. 394 7. Report benchmarks (for instability) 395 8. Stop applying all Instability Conditions, including BGP route 396 advertisement. 397 9. Report benchmarks (for recovery) 398 10. Optional - Change Configuration Set and/or Instability 399 Conditions for next iteration 400 Results 401 It is expected that there will be no additional packet loss from 402 the advertisement of routes from the BGP neighbor. Other 403 DUT operation should be stable without session loss. 405 4.4 BGP Policy Configuration 407 Objective 408 The purpose of this test is to benchmark the performance 409 of the DUT during stress conditions when there is continuous 410 reconfiguration of BGP Policy at the DUT. 412 Procedure 413 1. Report Configuration Set 414 2. Begin Startup Conditions with the DUT 415 3. Establish Configuration Sets with the DUT 416 4. Report benchmarks (for stability) 417 5. Apply Instability Conditions 418 6. Configure BGP Policy on the DUT for each established neighbor. 419 The BGP Policy SHOULD filter 25% of the routes learned from 420 that neighbor. Note that the specific policy configuration 421 to achieve the filtering may be device specific. 422 7. Every 30 minutes remove the BGP Policy configuration and then 423 configure it gain so that it is reapplied. 424 8. Report benchmarks (for instability) 425 9. Stop applying all Instability Conditions, including Policy 426 changes 427 10. Report benchmarks (for recovery) 428 11. Optional - Change Configuration Set and/or Instability 429 Conditions for next iteration 431 Results 432 It is expected that there will be no packet loss resulting from 433 the continuous configuration and removal of BGP Policy for BGP 434 neighbors. Other DUT operation should be stable without session 435 loss. 437 4.5 Persistent BGP Flapping 439 Objective 440 The purpose of this test is to benchmark the performance 441 of the DUT during stress conditions when flapping BGP Peering 442 sessions for an infinite period. 444 Procedure 445 1. Report Configuration Set 446 2. Begin Startup Conditions with the DUT 447 3. Establish Configuration Sets with the DUT 448 4. Report benchmarks (for stability) 449 5. Apply Instability Conditions 450 6. Repeatedly flap an IBGP and an EBGP peering session. 451 This SHOULD be achieved by losing physical layer connectivity 452 via a local interface shutdown/no shutdown every 180 seconds with 453 a delay of 10 seconds between the shut and no shut. 454 The loss of the EBGP peering session MUST cause the DUT to withdraw 455 100,000 or greater routes that are re-learned when the session 456 re-establishes. Route Flap Dampening SHOULD NOT be enabled. 457 7. Report benchmarks (for instability) 458 8. Stop applying all Instability Conditions, including flapping 459 9. Report benchmarks (for recovery) 460 10. Optional - Change Configuration Set and/or Instability 461 Conditions for next iteration 463 Results 464 It is expected that there will be significant packet loss 465 from repeated Convergence Events. Other DUT operation should be 466 stable without session loss. Recovery time should not be infinite. 468 4.6 BGP Route Flap Dampening 470 Objective 471 The purpose of this test is to benchmark the performance 472 of the DUT during stress conditions when flapping BGP Peering 473 sessions for an infinite period and route flap dampening is 474 enabled. 476 Procedure 477 1. Report Configuration Set 478 2. Configure Route Flap Dampening on the DUT with DEFAULT parameter 479 values. 480 3. Begin Startup Conditions with the DUT 481 4. Establish Configuration Sets with the DUT 482 5. Report benchmarks (for stability) 483 6. Apply Instability Conditions 484 7. Repeatedly flap an IBGP and an EBGP peering session. 485 This SHOULD be achieved by losing physical layer connectivity 486 via a local interface shutdown/no shutdown every 180 seconds with 487 a delay of 10 seconds between the shut and no shut. 488 The loss of the EBGP peering session MUST cause the DUT to withdraw 489 100,000 or greater routes that are re-learned when the session 490 re-establishes. 491 8. Report benchmarks (for instability) 492 9. Stop applying all Instability Conditions 493 10. Report benchmarks (for recovery) 494 11. Optional - Change Route Flap Dampening parameter values 495 12. Optional - Change Configuration Set and/or Instability 496 Conditions for next iteration 498 Results 499 It is expected that there will be significant packet loss 500 from repeated Convergence Events and flap dampening. Other DUT operation 501 should be stable without session loss. Recovery time should not be infinite. 503 4.7 Nested Convergence Events [5] 505 Objective 506 The purpose of this test is to benchmark the performance 507 of the DUT during stress conditions when flapping BGP Peering 508 sessions causes Nested Convergence Events. 510 Procedure 511 1. Report Configuration Set 512 2. Begin Startup Conditions with the DUT 513 3. Establish Configuration Sets with the DUT 514 4. Report benchmarks (for stability) 515 5. Apply Instability Conditions 516 6. Repeatedly flap an IBGP and an EBGP peering session. 517 This SHOULD be achieved by losing physical layer connectivity 518 via a local interface shutdown/no shutdown every 10 seconds with 519 a delay of 1 second between the shut and no shut. 520 The loss of the EBGP peering session MUST cause the DUT to withdraw 521 100,000 or greater routes that are re-learned when the session 522 re-establishes. Route Flap Dampening SHOULD NOT be enabled. 523 7. Report benchmarks (for instability) 524 8. Stop applying all Instability Conditions, including flapping 525 9. Report benchmarks (for recovery) 526 10. Optional - Change Configuration Set and/or Instability 527 Conditions for next iteration 529 Results 530 It is expected that there will be significant packet loss 531 from Nested Convergence Events. New Other DUT operation should be 532 stable without session loss. Recovery time should not be infinite. 534 4.8 Restart Under Load 536 Objective 537 The purpose of this test is to benchmark the performance of the DUT 538 during restart when stress conditions are applied. 540 Procedure 541 1. Report Configuration Set 542 2. Begin Startup Conditions with the DUT 543 3. Establish Configuration Sets with the DUT 544 4. Report benchmarks (for stability) 545 5. Restart DUT. This marks the beginning on the recovery period. 546 6. Report benchmarks (for recovery) 547 7. Optional - Change Configuration Set and/or Instability 548 Conditions for next iteration 549 NOTE 1: Restart via the DUT's Command Line Interface rather than 550 power cycle is typically more stressful than power cycle 551 since hardware can maintain state. 552 NOTE 2: Instability Conditions are not applied for this test case. 554 Results 555 DUT should re-establish all control protocol sessions and have 556 a Recovery Time [4] that is not infinite. 558 4.9 Destination Control Processor 560 Objective 561 The purpose of this test is to benchmark the performance 562 of the DUT during stress conditions when traffic is destined for 563 the Control Processor of the DUT. 565 Procedure 566 1. Report Configuration Set 567 2. Begin Startup Conditions with the DUT 568 3. Start Configuration Sets with the DUT, except Data Plane 569 Configuration Set 570 4. Report benchmarks (for stability) 571 5. Apply Instability Conditions 572 6. Send offered load at maximum forwarding rate of DUT interfaces 573 to all DUT interfaces. Traffic MUST be configured so that the 574 offered load has a destination address that is the DUT's central 575 control processor 576 7. Report benchmarks (for instability) 577 8. Stop applying all Instability Conditions, including data traffic 578 9. Report benchmarks (for recovery) 579 10. Optional - Change Configuration Set and/or Instability 580 Conditions for next iteration 582 Results 583 Results will vary with specific vendor implementations. 584 It is possible that significant session loss is observed. 586 4.10 Destination Control Processor with Rate-Limiting 588 Objective 589 The purpose of this test is to benchmark the performance 590 of the DUT during stress conditions when traffic is destined for 591 the Control processor of the DUT. 593 Procedure 594 1. Report Configuration Set 595 2. Apply policy filter to rate-limit traffic arriving at the Central 596 Processor to be only 1% of the offered load. 597 3. Begin Startup Conditions with the DUT 598 4. Start Configuration Sets with the DUT, except Data Plane 599 Configuration Set 600 5. Report benchmarks (for stability) 601 6. Apply Instability Conditions 602 7. Send offered load at maximum forwarding rate of DUT interfaces 603 to all DUT interfaces. Traffic MUST be configured so that the 604 offered load has a destination address that is the DUT's central 605 control processor 606 8. Report benchmarks (for instability) 607 9. Stop applying all Instability Conditions, including data traffic 608 10. Report benchmarks (for recovery) 609 11. Optional - Change Configuration Set and/or Instability 610 Conditions for next iteration 612 Results 613 Results will vary with specific vendor implementations. There should be 614 no session loss observed. 616 4.11 Destination Interfaces 617 Objective 618 The purpose of this test is to benchmark the performance 619 of the DUT during stress conditions when traffic is destined for 620 the interfaces of the DUT. 622 Procedure 623 1. Report Configuration Set 624 2. Begin Startup Conditions with the DUT 625 3. Start Configuration Sets with the DUT, except Data Plane 626 Configuration Set 627 4. Report benchmarks (for stability) 628 5. Apply Instability Conditions 629 6. Send offered load at maximum forwarding rate of DUT interfaces 630 to all DUT interfaces. Traffic MUST be configured so that the 631 offered load has destination addresses of the interfaces receiving 632 traffic. 633 7. Report benchmarks (for instability) 634 8. Stop applying all Instability Conditions, including data traffic 635 9. Report benchmarks (for recovery) 636 10. Optional - Change Configuration Set and/or Instability 637 Conditions for next iteration 639 Results 640 Results will vary with specific vendor implementations. 641 There should be no session loss observed. 643 4.12 DoS Attack 645 Objective 646 The purpose of this test is to benchmark the performance of the 647 DUT during stress conditions while experiencing a DoS attack. 649 Procedure 650 1. Report Configuration Set 651 2. Begin Startup Conditions with the DUT 652 3. Establish Configuration Sets with the DUT 653 4. Report benchmarks (for stability) 654 5. Apply Instability Conditions 655 6. Initiate DoS Attack against DUT. It is RECOMMENDED that 656 the SYN Flood attack be used for the DoS attack. 657 7. Report benchmarks (for instability) 658 8. Stop applying all Instability Conditions 659 9. Report benchmarks (for recovery) 660 10. Optional - Change Configuration Set and/or Instability 661 Conditions for next iteration 663 Results 664 DUT should be able to defend against DoS attack without additional 665 packet loss or session loss. 667 5. Security Considerations 668 Documents of this type do not directly affect the security of 669 the Internet or of corporate networks as long as benchmarking 670 is not performed on devices or systems connected to operating 671 networks. 673 6. Normative References 675 [1] Bradner, S., Editor, "Benchmarking Terminology for Network 676 Interconnection Devices", RFC 1242, February 1991. 678 [2] Mandeville, R., "Benchmarking Terminology for LAN Switching 679 Devices", RFC 2285, June 1998. 681 [3] Bradner, S. and McQuaid, J., "Benchmarking Methodology for 682 Network Interconnect Devices", RFC 2544, March 1999. 684 [4] Poretsky, S. and Rao, S., "Terminology for Accelerated 685 Stress Benchmarking", draft-ietf-bmwg-acc-bench-term-05, 686 work in progress, February 2005. 688 [5] Poretsky, S., "Benchmarking Terminology for IGP Data Plane 689 Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-05, 690 work in progress, February 2005. 692 7. Informative References 694 [RFC3871] RFC 3871 "Operational Security Requirements for Large 695 Internet Service Provider (ISP) IP Network Infrastructure. 696 G. Jones, Ed.. IETF, September 2004. 698 [NANOG25] "Core Router Evaluation for Higher Availability", Scott 699 Poretsky, NANOG 25, June 8, 2002, Toronto, CA. 701 [IEEECQR] "Router Stress Testing to Validate Readiness for Network 702 Deployment", Scott Poretsky, IEEE CQR 2003. 704 [CONVMETH] Poretsky, S., "Benchmarking Methodology for IGP Data Plane 705 Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-05, 706 work in progress, February 2005. 708 8. Author's Address 710 Scott Poretsky 711 Quarry Technologies 712 8 New England Executive Park 713 Burlington, MA 01803 714 USA 716 Phone: + 1 781 395 5090 717 EMail: sporetsky@quarrytech.com 719 Shankar Rao 720 950 17th Street 721 Suite 1900 722 Qwest Communications 723 Denver, CO 80210 724 USA 725 Phone: + 1 303 437 6643 726 Email: shankar.rao@qwest.com 727 Intellectual Property Statement 729 The IETF takes no position regarding the validity or scope of any Intel- 730 lectual Property Rights or other rights that might be claimed to pertain 731 to the implementation or use of the technology described in this docu- 732 ment or the extent to which any license under such rights might or might 733 not be available; nor does it represent that it has made any independent 734 effort to identify any such rights. Information on the procedures with 735 respect to rights in RFC documents can be found in BCP 78 and BCP 79. 737 Copies of IPR disclosures made to the IETF Secretariat and any 738 assurances of licenses to be made available, or the result of an attempt 739 made to obtain a general license or permission for the use of such 740 proprietary rights by implementers or users of this specification can be 741 obtained from the IETF on-line IPR repository at 742 http://www.ietf.org/ipr. 744 The IETF invites any interested party to bring to its attention any 745 copyrights, patents or patent applications, or other proprietary rights 746 that may cover technology that may be required to implement this stan- 747 dard. Please address the information to the IETF at ietf-ipr@ietf.org. 749 Disclaimer of Warranty 751 This document and the information contained herein are provided on an 752 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR 753 IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 754 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 755 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMA- 756 TION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 757 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 759 Copyright Statement 761 Copyright (C) The Internet Society (2005). This document is subject to 762 the rights, licenses and restrictions contained in BCP 78, and except as 763 set forth therein, the authors retain all their rights.