idnits 2.17.1 draft-ietf-bmwg-ospfconv-intraarea-04.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 14 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 15 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The abstract seems to contain references ([TERM]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 2003) is 7706 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TERM' is mentioned on line 123, but not defined == Missing Reference: 'OSPF' is mentioned on line 54, but not defined == Missing Reference: 'RFC2328' is mentioned on line 210, but not defined == Outdated reference: A later version (-07) exists of draft-ietf-bmwg-ospfconv-applicability-02 ** Downref: Normative reference to an Informational draft: draft-ietf-bmwg-ospfconv-applicability (ref. 'APPLICABILITY') Summary: 8 errors (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Vishwas Manral 3 Internet Draft Netplane Systems 4 Russ White 5 Cisco Systems 6 Aman Shaikh 7 Expiration Date: September 2003 University of California 8 File Name: draft-ietf-bmwg-ospfconv-intraarea-04.txt March 2003 10 Benchmarking Methodology for Basic OSPF Convergence 11 draft-ietf-bmwg-ospfconv-intraarea-04.txt 13 1. Status of this Memo 15 This document is an Internet-Draft and is in full conformance with 16 all provisions of Section 10 of RFC2026. 18 Internet Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its Areas, and its Working Groups. Note that other 20 groups may also distribute working documents as Internet Drafts. 22 Internet Drafts are draft documents valid for a maximum of six 23 months. Internet Drafts may be updated, replaced, or obsoleted by 24 other documents at any time. It is not appropriate to use Internet 25 Drafts as reference material or to cite them other than as a "working 26 draft" or "work in progress". 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 2. Abstract 36 This draft establishes standards for measuring OSPF convergence 37 performance on a single router (SR-Convergence [TERM]). Its initial 38 emphasis is on the control plane of single OSPF routers. We do not 39 address forwarding plane performance. 41 3. Motivation 43 There is a growing interest in routing protocol SR-Convergence 44 testing, with many people looking at various tests to determine how 45 long it takes for a network to converge after various conditions 46 occur. The major problem with this sort of testing is that the 47 framework of the tests has a major impact on the results; for 48 instance, determining when a network is converged, what parts of the 49 router's operation are considered within the testing, and other such 50 things will have a major impact on what apparent performance routing 51 protocols provide. 53 This document attempts to provide a framework within which Open 54 Shortest Path First [OSPF] performance testing can be placed, and 55 provide some tests with which some aspects of OSPF performance can be 56 measured. The motivation of the draft is to provide a set of tests 57 that can provide the user comparable data from various vendors with 58 which to evaluate the OSPF protocol performance on the devices. 60 4. Overview & Scope 62 While this document describes a specific set of tests aimed at 63 characterizing the SR-Convergence performance of OSPF processes in 64 routers or other boxes that incorporate OSPF functionality, a key 65 objective is to propose methodology that will standardize the 66 conducting and reporting of convergence-related measurements. 68 Things which are outside the scope of this document include: 70 o The interactions of SR-Convergence and forwarding; testing is 71 restricted to events occurring within the control plane. For- 72 warding performance is the primary focus in [INTERCONNECT] 73 and it is expected to be dealt with in work that ensues from 74 [FIB-TERM]. 76 o Inter area routing, AS-external routing, and simultaneous 77 traffic on the control and data paths within the DUT. 79 Other drafts in the future may cover some of the items noted as not 80 covered in the scope of this draft. For a discussion of the terminol- 81 ogy used in this draft (in relation to the tests themselves), refer 82 to [TERM]. For a discussion of the applicability of this draft, refer 83 to [APPLICABILITY]. 85 While this draft assumes OSPFv2, which only carries routing informa- 86 tion for IPv4 destinations, nothing in this draft precludes it from 87 use with OSPFv3, which carries IPv6 destinations. 89 5. Test Conditions 91 In all tests, the following test conditions will be assumed: 93 o The link speed should be high enough so that does not become 94 a bottleneck. Link speeds of 10MBps or higher are recom- 95 mended. The link speed between routers should be specified in 96 the test report. 98 o For all point-to-point links, it is assumed that a link 99 failure results in an immediate notification to the operating 100 system, and thus to the OSPF process; this is explained 101 thoroughly in [MILLISEC]. 103 o No data traffic will be running between the routers during 104 these tests. 106 o Optional capabilities which can reduce performance, such as 107 authentication, should be noted in the test results if they 108 are enabled. 110 o Optional changes in the default timer values, such as the 111 SPF, hello, router dead, and other intervals, should be noted 112 in the test results. 114 o All places where injecting a set of LSAs is referenced, the 115 set can include varying numbers of LSAs of varying types 116 representing a varying number of reachable destinations. See 117 [TERM] for further information about issues with LSA sets and 118 network topologies. 120 Tests should be run more than once, since a single test run 121 cannot be relied on to produce statistically sound results. 122 The number of test runs and any variations between the tests 123 should be recorded in the test results (see [TERM] for more 124 information on what items should be recorded in the test 125 results). 127 6. Reference Topologies 129 Several reference topologies will be used throughout the tests 130 described in the remainder of this document. Rather than repeating 131 these topologies, we've gathered them all in one section. 133 o Reference Topology 1 (Emulated Topology) 135 ( ) 136 DUT----Generator----( emulated topology ) 137 ( ) 139 A simple back-to-back configuration. It's assumed that the 140 link between the generator and the DUT is a point-to-point 141 link, while the connections within the generator represent 142 some emulated topology. 144 o Reference Topology 2 (Generator and Collector) 146 ( ) 147 Collector-----DUT-----Generator--( emulated topology ) 148 \ / ( ) 149 \------------/ 151 All routers are connected through point-to-point links. The 152 cost of all links is assumed to be the same unless otherwise 153 noted. 155 o Reference Topology 3 (Broadcast Network) 157 DUT R1 R2 158 | | | 159 -+------+------+-----..... 161 Any number of routers could be included on the common broad- 162 cast network. 164 o Reference Topology 4 (Parallel Links) 166 /--(link 1)-----\ ( ) 167 DUT Generator--( emulated topology ) 168 \--(link 2)-----/ ( ) 170 In all cases the tests and topologies are designed to allow perfor- 171 mance measurements to be taken all on a single device, whether the 172 DUT or some other device in the network. This eliminates the need for 173 syncronized clocks within the test networks. 175 7. Basic Process Performance Tests 177 These tests will measure aspects of the OSPF implementation as a pro- 178 cess on the device under test, including: 180 o Time required to process an LSA 182 o Flooding time 184 o Shortest Path First computation 186 7.1. Time required to process an LSA 188 o Using reference topology 1 (Emulated Topology), begin with 189 all links up and a full adjacency established between the DUT 190 and the generator. 192 o Send an LSA that is already there in the DUT (a duplicate 193 LSA), note the time difference between when the LSA is sent 194 to when the ack is received. This measures the time to pro- 195 pagate the LSA and the ack, as well as processing time of the 196 duplicate LSA. This is dupLSAprocTime. 198 o Send a new LSA from the generator to the DUT, followed 199 immediately by a duplicate LSA (LSA that already resides in 200 the database of DUT, but not the same as the one just sent). 202 o The DUT will acknowledge this second LSA immediately; note 203 the time of this acknowledgement. This is newLSAprocTime. 205 The amount of time required for an OSPF implementation to 206 process the new LSA can be computed by subtracting 207 dupLSAprocTime from newLSAprocTime. 209 Note: The duplicate LSA cannot be the same as the one just 210 sent because of the MinLSInterval restriction.[RFC2328] This 211 test is taken from [BLACKBOX]. 213 7.2. Flooding Time 215 o Using reference topology 2 (Generator and Colelctor), 216 enable OSPF on all links and allow the devices to build 217 full adjacencies. Configure the collector so it will block 218 all flooding towards the DUT, although it continues 219 receiving advertisements from the DUT. 221 o Inject a new set of LSAs from the generator towards the 222 collector and the DUT. 224 o On the collector, note the time the flooding is complete 225 across the link to the generator. Also note the time the 226 flooding is complete across the link from the DUT. 228 Two measurements can be taken from this test: 230 o The time between the last LSA is received on the collec- 231 tor from the generator and the time the last LSA is 232 received on the collector from the DUT. 234 o The time between the last LSA is received on the collector 235 from the generator and the time the first LSA is received 236 on the collector from the DUT. 238 Depending on the number of LSAs flooded, the sizes of the 239 LSAs, and the rate of flooding, these numbers could vary by 240 some amount. The settings and variances of these numbers 241 should be reported with the test results. 243 This time is important in link state protocols, since the 244 loop free nature of the network is reliant on the speed at 245 which revised topology information is flooded. 247 7.3. Shortest Path First Computation Time 249 o Use reference topology 1 (Emulated Toplogy), beginning 250 with the DUT and the generator fully adjacent. 252 o The default SPF timer on the DUT should be set to 0, so 253 that any new LSA that arrives, immediately results in the 254 SPF calculation [BLACKBOX]. 256 o The generator should inject a set of LSAs towards the DUT; 257 the DUT should be allowed to converge and install all best 258 paths in the local routing table, etc.. 260 o Send an LSA that is already there in the DUT (a duplicate 261 LSA), note the time difference between when the LSA is 262 sent to when the ack is received. This measures the time 263 to propagate the LSA and the ack, as well as processing 264 time of the duplicate LSA. This is dupLSAprocTime. 266 o Change the link cost between the generator and the emu- 267 lated network it is advertising. 269 o Immediately inject another LSA which is a duplicate of 270 some other LSA the generator has previously injected 271 (preferrably a stub network someplace within the emulated 272 network). 274 o Measure the time between transmitting the second (dupli- 275 cate) LSA and the acknowledgement for that LSA; this is 276 the totalSPFtime. The total time required to run SPF can 277 be computed by subtracting dupLSAprocTime from totalSPF- 278 time. 280 The accuracy of this test is crucially dependant on the 281 amount of time between the transmission of the first and 282 second LSAs. If there is too much time between them, the test 283 is meaningless because the SPF run will complete before the 284 second (duplicate) LSA is received. If there is too little 285 time between the LSAs being generated, then they will both be 286 handled before the SPF run is scheduled and started, and thus 287 the measurement would only be for the handling of the dupli- 288 cate LSA. 290 This test is also specified in [BLACKBOX]. 292 Note: This test may not be accurate on systems which 293 implement OSPF as a multithreaded process, where the 294 flooding takes place in a separate process (or on a dif- 295 ferent processor) than shortest path first computations. 297 It is also possible to measure the SPF time using white box 298 tests (using output supplied by the OSPF software impelem- 299 tor). For instance: 301 o Using reference topology 1 (Emulated Topology), establish 302 a full adjacency between the generator and the DUT. 304 o Inject a set of LSAs from the generator towards the DUT. 305 Allow the DUT to stabilize and install all best paths in 306 the routing table, etc. 308 o Change the link cost between the DUT and the generator (or 309 the link between the generator and the emulated network it 310 is advertising), such that a full SPF is required to run, 311 although only one piece of information is changed. 313 o Measure the amount of time required for the DUT to compute 314 new shortest path tree as a result of the topology changes 315 injected by the generator. These measurements should be 316 taken using available show and debug information on the 317 DUT. 319 Several caveats must be mentioned when using a white box 320 method of measuring SPF time; for instance, such white box 321 tests are only applicable when testing various versions or 322 variations within a single implementation of the OSPF proto- 323 col. Futher, the same set of commands must be used in each 324 iteration of such a test, to ensure consistent results. 326 There is some interesting relationship between the SPF times 327 reported by internal, white box, testing, and external, black 328 box testing; these two types of tests may be used as a "san- 329 ity check" on the other type of tests, by comparing the 330 results of the two tests. 332 See [APPLICABILITY] for further discussion. 334 8. Basic Intra-Area OSPF tests 336 These tests measure the performance of an OSPF implementation for 337 basic intra-area tasks, including: 339 o Forming Adjacencies on Point-to-Point Link (Initialization) 341 o Forming Adjacencies on Point-to-Point Links 343 o Link Up with Information Already in the Database 345 o Initial SR-Convergence Time on a Designated Router Electing 346 (Broadcast) Network 348 o Link Down with Layer 2 Detection 350 o Link Down with Layer 3 Detection 352 o Designated Router Election Time on A Broadcast Network 354 8.1. Forming Adjacencies on Point-to-Point Link (Initialization) 356 This test measures the time required to form an OSPF adjacency from 357 the time a layer two (data link) connection is formed between two 358 devices running OSPF. 360 o Use reference topology 1 (Emulated Topology), beginning 361 with the link between the generator and DUT disabled on 362 the DUT. OSPF should be configured and operating on both 363 devices. 365 o Inject a set of LSAs from the generator towards the DUT. 367 o Bring the link up at the DUT, noting the time that the 368 link carrier is established on the generator. 370 o Note the time the acknowledgement for the last LSA 371 transmitted from the DUT is received on the generator. 373 The time between the carrier establishment and the ack- 374 nowledgement for the last LSA transmitted by the generator 375 should be taken as the total amount of time required for the 376 OSPF process on the DUT to react to a link up event with the 377 set of LSAs injected, including the time required for the 378 operating system to notify the OSPF process about the link 379 up, etc.. The acknowledgement for the last LSA transmitted is 380 used instead of the last acknowledgement received in order to 381 prevent timing skews due to retransmitted acknowledgements or 382 LSAs. 384 8.2. Forming Adjacencies on Point-to-Point Links 386 This test measures the time required to form an adjacency from the 387 time the first communication occurs between two devices running OSPF. 389 o Using reference topology 1 (Emulated Topology), configure 390 the DUT and the generator so traffic can be passed along 391 the link between them. 393 o Configure the generator so OSPF is running on the point- 394 to-point link towards the DUT, and inject a set of LSAs. 396 o Configure the DUT so OSPF is initialized, but not running 397 on the point-to-point link between the DUT and the genera- 398 tor. 400 o Enable OSPF on the interface between the DUT and the gen- 401 erator on the DUT. 403 o Note the time of the first hello received from the DUT on 404 the generator. 406 o Note the time of the acknowledgement from the DUT for the 407 last LSA transmitted on the generator. 409 The time between the first hello received and the ack- 410 nowledgement for the last LSA transmitted by the generator 411 should be taken as the total amount of time required for the 412 OSPF process on the DUT to build a FULL neighbor adjacency 413 with the set of LSAs injected. The acknowledgement for the 414 last LSA transmitted is used instead of the last acknowledge- 415 ment received in order to prevent timing skews due to 416 retransmitted acknowledgements or LSAs. 418 8.3. Forming adjacencies with Information Already in the Database 420 o Using reference topology 2 (Generator and Collector), con- 421 figure all three devices to run OSPF. 423 o Configure the DUT so the link between the DUT and the gen- 424 erator is disabled . 426 o Inject a set of LSAs into the network from the generator; 427 the DUT should receive these LSAs through normal flooding 428 from the collector. 430 o Enable the link between the DUT and the generator. 432 o Note the time of the first hello received from the DUT on 433 the generator. 435 o Note the time of the acknowledgement from the DUT for the 436 last LSA transmitted on the generator. 438 The time between the hello received from the DUT by the gen- 439 erator and the acknowledgement for the last LSA transmitted 440 by the generator should be taken as the total amount of time 441 required for the OSPF process on the DUT to build a FULL 442 neighbor adjacency with the set of LSAs injected. In this 443 test, the DUT is already aware of the entire network topol- 444 ogy, so the time required should only include the processing 445 of each LSA from the generator and transmitting an 446 acknowledgement. The acknowledgement for the last LSA 447 transmitted is used instead of the last acknowledgement 448 received in order to prevent timing skews due to retransmit- 449 ted acknowledgements or LSAs. 451 8.4. Designated Router Election Time on A Broadcast Network 453 o Using reference topology 3 (Broadcast Network), configure 454 R1 to be the designated router on the link, and the DUT to 455 be the backup designated router. 457 o Enable OSPF on the common broadcast link on all the 458 routers in the test bed. 460 o Disble the broadcast link on R1. 462 o Note the time of the last hello received from R1 on R2. 464 o Note the time of the first network LSA generated by the 465 DUT as received on R2. 467 The time between the last hello received on R2 and the first 468 network LSA generated by the DUT should be taken as the 469 amount of time required for the DUT to complete a designated 470 router election computation. Note this test includes the dead 471 interval timer at the DUT, so this time can be factored out, 472 or the hello and dead intervals reduced to make these timers 473 impact the overall test times less. All changed timers, the 474 number of routers connected to the link, and other variable 475 factors should be noted in the test results. 477 8.5. Initial SR-Convergence Time on a Designated Router Electing (Broad- 478 cast) Network 480 o Using reference topology 3 (Broadcast Network), begin with 481 the DUT connected to the network with OSPF enabled. OSPF 482 should be enabled on R1, but the broadcast link should be 483 disabled. 485 o Enable the broadcast link between R1 and the DUT. Note the 486 time of the first hello received by R1. 488 o Note the time the first network LSA is flooded by the DUT 489 at R1. 491 o The differential between the first hello and the first 492 network LSA is the time required by the DUT to converge on 493 this new topology. 495 This test assumes that the DUT will be the designated router 496 on the broadcast link. A similar test could be designed to 497 test the SR-Convergence time when the DUT is not the desig- 498 nated router as well. 500 This test may be performed with varying numbers of devices 501 attached to the broadcast network, and varying sets of LSAs 502 being advertised to the DUT from the routers attached to the 503 broadcast network. Variations in the LSA sets and other fac- 504 tors should be noted in the test results. 506 The time required to elect a designated router, as measured 507 in Designated Router Election Time on A Broadcast Network, 508 above, may be subtracted from the results of this test to 509 provide just the convergence time across a broadcast network. 511 8.6. Link Down with Layer 2 Detection 513 o Using reference topology 4 (Parallel Links), begin with 514 OSPF in the full state between the generator and the DUT. 515 Both links should be point-to-point links with the ability 516 to notify the operating system immediately upon link 517 failure. 519 o Disable link 1; this should be done in such a way that the 520 keepalive timers at the data link layer will have no 521 impact on the DUT recognizing the link failure (the 522 operating system in the DUT should recognize this link 523 failure immediately). Disconnecting the cable on the gen- 524 erator end would be one possibility, or shutting the link 525 down. 527 o Note the time of the link failure on the generator. 529 o At the generator, note the time of the receipt of the new 530 router LSA from the DUT notifying the generator of the 531 link 2 failure. 533 The difference in the time between the initial link failure 534 and the receipt of the LSA on the generator across link 2 535 should be taken as the time required for an OSPF implementa- 536 tion to recognize and process a link failure. 538 8.7. Link Down with Layer 3 Detection 540 o Using reference topology 4 (Parallel Links), begin with 541 OSPF in the full state between the generator and the DUT. 543 o Disable OSPF processing on link 1 from the generator. This 544 should be done in such a way so it does not affect link 545 status; the DUT must note the failure of the adjacency 546 through the dead interval. 548 o At the generator, note the time of the receipt of the new 549 router LSA from the DUT notifying the generator of the 550 link 2 failure. 552 The difference in the time between the initial link failure 553 and the receipt of the LSA on the generator across link 2 554 should be taken as the time required for an OSPF implementa- 555 tion to recognize and process an adjacency failure. 557 9. Security Considerations 559 This draft adds no new security considerations nor does it resolve 560 any security considerations from the protocols tested. 562 10. Acknowledgements 564 Thanks to Howard Berkowitz, (hcb@clark.net), for his encouragement 565 and support. Thanks also to Gurpreet Singh 566 (Gurpreet.Singh@SpirentCom.COM) and Yasuhiro Ohara 567 (yasu@sfc.wide.ad.jp) for their comments as well. 569 11. Normative References 571 [OPSF]Moy, J., "OSPF Version 2", RFC 2328, April 1998. 573 [TERM]Manral, V., "OSPF Convergence Testing Terminiology and Concepts", 574 draft-ietf-bmwg-ospfconv-term-03, March 2003 576 [APPLICABILITY] 577 Manral, V., "Benchmarking Applicability for Basic OSPF Conver- 578 gence", draft-ietf-bmwg-ospfconv-applicability-02, March 2003 580 12. Informative References 582 [INTERCONNECT] 583 Bradner, S., McQuaid, J., "Benchmarking Methodology for Network 584 Interconnect Devices", RFC2544, March 1999. 586 [MILLISEC] 587 Alaettinoglu C., et al., "Towards Milli-Second IGP Convergence" 588 draft-alaettinoglu-isis-convergence 590 [FIB-TERM] 591 Trotter, G., "Terminology for Forwarding Information Base (FIB) 592 based Router Performance", RFC3222, October 2001. 594 [BLACKBOX] 595 Shaikh, Aman, Greenberg, Albert, "Experience in Black-Box OSPF 596 measurement" 598 13. Authors' Addresses 600 Vishwas Manral 601 Netplane Systems 602 189 Prashasan Nagar 603 Road number 72 604 Jubilee Hills 605 Hyderabad, India 607 vmanral@netplane.com 609 Russ White 610 Cisco Systems, Inc. 611 7025 Kit Creek Rd. 612 Research Triangle Park, NC 27709 614 riw@cisco.com 616 Aman Shaikh 617 University of California 618 School of Engineering 619 1156 High Street 620 Santa Cruz, CA 95064 622 aman@soe.ucsc.edu