idnits 2.17.1 draft-ietf-bmwg-ospfconv-intraarea-07.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 14 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 15 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 2003) is 7465 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'OSPF' is mentioned on line 55, but not defined == Missing Reference: 'TERM' is mentioned on line 131, but not defined == Missing Reference: 'RFC2328' is mentioned on line 218, but not defined == Outdated reference: A later version (-07) exists of draft-ietf-bmwg-ospfconv-applicability-03 ** Downref: Normative reference to an Informational draft: draft-ietf-bmwg-ospfconv-applicability (ref. 'APPLICABILITY') Summary: 7 errors (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group Vishwas Manral 2 Internet Draft Sinett Semiconductors 3 Russ White 4 Cisco Systems 5 Aman Shaikh 6 Expiration Date: May 2004 University of California 7 File Name: draft-ietf-bmwg-ospfconv-intraarea-07.txt November 2003 9 Benchmarking Basic OSPF Single Router Control Plane Convergence 10 draft-ietf-bmwg-ospfconv-intraarea-07.txt 12 1. Status of this Memo 14 This document is an Internet-Draft and is in full conformance with 15 all provisions of Section 10 of RFC2026. 17 Internet Drafts are working documents of the Internet Engineering 18 Task Force (IETF), its Areas, and its Working Groups. Note that other 19 groups may also distribute working documents as Internet Drafts. 21 Internet Drafts are draft documents valid for a maximum of six 22 months. Internet Drafts may be updated, replaced, or obsoleted by 23 other documents at any time. It is not appropriate to use Internet 24 Drafts as reference material or to cite them other than as a "working 25 draft" or "work in progress". 27 The list of current Internet-Drafts can be accessed at 28 http://www.ietf.org/ietf/1id-abstracts.txt 30 The list of Internet-Draft Shadow Directories can be accessed at 31 http://www.ietf.org/shadow.html. 33 2. Abstract 35 This draft provides suggestions for measuring OSPF single router 36 control plane convergence. Its initial emphasis is on the control 37 plane of single OSPF routers. We do not address forwarding plane 38 performance. 40 NOTE: Within this document, the word convergence relates to single 41 router control plane convergence only. 43 3. Motivation 45 There is a growing interest in routing protocol convergence testing, 46 with many people looking at various tests to determine how long it 47 takes for a network to converge after various conditions occur. The 48 major problem with this sort of testing is that the framework of the 49 tests has a major impact on the results; for instance, determining 50 when a network is converged, what parts of the router's operation are 51 considered within the testing, and other such things will have a 52 major impact on what apparent performance routing protocols provide. 54 This document attempts to provide a framework within which Open 55 Shortest Path First [OSPF] performance testing can be placed, and 56 provide some tests with which some aspects of OSPF performance can be 57 measured. The motivation of the draft is to provide a set of tests 58 that can provide the user comparable data from various vendors with 59 which to evaluate the OSPF protocol performance on the devices. 61 4. Overview & Scope 63 While this document describes a specific set of tests aimed at 64 characterizing the single router control plane convergence 65 performance of OSPF processes in routers or other boxes that 66 incorporate OSPF functionality, a key objective is to propose 67 methodologies that will prdouce directly comparable convergence 68 related measurements. 70 Things which are outside the scope of this document include: 72 o The interactions of convergence and forwarding; testing is 73 restricted to events occurring within the control plane. For- 74 warding performance is the primary focus in [INTERCONNECT] 75 and it is expected to be dealt with in work that ensues from 76 [FIB-TERM]. 78 o Inter-area route generation, AS-external route generation, 79 and simultaneous traffic on the control and data paths within 80 the DUT. While the tests outlined in this document measure 81 SPF time, flooding times, and other aspects of all OSPF con- 82 vergence performance, it does not provide tests for measuring 83 external or summary route generation, route translation, or 84 other OSPF inter-area and external routing performance. These 85 are expected to be dealt with in a later draft. 87 Other drafts in the future may cover some of the items noted as not 88 covered in the scope of this draft. For a discussion of the 89 terminology used in this draft (in relation to the tests themselves), 90 refer to [TERM]. For a discussion of the applicability of this draft, 91 refer to [APPLICABILITY]. 93 While this draft assumes OSPFv2, which only carries routing informa- 94 tion for IPv4 destinations, nothing in this draft precludes it from 95 use with OSPFv3, which carries IPv6 destinations. 97 5. Test Conditions 99 In all tests, the following test conditions will be assumed: 101 o The link speed should be high enough so that does not become 102 a bottleneck. Link speeds of 10MBps or higher are recom- 103 mended. The link speed between routers should be specified in 104 the test report. 106 o For all point-to-point links, it is assumed that a link 107 failure results in an immediate notification to the operating 108 system, and thus to the OSPF process; this is explained 109 thoroughly in [MILLISEC]. 111 o No data traffic will be running between the routers during 112 these tests. 114 o Optional capabilities which can reduce performance, such as 115 authentication, should be noted in the test results if they 116 are enabled. 118 o Optional changes in the default timer values, such as the 119 SPF, hello, router dead, and other intervals, should be noted 120 in the test results. 122 o All places where injecting a set of LSAs is referenced, the 123 set can include varying numbers of LSAs of varying types 124 representing a varying number of reachable destinations. See 125 [TERM] for further information about issues with LSA sets and 126 network topologies. 128 Tests should be run more than once, since a single test run 129 cannot be relied on to produce statistically sound results. 130 The number of test runs and any variations between the tests 131 should be recorded in the test results (see [TERM] for more 132 information on what items should be recorded in the test 133 results). 135 6. Reference Topologies 137 Several reference topologies will be used throughout the tests 138 described in the remainder of this document. Rather than repeating 139 these topologies, we've gathered them all in one section. 141 o Reference Topology 1 (Emulated Topology) 143 ( ) 144 DUT----Generator----( emulated topology ) 145 ( ) 147 A simple back-to-back configuration. It's assumed that the 148 link between the generator and the DUT is a point-to-point 149 link, while the connections within the generator represent 150 some emulated topology. 152 o Reference Topology 2 (Generator and Collector) 154 ( ) 155 Collector-----DUT-----Generator--( emulated topology ) 156 \ / ( ) 157 \------------/ 159 All routers are connected through point-to-point links. The 160 cost of all links is assumed to be the same unless otherwise 161 noted. 163 o Reference Topology 3 (Broadcast Network) 165 DUT R1 R2 166 | | | 167 -+------+------+-----..... 169 Any number of routers could be included on the common broad- 170 cast network. 172 o Reference Topology 4 (Parallel Links) 174 /--(link 1)-----\ ( ) 175 DUT Generator--( emulated topology ) 176 \--(link 2)-----/ ( ) 178 In all cases the tests and topologies are designed to allow perfor- 179 mance measurements to be taken all on a single device, whether the 180 DUT or some other device in the network. This eliminates the need for 181 syncronized clocks within the test networks. 183 7. Basic Process Performance Tests 185 These tests will measure aspects of the OSPF implementation as a pro- 186 cess on the device under test, including: 188 o Time required to process an LSA 190 o Flooding time 192 o Shortest Path First computation 194 7.1. Time required to process an LSA 196 o Using reference topology 1 (Emulated Topology), begin with 197 all links up and a full adjacency established between the DUT 198 and the generator. 200 o Send an LSA that is already there in the DUT (a duplicate 201 LSA), note the time difference between when the LSA is sent 202 to when the ack is received. This measures the time to pro- 203 pagate the LSA and the ack, as well as processing time of the 204 duplicate LSA. This is dupLSAprocTime. 206 o Send a new LSA from the generator to the DUT, followed 207 immediately by a duplicate LSA (LSA that already resides in 208 the database of DUT, but not the same as the one just sent). 210 o The DUT will acknowledge this second LSA immediately; note 211 the time of this acknowledgement. This is newLSAprocTime. 213 The amount of time required for an OSPF implementation to 214 process the new LSA can be computed by subtracting 215 dupLSAprocTime from newLSAprocTime. 217 Note: The duplicate LSA cannot be the same as the one just 218 sent because of the MinLSInterval restriction.[RFC2328] This 219 test is taken from [BLACKBOX]. 221 7.2. Flooding Time 223 o Using reference topology 2 (Generator and Collector), 224 enable OSPF on all links and allow the devices to build 225 full adjacencies. Configure the collector so it will block 226 all flooding towards the DUT, although it continues 227 receiving advertisements from the DUT. 229 o Inject a new set of LSAs from the generator towards the 230 collector and the DUT. 232 o On the collector, note the time the flooding is complete 233 across the link to the generator. Also note the time the 234 flooding is complete across the link from the DUT. 236 Two measurements can be taken from this test: 238 o The time between the last LSA is received on the collector 239 from the generator and the time the last LSA is received 240 on the collector from the DUT. 242 o The time between the last LSA is received on the collector 243 from the generator and the time the first LSA is received 244 on the collector from the DUT. 246 Depending on the number of LSAs flooded, the sizes of the 247 LSAs, the number of LSUs, and the rate of flooding, these 248 numbers could vary by some amount. The settings and variances 249 of these numbers should be reported with the test results. 251 This time is important in link state protocols, since the 252 loop free nature of the network is reliant on the speed at 253 which revised topology information is flooded. 255 7.3. Shortest Path First Computation Time 257 o Use reference topology 1 (Emulated Toplogy), beginning 258 with the DUT and the generator fully adjacent. 260 o The default SPF timer on the DUT should be set to 0, so 261 that any new LSA that arrives, immediately results in the 262 SPF calculation [BLACKBOX]. 264 o The generator should inject a set of LSAs towards the DUT; 265 the DUT should be allowed to converge and install all best 266 paths in the local routing table, etc.. 268 o Send an LSA that is already there in the DUT (a duplicate 269 LSA), note the time difference between when the LSA is 270 sent to when the ack is received. This measures the time 271 to propagate the LSA and the ack, as well as processing 272 time of the duplicate LSA. This is dupLSAprocTime. 274 o Change the link cost between the generator and the emu- 275 lated network it is advertising. 277 o Immediately inject another LSA which is a duplicate of 278 some other LSA the generator has previously injected 279 (preferrably a stub network someplace within the emulated 280 network). 282 o Measure the time between transmitting the second (dupli- 283 cate) LSA and the acknowledgement for that LSA; this is 284 the totalSPFtime. The total time required to run SPF can 285 be computed by subtracting dupLSAprocTime from totalSPF- 286 time. 288 The accuracy of this test is crucially dependant on the 289 amount of time between the transmission of the first and 290 second LSAs. If there is too much time between them, the test 291 is meaningless because the SPF run will complete before the 292 second (duplicate) LSA is received. If there is too little 293 time between the LSAs being generated, then they will both be 294 handled before the SPF run is scheduled and started, and thus 295 the measurement would only be for the handling of the dupli- 296 cate LSA. 298 This test is also specified in [BLACKBOX]. 300 Note: This test may not be accurate on systems which 301 implement OSPF as a multithreaded process, where the 302 flooding takes place in a separate process (or on a dif- 303 ferent processor) than shortest path first computations. 305 It is also possible to measure the SPF time using white box 306 tests (using output supplied by the OSPF software impelem- 307 tor). For instance: 309 o Using reference topology 1 (Emulated Topology), establish 310 a full adjacency between the generator and the DUT. 312 o Inject a set of LSAs from the generator towards the DUT. 313 Allow the DUT to stabilize and install all best paths in 314 the routing table, etc. 316 o Change the link cost between the DUT and the generator (or 317 the link between the generator and the emulated network it 318 is advertising), such that a full SPF is required to run, 319 although only one piece of information is changed. 321 o Measure the amount of time required for the DUT to compute 322 new shortest path tree as a result of the topology changes 323 injected by the generator. These measurements should be 324 taken using available show and debug information on the 325 DUT. 327 Several caveats must be mentioned when using a white box 328 method of measuring SPF time; for instance, such white box 329 tests are only applicable when testing various versions or 330 variations within a single implementation of the OSPF proto- 331 col. Futher, the same set of commands must be used in each 332 iteration of such a test, to ensure consistent results. 334 There is some interesting relationship between the SPF times 335 reported by white box (internal) testing, and black box 336 (external) testing; these two types of tests may be used as a 337 "sanity check" on the other type of tests, by comparing the 338 results of the two tests. 340 See [APPLICABILITY] for further discussion. 342 8. Basic Intra-Area OSPF tests 344 These tests measure the performance of an OSPF implementation for 345 basic intra-area tasks, including: 347 o Forming Adjacencies on Point-to-Point Link (Initialization) 349 o Forming Adjacencies on Point-to-Point Links 351 o Link Up with Information Already in the Database 353 o Initial convergence Time on a Designated Router Electing (Broad- 354 cast) Network 356 o Link Down with Layer 2 Detection 358 o Link Down with Layer 3 Detection 360 o Designated Router Election Time on A Broadcast Network 362 8.1. Forming Adjacencies on Point-to-Point Link (Initialization) 364 This test measures the time required to form an OSPF adjacency from 365 the time a layer two (data link) connection is formed between two 366 devices running OSPF. 368 o Use reference topology 1 (Emulated Topology), beginning 369 with the link between the generator and DUT disabled on 370 the DUT. OSPF should be configured and operating on both 371 devices. 373 o Inject a set of LSAs from the generator towards the DUT. 375 o Bring the link up at the DUT, noting the time that the 376 link carrier is established on the generator. 378 o Note the time the acknowledgement for the last LSA 379 transmitted from the DUT is received on the generator. 381 The time between the carrier establishment and the ack- 382 nowledgement for the last LSA transmitted by the generator 383 should be taken as the total amount of time required for the 384 OSPF process on the DUT to react to a link up event with the 385 set of LSAs injected, including the time required for the 386 operating system to notify the OSPF process about the link 387 up, etc.. The acknowledgement for the last LSA transmitted is 388 used instead of the last acknowledgement received in order to 389 prevent timing skews due to retransmitted acknowledgements or 390 LSAs. 392 8.2. Forming Adjacencies on Point-to-Point Links 394 This test measures the time required to form an adjacency from the 395 time the first communication occurs between two devices running OSPF. 397 o Using reference topology 1 (Emulated Topology), configure 398 the DUT and the generator so traffic can be passed along 399 the link between them. 401 o Configure the generator so OSPF is running on the point- 402 to-point link towards the DUT, and inject a set of LSAs. 404 o Configure the DUT so OSPF is initialized, but not running 405 on the point-to-point link between the DUT and the genera- 406 tor. 408 o Enable OSPF on the interface between the DUT and the gen- 409 erator on the DUT. 411 o Note the time of the first hello received from the DUT on 412 the generator. 414 o Note the time of the acknowledgement from the DUT for the 415 last LSA transmitted on the generator. 417 The time between the first hello received and the ack- 418 nowledgement for the last LSA transmitted by the generator 419 should be taken as the total amount of time required for the 420 OSPF process on the DUT to build a FULL neighbor adjacency 421 with the set of LSAs injected. The acknowledgement for the 422 last LSA transmitted is used instead of the last acknowledge- 423 ment received in order to prevent timing skews due to 424 retransmitted acknowledgements or LSAs. 426 8.3. Forming adjacencies with Information Already in the Database 428 o Using reference topology 2 (Generator and Collector), con- 429 figure all three devices to run OSPF. 431 o Configure the DUT so the link between the DUT and the gen- 432 erator is disabled . 434 o Inject a set of LSAs into the network from the generator; 435 the DUT should receive these LSAs through normal flooding 436 from the collector. 438 o Enable the link between the DUT and the generator. 440 o Note the time of the first hello received from the DUT on 441 the generator. 443 o Note the time of the acknowledgement from the DUT for the 444 last LSA transmitted on the generator. 446 The time between the hello received from the DUT by the gen- 447 erator and the acknowledgement for the last LSA transmitted 448 by the generator should be taken as the total amount of time 449 required for the OSPF process on the DUT to build a FULL 450 neighbor adjacency with the set of LSAs injected. In this 451 test, the DUT is already aware of the entire network topol- 452 ogy, so the time required should only include the processing 453 of each LSA from the generator and transmitting an 454 acknowledgement. The acknowledgement for the last LSA 455 transmitted is used instead of the last acknowledgement 456 received in order to prevent timing skews due to retransmit- 457 ted acknowledgements or LSAs. 459 8.4. Designated Router Election Time on A Broadcast Network 461 o Using reference topology 3 (Broadcast Network), configure 462 R1 to be the designated router on the link, and the DUT to 463 be the backup designated router. 465 o Enable OSPF on the common broadcast link on all the 466 routers in the test bed. 468 o Disble the broadcast link on R1. 470 o Note the time of the last hello received from R1 on R2. 472 o Note the time of the first network LSA generated by the 473 DUT as received on R2. 475 The time between the last hello received on R2 and the first 476 network LSA generated by the DUT should be taken as the 477 amount of time required for the DUT to complete a designated 478 router election computation. Note this test includes the dead 479 interval timer at the DUT, so this time may be factored out, 480 or the hello and dead intervals reduced to make these timers 481 impact the overall test times less. All changed timers, the 482 number of routers connected to the link, and other variable 483 factors should be noted in the test results. 485 8.5. Initial Convergence Time on a Broadcast Network, Test 1 487 o Using reference topology 3 (Broadcast Network), begin with 488 the DUT connected to the network with OSPF enabled. OSPF 489 should be enabled on R1, but the broadcast link should be 490 disabled. 492 o Enable the broadcast link between R1 and the DUT. Note the 493 time of the first hello received by R1. 495 o Note the time the first network LSA is flooded by the DUT 496 at R1. 498 o The differential between the first hello and the first 499 network LSA is the time required by the DUT to converge on 500 this new topology. 502 This test assumes that the DUT will be the designated router 503 on the broadcast link. A similar test could be designed to 504 test the convergence time when the DUT is not the designated 505 router as well. 507 This test may be performed with varying numbers of devices 508 attached to the broadcast network, and varying sets of LSAs 509 being advertised to the DUT from the routers attached to the 510 broadcast network. Variations in the LSA sets and other fac- 511 tors should be noted in the test results. 513 The time required to elect a designated router, as measured 514 in Designated Router Election Time on A Broadcast Network, 515 above, may be subtracted from the results of this test to 516 provide just the convergence time across a broadcast network. 518 8.6. Initial Convergence Time on a Broadcast Network, Test 2 520 o Using reference topology 3 (Broadcast Network), begin with 521 the DUT connected to the network with OSPF enabled. OSPF 522 should be enabled on R1, but the broadcast link should be 523 disabled. 525 o Enable the broadcast link between R1 and the DUT. Note the 526 time of the first hello transmitted by the DUT with a 527 designated router listed. 529 o Note the time the first network LSA is flooded by the DUT 530 at R1. 532 o The differential between the first hello with a designated 533 router lists and the first network LSA is the time 534 required by the DUT to converge on this new topology. 536 8.7. Link Down with Layer 2 Detection 538 o Using reference topology 4 (Parallel Links), begin with 539 OSPF in the full state between the generator and the DUT. 540 Both links should be point-to-point links with the ability 541 to notify the operating system immediately upon link 542 failure. 544 o Disable link 1; this should be done in such a way that the 545 keepalive timers at the data link layer will have no 546 impact on the DUT recognizing the link failure (the 547 operating system in the DUT should recognize this link 548 failure immediately). Disconnecting the cable on the gen- 549 erator end would be one possibility, or shutting the link 550 down. 552 o Note the time of the link failure on the generator. 554 o At the generator, note the time of the receipt of the new 555 router LSA from the DUT notifying the generator of the 556 link 2 failure. 558 The difference in the time between the initial link failure 559 and the receipt of the LSA on the generator across link 2 560 should be taken as the time required for an OSPF implementa- 561 tion to recognize and process a link failure. 563 8.8. Link Down with Layer 3 Detection 565 o Using reference topology 4 (Parallel Links), begin with 566 OSPF in the full state between the generator and the DUT. 568 o Disable OSPF processing on link 1 from the generator. This 569 should be done in such a way so it does not affect link 570 status; the DUT must note the failure of the adjacency 571 through the dead interval. 573 o At the generator, note the time of the receipt of the new 574 router LSA from the DUT notifying the generator of the 575 link 2 failure. 577 The difference in the time between the initial link failure 578 and the receipt of the LSA on the generator across link 2 579 should be taken as the time required for an OSPF implementa- 580 tion to recognize and process an adjacency failure. 582 9. Security Considerations 584 This draft adds no new security considerations nor does it resolve 585 any security considerations from the protocols tested. 587 10. Acknowledgements 589 Thanks to Howard Berkowitz, (hcb@clark.net), for his encouragement 590 and support. Thanks also to Gurpreet Singh 591 (Gurpreet.Singh@SpirentCom.COM) and Yasuhiro Ohara 592 (yasu@sfc.wide.ad.jp) for their comments as well. 594 11. Normative References 596 [OSPF]Moy, J., "OSPF Version 2", RFC 2328, April 1998. 598 [TERM]Manral, V., "OSPF Convergence Testing Terminiology and Concepts", 599 draft-ietf-bmwg-ospfconv-term-04, March 2003 601 [APPLICABILITY] 602 Manral, V., "Benchmarking Applicability for Basic OSPF Conver- 603 gence", draft-ietf-bmwg-ospfconv-applicability-03, March 2003 605 12. Informative References 607 [INTERCONNECT] 608 Bradner, S., McQuaid, J., "Benchmarking Methodology for Network 609 Interconnect Devices", RFC2544, March 1999. 611 [MILLISEC] 612 Alaettinoglu C., et al., "Towards Milli-Second IGP Convergence" 613 draft-alaettinoglu-isis-convergence 615 [FIB-TERM] 616 Trotter, G., "Terminology for Forwarding Information Base (FIB) 617 based Router Performance", RFC3222, October 2001. 619 [BLACKBOX] 620 Shaikh, Aman, Greenberg, Albert, "Experience in Black-Box OSPF 621 measurement" 623 13. Authors' Addresses 625 Vishwas Manral 626 Sinett Semiconductors 627 1st floor, Embassy Icon Annex, 2/1 Infantry Road, 628 Bangalore, India 630 email: vishwas@sinett.com 632 Russ White 633 Cisco Systems, Inc. 634 7025 Kit Creek Rd. 635 Research Triangle Park, NC 27709 637 riw@cisco.com 639 Aman Shaikh 640 University of California 641 School of Engineering 642 1156 High Street 643 Santa Cruz, CA 95064 645 aman@soe.ucsc.edu