idnits 2.17.1 draft-ietf-bmwg-ospfconv-intraarea-06.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 14 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 15 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The abstract seems to contain references ([TERM]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2003) is 7611 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TERM' is mentioned on line 132, but not defined == Missing Reference: 'OSPF' is mentioned on line 56, but not defined == Missing Reference: 'RFC2328' is mentioned on line 219, but not defined == Outdated reference: A later version (-07) exists of draft-ietf-bmwg-ospfconv-applicability-03 ** Downref: Normative reference to an Informational draft: draft-ietf-bmwg-ospfconv-applicability (ref. 'APPLICABILITY') Summary: 8 errors (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Vishwas Manral 3 Internet Draft Netplane Systems 4 Russ White 5 Cisco Systems 6 Aman Shaikh 7 Expiration Date: December 2003 University of California 8 File Name: draft-ietf-bmwg-ospfconv-intraarea-06.txt June 2003 10 Benchmarking Basic OPSF Single Router Control Plane Convergence 11 draft-ietf-bmwg-ospfconv-intraarea-06.txt 13 1. Status of this Memo 15 This document is an Internet-Draft and is in full conformance with 16 all provisions of Section 10 of RFC2026. 18 Internet Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its Areas, and its Working Groups. Note that other 20 groups may also distribute working documents as Internet Drafts. 22 Internet Drafts are draft documents valid for a maximum of six 23 months. Internet Drafts may be updated, replaced, or obsoleted by 24 other documents at any time. It is not appropriate to use Internet 25 Drafts as reference material or to cite them other than as a "working 26 draft" or "work in progress". 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 2. Abstract 36 This draft establishes standards for measuring OSPF single router 37 control plane convergence [TERM]. Its initial emphasis is on the 38 control plane of single OSPF routers. We do not address forwarding 39 plane performance. 41 NOTE: Within this document, the word convergence relates to single 42 router control plane convergence only. 44 3. Motivation 46 There is a growing interest in routing protocol convergence testing, 47 with many people looking at various tests to determine how long it 48 takes for a network to converge after various conditions occur. The 49 major problem with this sort of testing is that the framework of the 50 tests has a major impact on the results; for instance, determining 51 when a network is converged, what parts of the router's operation are 52 considered within the testing, and other such things will have a 53 major impact on what apparent performance routing protocols provide. 55 This document attempts to provide a framework within which Open 56 Shortest Path First [OSPF] performance testing can be placed, and 57 provide some tests with which some aspects of OSPF performance can be 58 measured. The motivation of the draft is to provide a set of tests 59 that can provide the user comparable data from various vendors with 60 which to evaluate the OSPF protocol performance on the devices. 62 4. Overview & Scope 64 While this document describes a specific set of tests aimed at 65 characterizing the single router control plane convergence 66 performance of OSPF processes in routers or other boxes that 67 incorporate OSPF functionality, a key objective is to propose 68 methodology that will standardize the conducting and reporting of 69 convergence-related measurements. 71 Things which are outside the scope of this document include: 73 o The interactions of convergence and forwarding; testing is 74 restricted to events occurring within the control plane. For- 75 warding performance is the primary focus in [INTERCONNECT] 76 and it is expected to be dealt with in work that ensues from 77 [FIB-TERM]. 79 o Inter area route generation, AS-external route generation, 80 and simultaneous traffic on the control and data paths within 81 the DUT. While the tests outlined in this document measure 82 SPF time, flooding times, and other aspects of all OSPF con- 83 vergence performance, it does not provide tests for measuring 84 external or summary route generation, route translation, or 85 other OSPF interarea and external routing performance. These 86 are expected to be dealt with in a later draft. 88 Other drafts in the future may cover some of the items noted as not 89 covered in the scope of this draft. For a discussion of the 90 terminology used in this draft (in relation to the tests themselves), 91 refer to [TERM]. For a discussion of the applicability of this draft, 92 refer to [APPLICABILITY]. 94 While this draft assumes OSPFv2, which only carries routing informa- 95 tion for IPv4 destinations, nothing in this draft precludes it from 96 use with OSPFv3, which carries IPv6 destinations. 98 5. Test Conditions 100 In all tests, the following test conditions will be assumed: 102 o The link speed should be high enough so that does not become 103 a bottleneck. Link speeds of 10MBps or higher are recom- 104 mended. The link speed between routers should be specified in 105 the test report. 107 o For all point-to-point links, it is assumed that a link 108 failure results in an immediate notification to the operating 109 system, and thus to the OSPF process; this is explained 110 thoroughly in [MILLISEC]. 112 o No data traffic will be running between the routers during 113 these tests. 115 o Optional capabilities which can reduce performance, such as 116 authentication, should be noted in the test results if they 117 are enabled. 119 o Optional changes in the default timer values, such as the 120 SPF, hello, router dead, and other intervals, should be noted 121 in the test results. 123 o All places where injecting a set of LSAs is referenced, the 124 set can include varying numbers of LSAs of varying types 125 representing a varying number of reachable destinations. See 126 [TERM] for further information about issues with LSA sets and 127 network topologies. 129 Tests should be run more than once, since a single test run 130 cannot be relied on to produce statistically sound results. 131 The number of test runs and any variations between the tests 132 should be recorded in the test results (see [TERM] for more 133 information on what items should be recorded in the test 134 results). 136 6. Reference Topologies 138 Several reference topologies will be used throughout the tests 139 described in the remainder of this document. Rather than repeating 140 these topologies, we've gathered them all in one section. 142 o Reference Topology 1 (Emulated Topology) 144 ( ) 145 DUT----Generator----( emulated topology ) 146 ( ) 148 A simple back-to-back configuration. It's assumed that the 149 link between the generator and the DUT is a point-to-point 150 link, while the connections within the generator represent 151 some emulated topology. 153 o Reference Topology 2 (Generator and Collector) 155 ( ) 156 Collector-----DUT-----Generator--( emulated topology ) 157 \ / ( ) 158 \------------/ 160 All routers are connected through point-to-point links. The 161 cost of all links is assumed to be the same unless otherwise 162 noted. 164 o Reference Topology 3 (Broadcast Network) 166 DUT R1 R2 167 | | | 168 -+------+------+-----..... 170 Any number of routers could be included on the common broad- 171 cast network. 173 o Reference Topology 4 (Parallel Links) 175 /--(link 1)-----\ ( ) 176 DUT Generator--( emulated topology ) 177 \--(link 2)-----/ ( ) 179 In all cases the tests and topologies are designed to allow perfor- 180 mance measurements to be taken all on a single device, whether the 181 DUT or some other device in the network. This eliminates the need for 182 syncronized clocks within the test networks. 184 7. Basic Process Performance Tests 186 These tests will measure aspects of the OSPF implementation as a pro- 187 cess on the device under test, including: 189 o Time required to process an LSA 191 o Flooding time 193 o Shortest Path First computation 195 7.1. Time required to process an LSA 197 o Using reference topology 1 (Emulated Topology), begin with 198 all links up and a full adjacency established between the DUT 199 and the generator. 201 o Send an LSA that is already there in the DUT (a duplicate 202 LSA), note the time difference between when the LSA is sent 203 to when the ack is received. This measures the time to pro- 204 pagate the LSA and the ack, as well as processing time of the 205 duplicate LSA. This is dupLSAprocTime. 207 o Send a new LSA from the generator to the DUT, followed 208 immediately by a duplicate LSA (LSA that already resides in 209 the database of DUT, but not the same as the one just sent). 211 o The DUT will acknowledge this second LSA immediately; note 212 the time of this acknowledgement. This is newLSAprocTime. 214 The amount of time required for an OSPF implementation to 215 process the new LSA can be computed by subtracting 216 dupLSAprocTime from newLSAprocTime. 218 Note: The duplicate LSA cannot be the same as the one just 219 sent because of the MinLSInterval restriction.[RFC2328] This 220 test is taken from [BLACKBOX]. 222 7.2. Flooding Time 224 o Using reference topology 2 (Generator and Collector), 225 enable OSPF on all links and allow the devices to build 226 full adjacencies. Configure the collector so it will block 227 all flooding towards the DUT, although it continues 228 receiving advertisements from the DUT. 230 o Inject a new set of LSAs from the generator towards the 231 collector and the DUT. 233 o On the collector, note the time the flooding is complete 234 across the link to the generator. Also note the time the 235 flooding is complete across the link from the DUT. 237 Two measurements can be taken from this test: 239 o The time between the last LSA is received on the collec- 240 tor from the generator and the time the last LSA is 241 received on the collector from the DUT. 243 o The time between the last LSA is received on the collector 244 from the generator and the time the first LSA is received 245 on the collector from the DUT. 247 Depending on the number of LSAs flooded, the sizes of the 248 LSAs, and the rate of flooding, these numbers could vary by 249 some amount. The settings and variances of these numbers 250 should be reported with the test results. 252 This time is important in link state protocols, since the 253 loop free nature of the network is reliant on the speed at 254 which revised topology information is flooded. 256 7.3. Shortest Path First Computation Time 258 o Use reference topology 1 (Emulated Toplogy), beginning 259 with the DUT and the generator fully adjacent. 261 o The default SPF timer on the DUT should be set to 0, so 262 that any new LSA that arrives, immediately results in the 263 SPF calculation [BLACKBOX]. 265 o The generator should inject a set of LSAs towards the DUT; 266 the DUT should be allowed to converge and install all best 267 paths in the local routing table, etc.. 269 o Send an LSA that is already there in the DUT (a duplicate 270 LSA), note the time difference between when the LSA is 271 sent to when the ack is received. This measures the time 272 to propagate the LSA and the ack, as well as processing 273 time of the duplicate LSA. This is dupLSAprocTime. 275 o Change the link cost between the generator and the emu- 276 lated network it is advertising. 278 o Immediately inject another LSA which is a duplicate of 279 some other LSA the generator has previously injected 280 (preferrably a stub network someplace within the emulated 281 network). 283 o Measure the time between transmitting the second (dupli- 284 cate) LSA and the acknowledgement for that LSA; this is 285 the totalSPFtime. The total time required to run SPF can 286 be computed by subtracting dupLSAprocTime from totalSPF- 287 time. 289 The accuracy of this test is crucially dependant on the 290 amount of time between the transmission of the first and 291 second LSAs. If there is too much time between them, the test 292 is meaningless because the SPF run will complete before the 293 second (duplicate) LSA is received. If there is too little 294 time between the LSAs being generated, then they will both be 295 handled before the SPF run is scheduled and started, and thus 296 the measurement would only be for the handling of the dupli- 297 cate LSA. 299 This test is also specified in [BLACKBOX]. 301 Note: This test may not be accurate on systems which 302 implement OSPF as a multithreaded process, where the 303 flooding takes place in a separate process (or on a dif- 304 ferent processor) than shortest path first computations. 306 It is also possible to measure the SPF time using white box 307 tests (using output supplied by the OSPF software impelem- 308 tor). For instance: 310 o Using reference topology 1 (Emulated Topology), establish 311 a full adjacency between the generator and the DUT. 313 o Inject a set of LSAs from the generator towards the DUT. 314 Allow the DUT to stabilize and install all best paths in 315 the routing table, etc. 317 o Change the link cost between the DUT and the generator (or 318 the link between the generator and the emulated network it 319 is advertising), such that a full SPF is required to run, 320 although only one piece of information is changed. 322 o Measure the amount of time required for the DUT to compute 323 new shortest path tree as a result of the topology changes 324 injected by the generator. These measurements should be 325 taken using available show and debug information on the 326 DUT. 328 Several caveats must be mentioned when using a white box 329 method of measuring SPF time; for instance, such white box 330 tests are only applicable when testing various versions or 331 variations within a single implementation of the OSPF proto- 332 col. Futher, the same set of commands must be used in each 333 iteration of such a test, to ensure consistent results. 335 There is some interesting relationship between the SPF times 336 reported by white box (internal) testing, and black box 337 (external) testing; these two types of tests may be used as a 338 "sanity check" on the other type of tests, by comparing the 339 results of the two tests. 341 See [APPLICABILITY] for further discussion. 343 8. Basic Intra-Area OSPF tests 345 These tests measure the performance of an OSPF implementation for 346 basic intra-area tasks, including: 348 o Forming Adjacencies on Point-to-Point Link (Initialization) 350 o Forming Adjacencies on Point-to-Point Links 352 o Link Up with Information Already in the Database 354 o Initial convergence Time on a Designated Router Electing (Broad- 355 cast) Network 357 o Link Down with Layer 2 Detection 359 o Link Down with Layer 3 Detection 361 o Designated Router Election Time on A Broadcast Network 363 8.1. Forming Adjacencies on Point-to-Point Link (Initialization) 365 This test measures the time required to form an OSPF adjacency from 366 the time a layer two (data link) connection is formed between two 367 devices running OSPF. 369 o Use reference topology 1 (Emulated Topology), beginning 370 with the link between the generator and DUT disabled on 371 the DUT. OSPF should be configured and operating on both 372 devices. 374 o Inject a set of LSAs from the generator towards the DUT. 376 o Bring the link up at the DUT, noting the time that the 377 link carrier is established on the generator. 379 o Note the time the acknowledgement for the last LSA 380 transmitted from the DUT is received on the generator. 382 The time between the carrier establishment and the ack- 383 nowledgement for the last LSA transmitted by the generator 384 should be taken as the total amount of time required for the 385 OSPF process on the DUT to react to a link up event with the 386 set of LSAs injected, including the time required for the 387 operating system to notify the OSPF process about the link 388 up, etc.. The acknowledgement for the last LSA transmitted is 389 used instead of the last acknowledgement received in order to 390 prevent timing skews due to retransmitted acknowledgements or 391 LSAs. 393 8.2. Forming Adjacencies on Point-to-Point Links 395 This test measures the time required to form an adjacency from the 396 time the first communication occurs between two devices running OSPF. 398 o Using reference topology 1 (Emulated Topology), configure 399 the DUT and the generator so traffic can be passed along 400 the link between them. 402 o Configure the generator so OSPF is running on the point- 403 to-point link towards the DUT, and inject a set of LSAs. 405 o Configure the DUT so OSPF is initialized, but not running 406 on the point-to-point link between the DUT and the genera- 407 tor. 409 o Enable OSPF on the interface between the DUT and the gen- 410 erator on the DUT. 412 o Note the time of the first hello received from the DUT on 413 the generator. 415 o Note the time of the acknowledgement from the DUT for the 416 last LSA transmitted on the generator. 418 The time between the first hello received and the ack- 419 nowledgement for the last LSA transmitted by the generator 420 should be taken as the total amount of time required for the 421 OSPF process on the DUT to build a FULL neighbor adjacency 422 with the set of LSAs injected. The acknowledgement for the 423 last LSA transmitted is used instead of the last acknowledge- 424 ment received in order to prevent timing skews due to 425 retransmitted acknowledgements or LSAs. 427 8.3. Forming adjacencies with Information Already in the Database 429 o Using reference topology 2 (Generator and Collector), con- 430 figure all three devices to run OSPF. 432 o Configure the DUT so the link between the DUT and the gen- 433 erator is disabled . 435 o Inject a set of LSAs into the network from the generator; 436 the DUT should receive these LSAs through normal flooding 437 from the collector. 439 o Enable the link between the DUT and the generator. 441 o Note the time of the first hello received from the DUT on 442 the generator. 444 o Note the time of the acknowledgement from the DUT for the 445 last LSA transmitted on the generator. 447 The time between the hello received from the DUT by the gen- 448 erator and the acknowledgement for the last LSA transmitted 449 by the generator should be taken as the total amount of time 450 required for the OSPF process on the DUT to build a FULL 451 neighbor adjacency with the set of LSAs injected. In this 452 test, the DUT is already aware of the entire network topol- 453 ogy, so the time required should only include the processing 454 of each LSA from the generator and transmitting an 455 acknowledgement. The acknowledgement for the last LSA 456 transmitted is used instead of the last acknowledgement 457 received in order to prevent timing skews due to retransmit- 458 ted acknowledgements or LSAs. 460 8.4. Designated Router Election Time on A Broadcast Network 462 o Using reference topology 3 (Broadcast Network), configure 463 R1 to be the designated router on the link, and the DUT to 464 be the backup designated router. 466 o Enable OSPF on the common broadcast link on all the 467 routers in the test bed. 469 o Disble the broadcast link on R1. 471 o Note the time of the last hello received from R1 on R2. 473 o Note the time of the first network LSA generated by the 474 DUT as received on R2. 476 The time between the last hello received on R2 and the first 477 network LSA generated by the DUT should be taken as the 478 amount of time required for the DUT to complete a designated 479 router election computation. Note this test includes the dead 480 interval timer at the DUT, so this time can be factored out, 481 or the hello and dead intervals reduced to make these timers 482 impact the overall test times less. All changed timers, the 483 number of routers connected to the link, and other variable 484 factors should be noted in the test results. 486 8.5. Initial convergence Time on a Designated Router Electing (Broad- 487 cast) Network 489 o Using reference topology 3 (Broadcast Network), begin with 490 the DUT connected to the network with OSPF enabled. OSPF 491 should be enabled on R1, but the broadcast link should be 492 disabled. 494 o Enable the broadcast link between R1 and the DUT. Note the 495 time of the first hello received by R1. 497 o Note the time the first network LSA is flooded by the DUT 498 at R1. 500 o The differential between the first hello and the first 501 network LSA is the time required by the DUT to converge on 502 this new topology. 504 This test assumes that the DUT will be the designated router 505 on the broadcast link. A similar test could be designed to 506 test the convergence time when the DUT is not the designated 507 router as well. 509 This test may be performed with varying numbers of devices 510 attached to the broadcast network, and varying sets of LSAs 511 being advertised to the DUT from the routers attached to the 512 broadcast network. Variations in the LSA sets and other fac- 513 tors should be noted in the test results. 515 The time required to elect a designated router, as measured 516 in Designated Router Election Time on A Broadcast Network, 517 above, may be subtracted from the results of this test to 518 provide just the convergence time across a broadcast network. 520 8.6. Link Down with Layer 2 Detection 522 o Using reference topology 4 (Parallel Links), begin with 523 OSPF in the full state between the generator and the DUT. 524 Both links should be point-to-point links with the ability 525 to notify the operating system immediately upon link 526 failure. 528 o Disable link 1; this should be done in such a way that the 529 keepalive timers at the data link layer will have no 530 impact on the DUT recognizing the link failure (the 531 operating system in the DUT should recognize this link 532 failure immediately). Disconnecting the cable on the gen- 533 erator end would be one possibility, or shutting the link 534 down. 536 o Note the time of the link failure on the generator. 538 o At the generator, note the time of the receipt of the new 539 router LSA from the DUT notifying the generator of the 540 link 2 failure. 542 The difference in the time between the initial link failure 543 and the receipt of the LSA on the generator across link 2 544 should be taken as the time required for an OSPF implementa- 545 tion to recognize and process a link failure. 547 8.7. Link Down with Layer 3 Detection 549 o Using reference topology 4 (Parallel Links), begin with 550 OSPF in the full state between the generator and the DUT. 552 o Disable OSPF processing on link 1 from the generator. This 553 should be done in such a way so it does not affect link 554 status; the DUT must note the failure of the adjacency 555 through the dead interval. 557 o At the generator, note the time of the receipt of the new 558 router LSA from the DUT notifying the generator of the 559 link 2 failure. 561 The difference in the time between the initial link failure 562 and the receipt of the LSA on the generator across link 2 563 should be taken as the time required for an OSPF implementa- 564 tion to recognize and process an adjacency failure. 566 9. Security Considerations 568 This draft adds no new security considerations nor does it resolve 569 any security considerations from the protocols tested. 571 10. Acknowledgements 573 Thanks to Howard Berkowitz, (hcb@clark.net), for his encouragement 574 and support. Thanks also to Gurpreet Singh 575 (Gurpreet.Singh@SpirentCom.COM) and Yasuhiro Ohara 576 (yasu@sfc.wide.ad.jp) for their comments as well. 578 11. Normative References 580 [OPSF]Moy, J., "OSPF Version 2", RFC 2328, April 1998. 582 [TERM]Manral, V., "OSPF Convergence Testing Terminiology and Concepts", 583 draft-ietf-bmwg-ospfconv-term-04, March 2003 585 [APPLICABILITY] 586 Manral, V., "Benchmarking Applicability for Basic OSPF Conver- 587 gence", draft-ietf-bmwg-ospfconv-applicability-03, March 2003 589 12. Informative References 591 [INTERCONNECT] 592 Bradner, S., McQuaid, J., "Benchmarking Methodology for Network 593 Interconnect Devices", RFC2544, March 1999. 595 [MILLISEC] 596 Alaettinoglu C., et al., "Towards Milli-Second IGP Convergence" 597 draft-alaettinoglu-isis-convergence 599 [FIB-TERM] 600 Trotter, G., "Terminology for Forwarding Information Base (FIB) 601 based Router Performance", RFC3222, October 2001. 603 [BLACKBOX] 604 Shaikh, Aman, Greenberg, Albert, "Experience in Black-Box OSPF 605 measurement" 607 13. Authors' Addresses 609 Vishwas Manral 610 Netplane Systems 611 189 Prashasan Nagar 612 Road number 72 613 Jubilee Hills 614 Hyderabad, India 616 vmanral@netplane.com 618 Russ White 619 Cisco Systems, Inc. 620 7025 Kit Creek Rd. 621 Research Triangle Park, NC 27709 623 riw@cisco.com 625 Aman Shaikh 626 University of California 627 School of Engineering 628 1156 High Street 629 Santa Cruz, CA 95064 631 aman@soe.ucsc.edu