idnits 2.17.1 draft-ietf-bmwg-ospfconv-intraarea-01.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 13 longer pages, the longest (page 2) being 60 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 14 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The abstract seems to contain references ([4]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 2002) is 8017 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '4' on line 544 looks like a reference -- Missing reference section? '1' on line 536 looks like a reference -- Missing reference section? '2' on line 539 looks like a reference -- Missing reference section? '3' on line 541 looks like a reference -- Missing reference section? '6' on line 549 looks like a reference -- Missing reference section? '5' on line 547 looks like a reference -- Missing reference section? 'RFC2328' on line 209 looks like a reference -- Missing reference section? '7' on line 552 looks like a reference -- Missing reference section? '8' on line 555 looks like a reference Summary: 8 errors (**), 0 flaws (~~), 4 warnings (==), 11 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Vishwas Manral 3 Internet Draft Netplane Systems 4 Russ White 5 Cisco Systems 6 Aman Shaikh 7 Expiration Date: October 2002 University of California 8 File Name: draft-ietf-bmwg-ospfconv-intraarea-01.txt May 2002 10 Benchmarking Methodology for Basic OSPF Convergence 11 draft-ietf-bmwg-ospfconv-intraarea-01.txt 13 1. Status of this Memo 15 This document is an Internet-Draft and is in full conformance with 16 all provisions of Section 10 of RFC2026. 18 Internet Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its Areas, and its Working Groups. Note that other 20 groups may also distribute working documents as Internet Drafts. 22 Internet Drafts are draft documents valid for a maximum of six 23 months. Internet Drafts may be updated, replaced, or obsoleted by 24 other documents at any time. It is not appropriate to use Internet 25 Drafts as reference material or to cite them other than as a "working 26 draft" or "work in progress". 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 2. Abstract 36 This draft establishes standards for measuring OSPF convergence 37 performance on a single router (SR-Convergence [4]). Its initial 38 emphasis is on the control plane of single OSPF routers. We do not 39 address forwarding plane performance. 41 3. Conventions used in this document 43 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 44 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 45 document are to be interpreted as described in [1]. 47 4. Motivation 49 There is a growing interest in routing protocol SR-Convergence 50 testing, with many people looking at various tests to determine how 51 long it takes for a network to converge after various conditions 52 occur. The major problem with this sort of testing is that the 53 framework of the tests has a major impact on the results; for 54 instance, determining when a network is converged, what parts of the 55 router's operation are considered within the testing, and other such 56 things will have a major impact on what apparent performance routing 57 protocols provide. 59 This document attempts to provide a framework within which Open 60 Shortest Path First (OSPF) [2] performance testing can be placed, and 61 provide some tests with which some aspects of OSPF performance can be 62 measured. The motivation of the draft is to provide a set of tests 63 that can provide the user comparable data from various vendors with 64 which to evaluate the OSPF protocol performance on the devices. 66 5. Overview & Scope 68 While this document describes a specific set of tests aimed at 69 characterizing the SR-Convergence performance of OSPF processes in 70 routers or other boxes that incorporate OSPF functionality, a key 71 objective is to propose methodology that will standardize the 72 conducting and reporting of convergence-related measurements. 74 Things which are outside the scope of this document include: 76 o The interactions of SR-Convergence and forwarding; testing is 77 restricted to events occurring within the control plane. For- 78 warding performance is the primary focus in [3] and it is 79 expected to be dealt with in work that ensues from [6]. 81 o Inter area routing, AS-external routing, and simultaneous 82 traffic on the control and data paths within the DUT. 84 Other drafts in the future may cover some of the items noted as not 85 covered in the scope of this draft. For a discussion of the 86 terminology used in this draft (in relation to the tests themselves), 87 refer to [4]. 89 6. Test Conditions 91 In all tests, the following test conditions will be assumed: 93 o The link speed should be high enough so that does not become 94 a bottleneck. Link speeds of 10MBps or higher are recom- 95 mended. The link speed between routers should be specified in 96 the test report. 98 o For all point-to-point links, it is assumed that a link 99 failure results in an immediate notification to the operating 100 system, and thus to the OSPF process; this is explained 101 thoroughly in [5]. 103 o No data traffic will be running between the routers during 104 these tests. 106 o Optional capabilities which can reduce performance, such as 107 authentication, should be noted in the test results if they 108 are enabled. 110 o Optional changes in the default timer values, such as the 111 SPF, hello, router dead, and other intervals, should be noted 112 in the test results. 114 o All places where injecting a set of LSAs is referenced, the 115 set can include varying numbers of LSAs of varying types 116 representing a varying number of reachable destinations. See 117 [4] for further information about issues with LSA sets and 118 network topologies. 120 Tests should be run more than once, since a single test run 121 cannot be relied on to produce statistically sound results. 122 The number of test runs and any variations between the tests 123 should be recorded in the test results (see [4] for more 124 information on what items should be recorded in the test 125 results). 127 7. Reference Topologies 129 Several reference topologies will be used throughout the tests 130 described in the remainder of this document. Rather than repeating 131 these topologies, we've gathered them all in one section. 133 o Reference Topology 1 135 ( ) 136 DUT----Generator----( emulated topology ) 137 ( ) 139 A simple back-to-back configuration. It's assumed that the 140 link between the generator and the DUT is a point-to-point 141 link, while the connections within the generator represent 142 some emulated topology. 144 o Reference Topology 2 146 ( ) 147 Collector-----DUT-----Generator--( emulated topology ) 148 \ / ( ) 149 \------------/ 151 All routers are connected through point-to-point links. The 152 cost of all links is assumed to be the same unless otherwise 153 noted. 155 o Reference Topology 3 157 DUT R1 R2 158 | | | 159 -+------+------+-----..... 161 Any number of routers could be included on the common broad- 162 cast network. 164 o Reference Topology 4 166 /--(link 1)-----\ ( ) 167 DUT Generator--( emulated topology ) 168 \--(link 2)-----/ ( ) 170 In all cases the tests and topologies are designed to allow perfor- 171 mance measurements to be taken all on a single device, whether the 172 DUT or some other device in the network. This eliminates the need for 173 syncronized clocks within the test networks. 175 8. Basic Process Performance Tests 177 These tests will measure aspects of the OSPF implementation as a pro- 178 cess on the device under test, including: 180 o Time required to process an LSA 182 o Flooding time 184 o Shortest Path First computation 186 8.1. Time required to process an LSA 188 o Using reference topology 1, begin with all links up and a 189 full adjacency established between the DUT and the generator. 191 o Send an LSA that is already there in the DUT (a duplicate 192 LSA), note the time difference between when the LSA is sent 193 to when the ack is received. This measures the time to pro- 194 pagate the LSA and the ack, as well as processing time of the 195 duplicate LSA. This is dupLSAprocTime. 197 o Send a new LSA from the generator to the DUT, followed 198 immediately by a duplicate LSA (LSA that already resides in 199 the database of DUT, but not the same as the one just sent). 201 o The DUT will acknowledge this second LSA immediately; note 202 the time of this acknowledgement. This is newLSAprocTime. 204 The amount of time required for an OSPF implementation to 205 process the new LSA can be computed by subtracting 206 dupLSAprocTime from newLSAprocTime. 208 Note: The duplicate LSA cannot be the same as the one just 209 sent because of the MinLSInterval restriction.[RFC2328] This 210 test is taken from [7]. 212 8.2. Flooding Time 214 o Using reference topology 2, enable OSPF on all links and 215 allow the devices to build full adjacencies. Configure the 216 collector so it will block all flooding towards the DUT, 217 although it continues receiving advertisements from the 218 DUT. 220 o Inject a new set of LSAs from the generator towards the 221 collector and the DUT. 223 o On the collector, note the time the flooding is complete 224 across the link to the generator. Also note the time the 225 flooding is complete across the link from the DUT. 227 Two measurements can be taken from this test: 229 o The time between the last LSA is received on the collec- 230 tor from the generator and the time the last LSA is 231 received on the collector from the DUT. 233 o The time between the last LSA is received on the collector 234 from the generator and the time the first LSA is received 235 on the collector from the DUT. 237 Depending on the number of LSAs flooded, the sizes of the 238 LSAs, and the rate of flooding, these numbers could vary by 239 some amount. The settings and variances of these numbers 240 should be reported with the test results. 242 This time is important in link state protocols, since the 243 loop free nature of the network is reliant on the speed at 244 which revised topology information is flooded. 246 8.3. Shortest Path First Computation Time (Internal Test) 248 o Using reference topology 1, establish a full adjacency 249 between the generator and the DUT. 251 o Inject a set of LSAs from the generator towards the DUT. 252 Allow the DUT to stabilize and install all best paths in 253 the routing table, etc. 255 o Change the link cost between the DUT and the generator (or 256 the link between the generator and the emulated network it 257 is advertising), such that a full SPF is required to run, 258 although only one piece of information is changed. 260 o Measure the amount of time required for the DUT to compute 261 new shortest path tree as a result of the topology changes 262 injected by the generator. These measurements should be 263 taken using available show and debug information on the 264 DUT. 266 The test results should note what available commands were 267 used to determine the length of the shortest path first run, 268 and should note any performance implications of monitoring 269 the SPF calculations using information available from the DUT 270 itself. 272 8.4. Shortest Path First Computation Time (External Test) 274 o Use reference topology 1, beginning with the DUT and the 275 generator fully adjacent. 277 o The default SPF timer on the DUT should be set to 0, so 278 that any new LSA that arrives, immediately results in the 279 SPF calculation [7]. 281 o The generator should inject a set of LSAs towards the DUT; 282 the DUT should be allowed to converge and install all best 283 paths in the local routing table, etc.. 285 o Send an LSA that is already there in the DUT (a duplicate 286 LSA), note the time difference between when the LSA is 287 sent to when the ack is received. This measures the time 288 to propagate the LSA and the ack, as well as processing 289 time of the duplicate LSA. This is dupLSAprocTime. 291 o Change the link cost between the generator and the emu- 292 lated network it is advertising. 294 o Immediately inject another LSA which is a duplicate of 295 some other LSA the generator has previously injected 296 (preferrably a stub network someplace within the emulated 297 network). 299 o Measure the time between transmitting the second (dupli- 300 cate) LSA and the acknowledgement for that LSA; this is 301 the totalSPFtime. The total time required to run SPF can 302 be computed by subtracting dupLSAprocTime from totalSPF- 303 time. 305 The accuracy of this test is crucially dependant on the 306 amount of time between the transmission of the first and 307 second LSAs. If there is too much time between them, the test 308 is meaningless because the SPF run will complete before the 309 second (duplicate) LSA is received. If there is too little 310 time between the LSAs being generated, then they will both be 311 handled before the SPF run is scheduled and started, and thus 312 the measurement would only be for the handling of the dupli- 313 cate LSA. 315 This test is also specified in [7]. 317 Note: This test may not be accurate on systems which imple- 318 ment OSPF as a multithreaded process, where the flooding 319 takes place in a separate process (or on a different proces- 320 sor) than shortest path first computations. 322 9. Basic Intra-Area OSPF tests 324 These tests measure the performance of an OSPF implementation for 325 basic intra-area tasks, including: 327 o Forming Adjacencies on Point-to-Point Link Initialization 329 o Forming Adjacencies on Point-to-Point Links 331 o Link Up with Information Already in the Database 333 o Initial SR-Convergence Time on a DR electing (broadcast) network 335 o Point-to-point Link Down with Layer 2 Detection 337 o Broadcast Media Link Down with Layer 3 Detection 339 o Designated Router Election Time on A Broadcast Network 341 9.1. Forming Adjacencies on Point-to-Point Link Initialization 343 o Use reference topology 1, beginning with the link between 344 the generator and DUT disabled on the DUT. OSPF should be 345 configured and operating on both devices. 347 o Inject a set of LSAs from the generator towards the DUT. 349 o Bring the link up at the DUT, noting the time that the 350 link carrier is established on the generator. 352 o Note the time the acknowledgement for the last LSA 353 transmitted from the DUT is received on the generator. 355 The time between the carrier establishment and the ack- 356 nowledgement for the last LSA transmitted by the generator 357 should be taken as the total amount of time required for the 358 OSPF process on the DUT to react to a link up event with the 359 set of LSAs injected, including the time required for the 360 operating system to notify the OSPF process about the link 361 up, etc.. The acknowledgement for the last LSA transmitted is 362 used instead of the last acknowledgement received in order to 363 prevent timing skews due to retransmitted acknowledgements or 364 LSAs. 366 9.2. Forming Adjacencies on Point-to-Point Links 368 o Using reference topology 1, configure the DUT and the gen- 369 erator so traffic can be passed along the link between 370 them. 372 o Configure the generator so OSPF is running on the point- 373 to-point link towards the DUT, and inject a set of LSAs. 375 o Configure the DUT so OSPF is initialized, but not running 376 on the point-to-point link between the DUT and the genera- 377 tor. 379 o Enable OSPF on the interface between the DUT and the gen- 380 erator on the DUT. 382 o Note the time of the first hello received from the DUT on 383 the generator. 385 o Note the time of the acknowledgement from the DUT for the 386 last LSA transmitted on the generator. 388 The time between the first hello received and the ack- 389 nowledgement for the last LSA transmitted by the generator 390 should be taken as the total amount of time required for the 391 OSPF process on the DUT to build a FULL neighbor adjacency 392 with the set of LSAs injected. The acknowledgement for the 393 last LSA transmitted is used instead of the last acknowledge- 394 ment received in order to prevent timing skews due to 395 retransmitted acknowledgements or LSAs. 397 9.3. Link Up with Information Already in the Database 399 o Using reference topology 2, configure all three devices to 400 run OSPF. 402 o Configure the DUT so the link between the DUT and the gen- 403 erator is disabled . 405 o Inject a set of LSAs into the network from the generator; 406 the DUT should receive these LSAs through normal flooding 407 from the collector. 409 o Enable the link between the DUT and the generator. 411 o Note the time of the first hello received from the DUT on 412 the generator. 414 o Note the time of the acknowledgement from the DUT for the 415 last LSA transmitted on the generator. 417 The time between the hello received from the DUT by the gen- 418 erator and the acknowledgement for the last LSA transmitted 419 by the generator should be taken as the total amount of time 420 required for the OSPF process on the DUT to build a FULL 421 neighbor adjacency with the set of LSAs injected. In this 422 test, the DUT is already aware of the entire network topol- 423 ogy, so the time required should only include the processing 424 of each LSA from the generator and transmitting an ack- 425 nowledgement. The acknowledgement for the last LSA transmit- 426 ted is used instead of the last acknowledgement received in 427 order to prevent timing skews due to retransmitted ack- 428 nowledgements or LSAs. 430 9.4. Initial SR-Convergence Time on a DR electing (broadcast) network 432 o Using reference topology 3, begin with the DUT connected 433 to the network with OSPF enabled. OSPF should be enabled 434 on R1, but the broadcast link should be disabled. 436 o Enable the broadcast link between R1 and the DUT. Note the 437 time of the first hello received by R1. 439 o Note the time the first network LSA is flooded by the DUT 440 at R1. 442 o The differential between the first hello and the first 443 network LSA is the time required by the DUT to converge on 444 this new topology. 446 This test assumes that the DUT will be the designated router 447 on the broadcast link. A similar test could be designed to 448 test the SR-Convergence time when the DUT is not the desig- 449 nated router as well. 451 This test may be performed with varying numbers of devices 452 attached to the broadcast network, and varying sets of LSAs 453 being advertised to the DUT from the routers attached to the 454 broadcast network. Variations in the LSA sets and other fac- 455 tors shoul dbe noted in the test results. 457 9.5. Point-to-point Link Down with Layer 2 Detection 459 o Using reference topology 4, begin with OSPF in the full 460 state between the generator and the DUT. Both links should 461 be point-to-point links with the ability to notify the 462 operating system immediately upon link failure. 464 o Disable link 1; this should be done in such a way that the 465 keepalive timers at the data link layer will have no 466 impact on the DUT recognizing the link failure (the 467 operating system in the DUT should recognize this link 468 failure immediately). Disconnecting the cable on the gen- 469 erator end would be one possibility, or shutting the link 470 down. 472 o Note the time of the link failure on the generator. 474 o At the generator, note the time of the receipt of the new 475 router LSA from the DUT notifying the generator of the 476 link 2 failure. 478 The difference in the time between the initial link failure 479 and the receipt of the LSA on the generator across link 2 480 should be taken as the time required for an OSPF implementa- 481 tion to recognize and process a link failure. 483 9.6. Broadcast Media Link Down with Layer 3 Detection 485 o Using reference topology 4, begin with OSPF in the full 486 state between the generator and the DUT. 488 o Disable OSPF processing on link 1 from the generator. This 489 should be done in such a way so it does not affect link 490 status; the DUT must note the failure of the adjacency 491 through the dead interval. 493 o At the generator, note the time of the receipt of the new 494 router LSA from the DUT notifying the generator of the 495 link 2 failure. 497 The difference in the time between the initial link failure 498 and the receipt of the LSA on the generator across link 2 499 should be taken as the time required for an OSPF implementa- 500 tion to recognize and process an adjacency failure. 502 9.7. Designated Router Election Time on A Broadcast Network 504 o Using reference topology 3, configure R1 to be the desig- 505 nated router on the link, and the DUT to be the backup 506 designated router. 508 o Enable OSPF on the common broadcast link on all the 509 routers in the test bed. 511 o Disble the broadcast link on R1. 513 o Note the time of the last hello received from R1 on R2. 515 o Note the time of the first network LSA generated by the 516 DUT as received on R2. 518 The time between the last hello received on R2 and the first 519 network LSA generated by the DUT should be taken as the 520 amount of time required for the DUT to complete a designated 521 router election computation. Note this test includes the dead 522 interval timer at the DUT, so this time can be factored out, 523 or the hello and dead intervals reduced to make these timers 524 impact the overall test times less. All changed timers, the 525 number of routers connected to the link, and other variable 526 factors should be noted in the test results. 528 10. Acknowledgements 530 Thanks to Howard Berkowitz, (hcb@clark.net), for his encouragement 531 and support. Thanks also to Gurpreet Singh 532 (Gurpreet.Singh@SpirentCom.COM) for his help as well. 534 11. References 536 [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement 537 Levels", RFC2119, March 1997. 539 [2] Moy, J., "OSPF Version 2", RFC 2328, April 1998. 541 [3] Bradner, S., McQuaid, J., "Benchmarking Methodology for Network 542 Interconnect Devices", RFC2544, March 1999. 544 [4] Manral, V., "OSPF Convergence Testing Terminiology and Concepts", 545 draft-bmwg-ospfconv-term-00.txt, My 2002 547 [5] draft-alaettinoglu-isis-convergence-01.txt 549 [6] Trotter, G., "Terminology for Forwarding Information Base (FIB) 550 based Router Performance", RFC3222, October 2001. 552 [7] Shaikh, Aman, Greenberg, Albert, "Experience in Black-Box OSPF 553 measurement" 555 >IP [8] draft-ietf-ospf-scalability-01.txt 557 12. Authors' Addresses 559 Vishwas Manral 560 Netplane Systems 561 189 Prashasan Nagar 562 Road number 72 563 Jubilee Hills 564 Hyderabad, India 566 vmanral@netplane.com 568 Russ White 569 Cisco Systems, Inc. 570 7025 Kit Creek Rd. 571 Research Triangle Park, NC 27709 573 riw@cisco.com 575 Aman Shaikh 576 University of California 577 School of Engineering 578 1156 High Street 579 Santa Cruz, CA 95064 581 aman@soe.ucsc.edu