idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-meth-18.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 761 has weird spacing: '... Tester secon...' == Line 770 has weird spacing: '... Pacing seco...' -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 13, 2009) is 5401 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-term-18 Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: January 14, 2010 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 July 13, 2009 10 Benchmarking Methodology for Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-meth-18 13 Status of this Memo 15 This Internet-Draft is submitted to IETF in full conformance with the 16 provisions of BCP 78 and BCP 79. This document may contain material 17 from IETF Documents or IETF Contributions published or made publicly 18 available before November 10, 2008. The person(s) controlling the 19 copyright in some of this material may not have granted the IETF 20 Trust the right to allow modifications of such material outside the 21 IETF Standards Process. Without obtaining an adequate license from 22 the person(s) controlling the copyright in such materials, this 23 document may not be modified outside the IETF Standards Process, and 24 derivative works of it may not be created outside the IETF Standards 25 Process, except to format it for publication as an RFC or to 26 translate it into languages other than English. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as Internet- 31 Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/ietf/1id-abstracts.txt. 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html. 44 This Internet-Draft will expire on January 14, 2010. 46 Copyright Notice 48 Copyright (c) 2009 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents in effect on the date of 53 publication of this document (http://trustee.ietf.org/license-info). 54 Please review these documents carefully, as they describe your rights 55 and restrictions with respect to this document. 57 Abstract 59 This document describes the methodology for benchmarking Link-State 60 Interior Gateway Protocol (IGP) Route Convergence. The methodology 61 is to be used for benchmarking IGP convergence time through 62 externally observable (black box) data plane measurements. The 63 methodology can be applied to any link-state IGP, such as ISIS and 64 OSPF. 66 Table of Contents 68 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 5 69 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 5 70 3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 5 71 3.1. Test topology for local changes . . . . . . . . . . . . . 5 72 3.2. Test topology for remote changes . . . . . . . . . . . . . 6 73 3.3. Test topology for local ECMP changes . . . . . . . . . . . 7 74 3.4. Test topology for remote ECMP changes . . . . . . . . . . 7 75 3.5. Test topology for Parallel Link changes . . . . . . . . . 8 76 4. Convergence Time and Loss of Connectivity Period . . . . . . . 9 77 5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 13 78 5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 13 79 5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 13 80 5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 13 81 5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 14 82 5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 14 83 5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 14 84 5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 15 85 5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 15 86 5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 15 87 6. Selection of Convergence Time Benchmark Metrics and Methods . 16 88 6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 16 89 6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 16 90 6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 16 91 6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 16 92 6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 17 93 6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 17 94 6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 17 95 6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 17 96 6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 17 97 6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 17 98 6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 18 99 6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 18 100 7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 18 101 8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 20 102 8.1. Interface failures . . . . . . . . . . . . . . . . . . . . 21 103 8.1.1. Convergence Due to Local Interface Failure . . . . . . 21 104 8.1.2. Convergence Due to Remote Interface Failure . . . . . 22 105 8.1.3. Convergence Due to ECMP Member Local Interface 106 Failure . . . . . . . . . . . . . . . . . . . . . . . 24 107 8.1.4. Convergence Due to ECMP Member Remote Interface 108 Failure . . . . . . . . . . . . . . . . . . . . . . . 25 109 8.1.5. Convergence Due to Parallel Link Interface Failure . . 26 110 8.2. Other failures . . . . . . . . . . . . . . . . . . . . . . 27 111 8.2.1. Convergence Due to Layer 2 Session Loss . . . . . . . 27 112 8.2.2. Convergence Due to Loss of IGP Adjacency . . . . . . . 28 113 8.2.3. Convergence Due to Route Withdrawal . . . . . . . . . 30 115 8.3. Administrative changes . . . . . . . . . . . . . . . . . . 31 116 8.3.1. Convergence Due to Local Adminstrative Shutdown . . . 31 117 8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 32 118 9. Security Considerations . . . . . . . . . . . . . . . . . . . 34 119 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 120 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 34 121 12. Normative References . . . . . . . . . . . . . . . . . . . . . 34 122 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 124 1. Introduction and Scope 126 This document describes the methodology for benchmarking Link-State 127 Interior Gateway Protocol (IGP) convergence. The motivation and 128 applicability for this benchmarking is described in [Po09a]. The 129 terminology to be used for this benchmarking is described in [Po09t]. 131 IGP convergence time is measured on the data plane at the Tester by 132 observing packet loss through the DUT. All factors contributing to 133 convergence time are accounted for by measuring on the data plane, as 134 discussed in [Po09a]. The test cases in this document are black-box 135 tests that emulate the network events that cause convergence, as 136 described in [Po09a]. 138 The methodology described in this document can be applied to IPv4 and 139 IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF 140 [Mo98][Co08], and others. 142 2. Existing Definitions 144 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 145 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 146 document are to be interpreted as described in BCP 14, RFC 2119 147 [Br97]. RFC 2119 defines the use of these key words to help make the 148 intent of standards track documents as clear as possible. While this 149 document uses these keywords, this document is not a standards track 150 document. 152 This document uses much of the terminology defined in [Po09t] and 153 uses existing terminology defined in other BMWG work. Examples 154 include, but are not limited to: 156 Throughput [Ref.[Br91], section 3.17] 157 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 158 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 159 Out-of-order Packet [Ref.[Po06], section 3.3.2] 160 Duplicate Packet [Ref.[Po06], section 3.3.3] 161 Stream [Ref.[Po06], section 3.3.2] 162 Loss Period [Ref.[Ko02], section 4] 164 3. Test Topologies 166 3.1. Test topology for local changes 168 Figure 1 shows the test topology to measure IGP convergence time due 169 to local Convergence Events such as Local Interface failure 170 (Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP 171 adjacency failure (Section 8.2.2). This topology is also used to 172 measure IGP convergence time due to the route withdrawal 173 (Section 8.2.3), and route cost change (Section 8.3.2) Convergence 174 Events. IGP adjancencies MUST be established between Tester and DUT, 175 one on the Preferred Egress Interface and one on the Next-Best Egress 176 Interface. For this purpose the Tester emulates two routers, each 177 establishing one adjacency with the DUT. An IGP adjacency MAY be 178 established on the Ingress Interface between Tester and DUT. 180 --------- Ingress Interface ---------- 181 | |<--------------------------------| | 182 | | | | 183 | | Preferred Egress Interface | | 184 | DUT |-------------------------------->| Tester | 185 | | | | 186 | |-------------------------------->| | 187 | | Next-Best Egress Interface | | 188 --------- ---------- 190 Figure 1: IGP convergence test topology for local changes 192 3.2. Test topology for remote changes 194 Figure 2 shows the test topology to measure IGP convergence time due 195 to Remote Interface failure (Section 8.1.2). In this topology the 196 two routers R1 and R2 are considered System Under Test (SUT) and 197 SHOULD be identically configured devices of the same model. IGP 198 adjancencies MUST be established between Tester and SUT, one on the 199 Preferred Egress Interface and one on the Next-Best Egress Interface. 200 For this purpose the Tester emulates one or two routers. An IGP 201 adjacency MAY be established on the Ingress Interface between Tester 202 and SUT. In this topology there is a possibility of a transient 203 microloop between R1 and R2 during convergence. 205 ------ ---------- 206 | | Preferred | | 207 ------ | R2 |--------------------->| | 208 | |-->| | Egress Interface | | 209 | | ------ | | 210 | R1 | | Tester | 211 | | Next-Best | | 212 | |------------------------------>| | 213 ------ Egress Interface | | 214 ^ ---------- 215 | | 216 --------------------------------------- 217 Ingress Interface 219 Figure 2: IGP convergence test topology for remote changes 221 3.3. Test topology for local ECMP changes 223 Figure 3 shows the test topology to measure IGP convergence time due 224 to local Convergence Events with members of an Equal Cost Multipath 225 (ECMP) set (Section 8.1.3). In this topology, the DUT is configured 226 with each egress interface as a member of a single ECMP set and the 227 Tester emulates N next-hop routers, one router for each member. IGP 228 adjancencies MUST be established between Tester and DUT, one on each 229 member of the ECMP set. For this purpose each of the N routers 230 emulated by the Tester establishes one adjacency with the DUT. An 231 IGP adjacency MAY be established on the Ingress Interface between 232 Tester and DUT. 234 --------- Ingress Interface ---------- 235 | |<--------------------------------| | 236 | | | | 237 | | ECMP set interface 1 | | 238 | |-------------------------------->| | 239 | DUT | . | Tester | 240 | | . | | 241 | | . | | 242 | |-------------------------------->| | 243 | | ECMP set interface N | | 244 --------- ---------- 246 Figure 3: IGP convergence test topology for local ECMP change 248 3.4. Test topology for remote ECMP changes 250 Figure 4 shows the test topology to measure IGP convergence time due 251 to remote Convergence Events with members of an Equal Cost Multipath 252 (ECMP) set (Section 8.1.4). In this topology the two routers R1 and 253 R2 are considered System Under Test (SUT) and MUST be identically 254 configured devices of the same model. Route R1 is configured with 255 each egress interface as a member of a single ECMP set and the Tester 256 emulates N next-hop routers, one router for each member. IGP 257 adjancencies MUST be established between Tester and SUT, one on each 258 egress interface of SUT. For this purpose each of the N routers 259 emulated by the Tester establishes one adjacency with the SUT. An 260 IGP adjacency MAY be established on the Ingress Interface between 261 Tester and SUT. In this topology there is a possibility of a 262 transient microloop between R1 and R2 during convergence. 264 ------ ---------- 265 | | | | 266 ------ ECMP set | R2 |---->| | 267 | |------------------->| | | | 268 | | Interface 1 ------ | | 269 | | | | 270 | | ECMP set interface 2 | | 271 | R1 |------------------------------>| Tester | 272 | | . | | 273 | | . | | 274 | | . | | 275 | |------------------------------>| | 276 ------ ECMP set interface N | | 277 ^ ---------- 278 | | 279 --------------------------------------- 280 Ingress Interface 282 Figure 4: IGP convergence test topology for remote ECMP convergence 284 3.5. Test topology for Parallel Link changes 286 Figure 5 shows the test topology to measure IGP convergence time due 287 to local Convergence Events with members of a Parallel Link 288 (Section 8.1.5). In this topology, the DUT is configured with each 289 egress interface as a member of a Parallel Link and the Tester 290 emulates the single next-hop router. IGP adjancencies MUST be 291 established on all N members of the Parallel Link between Tester and 292 DUT. For this purpose the router emulated by the Tester establishes 293 N adjacencies with the DUT. An IGP adjacency MAY be established on 294 the Ingress Interface between Tester and DUT. 296 --------- Ingress Interface ---------- 297 | |<--------------------------------| | 298 | | | | 299 | | Parallel Link Interface 1 | | 300 | |-------------------------------->| | 301 | DUT | . | Tester | 302 | | . | | 303 | | . | | 304 | |-------------------------------->| | 305 | | Parallel Link Interface N | | 306 --------- ---------- 308 Figure 5: IGP convergence test topology for Parallel Link changes 310 4. Convergence Time and Loss of Connectivity Period 312 Two concepts will be highlighted in this section: convergence time 313 and loss of connectivity period. 315 The Route Convergence [Po09t] time indicates the period in time 316 between the Convergence Event Instant [Po09t] and the instant in time 317 the DUT is ready to forward traffic for a specific route on its Next- 318 Best Egress Interface and maintains this state for the duration of 319 the Sustained Convergence Validation Time [Po09t]. To measure Route 320 Convergence time, the Convergence Event Instant and the traffic 321 received from the Next-Best Egress Interface need to be observed. 323 The Route Loss of Connectivity Period [Po09t] indicates the time 324 during which traffic to a specific route is lost following a 325 Convergence Event until Full Convergence [Po09t] completes. This 326 Route Loss of Connectivity Period can consist of one or more Loss 327 Periods [Ko02]. For the testcases described in this document it is 328 expected to have a single Loss Period. To measure Route Loss of 329 Connectivity Period, the traffic received from the Preferred Egress 330 Interface and the traffic received from the Next-Best Egress 331 Interface need to be observed. 333 The Route Loss of Connectivity Period is most important since that 334 has a direct impact on the network user's application performance. 336 In general the Route Convergence time is larger than or equal to the 337 Route Loss of Connectivity Period. Depending on which Convergence 338 Event occurs and how this Convergence Event is applied, traffic for a 339 route may still be forwarded over the Preferred Egress Interface 340 after the Convergence Event Instant, before converging to the Next- 341 Best Egress Interface. In that case the Route Loss of Connectivity 342 Period is shorter than the Route Convergence time. 344 At least one condition need to be fulfilled for Route Convergence 345 time to be equal to Route Loss of Connectivity Period. The condition 346 is that the Convergence Event causes an instantaneous traffic loss 347 for the measured route. A fiber cut on the Preferred Egress 348 Interface is an example of such a Convergence Event. For Convergence 349 Events caused by the Tester, such as an IGP cost change, the Tester 350 may start to drop all traffic received from the Preferred Egress 351 Interface at the Convergence Event Instant to achieve the same 352 result. 354 A second condition applies to Route Convergence time measurements 355 based on Connectivity Packet Loss [Po09t].This second condition is 356 that there is only a single Loss Period during Route Convergence. 357 For the testcases described in this document this is expected to be 358 the case. 360 To measure convergence time without real instantaneous traffic loss 361 at the Convergence Event Instant, such as a reversion of a link 362 failure Convergence Event, the Tester SHOULD collect a timestamp at 363 the time instant traffic starts and a timestamp at the Convergence 364 Event Instant, and only observe packet statistics on the Next-Best 365 Egress Interface. 367 The Convergence Event Instant together with the receive rate 368 observations on the Next-Best Egress Interface allow to derive the 369 convergence benchmarks using the Rate-Derived Method [Po09t]. 371 By observing lost packets on the Next-Best Egress Interface only, the 372 measured packet loss is the number of lost packets between traffic 373 start and Convergence Recovery Instant. To measure convergence times 374 using a loss-derived method, packet loss between the Convergence 375 Event Instant and the Convergence Recovery Instant is needed. The 376 time between traffic start and Convergence Event Instant must be 377 accounted for 379 Figure 6 illustrates a Convergence Event without instantaneous 380 traffic loss for all routes. The top graph shows the Forwarding Rate 381 over all routes, the bottem graph shows the Forwarding Rate for a 382 single route Rta. Some time after the Convergence Event Instant, 383 Forwarding Rate observed on the Preferred Egress Interface starts to 384 decrease. In the example route Rta is the first route to experience 385 packet loss at time Ta. Some time later, the Forwarding Rate 386 observed on the Next-Best Egress Interface starts to increase. In 387 the example route Rta is the first route to complete convergence at 388 time Ta'. 390 ^ 391 Fwd | 392 Rate |------------- ............ 393 | \ . 394 | \ . 395 | \ . 396 | \ . 397 |.................-.-.-.-.-.-.---------------- 398 +----+-------+---------------+-----------------> 399 ^ ^ ^ ^ time 400 T0 CEI Ta Ta' 402 ^ 403 Fwd | 404 Rate |------------- ................. 405 Rta | | . 406 | | . 407 |.............-.-.-.-.-.-.-.-.---------------- 408 +----+-------+---------------+-----------------> 409 ^ ^ ^ ^ time 410 T0 CEI Ta Ta' 412 Preferred Egress Interface: --- 413 Next-Best Egress Interface: ... 415 With CEI the Convergence Event Instant; T0 the time instant traffic 416 starts; Ta the time instant traffic loss for route Rta starts; Ta' 417 the time instant traffic loss for route Rta ends. 419 Figure 6 421 If only packets received on the Next-Best Egress Interface are 422 observed, the duration of the packet loss period for route Rta 423 observed on the Next-Best Egress Interface can be calculated from the 424 received packets as in Equation 1. Since the Convergence Event 425 Instant is the start time for convergence time measurement, the 426 period in time between T0 and CEI needs to be substracted from the 427 calculated result to become the convergence time, as in Equation 2. 429 Next-Best Egress Interface packet loss period 430 = (packets transmitted 431 - packets received from Next-Best Egress Interface) / tx rate 432 = Ta' - T0 434 Equation 1 436 convergence time 437 = Next-Best Egress Interface packet loss period - (CEI - T0) 438 = Ta' - CEI 440 Equation 2 442 Route Loss of Connectivity Period SHOULD be measured using the Route- 443 Specific Loss-Derived Method. Since the start instant and end 444 instant of the Route Loss of Connectivity Period can be different for 445 each route, these can not be accurately derived by only observing 446 global statistics over all routes. An example may clarify this. 448 Following a Convergence Event, route Rta is the first route for which 449 packet loss starts, the Route Loss of Connectivity Period for route 450 Rta starts at time Ta. Route Rtb is the last route for which packet 451 loss starts, the Route Loss of Connectivity Period for route Rtb 452 starts at time Tb with Tb>Ta. 454 ^ 455 Fwd | 456 Rate |-------- ----------- 457 | \ / 458 | \ / 459 | \ / 460 | \ / 461 | --------------- 462 +------------------------------------------> 463 ^ ^ ^ ^ time 464 Ta Tb Ta' Tb' 465 Tb'' Ta'' 467 Figure 7: Example Route Loss Of Connectivity Period 469 If the DUT implementation would be such that Route Rta would be the 470 first route for which traffic loss ends at time Ta' with Ta'>Tb. 471 Route Rtb would be the last route for which traffic loss ends at time 472 Tb' with Tb'>Ta'. By using only observing global traffic statistics 473 over all routes, the minimum Route Loss of Connectivity Period would 474 be measured as Ta'-Ta. The maximum calculated Route Loss of 475 Connectivity Period would be Tb'-Ta. The real minimum and maximum 476 Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb. 477 Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5, 478 would give a LoC Period between 3 and 5 derived from the global 479 traffic statistics, versus the real LoC Period between 3 and 4. 481 If the DUT implementation would be such that route Rtb would be the 482 first for which packet loss ends at time Tb'' and route Rta would be 483 the last for which packet loss ends at time Ta'', then the minimum 484 and maximum Route Loss of Connectivity Periods derived by observing 485 only global traffic statistics would be Tb''-Ta, and Ta''-Ta. The 486 real minimum and maximum Route Loss of Connectivity Periods are 487 Tb''-Tb and Ta''-Ta. Illustrating this with the numbers Ta=0, Tb=1, 488 Ta''=5, Tb''=3, would give a LoC Period between 3 and 5 derived from 489 the global traffic statistics, versus the real LoC Period between 2 490 and 5. 492 The two implementation variations in the above example would result 493 in the same derived minimum and maximum Route Loss of Connectivity 494 Periods when only observing the global packet statistics, while the 495 real Route Loss of Connectivity Periods are different. 497 5. Test Considerations 499 5.1. IGP Selection 501 The test cases described in section 4 MAY be used for link-state 502 IGPs, such as ISIS or OSPF. The IGP convergence time test 503 methodology is identical. 505 5.2. Routing Protocol Configuration 507 The obtained results for IGP convergence time may vary if other 508 routing protocols are enabled and routes learned via those protocols 509 are installed. IGP convergence times MUST be benchmarked without 510 routes installed from other protocols. 512 5.3. IGP Topology 514 The Tester emulates a single IGP topology. The DUT establishes IGP 515 adjacencies with one or more of the emulated routers in this single 516 IGP topology emulated by the Tester. See topology details in 517 Section 3. The emulated topology SHOULD only be advertised on the 518 DUT egress interfaces. 520 The number of IGP routes will impact the measured IGP route 521 convergence time. To obtain results similar to those that would be 522 observed in an operational network, it is RECOMMENDED that the number 523 of installed routes and nodes closely approximates that of the 524 network (e.g. thousands of routes with tens or hundreds of nodes). 526 The number of areas (for OSPF) and levels (for ISIS) can impact the 527 benchmark results. 529 5.4. Timers 531 There are timers that may impact the measured IGP convergence times. 532 The benchmark metrics MAY be measured at any fixed values for these 533 timers. To obtain results similar to those that would be observed in 534 an operational network, it is RECOMMENDED to configure the timers 535 with the values as configured in the operational network. 537 Examples of timers that may impact measured IGP convergence time 538 include, but are not limited to: 540 Interface failure indication 542 IGP hello timer 544 IGP dead-interval or hold-timer 546 LSA or LSP generation delay 548 LSA or LSP flood packet pacing 550 LSA or LSP retransmission packet pacing 552 SPF delay 554 5.5. Interface Types 556 All test cases in this methodology document MAY be executed with any 557 interface type. The type of media may dictate which test cases may 558 be executed. This is because each interface type has a unique 559 mechanism for detecting link failures and the speed at which that 560 mechanism operates will influence the measurement results. All 561 interfaces MUST be the same media and Throughput [Br91][Br99] for 562 each test case. All interfaces SHOULD be configured as point-to- 563 point. 565 5.6. Offered Load 567 The Throughput of the device, as defined in [Br91] and benchmarked in 568 [Br99] at a fixed packet size, needs to be determined over the 569 preferred path and over the next-best path. The Offered Load SHOULD 570 be the minumum of the measured Throughput of the device over the 571 primary path and over the backup path. The packet size is selectable 572 and MUST be recorded. Packet size is measured in bytes and includes 573 the IP header and payload. 575 In the Remote Interface failure testcases using topologies 2 and 4 576 there is a possibility of a transient microloop between R1 and R2 577 during convergence. The TTL value of the packets send by the Tester 578 may influence the benchmark measurements since it determines which 579 device in the topology may send an ICMP Time Exceeded Message for 580 looped packets. 582 The duration of the Offered Load MUST be greater than the convergence 583 time. 585 5.7. Measurement Accuracy 587 Since packet loss is observed to measure the Route Convergence Time, 588 the time between two successive packets offered to each individual 589 route is the highest possible accuracy of any packet loss based 590 measurement. When packet jitter is much less than the convergence 591 time, it is a negligible source of error and therefor it will be 592 ignored here. 594 5.8. Measurement Statistics 596 The benchmark measurements may vary for each trial, due to the 597 statistical nature of timer expirations, cpu scheduling, etc. 598 Evaluation of the test data must be done with an understanding of 599 generally accepted testing practices regarding repeatability, 600 variance and statistical significance of a small number of trials. 602 5.9. Tester Capabilities 604 It is RECOMMENDED that the Tester used to execute each test case has 605 the following capabilities: 607 1. Ability to establish IGP adjacencies and advertise a single IGP 608 topology to one or more peers. 610 2. Ability to insert a timestamp in each data packet's IP payload. 612 3. An internal time clock to control timestamping, time 613 measurements, and time calculations. 615 4. Ability to distinguish traffic load received on the Preferred and 616 Next-Best Interfaces [Po09t]. 618 5. Ability to disable or tune specific Layer-2 and Layer-3 protocol 619 functions on any interface(s). 621 The Tester MAY be capable to make non-data plane convergence 622 observations and use those observations for measurements. The Tester 623 MAY be capable to send and receive multiple traffic Streams [Po06]. 625 6. Selection of Convergence Time Benchmark Metrics and Methods 627 Different convergence time benchmark methods MAY be used to measure 628 convergence time benchmark metrics. The Tester capabilities are 629 important criteria to select a specific convergence time benchmark 630 method. The criteria to select a specific benchmark method include, 631 but are not limited to: 633 Tester capabilities: Sampling Interval, number of 634 Stream statistics to collect 635 Measurement accuracy: Sampling Interval, Offered Load 636 Test specification: number of routes 637 DUT capabilities: Throughput 639 6.1. Loss-Derived Method 641 6.1.1. Tester capabilities 643 The Offered Load SHOULD consist of a single Stream [Po06]. If 644 sending multiple Streams, the measured packet loss statistics for all 645 Streams MUST be added together. 647 The destination addresses for the Offered Load MUST be distributed 648 such that all routes are matched and each route is offered an equal 649 share of the total Offered Load. 651 In order to verify Full Convergence completion and the Sustained 652 Convergence Validation Time, the Tester MUST measure Forwarding Rate 653 each Packet Sampling Interval. 655 The total number of packets lost between the start of the traffic and 656 the end of the Sustained Convergence Validation Time is used to 657 calculate the Loss-Derived Convergence Time. 659 6.1.2. Benchmark Metrics 661 The Loss-Derived Method can be used to measure the Loss-Derived 662 Convergence Time, which is the average convergence time over all 663 routes, and to measure the Loss-Derived Loss of Connectivity Period, 664 which is the average Route Loss of Connectivity Period over all 665 routes. 667 6.1.3. Measurement Accuracy 669 TBD 671 6.2. Rate-Derived Method 673 6.2.1. Tester Capabilities 675 The Offered Load SHOULD consist of a single Stream. If sending 676 multiple Streams, the measured traffic rate statistics for all 677 Streams MUST be added together. 679 The destination addresses for the Offered Load MUST be distributed 680 such that all routes are matched and each route is offered an equal 681 share of the total Offered Load. 683 The Tester measures Forwarding Rate each Sampling Interval. The 684 Packet Sampling Interval influences the observation of the different 685 convergence time instants. If the Packet Sampling Interval is large 686 in comparison to the time between the convergence time instants, then 687 the different time instants may not be easily identifiable from the 688 Forwarding Rate observation. The requirements for the Packet 689 Sampling Interval are specified in [Po09t]. The RECOMMENDED value 690 for the Packet Sampling Interval is 10 milliseconds. The Packet 691 Sampling Interval MUST be reported. 693 6.2.2. Benchmark Metrics 695 The Rate-Derived Method SHOULD be used to measure First Route 696 Convergence Time and Full Convergence Time. It SHOULD NOT be used to 697 measure Loss of Connectivity Period (see Section Section 4). 699 6.2.3. Measurement Accuracy 701 The measurement accuracy of the Rate-Derived Method for transitions 702 that occur for all routes at the same instant is equal to the Packet 703 Sampling Interval and for other transitions the measurement accuracy 704 is equal to the Packet Sampling Interval plus the time between two 705 consecutive packets to the same destination. The latter is the case 706 since packets are sent in a particular order to all destinations in a 707 stream and when part of the routes experience packet loss, it is 708 unknown where in the transmit cycle packets to these routes are sent. 709 This uncertainty adds to the error. 711 6.3. Route-Specific Loss-Derived Method 713 6.3.1. Tester Capabilities 715 The Offered Load consists of multiple Streams. To measure Route- 716 Specific Convergence Times, the Tester sends one Stream to each route 717 in the FIB. The Tester MUST measure packet loss for each Stream 718 seperately. 720 In order to verify Full Convergence completion and the Sustained 721 Convergence Validation Time, the Tester MUST measure packet loss each 722 Packet Sampling Interval. This measurement at each Packet Sampling 723 Interval MAY be per Stream. 725 Only the total packet loss measured per Stream at the end of the 726 Sustained Convergence Validation Time is used to calculate the 727 benchmark metrics with this method. 729 6.3.2. Benchmark Metrics 731 The Route-Specific Loss-Derived Method SHOULD be used to measure 732 Route-Specific Convergence Times. It is the RECOMMENDED method to 733 measure Route Loss of Connectivity Period. 735 Under the conditions explained in Section 4, First Route Convergence 736 Time and Full Convergence Time as benchmarked using Rate-Derived 737 Method, may be equal to the minimum resp. maximum of the Route- 738 Specific Convergence Times. 740 6.3.3. Measurement Accuracy 742 The measurement accuracy of the Route-Specific Loss-Derived Method is 743 equal to the time between two consecutive packets to the same route. 745 7. Reporting Format 747 For each test case, it is recommended that the reporting tables below 748 are completed and all time values SHOULD be reported with resolution 749 as specified in [Po09t]. 751 Parameter Units 752 ----------------------------------- ----------------------- 753 Test Case test case number 754 Test Topology (1, 2, 3, 4, or 5) 755 IGP (ISIS, OSPF, other) 756 Interface Type (GigE, POS, ATM, other) 757 Packet Size offered to DUT bytes 758 Offered Load packets per second 759 IGP Routes advertised to DUT number of IGP routes 760 Nodes in emulated network number of nodes 761 Packet Sampling Interval on Tester seconds 762 Maximum Packet Delay Threshold seconds 764 Timer Values configured on DUT: 765 Interface failure indication delay seconds 766 IGP Hello Timer seconds 767 IGP Dead-Interval or hold-time seconds 768 LSA Generation Delay seconds 769 LSA Flood Packet Pacing seconds 770 LSA Retransmission Packet Pacing seconds 771 SPF Delay seconds 773 Complete the table below for the initial Convergence Event and the 774 reversion Convergence Event. 776 Parameter Units 777 ------------------------------------------ ---------------------- 778 Conversion Event (initial or reversion) 780 Traffic Forwarding Metrics: 781 Total number of packets offered to DUT number of Packets 782 Total number of packets forwarded by DUT number of Packets 783 Connectivity Packet Loss number of Packets 784 Convergence Packet Loss number of Packets 785 Out-of-Order Packets number of Packets 786 Duplicate Packets number of Packets 788 Convergence Benchmarks: 789 Rate-Derived Method: 790 First Route Convergence Time seconds 791 Full Convergence Time seconds 792 Loss-Derived Method: 793 Loss-Derived Convergence Time seconds 794 Route-Specific Loss-Derived Method: 795 Number of Routes Measured number of routes 796 Route-Specific Convergence Time[n] array of seconds 797 Minimum R-S Convergence Time seconds 798 Maximum R-S Convergence Time seconds 799 Median R-S Convergence Time seconds 800 Average R-S Convergence Time seconds 802 Loss of Connectivity Benchmarks: 803 Loss-Derived Method: 804 Loss-Derived Loss of Connectivity Period seconds 805 Route-Specific Loss-Derived Method: 806 Number of Routes Measured number of routes 807 Route LoC Period[n] array of seconds 808 Minimum Route LoC Period seconds 809 Maximum Route LoC Period seconds 810 Median Route LoC Period seconds 811 Average Route LoC Period seconds 813 8. Test Cases 815 It is RECOMMENDED that all applicable test cases be performed for 816 best characterization of the DUT. The test cases follow a generic 817 procedure tailored to the specific DUT configuration and Convergence 818 Event[Po09t]. This generic procedure is as follows: 820 1. Establish DUT and Tester configurations and advertise an IGP 821 topology from Tester to DUT. 823 2. Send Offered Load from Tester to DUT on ingress interface. 825 3. Verify traffic is routed correctly. 827 4. Introduce Convergence Event [Po09t]. 829 5. Measure First Route Convergence Time [Po09t]. 831 6. Measure Full Convergence Time [Po09t]. 833 7. Stop Offered Load. 835 8. Measure Route-Specific Convergence Times, Loss-Derived 836 Convergence Time, Route LoC Periods, and Loss-Derived LoC Period 837 [Po09t]. 839 9. Wait sufficient time for queues to drain. 841 10. Restart Offered Load. 843 11. Reverse Convergence Event. 845 12. Measure First Route Convergence Time. 847 13. Measure Full Convergence Time. 849 14. Stop Offered Load. 851 15. Measure Route-Specific Convergence Times, Loss-Derived 852 Convergence Time, Route LoC Periods, and Loss-Derived LoC 853 Period. 855 8.1. Interface failures 857 8.1.1. Convergence Due to Local Interface Failure 859 Objective 861 To obtain the IGP convergence times due to a Local Interface failure 862 event. 864 Procedure 866 1. Advertise an IGP topology from Tester to DUT using the topology 867 shown in Figure 1. 869 2. Send Offered Load from Tester to DUT on ingress interface. 871 3. Verify traffic is forwarded over Preferred Egress Interface. 873 4. Remove link on DUT's Preferred Egress Interface. This is the 874 Convergence Event. 876 5. Measure First Route Convergence Time. 878 6. Measure Full Convergence Time. 880 7. Stop Offered Load. 882 8. Measure Route-Specific Convergence Times and Loss-Derived 883 Convergence Time. 885 9. Wait sufficient time for queues to drain. 887 10. Restart Offered Load. 889 11. Restore link on DUT's Preferred Egress Interface. 891 12. Measure First Route Convergence Time. 893 13. Measure Full Convergence Time. 895 14. Stop Offered Load. 897 15. Measure Route-Specific Convergence Times, Loss-Derived 898 Convergence Time, Route LoC Periods, and Loss-Derived LoC 899 Period. 901 Results 903 The measured IGP convergence time may be influenced by the link 904 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 905 LSP flood packet pacing, SPF delay, SPF execution time, and routing 906 and forwarding tables update time [Po09a]. 908 8.1.2. Convergence Due to Remote Interface Failure 910 Objective 912 To obtain the IGP convergence time due to a Remote Interface failure 913 event. 915 Procedure 916 1. Advertise an IGP topology from Tester to SUT using the topology 917 shown in Figure 2. 919 2. Send Offered Load from Tester to SUT on ingress interface. 921 3. Verify traffic is forwarded over Preferred Egress Interface. 923 4. Remove link on Tester's interface [Po09t] connected to SUT's 924 Preferred Egress Interface. This is the Convergence Event. 926 5. Measure First Route Convergence Time. 928 6. Measure Full Convergence Time. 930 7. Stop Offered Load. 932 8. Measure Route-Specific Convergence Times and Loss-Derived 933 Convergence Time. 935 9. Wait sufficient time for queues to drain. 937 10. Restart Offered Load. 939 11. Restore link on Tester's interface connected to DUT's Preferred 940 Egress Interface. 942 12. Measure First Route Convergence Time. 944 13. Measure Full Convergence Time. 946 14. Stop Offered Load. 948 15. Measure Route-Specific Convergence Times, Loss-Derived 949 Convergence Time, Route LoC Periods, and Loss-Derived LoC 950 Period. 952 Results 954 The measured IGP convergence time may be influenced by the link 955 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 956 LSP flood packet pacing, SPF delay, SPF execution time, and routing 957 and forwarding tables update time. This test case may produce Stale 958 Forwarding [Po09t] due to a transient microloop between R1 and R2 959 during convergence, which may increase the measured convergence times 960 and loss of connectivity periods. 962 8.1.3. Convergence Due to ECMP Member Local Interface Failure 964 Objective 966 To obtain the IGP convergence time due to a Local Interface link 967 failure event of an ECMP Member. 969 Procedure 971 1. Advertise an IGP topology from Tester to DUT using the test 972 setup shown in Figure 3. 974 2. Send Offered Load from Tester to DUT on ingress interface. 976 3. Verify traffic is forwarded over the DUT's ECMP member interface 977 that will be failed in the next step. 979 4. Remove link on one of the DUT's ECMP member interfaces. This is 980 the Convergence Event. 982 5. Measure First Route Convergence Time. 984 6. Measure Full Convergence Time. 986 7. Stop Offered Load. 988 8. Measure Route-Specific Convergence Times and Loss-Derived 989 Convergence Time. At the same time measure Out-of-Order Packets 990 [Po06] and Duplicate Packets [Po06]. 992 9. Wait sufficient time for queues to drain. 994 10. Restart Offered Load. 996 11. Restore link on DUT's ECMP member interface. 998 12. Measure First Route Convergence Time. 1000 13. Measure Full Convergence Time. 1002 14. Stop Offered Load. 1004 15. Measure Route-Specific Convergence Times, Loss-Derived 1005 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1006 Period. At the same time measure Out-of-Order Packets [Po06] 1007 and Duplicate Packets [Po06]. 1009 Results 1010 The measured IGP Convergence time may be influenced by link failure 1011 indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP 1012 flood packet pacing, SPF delay, SPF execution time, and routing and 1013 forwarding tables update time [Po09a]. 1015 8.1.4. Convergence Due to ECMP Member Remote Interface Failure 1017 Objective 1019 To obtain the IGP convergence time due to a Remote Interface link 1020 failure event for an ECMP Member. 1022 Procedure 1024 1. Advertise an IGP topology from Tester to DUT using the test 1025 setup shown in Figure 4. 1027 2. Send Offered Load from Tester to DUT on ingress interface. 1029 3. Verify traffic is forwarded over the DUT's ECMP member interface 1030 that will be failed in the next step. 1032 4. Remove link on Tester's interface to R2. This is the 1033 Convergence Event Trigger. 1035 5. Measure First Route Convergence Time. 1037 6. Measure Full Convergence Time. 1039 7. Stop Offered Load. 1041 8. Measure Route-Specific Convergence Times and Loss-Derived 1042 Convergence Time. At the same time measure Out-of-Order Packets 1043 [Po06] and Duplicate Packets [Po06]. 1045 9. Wait sufficient time for queues to drain. 1047 10. Restart Offered Load. 1049 11. Restore link on Tester's interface to R2. 1051 12. Measure First Route Convergence Time. 1053 13. Measure Full Convergence Time. 1055 14. Stop Offered Load. 1057 15. Measure Route-Specific Convergence Times, Loss-Derived 1058 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1059 Period. At the same time measure Out-of-Order Packets [Po06] 1060 and Duplicate Packets [Po06]. 1062 Results 1064 The measured IGP convergence time may influenced by the link failure 1065 indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP 1066 flood packet pacing, SPF delay, SPF execution time, and routing and 1067 forwarding tables update time. This test case may produce Stale 1068 Forwarding [Po09t] due to a transient microloop between R1 and R2 1069 during convergence, which may increase the measured convergence times 1070 and loss of connectivity periods. 1072 8.1.5. Convergence Due to Parallel Link Interface Failure 1074 Objective 1076 To obtain the IGP convergence due to a local link failure event for a 1077 member of a parallel link. The links can be used for data load 1078 balancing 1080 Procedure 1082 1. Advertise an IGP topology from Tester to DUT using the test 1083 setup shown in Figure 5. 1085 2. Send Offered Load from Tester to DUT on ingress interface. 1087 3. Verify traffic is forwarded over the parallel link member that 1088 will be failed in the next step. 1090 4. Remove link on one of the DUT's parallel link member interfaces. 1091 This is the Convergence Event. 1093 5. Measure First Route Convergence Time. 1095 6. Measure Full Convergence Time. 1097 7. Stop Offered Load. 1099 8. Measure Route-Specific Convergence Times and Loss-Derived 1100 Convergence Time. At the same time measure Out-of-Order Packets 1101 [Po06] and Duplicate Packets [Po06]. 1103 9. Wait sufficient time for queues to drain. 1105 10. Restart Offered Load. 1107 11. Restore link on DUT's Parallel Link member interface. 1109 12. Measure First Route Convergence Time. 1111 13. Measure Full Convergence Time. 1113 14. Stop Offered Load. 1115 15. Measure Route-Specific Convergence Times, Loss-Derived 1116 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1117 Period. At the same time measure Out-of-Order Packets [Po06] 1118 and Duplicate Packets [Po06]. 1120 Results 1122 The measured IGP convergence time may be influenced by the link 1123 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1124 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1125 and forwarding tables update time [Po09a]. 1127 8.2. Other failures 1129 8.2.1. Convergence Due to Layer 2 Session Loss 1131 Objective 1133 To obtain the IGP convergence time due to a local layer 2 loss. 1135 Procedure 1137 1. Advertise an IGP topology from Tester to DUT using the topology 1138 shown in Figure 1. 1140 2. Send Offered Load from Tester to DUT on ingress interface. 1142 3. Verify traffic is routed over Preferred Egress Interface. 1144 4. Remove Layer 2 session from DUT's Preferred Egress Interface. 1145 This is the Convergence Event. 1147 5. Measure First Route Convergence Time. 1149 6. Measure Full Convergence Time. 1151 7. Stop Offered Load. 1153 8. Measure Route-Specific Convergence Times, Loss-Derived 1154 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1155 Period. 1157 9. Wait sufficient time for queues to drain. 1159 10. Restart Offered Load. 1161 11. Restore Layer 2 session on DUT's Preferred Egress Interface. 1163 12. Measure First Route Convergence Time. 1165 13. Measure Full Convergence Time. 1167 14. Stop Offered Load. 1169 15. Measure Route-Specific Convergence Times, Loss-Derived 1170 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1171 Period. 1173 Results 1175 The measured IGP Convergence time may be influenced by the Layer 2 1176 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1177 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1178 and forwarding tables update time [Po09a]. 1180 Discussion 1182 Configure IGP timers such that the IGP adjacency does not time out 1183 before layer 2 failure is detected. 1185 To measure convergence time, traffic SHOULD start dropping on the 1186 Preferred Egress Interface on the instant the layer 2 session is 1187 removed. Alternatively the Tester SHOULD record the time the instant 1188 layer 2 session is removed and traffic loss SHOULD only be measured 1189 on the Next-Best Egress Interface. 1191 8.2.2. Convergence Due to Loss of IGP Adjacency 1193 Objective 1195 To obtain the IGP convergence time due to loss of an IGP Adjacency. 1197 Procedure 1198 1. Advertise an IGP topology from Tester to DUT using the topology 1199 shown in Figure 1. 1201 2. Send Offered Load from Tester to DUT on ingress interface. 1203 3. Verify traffic is routed over Preferred Egress Interface. 1205 4. Remove IGP adjacency from the Preferred Egress Interface while 1206 the layer 2 session MUST be maintained. This is the Convergence 1207 Event. 1209 5. Measure First Route Convergence Time. 1211 6. Measure Full Convergence Time. 1213 7. Stop Offered Load. 1215 8. Measure Route-Specific Convergence Times, Loss-Derived 1216 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1217 Period. 1219 9. Wait sufficient time for queues to drain. 1221 10. Restart Offered Load. 1223 11. Restore IGP session on DUT's Preferred Egress Interface. 1225 12. Measure First Route Convergence Time. 1227 13. Measure Full Convergence Time. 1229 14. Stop Offered Load. 1231 15. Measure Route-Specific Convergence Times, Loss-Derived 1232 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1233 Period. 1235 Results 1237 The measured IGP Convergence time may be influenced by the IGP Hello 1238 Interval, IGP Dead Interval, LSA/LSP delay, LSA/LSP generation time, 1239 LSA/LSP flood packet pacing, SPF delay, SPF execution time, and 1240 routing and forwarding tables update time [Po09a]. 1242 Discussion 1244 Configure layer 2 such that layer 2 does not time out before IGP 1245 adjacency failure is detected. 1247 To measure convergence time, traffic SHOULD start dropping on the 1248 Preferred Egress Interface on the instant the IGP adjacency is 1249 removed. Alternatively the Tester SHOULD record the time the instant 1250 the IGP adjacency is removed and traffic loss SHOULD only be measured 1251 on the Next-Best Egress Interface. 1253 8.2.3. Convergence Due to Route Withdrawal 1255 Objective 1257 To obtain the IGP convergence time due to route withdrawal. 1259 Procedure 1261 1. Advertise an IGP topology from Tester to DUT using the topology 1262 shown in Figure 1. The routes that will be withdrawn MUST be a 1263 set of leaf routes advertised by at least two nodes in the 1264 emulated topology. The topology SHOULD be such that before the 1265 withdrawal the DUT prefers the leaf routes advertised by a node 1266 "nodeA" via the Preferred Egress Interface, and after the 1267 withdrawal the DUT prefers the leaf routes advertised by a node 1268 "nodeB" via the Next-Best Egress Interface. 1270 2. Send Offered Load from Tester to DUT on Ingress Interface. 1272 3. Verify traffic is routed over Preferred Egress Interface. 1274 4. The Tester withdraws the set of IGP leaf routes from nodeA. The 1275 withdrawal update message MUST be a single unfragmented packet. 1276 This is the Convergence Event. The Tester MAY record the time 1277 it sends the withdrawal message(s). 1279 5. Measure First Route Convergence Time. 1281 6. Measure Full Convergence Time. 1283 7. Stop Offered Load. 1285 8. Measure Route-Specific Convergence Times, Loss-Derived 1286 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1287 Period. 1289 9. Wait sufficient time for queues to drain. 1291 10. Restart Offered Load. 1293 11. Re-advertise the set of withdrawn IGP leaf routes from nodeA 1294 emulated by the Tester. The update message MUST be a single 1295 unfragmented packet. 1297 12. Measure First Route Convergence Time. 1299 13. Measure Full Convergence Time. 1301 14. Stop Offered Load. 1303 15. Measure Route-Specific Convergence Times, Loss-Derived 1304 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1305 Period. 1307 Results 1309 The measured IGP convergence time is influenced by SPF or route 1310 calculation delay, SPF or route calculation execution time, and 1311 routing and forwarding tables update time [Po09a]. 1313 Discussion 1315 To measure convergence time, traffic SHOULD start dropping on the 1316 Preferred Egress Interface on the instant the routes are withdrawn by 1317 the Tester. Alternatively the Tester SHOULD record the time the 1318 instant the routes are withdrawn and traffic loss SHOULD only be 1319 measured on the Next-Best Egress Interface. 1321 8.3. Administrative changes 1323 8.3.1. Convergence Due to Local Adminstrative Shutdown 1325 Objective 1327 To obtain the IGP convergence time due to taking the DUT's Local 1328 Interface administratively out of service. 1330 Procedure 1332 1. Advertise an IGP topology from Tester to DUT using the topology 1333 shown in Figure 1. 1335 2. Send Offered Load from Tester to DUT on ingress interface. 1337 3. Verify traffic is routed over Preferred Egress Interface. 1339 4. Take the DUT's Preferred Egress Interface administratively out 1340 of service. This is the Convergence Event. 1342 5. Measure First Route Convergence Time. 1344 6. Measure Full Convergence Time. 1346 7. Stop Offered Load. 1348 8. Measure Route-Specific Convergence Times, Loss-Derived 1349 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1350 Period. 1352 9. Wait sufficient time for queues to drain. 1354 10. Restart Offered Load. 1356 11. Restore Preferred Egress Interface by administratively enabling 1357 the interface. 1359 12. Measure First Route Convergence Time. 1361 13. Measure Full Convergence Time. 1363 14. Stop Offered Load. 1365 15. Measure Route-Specific Convergence Times, Loss-Derived 1366 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1367 Period. 1369 16. It is possible that no measured packet loss will be observed for 1370 this test case. 1372 Results 1374 The measured IGP Convergence time may be influenced by LSA/LSP delay, 1375 LSA/LSP generation time, LSA/LSP flood packet pacing, SPF delay, SPF 1376 execution time, and routing and forwarding tables update time 1377 [Po09a]. 1379 8.3.2. Convergence Due to Cost Change 1381 Objective 1383 To obtain the IGP convergence time due to route cost change. 1385 Procedure 1387 1. Advertise an IGP topology from Tester to DUT using the topology 1388 shown in Figure 1. 1390 2. Send Offered Load from Tester to DUT on ingress interface. 1392 3. Verify traffic is routed over Preferred Egress Interface. 1394 4. The Tester, emulating the neighbor node, increases the cost for 1395 all IGP routes at DUT's Preferred Egress Interface so that the 1396 Next-Best Egress Interface becomes preferred path. The update 1397 message advertising the higher cost MUST be a single 1398 unfragmented packet. This is the Convergence Event. The Tester 1399 MAY record the time it sends the update message advertising the 1400 higher cost on the Preferred Egress Interface. 1402 5. Measure First Route Convergence Time. 1404 6. Measure Full Convergence Time. 1406 7. Stop Offered Load. 1408 8. Measure Route-Specific Convergence Times, Loss-Derived 1409 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1410 Period. 1412 9. Wait sufficient time for queues to drain. 1414 10. Restart Offered Load. 1416 11. The Tester, emulating the neighbor node, decreases the cost for 1417 all IGP routes at DUT's Preferred Egress Interface so that the 1418 Preferred Egress Interface becomes preferred path. The update 1419 message advertising the lower cost MUST be a single unfragmented 1420 packet. 1422 12. Measure First Route Convergence Time. 1424 13. Measure Full Convergence Time. 1426 14. Stop Offered Load. 1428 15. Measure Route-Specific Convergence Times, Loss-Derived 1429 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1430 Period. 1432 Results 1434 The measured IGP Convergence time may be influenced by SPF delay, SPF 1435 execution time, and routing and forwarding tables update time 1436 [Po09a]. 1438 Discussion 1440 To measure convergence time, traffic SHOULD start dropping on the 1441 Preferred Egress Interface on the instant the cost is changed by the 1442 Tester. Alternatively the Tester SHOULD record the time the instant 1443 the cost is changed and traffic loss SHOULD only be measured on the 1444 Next-Best Egress Interface. 1446 9. Security Considerations 1448 Documents of this type do not directly affect the security of 1449 Internet or corporate networks as long as benchmarking is not 1450 performed on devices or systems connected to production networks. 1451 Security threats and how to counter these in SIP and the media layer 1452 is discussed in RFC3261, RFC3550, and RFC3711 and various other 1453 drafts. This document attempts to formalize a set of common 1454 methodology for benchmarking IGP convergence performance in a lab 1455 environment. 1457 10. IANA Considerations 1459 This document requires no IANA considerations. 1461 11. Acknowledgements 1463 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1464 Peter De Vriendt and the BMWG for their contributions to this work. 1466 12. Normative References 1468 [Br91] Bradner, S., "Benchmarking terminology for network 1469 interconnection devices", RFC 1242, July 1991. 1471 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1472 Requirement Levels", BCP 14, RFC 2119, March 1997. 1474 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1475 Network Interconnect Devices", RFC 2544, March 1999. 1477 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1478 environments", RFC 1195, December 1990. 1480 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1481 IPv6", RFC 5340, July 2008. 1483 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1484 October 2008. 1486 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1487 Metrics", RFC 3357, August 2002. 1489 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1490 Devices", RFC 2285, February 1998. 1492 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1494 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1495 "Terminology for Benchmarking Network-layer Traffic Control 1496 Mechanisms", RFC 4689, October 2006. 1498 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1499 IGP Data Plane Route Convergence", 1500 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1501 progress), March 2009. 1503 [Po09t] Poretsky, S. and B. Imhoff, "Terminology for Benchmarking 1504 Link-State IGP Data Plane Route Convergence", 1505 draft-ietf-bmwg-igp-dataplane-conv-term-18 (work in 1506 progress), July 2009. 1508 Authors' Addresses 1510 Scott Poretsky 1511 Allot Communications 1512 67 South Bedford Street, Suite 400 1513 Burlington, MA 01803 1514 USA 1516 Phone: + 1 508 309 2179 1517 Email: sporetsky@allot.com 1519 Brent Imhoff 1520 Juniper Networks 1521 1194 North Mathilda Ave 1522 Sunnyvale, CA 94089 1523 USA 1525 Phone: + 1 314 378 2571 1526 Email: bimhoff@planetspork.com 1527 Kris Michielsen 1528 Cisco Systems 1529 6A De Kleetlaan 1530 Diegem, BRABANT 1831 1531 Belgium 1533 Email: kmichiel@cisco.com