idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-meth-19.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 774 has weird spacing: '... Tester secon...' == Line 783 has weird spacing: '... Pacing seco...' -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 26, 2009) is 5296 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-term-18 Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: April 29, 2010 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 October 26, 2009 10 Benchmarking Methodology for Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-meth-19 13 Status of this Memo 15 This Internet-Draft is submitted to IETF in full conformance with the 16 provisions of BCP 78 and BCP 79. This document may contain material 17 from IETF Documents or IETF Contributions published or made publicly 18 available before November 10, 2008. The person(s) controlling the 19 copyright in some of this material may not have granted the IETF 20 Trust the right to allow modifications of such material outside the 21 IETF Standards Process. Without obtaining an adequate license from 22 the person(s) controlling the copyright in such materials, this 23 document may not be modified outside the IETF Standards Process, and 24 derivative works of it may not be created outside the IETF Standards 25 Process, except to format it for publication as an RFC or to 26 translate it into languages other than English. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as Internet- 31 Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/ietf/1id-abstracts.txt. 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html. 44 This Internet-Draft will expire on April 29, 2010. 46 Copyright Notice 48 Copyright (c) 2009 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents in effect on the date of 53 publication of this document (http://trustee.ietf.org/license-info). 54 Please review these documents carefully, as they describe your rights 55 and restrictions with respect to this document. 57 Abstract 59 This document describes the methodology for benchmarking Link-State 60 Interior Gateway Protocol (IGP) Route Convergence. The methodology 61 is to be used for benchmarking IGP convergence time through 62 externally observable (black box) data plane measurements. The 63 methodology can be applied to any link-state IGP, such as ISIS and 64 OSPF. 66 Table of Contents 68 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 5 69 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 5 70 3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 5 71 3.1. Test topology for local changes . . . . . . . . . . . . . 5 72 3.2. Test topology for remote changes . . . . . . . . . . . . . 6 73 3.3. Test topology for local ECMP changes . . . . . . . . . . . 7 74 3.4. Test topology for remote ECMP changes . . . . . . . . . . 7 75 3.5. Test topology for Parallel Link changes . . . . . . . . . 8 76 4. Convergence Time and Loss of Connectivity Period . . . . . . . 9 77 4.1. Convergence Events without instant traffic loss . . . . . 10 78 4.2. Loss of Connectivity . . . . . . . . . . . . . . . . . . . 12 79 5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 13 80 5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 13 81 5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 13 82 5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 13 83 5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 14 84 5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 14 85 5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 14 86 5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 15 87 5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 15 88 5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 15 89 6. Selection of Convergence Time Benchmark Metrics and Methods . 16 90 6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 16 91 6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 16 92 6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 17 93 6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 17 94 6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 17 95 6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 17 96 6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 17 97 6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 17 98 6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 18 99 6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 18 100 6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 18 101 6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 18 102 7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 18 103 8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 20 104 8.1. Interface failures . . . . . . . . . . . . . . . . . . . . 21 105 8.1.1. Convergence Due to Local Interface Failure . . . . . . 21 106 8.1.2. Convergence Due to Remote Interface Failure . . . . . 22 107 8.1.3. Convergence Due to ECMP Member Local Interface 108 Failure . . . . . . . . . . . . . . . . . . . . . . . 23 109 8.1.4. Convergence Due to ECMP Member Remote Interface 110 Failure . . . . . . . . . . . . . . . . . . . . . . . 25 111 8.1.5. Convergence Due to Parallel Link Interface Failure . . 26 112 8.2. Other failures . . . . . . . . . . . . . . . . . . . . . . 27 113 8.2.1. Convergence Due to Layer 2 Session Loss . . . . . . . 27 114 8.2.2. Convergence Due to Loss of IGP Adjacency . . . . . . . 28 115 8.2.3. Convergence Due to Route Withdrawal . . . . . . . . . 30 116 8.3. Administrative changes . . . . . . . . . . . . . . . . . . 31 117 8.3.1. Convergence Due to Local Adminstrative Shutdown . . . 31 118 8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 32 119 9. Security Considerations . . . . . . . . . . . . . . . . . . . 34 120 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 121 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 34 122 12. Normative References . . . . . . . . . . . . . . . . . . . . . 35 123 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 125 1. Introduction and Scope 127 This document describes the methodology for benchmarking Link-State 128 Interior Gateway Protocol (IGP) convergence. The motivation and 129 applicability for this benchmarking is described in [Po09a]. The 130 terminology to be used for this benchmarking is described in [Po09t]. 132 IGP convergence time is measured on the data plane at the Tester by 133 observing packet loss through the DUT. All factors contributing to 134 convergence time are accounted for by measuring on the data plane, as 135 discussed in [Po09a]. The test cases in this document are black-box 136 tests that emulate the network events that cause convergence, as 137 described in [Po09a]. 139 The methodology described in this document can be applied to IPv4 and 140 IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF 141 [Mo98][Co08], and others. 143 2. Existing Definitions 145 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 146 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 147 document are to be interpreted as described in BCP 14, RFC 2119 148 [Br97]. RFC 2119 defines the use of these key words to help make the 149 intent of standards track documents as clear as possible. While this 150 document uses these keywords, this document is not a standards track 151 document. 153 This document uses much of the terminology defined in [Po09t] and 154 uses existing terminology defined in other BMWG work. Examples 155 include, but are not limited to: 157 Throughput [Ref.[Br91], section 3.17] 158 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 159 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 160 Out-of-order Packet [Ref.[Po06], section 3.3.2] 161 Duplicate Packet [Ref.[Po06], section 3.3.3] 162 Stream [Ref.[Po06], section 3.3.2] 163 Loss Period [Ref.[Ko02], section 4] 165 3. Test Topologies 167 3.1. Test topology for local changes 169 Figure 1 shows the test topology to measure IGP convergence time due 170 to local Convergence Events such as Local Interface failure 171 (Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP 172 adjacency failure (Section 8.2.2). This topology is also used to 173 measure IGP convergence time due to the route withdrawal 174 (Section 8.2.3), and route cost change (Section 8.3.2) Convergence 175 Events. IGP adjancencies MUST be established between Tester and DUT, 176 one on the Preferred Egress Interface and one on the Next-Best Egress 177 Interface. For this purpose the Tester emulates two routers, each 178 establishing one adjacency with the DUT. An IGP adjacency SHOULD be 179 established on the Ingress Interface between Tester and DUT. 181 --------- Ingress Interface ---------- 182 | |<--------------------------------| | 183 | | | | 184 | | Preferred Egress Interface | | 185 | DUT |-------------------------------->| Tester | 186 | | | | 187 | |-------------------------------->| | 188 | | Next-Best Egress Interface | | 189 --------- ---------- 191 Figure 1: IGP convergence test topology for local changes 193 3.2. Test topology for remote changes 195 Figure 2 shows the test topology to measure IGP convergence time due 196 to Remote Interface failure (Section 8.1.2). In this topology the 197 two routers R1 and R2 are considered System Under Test (SUT) and 198 SHOULD be identically configured devices of the same model. IGP 199 adjancencies MUST be established between Tester and SUT, one on the 200 Preferred Egress Interface and one on the Next-Best Egress Interface. 201 For this purpose the Tester emulates one or two routers. An IGP 202 adjacency SHOULD be established on the Ingress Interface between 203 Tester and SUT. In this topology there is a possibility of a 204 transient microloop between R1 and R2 during convergence. 206 ------ ---------- 207 | | Preferred | | 208 ------ | R2 |--------------------->| | 209 | |-->| | Egress Interface | | 210 | | ------ | | 211 | R1 | | Tester | 212 | | Next-Best | | 213 | |------------------------------>| | 214 ------ Egress Interface | | 215 ^ ---------- 216 | | 217 --------------------------------------- 218 Ingress Interface 220 Figure 2: IGP convergence test topology for remote changes 222 3.3. Test topology for local ECMP changes 224 Figure 3 shows the test topology to measure IGP convergence time due 225 to local Convergence Events with members of an Equal Cost Multipath 226 (ECMP) set (Section 8.1.3). In this topology, the DUT is configured 227 with each egress interface as a member of a single ECMP set and the 228 Tester emulates N next-hop routers, one router for each member. IGP 229 adjancencies MUST be established between Tester and DUT, one on each 230 member of the ECMP set. For this purpose each of the N routers 231 emulated by the Tester establishes one adjacency with the DUT. An 232 IGP adjacency SHOULD be established on the Ingress Interface between 233 Tester and DUT. 235 --------- Ingress Interface ---------- 236 | |<--------------------------------| | 237 | | | | 238 | | ECMP set interface 1 | | 239 | |-------------------------------->| | 240 | DUT | . | Tester | 241 | | . | | 242 | | . | | 243 | |-------------------------------->| | 244 | | ECMP set interface N | | 245 --------- ---------- 247 Figure 3: IGP convergence test topology for local ECMP change 249 3.4. Test topology for remote ECMP changes 251 Figure 4 shows the test topology to measure IGP convergence time due 252 to remote Convergence Events with members of an Equal Cost Multipath 253 (ECMP) set (Section 8.1.4). In this topology the two routers R1 and 254 R2 are considered System Under Test (SUT) and MUST be identically 255 configured devices of the same model. Router R1 is configured with 256 each egress interface as a member of a single ECMP set and the Tester 257 emulates N next-hop routers, one router for each member. IGP 258 adjancencies MUST be established between Tester and SUT, one on each 259 egress interface of SUT. For this purpose each of the N routers 260 emulated by the Tester establishes one adjacency with the SUT. An 261 IGP adjacency SHOULD be established on the Ingress Interface between 262 Tester and SUT. In this topology there is a possibility of a 263 transient microloop between R1 and R2 during convergence. 265 ------ ---------- 266 | | | | 267 ------ ECMP set | R2 |---->| | 268 | |------------------->| | | | 269 | | Interface 1 ------ | | 270 | | | | 271 | | ECMP set interface 2 | | 272 | R1 |------------------------------>| Tester | 273 | | . | | 274 | | . | | 275 | | . | | 276 | |------------------------------>| | 277 ------ ECMP set interface N | | 278 ^ ---------- 279 | | 280 --------------------------------------- 281 Ingress Interface 283 Figure 4: IGP convergence test topology for remote ECMP convergence 285 3.5. Test topology for Parallel Link changes 287 Figure 5 shows the test topology to measure IGP convergence time due 288 to local Convergence Events with members of a Parallel Link 289 (Section 8.1.5). In this topology, the DUT is configured with each 290 egress interface as a member of a Parallel Link and the Tester 291 emulates the single next-hop router. IGP adjancencies MUST be 292 established on all N members of the Parallel Link between Tester and 293 DUT. For this purpose the router emulated by the Tester establishes 294 N adjacencies with the DUT. An IGP adjacency SHOULD be established 295 on the Ingress Interface between Tester and DUT. 297 --------- Ingress Interface ---------- 298 | |<--------------------------------| | 299 | | | | 300 | | Parallel Link Interface 1 | | 301 | |-------------------------------->| | 302 | DUT | . | Tester | 303 | | . | | 304 | | . | | 305 | |-------------------------------->| | 306 | | Parallel Link Interface N | | 307 --------- ---------- 309 Figure 5: IGP convergence test topology for Parallel Link changes 311 4. Convergence Time and Loss of Connectivity Period 313 Two concepts will be highlighted in this section: convergence time 314 and loss of connectivity period. 316 The Route Convergence [Po09t] time indicates the period in time 317 between the Convergence Event Instant [Po09t] and the instant in time 318 the DUT is ready to forward traffic for a specific route on its Next- 319 Best Egress Interface and maintains this state for the duration of 320 the Sustained Convergence Validation Time [Po09t]. To measure Route 321 Convergence time, the Convergence Event Instant and the traffic 322 received from the Next-Best Egress Interface need to be observed. 324 The Route Loss of Connectivity Period [Po09t] indicates the time 325 during which traffic to a specific route is lost following a 326 Convergence Event until Full Convergence [Po09t] completes. This 327 Route Loss of Connectivity Period can consist of one or more Loss 328 Periods [Ko02]. For the testcases described in this document it is 329 expected to have a single Loss Period. To measure Route Loss of 330 Connectivity Period, the traffic received from the Preferred Egress 331 Interface and the traffic received from the Next-Best Egress 332 Interface need to be observed. 334 The Route Loss of Connectivity Period is most important since that 335 has a direct impact on the network user's application performance. 337 In general the Route Convergence time is larger than or equal to the 338 Route Loss of Connectivity Period. Depending on which Convergence 339 Event occurs and how this Convergence Event is applied, traffic for a 340 route may still be forwarded over the Preferred Egress Interface 341 after the Convergence Event Instant, before converging to the Next- 342 Best Egress Interface. In that case the Route Loss of Connectivity 343 Period is shorter than the Route Convergence time. 345 At least one condition needs to be fulfilled for Route Convergence 346 time to be equal to Route Loss of Connectivity Period. The condition 347 is that the Convergence Event causes an instantaneous traffic loss 348 for the measured route. A fiber cut on the Preferred Egress 349 Interface is an example of such a Convergence Event. 351 A second condition applies to Route Convergence time measurements 352 based on Connectivity Packet Loss [Po09t]. This second condition is 353 that there is only a single Loss Period during Route Convergence. 354 For the testcases described in this document this is expected to be 355 the case. 357 4.1. Convergence Events without instant traffic loss 359 To measure convergence time benchmarks for Convergence Events caused 360 by a Tester, such as an IGP cost change, the Tester MAY start to 361 discard all traffic received from the Preferred Egress Interface at 362 the Convergence Event Instant, or MAY separately observe packets 363 received from the Preferred Egress Interface prior to the Convergence 364 Event Instant. This way these Convergence Events can be treated the 365 same as Convergence Events that cause instantaneous traffic loss. 367 To measure convergence time benchmarks without instantaneous traffic 368 loss (either real or induced by the Tester) at the Convergence Event 369 Instant, such as a reversion of a link failure Convergence Event, the 370 Tester SHALL only observe packet statistics on the Next-Best Egress 371 Interface. If using the Rate-Derived method to benchmark convergence 372 times for such Convergence Events, the Tester MUST collect a 373 timestamp at the Convergence Event Instant. If using a loss-derived 374 method to benchmark convergence times for such Convergence Events, 375 the Tester MUST measure the period in time between the Start Traffic 376 Instant and the Convergence Event Instant. To measure this period in 377 time the Tester can collect timestamps at the Start Traffic Instant 378 and the Convergence Event Instant. 380 The Convergence Event Instant together with the receive rate 381 observations on the Next-Best Egress Interface allow to derive the 382 convergence time benchmarks using the Rate-Derived Method [Po09t]. 384 By observing lost packets on the Next-Best Egress Interface only, the 385 observed packet loss is the number of lost packets between Traffic 386 Start Instant and Convergence Recovery Instant. To measure 387 convergence times using a loss-derived method, packet loss between 388 the Convergence Event Instant and the Convergence Recovery Instant is 389 needed. The time between Traffic Start Instant and Convergence Event 390 Instant must be accounted for. An example may clarify this. 392 Figure 6 illustrates a Convergence Event without instantaneous 393 traffic loss for all routes. The top graph shows the Forwarding Rate 394 over all routes, the bottom graph shows the Forwarding Rate for a 395 single route Rta. Some time after the Convergence Event Instant, 396 Forwarding Rate observed on the Preferred Egress Interface starts to 397 decrease. In the example, route Rta is the first route to experience 398 packet loss at time Ta. Some time later, the Forwarding Rate 399 observed on the Next-Best Egress Interface starts to increase. In 400 the example, route Rta is the first route to complete convergence at 401 time Ta'. 403 ^ 404 Fwd | 405 Rate |------------- ............ 406 | \ . 407 | \ . 408 | \ . 409 | \ . 410 |.................-.-.-.-.-.-.---------------- 411 +----+-------+---------------+-----------------> 412 ^ ^ ^ ^ time 413 T0 CEI Ta Ta' 415 ^ 416 Fwd | 417 Rate |------------- ................. 418 Rta | | . 419 | | . 420 |.............-.-.-.-.-.-.-.-.---------------- 421 +----+-------+---------------+-----------------> 422 ^ ^ ^ ^ time 423 T0 CEI Ta Ta' 425 Preferred Egress Interface: --- 426 Next-Best Egress Interface: ... 428 With T0 the Start Traffic Instant; CEI the Convergence Event Instant; 429 Ta the time instant traffic loss for route Rta starts; Ta' the time 430 instant traffic loss for route Rta ends. 432 Figure 6 434 If only packets received on the Next-Best Egress Interface are 435 observed, the duration of the packet loss period for route Rta can be 436 calculated from the received packets as in Equation 1. Since the 437 Convergence Event Instant is the start time for convergence time 438 measurement, the period in time between T0 and CEI needs to be 439 subtracted from the calculated result to become the convergence time, 440 as in Equation 2. 442 Next-Best Egress Interface packet loss period 443 = (packets transmitted 444 - packets received from Next-Best Egress Interface) / tx rate 445 = Ta' - T0 447 Equation 1 449 convergence time 450 = Next-Best Egress Interface packet loss period - (CEI - T0) 451 = Ta' - CEI 453 Equation 2 455 4.2. Loss of Connectivity 457 Route Loss of Connectivity Period SHOULD be measured using the Route- 458 Specific Loss-Derived Method. Since the start instant and end 459 instant of the Route Loss of Connectivity Period can be different for 460 each route, these can not be accurately derived by only observing 461 global statistics over all routes. An example may clarify this. 463 Following a Convergence Event, route Rta is the first route for which 464 packet loss starts, the Route Loss of Connectivity Period for route 465 Rta starts at time Ta. Route Rtb is the last route for which packet 466 loss starts, the Route Loss of Connectivity Period for route Rtb 467 starts at time Tb with Tb>Ta. 469 ^ 470 Fwd | 471 Rate |-------- ----------- 472 | \ / 473 | \ / 474 | \ / 475 | \ / 476 | --------------- 477 +------------------------------------------> 478 ^ ^ ^ ^ time 479 Ta Tb Ta' Tb' 480 Tb'' Ta'' 482 Figure 7: Example Route Loss Of Connectivity Period 484 If the DUT implementation would be such that Route Rta would be the 485 first route for which traffic loss ends at time Ta' with Ta'>Tb. 486 Route Rtb would be the last route for which traffic loss ends at time 487 Tb' with Tb'>Ta'. By using only observing global traffic statistics 488 over all routes, the minimum Route Loss of Connectivity Period would 489 be measured as Ta'-Ta. The maximum calculated Route Loss of 490 Connectivity Period would be Tb'-Ta. The real minimum and maximum 491 Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb. 492 Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5, 493 would give a LoC Period between 3 and 5 derived from the global 494 traffic statistics, versus the real LoC Period between 3 and 4. 496 If the DUT implementation would be such that route Rtb would be the 497 first for which packet loss ends at time Tb'' and route Rta would be 498 the last for which packet loss ends at time Ta'', then the minimum 499 and maximum Route Loss of Connectivity Periods derived by observing 500 only global traffic statistics would be Tb''-Ta, and Ta''-Ta. The 501 real minimum and maximum Route Loss of Connectivity Periods are 502 Tb''-Tb and Ta''-Ta. Illustrating this with the numbers Ta=0, Tb=1, 503 Ta''=5, Tb''=3, would give a LoC Period between 3 and 5 derived from 504 the global traffic statistics, versus the real LoC Period between 2 505 and 5. 507 The two implementation variations in the above example would result 508 in the same derived minimum and maximum Route Loss of Connectivity 509 Periods when only observing the global packet statistics, while the 510 real Route Loss of Connectivity Periods are different. 512 5. Test Considerations 514 5.1. IGP Selection 516 The test cases described in Section 8 MAY be used for link-state 517 IGPs, such as ISIS or OSPF. The IGP convergence time test 518 methodology is identical. 520 5.2. Routing Protocol Configuration 522 The obtained results for IGP convergence time may vary if other 523 routing protocols are enabled and routes learned via those protocols 524 are installed. IGP convergence times SHOULD be benchmarked without 525 routes installed from other protocols. 527 5.3. IGP Topology 529 The Tester emulates a single IGP topology. The DUT establishes IGP 530 adjacencies with one or more of the emulated routers in this single 531 IGP topology emulated by the Tester. See test topology details in 532 Section 3. The emulated topology SHOULD only be advertised on the 533 DUT egress interfaces. 535 The number of IGP routes will impact the measured IGP route 536 convergence time. To obtain results similar to those that would be 537 observed in an operational network, it is RECOMMENDED that the number 538 of installed routes and nodes closely approximate that of the network 539 (e.g. thousands of routes with tens or hundreds of nodes). 541 The number of areas (for OSPF) and levels (for ISIS) can impact the 542 benchmark results. 544 5.4. Timers 546 There are timers that may impact the measured IGP convergence times. 547 The benchmark metrics MAY be measured at any fixed values for these 548 timers. To obtain results similar to those that would be observed in 549 an operational network, it is RECOMMENDED to configure the timers 550 with the values as configured in the operational network. 552 Examples of timers that may impact measured IGP convergence time 553 include, but are not limited to: 555 Interface failure indication 557 IGP hello timer 559 IGP dead-interval or hold-timer 561 LSA or LSP generation delay 563 LSA or LSP flood packet pacing 565 SPF delay 567 5.5. Interface Types 569 All test cases in this methodology document MAY be executed with any 570 interface type. The type of media may dictate which test cases may 571 be executed. Each interface type has a unique mechanism for 572 detecting link failures and the speed at which that mechanism 573 operates will influence the measurement results. All interfaces MUST 574 be the same media and Throughput [Br91][Br99] for each test case. 575 All interfaces SHOULD be configured as point-to-point. 577 5.6. Offered Load 579 The Throughput of the device, as defined in [Br91] and benchmarked in 580 [Br99] at a fixed packet size, needs to be determined over the 581 preferred path and over the next-best path. The Offered Load SHOULD 582 be the minimum of the measured Throughput of the device over the 583 primary path and over the backup path. The packet size is selectable 584 and MUST be recorded. Packet size is measured in bytes and includes 585 the IP header and payload. 587 The destination addresses for the Offered Load MUST be distributed 588 such that all routes or a statistically representative subset of all 589 routes are matched and each of these routes is offered an equal share 590 of the Offered Load. It is RECOMMENDED to send traffic matching all 591 routes, but a statistically representative subset of all routes can 592 be used if required. 594 In the Remote Interface failure testcases using topologies 2 and 4 595 there is a possibility of a transient microloop between R1 and R2 596 during convergence. The TTL or Hop Limit value of the packets sent 597 by the Tester may influence the benchmark measurements since it 598 determines which device in the topology may send an ICMP Time 599 Exceeded Message for looped packets. 601 The duration of the Offered Load MUST be greater than the convergence 602 time. 604 5.7. Measurement Accuracy 606 Since packet loss is observed to measure the Route Convergence Time, 607 the time between two successive packets offered to each individual 608 route is the highest possible accuracy of any packet loss based 609 measurement. When packet jitter is much less than the convergence 610 time, it is a negligible source of error and therefore it will be 611 ignored here. 613 5.8. Measurement Statistics 615 The benchmark measurements may vary for each trial, due to the 616 statistical nature of timer expirations, cpu scheduling, etc. 617 Evaluation of the test data must be done with an understanding of 618 generally accepted testing practices regarding repeatability, 619 variance and statistical significance of a small number of trials. 621 5.9. Tester Capabilities 623 It is RECOMMENDED that the Tester used to execute each test case has 624 the following capabilities: 626 1. Ability to establish IGP adjacencies and advertise a single IGP 627 topology to one or more peers. 629 2. Ability to insert a timestamp in each data packet's IP payload. 631 3. An internal time clock to control timestamping, time 632 measurements, and time calculations. 634 4. Ability to distinguish traffic load received on the Preferred and 635 Next-Best Interfaces [Po09t]. 637 5. Ability to disable or tune specific Layer-2 and Layer-3 protocol 638 functions on any interface(s). 640 The Tester MAY be capable to make non-data plane convergence 641 observations and use those observations for measurements. The Tester 642 MAY be capable to send and receive multiple traffic Streams [Po06]. 644 Also see Section 6 for method-specific capabilities. 646 6. Selection of Convergence Time Benchmark Metrics and Methods 648 Different convergence time benchmark methods MAY be used to measure 649 convergence time benchmark metrics. The Tester capabilities are 650 important criteria to select a specific convergence time benchmark 651 method. The criteria to select a specific benchmark method include, 652 but are not limited to: 654 Tester capabilities: Sampling Interval, number of 655 Stream statistics to collect 656 Measurement accuracy: Sampling Interval, Offered Load 657 Test specification: number of routes 658 DUT capabilities: Throughput 660 6.1. Loss-Derived Method 662 6.1.1. Tester capabilities 664 The Offered Load SHOULD consist of a single Stream [Po06]. If 665 sending multiple Streams, the measured packet loss statistics for all 666 Streams MUST be added together. 668 In order to verify Full Convergence completion and the Sustained 669 Convergence Validation Time, the Tester MUST measure Forwarding Rate 670 each Packet Sampling Interval. 672 The total number of packets lost between the start of the traffic and 673 the end of the Sustained Convergence Validation Time is used to 674 calculate the Loss-Derived Convergence Time. 676 6.1.2. Benchmark Metrics 678 The Loss-Derived Method can be used to measure the Loss-Derived 679 Convergence Time, which is the average convergence time over all 680 routes, and to measure the Loss-Derived Loss of Connectivity Period, 681 which is the average Route Loss of Connectivity Period over all 682 routes. 684 6.1.3. Measurement Accuracy 686 The measurement accuracy of the Loss-Derived Method is equal to the 687 time between two consecutive packets to the same route. 689 6.2. Rate-Derived Method 691 6.2.1. Tester Capabilities 693 The Offered Load SHOULD consist of a single Stream. If sending 694 multiple Streams, the measured traffic rate statistics for all 695 Streams MUST be added together. 697 The Tester measures Forwarding Rate each Sampling Interval. The 698 Packet Sampling Interval influences the observation of the different 699 convergence time instants. If the Packet Sampling Interval is large 700 compared to the time between the convergence time instants, then the 701 different time instants may not be easily identifiable from the 702 Forwarding Rate observation. The requirements for the Packet 703 Sampling Interval are specified in [Po09t]. The RECOMMENDED value 704 for the Packet Sampling Interval is 10 milliseconds. The Packet 705 Sampling Interval MUST be reported. 707 6.2.2. Benchmark Metrics 709 The Rate-Derived Method SHOULD be used to measure First Route 710 Convergence Time and Full Convergence Time. It SHOULD NOT be used to 711 measure Loss of Connectivity Period (see Section 4). 713 6.2.3. Measurement Accuracy 715 The measurement accuracy of the Rate-Derived Method for transitions 716 that occur for all routes at the same instant is equal to the Packet 717 Sampling Interval and for other transitions the measurement accuracy 718 is equal to the Packet Sampling Interval plus the time between two 719 consecutive packets to the same destination. The latter is the case 720 since packets are sent in a particular order to all destinations in a 721 stream and when part of the routes experience packet loss, it is 722 unknown where in the transmit cycle packets to these routes are sent. 723 This uncertainty adds to the error. 725 6.3. Route-Specific Loss-Derived Method 727 6.3.1. Tester Capabilities 729 The Offered Load consists of multiple Streams. The Tester MUST 730 measure packet loss for each Stream separately. 732 In order to verify Full Convergence completion and the Sustained 733 Convergence Validation Time, the Tester MUST measure packet loss each 734 Packet Sampling Interval. This measurement at each Packet Sampling 735 Interval MAY be per Stream. 737 Only the total packet loss measured per Stream at the end of the 738 Sustained Convergence Validation Time is used to calculate the 739 benchmark metrics with this method. 741 6.3.2. Benchmark Metrics 743 The Route-Specific Loss-Derived Method SHOULD be used to measure 744 Route-Specific Convergence Times. It is the RECOMMENDED method to 745 measure Route Loss of Connectivity Period. 747 Under the conditions explained in Section 4, First Route Convergence 748 Time and Full Convergence Time as benchmarked using Rate-Derived 749 Method, may be equal to the minimum resp. maximum of the Route- 750 Specific Convergence Times. 752 6.3.3. Measurement Accuracy 754 The measurement accuracy of the Route-Specific Loss-Derived Method is 755 equal to the time between two consecutive packets to the same route. 757 7. Reporting Format 759 For each test case, it is recommended that the reporting tables below 760 are completed and all time values SHOULD be reported with resolution 761 as specified in [Po09t]. 763 Parameter Units 764 ----------------------------------- ----------------------- 765 Test Case test case number 766 Test Topology (1, 2, 3, 4, or 5) 767 IGP (ISIS, OSPF, other) 768 Interface Type (GigE, POS, ATM, other) 769 Packet Size offered to DUT bytes 770 Offered Load packets per second 771 IGP Routes advertised to DUT number of IGP routes 772 Nodes in emulated network number of nodes 773 Number of Routes measured number of routes 774 Packet Sampling Interval on Tester seconds 775 Forwarding Delay Threshold seconds 777 Timer Values configured on DUT: 778 Interface failure indication delay seconds 779 IGP Hello Timer seconds 780 IGP Dead-Interval or hold-time seconds 781 LSA Generation Delay seconds 782 LSA Flood Packet Pacing seconds 783 LSA Retransmission Packet Pacing seconds 784 SPF Delay seconds 786 Test Details: 788 If the Offered Load matches a subset of routes, describe how this 789 subset is selected. 791 Describe how the Convergence Event is applied; does it cause 792 instantaneous traffic loss or not. 794 Complete the table below for the initial Convergence Event and the 795 reversion Convergence Event. 797 Parameter Units 798 ------------------------------------------ ---------------------- 799 Conversion Event (initial or reversion) 801 Traffic Forwarding Metrics: 802 Total number of packets offered to DUT number of Packets 803 Total number of packets forwarded by DUT number of Packets 804 Connectivity Packet Loss number of Packets 805 Convergence Packet Loss number of Packets 806 Out-of-Order Packets number of Packets 807 Duplicate Packets number of Packets 809 Convergence Benchmarks: 810 Rate-Derived Method: 811 First Route Convergence Time seconds 812 Full Convergence Time seconds 813 Loss-Derived Method: 814 Loss-Derived Convergence Time seconds 815 Route-Specific Loss-Derived Method: 816 Route-Specific Convergence Time[n] array of seconds 817 Minimum R-S Convergence Time seconds 818 Maximum R-S Convergence Time seconds 819 Median R-S Convergence Time seconds 820 Average R-S Convergence Time seconds 822 Loss of Connectivity Benchmarks: 823 Loss-Derived Method: 824 Loss-Derived Loss of Connectivity Period seconds 825 Route-Specific Loss-Derived Method: 826 Route LoC Period[n] array of seconds 827 Minimum Route LoC Period seconds 828 Maximum Route LoC Period seconds 829 Median Route LoC Period seconds 830 Average Route LoC Period seconds 832 8. Test Cases 834 It is RECOMMENDED that all applicable test cases be performed for 835 best characterization of the DUT. The test cases follow a generic 836 procedure tailored to the specific DUT configuration and Convergence 837 Event [Po09t]. This generic procedure is as follows: 839 1. Establish DUT and Tester configurations and advertise an IGP 840 topology from Tester to DUT. 842 2. Send Offered Load from Tester to DUT on ingress interface. 844 3. Verify traffic is routed correctly. 846 4. Introduce Convergence Event [Po09t]. 848 5. Measure First Route Convergence Time [Po09t]. 850 6. Measure Full Convergence Time [Po09t]. 852 7. Stop Offered Load. 854 8. Measure Route-Specific Convergence Times, Loss-Derived 855 Convergence Time, Route LoC Periods, and Loss-Derived LoC Period 856 [Po09t]. 858 9. Wait sufficient time for queues to drain. 860 10. Restart Offered Load. 862 11. Reverse Convergence Event. 864 12. Measure First Route Convergence Time. 866 13. Measure Full Convergence Time. 868 14. Stop Offered Load. 870 15. Measure Route-Specific Convergence Times, Loss-Derived 871 Convergence Time, Route LoC Periods, and Loss-Derived LoC 872 Period. 874 8.1. Interface failures 876 8.1.1. Convergence Due to Local Interface Failure 878 Objective 880 To obtain the IGP convergence times due to a Local Interface failure 881 event. 883 Procedure 885 1. Advertise an IGP topology from Tester to DUT using the topology 886 shown in Figure 1. 888 2. Send Offered Load from Tester to DUT on ingress interface. 890 3. Verify traffic is forwarded over Preferred Egress Interface. 892 4. Remove link on DUT's Preferred Egress Interface. This is the 893 Convergence Event. 895 5. Measure First Route Convergence Time. 897 6. Measure Full Convergence Time. 899 7. Stop Offered Load. 901 8. Measure Route-Specific Convergence Times and Loss-Derived 902 Convergence Time. 904 9. Wait sufficient time for queues to drain. 906 10. Restart Offered Load. 908 11. Restore link on DUT's Preferred Egress Interface. 910 12. Measure First Route Convergence Time. 912 13. Measure Full Convergence Time. 914 14. Stop Offered Load. 916 15. Measure Route-Specific Convergence Times, Loss-Derived 917 Convergence Time, Route LoC Periods, and Loss-Derived LoC 918 Period. 920 Results 922 The measured IGP convergence time may be influenced by the link 923 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 924 LSP flood packet pacing, SPF delay, SPF execution time, and routing 925 and forwarding tables update time [Po09a]. 927 8.1.2. Convergence Due to Remote Interface Failure 929 Objective 931 To obtain the IGP convergence time due to a Remote Interface failure 932 event. 934 Procedure 936 1. Advertise an IGP topology from Tester to SUT using the topology 937 shown in Figure 2. 939 2. Send Offered Load from Tester to SUT on ingress interface. 941 3. Verify traffic is forwarded over Preferred Egress Interface. 943 4. Remove link on Tester's interface [Po09t] connected to SUT's 944 Preferred Egress Interface. This is the Convergence Event. 946 5. Measure First Route Convergence Time. 948 6. Measure Full Convergence Time. 950 7. Stop Offered Load. 952 8. Measure Route-Specific Convergence Times and Loss-Derived 953 Convergence Time. 955 9. Wait sufficient time for queues to drain. 957 10. Restart Offered Load. 959 11. Restore link on Tester's interface connected to DUT's Preferred 960 Egress Interface. 962 12. Measure First Route Convergence Time. 964 13. Measure Full Convergence Time. 966 14. Stop Offered Load. 968 15. Measure Route-Specific Convergence Times, Loss-Derived 969 Convergence Time, Route LoC Periods, and Loss-Derived LoC 970 Period. 972 Results 974 The measured IGP convergence time may be influenced by the link 975 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 976 LSP flood packet pacing, SPF delay, SPF execution time, and routing 977 and forwarding tables update time. This test case may produce Stale 978 Forwarding [Po09t] due to a transient microloop between R1 and R2 979 during convergence, which may increase the measured convergence times 980 and loss of connectivity periods. 982 8.1.3. Convergence Due to ECMP Member Local Interface Failure 984 Objective 986 To obtain the IGP convergence time due to a Local Interface link 987 failure event of an ECMP Member. 989 Procedure 991 1. Advertise an IGP topology from Tester to DUT using the test 992 setup shown in Figure 3. 994 2. Send Offered Load from Tester to DUT on ingress interface. 996 3. Verify traffic is forwarded over the DUT's ECMP member interface 997 that will be failed in the next step. 999 4. Remove link on one of the DUT's ECMP member interfaces. This is 1000 the Convergence Event. 1002 5. Measure First Route Convergence Time. 1004 6. Measure Full Convergence Time. 1006 7. Stop Offered Load. 1008 8. Measure Route-Specific Convergence Times and Loss-Derived 1009 Convergence Time. At the same time measure Out-of-Order Packets 1010 [Po06] and Duplicate Packets [Po06]. 1012 9. Wait sufficient time for queues to drain. 1014 10. Restart Offered Load. 1016 11. Restore link on DUT's ECMP member interface. 1018 12. Measure First Route Convergence Time. 1020 13. Measure Full Convergence Time. 1022 14. Stop Offered Load. 1024 15. Measure Route-Specific Convergence Times, Loss-Derived 1025 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1026 Period. At the same time measure Out-of-Order Packets [Po06] 1027 and Duplicate Packets [Po06]. 1029 Results 1031 The measured IGP Convergence time may be influenced by link failure 1032 indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP 1033 flood packet pacing, SPF delay, SPF execution time, and routing and 1034 forwarding tables update time [Po09a]. 1036 8.1.4. Convergence Due to ECMP Member Remote Interface Failure 1038 Objective 1040 To obtain the IGP convergence time due to a Remote Interface link 1041 failure event for an ECMP Member. 1043 Procedure 1045 1. Advertise an IGP topology from Tester to DUT using the test 1046 setup shown in Figure 4. 1048 2. Send Offered Load from Tester to DUT on ingress interface. 1050 3. Verify traffic is forwarded over the DUT's ECMP member interface 1051 that will be failed in the next step. 1053 4. Remove link on Tester's interface to R2. This is the 1054 Convergence Event Trigger. 1056 5. Measure First Route Convergence Time. 1058 6. Measure Full Convergence Time. 1060 7. Stop Offered Load. 1062 8. Measure Route-Specific Convergence Times and Loss-Derived 1063 Convergence Time. At the same time measure Out-of-Order Packets 1064 [Po06] and Duplicate Packets [Po06]. 1066 9. Wait sufficient time for queues to drain. 1068 10. Restart Offered Load. 1070 11. Restore link on Tester's interface to R2. 1072 12. Measure First Route Convergence Time. 1074 13. Measure Full Convergence Time. 1076 14. Stop Offered Load. 1078 15. Measure Route-Specific Convergence Times, Loss-Derived 1079 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1080 Period. At the same time measure Out-of-Order Packets [Po06] 1081 and Duplicate Packets [Po06]. 1083 Results 1084 The measured IGP convergence time may influenced by the link failure 1085 indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP 1086 flood packet pacing, SPF delay, SPF execution time, and routing and 1087 forwarding tables update time. This test case may produce Stale 1088 Forwarding [Po09t] due to a transient microloop between R1 and R2 1089 during convergence, which may increase the measured convergence times 1090 and loss of connectivity periods. 1092 8.1.5. Convergence Due to Parallel Link Interface Failure 1094 Objective 1096 To obtain the IGP convergence due to a local link failure event for a 1097 member of a parallel link. The links can be used for data load 1098 balancing 1100 Procedure 1102 1. Advertise an IGP topology from Tester to DUT using the test 1103 setup shown in Figure 5. 1105 2. Send Offered Load from Tester to DUT on ingress interface. 1107 3. Verify traffic is forwarded over the parallel link member that 1108 will be failed in the next step. 1110 4. Remove link on one of the DUT's parallel link member interfaces. 1111 This is the Convergence Event. 1113 5. Measure First Route Convergence Time. 1115 6. Measure Full Convergence Time. 1117 7. Stop Offered Load. 1119 8. Measure Route-Specific Convergence Times and Loss-Derived 1120 Convergence Time. At the same time measure Out-of-Order Packets 1121 [Po06] and Duplicate Packets [Po06]. 1123 9. Wait sufficient time for queues to drain. 1125 10. Restart Offered Load. 1127 11. Restore link on DUT's Parallel Link member interface. 1129 12. Measure First Route Convergence Time. 1131 13. Measure Full Convergence Time. 1133 14. Stop Offered Load. 1135 15. Measure Route-Specific Convergence Times, Loss-Derived 1136 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1137 Period. At the same time measure Out-of-Order Packets [Po06] 1138 and Duplicate Packets [Po06]. 1140 Results 1142 The measured IGP convergence time may be influenced by the link 1143 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1144 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1145 and forwarding tables update time [Po09a]. 1147 8.2. Other failures 1149 8.2.1. Convergence Due to Layer 2 Session Loss 1151 Objective 1153 To obtain the IGP convergence time due to a local layer 2 loss. 1155 Procedure 1157 1. Advertise an IGP topology from Tester to DUT using the topology 1158 shown in Figure 1. 1160 2. Send Offered Load from Tester to DUT on ingress interface. 1162 3. Verify traffic is routed over Preferred Egress Interface. 1164 4. Remove Layer 2 session from DUT's Preferred Egress Interface. 1165 This is the Convergence Event. 1167 5. Measure First Route Convergence Time. 1169 6. Measure Full Convergence Time. 1171 7. Stop Offered Load. 1173 8. Measure Route-Specific Convergence Times, Loss-Derived 1174 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1175 Period. 1177 9. Wait sufficient time for queues to drain. 1179 10. Restart Offered Load. 1181 11. Restore Layer 2 session on DUT's Preferred Egress Interface. 1183 12. Measure First Route Convergence Time. 1185 13. Measure Full Convergence Time. 1187 14. Stop Offered Load. 1189 15. Measure Route-Specific Convergence Times, Loss-Derived 1190 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1191 Period. 1193 Results 1195 The measured IGP Convergence time may be influenced by the Layer 2 1196 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1197 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1198 and forwarding tables update time [Po09a]. 1200 Discussion 1202 Configure IGP timers such that the IGP adjacency does not time out 1203 before layer 2 failure is detected. 1205 To measure convergence time, traffic SHOULD start dropping on the 1206 Preferred Egress Interface on the instant the layer 2 session is 1207 removed. Alternatively the Tester SHOULD record the time the instant 1208 layer 2 session is removed and traffic loss SHOULD only be measured 1209 on the Next-Best Egress Interface. For loss-derived benchmarks the 1210 time of the Start Traffic Instant SHOULD be recorded as well. See 1211 Section 4.1. 1213 8.2.2. Convergence Due to Loss of IGP Adjacency 1215 Objective 1217 To obtain the IGP convergence time due to loss of an IGP Adjacency. 1219 Procedure 1221 1. Advertise an IGP topology from Tester to DUT using the topology 1222 shown in Figure 1. 1224 2. Send Offered Load from Tester to DUT on ingress interface. 1226 3. Verify traffic is routed over Preferred Egress Interface. 1228 4. Remove IGP adjacency from the Preferred Egress Interface while 1229 the layer 2 session MUST be maintained. This is the Convergence 1230 Event. 1232 5. Measure First Route Convergence Time. 1234 6. Measure Full Convergence Time. 1236 7. Stop Offered Load. 1238 8. Measure Route-Specific Convergence Times, Loss-Derived 1239 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1240 Period. 1242 9. Wait sufficient time for queues to drain. 1244 10. Restart Offered Load. 1246 11. Restore IGP session on DUT's Preferred Egress Interface. 1248 12. Measure First Route Convergence Time. 1250 13. Measure Full Convergence Time. 1252 14. Stop Offered Load. 1254 15. Measure Route-Specific Convergence Times, Loss-Derived 1255 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1256 Period. 1258 Results 1260 The measured IGP Convergence time may be influenced by the IGP Hello 1261 Interval, IGP Dead Interval, LSA/LSP delay, LSA/LSP generation time, 1262 LSA/LSP flood packet pacing, SPF delay, SPF execution time, and 1263 routing and forwarding tables update time [Po09a]. 1265 Discussion 1267 Configure layer 2 such that layer 2 does not time out before IGP 1268 adjacency failure is detected. 1270 To measure convergence time, traffic SHOULD start dropping on the 1271 Preferred Egress Interface on the instant the IGP adjacency is 1272 removed. Alternatively the Tester SHOULD record the time the instant 1273 the IGP adjacency is removed and traffic loss SHOULD only be measured 1274 on the Next-Best Egress Interface. For loss-derived benchmarks the 1275 time of the Start Traffic Instant SHOULD be recorded as well. See 1276 Section 4.1. 1278 8.2.3. Convergence Due to Route Withdrawal 1280 Objective 1282 To obtain the IGP convergence time due to route withdrawal. 1284 Procedure 1286 1. Advertise an IGP topology from Tester to DUT using the topology 1287 shown in Figure 1. The routes that will be withdrawn MUST be a 1288 set of leaf routes advertised by at least two nodes in the 1289 emulated topology. The topology SHOULD be such that before the 1290 withdrawal the DUT prefers the leaf routes advertised by a node 1291 "nodeA" via the Preferred Egress Interface, and after the 1292 withdrawal the DUT prefers the leaf routes advertised by a node 1293 "nodeB" via the Next-Best Egress Interface. 1295 2. Send Offered Load from Tester to DUT on Ingress Interface. 1297 3. Verify traffic is routed over Preferred Egress Interface. 1299 4. The Tester withdraws the set of IGP leaf routes from nodeA. 1300 This is the Convergence Event. The withdrawal update message 1301 SHOULD be a single unfragmented packet. If the routes cannot be 1302 withdrawn by a single packet, the messages SHOULD be sent using 1303 the same pacing characteristics as the DUT. The Tester MAY 1304 record the time it sends the withdrawal message(s). 1306 5. Measure First Route Convergence Time. 1308 6. Measure Full Convergence Time. 1310 7. Stop Offered Load. 1312 8. Measure Route-Specific Convergence Times, Loss-Derived 1313 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1314 Period. 1316 9. Wait sufficient time for queues to drain. 1318 10. Restart Offered Load. 1320 11. Re-advertise the set of withdrawn IGP leaf routes from nodeA 1321 emulated by the Tester. The update message SHOULD be a single 1322 unfragmented packet. If the routes cannot be advertised by a 1323 single packet, the messages SHOULD be sent using the same pacing 1324 characteristics as the DUT. The Tester MAY record the time it 1325 sends the update message(s). 1327 12. Measure First Route Convergence Time. 1329 13. Measure Full Convergence Time. 1331 14. Stop Offered Load. 1333 15. Measure Route-Specific Convergence Times, Loss-Derived 1334 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1335 Period. 1337 Results 1339 The measured IGP convergence time is influenced by SPF or route 1340 calculation delay, SPF or route calculation execution time, and 1341 routing and forwarding tables update time [Po09a]. 1343 Discussion 1345 To measure convergence time, traffic SHOULD start dropping on the 1346 Preferred Egress Interface on the instant the routes are withdrawn by 1347 the Tester. Alternatively the Tester SHOULD record the time the 1348 instant the routes are withdrawn and traffic loss SHOULD only be 1349 measured on the Next-Best Egress Interface. For loss-derived 1350 benchmarks the time of the Start Traffic Instant SHOULD be recorded 1351 as well. See Section 4.1. 1353 8.3. Administrative changes 1355 8.3.1. Convergence Due to Local Adminstrative Shutdown 1357 Objective 1359 To obtain the IGP convergence time due to taking the DUT's Local 1360 Interface administratively out of service. 1362 Procedure 1364 1. Advertise an IGP topology from Tester to DUT using the topology 1365 shown in Figure 1. 1367 2. Send Offered Load from Tester to DUT on ingress interface. 1369 3. Verify traffic is routed over Preferred Egress Interface. 1371 4. Take the DUT's Preferred Egress Interface administratively out 1372 of service. This is the Convergence Event. 1374 5. Measure First Route Convergence Time. 1376 6. Measure Full Convergence Time. 1378 7. Stop Offered Load. 1380 8. Measure Route-Specific Convergence Times, Loss-Derived 1381 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1382 Period. 1384 9. Wait sufficient time for queues to drain. 1386 10. Restart Offered Load. 1388 11. Restore Preferred Egress Interface by administratively enabling 1389 the interface. 1391 12. Measure First Route Convergence Time. 1393 13. Measure Full Convergence Time. 1395 14. Stop Offered Load. 1397 15. Measure Route-Specific Convergence Times, Loss-Derived 1398 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1399 Period. 1401 16. It is possible that no measured packet loss will be observed for 1402 this test case. 1404 Results 1406 The measured IGP Convergence time may be influenced by LSA/LSP delay, 1407 LSA/LSP generation time, LSA/LSP flood packet pacing, SPF delay, SPF 1408 execution time, and routing and forwarding tables update time 1409 [Po09a]. 1411 8.3.2. Convergence Due to Cost Change 1413 Objective 1415 To obtain the IGP convergence time due to route cost change. 1417 Procedure 1419 1. Advertise an IGP topology from Tester to DUT using the topology 1420 shown in Figure 1. 1422 2. Send Offered Load from Tester to DUT on ingress interface. 1424 3. Verify traffic is routed over Preferred Egress Interface. 1426 4. The Tester, emulating the neighbor node, increases the cost for 1427 all IGP routes at DUT's Preferred Egress Interface so that the 1428 Next-Best Egress Interface becomes preferred path. The update 1429 message advertising the higher cost MUST be a single 1430 unfragmented packet. This is the Convergence Event. The Tester 1431 MAY record the time it sends the update message advertising the 1432 higher cost on the Preferred Egress Interface. 1434 5. Measure First Route Convergence Time. 1436 6. Measure Full Convergence Time. 1438 7. Stop Offered Load. 1440 8. Measure Route-Specific Convergence Times, Loss-Derived 1441 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1442 Period. 1444 9. Wait sufficient time for queues to drain. 1446 10. Restart Offered Load. 1448 11. The Tester, emulating the neighbor node, decreases the cost for 1449 all IGP routes at DUT's Preferred Egress Interface so that the 1450 Preferred Egress Interface becomes preferred path. The update 1451 message advertising the lower cost MUST be a single unfragmented 1452 packet. 1454 12. Measure First Route Convergence Time. 1456 13. Measure Full Convergence Time. 1458 14. Stop Offered Load. 1460 15. Measure Route-Specific Convergence Times, Loss-Derived 1461 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1462 Period. 1464 Results 1465 The measured IGP Convergence time may be influenced by SPF delay, SPF 1466 execution time, and routing and forwarding tables update time 1467 [Po09a]. 1469 Discussion 1471 To measure convergence time, traffic SHOULD start dropping on the 1472 Preferred Egress Interface on the instant the cost is changed by the 1473 Tester. Alternatively the Tester SHOULD record the time the instant 1474 the cost is changed and traffic loss SHOULD only be measured on the 1475 Next-Best Egress Interface. For loss-derived benchmarks the time of 1476 the Start Traffic Instant SHOULD be recorded as well. See Section 1477 4.1. 1479 9. Security Considerations 1481 Benchmarking activities as described in this memo are limited to 1482 technology characterization using controlled stimuli in a laboratory 1483 environment, with dedicated address space and the constraints 1484 specified in the sections above. 1486 The benchmarking network topology will be an independent test setup 1487 and MUST NOT be connected to devices that may forward the test 1488 traffic into a production network, or misroute traffic to the test 1489 management network. 1491 Further, benchmarking is performed on a "black-box" basis, relying 1492 solely on measurements observable external to the DUT/SUT. 1494 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 1495 benchmarking purposes. Any implications for network security arising 1496 from the DUT/SUT SHOULD be identical in the lab and in production 1497 networks. 1499 10. IANA Considerations 1501 This document requires no IANA considerations. 1503 11. Acknowledgements 1505 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1506 Peter De Vriendt and the BMWG for their contributions to this work. 1508 12. Normative References 1510 [Br91] Bradner, S., "Benchmarking terminology for network 1511 interconnection devices", RFC 1242, July 1991. 1513 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1514 Requirement Levels", BCP 14, RFC 2119, March 1997. 1516 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1517 Network Interconnect Devices", RFC 2544, March 1999. 1519 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1520 environments", RFC 1195, December 1990. 1522 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1523 IPv6", RFC 5340, July 2008. 1525 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1526 October 2008. 1528 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1529 Metrics", RFC 3357, August 2002. 1531 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1532 Devices", RFC 2285, February 1998. 1534 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1536 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1537 "Terminology for Benchmarking Network-layer Traffic Control 1538 Mechanisms", RFC 4689, October 2006. 1540 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1541 IGP Data Plane Route Convergence", 1542 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1543 progress), March 2009. 1545 [Po09t] Poretsky, S. and B. Imhoff, "Terminology for Benchmarking 1546 Link-State IGP Data Plane Route Convergence", 1547 draft-ietf-bmwg-igp-dataplane-conv-term-18 (work in 1548 progress), July 2009. 1550 Authors' Addresses 1552 Scott Poretsky 1553 Allot Communications 1554 67 South Bedford Street, Suite 400 1555 Burlington, MA 01803 1556 USA 1558 Phone: + 1 508 309 2179 1559 Email: sporetsky@allot.com 1561 Brent Imhoff 1562 Juniper Networks 1563 1194 North Mathilda Ave 1564 Sunnyvale, CA 94089 1565 USA 1567 Phone: + 1 314 378 2571 1568 Email: bimhoff@planetspork.com 1570 Kris Michielsen 1571 Cisco Systems 1572 6A De Kleetlaan 1573 Diegem, BRABANT 1831 1574 Belgium 1576 Email: kmichiel@cisco.com