idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-meth-22.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 928 has weird spacing: '...P links num...' == Line 930 has weird spacing: '... Tester secon...' == Line 939 has weird spacing: '... Pacing seco...' -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 8, 2010) is 4917 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-term-20 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: May 12, 2011 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 November 8, 2010 10 Benchmarking Methodology for Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-meth-22 13 Abstract 15 This document describes the methodology for benchmarking Link-State 16 Interior Gateway Protocol (IGP) Route Convergence. The methodology 17 is to be used for benchmarking IGP convergence time through 18 externally observable (black box) data plane measurements. The 19 methodology can be applied to any link-state IGP, such as ISIS and 20 OSPF. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on May 12, 2011. 39 Copyright Notice 41 Copyright (c) 2010 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 This document may contain material from IETF Documents or IETF 55 Contributions published or made publicly available before November 56 10, 2008. The person(s) controlling the copyright in some of this 57 material may not have granted the IETF Trust the right to allow 58 modifications of such material outside the IETF Standards Process. 59 Without obtaining an adequate license from the person(s) controlling 60 the copyright in such materials, this document may not be modified 61 outside the IETF Standards Process, and derivative works of it may 62 not be created outside the IETF Standards Process, except to format 63 it for publication as an RFC or to translate it into languages other 64 than English. 66 Table of Contents 68 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 5 69 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 5 70 3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 5 71 3.1. Test topology for local changes . . . . . . . . . . . . . 6 72 3.2. Test topology for remote changes . . . . . . . . . . . . . 7 73 3.3. Test topology for local ECMP changes . . . . . . . . . . . 8 74 3.4. Test topology for remote ECMP changes . . . . . . . . . . 9 75 3.5. Test topology for Parallel Link changes . . . . . . . . . 10 76 4. Convergence Time and Loss of Connectivity Period . . . . . . . 11 77 4.1. Convergence Events without instant traffic loss . . . . . 12 78 4.2. Loss of Connectivity . . . . . . . . . . . . . . . . . . . 14 79 5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 15 80 5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 15 81 5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 15 82 5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 15 83 5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 16 84 5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 16 85 5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 16 86 5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 17 87 5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 17 88 5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 17 89 6. Selection of Convergence Time Benchmark Metrics and Methods . 18 90 6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 18 91 6.1.1. Tester capabilities . . . . . . . . . . . . . . . . . 18 92 6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 19 93 6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 19 94 6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 19 95 6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 19 96 6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 20 97 6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 20 98 6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 21 99 6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 21 100 6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 21 101 6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 22 102 7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 22 103 8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 23 104 8.1. Interface failures . . . . . . . . . . . . . . . . . . . . 24 105 8.1.1. Convergence Due to Local Interface Failure . . . . . . 24 106 8.1.2. Convergence Due to Remote Interface Failure . . . . . 25 107 8.1.3. Convergence Due to ECMP Member Local Interface 108 Failure . . . . . . . . . . . . . . . . . . . . . . . 27 109 8.1.4. Convergence Due to ECMP Member Remote Interface 110 Failure . . . . . . . . . . . . . . . . . . . . . . . 28 111 8.1.5. Convergence Due to Parallel Link Interface Failure . . 29 112 8.2. Other failures . . . . . . . . . . . . . . . . . . . . . . 30 113 8.2.1. Convergence Due to Layer 2 Session Loss . . . . . . . 30 114 8.2.2. Convergence Due to Loss of IGP Adjacency . . . . . . . 31 115 8.2.3. Convergence Due to Route Withdrawal . . . . . . . . . 33 116 8.3. Administrative changes . . . . . . . . . . . . . . . . . . 34 117 8.3.1. Convergence Due to Local Adminstrative Shutdown . . . 34 118 8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 36 119 9. Security Considerations . . . . . . . . . . . . . . . . . . . 37 120 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38 121 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 38 122 12. Normative References . . . . . . . . . . . . . . . . . . . . . 38 123 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 39 125 1. Introduction and Scope 127 This document describes the methodology for benchmarking Link-State 128 Interior Gateway Protocol (IGP) convergence. The motivation and 129 applicability for this benchmarking is described in [Po09a]. The 130 terminology to be used for this benchmarking is described in [Po10t]. 132 IGP convergence time is measured on the data plane at the Tester by 133 observing packet loss through the DUT. All factors contributing to 134 convergence time are accounted for by measuring on the data plane, as 135 discussed in [Po09a]. The test cases in this document are black-box 136 tests that emulate the network events that cause convergence, as 137 described in [Po09a]. 139 The methodology described in this document can be applied to IPv4 and 140 IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF 141 [Mo98][Co08], and others. 143 2. Existing Definitions 145 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 146 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 147 document are to be interpreted as described in BCP 14, RFC 2119 148 [Br97]. RFC 2119 defines the use of these key words to help make the 149 intent of standards track documents as clear as possible. While this 150 document uses these keywords, this document is not a standards track 151 document. 153 This document uses much of the terminology defined in [Po10t] and 154 uses existing terminology defined in other BMWG work. Examples 155 include, but are not limited to: 157 Throughput [Ref.[Br91], section 3.17] 158 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 159 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 160 Out-of-Order Packet [Ref.[Po06], section 3.3.4] 161 Duplicate Packet [Ref.[Po06], section 3.3.5] 162 Stream [Ref.[Po06], section 3.3.2] 163 Loss Period [Ref.[Ko02], section 4] 164 Forwarding Delay [Ref.[Po06], section 3.2.4] 165 IP Packet Delay Variation (IPDV) [Ref.[De02], section 1.2] 167 3. Test Topologies 168 3.1. Test topology for local changes 170 Figure 1 shows the test topology to measure IGP convergence time due 171 to local Convergence Events such as Local Interface failure 172 (Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP 173 adjacency failure (Section 8.2.2). This topology is also used to 174 measure IGP convergence time due to the route withdrawal 175 (Section 8.2.3), and route cost change (Section 8.3.2) Convergence 176 Events. IGP adjacencies MUST be established between Tester and DUT, 177 one on the Preferred Egress Interface and one on the Next-Best Egress 178 Interface. For this purpose the Tester emulates two routers, each 179 establishing one adjacency with the DUT. An IGP adjacency SHOULD be 180 established on the Ingress Interface between Tester and DUT. 182 --------- Ingress Interface ---------- 183 | |<--------------------------------| | 184 | | | | 185 | | Preferred Egress Interface | | 186 | DUT |-------------------------------->| Tester | 187 | | | | 188 | |-------------------------------->| | 189 | | Next-Best Egress Interface | | 190 --------- ---------- 192 Figure 1: IGP convergence test topology for local changes 194 Figure 2 shows the test topology to measure IGP convergence time due 195 to local Convergence Events with a non-ECMP Preferred Egress 196 Interface and ECMP Next-Best Egress Interfaces (Section 8.1.1). In 197 this topology, the DUT is configured with each Next-Best Egress 198 interface as a member of a single ECMP set. The Preferred Egress 199 Interface is not a member of an ECMP set. The Tester emulates N+1 200 next-hop routers, one router for the Preferred Egress Interface and N 201 routers for the members of the ECMP set. IGP adjacencies MUST be 202 established between Tester and DUT, one on the Preferred Egress 203 Interface, an one on each member of the ECMP set. For this purpose 204 each of the N+1 routers emulated by the Tester establishes one 205 adjacency with the DUT. An IGP adjacency SHOULD be established on 206 the Ingress Interface between Tester and DUT. When the test 207 specifies to observe the Next-Best Egress Interface statistics, the 208 combined statistics for all ECMP members should be observed. 210 --------- Ingress Interface ---------- 211 | |<--------------------------------| | 212 | | Preferred Egress Interface | | 213 | |-------------------------------->| | 214 | | ECMP set interface 1 | | 215 | DUT |-------------------------------->| Tester | 216 | | . | | 217 | | . | | 218 | |-------------------------------->| | 219 | | ECMP set interface N | | 220 --------- ---------- 222 Figure 2: IGP convergence test topology for local changes with non- 223 ECMP to ECMP convergence 225 3.2. Test topology for remote changes 227 Figure 3 shows the test topology to measure IGP convergence time due 228 to Remote Interface failure (Section 8.1.2). In this topology the 229 two routers R1 and R2 are considered System Under Test (SUT) and 230 SHOULD be identically configured devices of the same model. IGP 231 adjacencies MUST be established between Tester and SUT, one on the 232 Preferred Egress Interface and one on the Next-Best Egress Interface. 233 For this purpose the Tester emulates one or two routers. An IGP 234 adjacency SHOULD be established on the Ingress Interface between 235 Tester and SUT. In this topology there is a possibility of a 236 transient microloop between R1 and R2 during convergence. 238 ------ ---------- 239 | | Preferred | | 240 ------ | R2 |--------------------->| | 241 | |-->| | Egress Interface | | 242 | | ------ | | 243 | R1 | | Tester | 244 | | Next-Best | | 245 | |------------------------------>| | 246 ------ Egress Interface | | 247 ^ ---------- 248 | | 249 --------------------------------------- 250 Ingress Interface 252 Figure 3: IGP convergence test topology for remote changes 254 Figure 4 shows the test topology to measure IGP convergence time due 255 to remote Convergence Events with a non-ECMP Preferred Egress 256 Interface and ECMP Next-Best Egress Interfaces (Section 8.1.2). In 257 this topology the two routers R1 and R2 are considered System Under 258 Test (SUT) and MUST be identically configured devices of the same 259 model. Router R1 is configured with each Next-Best Egress interface 260 as a member of the same ECMP set. The Preferred Egress Interface of 261 R1 is not a member of an ECMP set. The Tester emulates N+1 next-hop 262 routers, one for R2 and one for each member of the ECMP set. IGP 263 adjacencies MUST be established between Tester and SUT, one on each 264 egress interface of SUT. For this purpose each of the N+1 routers 265 emulated by the Tester establishes one adjacency with the SUT. An 266 IGP adjacency SHOULD be established on the Ingress Interface between 267 Tester and SUT. In this topology there is a possibility of a 268 transient microloop between R1 and R2 during convergence. When the 269 test specifies to observe the Next-Best Egress Interface statistics, 270 the combined statistics for all ECMP members should be observed. 272 ------ ---------- 273 | | | | 274 ------ Preferred | R2 |---->| | 275 | |------------------->| | | | 276 | | Egress Interface ------ | | 277 | | | | 278 | | ECMP set interface 1 | | 279 | R1 |------------------------------>| Tester | 280 | | . | | 281 | | . | | 282 | | . | | 283 | |------------------------------>| | 284 ------ ECMP set interface N | | 285 ^ ---------- 286 | | 287 --------------------------------------- 288 Ingress Interface 290 Figure 4: IGP convergence test topology for remote changes with non- 291 ECMP to ECMP convergence 293 3.3. Test topology for local ECMP changes 295 Figure 5 shows the test topology to measure IGP convergence time due 296 to local Convergence Events of a member of an Equal Cost Multipath 297 (ECMP) set (Section 8.1.3). In this topology, the DUT is configured 298 with each egress interface as a member of a single ECMP set and the 299 Tester emulates N next-hop routers, one router for each member. IGP 300 adjacencies MUST be established between Tester and DUT, one on each 301 member of the ECMP set. For this purpose each of the N routers 302 emulated by the Tester establishes one adjacency with the DUT. An 303 IGP adjacency SHOULD be established on the Ingress Interface between 304 Tester and DUT. When the test specifies to observe the Next-Best 305 Egress Interface statistics, the combined statistics for all ECMP 306 members except the one affected by the Convergence Event, should be 307 observed. 309 --------- Ingress Interface ---------- 310 | |<--------------------------------| | 311 | | | | 312 | | ECMP set interface 1 | | 313 | |-------------------------------->| | 314 | DUT | . | Tester | 315 | | . | | 316 | | . | | 317 | |-------------------------------->| | 318 | | ECMP set interface N | | 319 --------- ---------- 321 Figure 5: IGP convergence test topology for local ECMP changes 323 3.4. Test topology for remote ECMP changes 325 Figure 6 shows the test topology to measure IGP convergence time due 326 to remote Convergence Events of a member of an Equal Cost Multipath 327 (ECMP) set (Section 8.1.4). In this topology the two routers R1 and 328 R2 are considered System Under Test (SUT) and MUST be identically 329 configured devices of the same model. Router R1 is configured with 330 each egress interface as a member of a single ECMP set and the Tester 331 emulates N next-hop routers, one router for each member. IGP 332 adjacencies MUST be established between Tester and SUT, one on each 333 egress interface of SUT. For this purpose each of the N routers 334 emulated by the Tester establishes one adjacency with the SUT. An 335 IGP adjacency SHOULD be established on the Ingress Interface between 336 Tester and SUT. In this topology there is a possibility of a 337 transient microloop between R1 and R2 during convergence. When the 338 test specifies to observe the Next-Best Egress Interface statistics, 339 the combined statistics for all ECMP members except the one affected 340 by the Convergence Event, should be observed. 342 ------ ---------- 343 | | | | 344 ------ ECMP set | R2 |---->| | 345 | |------------------->| | | | 346 | | Interface 1 ------ | | 347 | | | | 348 | | ECMP set interface 2 | | 349 | R1 |------------------------------>| Tester | 350 | | . | | 351 | | . | | 352 | | . | | 353 | |------------------------------>| | 354 ------ ECMP set interface N | | 355 ^ ---------- 356 | | 357 --------------------------------------- 358 Ingress Interface 360 Figure 6: IGP convergence test topology for remote ECMP changes 362 3.5. Test topology for Parallel Link changes 364 Figure 7 shows the test topology to measure IGP convergence time due 365 to local Convergence Events with members of a Parallel Link 366 (Section 8.1.5). In this topology, the DUT is configured with each 367 egress interface as a member of a Parallel Link and the Tester 368 emulates the single next-hop router. IGP adjacencies MUST be 369 established on all N members of the Parallel Link between Tester and 370 DUT. For this purpose the router emulated by the Tester establishes 371 N adjacencies with the DUT. An IGP adjacency SHOULD be established 372 on the Ingress Interface between Tester and DUT. When the test 373 specifies to observe the Next-Best Egress Interface statistics, the 374 combined statistics for all Parallel Link members except the one 375 affected by the Convergence Event, should be observed. 377 --------- Ingress Interface ---------- 378 | |<--------------------------------| | 379 | | | | 380 | | Parallel Link Interface 1 | | 381 | |-------------------------------->| | 382 | DUT | . | Tester | 383 | | . | | 384 | | . | | 385 | |-------------------------------->| | 386 | | Parallel Link Interface N | | 387 --------- ---------- 389 Figure 7: IGP convergence test topology for Parallel Link changes 391 4. Convergence Time and Loss of Connectivity Period 393 Two concepts will be highlighted in this section: convergence time 394 and loss of connectivity period. 396 The Route Convergence [Po10t] time indicates the period in time 397 between the Convergence Event Instant [Po10t] and the instant in time 398 the DUT is ready to forward traffic for a specific route on its Next- 399 Best Egress Interface and maintains this state for the duration of 400 the Sustained Convergence Validation Time [Po10t]. To measure Route 401 Convergence time, the Convergence Event Instant and the traffic 402 received from the Next-Best Egress Interface need to be observed. 404 The Route Loss of Connectivity Period [Po10t] indicates the time 405 during which traffic to a specific route is lost following a 406 Convergence Event until Full Convergence [Po10t] completes. This 407 Route Loss of Connectivity Period can consist of one or more Loss 408 Periods [Ko02]. For the testcases described in this document it is 409 expected to have a single Loss Period. To measure Route Loss of 410 Connectivity Period, the traffic received from the Preferred Egress 411 Interface and the traffic received from the Next-Best Egress 412 Interface need to be observed. 414 The Route Loss of Connectivity Period is most important since that 415 has a direct impact on the network user's application performance. 417 In general the Route Convergence time is larger than or equal to the 418 Route Loss of Connectivity Period. Depending on which Convergence 419 Event occurs and how this Convergence Event is applied, traffic for a 420 route may still be forwarded over the Preferred Egress Interface 421 after the Convergence Event Instant, before converging to the Next- 422 Best Egress Interface. In that case the Route Loss of Connectivity 423 Period is shorter than the Route Convergence time. 425 At least one condition needs to be fulfilled for Route Convergence 426 time to be equal to Route Loss of Connectivity Period. The condition 427 is that the Convergence Event causes an instantaneous traffic loss 428 for the measured route. A fiber cut on the Preferred Egress 429 Interface is an example of such a Convergence Event. 431 A second condition applies to Route Convergence time measurements 432 based on Connectivity Packet Loss [Po10t]. This second condition is 433 that there is only a single Loss Period during Route Convergence. 434 For the testcases described in this document this is expected to be 435 the case. 437 4.1. Convergence Events without instant traffic loss 439 To measure convergence time benchmarks for Convergence Events caused 440 by a Tester, such as an IGP cost change, the Tester MAY start to 441 discard all traffic received from the Preferred Egress Interface at 442 the Convergence Event Instant, or MAY separately observe packets 443 received from the Preferred Egress Interface prior to the Convergence 444 Event Instant. This way these Convergence Events can be treated the 445 same as Convergence Events that cause instantaneous traffic loss. 447 To measure convergence time benchmarks without instantaneous traffic 448 loss (either real or induced by the Tester) at the Convergence Event 449 Instant, such as a reversion of a link failure Convergence Event, the 450 Tester SHALL only observe packet statistics on the Next-Best Egress 451 Interface. If using the Rate-Derived method to benchmark convergence 452 times for such Convergence Events, the Tester MUST collect a 453 timestamp at the Convergence Event Instant. If using a loss-derived 454 method to benchmark convergence times for such Convergence Events, 455 the Tester MUST measure the period in time between the Start Traffic 456 Instant and the Convergence Event Instant. To measure this period in 457 time the Tester can collect timestamps at the Start Traffic Instant 458 and the Convergence Event Instant. 460 The Convergence Event Instant together with the receive rate 461 observations on the Next-Best Egress Interface allow to derive the 462 convergence time benchmarks using the Rate-Derived Method [Po10t]. 464 By observing lost packets on the Next-Best Egress Interface only, the 465 observed packet loss is the number of lost packets between Traffic 466 Start Instant and Convergence Recovery Instant. To measure 467 convergence times using a loss-derived method, packet loss between 468 the Convergence Event Instant and the Convergence Recovery Instant is 469 needed. The time between Traffic Start Instant and Convergence Event 470 Instant must be accounted for. An example may clarify this. 472 Figure 8 illustrates a Convergence Event without instantaneous 473 traffic loss for all routes. The top graph shows the Forwarding Rate 474 over all routes, the bottom graph shows the Forwarding Rate for a 475 single route Rta. Some time after the Convergence Event Instant, 476 Forwarding Rate observed on the Preferred Egress Interface starts to 477 decrease. In the example, route Rta is the first route to experience 478 packet loss at time Ta. Some time later, the Forwarding Rate 479 observed on the Next-Best Egress Interface starts to increase. In 480 the example, route Rta is the first route to complete convergence at 481 time Ta'. 483 ^ 484 Fwd | 485 Rate |------------- ............ 486 | \ . 487 | \ . 488 | \ . 489 | \ . 490 |.................-.-.-.-.-.-.---------------- 491 +----+-------+---------------+-----------------> 492 ^ ^ ^ ^ time 493 T0 CEI Ta Ta' 495 ^ 496 Fwd | 497 Rate |------------- ................. 498 Rta | | . 499 | | . 500 |.............-.-.-.-.-.-.-.-.---------------- 501 +----+-------+---------------+-----------------> 502 ^ ^ ^ ^ time 503 T0 CEI Ta Ta' 505 Preferred Egress Interface: --- 506 Next-Best Egress Interface: ... 508 With T0 the Start Traffic Instant; CEI the Convergence Event Instant; 509 Ta the time instant traffic loss for route Rta starts; Ta' the time 510 instant traffic loss for route Rta ends. 512 Figure 8 514 If only packets received on the Next-Best Egress Interface are 515 observed, the duration of the packet loss period for route Rta can be 516 calculated from the received packets as in Equation 1. Since the 517 Convergence Event Instant is the start time for convergence time 518 measurement, the period in time between T0 and CEI needs to be 519 subtracted from the calculated result to become the convergence time, 520 as in Equation 2. 522 Next-Best Egress Interface packet loss period 523 = (packets transmitted 524 - packets received from Next-Best Egress Interface) / tx rate 525 = Ta' - T0 527 Equation 1 529 convergence time 530 = Next-Best Egress Interface packet loss period - (CEI - T0) 531 = Ta' - CEI 533 Equation 2 535 4.2. Loss of Connectivity 537 Route Loss of Connectivity Period SHOULD be measured using the Route- 538 Specific Loss-Derived Method. Since the start instant and end 539 instant of the Route Loss of Connectivity Period can be different for 540 each route, these can not be accurately derived by only observing 541 global statistics over all routes. An example may clarify this. 543 Following a Convergence Event, route Rta is the first route for which 544 packet loss starts, the Route Loss of Connectivity Period for route 545 Rta starts at time Ta. Route Rtb is the last route for which packet 546 loss starts, the Route Loss of Connectivity Period for route Rtb 547 starts at time Tb with Tb>Ta. 549 ^ 550 Fwd | 551 Rate |-------- ----------- 552 | \ / 553 | \ / 554 | \ / 555 | \ / 556 | --------------- 557 +------------------------------------------> 558 ^ ^ ^ ^ time 559 Ta Tb Ta' Tb' 560 Tb'' Ta'' 562 Figure 9: Example Route Loss Of Connectivity Period 564 If the DUT implementation would be such that Route Rta would be the 565 first route for which traffic loss ends at time Ta' with Ta'>Tb. 566 Route Rtb would be the last route for which traffic loss ends at time 567 Tb' with Tb'>Ta'. By using only observing global traffic statistics 568 over all routes, the minimum Route Loss of Connectivity Period would 569 be measured as Ta'-Ta. The maximum calculated Route Loss of 570 Connectivity Period would be Tb'-Ta. The real minimum and maximum 571 Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb. 572 Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5, 573 would give a LoC Period between 3 and 5 derived from the global 574 traffic statistics, versus the real LoC Period between 3 and 4. 576 If the DUT implementation would be such that route Rtb would be the 577 first for which packet loss ends at time Tb'' and route Rta would be 578 the last for which packet loss ends at time Ta'', then the minimum 579 and maximum Route Loss of Connectivity Periods derived by observing 580 only global traffic statistics would be Tb''-Ta, and Ta''-Ta. The 581 real minimum and maximum Route Loss of Connectivity Periods are 582 Tb''-Tb and Ta''-Ta. Illustrating this with the numbers Ta=0, Tb=1, 583 Ta''=5, Tb''=3, would give a LoC Period between 3 and 5 derived from 584 the global traffic statistics, versus the real LoC Period between 2 585 and 5. 587 The two implementation variations in the above example would result 588 in the same derived minimum and maximum Route Loss of Connectivity 589 Periods when only observing the global packet statistics, while the 590 real Route Loss of Connectivity Periods are different. 592 5. Test Considerations 594 5.1. IGP Selection 596 The test cases described in Section 8 MAY be used for link-state 597 IGPs, such as ISIS or OSPF. The IGP convergence time test 598 methodology is identical. 600 5.2. Routing Protocol Configuration 602 The obtained results for IGP convergence time may vary if other 603 routing protocols are enabled and routes learned via those protocols 604 are installed. IGP convergence times SHOULD be benchmarked without 605 routes installed from other protocols. 607 5.3. IGP Topology 609 The Tester emulates a single IGP topology. The DUT establishes IGP 610 adjacencies with one or more of the emulated routers in this single 611 IGP topology emulated by the Tester. See test topology details in 612 Section 3. The emulated topology SHOULD only be advertised on the 613 DUT egress interfaces. 615 The number of IGP routes and number of nodes in the topology, and the 616 type of topology will impact the measured IGP convergence time. To 617 obtain results similar to those that would be observed in an 618 operational network, it is RECOMMENDED that the number of installed 619 routes and nodes closely approximate that of the network (e.g. 620 thousands of routes with tens or hundreds of nodes). 622 The number of areas (for OSPF) and levels (for ISIS) can impact the 623 benchmark results. 625 5.4. Timers 627 There are timers that may impact the measured IGP convergence times. 628 The benchmark metrics MAY be measured at any fixed values for these 629 timers. To obtain results similar to those that would be observed in 630 an operational network, it is RECOMMENDED to configure the timers 631 with the values as configured in the operational network. 633 Examples of timers that may impact measured IGP convergence time 634 include, but are not limited to: 636 Interface failure indication 638 IGP hello timer 640 IGP dead-interval or hold-timer 642 LSA or LSP generation delay 644 LSA or LSP flood packet pacing 646 SPF delay 648 5.5. Interface Types 650 All test cases in this methodology document MAY be executed with any 651 interface type. The type of media may dictate which test cases may 652 be executed. Each interface type has a unique mechanism for 653 detecting link failures and the speed at which that mechanism 654 operates will influence the measurement results. All interfaces MUST 655 be the same media and Throughput [Br91][Br99] for each test case. 656 All interfaces SHOULD be configured as point-to-point. 658 5.6. Offered Load 660 The Throughput of the device, as defined in [Br91] and benchmarked in 661 [Br99] at a fixed packet size, needs to be determined over the 662 preferred path and over the next-best path. The Offered Load SHOULD 663 be the minimum of the measured Throughput of the device over the 664 primary path and over the backup path. The packet size is selectable 665 and MUST be recorded. Packet size is measured in bytes and includes 666 the IP header and payload. 668 The destination addresses for the Offered Load MUST be distributed 669 such that all routes or a statistically representative subset of all 670 routes are matched and each of these routes is offered an equal share 671 of the Offered Load. It is RECOMMENDED to send traffic matching all 672 routes, but a statistically representative subset of all routes can 673 be used if required. 675 In the Remote Interface failure testcases using topologies 3, 4, and 676 6 there is a possibility of a transient microloop between R1 and R2 677 during convergence. The TTL or Hop Limit value of the packets sent 678 by the Tester may influence the benchmark measurements since it 679 determines which device in the topology may send an ICMP Time 680 Exceeded Message for looped packets. 682 The duration of the Offered Load MUST be greater than the convergence 683 time plus the Sustained Convergence Validation Time. 685 Offered load should send a packet to each destination before sending 686 another packet to the same destination. It is RECOMMENDED that the 687 packets are transmitted in a round-robin fashion with a uniform 688 interpacket delay. 690 5.7. Measurement Accuracy 692 Since packet loss is observed to measure the Route Convergence Time, 693 the time between two successive packets offered to each individual 694 route is the highest possible accuracy of any packet loss based 695 measurement. The higher the traffic rate offered to each route the 696 higher the possible measurement accuracy. 698 Also see Section 6 for method-specific measurement accuracy. 700 5.8. Measurement Statistics 702 The benchmark measurements may vary for each trial, due to the 703 statistical nature of timer expirations, cpu scheduling, etc. 704 Evaluation of the test data must be done with an understanding of 705 generally accepted testing practices regarding repeatability, 706 variance and statistical significance of a small number of trials. 708 5.9. Tester Capabilities 710 It is RECOMMENDED that the Tester used to execute each test case has 711 the following capabilities: 713 1. Ability to establish IGP adjacencies and advertise a single IGP 714 topology to one or more peers. 716 2. Ability to measure Forwarding Delay, Duplicate Packets and Out- 717 of-Order Packets. 719 3. An internal time clock to control timestamping, time 720 measurements, and time calculations. 722 4. Ability to distinguish traffic load received on the Preferred and 723 Next-Best Interfaces [Po10t]. 725 5. Ability to disable or tune specific Layer-2 and Layer-3 protocol 726 functions on any interface(s). 728 The Tester MAY be capable to make non-data plane convergence 729 observations and use those observations for measurements. The Tester 730 MAY be capable to send and receive multiple traffic Streams [Po06]. 732 Also see Section 6 for method-specific capabilities. 734 6. Selection of Convergence Time Benchmark Metrics and Methods 736 Different convergence time benchmark methods MAY be used to measure 737 convergence time benchmark metrics. The Tester capabilities are 738 important criteria to select a specific convergence time benchmark 739 method. The criteria to select a specific benchmark method include, 740 but are not limited to: 742 Tester capabilities: Sampling Interval, number of 743 Stream statistics to collect 744 Measurement accuracy: Sampling Interval, Offered Load, 745 number of routes 746 Test specification: number of routes 747 DUT capabilities: Throughput, IP Packet Delay 748 Variation 750 6.1. Loss-Derived Method 752 6.1.1. Tester capabilities 754 The Offered Load SHOULD consist of a single Stream [Po06]. If 755 sending multiple Streams, the measured packet loss statistics for all 756 Streams MUST be added together. 758 In order to verify Full Convergence completion and the Sustained 759 Convergence Validation Time, the Tester MUST measure Forwarding Rate 760 each Packet Sampling Interval. 762 The total number of packets lost between the start of the traffic and 763 the end of the Sustained Convergence Validation Time is used to 764 calculate the Loss-Derived Convergence Time. 766 6.1.2. Benchmark Metrics 768 The Loss-Derived Method can be used to measure the Loss-Derived 769 Convergence Time, which is the average convergence time over all 770 routes, and to measure the Loss-Derived Loss of Connectivity Period, 771 which is the average Route Loss of Connectivity Period over all 772 routes. 774 6.1.3. Measurement Accuracy 776 The actual value falls within the accuracy interval [-(number of 777 destinations/Offered Load), +(number of destinations/Offered Load)] 778 around the value as measured using the Loss-Derived Method. 780 6.2. Rate-Derived Method 782 6.2.1. Tester Capabilities 784 The Offered Load SHOULD consist of a single Stream. If sending 785 multiple Streams, the measured traffic rate statistics for all 786 Streams MUST be added together. 788 The Tester measures Forwarding Rate each Sampling Interval. The 789 Packet Sampling Interval influences the observation of the different 790 convergence time instants. If the Packet Sampling Interval is large 791 compared to the time between the convergence time instants, then the 792 different time instants may not be easily identifiable from the 793 Forwarding Rate observation. The presence of IPDV [De02] may cause 794 fluctuations of the Forwarding Rate observation and can prevent 795 correct observation of the different convergence time instants. 797 The Packet Sampling Interval MUST be larger than or equal to the time 798 between two consecutive packets to the same destination. For maximum 799 accuracy the value for the Packet Sampling Interval SHOULD be as 800 small as possible, but the presence of IPDV may enforce using a 801 larger Packet Sampling Interval. The Packet Sampling Interval MUST 802 be reported. 804 IPDV causes fluctuations in the number of received packets during 805 each Packet Sampling Interval. To account for the presence of IPDV 806 in determining if a convergence instant has been reached, Forwarding 807 Delay SHOULD be observed during each Packet Sampling Interval. The 808 minimum and maximum number of packets expected in a Packet Sampling 809 Interval in presence of IPDV can be calculated with Equation 3. 811 number of packets expected in a Packet Sampling Interval 812 in presence of IP Packet Delay Variation 813 = expected number of packets without IP Packet Delay Variation 814 +/-( (maxDelay - minDelay) * Offered Load) 815 with minDelay and maxDelay the minimum resp. maximum Forwarding Delay 816 of packets received during the Packet Sampling Interval 818 Equation 3 820 To determine if a convergence instant has been reached the number of 821 packets received in a Packet Sampling Interval is compared with the 822 range of expected number of packets calculated in Equation 3. 824 6.2.2. Benchmark Metrics 826 The Rate-Derived Method SHOULD be used to measure First Route 827 Convergence Time and Full Convergence Time. It SHOULD NOT be used to 828 measure Loss of Connectivity Period (see Section 4). 830 6.2.3. Measurement Accuracy 832 The measurement accuracy interval of the Rate-Derived Method depends 833 on the metric being measured or calculated and the characteristics of 834 the related transition. IPDV [De02] adds uncertainty to the amount 835 of packets received in a Packet Sampling Interval and this 836 uncertainty adds to the measurement error. The effect of IPDV is not 837 accounted for in the calculation of the accuracy intervals below. 838 IPDV is of importance for the convergence instants were a variation 839 in Forwarding Rate needs to be observed (Convergence Recovery Instant 840 and for topologies with ECMP also Convergence Event Instant and First 841 Route Convergence Instant). 843 If the Convergence Event Instant is observed on the dataplane using 844 the Rate Derived Method, it needs to be instantaneous for all routes 845 (see Section 4.1). The actual value of the Convergence Event Instant 846 falls within the accuracy interval [-(Packet Sampling Interval + 847 1/Offered Load), +0] around the value as measured using the Rate- 848 Derived Method. 850 If the Convergence Recovery Transition is non-instantaneous for all 851 routes then the actual value of the First Route Convergence Instant 852 falls within the accuracy interval [-(Packet Sampling Interval + time 853 between two consecutive packets to the same destination), +0] around 854 the value as measured using the Rate-Derived Method, and the actual 855 value of the Convergence Recovery Instant falls within the accuracy 856 interval [-(2 * Packet Sampling Interval), -(Packet Sampling Interval 857 - time between two consecutive packets to the same destination)] 858 around the value as measured using the Rate-Derived Method. 860 The term "time between two consecutive packets to the same 861 destination" is added in the above accuracy intervals since packets 862 are sent in a particular order to all destinations in a stream and 863 when part of the routes experience packet loss, it is unknown where 864 in the transmit cycle packets to these routes are sent. This 865 uncertainty adds to the error. 867 The accuracy intervals of the derived metrics First Route Convergence 868 Time and Rate-Derived Convergence Time are calculated from the above 869 convergence instants accuracy intervals. The actual value of First 870 Route Convergence Time falls within the accuracy interval [-(Packet 871 Sampling Interval + time between two consecutive packets to the same 872 destination), +(Packet Sampling Interval + 1/Offered Load)] around 873 the calculated value. The actual value of Rate-Derived Convergence 874 Time falls within the accuracy interval [-(2 * Packet Sampling 875 Interval), +(time between two consecutive packets to the same 876 destination + 1/Offered Load)] around the calculated value. 878 6.3. Route-Specific Loss-Derived Method 880 6.3.1. Tester Capabilities 882 The Offered Load consists of multiple Streams. The Tester MUST 883 measure packet loss for each Stream separately. 885 In order to verify Full Convergence completion and the Sustained 886 Convergence Validation Time, the Tester MUST measure packet loss each 887 Packet Sampling Interval. This measurement at each Packet Sampling 888 Interval MAY be per Stream. 890 Only the total packet loss measured per Stream at the end of the 891 Sustained Convergence Validation Time is used to calculate the 892 benchmark metrics with this method. 894 6.3.2. Benchmark Metrics 896 The Route-Specific Loss-Derived Method SHOULD be used to measure 897 Route-Specific Convergence Times. It is the RECOMMENDED method to 898 measure Route Loss of Connectivity Period. 900 Under the conditions explained in Section 4, First Route Convergence 901 Time and Full Convergence Time as benchmarked using Rate-Derived 902 Method, may be equal to the minimum resp. maximum of the Route- 903 Specific Convergence Times. 905 6.3.3. Measurement Accuracy 907 The actual value falls within the accuracy interval [-(number of 908 destinations/Offered Load), +(number of destinations/Offered Load)] 909 around the value as measured using the Route-Specific Loss-Derived 910 Method. 912 7. Reporting Format 914 For each test case, it is recommended that the reporting tables below 915 are completed and all time values SHOULD be reported with resolution 916 as specified in [Po10t]. 918 Parameter Units 919 ----------------------------------- --------------------------- 920 Test Case test case number 921 Test Topology Test Topology Figure number 922 IGP (ISIS, OSPF, other) 923 Interface Type (GigE, POS, ATM, other) 924 Packet Size offered to DUT bytes 925 Offered Load packets per second 926 IGP Routes advertised to DUT number of IGP routes 927 Nodes in emulated network number of nodes 928 Number of Parallel or ECMP links number of links 929 Number of Routes measured number of routes 930 Packet Sampling Interval on Tester seconds 931 Forwarding Delay Threshold seconds 933 Timer Values configured on DUT: 934 Interface failure indication delay seconds 935 IGP Hello Timer seconds 936 IGP Dead-Interval or hold-time seconds 937 LSA Generation Delay seconds 938 LSA Flood Packet Pacing seconds 939 LSA Retransmission Packet Pacing seconds 940 SPF Delay seconds 942 Test Details: 944 If the Offered Load matches a subset of routes, describe how this 945 subset is selected. 947 Describe how the Convergence Event is applied; does it cause 948 instantaneous traffic loss or not. 950 Complete the table below for the initial Convergence Event and the 951 reversion Convergence Event. 953 Parameter Units 954 ------------------------------------------ ---------------------- 955 Conversion Event (initial or reversion) 957 Traffic Forwarding Metrics: 958 Total number of packets offered to DUT number of Packets 959 Total number of packets forwarded by DUT number of Packets 960 Connectivity Packet Loss number of Packets 961 Convergence Packet Loss number of Packets 962 Out-of-Order Packets number of Packets 963 Duplicate Packets number of Packets 965 Convergence Benchmarks: 966 Rate-Derived Method: 967 First Route Convergence Time seconds 968 Full Convergence Time seconds 969 Loss-Derived Method: 970 Loss-Derived Convergence Time seconds 971 Route-Specific Loss-Derived Method: 972 Route-Specific Convergence Time[n] array of seconds 973 Minimum R-S Convergence Time seconds 974 Maximum R-S Convergence Time seconds 975 Median R-S Convergence Time seconds 976 Average R-S Convergence Time seconds 978 Loss of Connectivity Benchmarks: 979 Loss-Derived Method: 980 Loss-Derived Loss of Connectivity Period seconds 981 Route-Specific Loss-Derived Method: 982 Route LoC Period[n] array of seconds 983 Minimum Route LoC Period seconds 984 Maximum Route LoC Period seconds 985 Median Route LoC Period seconds 986 Average Route LoC Period seconds 988 8. Test Cases 990 It is RECOMMENDED that all applicable test cases be performed for 991 best characterization of the DUT. The test cases follow a generic 992 procedure tailored to the specific DUT configuration and Convergence 993 Event [Po10t]. This generic procedure is as follows: 995 1. Establish DUT and Tester configurations and advertise an IGP 996 topology from Tester to DUT. 998 2. Send Offered Load from Tester to DUT on ingress interface. 1000 3. Verify traffic is routed correctly. Verify if traffic is 1001 forwarded without drops, without Out-of-Order Packets, and 1002 without exceeding the Forwarding Delay Threshold [Po06]. 1004 4. Introduce Convergence Event [Po10t]. 1006 5. Measure First Route Convergence Time [Po10t]. 1008 6. Measure Full Convergence Time [Po10t]. 1010 7. Stop Offered Load. 1012 8. Measure Route-Specific Convergence Times, Loss-Derived 1013 Convergence Time, Route LoC Periods, and Loss-Derived LoC Period 1014 [Po10t]. 1016 9. Wait sufficient time for queues to drain. The duration of this 1017 time period is equal to the Forwarding Delay Threshold. In 1018 absence of a Forwarding Delay Threshold specification the 1019 duration of this time period is 2 seconds [Br99]. 1021 10. Restart Offered Load. 1023 11. Reverse Convergence Event. 1025 12. Measure First Route Convergence Time. 1027 13. Measure Full Convergence Time. 1029 14. Stop Offered Load. 1031 15. Measure Route-Specific Convergence Times, Loss-Derived 1032 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1033 Period. 1035 8.1. Interface failures 1037 8.1.1. Convergence Due to Local Interface Failure 1039 Objective 1041 To obtain the IGP convergence times due to a Local Interface failure 1042 event. The Next-Best Egress Interface can be a single interface 1043 (Figure 1) or an ECMP set (Figure 2). The test with ECMP topology 1044 (Figure 2) is OPTIONAL. 1046 Procedure 1047 1. Advertise an IGP topology from Tester to DUT using the topology 1048 shown in Figure 1 or 2. 1050 2. Send Offered Load from Tester to DUT on ingress interface. 1052 3. Verify traffic is forwarded over Preferred Egress Interface. 1054 4. Remove link on DUT's Preferred Egress Interface. This is the 1055 Convergence Event. 1057 5. Measure First Route Convergence Time. 1059 6. Measure Full Convergence Time. 1061 7. Stop Offered Load. 1063 8. Measure Route-Specific Convergence Times and Loss-Derived 1064 Convergence Time. 1066 9. Wait sufficient time for queues to drain. 1068 10. Restart Offered Load. 1070 11. Restore link on DUT's Preferred Egress Interface. 1072 12. Measure First Route Convergence Time. 1074 13. Measure Full Convergence Time. 1076 14. Stop Offered Load. 1078 15. Measure Route-Specific Convergence Times, Loss-Derived 1079 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1080 Period. 1082 Results 1084 The measured IGP convergence time may be influenced by the link 1085 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1086 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1087 and forwarding tables update time [Po09a]. 1089 8.1.2. Convergence Due to Remote Interface Failure 1091 Objective 1093 To obtain the IGP convergence time due to a Remote Interface failure 1094 event. The Next-Best Egress Interface can be a single interface 1095 (Figure 3) or an ECMP set (Figure 4). The test with ECMP topology 1096 (Figure 4) is OPTIONAL. 1098 Procedure 1100 1. Advertise an IGP topology from Tester to SUT using the topology 1101 shown in Figure 3 or 4. 1103 2. Send Offered Load from Tester to SUT on ingress interface. 1105 3. Verify traffic is forwarded over Preferred Egress Interface. 1107 4. Remove link on Tester's interface [Po10t] connected to SUT's 1108 Preferred Egress Interface. This is the Convergence Event. 1110 5. Measure First Route Convergence Time. 1112 6. Measure Full Convergence Time. 1114 7. Stop Offered Load. 1116 8. Measure Route-Specific Convergence Times and Loss-Derived 1117 Convergence Time. 1119 9. Wait sufficient time for queues to drain. 1121 10. Restart Offered Load. 1123 11. Restore link on Tester's interface connected to DUT's Preferred 1124 Egress Interface. 1126 12. Measure First Route Convergence Time. 1128 13. Measure Full Convergence Time. 1130 14. Stop Offered Load. 1132 15. Measure Route-Specific Convergence Times, Loss-Derived 1133 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1134 Period. 1136 Results 1138 The measured IGP convergence time may be influenced by the link 1139 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1140 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1141 and forwarding tables update time. This test case may produce Stale 1142 Forwarding [Po10t] due to a transient microloop between R1 and R2 1143 during convergence, which may increase the measured convergence times 1144 and loss of connectivity periods. 1146 8.1.3. Convergence Due to ECMP Member Local Interface Failure 1148 Objective 1150 To obtain the IGP convergence time due to a Local Interface link 1151 failure event of an ECMP Member. 1153 Procedure 1155 1. Advertise an IGP topology from Tester to DUT using the test 1156 setup shown in Figure 5. 1158 2. Send Offered Load from Tester to DUT on ingress interface. 1160 3. Verify traffic is forwarded over the DUT's ECMP member interface 1161 that will be failed in the next step. 1163 4. Remove link on one of the DUT's ECMP member interfaces. This is 1164 the Convergence Event. 1166 5. Measure First Route Convergence Time. 1168 6. Measure Full Convergence Time. 1170 7. Stop Offered Load. 1172 8. Measure Route-Specific Convergence Times and Loss-Derived 1173 Convergence Time. At the same time measure Out-of-Order Packets 1174 [Po06] and Duplicate Packets [Po06]. 1176 9. Wait sufficient time for queues to drain. 1178 10. Restart Offered Load. 1180 11. Restore link on DUT's ECMP member interface. 1182 12. Measure First Route Convergence Time. 1184 13. Measure Full Convergence Time. 1186 14. Stop Offered Load. 1188 15. Measure Route-Specific Convergence Times, Loss-Derived 1189 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1190 Period. At the same time measure Out-of-Order Packets [Po06] 1191 and Duplicate Packets [Po06]. 1193 Results 1195 The measured IGP Convergence time may be influenced by link failure 1196 indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP 1197 flood packet pacing, SPF delay, SPF execution time, and routing and 1198 forwarding tables update time [Po09a]. 1200 8.1.4. Convergence Due to ECMP Member Remote Interface Failure 1202 Objective 1204 To obtain the IGP convergence time due to a Remote Interface link 1205 failure event for an ECMP Member. 1207 Procedure 1209 1. Advertise an IGP topology from Tester to DUT using the test 1210 setup shown in Figure 6. 1212 2. Send Offered Load from Tester to DUT on ingress interface. 1214 3. Verify traffic is forwarded over the DUT's ECMP member interface 1215 that will be failed in the next step. 1217 4. Remove link on Tester's interface to R2. This is the 1218 Convergence Event Trigger. 1220 5. Measure First Route Convergence Time. 1222 6. Measure Full Convergence Time. 1224 7. Stop Offered Load. 1226 8. Measure Route-Specific Convergence Times and Loss-Derived 1227 Convergence Time. At the same time measure Out-of-Order Packets 1228 [Po06] and Duplicate Packets [Po06]. 1230 9. Wait sufficient time for queues to drain. 1232 10. Restart Offered Load. 1234 11. Restore link on Tester's interface to R2. 1236 12. Measure First Route Convergence Time. 1238 13. Measure Full Convergence Time. 1240 14. Stop Offered Load. 1242 15. Measure Route-Specific Convergence Times, Loss-Derived 1243 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1244 Period. At the same time measure Out-of-Order Packets [Po06] 1245 and Duplicate Packets [Po06]. 1247 Results 1249 The measured IGP convergence time may influenced by the link failure 1250 indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP 1251 flood packet pacing, SPF delay, SPF execution time, and routing and 1252 forwarding tables update time. This test case may produce Stale 1253 Forwarding [Po10t] due to a transient microloop between R1 and R2 1254 during convergence, which may increase the measured convergence times 1255 and loss of connectivity periods. 1257 8.1.5. Convergence Due to Parallel Link Interface Failure 1259 Objective 1261 To obtain the IGP convergence due to a local link failure event for a 1262 member of a parallel link. The links can be used for data load 1263 balancing 1265 Procedure 1267 1. Advertise an IGP topology from Tester to DUT using the test 1268 setup shown in Figure 7. 1270 2. Send Offered Load from Tester to DUT on ingress interface. 1272 3. Verify traffic is forwarded over the parallel link member that 1273 will be failed in the next step. 1275 4. Remove link on one of the DUT's parallel link member interfaces. 1276 This is the Convergence Event. 1278 5. Measure First Route Convergence Time. 1280 6. Measure Full Convergence Time. 1282 7. Stop Offered Load. 1284 8. Measure Route-Specific Convergence Times and Loss-Derived 1285 Convergence Time. At the same time measure Out-of-Order Packets 1287 [Po06] and Duplicate Packets [Po06]. 1289 9. Wait sufficient time for queues to drain. 1291 10. Restart Offered Load. 1293 11. Restore link on DUT's Parallel Link member interface. 1295 12. Measure First Route Convergence Time. 1297 13. Measure Full Convergence Time. 1299 14. Stop Offered Load. 1301 15. Measure Route-Specific Convergence Times, Loss-Derived 1302 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1303 Period. At the same time measure Out-of-Order Packets [Po06] 1304 and Duplicate Packets [Po06]. 1306 Results 1308 The measured IGP convergence time may be influenced by the link 1309 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1310 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1311 and forwarding tables update time [Po09a]. 1313 8.2. Other failures 1315 8.2.1. Convergence Due to Layer 2 Session Loss 1317 Objective 1319 To obtain the IGP convergence time due to a local layer 2 loss. 1321 Procedure 1323 1. Advertise an IGP topology from Tester to DUT using the topology 1324 shown in Figure 1. 1326 2. Send Offered Load from Tester to DUT on ingress interface. 1328 3. Verify traffic is routed over Preferred Egress Interface. 1330 4. Remove Layer 2 session from DUT's Preferred Egress Interface. 1331 This is the Convergence Event. 1333 5. Measure First Route Convergence Time. 1335 6. Measure Full Convergence Time. 1337 7. Stop Offered Load. 1339 8. Measure Route-Specific Convergence Times, Loss-Derived 1340 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1341 Period. 1343 9. Wait sufficient time for queues to drain. 1345 10. Restart Offered Load. 1347 11. Restore Layer 2 session on DUT's Preferred Egress Interface. 1349 12. Measure First Route Convergence Time. 1351 13. Measure Full Convergence Time. 1353 14. Stop Offered Load. 1355 15. Measure Route-Specific Convergence Times, Loss-Derived 1356 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1357 Period. 1359 Results 1361 The measured IGP Convergence time may be influenced by the Layer 2 1362 failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/ 1363 LSP flood packet pacing, SPF delay, SPF execution time, and routing 1364 and forwarding tables update time [Po09a]. 1366 Discussion 1368 Configure IGP timers such that the IGP adjacency does not time out 1369 before layer 2 failure is detected. 1371 To measure convergence time, traffic SHOULD start dropping on the 1372 Preferred Egress Interface on the instant the layer 2 session is 1373 removed. Alternatively the Tester SHOULD record the time the instant 1374 layer 2 session is removed and traffic loss SHOULD only be measured 1375 on the Next-Best Egress Interface. For loss-derived benchmarks the 1376 time of the Start Traffic Instant SHOULD be recorded as well. See 1377 Section 4.1. 1379 8.2.2. Convergence Due to Loss of IGP Adjacency 1381 Objective 1382 To obtain the IGP convergence time due to loss of an IGP Adjacency. 1384 Procedure 1386 1. Advertise an IGP topology from Tester to DUT using the topology 1387 shown in Figure 1. 1389 2. Send Offered Load from Tester to DUT on ingress interface. 1391 3. Verify traffic is routed over Preferred Egress Interface. 1393 4. Remove IGP adjacency from the Preferred Egress Interface while 1394 the layer 2 session MUST be maintained. This is the Convergence 1395 Event. 1397 5. Measure First Route Convergence Time. 1399 6. Measure Full Convergence Time. 1401 7. Stop Offered Load. 1403 8. Measure Route-Specific Convergence Times, Loss-Derived 1404 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1405 Period. 1407 9. Wait sufficient time for queues to drain. 1409 10. Restart Offered Load. 1411 11. Restore IGP session on DUT's Preferred Egress Interface. 1413 12. Measure First Route Convergence Time. 1415 13. Measure Full Convergence Time. 1417 14. Stop Offered Load. 1419 15. Measure Route-Specific Convergence Times, Loss-Derived 1420 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1421 Period. 1423 Results 1425 The measured IGP Convergence time may be influenced by the IGP Hello 1426 Interval, IGP Dead Interval, LSA/LSP delay, LSA/LSP generation time, 1427 LSA/LSP flood packet pacing, SPF delay, SPF execution time, and 1428 routing and forwarding tables update time [Po09a]. 1430 Discussion 1432 Configure layer 2 such that layer 2 does not time out before IGP 1433 adjacency failure is detected. 1435 To measure convergence time, traffic SHOULD start dropping on the 1436 Preferred Egress Interface on the instant the IGP adjacency is 1437 removed. Alternatively the Tester SHOULD record the time the instant 1438 the IGP adjacency is removed and traffic loss SHOULD only be measured 1439 on the Next-Best Egress Interface. For loss-derived benchmarks the 1440 time of the Start Traffic Instant SHOULD be recorded as well. See 1441 Section 4.1. 1443 8.2.3. Convergence Due to Route Withdrawal 1445 Objective 1447 To obtain the IGP convergence time due to route withdrawal. 1449 Procedure 1451 1. Advertise an IGP topology from Tester to DUT using the topology 1452 shown in Figure 1. The routes that will be withdrawn MUST be a 1453 set of leaf routes advertised by at least two nodes in the 1454 emulated topology. The topology SHOULD be such that before the 1455 withdrawal the DUT prefers the leaf routes advertised by a node 1456 "nodeA" via the Preferred Egress Interface, and after the 1457 withdrawal the DUT prefers the leaf routes advertised by a node 1458 "nodeB" via the Next-Best Egress Interface. 1460 2. Send Offered Load from Tester to DUT on Ingress Interface. 1462 3. Verify traffic is routed over Preferred Egress Interface. 1464 4. The Tester withdraws the set of IGP leaf routes from nodeA. 1465 This is the Convergence Event. The withdrawal update message 1466 SHOULD be a single unfragmented packet. If the routes cannot be 1467 withdrawn by a single packet, the messages SHOULD be sent using 1468 the same pacing characteristics as the DUT. The Tester MAY 1469 record the time it sends the withdrawal message(s). 1471 5. Measure First Route Convergence Time. 1473 6. Measure Full Convergence Time. 1475 7. Stop Offered Load. 1477 8. Measure Route-Specific Convergence Times, Loss-Derived 1478 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1479 Period. 1481 9. Wait sufficient time for queues to drain. 1483 10. Restart Offered Load. 1485 11. Re-advertise the set of withdrawn IGP leaf routes from nodeA 1486 emulated by the Tester. The update message SHOULD be a single 1487 unfragmented packet. If the routes cannot be advertised by a 1488 single packet, the messages SHOULD be sent using the same pacing 1489 characteristics as the DUT. The Tester MAY record the time it 1490 sends the update message(s). 1492 12. Measure First Route Convergence Time. 1494 13. Measure Full Convergence Time. 1496 14. Stop Offered Load. 1498 15. Measure Route-Specific Convergence Times, Loss-Derived 1499 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1500 Period. 1502 Results 1504 The measured IGP convergence time is influenced by SPF or route 1505 calculation delay, SPF or route calculation execution time, and 1506 routing and forwarding tables update time [Po09a]. 1508 Discussion 1510 To measure convergence time, traffic SHOULD start dropping on the 1511 Preferred Egress Interface on the instant the routes are withdrawn by 1512 the Tester. Alternatively the Tester SHOULD record the time the 1513 instant the routes are withdrawn and traffic loss SHOULD only be 1514 measured on the Next-Best Egress Interface. For loss-derived 1515 benchmarks the time of the Start Traffic Instant SHOULD be recorded 1516 as well. See Section 4.1. 1518 8.3. Administrative changes 1520 8.3.1. Convergence Due to Local Adminstrative Shutdown 1522 Objective 1524 To obtain the IGP convergence time due to taking the DUT's Local 1525 Interface administratively out of service. 1527 Procedure 1529 1. Advertise an IGP topology from Tester to DUT using the topology 1530 shown in Figure 1. 1532 2. Send Offered Load from Tester to DUT on ingress interface. 1534 3. Verify traffic is routed over Preferred Egress Interface. 1536 4. Take the DUT's Preferred Egress Interface administratively out 1537 of service. This is the Convergence Event. 1539 5. Measure First Route Convergence Time. 1541 6. Measure Full Convergence Time. 1543 7. Stop Offered Load. 1545 8. Measure Route-Specific Convergence Times, Loss-Derived 1546 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1547 Period. 1549 9. Wait sufficient time for queues to drain. 1551 10. Restart Offered Load. 1553 11. Restore Preferred Egress Interface by administratively enabling 1554 the interface. 1556 12. Measure First Route Convergence Time. 1558 13. Measure Full Convergence Time. 1560 14. Stop Offered Load. 1562 15. Measure Route-Specific Convergence Times, Loss-Derived 1563 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1564 Period. 1566 16. It is possible that no measured packet loss will be observed for 1567 this test case. 1569 Results 1571 The measured IGP Convergence time may be influenced by LSA/LSP delay, 1572 LSA/LSP generation time, LSA/LSP flood packet pacing, SPF delay, SPF 1573 execution time, and routing and forwarding tables update time 1574 [Po09a]. 1576 8.3.2. Convergence Due to Cost Change 1578 Objective 1580 To obtain the IGP convergence time due to route cost change. 1582 Procedure 1584 1. Advertise an IGP topology from Tester to DUT using the topology 1585 shown in Figure 1. 1587 2. Send Offered Load from Tester to DUT on ingress interface. 1589 3. Verify traffic is routed over Preferred Egress Interface. 1591 4. The Tester, emulating the neighbor node, increases the cost for 1592 all IGP routes at DUT's Preferred Egress Interface so that the 1593 Next-Best Egress Interface becomes preferred path. The update 1594 message advertising the higher cost MUST be a single 1595 unfragmented packet. This is the Convergence Event. The Tester 1596 MAY record the time it sends the update message advertising the 1597 higher cost on the Preferred Egress Interface. 1599 5. Measure First Route Convergence Time. 1601 6. Measure Full Convergence Time. 1603 7. Stop Offered Load. 1605 8. Measure Route-Specific Convergence Times, Loss-Derived 1606 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1607 Period. 1609 9. Wait sufficient time for queues to drain. 1611 10. Restart Offered Load. 1613 11. The Tester, emulating the neighbor node, decreases the cost for 1614 all IGP routes at DUT's Preferred Egress Interface so that the 1615 Preferred Egress Interface becomes preferred path. The update 1616 message advertising the lower cost MUST be a single unfragmented 1617 packet. 1619 12. Measure First Route Convergence Time. 1621 13. Measure Full Convergence Time. 1623 14. Stop Offered Load. 1625 15. Measure Route-Specific Convergence Times, Loss-Derived 1626 Convergence Time, Route LoC Periods, and Loss-Derived LoC 1627 Period. 1629 Results 1631 The measured IGP Convergence time may be influenced by SPF delay, SPF 1632 execution time, and routing and forwarding tables update time 1633 [Po09a]. 1635 Discussion 1637 To measure convergence time, traffic SHOULD start dropping on the 1638 Preferred Egress Interface on the instant the cost is changed by the 1639 Tester. Alternatively the Tester SHOULD record the time the instant 1640 the cost is changed and traffic loss SHOULD only be measured on the 1641 Next-Best Egress Interface. For loss-derived benchmarks the time of 1642 the Start Traffic Instant SHOULD be recorded as well. See Section 1643 4.1. 1645 9. Security Considerations 1647 Benchmarking activities as described in this memo are limited to 1648 technology characterization using controlled stimuli in a laboratory 1649 environment, with dedicated address space and the constraints 1650 specified in the sections above. 1652 The benchmarking network topology will be an independent test setup 1653 and MUST NOT be connected to devices that may forward the test 1654 traffic into a production network, or misroute traffic to the test 1655 management network. 1657 Further, benchmarking is performed on a "black-box" basis, relying 1658 solely on measurements observable external to the DUT/SUT. 1660 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 1661 benchmarking purposes. Any implications for network security arising 1662 from the DUT/SUT SHOULD be identical in the lab and in production 1663 networks. 1665 10. IANA Considerations 1667 This document requires no IANA considerations. 1669 11. Acknowledgements 1671 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1672 Peter De Vriendt, Anuj Dewagan and the BMWG for their contributions 1673 to this work. 1675 12. Normative References 1677 [Br91] Bradner, S., "Benchmarking terminology for network 1678 interconnection devices", RFC 1242, July 1991. 1680 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1681 Requirement Levels", BCP 14, RFC 2119, March 1997. 1683 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1684 Network Interconnect Devices", RFC 2544, March 1999. 1686 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1687 environments", RFC 1195, December 1990. 1689 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1690 IPv6", RFC 5340, July 2008. 1692 [De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1693 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1694 November 2002. 1696 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1697 October 2008. 1699 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1700 Metrics", RFC 3357, August 2002. 1702 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1703 Devices", RFC 2285, February 1998. 1705 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1707 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1708 "Terminology for Benchmarking Network-layer Traffic Control 1709 Mechanisms", RFC 4689, October 2006. 1711 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1712 IGP Data Plane Route Convergence", 1713 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1714 progress), March 2009. 1716 [Po10t] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology 1717 for Benchmarking Link-State IGP Data Plane Route 1718 Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-20 1719 (work in progress), March 2010. 1721 Authors' Addresses 1723 Scott Poretsky 1724 Allot Communications 1725 67 South Bedford Street, Suite 400 1726 Burlington, MA 01803 1727 USA 1729 Phone: + 1 508 309 2179 1730 Email: sporetsky@allot.com 1732 Brent Imhoff 1733 Juniper Networks 1734 1194 North Mathilda Ave 1735 Sunnyvale, CA 94089 1736 USA 1738 Phone: + 1 314 378 2571 1739 Email: bimhoff@planetspork.com 1741 Kris Michielsen 1742 Cisco Systems 1743 6A De Kleetlaan 1744 Diegem, BRABANT 1831 1745 Belgium 1747 Email: kmichiel@cisco.com