idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-term-23.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 16, 2011) is 4815 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: August 13, 2011 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 February 16, 2011 10 Terminology for Benchmarking Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-term-23 13 Abstract 15 This document describes the terminology for benchmarking link-state 16 Interior Gateway Protocol (IGP) route convergence. The terminology 17 is to be used for benchmarking IGP convergence time through 18 externally observable (black box) data plane measurements. The 19 terminology can be applied to any link-state IGP, such as 20 Intermediate System to Intermediate System (IS-IS) and Open Shortest 21 Path First (OSPF). 23 Status of this Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on August 13, 2011. 40 Copyright Notice 42 Copyright (c) 2011 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 This document may contain material from IETF Documents or IETF 56 Contributions published or made publicly available before November 57 10, 2008. The person(s) controlling the copyright in some of this 58 material may not have granted the IETF Trust the right to allow 59 modifications of such material outside the IETF Standards Process. 60 Without obtaining an adequate license from the person(s) controlling 61 the copyright in such materials, this document may not be modified 62 outside the IETF Standards Process, and derivative works of it may 63 not be created outside the IETF Standards Process, except to format 64 it for publication as an RFC or to translate it into languages other 65 than English. 67 Table of Contents 69 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 4 70 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 4 71 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 72 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 4 73 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 5 74 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 5 75 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 6 76 3.2.1. Traffic Start Instant . . . . . . . . . . . . . . . . 6 77 3.2.2. Convergence Event Instant . . . . . . . . . . . . . . 6 78 3.2.3. Convergence Recovery Instant . . . . . . . . . . . . . 7 79 3.2.4. First Route Convergence Instant . . . . . . . . . . . 7 80 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 8 81 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 8 82 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 9 83 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 9 84 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 9 85 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 9 86 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 10 87 3.4.4. Next-Best Egress Interface . . . . . . . . . . . . . . 10 88 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 11 89 3.5.1. Rate-Derived Method . . . . . . . . . . . . . . . . . 11 90 3.5.2. Loss-Derived Method . . . . . . . . . . . . . . . . . 13 91 3.5.3. Route-Specific Loss-Derived Method . . . . . . . . . . 14 92 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15 93 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 15 94 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 16 95 3.6.3. Route-Specific Convergence Time . . . . . . . . . . . 17 96 3.6.4. Loss-Derived Convergence Time . . . . . . . . . . . . 18 97 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 19 98 3.6.6. Loss-Derived Loss of Connectivity Period . . . . . . . 20 99 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 21 100 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 21 101 3.7.2. Convergence Packet Loss . . . . . . . . . . . . . . . 21 102 3.7.3. Connectivity Packet Loss . . . . . . . . . . . . . . . 22 103 3.7.4. Packet Sampling Interval . . . . . . . . . . . . . . . 22 104 3.7.5. Sustained Convergence Validation Time . . . . . . . . 23 105 3.7.6. Forwarding Delay Threshold . . . . . . . . . . . . . . 24 106 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 24 107 3.8.1. Impaired Packet . . . . . . . . . . . . . . . . . . . 24 108 4. Security Considerations . . . . . . . . . . . . . . . . . . . 24 109 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 110 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 111 7. Normative References . . . . . . . . . . . . . . . . . . . . . 25 112 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 114 1. Introduction and Scope 116 This document is a companion to [Po11m] which the methodology to be 117 used for benchmarking link-state Interior Gateway Protocol (IGP) 118 Convergence by observing the data plane. The purpose of this 119 document is to introduce new terms required to complete execution of 120 the Link-State IGP Data Plane Route Convergence methodology [Po11m]. 122 IGP convergence time is measured by observing the dataplane through 123 the Device Under Test (DUT) at the Tester. The methodology and 124 terminology to be used for benchmarking IGP Convergence can be 125 applied to IPv4 and IPv6 traffic and link-state IGPs such as 126 Intermediate System to Intermediate System (IS-IS) [Ca90][Ho08], Open 127 Shortest Path First (OSPF) [Mo98][Co08], and others. 129 2. Existing Definitions 131 This document uses existing terminology defined in other IETF 132 documents. Examples include, but are not limited to: 134 Throughput [Ref.[Br91], section 3.17] 135 Offered Load [Ref.[Ma98], section 3.5.2] 136 Forwarding Rate [Ref.[Ma98], section 3.6.1] 137 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 138 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 139 Out-of-Order Packet [Ref.[Po06], section 3.3.4] 140 Duplicate Packet [Ref.[Po06], section 3.3.5] 141 Stream [Ref.[Po06], section 3.3.2] 142 Forwarding Delay [Ref.[Po06], section 3.2.4] 143 IP Packet Delay Variation (IPDV) [Ref.[De02], section 1.2] 144 Loss Period [Ref.[Ko02], section 4] 146 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 147 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 148 document are to be interpreted as described in BCP 14, RFC 2119 149 [Br97]. RFC 2119 defines the use of these key words to help make the 150 intent of standards track documents as clear as possible. While this 151 document uses these keywords, this document is not a standards track 152 document. 154 3. Term Definitions 156 3.1. Convergence Types 157 3.1.1. Route Convergence 159 Definition: 161 The process of updating all components of the router, including the 162 Routing Information Base (RIB) and Forwarding Information Base (FIB), 163 along with software and hardware tables, with the most recent route 164 change(s) such that forwarding for a route entry is successful on the 165 Next-Best Egress Interface [Section 3.4.4]. 167 Discussion: 169 In general IGP convergence does not necessarily result in a change in 170 forwarding. But the test cases in [Po11m] are specified such that 171 the IGP convergence results in a change of egress interface for the 172 measurement dataplane traffic. Due to this property of the test case 173 specifications, Route Convergence can be observed externally by the 174 rerouting of the measurement dataplane traffic to the Next-best 175 Egress Interface [Section 3.4.4]. 177 Measurement Units: N/A 179 See Also: 181 Next-Best Egress Interface, Full Convergence 183 3.1.2. Full Convergence 185 Definition: 187 Route Convergence for all routes in the Forwarding Information Base 188 (FIB). 190 Discussion: 192 In general IGP convergence does not necessarily result in a change in 193 forwarding. But the test cases in [Po11m] are specified such that 194 the IGP convergence results in a change of egress interface for the 195 measurement dataplane traffic. Due to this property of the test 196 cases specifications, Full Convergence can be observed externally by 197 the rerouting of the measurement dataplane traffic to the Next-best 198 Egress Interface [Section 3.4.4]. 200 Measurement Units: N/A 202 See Also: 204 Next-Best Egress Interface, Route Convergence 206 3.2. Instants 208 3.2.1. Traffic Start Instant 210 Definition: 212 The time instant the Tester sends out the first data packet to the 213 Device Under Test (DUT). 215 Discussion: 217 If using the Loss-Derived Method [Section 3.5.2] or the Route- 218 Specific Loss-Derived Method [Section 3.5.3] to benchmark IGP 219 convergence time, and the applied Convergence Event [Section 3.7.1] 220 does not cause instantaneous traffic loss for all routes at the 221 Convergence Event Instant [Section 3.2.2] then the Tester SHOULD 222 collect a timestamp on the Traffic Start Instant in order to measure 223 the period of time between the Traffic Start Instant and Convergence 224 Event Instant. 226 Measurement Units: 228 seconds (and fractions), reported with resolution sufficient to 229 distinguish between different instants 231 See Also: 233 Loss-Derived Method, Route-Specific Loss-Derived Method, Convergence 234 Event, Convergence Event Instant 236 3.2.2. Convergence Event Instant 238 Definition: 240 The time instant that a Convergence Event [Section 3.7.1] occurs. 242 Discussion: 244 If the Convergence Event [Section 3.7.1] causes instantaneous traffic 245 loss on the Preferred Egress Interface [Section 3.4.3], the 246 Convergence Event Instant is observable from the data plane as the 247 instant that no more packets are received on the Preferred Egress 248 Interface. 250 The Tester SHOULD collect a timestamp on the Convergence Event 251 Instant if it the Convergence Event does not cause instantaneous 252 traffic loss on the Preferred Egress Interface [Section 3.4.3]. 254 Measurement Units: 256 seconds (and fractions), reported with resolution sufficient to 257 distinguish between different instants 259 See Also: 261 Convergence Event, Preferred Egress Interface 263 3.2.3. Convergence Recovery Instant 265 Definition: 267 The time instant that Full Convergence [Section 3.1.2] has completed. 269 Discussion: 271 The Full Convergence completed state MUST be maintained for an 272 interval of duration equal to the Sustained Convergence Validation 273 Time [Section 3.7.5] in order to validate the Convergence Recovery 274 Instant. 276 The Convergence Recovery Instant is observable from the data plane as 277 the instant the Device Under Test (DUT) forwards traffic to all 278 destinations over the Next-Best Egress Interface [Section 3.4.4] 279 without impairments. 281 Measurement Units: 283 seconds (and fractions), reported with resolution sufficient to 284 distinguish between different instants 286 See Also: 288 Sustained Convergence Validation Time, Full Convergence, Next-Best 289 Egress Interface 291 3.2.4. First Route Convergence Instant 293 Definition: 295 The time instant the first route entry completes Route Convergence 296 [Section 3.1.1] 298 Discussion: 300 Any route may be the first to complete Route Convergence. The First 301 Route Convergence Instant is observable from the data plane as the 302 instant that the first packet that is not an Impaired Packet 303 [Section 3.8.1] is received from the Next-Best Egress Interface 304 [Section 3.4.4] or, for the test cases with Equal Cost Multi-Path 305 (ECMP) or Parallel Links, the instant that the Forwarding Rate on the 306 Next-Best Egress Interface [Section 3.4.4] starts to increase. 308 Measurement Units: 310 seconds (and fractions), reported with resolution sufficient to 311 distinguish between different instants 313 See Also: 315 Route Convergence, Impaired Packet, Next-Best Egress Interface 317 3.3. Transitions 319 3.3.1. Convergence Event Transition 321 Definition: 323 A time interval following a Convergence Event [Section 3.7.1] in 324 which Forwarding Rate on the Preferred Egress Interface 325 [Section 3.4.3] gradually reduces to zero. 327 Discussion: 329 The Forwarding Rate during a Convergence Event Transition may or may 330 not decrease linearly. 332 The Forwarding Rate observed on the Device Under Test (DUT) egress 333 interface(s) may or may not decrease to zero. 335 The Offered Load, the number of routes, and the Packet Sampling 336 Interval [Section 3.7.4] influence the observations of the 337 Convergence Event Transition using the Rate-Derived Method 338 [Section 3.5.1]. 340 Measurement Units: seconds (and fractions) 342 See Also: 344 Convergence Event, Preferred Egress Interface, Packet Sampling 345 Interva, Rate-Derived Method 347 3.3.2. Convergence Recovery Transition 349 Definition: 351 A time interval following the First Route Convergence Instant 352 [Section 3.4.4] in which Forwarding Rate on the Device Under Test 353 (DUT) egress interface(s) gradually increases to equal the Offered 354 Load. 356 Discussion: 358 The Forwarding Rate observed during a Convergence Recovery Transition 359 may or may not increase linearly. 361 The Offered Load, the number of routes, and the Packet Sampling 362 Interval [Section 3.7.4] influence the observations of the 363 Convergence Recovery Transition using the Rate-Derived Method 364 [Section 3.5.1]. 366 Measurement Units: seconds (and fractions) 368 See Also: 370 First Route Convergence Instant, Packet Sampling Interva, Rate- 371 Derived Method 373 3.4. Interfaces 375 3.4.1. Local Interface 377 Definition: 379 An interface on the Device Under Test (DUT). 381 Discussion: 383 A failure of a Local Interface indicates that the failure occurred 384 directly on the Device Under Test (DUT). 386 Measurement Units: N/A 388 See Also: Remote Interface 390 3.4.2. Remote Interface 392 Definition: 394 An interface on a neighboring router that is not directly connected 395 to any interface on the Device Under Test (DUT). 397 Discussion: 399 A failure of a Remote Interface indicates that the failure occurred 400 on a neighbor router's interface that is not directly connected to 401 the Device Under Test (DUT). 403 Measurement Units: N/A 405 See Also: Local Interface 407 3.4.3. Preferred Egress Interface 409 Definition: 411 The outbound interface from the Device Under Test (DUT) for traffic 412 routed to the preferred next-hop. 414 Discussion: 416 The Preferred Egress Interface is the egress interface prior to a 417 Convergence Event [Section 3.7.1]. 419 Measurement Units: N/A 421 See Also: Convergence Event, Next-Best Egress Interface 423 3.4.4. Next-Best Egress Interface 425 Definition: 427 The outbound interface or set of outbound interfaces in an Equal Cost 428 Multipath (ECMP) set or parallel link set of the Device Under Test 429 (DUT) for traffic routed to the second-best next-hop. 431 Discussion: 433 The Next-Best Egress Interface becomes the egress interface after a 434 Convergence Event [Section 3.4.4]. 436 For the test cases in [Po11m] using test topologies with an ECMP set 437 or parallel link set, the term Preferred Egress Interface refers to 438 all members of the link set. 440 Measurement Units: N/A 442 See Also: Convergence Event, Preferred Egress Interface 444 3.5. Benchmarking Methods 446 3.5.1. Rate-Derived Method 448 Definition: 450 The method to calculate convergence time benchmarks from observing 451 Forwarding Rate each Packet Sampling Interval [Section 3.7.4]. 453 Discussion: 455 Figure 1 shows an example of the Forwarding Rate change in time 456 during convergence as observed when using the Rate-Derived Method. 458 ^ Traffic Convergence 459 Fwd | Start Recovery 460 Rate | Instant Instant 461 | Offered ^ ^ 462 | Load --> ----------\ /----------- 463 | \ /<--- Convergence 464 | \ Packet / Recovery 465 | Convergence --->\ Loss / Transition 466 | Event \ / 467 | Transition \---------/ <-- Max Packet Loss 468 | 469 +---------------------------------------------------------> 470 ^ ^ time 471 Convergence First Route 472 Event Instant Convergence Instant 474 Figure 1: Rate-Derived Convergence Graph 476 To enable collecting statistics of Out-of-Order Packets per flow (See 477 [Th00], Section 3) the Offered Load SHOULD consist of multiple 478 Streams [Po06] and each Stream SHOULD consist of a single flow . If 479 sending multiple Streams, the measured traffic statistics for all 480 Streams MUST be added together. 482 The destination addresses for the Offered Load MUST be distributed 483 such that all routes or a statistically representative subset of all 484 routes are matched and each of these routes is offered an equal share 485 of the Offered Load. It is RECOMMENDED to send traffic to all 486 routes, but a statistically representative subset of all routes can 487 be used if required. 489 At least one packet per route for all routes matched in the Offered 490 Load MUST be offered to the DUT within each Packet Sampling Interval. 491 For maximum accuracy the value for the Packet Sampling Interval 492 SHOULD be as small as possible, but the presence of IP Packet Delay 493 Variation (IPDV) [De02] may enforce using a larger Packet Sampling 494 Interval. 496 The Offered Load, IPDV, the number of routes, and the Packet Sampling 497 Interval influence the observations for the Rate-Derived Method. It 498 may be difficult to identify the different convergence time instants 499 in the Rate-Derived Convergence Graph. For example, it is possible 500 that a Convergence Event causes the Forwarding Rate to drop to zero, 501 while this may not be observed in the Forwarding Rate measurements if 502 the Packet Sampling Interval is too large. 504 IPDV causes fluctuations in the number of received packets during 505 each Packet Sampling Interval. To account for the presence of IPDV 506 in determining if a convergence instant has been reached, Forwarding 507 Delay SHOULD be observed during each Packet Sampling Interval. The 508 minimum and maximum number of packets expected in a Packet Sampling 509 Interval in presence of IPDV can be calculated with Equation 1. 511 number of packets expected in a Packet Sampling Interval 512 in presence of IP Packet Delay Variation 513 = expected number of packets without IP Packet Delay Variation 514 +/-( (maxDelay - minDelay) * Offered Load) 515 with minDelay and maxDelay the minimum resp. maximum Forwarding Delay 516 of packets received during the Packet Sampling Interval 518 Equation 1 520 To determine if a convergence instant has been reached the number of 521 packets received in a Packet Sampling Interval is compared with the 522 range of expected number of packets calculated in Equation 1. 524 If packets are going over multiple ECMP members and one or more of 525 the members has failed then the number of received packets during 526 each Packet Sampling Interval may vary, even excluding presence of 527 IPDV. To prevent fluctuation of the number of received packets 528 during each Packet Sampling Interval for this reason, the Packet 529 Sampling Interval duration SHOULD be a whole multiple of the time 530 between two consecutive packets sent to the same destination. 532 Metrics measured at the Packet Sampling Interval MUST include 533 Forwarding Rate and Impaired Packet count. 535 To measure convergence time benchmarks for Convergence Events 536 [Section 3.7.1] that do not cause instantaneous traffic loss for all 537 routes at the Convergence Event Instant, the Tester SHOULD collect a 538 timestamp of the Convergence Event Instant [Section 3.2.2] and the 539 Tester SHOULD observe Forwarding Rate separately on the Next-Best 540 Egress Interface. 542 Since the Rate-Derived Method does not distinguish between individual 543 traffic destinations, it SHOULD NOT be used for any route specific 544 measurements. Therefor Rate-Derived Method SHOULD NOT be used to 545 benchmark Route Loss of Connectivity Period [Section 3.6.5]. 547 Measurement Units: N/A 549 See Also: 551 Packet Sampling Interval, Convergence Event, Convergence Event 552 Instant, Next-Best Egress Interface, Route Loss of Connectivity 553 Period 555 3.5.2. Loss-Derived Method 557 Definition: 559 The method to calculate the Loss-Derived Convergence Time 560 [Section 3.6.4] and Loss-Derived Loss of Connectivity Period 561 [Section 3.6.6] benchmarks from the amount of Impaired Packets 562 [Section 3.8.1]. 564 Discussion: 566 To enable collecting statistics of Out-of-Order Packets per flow (See 567 [Th00], Section 3) the Offered Load SHOULD consist of multiple 568 Streams [Po06] and each Stream SHOULD consist of a single flow . If 569 sending multiple Streams, the measured traffic statistics for all 570 Streams MUST be added together. 572 The destination addresses for the Offered Load MUST be distributed 573 such that all routes or a statistically representative subset of all 574 routes are matched and each of these routes is offered an equal share 575 of the Offered Load. It is RECOMMENDED to send traffic to all 576 routes, but a statistically representative subset of all routes can 577 be used if required. 579 Loss-Derived Method SHOULD always be combined with Rate-Derived 580 Method in order to observe Full Convergence completion. The total 581 amount of Convergence Packet Loss is collected after Full Convergence 582 completion. 584 To measure convergence time and loss of connectivity benchmarks for 585 Convergence Events that cause instantaneous traffic loss for all 586 routes at the Convergence Event Instant, the Tester SHOULD observe 587 Impaired Packet count on all DUT egress interfaces (see Connectivity 588 Packet Loss [Section 3.7.3]). 590 To measure convergence time benchmarks for Convergence Events that do 591 not cause instantaneous traffic loss for all routes at the 592 Convergence Event Instant, the Tester SHOULD collect timestamps of 593 the Start Traffic Instant and of the Convergence Event Instant, and 594 the Tester SHOULD observe Impaired Packet count separately on the 595 Next-Best Egress Interface (See Convergence Packet Loss 596 [Section 3.7.2]). 598 Since Loss-Derived Method does not distinguish between traffic 599 destinations and the Impaired Packet statistics are only collected 600 after Full Convergence completion, this method can only be used to 601 measure average values over all routes. For these reasons Loss- 602 Derived Method can only be used to benchmark Loss-Derived Convergence 603 Time [Section 3.6.4] and Loss-Derived Loss of Connectivity Period 604 [Section 3.6.6]. 606 Note that the Loss-Derived Method measures an average over all 607 routes, including the routes that may not be impacted by the 608 Convergence Event, such as routes via non-impacted members of ECMP or 609 parallel links. 611 Measurement Units: N/A 613 See Also: 615 Loss-Derived Convergence Time, Loss-Derived Loss of Connectivity 616 Period, Connectivity Packet Loss, Convergence Packet Loss 618 3.5.3. Route-Specific Loss-Derived Method 620 Definition: 622 The method to calculate the Route-Specific Convergence Time 623 [Section 3.6.3] benchmark from the amount of Impaired Packets 624 [Section 3.8.1] during convergence for a specific route entry. 626 Discussion: 628 To benchmark Route-Specific Convergence Time, the Tester provides an 629 Offered Load that consists of multiple Streams [Po06]. Each Stream 630 has a single destination address matching a different route entry, 631 for all routes or a statistically representative subset of all 632 routes. Each Stream SHOULD consist of a single flow (See [Th00], 633 Section 3). Convergence Packet Loss is measured for each Stream 634 separately. 636 Route-Specific Loss-Derived Method SHOULD always be combined with 637 Rate-Derived Method in order to observe Full Convergence completion. 638 The total amount of Convergence Packet Loss [Section 3.7.2] for each 639 Stream is collected after Full Convergence completion. 641 Route-Specific Loss-Derived Method is the RECOMMENDED method to 642 measure convergence time benchmarks. 644 To measure convergence time and loss of connectivity benchmarks for 645 Convergence Events that cause instantaneous traffic loss for all 646 routes at the Convergence Event Instant, the Tester SHOULD observe 647 Impaired Packet count on all DUT egress interfaces (see Connectivity 648 Packet Loss [Section 3.7.3]). 650 To measure convergence time benchmarks for Convergence Events that do 651 not cause instantaneous traffic loss for all routes at the 652 Convergence Event Instant, the Tester SHOULD collect timestamps of 653 the Start Traffic Instant and of the Convergence Event Instant, and 654 the Tester SHOULD observe packet loss separately on the Next-Best 655 Egress Interface (See Convergence Packet Loss [Section 3.7.2]). 657 Since Route-Specific Loss-Derived Method uses traffic streams to 658 individual routes, it observes Impaired Packet count as it would be 659 experienced by a network user. For this reason Route-Specific Loss- 660 Derived Method is RECOMMENDED to measure Route-Specific Convergence 661 Time benchmarks and Route Loss of Connectivity Period benchmarks. 663 Measurement Units: N/A 665 See Also: 667 Route-Specific Convergence Time, Route Loss of Connectivity Period, 668 Connectivity Packet Loss, Convergence Packet Loss 670 3.6. Benchmarks 672 3.6.1. Full Convergence Time 674 Definition: 676 The time duration of the period between the Convergence Event Instant 677 and the Convergence Recovery Instant as observed using the Rate- 678 Derived Method. 680 Discussion: 682 Using the Rate-Derived Method, Full Convergence Time can be 683 calculated as the time difference between the Convergence Event 684 Instant and the Convergence Recovery Instant, as shown in Equation 2. 686 Full Convergence Time = 687 Convergence Recovery Instant - Convergence Event Instant 689 Equation 2 691 The Convergence Event Instant can be derived from the Forwarding Rate 692 observation or from a timestamp collected by the Tester. 694 For the test cases described in [Po11m], it is expected that Full 695 Convergence Time equals the maximum Route-Specific Convergence Time 696 when benchmarking all routes in FIB using the Route-Specific Loss- 697 Derived Method. 699 It is not possible to measure Full Convergence Time using the Loss- 700 Derived Method. 702 Measurement Units: seconds (and fractions) 704 See Also: 706 Full Convergence, Rate-Derived Method, Route-Specific Loss-Derived 707 Method, Convergence Event Instant, Convergence Recovery Instant 709 3.6.2. First Route Convergence Time 711 Definition: 713 The duration of the period between the Convergence Event Instant and 714 the First Route Convergence Instant as observed using the Rate- 715 Derived Method. 717 Discussion: 719 Using the Rate-Derived Method, First Route Convergence Time can be 720 calculated as the time difference between the Convergence Event 721 Instant and the First Route Convergence Instant, as shown with 722 Equation 3. 724 First Route Convergence Time = 725 First Route Convergence Instant - Convergence Event Instant 727 Equation 3 729 The Convergence Event Instant can be derived from the Forwarding Rate 730 observation or from a timestamp collected by the Tester. 732 For the test cases described in [Po11m], it is expected that First 733 Route Convergence Time equals the minimum Route-Specific Convergence 734 Time when benchmarking all routes in FIB using the Route-Specific 735 Loss-Derived Method. 737 It is not possible to measure First Route Convergence Time using the 738 Loss-Derived Method. 740 Measurement Units: seconds (and fractions) 742 See Also: 744 Rate-Derived Method, Route-Specific Loss-Derived Method, Convergence 745 Event Instant, First Route Convergence Instant 747 3.6.3. Route-Specific Convergence Time 749 Definition: 751 The amount of time it takes for Route Convergence to be completed for 752 a specific route, as calculated from the amount of Impaired Packets 753 [Section 3.8.1] during convergence for a single route entry. 755 Discussion: 757 Route-Specific Convergence Time can only be measured using the Route- 758 Specific Loss-Derived Method. 760 If the applied Convergence Event causes instantaneous traffic loss 761 for all routes at the Convergence Event Instant, Connectivity Packet 762 Loss should be observed. Connectivity Packet Loss is the combined 763 Impaired Packet count observed on Preferred Egress Interface and 764 Next-Best Egress Interface. When benchmarking Route-Specific 765 Convergence Time, Connectivity Packet Loss is measured and Equation 4 766 is applied for each measured route. The calculation is equal to 767 Equation 8 in Section 3.6.5. 769 Route-Specific Convergence Time = 770 Connectivity Packet Loss for specific route/Offered Load per route 772 Equation 4 774 If the applied Convergence Event does not cause instantaneous traffic 775 loss for all routes at the Convergence Event Instant, then the Tester 776 SHOULD collect timestamps of the Traffic Start Instant and of the 777 Convergence Event Instant, and the Tester SHOULD observe Convergence 778 Packet Loss separately on the Next-Best Egress Interface. When 779 benchmarking Route-Specific Convergence Time, Convergence Packet Loss 780 is measured and Equation 5 is applied for each measured route. 782 Route-Specific Convergence Time = 783 Convergence Packet Loss for specific route/Offered Load per route 784 - (Convergence Event Instant - Traffic Start Instant) 786 Equation 5 788 The Route-Specific Convergence Time benchmarks enable minimum, 789 maximum, average, and median convergence time measurements to be 790 reported by comparing the results for the different route entries. 791 It also enables benchmarking of convergence time when configuring a 792 priority value for route entry(ies). Since multiple Route-Specific 793 Convergence Times can be measured it is possible to have an array of 794 results. The format for reporting Route-Specific Convergence Time is 795 provided in [Po11m]. 797 Measurement Units: seconds (and fractions) 799 See Also: 801 Route-Specific Loss-Derived Method, Convergence Event, Convergence 802 Event Instant, Convergence Packet Loss, Connectivity Packet Loss, 803 Route Convergence 805 3.6.4. Loss-Derived Convergence Time 807 Definition: 809 The average Route Convergence time for all routes in the Forwarding 810 Information Base (FIB), as calculated from the amount of Impaired 811 Packets [Section 3.8.1] during convergence. 813 Discussion: 815 Loss-Derived Convergence Time is measured using the Loss-Derived 816 Method. 818 If the applied Convergence Event causes instantaneous traffic loss 819 for all routes at the Convergence Event Instant, Connectivity Packet 820 Loss [Section 3.7.3] should be observed. Connectivity Packet Loss is 821 the combined Impaired Packet count observed on Preferred Egress 822 Interface and Next-Best Egress Interface. When benchmarking Loss- 823 Derived Convergence Time, Connectivity Packet Loss is measured and 824 Equation 6 is applied. 826 Loss-Derived Convergence Time = 827 Connectivity Packet Loss/Offered Load 829 Equation 6 831 If the applied Convergence Event does not cause instantaneous traffic 832 loss for all routes at the Convergence Event Instant, then the Tester 833 SHOULD collect timestamps of the Start Traffic Instant and of the 834 Convergence Event Instant and the Tester SHOULD observe Convergence 835 Packet Loss [Section 3.7.2] separately on the Next-Best Egress 836 Interface. When benchmarking Loss-Derived Convergence Time, 837 Convergence Packet Loss is measured and Equation 7 is applied. 839 Loss-Derived Convergence Time = 840 Convergence Packet Loss/Offered Load 841 - (Convergence Event Instant - Traffic Start Instant) 843 Equation 7 845 Measurement Units: seconds (and fractions) 847 See Also: 849 Convergence Packet Loss, Connectivity Packet Loss, Route Convergence, 850 Loss-Derived Method 852 3.6.5. Route Loss of Connectivity Period 854 Definition: 856 The time duration of packet impairments for a specific route entry 857 following a Convergence Event until Full Convergence completion, as 858 observed using the Route-Specific Loss-Derived Method. 860 Discussion: 862 In general the Route Loss of Connectivity Period is not equal to the 863 Route-Specific Convergence Time. If the DUT continues to forward 864 traffic to the Preferred Egress Interface after the Convergence Event 865 is applied then the Route Loss of Connectivity Period will be smaller 866 than the Route-Specific Convergence Time. This is also specifically 867 the case after reversing a failure event. 869 The Route Loss of Connectivity Period may be equal to the Route- 870 Specific Convergence Time if, as a characteristic of the Convergence 871 Event, traffic for all routes starts dropping instantaneously on the 872 Convergence Event Instant. See discussion in [Po11m]. 874 For the test cases described in [Po11m] the Route Loss of 875 Connectivity Period is expected to be a single Loss Period [Ko02]. 877 When benchmarking Route Loss of Connectivity Period, Connectivity 878 Packet Loss is measured for each route and Equation 8 is applied for 879 each measured route entry. The calculation is equal to Equation 4 in 880 Section 3.6.3. 882 Route Loss of Connectivity Period = 883 Connectivity Packet Loss for specific route/Offered Load per route 885 Equation 8 887 Route Loss of Connectivity Period SHOULD be measured using Route- 888 Specific Loss-Derived Method. 890 Measurement Units: seconds (and fractions) 892 See Also: 894 Route-Specific Convergence Time, Route-Specific Loss-Derived Method, 895 Connectivity Packet Loss 897 3.6.6. Loss-Derived Loss of Connectivity Period 899 Definition: 901 The average time duration of packet impairments for all routes 902 following a Convergence Event until Full Convergence completion, as 903 observed using the Loss-Derived Method. 905 Discussion: 907 In general the Loss-Derived Loss of Connectivity Period is not equal 908 to the Loss-Derived Convergence Time. If the DUT continues to 909 forward traffic to the Preferred Egress Interface after the 910 Convergence Event is applied then the Loss-Derived Loss of 911 Connectivity Period will be smaller than the Loss-Derived Convergence 912 Time. This is also specifically the case after reversing a failure 913 event. 915 The Loss-Derived Loss of Connectivity Period may be equal to the 916 Loss-Derived Convergence Time if, as a characteristic of the 917 Convergence Event, traffic for all routes starts dropping 918 instantaneously on the Convergence Event Instant. See discussion in 919 [Po11m]. 921 For the test cases described in [Po11m] each route's Route Loss of 922 Connectivity Period is expected to be a single Loss Period [Ko02]. 924 When benchmarking Loss-Derived Loss of Connectivity Period, 925 Connectivity Packet Loss is measured for all routes and Equation 9 is 926 applied. The calculation is equal to Equation 6 in Section 3.6.4. 928 Loss-Derived Loss of Connectivity Period = 929 Connectivity Packet Loss for all routes/Offered Load 931 Equation 9 933 Loss-Derived Loss of Connectivity Period SHOULD be measured using 934 Loss-Derived Method. 936 Measurement Units: seconds (and fractions) 938 See Also: 940 Loss-Derived Convergence Time, Loss-Derived Method, Connectivity 941 Packet Loss 943 3.7. Measurement Terms 945 3.7.1. Convergence Event 947 Definition: 949 The occurrence of an event in the network that will result in a 950 change in the egress interface of the Device Under Test (DUT) for 951 routed packets. 953 Discussion: 955 All test cases in [Po11m] are defined such that a Convergence Event 956 results in a change of egress interface of the DUT. Local or remote 957 triggers that cause a route calculation which does not result in a 958 change in forwarding are not considered. 960 Measurement Units: N/A 962 See Also: Convergence Event Instant 964 3.7.2. Convergence Packet Loss 966 Definition: 968 The number of Impaired Packets [Section 3.8.1] as observed on the 969 Next-Best Egress Interface of the DUT during convergence. 971 Discussion: 973 An Impaired Packet is considered as a lost packet. 975 Measurement Units: number of packets 977 See Also: 979 Connectivity Packet Loss 981 3.7.3. Connectivity Packet Loss 983 Definition: 985 The number of Impaired Packets observed on all DUT egress interfaces 986 during convergence. 988 Discussion: 990 An Impaired Packet is considered as a lost packet. Connectivity 991 Packet Loss is equal to Convergence Packet Loss if the Convergence 992 Event causes instantaneous traffic loss for all egress interfaces of 993 the DUT except for the Next-Best Egress Interface. 995 Measurement Units: number of packets 997 See Also: 999 Convergence Packet Loss 1001 3.7.4. Packet Sampling Interval 1003 Definition: 1005 The interval at which the Tester (test equipment) polls to make 1006 measurements for arriving packets. 1008 Discussion: 1010 At least one packet per route for all routes matched in the Offered 1011 Load MUST be offered to the DUT within the Packet Sampling Interval. 1012 Metrics measured at the Packet Sampling Interval MUST include 1013 Forwarding Rate and received packets. 1015 Packet Sampling Interval can influence the convergence graph as 1016 observed with the Rate-Derived Method. This is particularly true 1017 when implementations complete Full Convergence in less time than the 1018 Packet Sampling Interval. The Convergence Event Instant and First 1019 Route Convergence Instant may not be easily identifiable and the 1020 Rate-Derived Method may produce a larger than actual convergence 1021 time. 1023 Using a small Packet Sampling Interval in the presence of IPDV [De02] 1024 may cause fluctuations of the Forwarding Rate observation and can 1025 prevent correct observation of the different convergence time 1026 instants. 1028 The value of the Packet Sampling Interval only contributes to the 1029 measurement accuracy of the Rate-Derived Method. For maximum 1030 accuracy the value for the Packet Sampling Interval SHOULD be as 1031 small as possible, but the presence of IPDV may enforce using a 1032 larger Packet Sampling Interval. 1034 Measurement Units: seconds (and fractions) 1036 See Also: Rate-Derived Method 1038 3.7.5. Sustained Convergence Validation Time 1040 Definition: 1042 The amount of time for which the completion of Full Convergence is 1043 maintained without additional Impaired Packets being observed. 1045 Discussion: 1047 The purpose of the Sustained Convergence Validation Time is to 1048 produce convergence benchmarks protected against fluctuation in 1049 Forwarding Rate after the completion of Full Convergence is observed. 1050 The RECOMMENDED Sustained Convergence Validation Time to be used is 1051 the time to send 5 consecutive packets to each destination with a 1052 minimum of 5 seconds. The Benchmarking Methodology Working Group 1053 (BMWG) selected 5 seconds based upon [Br99] which recommends waiting 1054 2 seconds for residual frames to arrive (this is the Forwarding Delay 1055 Threshold for the last packet sent) and 5 seconds for DUT 1056 restabilization. 1058 Measurement Units: seconds (and fractions) 1060 See Also: 1062 Full Convergence, Convergence Recovery Instant 1064 3.7.6. Forwarding Delay Threshold 1066 Definition: 1068 The maximum waiting time threshold used to distinguish between 1069 packets with very long delay and lost packets that will never arrive. 1071 Discussion: 1073 Applying a Forwarding Delay Threshold allows to consider packets with 1074 a too large Forwarding Delay as being lost, as is required for some 1075 applications (e.g. voice, video, etc.). The Forwarding Delay 1076 Threshold is a parameter of the methodology, and it MUST be reported. 1077 [Br99] recommends waiting 2 seconds for residual frames to arrive. 1079 Measurement Units: seconds (and fractions) 1081 See Also: 1083 Convergence Packet Loss, Connectivity Packet Loss 1085 3.8. Miscellaneous Terms 1087 3.8.1. Impaired Packet 1089 Definition: 1091 A packet that experienced at least one of the following impairments: 1092 loss, excessive Forwarding Delay, corruption, duplication, 1093 reordering. 1095 Discussion: 1097 A lost packet, a packet with a Forwarding Delay exceeding the 1098 Forwarding Delay Threshold, a corrupted packet, a Duplicate Packet 1099 [Po06], and an Out-of-Order Packet [Po06] are Impaired Packets. 1101 Packet ordering is observed for each individual flow (See [Th00], 1102 Section 3) of the Offered Load. 1104 Measurement Units: N/A 1106 See Also: Forwarding Delay Threshold 1108 4. Security Considerations 1110 Benchmarking activities as described in this memo are limited to 1111 technology characterization using controlled stimuli in a laboratory 1112 environment, with dedicated address space and the constraints 1113 specified in the sections above. 1115 The benchmarking network topology will be an independent test setup 1116 and MUST NOT be connected to devices that may forward the test 1117 traffic into a production network, or misroute traffic to the test 1118 management network. 1120 Further, benchmarking is performed on a "black-box" basis, relying 1121 solely on measurements observable external to the DUT/SUT. 1123 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 1124 benchmarking purposes. Any implications for network security arising 1125 from the DUT/SUT SHOULD be identical in the lab and in production 1126 networks. 1128 5. IANA Considerations 1130 This document requires no IANA considerations. 1132 6. Acknowledgements 1134 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1135 Peter De Vriendt, Anuj Dewagan, Adrian Farrel, Stewart Bryant, 1136 Francis Dupont, and the Benchmarking Methodology Working Group for 1137 their contributions to this work. 1139 7. Normative References 1141 [Br91] Bradner, S., "Benchmarking terminology for network 1142 interconnection devices", RFC 1242, July 1991. 1144 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1145 Requirement Levels", BCP 14, RFC 2119, March 1997. 1147 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1148 Network Interconnect Devices", RFC 2544, March 1999. 1150 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1151 environments", RFC 1195, December 1990. 1153 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1154 IPv6", RFC 5340, July 2008. 1156 [De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1157 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1158 November 2002. 1160 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1161 October 2008. 1163 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1164 Metrics", RFC 3357, August 2002. 1166 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1167 Devices", RFC 2285, February 1998. 1169 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1171 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1172 "Terminology for Benchmarking Network-layer Traffic Control 1173 Mechanisms", RFC 4689, October 2006. 1175 [Po11m] Poretsky, S., Imhoff, B., and K. Michielsen, "Benchmarking 1176 Methodology for Link-State IGP Data Plane Route 1177 Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-23 1178 (work in progress), January 2011. 1180 [Th00] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and 1181 Multicast Next-Hop Selection", RFC 2991, November 2000. 1183 Authors' Addresses 1185 Scott Poretsky 1186 Allot Communications 1187 67 South Bedford Street, Suite 400 1188 Burlington, MA 01803 1189 USA 1191 Phone: + 1 508 309 2179 1192 Email: sporetsky@allot.com 1193 Brent Imhoff 1194 Juniper Networks 1195 1194 North Mathilda Ave 1196 Sunnyvale, CA 94089 1197 USA 1199 Phone: + 1 314 378 2571 1200 Email: bimhoff@planetspork.com 1202 Kris Michielsen 1203 Cisco Systems 1204 6A De Kleetlaan 1205 Diegem, BRABANT 1831 1206 Belgium 1208 Email: kmichiel@cisco.com