idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-term-18.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 13, 2009) is 5400 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-meth-18 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: January 14, 2010 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 July 13, 2009 10 Terminology for Benchmarking Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-term-18 13 Status of this Memo 15 This Internet-Draft is submitted to IETF in full conformance with the 16 provisions of BCP 78 and BCP 79. This document may contain material 17 from IETF Documents or IETF Contributions published or made publicly 18 available before November 10, 2008. The person(s) controlling the 19 copyright in some of this material may not have granted the IETF 20 Trust the right to allow modifications of such material outside the 21 IETF Standards Process. Without obtaining an adequate license from 22 the person(s) controlling the copyright in such materials, this 23 document may not be modified outside the IETF Standards Process, and 24 derivative works of it may not be created outside the IETF Standards 25 Process, except to format it for publication as an RFC or to 26 translate it into languages other than English. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as Internet- 31 Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/ietf/1id-abstracts.txt. 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html. 44 This Internet-Draft will expire on January 14, 2010. 46 Copyright Notice 48 Copyright (c) 2009 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents in effect on the date of 53 publication of this document (http://trustee.ietf.org/license-info). 54 Please review these documents carefully, as they describe your rights 55 and restrictions with respect to this document. 57 Abstract 59 This document describes the terminology for benchmarking Interior 60 Gateway Protocol (IGP) Route Convergence. The terminology is to be 61 used for benchmarking IGP convergence time through externally 62 observable (black box) data plane measurements. The terminology can 63 be applied to any link-state IGP, such as ISIS and OSPF. 65 Table of Contents 67 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 4 68 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 4 69 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 70 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 5 71 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 5 72 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 5 73 3.1.3. Network Convergence . . . . . . . . . . . . . . . . . 6 74 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 6 75 3.2.1. Convergence Event Instant . . . . . . . . . . . . . . 6 76 3.2.2. Convergence Recovery Instant . . . . . . . . . . . . . 7 77 3.2.3. First Route Convergence Instant . . . . . . . . . . . 7 78 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 8 79 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 8 80 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 9 81 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 9 82 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 9 83 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 10 84 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 10 85 3.4.4. Next-Best Egress Interface . . . . . . . . . . . . . . 10 86 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 11 87 3.5.1. Rate-Derived Method . . . . . . . . . . . . . . . . . 11 88 3.5.2. Loss-Derived Method . . . . . . . . . . . . . . . . . 12 89 3.5.3. Route-Specific Loss-Derived Method . . . . . . . . . . 13 90 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15 91 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 15 92 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 15 93 3.6.3. Route-Specific Convergence Time . . . . . . . . . . . 16 94 3.6.4. Loss-Derived Convergence Time . . . . . . . . . . . . 18 95 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 19 96 3.6.6. Loss-Derived Loss of Connectivity Period . . . . . . . 20 97 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 21 98 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 21 99 3.7.2. Packet Loss . . . . . . . . . . . . . . . . . . . . . 21 100 3.7.3. Convergence Packet Loss . . . . . . . . . . . . . . . 21 101 3.7.4. Connectivity Packet Loss . . . . . . . . . . . . . . . 22 102 3.7.5. Packet Sampling Interval . . . . . . . . . . . . . . . 23 103 3.7.6. Sustained Convergence Validation Time . . . . . . . . 23 104 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 24 105 3.8.1. Stale Forwarding . . . . . . . . . . . . . . . . . . . 24 106 3.8.2. Nested Convergence Event . . . . . . . . . . . . . . . 24 107 4. Security Considerations . . . . . . . . . . . . . . . . . . . 25 108 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 109 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 110 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25 111 7.1. Normative References . . . . . . . . . . . . . . . . . . . 25 112 7.2. Informative References . . . . . . . . . . . . . . . . . . 26 113 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 115 1. Introduction and Scope 117 This draft describes the terminology for benchmarking Link-State 118 Interior Gateway Protocol (IGP) Convergence. The motivation and 119 applicability for this benchmarking is provided in [Po09a]. The 120 methodology to be used for this benchmarking is described in [Po09m]. 121 The purpose of this document is to introduce new terms required to 122 complete execution of the IGP Route Methodology [Po09m]. 124 IGP convergence time is measured on the data plane at the Tester by 125 observing packet loss through the DUT. The methodology and 126 terminology to be used for benchmarking IGP Convergence can be 127 applied to IPv4 and IPv6 traffic and link-state IGPs such as ISIS 128 [Ca90][Ho08], OSPF [Mo98][Co08], and others. 130 2. Existing Definitions 132 This document uses existing terminology defined in other BMWG work. 133 Examples include, but are not limited to: 135 Frame Loss Rate [Ref.[Br91], section 3.6] 136 Throughput [Ref.[Br91], section 3.17] 137 Offered Load [Ref.[Ma98], section 3.5.2] 138 Forwarding Rate [Ref.[Ma98], section 3.6.1] 139 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 140 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 141 Out-of-order Packet [Ref.[Po06], section 3.3.2] 142 Duplicate Packet [Ref.[Po06], section 3.3.3] 143 Packet Reordering [Ref.[Mo06], section 3.3] 144 Stream [Ref.[Po06], section 3.3.2] 145 Flow [Ref.[Po06], section 3.1.5] 146 Forwarding Delay [Ref.[Po06], section 3.2.4] 147 Loss Period [Ref.[Ko02], section 4] 149 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 150 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 151 document are to be interpreted as described in BCP 14, RFC 2119 152 [Br97]. RFC 2119 defines the use of these key words to help make the 153 intent of standards track documents as clear as possible. While this 154 document uses these keywords, this document is not a standards track 155 document. 157 3. Term Definitions 158 3.1. Convergence Types 160 3.1.1. Route Convergence 162 Definition: 164 The process of updating all components of the router, including the 165 Routing Information Base (RIB) and Forwarding Information Base (FIB), 166 along with software and hardware tables, with the most recent route 167 change(s) such that forwarding for a route entry is successful on the 168 Next-Best Egress Interface. 170 Discussion: 172 Route Convergence MUST occur after a Convergence Event. Route 173 Convergence can be observed externally by the rerouting of data 174 traffic for a destination matching a route entry to the Next-best 175 Egress Interface. Completion of Route Convergence may or may not be 176 sustained over time. 178 Measurement Units: N/A 180 Issues: None 182 See Also: 184 Network Convergence, Full Convergence, Convergence Event 186 3.1.2. Full Convergence 188 Definition: 190 Route Convergence for all routes in the FIB. 192 Discussion: 194 Full Convergence MUST occur after a Convergence Event. Full 195 Convergence can be observed externally by the rerouting of data 196 traffic to destinations matching all route entries to the Next-best 197 Egress Interface. Completion of Full Convergence is externally 198 observable from the data plane when the Forwarding Rate of the data 199 plane traffic on the Next-Best Egress Interface equals the Offered 200 Load. 202 Completion of Full Convergence may or may not be sustained over time. 204 Measurement Units: N/A 205 Issues: None 207 See Also: 209 Network Convergence, Route Convergence, Convergence Event, Full 210 Convergence Time, Convergence Recovery Instant 212 3.1.3. Network Convergence 214 Definition: 216 Full Convergence in all routers throughout the network. 218 Discussion: 220 Network Convergence includes all Route Convergence operations for all 221 routers in the network following a Convergence Event. 223 Completion of Network Convergence can be observed by recovery of the 224 network Forwarding Rate to equal the Offered Load, with no Stale 225 Forwarding, and no Blenders [Ca01][Ci03]. 227 Completion of Network Convergence may or may not be sustained over 228 time. 230 Measurement Units: N/A 232 Issues: None 234 See Also: 236 Route Convergence, Full Convergence, Stale Forwarding 238 3.2. Instants 240 3.2.1. Convergence Event Instant 242 Definition: 244 The time instant that a Convergence Event occurs. 246 Discussion: 248 If the Convergence Event causes instantaneous traffic loss on the 249 Preferred Egress Interface, the Convergence Event Instant is 250 observable from the data plane as the instant that the DUT begins to 251 exhibit packet loss. 253 The Tester SHOULD collect a timestamp on the Convergence Event 254 Instant if it is not observable from the data plane. 256 Measurement Units: 258 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 259 microseconds. 261 Issues: None 263 See Also: Convergence Event 265 3.2.2. Convergence Recovery Instant 267 Definition: 269 The time instant that Full Convergence has completed. 271 Discussion: 273 The Full Convergence completed state MUST be maintained for an 274 interval of duration equal to the Sustained Convergence Validation 275 Time in order to validate the Convergence Recovery Instant. 277 The Convergence Recovery Instant is observable from the data plane as 278 the instant the DUT forwards traffic to all destinations over the 279 Next-Best Egress Interface. 281 When using the Rate-Derived Method, the Convergence Recovery Instant 282 falls within the Packet Sampling Interval preceding the first 283 interval where the observed Forwarding Rate on the Next-Best Egress 284 Interface equals the Offered Load. 286 Measurement Units: 288 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 289 microseconds. 291 Issues: None 293 See Also: 295 Sustained Convergence Validation Time, Full Convergence 297 3.2.3. First Route Convergence Instant 299 Definition: 301 The time instant the first route entry completes Route Convergence 302 following a Convergence Event 304 Discussion: 306 Any route may be the first to complete Route Convergence. The First 307 Route Convergence Instant is observable from the data plane as the 308 instant that the first packet is received from the Next-Best Egress 309 Interface. 311 Measurement Units: 313 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 314 microseconds. 316 Issues: None 318 See Also: Route Convergence 320 3.3. Transitions 322 3.3.1. Convergence Event Transition 324 Definition: 326 A time interval following a Convergence Event in which Forwarding 327 Rate on the Preferred Egress Interface gradually reduces to zero. 329 Discussion: 331 The Forwarding Rate during a Convergence Event Transition may not 332 decrease linearly. 334 The Forwarding Rate observed on all DUT egress interfaces may or may 335 not decrease to zero. 337 The Offered Load, the number of routes, and the Packet Sampling 338 Interval influence the observations of the Convergence Event 339 Transition using the Rate-Derived Method. This is further discussed 340 with the term "Rate-Derived Method". 342 Measurement Units: seconds 344 Issues: None 346 See Also: 348 Convergence Event, Rate-Derived Method 350 3.3.2. Convergence Recovery Transition 352 Definition: 354 A time interval following the First Route Convergence Instant in 355 which Forwarding Rate on the Next-Best Egress Interface gradually 356 increases to equal the Offered Load. 358 Discussion: 360 The Forwarding Rate observed during a Convergence Recovery Transition 361 may not increase linearly. 363 The Offered Load, the number of routes, and the Packet Sampling 364 Interval influence the observations of the Convergence Recovery 365 Transition using the Rate-Derived Method. This is further discussed 366 with the term "Rate-Derived Method". 368 Measurement Units: seconds 370 Issues: None 372 See Also: 374 Full Convergence,First Route Convergence Instant, Rate-Derived Method 376 3.4. Interfaces 378 3.4.1. Local Interface 380 Definition: 382 An interface on the DUT. 384 Discussion: 386 A failure of the Local Interface indicates that the failure occurred 387 directly on the DUT. 389 Measurement Units: N/A 391 Issues: None 393 See Also: Remote Interface 395 3.4.2. Remote Interface 397 Definition: 399 An interface on a neighboring router that is not directly connected 400 to any interface on the DUT. 402 Discussion: 404 A failure of a Remote Interface indicates that the failure occurred 405 on a neighbor router's interface that is not directly connected to 406 the DUT. 408 Measurement Units: N/A 410 Issues: None 412 See Also: Local Interface 414 3.4.3. Preferred Egress Interface 416 Definition: 418 The outbound interface from the DUT for traffic routed to the 419 preferred next-hop. 421 Discussion: 423 The Preferred Egress Interface is the egress interface prior to a 424 Convergence Event. 426 Measurement Units: N/A 428 Issues: None 430 See Also: Next-Best Egress Interface 432 3.4.4. Next-Best Egress Interface 434 Definition: 436 The outbound interface from the DUT for traffic routed to the second- 437 best next-hop. 439 Discussion: 441 The Next-Best Egress Interface becomes the egress interface after a 442 Convergence Event. 444 The Next-Best Egress Interface is of the same media type and link 445 speed as the Preferred Egress Interface. 447 Measurement Units: N/A 449 Issues: None 451 See Also: Preferred Egress Interface 453 3.5. Benchmarking Methods 455 3.5.1. Rate-Derived Method 457 Definition: 459 The method to calculate convergence time benchmarks from observing 460 Forwarding Rate each Packet Sampling Interval. 462 Discussion: 464 Figure 1 shows an example of the Forwarding Rate change in time 465 during convergence as observed when using the Rate-Derived Method. 467 ^ Convergence 468 Fwd | Recovery 469 Rate | Instant 470 | Offered ^ 471 | Load --> ----------\ /----------- 472 | \ /<--- Convergence 473 | \ Packet / Recovery 474 | Convergence --->\ Loss / Transition 475 | Event \ / 476 | Transition \---------/ <-- Max Packet Loss 477 | 478 +---------------------------------------------------------> 479 ^ ^ time 480 Convergence First Route 481 Event Instant Convergence Instant 483 Figure 1: Rate-Derived Convergence Graph 485 The Offered Load SHOULD consists of a single Stream [Po06]. If 486 sending multiple Streams, the measured traffic rate statistics for 487 all Streams MUST be added together. 489 The destination addresses for the Offered Load MUST be distributed 490 such that all routes in the FIB are matched and each route is offered 491 an equal share of the total Offered Load. 493 At least one packet per route in the FIB for all routes in the FIB 494 MUST be offered to the DUT within each Packet Sampling Interval. 496 The Offered Load, the number of routes, and the Packet Sampling 497 Interval influence the observations for the Rate-Derived Method. It 498 may be difficult to identify the different convergence time instants 499 in the Rate-Derived Convergence Graph. For example, it is possible 500 that a Convergence Event causes the Forwarding Rate to drop to zero, 501 while this may not be observed in the Forwarding Rate measurements if 502 the Packet Sampling Interval is too high. 504 Metrics measured at the Packet Sampling Interval MUST include 505 Forwarding Rate and packet loss. 507 Rate-Derived Method is a RECOMMENDED method to measure convergence 508 time benchmarks. 510 To measure convergence time benchmarks for Convergence Events that do 511 not cause instantaneous traffic loss for all routes at the 512 Convergence Event Instant, the Tester SHOULD collect a timestamp of 513 the Convergence Event Instant and the Tester SHOULD observe 514 Forwarding Rate seperately on the Next-Best Egress Interface. 516 Since the Rate-Derived Method does not distinguish between individual 517 traffic destinations, it SHOULD NOT be used for any route specific 518 measurements. Therefor Rate-Derived Method SHOULD NOT be used to 519 benchmark Route Loss of Connectivity Period. 521 Measurement Units: N/A 523 Issues: None 525 See Also: 527 Packet Sampling Interval, Convergence Event, Convergence Event 528 Instant, Full Convergence 530 3.5.2. Loss-Derived Method 532 Definition: 534 The method to calculate the Loss-Derived Convergence Time and Loss- 535 Derived Loss of Connectivity Period benchmarks from the amount of 536 packet loss. 538 Discussion: 540 The Offered Load SHOULD consists of a single Stream [Po06]. If 541 sending multiple Streams, the measured traffic rate statistics for 542 all Streams MUST be added together. 544 The destination addresses for the Offered Load MUST be distributed 545 such that all routes in the FIB are matched and each route is offered 546 an equal share of the total Offered Load. 548 Loss-Derived Method SHOULD always be combined with Rate-Derived 549 Method in order to observe Full Convergence completion. The total 550 amount of Convergence Packet Loss is collected after Full Convergence 551 completion. 553 To measure convergence time and loss of connectivity benchmarks, the 554 Tester SHOULD in general observe packet loss on all DUT egress 555 interfaces (Connectivity Packet Loss). 557 To measure convergence time benchmarks for Convergence Events that do 558 not cause instantaneous traffic loss for all routes at the 559 Convergence Event Instant, the Tester SHOULD collect a timestamp of 560 the Convergence Event Instant and the Tester SHOULD observe packet 561 loss seperately on the Next-Best Egress Interface (Convergence Packet 562 Loss). 564 Since Loss-Derived Method does not distinguish between traffic 565 destinations and the packet loss statistics are only collected after 566 Full Convergence completion, this method can only be used to measure 567 average values over all routes. For these reasons Loss-Derived 568 Method can only be used to benchmark Loss-Derived Convergence Time 569 and Loss-Derived Loss of Connectivity Period. 571 Note that the Loss-Derived Method measures an average over all 572 routes, including the routes that may not be impacted by the 573 Convergence Event, such as routes via non-impacted members of ECMP or 574 parallel links. 576 Measurement Units: seconds 578 Issues: None 580 See Also: 582 Loss-Derived Convergence Time, Loss-Derived Loss of Connectivity 583 Period, Convergence Packet Loss 585 3.5.3. Route-Specific Loss-Derived Method 587 Definition: 589 The method to calculate the Route-Specific Convergence Time benchmark 590 from the amount of packet loss during convergence for a specific 591 route entry. 593 Discussion: 595 To benchmark Route-Specific Convergence Time, the Tester provides an 596 Offered Load that consists of multiple Streams [Po06]. Each Stream 597 has a single destination address matching a different route entry, 598 for every route entry in the FIB. Convergence Packet Loss is 599 measured for each Stream separately. 601 Route-Specific Loss-Derived Method SHOULD always be combined with 602 Rate-Derived Method in order to observe Full Convergence completion. 603 The total amount of Convergence Packet Loss for each Stream is 604 collected after Full Convergence completion. 606 Route-Specific Loss-Derived Method is a RECOMMENDED method to measure 607 convergence time benchmarks. 609 To measure convergence time and loss of connectivity benchmarks, the 610 Tester SHOULD in general observe packet loss on all DUT egress 611 interfaces (Connectivity Packet Loss). 613 To measure convergence time benchmarks for Convergence Events that do 614 not cause instantaneous traffic loss for all routes at the 615 Convergence Event Instant, the Tester SHOULD collect a timestamp of 616 the Convergence Event Instant and the Tester SHOULD observe packet 617 loss seperately on the Next-Best Egress Interface (Convergence Packet 618 Loss). 620 Since Route-Specific Loss-Derived Method uses traffic streams to 621 individual routes, it measures packet loss as it would be experienced 622 by a network user. For this reason Route-Specific Loss-Derived 623 Method is RECOMMENDED to measure Route-Specific Convergence Time 624 benchmarks and Route Loss of Connectivity Period benchmarks. 626 Measurement Units: seconds 628 Issues: None 630 See Also: 632 Route-Specific Convergence Time, Route Loss of Connectivity Period, 633 Convergence Packet Loss 635 3.6. Benchmarks 637 3.6.1. Full Convergence Time 639 Definition: 641 The time duration of the period between the Convergence Event Instant 642 and the Convergence Recovery Instant as observed using the Rate- 643 Derived Method. 645 Discussion: 647 Using the Rate-Derived Method, Full Convergence Time can be 648 calculated as the time difference between the Convergence Event 649 Instant and the Convergence Recovery Instant, as shown in Equation 1. 651 Full Convergence Time = 652 Convergence Recovery Instant - Convergence Event Instant 654 Equation 1 656 The Convergence Event Instant can be derived from the Forwarding Rate 657 observation or from a timestamp collected by the Tester. 659 For the testcases described in [Po09m], it is expected that Full 660 Convergence Time equals the maximum Route-Specific Convergence Time 661 when benchmarking all routes in FIB using the Route-Specific Loss- 662 Derived Method. 664 It is not possible to measure Full Convergence Time using the Loss- 665 Derived Method. 667 Measurement Units: seconds 669 Issues: None 671 See Also: 673 Full Convergence, Rate-Derived Method, Route-Specific Loss-Derived 674 Method 676 3.6.2. First Route Convergence Time 678 Definition: 680 The duration of the period between the Convergence Event Instant and 681 the First Route Convergence Instant as observed using the Rate- 682 Derived Method. 684 Discussion: 686 Using the Rate-Derived Method, First Route Convergence Time can be 687 calculated as the time difference between the Convergence Event 688 Instant and the First Route Convergence Instant, as shown with 689 Equation 2. 691 First Route Convergence Time = 692 First Route Convergence Instant - Convergence Event Instant 694 Equation 2 696 The Convergence Event Instant can be derived from the Forwarding Rate 697 observation or from a timestamp collected by the Tester. 699 For the testcases described in [Po09m], it is expected that First 700 Route Convergence Time equals the minimum Route-Specific Convergence 701 Time when benchmarking all routes in FIB using the Route-Specific 702 Loss-Derived Method. 704 It is not possible to measure First Route Convergence Time using the 705 Loss-Derived Method. 707 Measurement Units: seconds 709 Issues: None 711 See Also: 713 Rate-Derived Method, Route-Specific Loss-Derived Method, First Route 714 Convergence Instant 716 3.6.3. Route-Specific Convergence Time 718 Definition: 720 The amount of time it takes for Route Convergence to be completed for 721 a specific route, as calculated from the amount of packet loss during 722 convergence for a single route entry. 724 Discussion: 726 Route-Specific Convergence Time can only be measured using the Route- 727 Specific Loss-Derived Method. 729 If the applied Convergence Event causes instantaneous traffic loss 730 for all routes at the Convergence Event Instant, Connectivity Packet 731 Loss should be observed. Connectivity Packet Loss is the combined 732 packet loss observed on Preferred Egress Interface and Next-Best 733 Egress Interface. When benchmarking Route-Specific Convergence Time, 734 Connectivity Packet Loss is measured and Equation 3 is applied for 735 each measured route. The calculation is equal to Equation 7 in 736 Section 3.6.5. 738 Route-Specific Convergence Time = 739 Connectivity Packet Loss for specific route/Offered Load per route 741 Equation 3 743 If the applied Convergence Event does not cause instantaneous traffic 744 loss for all routes at the Convergence Event Instant, then the Tester 745 SHOULD collect a timestamp of the Convergence Event Instant and the 746 Tester SHOULD observe Convergence Packet Loss separately on the Next- 747 Best Egress Interface. When benchmarking Route-Specific Convergence 748 Time, Convergence Packet Loss is measured and Equation 4 is applied 749 for each measured route. 751 Route-Specific Convergence Time = 752 Convergence Packet Loss for specific route/Offered Load per route 753 - (Convergence Event Instant - start traffic instant) 755 Equation 4 757 The Convergence Event Instant and start traffic instant SHOULD be 758 collected by the Tester. 760 The Route-Specific Convergence Time benchmarks enable minimum, 761 maximum, average, and median convergence time measurements to be 762 reported by comparing the results for the different route entries. 763 It also enables benchmarking of convergence time when configuring a 764 priority value for route entry(ies). Since multiple Route-Specific 765 Convergence Times can be measured it is possible to have an array of 766 results. The format for reporting Route-Specific Convergence Time is 767 provided in [Po09m]. 769 Measurement Units: seconds 771 Issues: None 773 See Also: 775 Convergence Event, Convergence Packet Loss, Connectivity Packet Loss, 776 Route Convergence 778 3.6.4. Loss-Derived Convergence Time 780 Definition: 782 The average Route Convergence time for all routes in FIB, as 783 calculated from the amount of packet loss during convergence. 785 Discussion: 787 Loss-Derived Convergence Time is measured using the Loss-Derived 788 Method. 790 If the applied Convergence Event causes instantaneous traffic loss 791 for all routes at the Convergence Event Instant, Connectivity Packet 792 Loss should be observed. Connectivity Packet Loss is the combined 793 packet loss observed on Preferred Egress Interface and Next-Best 794 Egress Interface. When benchmarking Loss-Derived Convergence Time, 795 Connectivity Packet Loss is measured and Equation 5 is applied. 797 Loss-Derived Convergence Time = 798 Connectivity Packet Loss/Offered Load 800 Equation 5 802 If the applied Convergence Event does not cause instantaneous traffic 803 loss for all routes at the Convergence Event Instant, then the Tester 804 SHOULD collect a timestamp of the Convergence Event Instant and the 805 Tester SHOULD observe Convergence Packet Loss separately on the Next- 806 Best Egress Interface. When benchmarking Loss-Derived Convergence 807 Time, Convergence Packet Loss is measured and Equation 6 is applied. 809 Loss-Derived Convergence Time = 810 Convergence Packet Loss/Offered Load 811 - (Convergence Event Instant - start traffic instant) 813 Equation 6 815 The Convergence Event Instant and start traffic instant SHOULD be 816 collected by the Tester. 818 Measurement Units: seconds 820 Issues: None 822 See Also: 824 Convergence Packet Loss, Connectivity Packet Loss, Route Convergence 826 3.6.5. Route Loss of Connectivity Period 828 Definition: 830 The time duration of traffic loss for a specific route entry 831 following a Convergence Event until Full Convergence completion, as 832 observed using the Route-Specific Loss-Derived Method. 834 Discussion: 836 In general the Route Loss of Connectivity Period is not equal to the 837 Route-Specific Convergence Time. If the DUT continues to forward 838 traffic to the Preferred Egress Interface after the Convergence Event 839 is applied then the Route Loss of Connectivity Period will be smaller 840 than the Route-Specific Convergence Time. This is also specifically 841 the case after reversing a failure event. 843 The Route Loss of Connectivity Period may be equal to the Route- 844 Specific Convergence Time, as a characteristic of the Convergence 845 Event, traffic for all routes starts dropping instantaneously on the 846 Convergence Event Instant. See discussion in [Po09m]. 848 For the testcases described in [Po09m] the Route Loss of Connectivity 849 Period is expected to be a single Loss Period [Ko02]. 851 When benchmarking Route Loss of Connectivity Period, Connectivity 852 Packet Loss is measured for each route and Equation 7 is applied for 853 each measured route entry. The calculation is equal to Equation 3 in 854 Section 3.6.3. 856 Route Loss of Connectivity Period = 857 Connectivity Packet Loss for specific route/Offered Load per route 859 Equation 7 861 Route Loss of Connectivity Period SHOULD be measured using Route- 862 Specific Loss-Derived Method. 864 Measurement Units: seconds 866 Issues: None 868 See Also: 870 Route-Specific Convergence Time, Route-Specific Loss-Derived Method, 871 Connectivity Packet Loss 873 3.6.6. Loss-Derived Loss of Connectivity Period 875 Definition: 877 The average time duration of traffic loss for all routes following a 878 Convergence Event until Full Convergence completion, as observed 879 using the Loss-Derived Method. 881 Discussion: 883 In general the Loss-Derived Loss of Connectivity Period is not equal 884 to the Loss-Derived Convergence Time. If the DUT continues to 885 forward traffic to the Preferred Egress Interface after the 886 Convergence Event is applied then the Loss-Derived Loss of 887 Connectivity Period will be smaller than the Loss-Derived Convergence 888 Time. This is also specifically the case after reversing a failure 889 event. 891 The Loss-Derived Loss of Connectivity Period may be equal to the 892 Loss-Derived Convergence Time if, as a characteristic of the 893 Convergence Event, traffic for all routes starts dropping 894 instantaneously on the Convergence Event Instant. See discussion in 895 [Po09m]. 897 For the testcases described in [Po09m] each route's Route Loss of 898 Connectivity Period is expected to be a single Loss Period [Ko02]. 900 When benchmarking Loss-Derived Loss of Connectivity Period, 901 Connectivity Packet Loss is measured for all routes and Equation 8 is 902 applied. The calculation is equal to Equation 5 in Section 3.6.4. 904 Loss-Derived Loss of Connectivity Period = 905 Connectivity Packet Loss for all routes/Offered Load 907 Equation 8 909 Loss-Derived Loss of Connectivity Period SHOULD be measured using 910 Loss-Derived Method. 912 Measurement Units: seconds 914 Issues: None 916 See Also: 918 Loss-Derived Convergence Time, Loss-Derived Method, Connectivity 919 Packet Loss 921 3.7. Measurement Terms 923 3.7.1. Convergence Event 925 Definition: 927 The occurrence of a planned or unplanned event in the network that 928 will result in a change in the egress interface of the Device Under 929 Test (DUT) for routed packets. 931 Discussion: 933 Convergence Events include but are not limited to link loss, routing 934 protocol session loss, router failure, configuration change, and 935 better next-hop learned via a routing protocol. 937 Measurement Units: N/A 939 Issues: None 941 See Also: Convergence Event Instant 943 3.7.2. Packet Loss 945 Definition: 947 The number of packets that should have been forwarded by a DUT under 948 a constant Offered Load that were not forwarded due to lack of 949 resources. 951 Discussion: 953 Packet Loss is a modified version of the term "Frame Loss Rate" as 954 defined in [Br91]. The term "Frame Loss" is intended for Ethernet 955 Frames while "Packet Loss" is intended for IP packets. 957 Measurement units: 959 Number of offered packets that are not forwarded. 961 Issues: None 963 See Also: Convergence Packet Loss 965 3.7.3. Convergence Packet Loss 967 Definition: 969 The number of packets lost due to a Convergence Event until Full 970 Convergence completes, as observed on the Next-Best Egress Interface. 972 Discussion: 974 Convergence Packet Loss is observed on the Next-Best Egress 975 Interface. It only needs to be observed for Convergence Events that 976 do not cause instantaneous traffic loss at Convergence Event Instant. 978 Convergence Packet Loss includes packets that were lost and packets 979 that were delayed due to buffering. The magnitude of an acceptable 980 Forwarding Delay is a parameter of the methodology. If a maximum 981 acceptable Forwarding Delay threshold is applied it MUST be reported. 983 Measurement Units: number of packets 985 Issues: None 987 See Also: 989 Packet Loss, Full Convergence, Convergence Event, Connectivity Packet 990 Loss 992 3.7.4. Connectivity Packet Loss 994 Definition: 996 The number of packets lost due to a Convergence Event until Full 997 Convergence completes. 999 Discussion: 1001 Connectivity Packet Loss is observed on all DUT egress interfaces. 1003 Convergence Packet Loss includes packets that were lost and packets 1004 that were delayed due to buffering. The magnitude of an acceptable 1005 Forwarding Delay is a parameter of the methodology. If a maximum 1006 acceptable Forwarding Delay threshold is applied it MUST be reported. 1008 Measurement Units: number of packets 1010 Issues: None 1012 See Also: 1014 Packet Loss, Route Loss of Connectivity Period, Convergence Event, 1015 Convergence Packet Loss 1017 3.7.5. Packet Sampling Interval 1019 Definition: 1021 The interval at which the Tester (test equipment) polls to make 1022 measurements for arriving packets. 1024 Discussion: 1026 At least one packet per route in the FIB for all routes MUST be 1027 offered to the DUT within the Packet Sampling Interval. Metrics 1028 measured at the Packet Sampling Interval MUST include Forwarding Rate 1029 and received packets. 1031 Packet Sampling Interval can influence the Convergence Graph as 1032 observed with the Rate-Derived Method. This is particularly true 1033 when implementations complete Full Convergence in less time than the 1034 Packet Sampling Interval. The Convergence Event Instant and First 1035 Route Convergence Instant may not be easily identifiable and the 1036 Rate-Derived Method may produce a larger than actual convergence 1037 time. 1039 The recommended value for configuration of the Packet Sampling 1040 Interval when using the Rate-Derived Method is provided in [Po09m]. 1041 For the other benchmark methods the value of the Packet Sampling 1042 Interval does not contribute to the measurement accuracy. 1044 Measurement Units: seconds 1046 Issues: None 1048 See Also: Rate-Derived Method 1050 3.7.6. Sustained Convergence Validation Time 1052 Definition: 1054 The amount of time for which the completion of Full Convergence is 1055 maintained without additional packet loss. 1057 Discussion: 1059 The purpose of the Sustained Convergence Validation Time is to 1060 produce convergence benchmarks protected against fluctuation in 1061 Forwarding Rate after the completion of Full Convergence is observed. 1062 The RECOMMENDED Sustained Convergence Validation Time to be used is 5 1063 seconds. The BMWG selected 5 seconds based upon RFC 2544 [Br99] 1064 which recommends waiting 2 seconds for residual frames to arrive and 1065 5 seconds for DUT restabilization. 1067 Measurement Units: seconds 1069 Issues: None 1071 See Also: 1073 Full Convergence, Convergence Recovery Instant 1075 3.8. Miscellaneous Terms 1077 3.8.1. Stale Forwarding 1079 Definition: 1081 Forwarding of traffic to route entries that no longer exist or to 1082 route entries with next-hops that are no longer preferred. 1084 Discussion: 1086 Stale Forwarding can be caused by a Convergence Event and can 1087 manifest as a "black-hole" or microloop that produces packet loss, or 1088 out-of-order packets, or delayed packets. Stale Forwarding can exist 1089 until Network Convergence is completed. 1091 Measurement Units: N/A 1093 Issues: None 1095 See Also: Network Convergence 1097 3.8.2. Nested Convergence Event 1099 Definition: 1101 The occurrence of a Convergence Event while the route table is 1102 converging from a prior Convergence Event. 1104 Discussion: 1106 The Convergence Events for a Nested Convergence Event MUST occur with 1107 different neighbors. A possible observation from a Nested 1108 Convergence Event will be the withdrawal of routes from one neighbor 1109 while the routes of another neighbor are being installed. 1111 Measurement Units: N/A 1112 Issues: None 1114 See Also: Convergence Event 1116 4. Security Considerations 1118 Documents of this type do not directly affect the security of 1119 Internet or corporate networks as long as benchmarking is not 1120 performed on devices or systems connected to production networks. 1121 Security threats and how to counter these in SIP and the media layer 1122 is discussed in RFC3261, RFC3550, and RFC3711 and various other 1123 drafts. This document attempts to formalize a set of common 1124 methodology for benchmarking IGP convergence performance in a lab 1125 environment. 1127 5. IANA Considerations 1129 This document requires no IANA considerations. 1131 6. Acknowledgements 1133 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1134 Peter De Vriendt and the BMWG for their contributions to this work. 1136 7. References 1138 7.1. Normative References 1140 [Br91] Bradner, S., "Benchmarking terminology for network 1141 interconnection devices", RFC 1242, July 1991. 1143 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1144 Requirement Levels", BCP 14, RFC 2119, March 1997. 1146 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1147 Network Interconnect Devices", RFC 2544, March 1999. 1149 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1150 environments", RFC 1195, December 1990. 1152 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1153 IPv6", RFC 5340, July 2008. 1155 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1156 October 2008. 1158 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1159 Metrics", RFC 3357, August 2002. 1161 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1162 Devices", RFC 2285, February 1998. 1164 [Mo06] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., 1165 and J. Perser, "Packet Reordering Metrics", RFC 4737, 1166 November 2006. 1168 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1170 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1171 "Terminology for Benchmarking Network-layer Traffic Control 1172 Mechanisms", RFC 4689, October 2006. 1174 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1175 IGP Data Plane Route Convergence", 1176 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1177 progress), March 2009. 1179 [Po09m] Poretsky, S. and B. Imhoff, "Benchmarking Methodology for 1180 Link-State IGP Data Plane Route Convergence", 1181 draft-ietf-bmwg-igp-dataplane-conv-meth-18 (work in 1182 progress), July 2009. 1184 7.2. Informative References 1186 [Ca01] Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained 1187 View of High Performance Networking", NANOG 22, June 2001. 1189 [Ci03] Ciavattone, L., Morton, A., and G. Ramachandran, 1190 "Standardized Active Measurements on a Tier 1 IP Backbone", 1191 IEEE Communications Magazine p90-97, May 2003. 1193 Authors' Addresses 1195 Scott Poretsky 1196 Allot Communications 1197 67 South Bedford Street, Suite 400 1198 Burlington, MA 01803 1199 USA 1201 Phone: + 1 508 309 2179 1202 Email: sporetsky@allot.com 1204 Brent Imhoff 1205 Juniper Networks 1206 1194 North Mathilda Ave 1207 Sunnyvale, CA 94089 1208 USA 1210 Phone: + 1 314 378 2571 1211 Email: bimhoff@planetspork.com 1213 Kris Michielsen 1214 Cisco Systems 1215 6A De Kleetlaan 1216 Diegem, BRABANT 1831 1217 Belgium 1219 Email: kmichiel@cisco.com