idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-term-19.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 26, 2009) is 5290 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-meth-18 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: April 29, 2010 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 October 26, 2009 10 Terminology for Benchmarking Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-term-19 13 Status of this Memo 15 This Internet-Draft is submitted to IETF in full conformance with the 16 provisions of BCP 78 and BCP 79. This document may contain material 17 from IETF Documents or IETF Contributions published or made publicly 18 available before November 10, 2008. The person(s) controlling the 19 copyright in some of this material may not have granted the IETF 20 Trust the right to allow modifications of such material outside the 21 IETF Standards Process. Without obtaining an adequate license from 22 the person(s) controlling the copyright in such materials, this 23 document may not be modified outside the IETF Standards Process, and 24 derivative works of it may not be created outside the IETF Standards 25 Process, except to format it for publication as an RFC or to 26 translate it into languages other than English. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as Internet- 31 Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/ietf/1id-abstracts.txt. 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html. 44 This Internet-Draft will expire on April 29, 2010. 46 Copyright Notice 48 Copyright (c) 2009 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents in effect on the date of 53 publication of this document (http://trustee.ietf.org/license-info). 54 Please review these documents carefully, as they describe your rights 55 and restrictions with respect to this document. 57 Abstract 59 This document describes the terminology for benchmarking Interior 60 Gateway Protocol (IGP) Route Convergence. The terminology is to be 61 used for benchmarking IGP convergence time through externally 62 observable (black box) data plane measurements. The terminology can 63 be applied to any link-state IGP, such as ISIS and OSPF. 65 Table of Contents 67 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 4 68 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 4 69 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 70 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 5 71 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 5 72 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 5 73 3.1.3. Network Convergence . . . . . . . . . . . . . . . . . 6 74 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 6 75 3.2.1. Traffic Start Instant . . . . . . . . . . . . . . . . 6 76 3.2.2. Convergence Event Instant . . . . . . . . . . . . . . 7 77 3.2.3. Convergence Recovery Instant . . . . . . . . . . . . . 7 78 3.2.4. First Route Convergence Instant . . . . . . . . . . . 8 79 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 9 80 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 9 81 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 9 82 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 10 83 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 10 84 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 10 85 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 11 86 3.4.4. Next-Best Egress Interface . . . . . . . . . . . . . . 11 87 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 11 88 3.5.1. Rate-Derived Method . . . . . . . . . . . . . . . . . 11 89 3.5.2. Loss-Derived Method . . . . . . . . . . . . . . . . . 13 90 3.5.3. Route-Specific Loss-Derived Method . . . . . . . . . . 14 91 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15 92 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 15 93 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 16 94 3.6.3. Route-Specific Convergence Time . . . . . . . . . . . 17 95 3.6.4. Loss-Derived Convergence Time . . . . . . . . . . . . 18 96 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 19 97 3.6.6. Loss-Derived Loss of Connectivity Period . . . . . . . 20 98 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 21 99 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 21 100 3.7.2. Packet Loss . . . . . . . . . . . . . . . . . . . . . 22 101 3.7.3. Convergence Packet Loss . . . . . . . . . . . . . . . 22 102 3.7.4. Connectivity Packet Loss . . . . . . . . . . . . . . . 23 103 3.7.5. Packet Sampling Interval . . . . . . . . . . . . . . . 23 104 3.7.6. Sustained Convergence Validation Time . . . . . . . . 24 105 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 24 106 3.8.1. Stale Forwarding . . . . . . . . . . . . . . . . . . . 24 107 3.8.2. Nested Convergence Event . . . . . . . . . . . . . . . 25 108 4. Security Considerations . . . . . . . . . . . . . . . . . . . 25 109 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 110 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 26 111 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 26 112 7.1. Normative References . . . . . . . . . . . . . . . . . . . 26 113 7.2. Informative References . . . . . . . . . . . . . . . . . . 27 114 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 27 116 1. Introduction and Scope 118 This draft describes the terminology for benchmarking Link-State 119 Interior Gateway Protocol (IGP) Convergence. The motivation and 120 applicability for this benchmarking is provided in [Po09a]. The 121 methodology to be used for this benchmarking is described in [Po09m]. 122 The purpose of this document is to introduce new terms required to 123 complete execution of the IGP Route Methodology [Po09m]. 125 IGP convergence time is measured on the data plane at the Tester by 126 observing packet loss through the DUT. The methodology and 127 terminology to be used for benchmarking IGP Convergence can be 128 applied to IPv4 and IPv6 traffic and link-state IGPs such as ISIS 129 [Ca90][Ho08], OSPF [Mo98][Co08], and others. 131 2. Existing Definitions 133 This document uses existing terminology defined in other BMWG work. 134 Examples include, but are not limited to: 136 Frame Loss Rate [Ref.[Br91], section 3.6] 137 Throughput [Ref.[Br91], section 3.17] 138 Offered Load [Ref.[Ma98], section 3.5.2] 139 Forwarding Rate [Ref.[Ma98], section 3.6.1] 140 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 141 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 142 Out-of-order Packet [Ref.[Po06], section 3.3.2] 143 Duplicate Packet [Ref.[Po06], section 3.3.3] 144 Packet Reordering [Ref.[Mo06], section 3.3] 145 Stream [Ref.[Po06], section 3.3.2] 146 Forwarding Delay [Ref.[Po06], section 3.2.4] 147 Loss Period [Ref.[Ko02], section 4] 149 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 150 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 151 document are to be interpreted as described in BCP 14, RFC 2119 152 [Br97]. RFC 2119 defines the use of these key words to help make the 153 intent of standards track documents as clear as possible. While this 154 document uses these keywords, this document is not a standards track 155 document. 157 3. Term Definitions 158 3.1. Convergence Types 160 3.1.1. Route Convergence 162 Definition: 164 The process of updating all components of the router, including the 165 Routing Information Base (RIB) and Forwarding Information Base (FIB), 166 along with software and hardware tables, with the most recent route 167 change(s) such that forwarding for a route entry is successful on the 168 Next-Best Egress Interface. 170 Discussion: 172 Route Convergence MUST occur after a Convergence Event. Route 173 Convergence can be observed externally by the rerouting of data 174 traffic for a destination matching a route entry to the Next-best 175 Egress Interface. Completion of Route Convergence may or may not be 176 sustained over time. 178 Measurement Units: N/A 180 Issues: None 182 See Also: 184 Network Convergence, Full Convergence, Convergence Event 186 3.1.2. Full Convergence 188 Definition: 190 Route Convergence for all routes in the FIB. 192 Discussion: 194 Full Convergence MUST occur after a Convergence Event. Full 195 Convergence can be observed externally by the rerouting of data 196 traffic to destinations matching all route entries to the Next-best 197 Egress Interface. Completion of Full Convergence is externally 198 observable from the data plane when the Forwarding Rate of the data 199 plane traffic on the Next-Best Egress Interface equals the Offered 200 Load. 202 Completion of Full Convergence may or may not be sustained over time. 204 Measurement Units: N/A 205 Issues: None 207 See Also: 209 Network Convergence, Route Convergence, Convergence Event, Full 210 Convergence Time, Convergence Recovery Instant 212 3.1.3. Network Convergence 214 Definition: 216 Full Convergence in all routers throughout the network. 218 Discussion: 220 Network Convergence includes all Route Convergence operations for all 221 routers in the network following a Convergence Event. 223 Completion of Network Convergence can be observed by recovery of the 224 network Forwarding Rate to equal the Offered Load, with no Stale 225 Forwarding, and no Blenders [Ca01][Ci03]. 227 Completion of Network Convergence may or may not be sustained over 228 time. 230 Measurement Units: N/A 232 Issues: None 234 See Also: 236 Route Convergence, Full Convergence, Stale Forwarding 238 3.2. Instants 240 3.2.1. Traffic Start Instant 242 Definition: 244 The time instant the Tester sends out the first data packet to the 245 DUT. 247 Discussion: 249 If using the Loss-Derived Method or the Route-Specific Loss-Derived 250 Method to benchmark IGP convergence time, and the applied Convergence 251 Event does not cause instantaneous traffic loss for all routes at the 252 Convergence Event Instant then the Tester SHOULD collect a timestamp 253 on the Traffic Start Instant in order to measure the period of time 254 between the Traffic Start Instant and Convergence Event Instant. 256 Measurement Units: 258 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 259 microseconds. 261 Issues: None 263 See Also: 265 Convergence Event Instant, Route-Specific Convergence Time, Loss- 266 Derived Convergence Time. 268 3.2.2. Convergence Event Instant 270 Definition: 272 The time instant that a Convergence Event occurs. 274 Discussion: 276 If the Convergence Event causes instantaneous traffic loss on the 277 Preferred Egress Interface, the Convergence Event Instant is 278 observable from the data plane as the instant that the DUT begins to 279 exhibit packet loss. 281 The Tester SHOULD collect a timestamp on the Convergence Event 282 Instant if it is not observable from the data plane. 284 Measurement Units: 286 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 287 microseconds. 289 Issues: None 291 See Also: Convergence Event 293 3.2.3. Convergence Recovery Instant 295 Definition: 297 The time instant that Full Convergence has completed. 299 Discussion: 301 The Full Convergence completed state MUST be maintained for an 302 interval of duration equal to the Sustained Convergence Validation 303 Time in order to validate the Convergence Recovery Instant. 305 The Convergence Recovery Instant is observable from the data plane as 306 the instant the DUT forwards traffic to all destinations over the 307 Next-Best Egress Interface. 309 When using the Rate-Derived Method, the Convergence Recovery Instant 310 falls within the Packet Sampling Interval preceding the first 311 interval where the observed Forwarding Rate on the Next-Best Egress 312 Interface equals the Offered Load. 314 Measurement Units: 316 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 317 microseconds. 319 Issues: None 321 See Also: 323 Sustained Convergence Validation Time, Full Convergence 325 3.2.4. First Route Convergence Instant 327 Definition: 329 The time instant the first route entry completes Route Convergence 330 following a Convergence Event 332 Discussion: 334 Any route may be the first to complete Route Convergence. The First 335 Route Convergence Instant is observable from the data plane as the 336 instant that the first packet is received from the Next-Best Egress 337 Interface. 339 Measurement Units: 341 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 342 microseconds. 344 Issues: None 346 See Also: Route Convergence 348 3.3. Transitions 350 3.3.1. Convergence Event Transition 352 Definition: 354 A time interval following a Convergence Event in which Forwarding 355 Rate on the Preferred Egress Interface gradually reduces to zero. 357 Discussion: 359 The Forwarding Rate during a Convergence Event Transition may not 360 decrease linearly. 362 The Forwarding Rate observed on all DUT egress interfaces may or may 363 not decrease to zero. 365 The Offered Load, the number of routes, and the Packet Sampling 366 Interval influence the observations of the Convergence Event 367 Transition using the Rate-Derived Method. This is further discussed 368 with the term "Rate-Derived Method". 370 Measurement Units: seconds 372 Issues: None 374 See Also: 376 Convergence Event, Rate-Derived Method 378 3.3.2. Convergence Recovery Transition 380 Definition: 382 A time interval following the First Route Convergence Instant in 383 which Forwarding Rate on the Next-Best Egress Interface gradually 384 increases to equal the Offered Load. 386 Discussion: 388 The Forwarding Rate observed during a Convergence Recovery Transition 389 may not increase linearly. 391 The Offered Load, the number of routes, and the Packet Sampling 392 Interval influence the observations of the Convergence Recovery 393 Transition using the Rate-Derived Method. This is further discussed 394 with the term "Rate-Derived Method". 396 Measurement Units: seconds 398 Issues: None 400 See Also: 402 Full Convergence,First Route Convergence Instant, Rate-Derived Method 404 3.4. Interfaces 406 3.4.1. Local Interface 408 Definition: 410 An interface on the DUT. 412 Discussion: 414 A failure of the Local Interface indicates that the failure occurred 415 directly on the DUT. 417 Measurement Units: N/A 419 Issues: None 421 See Also: Remote Interface 423 3.4.2. Remote Interface 425 Definition: 427 An interface on a neighboring router that is not directly connected 428 to any interface on the DUT. 430 Discussion: 432 A failure of a Remote Interface indicates that the failure occurred 433 on a neighbor router's interface that is not directly connected to 434 the DUT. 436 Measurement Units: N/A 438 Issues: None 440 See Also: Local Interface 442 3.4.3. Preferred Egress Interface 444 Definition: 446 The outbound interface from the DUT for traffic routed to the 447 preferred next-hop. 449 Discussion: 451 The Preferred Egress Interface is the egress interface prior to a 452 Convergence Event. 454 Measurement Units: N/A 456 Issues: None 458 See Also: Next-Best Egress Interface 460 3.4.4. Next-Best Egress Interface 462 Definition: 464 The outbound interface from the DUT for traffic routed to the second- 465 best next-hop. 467 Discussion: 469 The Next-Best Egress Interface becomes the egress interface after a 470 Convergence Event. 472 The Next-Best Egress Interface is of the same media type and link 473 speed as the Preferred Egress Interface. 475 Measurement Units: N/A 477 Issues: None 479 See Also: Preferred Egress Interface 481 3.5. Benchmarking Methods 483 3.5.1. Rate-Derived Method 485 Definition: 487 The method to calculate convergence time benchmarks from observing 488 Forwarding Rate each Packet Sampling Interval. 490 Discussion: 492 Figure 1 shows an example of the Forwarding Rate change in time 493 during convergence as observed when using the Rate-Derived Method. 495 ^ Traffic Convergence 496 Fwd | Start Recovery 497 Rate | Instant Instant 498 | Offered ^ ^ 499 | Load --> ----------\ /----------- 500 | \ /<--- Convergence 501 | \ Packet / Recovery 502 | Convergence --->\ Loss / Transition 503 | Event \ / 504 | Transition \---------/ <-- Max Packet Loss 505 | 506 +---------------------------------------------------------> 507 ^ ^ time 508 Convergence First Route 509 Event Instant Convergence Instant 511 Figure 1: Rate-Derived Convergence Graph 513 The Offered Load SHOULD consist of a single Stream [Po06]. If 514 sending multiple Streams, the measured traffic rate statistics for 515 all Streams MUST be added together. 517 The destination addresses for the Offered Load MUST be distributed 518 such that all routes or a statistically representative subset of all 519 routes are matched and each of these routes is offered an equal share 520 of the Offered Load. It is RECOMMENDED to send traffic to all 521 routes, but a statistically representative subset of all routes can 522 be used if required. 524 At least one packet per route for all routes matched in the Offered 525 Load MUST be offered to the DUT within each Packet Sampling Interval. 527 The Offered Load, the number of routes, and the Packet Sampling 528 Interval influence the observations for the Rate-Derived Method. It 529 may be difficult to identify the different convergence time instants 530 in the Rate-Derived Convergence Graph. For example, it is possible 531 that a Convergence Event causes the Forwarding Rate to drop to zero, 532 while this may not be observed in the Forwarding Rate measurements if 533 the Packet Sampling Interval is too large. 535 Metrics measured at the Packet Sampling Interval MUST include 536 Forwarding Rate and packet loss. 538 Rate-Derived Method is a RECOMMENDED method to measure convergence 539 time benchmarks. 541 To measure convergence time benchmarks for Convergence Events that do 542 not cause instantaneous traffic loss for all routes at the 543 Convergence Event Instant, the Tester SHOULD collect a timestamp of 544 the Convergence Event Instant and the Tester SHOULD observe 545 Forwarding Rate separately on the Next-Best Egress Interface. 547 Since the Rate-Derived Method does not distinguish between individual 548 traffic destinations, it SHOULD NOT be used for any route specific 549 measurements. Therefor Rate-Derived Method SHOULD NOT be used to 550 benchmark Route Loss of Connectivity Period. 552 Measurement Units: N/A 554 Issues: None 556 See Also: 558 Packet Sampling Interval, Convergence Event, Convergence Event 559 Instant, Full Convergence 561 3.5.2. Loss-Derived Method 563 Definition: 565 The method to calculate the Loss-Derived Convergence Time and Loss- 566 Derived Loss of Connectivity Period benchmarks from the amount of 567 packet loss. 569 Discussion: 571 The Offered Load SHOULD consist of a single Stream [Po06]. If 572 sending multiple Streams, the measured traffic rate statistics for 573 all Streams MUST be added together. 575 The destination addresses for the Offered Load MUST be distributed 576 such that all routes or a statistically representative subset of all 577 routes are matched and each of these routes is offered an equal share 578 of the Offered Load. It is RECOMMENDED to send traffic to all 579 routes, but a statistically representative subset of all routes can 580 be used if required. 582 Loss-Derived Method SHOULD always be combined with Rate-Derived 583 Method in order to observe Full Convergence completion. The total 584 amount of Convergence Packet Loss is collected after Full Convergence 585 completion. 587 To measure convergence time and loss of connectivity benchmarks, the 588 Tester SHOULD in general observe packet loss on all DUT egress 589 interfaces (Connectivity Packet Loss). 591 To measure convergence time benchmarks for Convergence Events that do 592 not cause instantaneous traffic loss for all routes at the 593 Convergence Event Instant, the Tester SHOULD collect timestamps of 594 the Start Traffic Instant and of the Convergence Event Instant, and 595 the Tester SHOULD observe packet loss separately on the Next-Best 596 Egress Interface (Convergence Packet Loss). 598 Since Loss-Derived Method does not distinguish between traffic 599 destinations and the packet loss statistics are only collected after 600 Full Convergence completion, this method can only be used to measure 601 average values over all routes. For these reasons Loss-Derived 602 Method can only be used to benchmark Loss-Derived Convergence Time 603 and Loss-Derived Loss of Connectivity Period. 605 Note that the Loss-Derived Method measures an average over all 606 routes, including the routes that may not be impacted by the 607 Convergence Event, such as routes via non-impacted members of ECMP or 608 parallel links. 610 Measurement Units: seconds 612 Issues: None 614 See Also: 616 Loss-Derived Convergence Time, Loss-Derived Loss of Connectivity 617 Period, Convergence Packet Loss 619 3.5.3. Route-Specific Loss-Derived Method 621 Definition: 623 The method to calculate the Route-Specific Convergence Time benchmark 624 from the amount of packet loss during convergence for a specific 625 route entry. 627 Discussion: 629 To benchmark Route-Specific Convergence Time, the Tester provides an 630 Offered Load that consists of multiple Streams [Po06]. Each Stream 631 has a single destination address matching a different route entry, 632 for all routes or a statistically representative subset of all 633 routes. Convergence Packet Loss is measured for each Stream 634 separately. 636 Route-Specific Loss-Derived Method SHOULD always be combined with 637 Rate-Derived Method in order to observe Full Convergence completion. 638 The total amount of Convergence Packet Loss for each Stream is 639 collected after Full Convergence completion. 641 Route-Specific Loss-Derived Method is a RECOMMENDED method to measure 642 convergence time benchmarks. 644 To measure convergence time and loss of connectivity benchmarks, the 645 Tester SHOULD in general observe packet loss on all DUT egress 646 interfaces (Connectivity Packet Loss). 648 To measure convergence time benchmarks for Convergence Events that do 649 not cause instantaneous traffic loss for all routes at the 650 Convergence Event Instant, the Tester SHOULD collect timestamps of 651 the Start Traffic Instant and of the Convergence Event Instant, and 652 the Tester SHOULD observe packet loss separately on the Next-Best 653 Egress Interface (Convergence Packet Loss). 655 Since Route-Specific Loss-Derived Method uses traffic streams to 656 individual routes, it measures packet loss as it would be experienced 657 by a network user. For this reason Route-Specific Loss-Derived 658 Method is RECOMMENDED to measure Route-Specific Convergence Time 659 benchmarks and Route Loss of Connectivity Period benchmarks. 661 Measurement Units: seconds 663 Issues: None 665 See Also: 667 Route-Specific Convergence Time, Route Loss of Connectivity Period, 668 Convergence Packet Loss 670 3.6. Benchmarks 672 3.6.1. Full Convergence Time 674 Definition: 676 The time duration of the period between the Convergence Event Instant 677 and the Convergence Recovery Instant as observed using the Rate- 678 Derived Method. 680 Discussion: 682 Using the Rate-Derived Method, Full Convergence Time can be 683 calculated as the time difference between the Convergence Event 684 Instant and the Convergence Recovery Instant, as shown in Equation 1. 686 Full Convergence Time = 687 Convergence Recovery Instant - Convergence Event Instant 689 Equation 1 691 The Convergence Event Instant can be derived from the Forwarding Rate 692 observation or from a timestamp collected by the Tester. 694 For the testcases described in [Po09m], it is expected that Full 695 Convergence Time equals the maximum Route-Specific Convergence Time 696 when benchmarking all routes in FIB using the Route-Specific Loss- 697 Derived Method. 699 It is not possible to measure Full Convergence Time using the Loss- 700 Derived Method. 702 Measurement Units: seconds 704 Issues: None 706 See Also: 708 Full Convergence, Rate-Derived Method, Route-Specific Loss-Derived 709 Method 711 3.6.2. First Route Convergence Time 713 Definition: 715 The duration of the period between the Convergence Event Instant and 716 the First Route Convergence Instant as observed using the Rate- 717 Derived Method. 719 Discussion: 721 Using the Rate-Derived Method, First Route Convergence Time can be 722 calculated as the time difference between the Convergence Event 723 Instant and the First Route Convergence Instant, as shown with 724 Equation 2. 726 First Route Convergence Time = 727 First Route Convergence Instant - Convergence Event Instant 729 Equation 2 731 The Convergence Event Instant can be derived from the Forwarding Rate 732 observation or from a timestamp collected by the Tester. 734 For the testcases described in [Po09m], it is expected that First 735 Route Convergence Time equals the minimum Route-Specific Convergence 736 Time when benchmarking all routes in FIB using the Route-Specific 737 Loss-Derived Method. 739 It is not possible to measure First Route Convergence Time using the 740 Loss-Derived Method. 742 Measurement Units: seconds 744 Issues: None 746 See Also: 748 Rate-Derived Method, Route-Specific Loss-Derived Method, First Route 749 Convergence Instant 751 3.6.3. Route-Specific Convergence Time 753 Definition: 755 The amount of time it takes for Route Convergence to be completed for 756 a specific route, as calculated from the amount of packet loss during 757 convergence for a single route entry. 759 Discussion: 761 Route-Specific Convergence Time can only be measured using the Route- 762 Specific Loss-Derived Method. 764 If the applied Convergence Event causes instantaneous traffic loss 765 for all routes at the Convergence Event Instant, Connectivity Packet 766 Loss should be observed. Connectivity Packet Loss is the combined 767 packet loss observed on Preferred Egress Interface and Next-Best 768 Egress Interface. When benchmarking Route-Specific Convergence Time, 769 Connectivity Packet Loss is measured and Equation 3 is applied for 770 each measured route. The calculation is equal to Equation 7 in 771 Section 3.6.5. 773 Route-Specific Convergence Time = 774 Connectivity Packet Loss for specific route/Offered Load per route 776 Equation 3 778 If the applied Convergence Event does not cause instantaneous traffic 779 loss for all routes at the Convergence Event Instant, then the Tester 780 SHOULD collect timestamps of the Traffic Start Instant and of the 781 Convergence Event Instant, and the Tester SHOULD observe Convergence 782 Packet Loss separately on the Next-Best Egress Interface. When 783 benchmarking Route-Specific Convergence Time, Convergence Packet Loss 784 is measured and Equation 4 is applied for each measured route. 786 Route-Specific Convergence Time = 787 Convergence Packet Loss for specific route/Offered Load per route 788 - (Convergence Event Instant - Traffic Start Instant) 790 Equation 4 792 The Convergence Event Instant and Traffic Start Instant SHOULD be 793 collected by the Tester. 795 The Route-Specific Convergence Time benchmarks enable minimum, 796 maximum, average, and median convergence time measurements to be 797 reported by comparing the results for the different route entries. 798 It also enables benchmarking of convergence time when configuring a 799 priority value for route entry(ies). Since multiple Route-Specific 800 Convergence Times can be measured it is possible to have an array of 801 results. The format for reporting Route-Specific Convergence Time is 802 provided in [Po09m]. 804 Measurement Units: seconds 806 Issues: None 808 See Also: 810 Convergence Event, Convergence Packet Loss, Connectivity Packet Loss, 811 Route Convergence 813 3.6.4. Loss-Derived Convergence Time 815 Definition: 817 The average Route Convergence time for all routes in FIB, as 818 calculated from the amount of packet loss during convergence. 820 Discussion: 822 Loss-Derived Convergence Time is measured using the Loss-Derived 823 Method. 825 If the applied Convergence Event causes instantaneous traffic loss 826 for all routes at the Convergence Event Instant, Connectivity Packet 827 Loss should be observed. Connectivity Packet Loss is the combined 828 packet loss observed on Preferred Egress Interface and Next-Best 829 Egress Interface. When benchmarking Loss-Derived Convergence Time, 830 Connectivity Packet Loss is measured and Equation 5 is applied. 832 Loss-Derived Convergence Time = 833 Connectivity Packet Loss/Offered Load 835 Equation 5 837 If the applied Convergence Event does not cause instantaneous traffic 838 loss for all routes at the Convergence Event Instant, then the Tester 839 SHOULD collect timestamps of the Start Traffic Instant and of the 840 Convergence Event Instant and the Tester SHOULD observe Convergence 841 Packet Loss separately on the Next-Best Egress Interface. When 842 benchmarking Loss-Derived Convergence Time, Convergence Packet Loss 843 is measured and Equation 6 is applied. 845 Loss-Derived Convergence Time = 846 Convergence Packet Loss/Offered Load 847 - (Convergence Event Instant - Traffic Start Instant) 849 Equation 6 851 The Convergence Event Instant and Traffic Start Instant SHOULD be 852 collected by the Tester. 854 Measurement Units: seconds 856 Issues: None 858 See Also: 860 Convergence Packet Loss, Connectivity Packet Loss, Route Convergence 862 3.6.5. Route Loss of Connectivity Period 864 Definition: 866 The time duration of traffic loss for a specific route entry 867 following a Convergence Event until Full Convergence completion, as 868 observed using the Route-Specific Loss-Derived Method. 870 Discussion: 872 In general the Route Loss of Connectivity Period is not equal to the 873 Route-Specific Convergence Time. If the DUT continues to forward 874 traffic to the Preferred Egress Interface after the Convergence Event 875 is applied then the Route Loss of Connectivity Period will be smaller 876 than the Route-Specific Convergence Time. This is also specifically 877 the case after reversing a failure event. 879 The Route Loss of Connectivity Period may be equal to the Route- 880 Specific Convergence Time if, as a characteristic of the Convergence 881 Event, traffic for all routes starts dropping instantaneously on the 882 Convergence Event Instant. See discussion in [Po09m]. 884 For the testcases described in [Po09m] the Route Loss of Connectivity 885 Period is expected to be a single Loss Period [Ko02]. 887 When benchmarking Route Loss of Connectivity Period, Connectivity 888 Packet Loss is measured for each route and Equation 7 is applied for 889 each measured route entry. The calculation is equal to Equation 3 in 890 Section 3.6.3. 892 Route Loss of Connectivity Period = 893 Connectivity Packet Loss for specific route/Offered Load per route 895 Equation 7 897 Route Loss of Connectivity Period SHOULD be measured using Route- 898 Specific Loss-Derived Method. 900 Measurement Units: seconds 902 Issues: None 904 See Also: 906 Route-Specific Convergence Time, Route-Specific Loss-Derived Method, 907 Connectivity Packet Loss 909 3.6.6. Loss-Derived Loss of Connectivity Period 911 Definition: 913 The average time duration of traffic loss for all routes following a 914 Convergence Event until Full Convergence completion, as observed 915 using the Loss-Derived Method. 917 Discussion: 919 In general the Loss-Derived Loss of Connectivity Period is not equal 920 to the Loss-Derived Convergence Time. If the DUT continues to 921 forward traffic to the Preferred Egress Interface after the 922 Convergence Event is applied then the Loss-Derived Loss of 923 Connectivity Period will be smaller than the Loss-Derived Convergence 924 Time. This is also specifically the case after reversing a failure 925 event. 927 The Loss-Derived Loss of Connectivity Period may be equal to the 928 Loss-Derived Convergence Time if, as a characteristic of the 929 Convergence Event, traffic for all routes starts dropping 930 instantaneously on the Convergence Event Instant. See discussion in 931 [Po09m]. 933 For the testcases described in [Po09m] each route's Route Loss of 934 Connectivity Period is expected to be a single Loss Period [Ko02]. 936 When benchmarking Loss-Derived Loss of Connectivity Period, 937 Connectivity Packet Loss is measured for all routes and Equation 8 is 938 applied. The calculation is equal to Equation 5 in Section 3.6.4. 940 Loss-Derived Loss of Connectivity Period = 941 Connectivity Packet Loss for all routes/Offered Load 943 Equation 8 945 Loss-Derived Loss of Connectivity Period SHOULD be measured using 946 Loss-Derived Method. 948 Measurement Units: seconds 950 Issues: None 952 See Also: 954 Loss-Derived Convergence Time, Loss-Derived Method, Connectivity 955 Packet Loss 957 3.7. Measurement Terms 959 3.7.1. Convergence Event 961 Definition: 963 The occurrence of a planned or unplanned event in the network that 964 will result in a change in the egress interface of the Device Under 965 Test (DUT) for routed packets. 967 Discussion: 969 Convergence Events include but are not limited to link loss, routing 970 protocol session loss, router failure, configuration change, and 971 better next-hop learned via a routing protocol. 973 Measurement Units: N/A 975 Issues: None 977 See Also: Convergence Event Instant 979 3.7.2. Packet Loss 981 Definition: 983 The number of packets that should have been forwarded by a DUT under 984 a constant Offered Load that were not forwarded due to lack of 985 resources. 987 Discussion: 989 Packet Loss is a modified version of the term "Frame Loss Rate" as 990 defined in [Br91]. The term "Frame Loss" is intended for Ethernet 991 Frames while "Packet Loss" is intended for IP packets. 993 Measurement units: Number of offered packets that are not forwarded. 995 Issues: None 997 See Also: Convergence Packet Loss 999 3.7.3. Convergence Packet Loss 1001 Definition: 1003 The number of packets lost due to a Convergence Event until Full 1004 Convergence completes, as observed on the Next-Best Egress Interface. 1006 Discussion: 1008 Convergence Packet Loss is observed on the Next-Best Egress 1009 Interface. It only needs to be observed for Convergence Events that 1010 do not cause instantaneous traffic loss at Convergence Event Instant. 1012 Convergence Packet Loss includes packets that were lost and packets 1013 that were delayed due to buffering. The magnitude of an acceptable 1014 Forwarding Delay is a parameter of the methodology. If a maximum 1015 acceptable Forwarding Delay threshold is applied it MUST be reported. 1017 Measurement Units: number of packets 1019 Issues: None 1020 See Also: 1022 Packet Loss, Full Convergence, Convergence Event, Connectivity Packet 1023 Loss 1025 3.7.4. Connectivity Packet Loss 1027 Definition: 1029 The number of packets lost due to a Convergence Event until Full 1030 Convergence completes. 1032 Discussion: 1034 Connectivity Packet Loss is observed on all DUT egress interfaces. 1036 Convergence Packet Loss includes packets that were lost and packets 1037 that were delayed due to buffering. The magnitude of an acceptable 1038 Forwarding Delay is a parameter of the methodology. If a maximum 1039 acceptable Forwarding Delay threshold is applied it MUST be reported. 1041 Measurement Units: number of packets 1043 Issues: None 1045 See Also: 1047 Packet Loss, Route Loss of Connectivity Period, Convergence Event, 1048 Convergence Packet Loss 1050 3.7.5. Packet Sampling Interval 1052 Definition: 1054 The interval at which the Tester (test equipment) polls to make 1055 measurements for arriving packets. 1057 Discussion: 1059 At least one packet per route for all routes matched in the Offered 1060 Load MUST be offered to the DUT within the Packet Sampling Interval. 1061 Metrics measured at the Packet Sampling Interval MUST include 1062 Forwarding Rate and received packets. 1064 Packet Sampling Interval can influence the convergence graph as 1065 observed with the Rate-Derived Method. This is particularly true 1066 when implementations complete Full Convergence in less time than the 1067 Packet Sampling Interval. The Convergence Event Instant and First 1068 Route Convergence Instant may not be easily identifiable and the 1069 Rate-Derived Method may produce a larger than actual convergence 1070 time. 1072 The recommended value for configuration of the Packet Sampling 1073 Interval when using the Rate-Derived Method is provided in [Po09m]. 1074 For the other benchmark methods the value of the Packet Sampling 1075 Interval does not contribute to the measurement accuracy. 1077 Measurement Units: seconds 1079 Issues: None 1081 See Also: Rate-Derived Method 1083 3.7.6. Sustained Convergence Validation Time 1085 Definition: 1087 The amount of time for which the completion of Full Convergence is 1088 maintained without additional packet loss. 1090 Discussion: 1092 The purpose of the Sustained Convergence Validation Time is to 1093 produce convergence benchmarks protected against fluctuation in 1094 Forwarding Rate after the completion of Full Convergence is observed. 1095 The RECOMMENDED Sustained Convergence Validation Time to be used is 5 1096 seconds. The BMWG selected 5 seconds based upon RFC 2544 [Br99] 1097 which recommends waiting 2 seconds for residual frames to arrive 1098 (this is the Forwarding Delay threshold for the last packet sent) and 1099 5 seconds for DUT restabilization. 1101 Measurement Units: seconds 1103 Issues: None 1105 See Also: 1107 Full Convergence, Convergence Recovery Instant 1109 3.8. Miscellaneous Terms 1111 3.8.1. Stale Forwarding 1113 Definition: 1115 Forwarding of traffic to route entries that no longer exist or to 1116 route entries with next-hops that are no longer preferred. 1118 Discussion: 1120 Stale Forwarding can be caused by a Convergence Event and can 1121 manifest as a "black-hole" or microloop that produces packet loss, or 1122 out-of-order packets, or delayed packets. Stale Forwarding can exist 1123 until Network Convergence is completed. 1125 Measurement Units: N/A 1127 Issues: None 1129 See Also: Network Convergence 1131 3.8.2. Nested Convergence Event 1133 Definition: 1135 The occurrence of a Convergence Event while the route table is 1136 converging from a prior Convergence Event. 1138 Discussion: 1140 The Convergence Events for a Nested Convergence Event MUST occur with 1141 different neighbors. A possible observation from a Nested 1142 Convergence Event will be the withdrawal of routes from one neighbor 1143 while the routes of another neighbor are being installed. 1145 Measurement Units: N/A 1147 Issues: None 1149 See Also: Convergence Event 1151 4. Security Considerations 1153 Benchmarking activities as described in this memo are limited to 1154 technology characterization using controlled stimuli in a laboratory 1155 environment, with dedicated address space and the constraints 1156 specified in the sections above. 1158 The benchmarking network topology will be an independent test setup 1159 and MUST NOT be connected to devices that may forward the test 1160 traffic into a production network, or misroute traffic to the test 1161 management network. 1163 Further, benchmarking is performed on a "black-box" basis, relying 1164 solely on measurements observable external to the DUT/SUT. 1166 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 1167 benchmarking purposes. Any implications for network security arising 1168 from the DUT/SUT SHOULD be identical in the lab and in production 1169 networks. 1171 5. IANA Considerations 1173 This document requires no IANA considerations. 1175 6. Acknowledgements 1177 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1178 Peter De Vriendt and the BMWG for their contributions to this work. 1180 7. References 1182 7.1. Normative References 1184 [Br91] Bradner, S., "Benchmarking terminology for network 1185 interconnection devices", RFC 1242, July 1991. 1187 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1188 Requirement Levels", BCP 14, RFC 2119, March 1997. 1190 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1191 Network Interconnect Devices", RFC 2544, March 1999. 1193 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1194 environments", RFC 1195, December 1990. 1196 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1197 IPv6", RFC 5340, July 2008. 1199 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1200 October 2008. 1202 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1203 Metrics", RFC 3357, August 2002. 1205 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1206 Devices", RFC 2285, February 1998. 1208 [Mo06] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., 1209 and J. Perser, "Packet Reordering Metrics", RFC 4737, 1210 November 2006. 1212 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1214 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1215 "Terminology for Benchmarking Network-layer Traffic Control 1216 Mechanisms", RFC 4689, October 2006. 1218 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1219 IGP Data Plane Route Convergence", 1220 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1221 progress), March 2009. 1223 [Po09m] Poretsky, S. and B. Imhoff, "Benchmarking Methodology for 1224 Link-State IGP Data Plane Route Convergence", 1225 draft-ietf-bmwg-igp-dataplane-conv-meth-18 (work in 1226 progress), July 2009. 1228 7.2. Informative References 1230 [Ca01] Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained 1231 View of High Performance Networking", NANOG 22, June 2001. 1233 [Ci03] Ciavattone, L., Morton, A., and G. Ramachandran, 1234 "Standardized Active Measurements on a Tier 1 IP Backbone", 1235 IEEE Communications Magazine p90-97, May 2003. 1237 Authors' Addresses 1239 Scott Poretsky 1240 Allot Communications 1241 67 South Bedford Street, Suite 400 1242 Burlington, MA 01803 1243 USA 1245 Phone: + 1 508 309 2179 1246 Email: sporetsky@allot.com 1247 Brent Imhoff 1248 Juniper Networks 1249 1194 North Mathilda Ave 1250 Sunnyvale, CA 94089 1251 USA 1253 Phone: + 1 314 378 2571 1254 Email: bimhoff@planetspork.com 1256 Kris Michielsen 1257 Cisco Systems 1258 6A De Kleetlaan 1259 Diegem, BRABANT 1831 1260 Belgium 1262 Email: kmichiel@cisco.com