idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-term-20.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 8, 2010) is 5156 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-meth-18 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: September 9, 2010 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 March 8, 2010 10 Terminology for Benchmarking Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-term-20 13 Abstract 15 This document describes the terminology for benchmarking Interior 16 Gateway Protocol (IGP) Route Convergence. The terminology is to be 17 used for benchmarking IGP convergence time through externally 18 observable (black box) data plane measurements. The terminology can 19 be applied to any link-state IGP, such as ISIS and OSPF. 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt. 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html. 42 This Internet-Draft will expire on September 9, 2010. 44 Copyright Notice 46 Copyright (c) 2010 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the BSD License. 59 This document may contain material from IETF Documents or IETF 60 Contributions published or made publicly available before November 61 10, 2008. The person(s) controlling the copyright in some of this 62 material may not have granted the IETF Trust the right to allow 63 modifications of such material outside the IETF Standards Process. 64 Without obtaining an adequate license from the person(s) controlling 65 the copyright in such materials, this document may not be modified 66 outside the IETF Standards Process, and derivative works of it may 67 not be created outside the IETF Standards Process, except to format 68 it for publication as an RFC or to translate it into languages other 69 than English. 71 Table of Contents 73 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 5 74 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 5 75 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 5 76 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 6 77 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 6 78 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 6 79 3.1.3. Network Convergence . . . . . . . . . . . . . . . . . 7 80 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 7 81 3.2.1. Traffic Start Instant . . . . . . . . . . . . . . . . 7 82 3.2.2. Convergence Event Instant . . . . . . . . . . . . . . 8 83 3.2.3. Convergence Recovery Instant . . . . . . . . . . . . . 8 84 3.2.4. First Route Convergence Instant . . . . . . . . . . . 9 85 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 9 86 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 9 87 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 10 88 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 11 89 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 11 90 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 11 91 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 11 92 3.4.4. Next-Best Egress Interface . . . . . . . . . . . . . . 12 93 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 12 94 3.5.1. Rate-Derived Method . . . . . . . . . . . . . . . . . 12 95 3.5.2. Loss-Derived Method . . . . . . . . . . . . . . . . . 14 96 3.5.3. Route-Specific Loss-Derived Method . . . . . . . . . . 16 97 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 17 98 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 17 99 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 17 100 3.6.3. Route-Specific Convergence Time . . . . . . . . . . . 18 101 3.6.4. Loss-Derived Convergence Time . . . . . . . . . . . . 20 102 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 21 103 3.6.6. Loss-Derived Loss of Connectivity Period . . . . . . . 22 104 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 23 105 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 23 106 3.7.2. Packet Loss . . . . . . . . . . . . . . . . . . . . . 23 107 3.7.3. Convergence Packet Loss . . . . . . . . . . . . . . . 23 108 3.7.4. Connectivity Packet Loss . . . . . . . . . . . . . . . 24 109 3.7.5. Packet Sampling Interval . . . . . . . . . . . . . . . 25 110 3.7.6. Sustained Convergence Validation Time . . . . . . . . 25 111 3.7.7. Forwarding Delay Threshold . . . . . . . . . . . . . . 26 112 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 26 113 3.8.1. Stale Forwarding . . . . . . . . . . . . . . . . . . . 26 114 3.8.2. Nested Convergence Event . . . . . . . . . . . . . . . 27 115 4. Security Considerations . . . . . . . . . . . . . . . . . . . 27 116 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28 117 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 28 118 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 28 119 7.1. Normative References . . . . . . . . . . . . . . . . . . . 28 120 7.2. Informative References . . . . . . . . . . . . . . . . . . 29 121 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 29 123 1. Introduction and Scope 125 This draft describes the terminology for benchmarking Link-State 126 Interior Gateway Protocol (IGP) Convergence. The motivation and 127 applicability for this benchmarking is provided in [Po09a]. The 128 methodology to be used for this benchmarking is described in [Po09m]. 129 The purpose of this document is to introduce new terms required to 130 complete execution of the IGP Route Methodology [Po09m]. 132 IGP convergence time is measured on the data plane at the Tester by 133 observing packet loss through the DUT. The methodology and 134 terminology to be used for benchmarking IGP Convergence can be 135 applied to IPv4 and IPv6 traffic and link-state IGPs such as ISIS 136 [Ca90][Ho08], OSPF [Mo98][Co08], and others. 138 2. Existing Definitions 140 This document uses existing terminology defined in other BMWG work. 141 Examples include, but are not limited to: 143 Frame Loss Rate [Ref.[Br91], section 3.6] 144 Throughput [Ref.[Br91], section 3.17] 145 Offered Load [Ref.[Ma98], section 3.5.2] 146 Forwarding Rate [Ref.[Ma98], section 3.6.1] 147 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 148 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 149 Out-of-Order Packet [Ref.[Po06], section 3.3.4] 150 Duplicate Packet [Ref.[Po06], section 3.3.5] 151 Packet Reordering [Ref.[Mo06], section 3.3] 152 Stream [Ref.[Po06], section 3.3.2] 153 Forwarding Delay [Ref.[Po06], section 3.2.4] 154 Jitter [Ref.[Po06], section 3.2.5] 155 Loss Period [Ref.[Ko02], section 4] 157 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 158 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 159 document are to be interpreted as described in BCP 14, RFC 2119 160 [Br97]. RFC 2119 defines the use of these key words to help make the 161 intent of standards track documents as clear as possible. While this 162 document uses these keywords, this document is not a standards track 163 document. 165 3. Term Definitions 166 3.1. Convergence Types 168 3.1.1. Route Convergence 170 Definition: 172 The process of updating all components of the router, including the 173 Routing Information Base (RIB) and Forwarding Information Base (FIB), 174 along with software and hardware tables, with the most recent route 175 change(s) such that forwarding for a route entry is successful on the 176 Next-Best Egress Interface. 178 Discussion: 180 Route Convergence MUST occur after a Convergence Event. Route 181 Convergence can be observed externally by the rerouting of data 182 traffic for a destination matching a route entry to the Next-best 183 Egress Interface. Completion of Route Convergence may or may not be 184 sustained over time. 186 Measurement Units: N/A 188 Issues: None 190 See Also: 192 Network Convergence, Full Convergence, Convergence Event 194 3.1.2. Full Convergence 196 Definition: 198 Route Convergence for all routes in the FIB. 200 Discussion: 202 Full Convergence MUST occur after a Convergence Event. Full 203 Convergence can be observed externally by the rerouting of data 204 traffic to destinations matching all route entries to the Next-best 205 Egress Interface. Completion of Full Convergence is externally 206 observable from the data plane when the Forwarding Rate of the data 207 plane traffic on the Next-Best Egress Interface equals the Offered 208 Load. 210 Completion of Full Convergence may or may not be sustained over time. 212 Measurement Units: N/A 213 Issues: None 215 See Also: 217 Network Convergence, Route Convergence, Convergence Event, Full 218 Convergence Time, Convergence Recovery Instant 220 3.1.3. Network Convergence 222 Definition: 224 Full Convergence in all routers throughout the network. 226 Discussion: 228 Network Convergence includes all Route Convergence operations for all 229 routers in the network following a Convergence Event. 231 Completion of Network Convergence can be observed by recovery of the 232 network Forwarding Rate to equal the Offered Load, with no Stale 233 Forwarding, and no Blenders [Ca01][Ci03]. 235 Completion of Network Convergence may or may not be sustained over 236 time. 238 Measurement Units: N/A 240 Issues: None 242 See Also: 244 Route Convergence, Full Convergence, Stale Forwarding 246 3.2. Instants 248 3.2.1. Traffic Start Instant 250 Definition: 252 The time instant the Tester sends out the first data packet to the 253 DUT. 255 Discussion: 257 If using the Loss-Derived Method or the Route-Specific Loss-Derived 258 Method to benchmark IGP convergence time, and the applied Convergence 259 Event does not cause instantaneous traffic loss for all routes at the 260 Convergence Event Instant then the Tester SHOULD collect a timestamp 261 on the Traffic Start Instant in order to measure the period of time 262 between the Traffic Start Instant and Convergence Event Instant. 264 Measurement Units: 266 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 267 microseconds. 269 Issues: None 271 See Also: 273 Convergence Event Instant, Route-Specific Convergence Time, Loss- 274 Derived Convergence Time. 276 3.2.2. Convergence Event Instant 278 Definition: 280 The time instant that a Convergence Event occurs. 282 Discussion: 284 If the Convergence Event causes instantaneous traffic loss on the 285 Preferred Egress Interface, the Convergence Event Instant is 286 observable from the data plane as the instant that the DUT begins to 287 exhibit packet loss. 289 The Tester SHOULD collect a timestamp on the Convergence Event 290 Instant if it is not observable from the data plane. 292 Measurement Units: 294 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 295 microseconds. 297 Issues: None 299 See Also: Convergence Event 301 3.2.3. Convergence Recovery Instant 303 Definition: 305 The time instant that Full Convergence has completed. 307 Discussion: 309 The Full Convergence completed state MUST be maintained for an 310 interval of duration equal to the Sustained Convergence Validation 311 Time in order to validate the Convergence Recovery Instant. 313 The Convergence Recovery Instant is observable from the data plane as 314 the instant the DUT forwards traffic to all destinations over the 315 Next-Best Egress Interface. 317 Measurement Units: 319 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 320 microseconds. 322 Issues: None 324 See Also: 326 Sustained Convergence Validation Time, Full Convergence 328 3.2.4. First Route Convergence Instant 330 Definition: 332 The time instant the first route entry completes Route Convergence 333 following a Convergence Event 335 Discussion: 337 Any route may be the first to complete Route Convergence. The First 338 Route Convergence Instant is observable from the data plane as the 339 instant that the first packet is received from the Next-Best Egress 340 Interface. 342 Measurement Units: 344 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 345 microseconds. 347 Issues: None 349 See Also: Route Convergence 351 3.3. Transitions 353 3.3.1. Convergence Event Transition 355 Definition: 357 A time interval following a Convergence Event in which Forwarding 358 Rate on the Preferred Egress Interface gradually reduces to zero. 360 Discussion: 362 The Forwarding Rate during a Convergence Event Transition may not 363 decrease linearly. 365 The Forwarding Rate observed on all DUT egress interfaces may or may 366 not decrease to zero. 368 The Offered Load, the number of routes, and the Packet Sampling 369 Interval influence the observations of the Convergence Event 370 Transition using the Rate-Derived Method. This is further discussed 371 with the term "Rate-Derived Method". 373 Measurement Units: seconds 375 Issues: None 377 See Also: 379 Convergence Event, Rate-Derived Method 381 3.3.2. Convergence Recovery Transition 383 Definition: 385 A time interval following the First Route Convergence Instant in 386 which Forwarding Rate on the Next-Best Egress Interface gradually 387 increases to equal the Offered Load. 389 Discussion: 391 The Forwarding Rate observed during a Convergence Recovery Transition 392 may not increase linearly. 394 The Offered Load, the number of routes, and the Packet Sampling 395 Interval influence the observations of the Convergence Recovery 396 Transition using the Rate-Derived Method. This is further discussed 397 with the term "Rate-Derived Method". 399 Measurement Units: seconds 401 Issues: None 403 See Also: 405 Full Convergence,First Route Convergence Instant, Rate-Derived Method 407 3.4. Interfaces 409 3.4.1. Local Interface 411 Definition: 413 An interface on the DUT. 415 Discussion: 417 A failure of the Local Interface indicates that the failure occurred 418 directly on the DUT. 420 Measurement Units: N/A 422 Issues: None 424 See Also: Remote Interface 426 3.4.2. Remote Interface 428 Definition: 430 An interface on a neighboring router that is not directly connected 431 to any interface on the DUT. 433 Discussion: 435 A failure of a Remote Interface indicates that the failure occurred 436 on a neighbor router's interface that is not directly connected to 437 the DUT. 439 Measurement Units: N/A 441 Issues: None 443 See Also: Local Interface 445 3.4.3. Preferred Egress Interface 447 Definition: 449 The outbound interface from the DUT for traffic routed to the 450 preferred next-hop. 452 Discussion: 454 The Preferred Egress Interface is the egress interface prior to a 455 Convergence Event. 457 Measurement Units: N/A 459 Issues: None 461 See Also: Next-Best Egress Interface 463 3.4.4. Next-Best Egress Interface 465 Definition: 467 The outbound interface from the DUT for traffic routed to the second- 468 best next-hop. 470 Discussion: 472 The Next-Best Egress Interface becomes the egress interface after a 473 Convergence Event. 475 Measurement Units: N/A 477 Issues: None 479 See Also: Preferred Egress Interface 481 3.5. Benchmarking Methods 483 3.5.1. Rate-Derived Method 485 Definition: 487 The method to calculate convergence time benchmarks from observing 488 Forwarding Rate each Packet Sampling Interval. 490 Discussion: 492 Figure 1 shows an example of the Forwarding Rate change in time 493 during convergence as observed when using the Rate-Derived Method. 495 ^ Traffic Convergence 496 Fwd | Start Recovery 497 Rate | Instant Instant 498 | Offered ^ ^ 499 | Load --> ----------\ /----------- 500 | \ /<--- Convergence 501 | \ Packet / Recovery 502 | Convergence --->\ Loss / Transition 503 | Event \ / 504 | Transition \---------/ <-- Max Packet Loss 505 | 506 +---------------------------------------------------------> 507 ^ ^ time 508 Convergence First Route 509 Event Instant Convergence Instant 511 Figure 1: Rate-Derived Convergence Graph 513 The Offered Load SHOULD consist of a single Stream [Po06]. If 514 sending multiple Streams, the measured traffic rate statistics for 515 all Streams MUST be added together. 517 The destination addresses for the Offered Load MUST be distributed 518 such that all routes or a statistically representative subset of all 519 routes are matched and each of these routes is offered an equal share 520 of the Offered Load. It is RECOMMENDED to send traffic to all 521 routes, but a statistically representative subset of all routes can 522 be used if required. 524 At least one packet per route for all routes matched in the Offered 525 Load MUST be offered to the DUT within each Packet Sampling Interval. 526 For maximum accuracy the value for the Packet Sampling Interval 527 SHOULD be as small as possible, but the presence of Jitter [Po06] may 528 enforce using a larger Packet Sampling Interval. 530 The Offered Load, Jitter, the number of routes, and the Packet 531 Sampling Interval influence the observations for the Rate-Derived 532 Method. It may be difficult to identify the different convergence 533 time instants in the Rate-Derived Convergence Graph. For example, it 534 is possible that a Convergence Event causes the Forwarding Rate to 535 drop to zero, while this may not be observed in the Forwarding Rate 536 measurements if the Packet Sampling Interval is too large. 538 Jitter causes fluctuations in the number of received packets during 539 each Packet Sampling Interval. To account for the presence of Jitter 540 in determining if a convergence instant has been reached, Jitter 541 SHOULD be observed during each Packet Sampling Interval. The minimum 542 and maximum number of packets expected in a Packet Sampling Interval 543 in presence of Jitter can be calculated with Equation 1. 545 number of packets expected in a Packet Sampling Interval 546 in presence of Jitter 547 = expected number of packets without Jitter 548 +/-(max Jitter during Packet Sampling Interval * Offered Load) 550 Equation 1 552 To determine if a convergence instant has been reached the number of 553 packets received in a Packet Sampling Interval is compared with the 554 range of expected number of packets calculated in Equation 1. 556 Metrics measured at the Packet Sampling Interval MUST include 557 Forwarding Rate and packet loss. 559 Rate-Derived Method is a RECOMMENDED method to measure convergence 560 time benchmarks. 562 To measure convergence time benchmarks for Convergence Events that do 563 not cause instantaneous traffic loss for all routes at the 564 Convergence Event Instant, the Tester SHOULD collect a timestamp of 565 the Convergence Event Instant and the Tester SHOULD observe 566 Forwarding Rate separately on the Next-Best Egress Interface. 568 Since the Rate-Derived Method does not distinguish between individual 569 traffic destinations, it SHOULD NOT be used for any route specific 570 measurements. Therefor Rate-Derived Method SHOULD NOT be used to 571 benchmark Route Loss of Connectivity Period. 573 Measurement Units: N/A 575 Issues: None 577 See Also: 579 Packet Sampling Interval, Convergence Event, Convergence Event 580 Instant, Full Convergence 582 3.5.2. Loss-Derived Method 584 Definition: 586 The method to calculate the Loss-Derived Convergence Time and Loss- 587 Derived Loss of Connectivity Period benchmarks from the amount of 588 packet loss. 590 Discussion: 592 The Offered Load SHOULD consist of a single Stream [Po06]. If 593 sending multiple Streams, the measured traffic rate statistics for 594 all Streams MUST be added together. 596 The destination addresses for the Offered Load MUST be distributed 597 such that all routes or a statistically representative subset of all 598 routes are matched and each of these routes is offered an equal share 599 of the Offered Load. It is RECOMMENDED to send traffic to all 600 routes, but a statistically representative subset of all routes can 601 be used if required. 603 Loss-Derived Method SHOULD always be combined with Rate-Derived 604 Method in order to observe Full Convergence completion. The total 605 amount of Convergence Packet Loss is collected after Full Convergence 606 completion. 608 To measure convergence time and loss of connectivity benchmarks, the 609 Tester SHOULD in general observe packet loss on all DUT egress 610 interfaces (Connectivity Packet Loss). 612 To measure convergence time benchmarks for Convergence Events that do 613 not cause instantaneous traffic loss for all routes at the 614 Convergence Event Instant, the Tester SHOULD collect timestamps of 615 the Start Traffic Instant and of the Convergence Event Instant, and 616 the Tester SHOULD observe packet loss separately on the Next-Best 617 Egress Interface (Convergence Packet Loss). 619 Since Loss-Derived Method does not distinguish between traffic 620 destinations and the packet loss statistics are only collected after 621 Full Convergence completion, this method can only be used to measure 622 average values over all routes. For these reasons Loss-Derived 623 Method can only be used to benchmark Loss-Derived Convergence Time 624 and Loss-Derived Loss of Connectivity Period. 626 Note that the Loss-Derived Method measures an average over all 627 routes, including the routes that may not be impacted by the 628 Convergence Event, such as routes via non-impacted members of ECMP or 629 parallel links. 631 Measurement Units: seconds 633 Issues: None 635 See Also: 637 Loss-Derived Convergence Time, Loss-Derived Loss of Connectivity 638 Period, Convergence Packet Loss 640 3.5.3. Route-Specific Loss-Derived Method 642 Definition: 644 The method to calculate the Route-Specific Convergence Time benchmark 645 from the amount of packet loss during convergence for a specific 646 route entry. 648 Discussion: 650 To benchmark Route-Specific Convergence Time, the Tester provides an 651 Offered Load that consists of multiple Streams [Po06]. Each Stream 652 has a single destination address matching a different route entry, 653 for all routes or a statistically representative subset of all 654 routes. Convergence Packet Loss is measured for each Stream 655 separately. 657 Route-Specific Loss-Derived Method SHOULD always be combined with 658 Rate-Derived Method in order to observe Full Convergence completion. 659 The total amount of Convergence Packet Loss for each Stream is 660 collected after Full Convergence completion. 662 Route-Specific Loss-Derived Method is a RECOMMENDED method to measure 663 convergence time benchmarks. 665 To measure convergence time and loss of connectivity benchmarks, the 666 Tester SHOULD in general observe packet loss on all DUT egress 667 interfaces (Connectivity Packet Loss). 669 To measure convergence time benchmarks for Convergence Events that do 670 not cause instantaneous traffic loss for all routes at the 671 Convergence Event Instant, the Tester SHOULD collect timestamps of 672 the Start Traffic Instant and of the Convergence Event Instant, and 673 the Tester SHOULD observe packet loss separately on the Next-Best 674 Egress Interface (Convergence Packet Loss). 676 Since Route-Specific Loss-Derived Method uses traffic streams to 677 individual routes, it measures packet loss as it would be experienced 678 by a network user. For this reason Route-Specific Loss-Derived 679 Method is RECOMMENDED to measure Route-Specific Convergence Time 680 benchmarks and Route Loss of Connectivity Period benchmarks. 682 Measurement Units: seconds 684 Issues: None 686 See Also: 688 Route-Specific Convergence Time, Route Loss of Connectivity Period, 689 Convergence Packet Loss 691 3.6. Benchmarks 693 3.6.1. Full Convergence Time 695 Definition: 697 The time duration of the period between the Convergence Event Instant 698 and the Convergence Recovery Instant as observed using the Rate- 699 Derived Method. 701 Discussion: 703 Using the Rate-Derived Method, Full Convergence Time can be 704 calculated as the time difference between the Convergence Event 705 Instant and the Convergence Recovery Instant, as shown in Equation 2. 707 Full Convergence Time = 708 Convergence Recovery Instant - Convergence Event Instant 710 Equation 2 712 The Convergence Event Instant can be derived from the Forwarding Rate 713 observation or from a timestamp collected by the Tester. 715 For the testcases described in [Po09m], it is expected that Full 716 Convergence Time equals the maximum Route-Specific Convergence Time 717 when benchmarking all routes in FIB using the Route-Specific Loss- 718 Derived Method. 720 It is not possible to measure Full Convergence Time using the Loss- 721 Derived Method. 723 Measurement Units: seconds 725 Issues: None 727 See Also: 729 Full Convergence, Rate-Derived Method, Route-Specific Loss-Derived 730 Method 732 3.6.2. First Route Convergence Time 734 Definition: 736 The duration of the period between the Convergence Event Instant and 737 the First Route Convergence Instant as observed using the Rate- 738 Derived Method. 740 Discussion: 742 Using the Rate-Derived Method, First Route Convergence Time can be 743 calculated as the time difference between the Convergence Event 744 Instant and the First Route Convergence Instant, as shown with 745 Equation 3. 747 First Route Convergence Time = 748 First Route Convergence Instant - Convergence Event Instant 750 Equation 3 752 The Convergence Event Instant can be derived from the Forwarding Rate 753 observation or from a timestamp collected by the Tester. 755 For the testcases described in [Po09m], it is expected that First 756 Route Convergence Time equals the minimum Route-Specific Convergence 757 Time when benchmarking all routes in FIB using the Route-Specific 758 Loss-Derived Method. 760 It is not possible to measure First Route Convergence Time using the 761 Loss-Derived Method. 763 Measurement Units: seconds 765 Issues: None 767 See Also: 769 Rate-Derived Method, Route-Specific Loss-Derived Method, First Route 770 Convergence Instant 772 3.6.3. Route-Specific Convergence Time 774 Definition: 776 The amount of time it takes for Route Convergence to be completed for 777 a specific route, as calculated from the amount of packet loss during 778 convergence for a single route entry. 780 Discussion: 782 Route-Specific Convergence Time can only be measured using the Route- 783 Specific Loss-Derived Method. 785 If the applied Convergence Event causes instantaneous traffic loss 786 for all routes at the Convergence Event Instant, Connectivity Packet 787 Loss should be observed. Connectivity Packet Loss is the combined 788 packet loss observed on Preferred Egress Interface and Next-Best 789 Egress Interface. When benchmarking Route-Specific Convergence Time, 790 Connectivity Packet Loss is measured and Equation 4 is applied for 791 each measured route. The calculation is equal to Equation 8 in 792 Section 3.6.5. 794 Route-Specific Convergence Time = 795 Connectivity Packet Loss for specific route/Offered Load per route 797 Equation 4 799 If the applied Convergence Event does not cause instantaneous traffic 800 loss for all routes at the Convergence Event Instant, then the Tester 801 SHOULD collect timestamps of the Traffic Start Instant and of the 802 Convergence Event Instant, and the Tester SHOULD observe Convergence 803 Packet Loss separately on the Next-Best Egress Interface. When 804 benchmarking Route-Specific Convergence Time, Convergence Packet Loss 805 is measured and Equation 5 is applied for each measured route. 807 Route-Specific Convergence Time = 808 Convergence Packet Loss for specific route/Offered Load per route 809 - (Convergence Event Instant - Traffic Start Instant) 811 Equation 5 813 The Convergence Event Instant and Traffic Start Instant SHOULD be 814 collected by the Tester. 816 The Route-Specific Convergence Time benchmarks enable minimum, 817 maximum, average, and median convergence time measurements to be 818 reported by comparing the results for the different route entries. 819 It also enables benchmarking of convergence time when configuring a 820 priority value for route entry(ies). Since multiple Route-Specific 821 Convergence Times can be measured it is possible to have an array of 822 results. The format for reporting Route-Specific Convergence Time is 823 provided in [Po09m]. 825 Measurement Units: seconds 827 Issues: None 829 See Also: 831 Convergence Event, Convergence Packet Loss, Connectivity Packet Loss, 832 Route Convergence 834 3.6.4. Loss-Derived Convergence Time 836 Definition: 838 The average Route Convergence time for all routes in FIB, as 839 calculated from the amount of packet loss during convergence. 841 Discussion: 843 Loss-Derived Convergence Time is measured using the Loss-Derived 844 Method. 846 If the applied Convergence Event causes instantaneous traffic loss 847 for all routes at the Convergence Event Instant, Connectivity Packet 848 Loss should be observed. Connectivity Packet Loss is the combined 849 packet loss observed on Preferred Egress Interface and Next-Best 850 Egress Interface. When benchmarking Loss-Derived Convergence Time, 851 Connectivity Packet Loss is measured and Equation 6 is applied. 853 Loss-Derived Convergence Time = 854 Connectivity Packet Loss/Offered Load 856 Equation 6 858 If the applied Convergence Event does not cause instantaneous traffic 859 loss for all routes at the Convergence Event Instant, then the Tester 860 SHOULD collect timestamps of the Start Traffic Instant and of the 861 Convergence Event Instant and the Tester SHOULD observe Convergence 862 Packet Loss separately on the Next-Best Egress Interface. When 863 benchmarking Loss-Derived Convergence Time, Convergence Packet Loss 864 is measured and Equation 7 is applied. 866 Loss-Derived Convergence Time = 867 Convergence Packet Loss/Offered Load 868 - (Convergence Event Instant - Traffic Start Instant) 870 Equation 7 872 The Convergence Event Instant and Traffic Start Instant SHOULD be 873 collected by the Tester. 875 Measurement Units: seconds 877 Issues: None 879 See Also: 881 Convergence Packet Loss, Connectivity Packet Loss, Route Convergence 883 3.6.5. Route Loss of Connectivity Period 885 Definition: 887 The time duration of traffic loss for a specific route entry 888 following a Convergence Event until Full Convergence completion, as 889 observed using the Route-Specific Loss-Derived Method. 891 Discussion: 893 In general the Route Loss of Connectivity Period is not equal to the 894 Route-Specific Convergence Time. If the DUT continues to forward 895 traffic to the Preferred Egress Interface after the Convergence Event 896 is applied then the Route Loss of Connectivity Period will be smaller 897 than the Route-Specific Convergence Time. This is also specifically 898 the case after reversing a failure event. 900 The Route Loss of Connectivity Period may be equal to the Route- 901 Specific Convergence Time if, as a characteristic of the Convergence 902 Event, traffic for all routes starts dropping instantaneously on the 903 Convergence Event Instant. See discussion in [Po09m]. 905 For the testcases described in [Po09m] the Route Loss of Connectivity 906 Period is expected to be a single Loss Period [Ko02]. 908 When benchmarking Route Loss of Connectivity Period, Connectivity 909 Packet Loss is measured for each route and Equation 8 is applied for 910 each measured route entry. The calculation is equal to Equation 4 in 911 Section 3.6.3. 913 Route Loss of Connectivity Period = 914 Connectivity Packet Loss for specific route/Offered Load per route 916 Equation 8 918 Route Loss of Connectivity Period SHOULD be measured using Route- 919 Specific Loss-Derived Method. 921 Measurement Units: seconds 923 Issues: None 925 See Also: 927 Route-Specific Convergence Time, Route-Specific Loss-Derived Method, 928 Connectivity Packet Loss 930 3.6.6. Loss-Derived Loss of Connectivity Period 932 Definition: 934 The average time duration of traffic loss for all routes following a 935 Convergence Event until Full Convergence completion, as observed 936 using the Loss-Derived Method. 938 Discussion: 940 In general the Loss-Derived Loss of Connectivity Period is not equal 941 to the Loss-Derived Convergence Time. If the DUT continues to 942 forward traffic to the Preferred Egress Interface after the 943 Convergence Event is applied then the Loss-Derived Loss of 944 Connectivity Period will be smaller than the Loss-Derived Convergence 945 Time. This is also specifically the case after reversing a failure 946 event. 948 The Loss-Derived Loss of Connectivity Period may be equal to the 949 Loss-Derived Convergence Time if, as a characteristic of the 950 Convergence Event, traffic for all routes starts dropping 951 instantaneously on the Convergence Event Instant. See discussion in 952 [Po09m]. 954 For the testcases described in [Po09m] each route's Route Loss of 955 Connectivity Period is expected to be a single Loss Period [Ko02]. 957 When benchmarking Loss-Derived Loss of Connectivity Period, 958 Connectivity Packet Loss is measured for all routes and Equation 9 is 959 applied. The calculation is equal to Equation 6 in Section 3.6.4. 961 Loss-Derived Loss of Connectivity Period = 962 Connectivity Packet Loss for all routes/Offered Load 964 Equation 9 966 Loss-Derived Loss of Connectivity Period SHOULD be measured using 967 Loss-Derived Method. 969 Measurement Units: seconds 971 Issues: None 973 See Also: 975 Loss-Derived Convergence Time, Loss-Derived Method, Connectivity 976 Packet Loss 978 3.7. Measurement Terms 980 3.7.1. Convergence Event 982 Definition: 984 The occurrence of a planned or unplanned event in the network that 985 will result in a change in the egress interface of the Device Under 986 Test (DUT) for routed packets. 988 Discussion: 990 Convergence Events include but are not limited to link loss, routing 991 protocol session loss, router failure, configuration change, and 992 better next-hop learned via a routing protocol. 994 Measurement Units: N/A 996 Issues: None 998 See Also: Convergence Event Instant 1000 3.7.2. Packet Loss 1002 Definition: 1004 The number of packets that should have been forwarded by a DUT under 1005 a constant Offered Load that were not forwarded due to lack of 1006 resources. 1008 Discussion: 1010 Packet Loss is a modified version of the term "Frame Loss Rate" as 1011 defined in [Br91]. The term "Frame Loss" is intended for Ethernet 1012 Frames while "Packet Loss" is intended for IP packets. 1014 Measurement units: Number of offered packets that are not forwarded. 1016 Issues: None 1018 See Also: Convergence Packet Loss 1020 3.7.3. Convergence Packet Loss 1022 Definition: 1024 The number of packets lost due to a Convergence Event until Full 1025 Convergence completes, as observed on the Next-Best Egress Interface. 1027 Discussion: 1029 Convergence Packet Loss is observed on the Next-Best Egress 1030 Interface. It only needs to be observed for Convergence Events that 1031 do not cause instantaneous traffic loss at Convergence Event Instant. 1033 Convergence Packet Loss includes packets that were lost and packets 1034 that were delayed due to buffering. The maximum acceptable 1035 Forwarding Delay (Forwarding Delay Threshold) is a parameter of the 1036 methodology, if it is applied it MUST be reported. 1038 Measurement Units: number of packets 1040 Issues: None 1042 See Also: 1044 Packet Loss, Full Convergence, Convergence Event, Connectivity Packet 1045 Loss 1047 3.7.4. Connectivity Packet Loss 1049 Definition: 1051 The number of packets lost due to a Convergence Event until Full 1052 Convergence completes. 1054 Discussion: 1056 Connectivity Packet Loss is observed on all DUT egress interfaces. 1058 Connectivity Packet Loss includes packets that were lost and packets 1059 that were delayed due to buffering. The maximum acceptable 1060 Forwarding Delay (Forwarding Delay Threshold) is a parameter of the 1061 methodology, if it is applied it MUST be reported. 1063 Measurement Units: number of packets 1065 Issues: None 1067 See Also: 1069 Packet Loss, Route Loss of Connectivity Period, Convergence Event, 1070 Convergence Packet Loss 1072 3.7.5. Packet Sampling Interval 1074 Definition: 1076 The interval at which the Tester (test equipment) polls to make 1077 measurements for arriving packets. 1079 Discussion: 1081 At least one packet per route for all routes matched in the Offered 1082 Load MUST be offered to the DUT within the Packet Sampling Interval. 1083 Metrics measured at the Packet Sampling Interval MUST include 1084 Forwarding Rate and received packets. 1086 Packet Sampling Interval can influence the convergence graph as 1087 observed with the Rate-Derived Method. This is particularly true 1088 when implementations complete Full Convergence in less time than the 1089 Packet Sampling Interval. The Convergence Event Instant and First 1090 Route Convergence Instant may not be easily identifiable and the 1091 Rate-Derived Method may produce a larger than actual convergence 1092 time. 1094 Using a small Packet Sampling Interval in the presence of Jitter 1095 [Po06] may cause fluctuations of the Forwarding Rate observation and 1096 can prevent correct observation of the different convergence time 1097 instants. 1099 The value of the Packet Sampling Interval only contributes to the 1100 measurement accuracy of the Rate-Derived Method. For maximum 1101 accuracy the value for the Packet Sampling Interval SHOULD be as 1102 small as possible, but the presence of Jitter may enforce using a 1103 larger Packet Sampling Interval. 1105 Measurement Units: seconds 1107 Issues: None 1109 See Also: Rate-Derived Method 1111 3.7.6. Sustained Convergence Validation Time 1113 Definition: 1115 The amount of time for which the completion of Full Convergence is 1116 maintained without additional packet loss. 1118 Discussion: 1120 The purpose of the Sustained Convergence Validation Time is to 1121 produce convergence benchmarks protected against fluctuation in 1122 Forwarding Rate after the completion of Full Convergence is observed. 1123 The RECOMMENDED Sustained Convergence Validation Time to be used is 1124 the time to send 5 consecutive packets to each destination with a 1125 minimum of 5 seconds. The BMWG selected 5 seconds based upon [Br99] 1126 which recommends waiting 2 seconds for residual frames to arrive 1127 (this is the Forwarding Delay Threshold for the last packet sent) and 1128 5 seconds for DUT restabilization. 1130 Measurement Units: seconds 1132 Issues: None 1134 See Also: 1136 Full Convergence, Convergence Recovery Instant 1138 3.7.7. Forwarding Delay Threshold 1140 Definition: 1142 The maximum Forwarding Delay for a packet to be accepted. 1144 Discussion: 1146 Applying a Forwarding Delay Threshold allows to consider packets with 1147 a too large Forwarding Delay as being lost, as is required for some 1148 applications (e.g. voice, video, etc.). The Forwarding Delay 1149 Threshold is a parameter of the methodology, if it is applied it MUST 1150 be reported. 1152 Measurement Units: seconds 1154 Issues: None 1156 See Also: 1158 Convergence Packet Loss, Connectivity Packet Loss 1160 3.8. Miscellaneous Terms 1162 3.8.1. Stale Forwarding 1164 Definition: 1166 Forwarding of traffic to route entries that no longer exist or to 1167 route entries with next-hops that are no longer preferred. 1169 Discussion: 1171 Stale Forwarding can be caused by a Convergence Event and can 1172 manifest as a "black-hole" or microloop that produces packet loss, or 1173 out-of-order packets, or delayed packets. Stale Forwarding can exist 1174 until Network Convergence is completed. 1176 Measurement Units: N/A 1178 Issues: None 1180 See Also: Network Convergence 1182 3.8.2. Nested Convergence Event 1184 Definition: 1186 The occurrence of a Convergence Event while the route table is 1187 converging from a prior Convergence Event. 1189 Discussion: 1191 The Convergence Events for a Nested Convergence Event MUST occur with 1192 different neighbors. A possible observation from a Nested 1193 Convergence Event will be the withdrawal of routes from one neighbor 1194 while the routes of another neighbor are being installed. 1196 Measurement Units: N/A 1198 Issues: None 1200 See Also: Convergence Event 1202 4. Security Considerations 1204 Benchmarking activities as described in this memo are limited to 1205 technology characterization using controlled stimuli in a laboratory 1206 environment, with dedicated address space and the constraints 1207 specified in the sections above. 1209 The benchmarking network topology will be an independent test setup 1210 and MUST NOT be connected to devices that may forward the test 1211 traffic into a production network, or misroute traffic to the test 1212 management network. 1214 Further, benchmarking is performed on a "black-box" basis, relying 1215 solely on measurements observable external to the DUT/SUT. 1217 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 1218 benchmarking purposes. Any implications for network security arising 1219 from the DUT/SUT SHOULD be identical in the lab and in production 1220 networks. 1222 5. IANA Considerations 1224 This document requires no IANA considerations. 1226 6. Acknowledgements 1228 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1229 Peter De Vriendt, Anuj Dewagan and the BMWG for their contributions 1230 to this work. 1232 7. References 1234 7.1. Normative References 1236 [Br91] Bradner, S., "Benchmarking terminology for network 1237 interconnection devices", RFC 1242, July 1991. 1239 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1240 Requirement Levels", BCP 14, RFC 2119, March 1997. 1242 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1243 Network Interconnect Devices", RFC 2544, March 1999. 1245 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1246 environments", RFC 1195, December 1990. 1248 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1249 IPv6", RFC 5340, July 2008. 1251 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1252 October 2008. 1254 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1255 Metrics", RFC 3357, August 2002. 1257 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1258 Devices", RFC 2285, February 1998. 1260 [Mo06] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., 1261 and J. Perser, "Packet Reordering Metrics", RFC 4737, 1262 November 2006. 1264 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1266 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1267 "Terminology for Benchmarking Network-layer Traffic Control 1268 Mechanisms", RFC 4689, October 2006. 1270 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1271 IGP Data Plane Route Convergence", 1272 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1273 progress), March 2009. 1275 [Po09m] Poretsky, S. and B. Imhoff, "Benchmarking Methodology for 1276 Link-State IGP Data Plane Route Convergence", 1277 draft-ietf-bmwg-igp-dataplane-conv-meth-18 (work in 1278 progress), July 2009. 1280 7.2. Informative References 1282 [Ca01] Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained 1283 View of High Performance Networking", NANOG 22, June 2001. 1285 [Ci03] Ciavattone, L., Morton, A., and G. Ramachandran, 1286 "Standardized Active Measurements on a Tier 1 IP Backbone", 1287 IEEE Communications Magazine p90-97, May 2003. 1289 Authors' Addresses 1291 Scott Poretsky 1292 Allot Communications 1293 67 South Bedford Street, Suite 400 1294 Burlington, MA 01803 1295 USA 1297 Phone: + 1 508 309 2179 1298 Email: sporetsky@allot.com 1299 Brent Imhoff 1300 Juniper Networks 1301 1194 North Mathilda Ave 1302 Sunnyvale, CA 94089 1303 USA 1305 Phone: + 1 314 378 2571 1306 Email: bimhoff@planetspork.com 1308 Kris Michielsen 1309 Cisco Systems 1310 6A De Kleetlaan 1311 Diegem, BRABANT 1831 1312 Belgium 1314 Email: kmichiel@cisco.com