idnits 2.17.1 draft-ietf-bmwg-igp-dataplane-conv-term-22.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 8, 2010) is 4917 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-23) exists of draft-ietf-bmwg-igp-dataplane-conv-meth-20 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Poretsky 3 Internet-Draft Allot Communications 4 Intended status: Informational B. Imhoff 5 Expires: May 12, 2011 Juniper Networks 6 K. Michielsen 7 Cisco Systems 8 November 8, 2010 10 Terminology for Benchmarking Link-State IGP Data Plane Route Convergence 11 draft-ietf-bmwg-igp-dataplane-conv-term-22 13 Abstract 15 This document describes the terminology for benchmarking Interior 16 Gateway Protocol (IGP) Route Convergence. The terminology is to be 17 used for benchmarking IGP convergence time through externally 18 observable (black box) data plane measurements. The terminology can 19 be applied to any link-state IGP, such as ISIS and OSPF. 21 Status of this Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on May 12, 2011. 38 Copyright Notice 40 Copyright (c) 2010 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 This document may contain material from IETF Documents or IETF 54 Contributions published or made publicly available before November 55 10, 2008. The person(s) controlling the copyright in some of this 56 material may not have granted the IETF Trust the right to allow 57 modifications of such material outside the IETF Standards Process. 58 Without obtaining an adequate license from the person(s) controlling 59 the copyright in such materials, this document may not be modified 60 outside the IETF Standards Process, and derivative works of it may 61 not be created outside the IETF Standards Process, except to format 62 it for publication as an RFC or to translate it into languages other 63 than English. 65 Table of Contents 67 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 4 68 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 4 69 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 70 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 5 71 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 5 72 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 5 73 3.1.3. Network Convergence . . . . . . . . . . . . . . . . . 6 74 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 6 75 3.2.1. Traffic Start Instant . . . . . . . . . . . . . . . . 6 76 3.2.2. Convergence Event Instant . . . . . . . . . . . . . . 7 77 3.2.3. Convergence Recovery Instant . . . . . . . . . . . . . 7 78 3.2.4. First Route Convergence Instant . . . . . . . . . . . 8 79 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 8 80 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 8 81 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 9 82 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 10 83 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 10 84 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 10 85 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 10 86 3.4.4. Next-Best Egress Interface . . . . . . . . . . . . . . 11 87 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 11 88 3.5.1. Rate-Derived Method . . . . . . . . . . . . . . . . . 11 89 3.5.2. Loss-Derived Method . . . . . . . . . . . . . . . . . 14 90 3.5.3. Route-Specific Loss-Derived Method . . . . . . . . . . 15 91 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 16 92 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 16 93 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 17 94 3.6.3. Route-Specific Convergence Time . . . . . . . . . . . 18 95 3.6.4. Loss-Derived Convergence Time . . . . . . . . . . . . 19 96 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 20 97 3.6.6. Loss-Derived Loss of Connectivity Period . . . . . . . 21 98 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 22 99 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 22 100 3.7.2. Packet Loss . . . . . . . . . . . . . . . . . . . . . 22 101 3.7.3. Convergence Packet Loss . . . . . . . . . . . . . . . 23 102 3.7.4. Connectivity Packet Loss . . . . . . . . . . . . . . . 23 103 3.7.5. Packet Sampling Interval . . . . . . . . . . . . . . . 24 104 3.7.6. Sustained Convergence Validation Time . . . . . . . . 25 105 3.7.7. Forwarding Delay Threshold . . . . . . . . . . . . . . 25 106 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 26 107 3.8.1. Stale Forwarding . . . . . . . . . . . . . . . . . . . 26 108 3.8.2. Nested Convergence Event . . . . . . . . . . . . . . . 26 109 4. Security Considerations . . . . . . . . . . . . . . . . . . . 27 110 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 111 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 27 112 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 27 113 7.1. Normative References . . . . . . . . . . . . . . . . . . . 27 114 7.2. Informative References . . . . . . . . . . . . . . . . . . 28 115 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 117 1. Introduction and Scope 119 This draft describes the terminology for benchmarking Link-State 120 Interior Gateway Protocol (IGP) Convergence. The motivation and 121 applicability for this benchmarking is provided in [Po09a]. The 122 methodology to be used for this benchmarking is described in [Po10m]. 123 The purpose of this document is to introduce new terms required to 124 complete execution of the IGP Route Methodology [Po10m]. 126 IGP convergence time is measured on the data plane at the Tester by 127 observing packet loss through the DUT. The methodology and 128 terminology to be used for benchmarking IGP Convergence can be 129 applied to IPv4 and IPv6 traffic and link-state IGPs such as ISIS 130 [Ca90][Ho08], OSPF [Mo98][Co08], and others. 132 2. Existing Definitions 134 This document uses existing terminology defined in other BMWG work. 135 Examples include, but are not limited to: 137 Frame Loss Rate [Ref.[Br91], section 3.6] 138 Throughput [Ref.[Br91], section 3.17] 139 Offered Load [Ref.[Ma98], section 3.5.2] 140 Forwarding Rate [Ref.[Ma98], section 3.6.1] 141 Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] 142 System Under Test (SUT) [Ref.[Ma98], section 3.1.2] 143 Out-of-Order Packet [Ref.[Po06], section 3.3.4] 144 Duplicate Packet [Ref.[Po06], section 3.3.5] 145 Packet Reordering [Ref.[Mo06], section 3.3] 146 Stream [Ref.[Po06], section 3.3.2] 147 Forwarding Delay [Ref.[Po06], section 3.2.4] 148 IP Packet Delay Variation (IPDV) [Ref.[De02], section 1.2] 149 Loss Period [Ref.[Ko02], section 4] 151 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 152 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 153 document are to be interpreted as described in BCP 14, RFC 2119 154 [Br97]. RFC 2119 defines the use of these key words to help make the 155 intent of standards track documents as clear as possible. While this 156 document uses these keywords, this document is not a standards track 157 document. 159 3. Term Definitions 160 3.1. Convergence Types 162 3.1.1. Route Convergence 164 Definition: 166 The process of updating all components of the router, including the 167 Routing Information Base (RIB) and Forwarding Information Base (FIB), 168 along with software and hardware tables, with the most recent route 169 change(s) such that forwarding for a route entry is successful on the 170 Next-Best Egress Interface. 172 Discussion: 174 Route Convergence MUST occur after a Convergence Event. Route 175 Convergence can be observed externally by the rerouting of data 176 traffic for a destination matching a route entry to the Next-best 177 Egress Interface. Completion of Route Convergence may or may not be 178 sustained over time. 180 Measurement Units: N/A 182 Issues: None 184 See Also: 186 Network Convergence, Full Convergence, Convergence Event 188 3.1.2. Full Convergence 190 Definition: 192 Route Convergence for all routes in the FIB. 194 Discussion: 196 Full Convergence MUST occur after a Convergence Event. Full 197 Convergence can be observed externally by the rerouting of data 198 traffic to destinations matching all route entries to the Next-best 199 Egress Interface. Completion of Full Convergence is externally 200 observable from the data plane when the Forwarding Rate of the data 201 plane traffic on the Next-Best Egress Interface equals the Offered 202 Load. 204 Completion of Full Convergence may or may not be sustained over time. 206 Measurement Units: N/A 207 Issues: None 209 See Also: 211 Network Convergence, Route Convergence, Convergence Event, Full 212 Convergence Time, Convergence Recovery Instant 214 3.1.3. Network Convergence 216 Definition: 218 Full Convergence in all routers throughout the network. 220 Discussion: 222 Network Convergence includes all Route Convergence operations for all 223 routers in the network following a Convergence Event. 225 Completion of Network Convergence can be observed by recovery of the 226 network Forwarding Rate to equal the Offered Load, with no Stale 227 Forwarding, and no Blenders [Ca01][Ci03]. 229 Completion of Network Convergence may or may not be sustained over 230 time. 232 Measurement Units: N/A 234 Issues: None 236 See Also: 238 Route Convergence, Full Convergence, Stale Forwarding 240 3.2. Instants 242 3.2.1. Traffic Start Instant 244 Definition: 246 The time instant the Tester sends out the first data packet to the 247 DUT. 249 Discussion: 251 If using the Loss-Derived Method or the Route-Specific Loss-Derived 252 Method to benchmark IGP convergence time, and the applied Convergence 253 Event does not cause instantaneous traffic loss for all routes at the 254 Convergence Event Instant then the Tester SHOULD collect a timestamp 255 on the Traffic Start Instant in order to measure the period of time 256 between the Traffic Start Instant and Convergence Event Instant. 258 Measurement Units: 260 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 261 microseconds. 263 Issues: None 265 See Also: 267 Convergence Event Instant, Route-Specific Convergence Time, Loss- 268 Derived Convergence Time. 270 3.2.2. Convergence Event Instant 272 Definition: 274 The time instant that a Convergence Event occurs. 276 Discussion: 278 If the Convergence Event causes instantaneous traffic loss on the 279 Preferred Egress Interface, the Convergence Event Instant is 280 observable from the data plane as the instant that the DUT begins to 281 exhibit packet loss. 283 The Tester SHOULD collect a timestamp on the Convergence Event 284 Instant if it is not observable from the data plane. 286 Measurement Units: 288 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 289 microseconds. 291 Issues: None 293 See Also: Convergence Event 295 3.2.3. Convergence Recovery Instant 297 Definition: 299 The time instant that Full Convergence has completed. 301 Discussion: 303 The Full Convergence completed state MUST be maintained for an 304 interval of duration equal to the Sustained Convergence Validation 305 Time in order to validate the Convergence Recovery Instant. 307 The Convergence Recovery Instant is observable from the data plane as 308 the instant the DUT forwards traffic to all destinations over the 309 Next-Best Egress Interface. 311 Measurement Units: 313 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 314 microseconds. 316 Issues: None 318 See Also: 320 Sustained Convergence Validation Time, Full Convergence 322 3.2.4. First Route Convergence Instant 324 Definition: 326 The time instant the first route entry completes Route Convergence 327 following a Convergence Event 329 Discussion: 331 Any route may be the first to complete Route Convergence. The First 332 Route Convergence Instant is observable from the data plane as the 333 instant that the first packet is received from the Next-Best Egress 334 Interface. 336 Measurement Units: 338 hh:mm:ss:nnn:uuu, where 'nnn' is milliseconds and 'uuu' is 339 microseconds. 341 Issues: None 343 See Also: Route Convergence 345 3.3. Transitions 347 3.3.1. Convergence Event Transition 349 Definition: 351 A time interval following a Convergence Event in which Forwarding 352 Rate on the Preferred Egress Interface gradually reduces to zero. 354 Discussion: 356 The Forwarding Rate during a Convergence Event Transition may not 357 decrease linearly. 359 The Forwarding Rate observed on all DUT egress interfaces may or may 360 not decrease to zero. 362 The Offered Load, the number of routes, and the Packet Sampling 363 Interval influence the observations of the Convergence Event 364 Transition using the Rate-Derived Method. This is further discussed 365 with the term "Rate-Derived Method". 367 Measurement Units: seconds 369 Issues: None 371 See Also: 373 Convergence Event, Rate-Derived Method 375 3.3.2. Convergence Recovery Transition 377 Definition: 379 A time interval following the First Route Convergence Instant in 380 which Forwarding Rate on the Next-Best Egress Interface gradually 381 increases to equal the Offered Load. 383 Discussion: 385 The Forwarding Rate observed during a Convergence Recovery Transition 386 may not increase linearly. 388 The Offered Load, the number of routes, and the Packet Sampling 389 Interval influence the observations of the Convergence Recovery 390 Transition using the Rate-Derived Method. This is further discussed 391 with the term "Rate-Derived Method". 393 Measurement Units: seconds 395 Issues: None 397 See Also: 399 Full Convergence,First Route Convergence Instant, Rate-Derived Method 401 3.4. Interfaces 403 3.4.1. Local Interface 405 Definition: 407 An interface on the DUT. 409 Discussion: 411 A failure of the Local Interface indicates that the failure occurred 412 directly on the DUT. 414 Measurement Units: N/A 416 Issues: None 418 See Also: Remote Interface 420 3.4.2. Remote Interface 422 Definition: 424 An interface on a neighboring router that is not directly connected 425 to any interface on the DUT. 427 Discussion: 429 A failure of a Remote Interface indicates that the failure occurred 430 on a neighbor router's interface that is not directly connected to 431 the DUT. 433 Measurement Units: N/A 435 Issues: None 437 See Also: Local Interface 439 3.4.3. Preferred Egress Interface 441 Definition: 443 The outbound interface from the DUT for traffic routed to the 444 preferred next-hop. 446 Discussion: 448 The Preferred Egress Interface is the egress interface prior to a 449 Convergence Event. 451 Measurement Units: N/A 453 Issues: None 455 See Also: Next-Best Egress Interface 457 3.4.4. Next-Best Egress Interface 459 Definition: 461 The outbound interface from the DUT for traffic routed to the second- 462 best next-hop. 464 Discussion: 466 The Next-Best Egress Interface becomes the egress interface after a 467 Convergence Event. 469 Measurement Units: N/A 471 Issues: None 473 See Also: Preferred Egress Interface 475 3.5. Benchmarking Methods 477 3.5.1. Rate-Derived Method 479 Definition: 481 The method to calculate convergence time benchmarks from observing 482 Forwarding Rate each Packet Sampling Interval. 484 Discussion: 486 Figure 1 shows an example of the Forwarding Rate change in time 487 during convergence as observed when using the Rate-Derived Method. 489 ^ Traffic Convergence 490 Fwd | Start Recovery 491 Rate | Instant Instant 492 | Offered ^ ^ 493 | Load --> ----------\ /----------- 494 | \ /<--- Convergence 495 | \ Packet / Recovery 496 | Convergence --->\ Loss / Transition 497 | Event \ / 498 | Transition \---------/ <-- Max Packet Loss 499 | 500 +---------------------------------------------------------> 501 ^ ^ time 502 Convergence First Route 503 Event Instant Convergence Instant 505 Figure 1: Rate-Derived Convergence Graph 507 The Offered Load SHOULD consist of a single Stream [Po06]. If 508 sending multiple Streams, the measured traffic rate statistics for 509 all Streams MUST be added together. 511 The destination addresses for the Offered Load MUST be distributed 512 such that all routes or a statistically representative subset of all 513 routes are matched and each of these routes is offered an equal share 514 of the Offered Load. It is RECOMMENDED to send traffic to all 515 routes, but a statistically representative subset of all routes can 516 be used if required. 518 At least one packet per route for all routes matched in the Offered 519 Load MUST be offered to the DUT within each Packet Sampling Interval. 520 For maximum accuracy the value for the Packet Sampling Interval 521 SHOULD be as small as possible, but the presence of IP Packet Delay 522 Variation (IPDV) [De02] may enforce using a larger Packet Sampling 523 Interval. 525 The Offered Load, IPDV, the number of routes, and the Packet Sampling 526 Interval influence the observations for the Rate-Derived Method. It 527 may be difficult to identify the different convergence time instants 528 in the Rate-Derived Convergence Graph. For example, it is possible 529 that a Convergence Event causes the Forwarding Rate to drop to zero, 530 while this may not be observed in the Forwarding Rate measurements if 531 the Packet Sampling Interval is too large. 533 IPDV causes fluctuations in the number of received packets during 534 each Packet Sampling Interval. To account for the presence of IPDV 535 in determining if a convergence instant has been reached, Forwarding 536 Delay SHOULD be observed during each Packet Sampling Interval. The 537 minimum and maximum number of packets expected in a Packet Sampling 538 Interval in presence of IPDV can be calculated with Equation 1. 540 number of packets expected in a Packet Sampling Interval 541 in presence of IP Packet Delay Variation 542 = expected number of packets without IP Packet Delay Variation 543 +/-( (maxDelay - minDelay) * Offered Load) 544 with minDelay and maxDelay the minimum resp. maximum Forwarding Delay 545 of packets received during the Packet Sampling Interval 547 Equation 1 549 To determine if a convergence instant has been reached the number of 550 packets received in a Packet Sampling Interval is compared with the 551 range of expected number of packets calculated in Equation 1. 553 If packets are going over multiple ECMP members and one or more of 554 the members has failed then the number of received packets during 555 each Packet Sampling Interval may vary, even excluding presence of 556 IPDV. To prevent fluctuation of the number of received packets 557 during each Packet Sampling Interval for this reason, the Packet 558 Sampling Interval duration SHOULD be a whole multiple of the time 559 between two consecutive packets sent to the same destination. 561 Metrics measured at the Packet Sampling Interval MUST include 562 Forwarding Rate and packet loss. 564 Rate-Derived Method is a RECOMMENDED method to measure convergence 565 time benchmarks. 567 To measure convergence time benchmarks for Convergence Events that do 568 not cause instantaneous traffic loss for all routes at the 569 Convergence Event Instant, the Tester SHOULD collect a timestamp of 570 the Convergence Event Instant and the Tester SHOULD observe 571 Forwarding Rate separately on the Next-Best Egress Interface. 573 Since the Rate-Derived Method does not distinguish between individual 574 traffic destinations, it SHOULD NOT be used for any route specific 575 measurements. Therefor Rate-Derived Method SHOULD NOT be used to 576 benchmark Route Loss of Connectivity Period. 578 Measurement Units: N/A 580 Issues: None 582 See Also: 584 Packet Sampling Interval, Convergence Event, Convergence Event 585 Instant, Full Convergence 587 3.5.2. Loss-Derived Method 589 Definition: 591 The method to calculate the Loss-Derived Convergence Time and Loss- 592 Derived Loss of Connectivity Period benchmarks from the amount of 593 packet loss. 595 Discussion: 597 The Offered Load SHOULD consist of a single Stream [Po06]. If 598 sending multiple Streams, the measured traffic rate statistics for 599 all Streams MUST be added together. 601 The destination addresses for the Offered Load MUST be distributed 602 such that all routes or a statistically representative subset of all 603 routes are matched and each of these routes is offered an equal share 604 of the Offered Load. It is RECOMMENDED to send traffic to all 605 routes, but a statistically representative subset of all routes can 606 be used if required. 608 Loss-Derived Method SHOULD always be combined with Rate-Derived 609 Method in order to observe Full Convergence completion. The total 610 amount of Convergence Packet Loss is collected after Full Convergence 611 completion. 613 To measure convergence time and loss of connectivity benchmarks, the 614 Tester SHOULD in general observe packet loss on all DUT egress 615 interfaces (Connectivity Packet Loss). 617 To measure convergence time benchmarks for Convergence Events that do 618 not cause instantaneous traffic loss for all routes at the 619 Convergence Event Instant, the Tester SHOULD collect timestamps of 620 the Start Traffic Instant and of the Convergence Event Instant, and 621 the Tester SHOULD observe packet loss separately on the Next-Best 622 Egress Interface (Convergence Packet Loss). 624 Since Loss-Derived Method does not distinguish between traffic 625 destinations and the packet loss statistics are only collected after 626 Full Convergence completion, this method can only be used to measure 627 average values over all routes. For these reasons Loss-Derived 628 Method can only be used to benchmark Loss-Derived Convergence Time 629 and Loss-Derived Loss of Connectivity Period. 631 Note that the Loss-Derived Method measures an average over all 632 routes, including the routes that may not be impacted by the 633 Convergence Event, such as routes via non-impacted members of ECMP or 634 parallel links. 636 Measurement Units: seconds 638 Issues: None 640 See Also: 642 Loss-Derived Convergence Time, Loss-Derived Loss of Connectivity 643 Period, Convergence Packet Loss 645 3.5.3. Route-Specific Loss-Derived Method 647 Definition: 649 The method to calculate the Route-Specific Convergence Time benchmark 650 from the amount of packet loss during convergence for a specific 651 route entry. 653 Discussion: 655 To benchmark Route-Specific Convergence Time, the Tester provides an 656 Offered Load that consists of multiple Streams [Po06]. Each Stream 657 has a single destination address matching a different route entry, 658 for all routes or a statistically representative subset of all 659 routes. Convergence Packet Loss is measured for each Stream 660 separately. 662 Route-Specific Loss-Derived Method SHOULD always be combined with 663 Rate-Derived Method in order to observe Full Convergence completion. 664 The total amount of Convergence Packet Loss for each Stream is 665 collected after Full Convergence completion. 667 Route-Specific Loss-Derived Method is a RECOMMENDED method to measure 668 convergence time benchmarks. 670 To measure convergence time and loss of connectivity benchmarks, the 671 Tester SHOULD in general observe packet loss on all DUT egress 672 interfaces (Connectivity Packet Loss). 674 To measure convergence time benchmarks for Convergence Events that do 675 not cause instantaneous traffic loss for all routes at the 676 Convergence Event Instant, the Tester SHOULD collect timestamps of 677 the Start Traffic Instant and of the Convergence Event Instant, and 678 the Tester SHOULD observe packet loss separately on the Next-Best 679 Egress Interface (Convergence Packet Loss). 681 Since Route-Specific Loss-Derived Method uses traffic streams to 682 individual routes, it measures packet loss as it would be experienced 683 by a network user. For this reason Route-Specific Loss-Derived 684 Method is RECOMMENDED to measure Route-Specific Convergence Time 685 benchmarks and Route Loss of Connectivity Period benchmarks. 687 Measurement Units: seconds 689 Issues: None 691 See Also: 693 Route-Specific Convergence Time, Route Loss of Connectivity Period, 694 Convergence Packet Loss 696 3.6. Benchmarks 698 3.6.1. Full Convergence Time 700 Definition: 702 The time duration of the period between the Convergence Event Instant 703 and the Convergence Recovery Instant as observed using the Rate- 704 Derived Method. 706 Discussion: 708 Using the Rate-Derived Method, Full Convergence Time can be 709 calculated as the time difference between the Convergence Event 710 Instant and the Convergence Recovery Instant, as shown in Equation 2. 712 Full Convergence Time = 713 Convergence Recovery Instant - Convergence Event Instant 715 Equation 2 717 The Convergence Event Instant can be derived from the Forwarding Rate 718 observation or from a timestamp collected by the Tester. 720 For the testcases described in [Po10m], it is expected that Full 721 Convergence Time equals the maximum Route-Specific Convergence Time 722 when benchmarking all routes in FIB using the Route-Specific Loss- 723 Derived Method. 725 It is not possible to measure Full Convergence Time using the Loss- 726 Derived Method. 728 Measurement Units: seconds 729 Issues: None 731 See Also: 733 Full Convergence, Rate-Derived Method, Route-Specific Loss-Derived 734 Method 736 3.6.2. First Route Convergence Time 738 Definition: 740 The duration of the period between the Convergence Event Instant and 741 the First Route Convergence Instant as observed using the Rate- 742 Derived Method. 744 Discussion: 746 Using the Rate-Derived Method, First Route Convergence Time can be 747 calculated as the time difference between the Convergence Event 748 Instant and the First Route Convergence Instant, as shown with 749 Equation 3. 751 First Route Convergence Time = 752 First Route Convergence Instant - Convergence Event Instant 754 Equation 3 756 The Convergence Event Instant can be derived from the Forwarding Rate 757 observation or from a timestamp collected by the Tester. 759 For the testcases described in [Po10m], it is expected that First 760 Route Convergence Time equals the minimum Route-Specific Convergence 761 Time when benchmarking all routes in FIB using the Route-Specific 762 Loss-Derived Method. 764 It is not possible to measure First Route Convergence Time using the 765 Loss-Derived Method. 767 Measurement Units: seconds 769 Issues: None 771 See Also: 773 Rate-Derived Method, Route-Specific Loss-Derived Method, First Route 774 Convergence Instant 776 3.6.3. Route-Specific Convergence Time 778 Definition: 780 The amount of time it takes for Route Convergence to be completed for 781 a specific route, as calculated from the amount of packet loss during 782 convergence for a single route entry. 784 Discussion: 786 Route-Specific Convergence Time can only be measured using the Route- 787 Specific Loss-Derived Method. 789 If the applied Convergence Event causes instantaneous traffic loss 790 for all routes at the Convergence Event Instant, Connectivity Packet 791 Loss should be observed. Connectivity Packet Loss is the combined 792 packet loss observed on Preferred Egress Interface and Next-Best 793 Egress Interface. When benchmarking Route-Specific Convergence Time, 794 Connectivity Packet Loss is measured and Equation 4 is applied for 795 each measured route. The calculation is equal to Equation 8 in 796 Section 3.6.5. 798 Route-Specific Convergence Time = 799 Connectivity Packet Loss for specific route/Offered Load per route 801 Equation 4 803 If the applied Convergence Event does not cause instantaneous traffic 804 loss for all routes at the Convergence Event Instant, then the Tester 805 SHOULD collect timestamps of the Traffic Start Instant and of the 806 Convergence Event Instant, and the Tester SHOULD observe Convergence 807 Packet Loss separately on the Next-Best Egress Interface. When 808 benchmarking Route-Specific Convergence Time, Convergence Packet Loss 809 is measured and Equation 5 is applied for each measured route. 811 Route-Specific Convergence Time = 812 Convergence Packet Loss for specific route/Offered Load per route 813 - (Convergence Event Instant - Traffic Start Instant) 815 Equation 5 817 The Convergence Event Instant and Traffic Start Instant SHOULD be 818 collected by the Tester. 820 The Route-Specific Convergence Time benchmarks enable minimum, 821 maximum, average, and median convergence time measurements to be 822 reported by comparing the results for the different route entries. 823 It also enables benchmarking of convergence time when configuring a 824 priority value for route entry(ies). Since multiple Route-Specific 825 Convergence Times can be measured it is possible to have an array of 826 results. The format for reporting Route-Specific Convergence Time is 827 provided in [Po10m]. 829 Measurement Units: seconds 831 Issues: None 833 See Also: 835 Convergence Event, Convergence Packet Loss, Connectivity Packet Loss, 836 Route Convergence 838 3.6.4. Loss-Derived Convergence Time 840 Definition: 842 The average Route Convergence time for all routes in FIB, as 843 calculated from the amount of packet loss during convergence. 845 Discussion: 847 Loss-Derived Convergence Time is measured using the Loss-Derived 848 Method. 850 If the applied Convergence Event causes instantaneous traffic loss 851 for all routes at the Convergence Event Instant, Connectivity Packet 852 Loss should be observed. Connectivity Packet Loss is the combined 853 packet loss observed on Preferred Egress Interface and Next-Best 854 Egress Interface. When benchmarking Loss-Derived Convergence Time, 855 Connectivity Packet Loss is measured and Equation 6 is applied. 857 Loss-Derived Convergence Time = 858 Connectivity Packet Loss/Offered Load 860 Equation 6 862 If the applied Convergence Event does not cause instantaneous traffic 863 loss for all routes at the Convergence Event Instant, then the Tester 864 SHOULD collect timestamps of the Start Traffic Instant and of the 865 Convergence Event Instant and the Tester SHOULD observe Convergence 866 Packet Loss separately on the Next-Best Egress Interface. When 867 benchmarking Loss-Derived Convergence Time, Convergence Packet Loss 868 is measured and Equation 7 is applied. 870 Loss-Derived Convergence Time = 871 Convergence Packet Loss/Offered Load 872 - (Convergence Event Instant - Traffic Start Instant) 874 Equation 7 876 The Convergence Event Instant and Traffic Start Instant SHOULD be 877 collected by the Tester. 879 Measurement Units: seconds 881 Issues: None 883 See Also: 885 Convergence Packet Loss, Connectivity Packet Loss, Route Convergence 887 3.6.5. Route Loss of Connectivity Period 889 Definition: 891 The time duration of traffic loss for a specific route entry 892 following a Convergence Event until Full Convergence completion, as 893 observed using the Route-Specific Loss-Derived Method. 895 Discussion: 897 In general the Route Loss of Connectivity Period is not equal to the 898 Route-Specific Convergence Time. If the DUT continues to forward 899 traffic to the Preferred Egress Interface after the Convergence Event 900 is applied then the Route Loss of Connectivity Period will be smaller 901 than the Route-Specific Convergence Time. This is also specifically 902 the case after reversing a failure event. 904 The Route Loss of Connectivity Period may be equal to the Route- 905 Specific Convergence Time if, as a characteristic of the Convergence 906 Event, traffic for all routes starts dropping instantaneously on the 907 Convergence Event Instant. See discussion in [Po10m]. 909 For the testcases described in [Po10m] the Route Loss of Connectivity 910 Period is expected to be a single Loss Period [Ko02]. 912 When benchmarking Route Loss of Connectivity Period, Connectivity 913 Packet Loss is measured for each route and Equation 8 is applied for 914 each measured route entry. The calculation is equal to Equation 4 in 915 Section 3.6.3. 917 Route Loss of Connectivity Period = 918 Connectivity Packet Loss for specific route/Offered Load per route 920 Equation 8 922 Route Loss of Connectivity Period SHOULD be measured using Route- 923 Specific Loss-Derived Method. 925 Measurement Units: seconds 927 Issues: None 929 See Also: 931 Route-Specific Convergence Time, Route-Specific Loss-Derived Method, 932 Connectivity Packet Loss 934 3.6.6. Loss-Derived Loss of Connectivity Period 936 Definition: 938 The average time duration of traffic loss for all routes following a 939 Convergence Event until Full Convergence completion, as observed 940 using the Loss-Derived Method. 942 Discussion: 944 In general the Loss-Derived Loss of Connectivity Period is not equal 945 to the Loss-Derived Convergence Time. If the DUT continues to 946 forward traffic to the Preferred Egress Interface after the 947 Convergence Event is applied then the Loss-Derived Loss of 948 Connectivity Period will be smaller than the Loss-Derived Convergence 949 Time. This is also specifically the case after reversing a failure 950 event. 952 The Loss-Derived Loss of Connectivity Period may be equal to the 953 Loss-Derived Convergence Time if, as a characteristic of the 954 Convergence Event, traffic for all routes starts dropping 955 instantaneously on the Convergence Event Instant. See discussion in 956 [Po10m]. 958 For the testcases described in [Po10m] each route's Route Loss of 959 Connectivity Period is expected to be a single Loss Period [Ko02]. 961 When benchmarking Loss-Derived Loss of Connectivity Period, 962 Connectivity Packet Loss is measured for all routes and Equation 9 is 963 applied. The calculation is equal to Equation 6 in Section 3.6.4. 965 Loss-Derived Loss of Connectivity Period = 966 Connectivity Packet Loss for all routes/Offered Load 968 Equation 9 970 Loss-Derived Loss of Connectivity Period SHOULD be measured using 971 Loss-Derived Method. 973 Measurement Units: seconds 975 Issues: None 977 See Also: 979 Loss-Derived Convergence Time, Loss-Derived Method, Connectivity 980 Packet Loss 982 3.7. Measurement Terms 984 3.7.1. Convergence Event 986 Definition: 988 The occurrence of a planned or unplanned event in the network that 989 will result in a change in the egress interface of the Device Under 990 Test (DUT) for routed packets. 992 Discussion: 994 Convergence Events include but are not limited to link loss, routing 995 protocol session loss, router failure, configuration change, and 996 better next-hop learned via a routing protocol. 998 Measurement Units: N/A 1000 Issues: None 1002 See Also: Convergence Event Instant 1004 3.7.2. Packet Loss 1006 Definition: 1008 The number of packets that should have been forwarded by a DUT under 1009 a constant Offered Load that were not forwarded due to lack of 1010 resources. 1012 Discussion: 1014 Packet Loss is a modified version of the term "Frame Loss Rate" as 1015 defined in [Br91]. The term "Frame Loss" is intended for Ethernet 1016 Frames while "Packet Loss" is intended for IP packets. 1018 Measurement units: Number of offered packets that are not forwarded. 1020 Issues: None 1022 See Also: Convergence Packet Loss 1024 3.7.3. Convergence Packet Loss 1026 Definition: 1028 The number of packets lost due to a Convergence Event until Full 1029 Convergence completes, as observed on the Next-Best Egress Interface. 1031 Discussion: 1033 Convergence Packet Loss is observed on the Next-Best Egress 1034 Interface. It only needs to be observed for Convergence Events that 1035 do not cause instantaneous traffic loss at Convergence Event Instant. 1037 Convergence Packet Loss includes packets that were lost and packets 1038 that were delayed due to buffering. The maximum acceptable 1039 Forwarding Delay (Forwarding Delay Threshold) is a parameter of the 1040 methodology, if it is applied it MUST be reported. 1042 Measurement Units: number of packets 1044 Issues: None 1046 See Also: 1048 Packet Loss, Full Convergence, Convergence Event, Connectivity Packet 1049 Loss 1051 3.7.4. Connectivity Packet Loss 1053 Definition: 1055 The number of packets lost due to a Convergence Event until Full 1056 Convergence completes. 1058 Discussion: 1060 Connectivity Packet Loss is observed on all DUT egress interfaces. 1062 Connectivity Packet Loss includes packets that were lost and packets 1063 that were delayed due to buffering. The maximum acceptable 1064 Forwarding Delay (Forwarding Delay Threshold) is a parameter of the 1065 methodology, if it is applied it MUST be reported. 1067 Measurement Units: number of packets 1069 Issues: None 1071 See Also: 1073 Packet Loss, Route Loss of Connectivity Period, Convergence Event, 1074 Convergence Packet Loss 1076 3.7.5. Packet Sampling Interval 1078 Definition: 1080 The interval at which the Tester (test equipment) polls to make 1081 measurements for arriving packets. 1083 Discussion: 1085 At least one packet per route for all routes matched in the Offered 1086 Load MUST be offered to the DUT within the Packet Sampling Interval. 1087 Metrics measured at the Packet Sampling Interval MUST include 1088 Forwarding Rate and received packets. 1090 Packet Sampling Interval can influence the convergence graph as 1091 observed with the Rate-Derived Method. This is particularly true 1092 when implementations complete Full Convergence in less time than the 1093 Packet Sampling Interval. The Convergence Event Instant and First 1094 Route Convergence Instant may not be easily identifiable and the 1095 Rate-Derived Method may produce a larger than actual convergence 1096 time. 1098 Using a small Packet Sampling Interval in the presence of IPDV [De02] 1099 may cause fluctuations of the Forwarding Rate observation and can 1100 prevent correct observation of the different convergence time 1101 instants. 1103 The value of the Packet Sampling Interval only contributes to the 1104 measurement accuracy of the Rate-Derived Method. For maximum 1105 accuracy the value for the Packet Sampling Interval SHOULD be as 1106 small as possible, but the presence of IPDV may enforce using a 1107 larger Packet Sampling Interval. 1109 Measurement Units: seconds 1110 Issues: None 1112 See Also: Rate-Derived Method 1114 3.7.6. Sustained Convergence Validation Time 1116 Definition: 1118 The amount of time for which the completion of Full Convergence is 1119 maintained without additional packet loss. 1121 Discussion: 1123 The purpose of the Sustained Convergence Validation Time is to 1124 produce convergence benchmarks protected against fluctuation in 1125 Forwarding Rate after the completion of Full Convergence is observed. 1126 The RECOMMENDED Sustained Convergence Validation Time to be used is 1127 the time to send 5 consecutive packets to each destination with a 1128 minimum of 5 seconds. The BMWG selected 5 seconds based upon [Br99] 1129 which recommends waiting 2 seconds for residual frames to arrive 1130 (this is the Forwarding Delay Threshold for the last packet sent) and 1131 5 seconds for DUT restabilization. 1133 Measurement Units: seconds 1135 Issues: None 1137 See Also: 1139 Full Convergence, Convergence Recovery Instant 1141 3.7.7. Forwarding Delay Threshold 1143 Definition: 1145 The maximum waiting time threshold used to distinguish between 1146 packets with very long delay and lost packets that will never arrive. 1148 Discussion: 1150 Applying a Forwarding Delay Threshold allows to consider packets with 1151 a too large Forwarding Delay as being lost, as is required for some 1152 applications (e.g. voice, video, etc.). The Forwarding Delay 1153 Threshold is a parameter of the methodology, if it is applied it MUST 1154 be reported. 1156 Measurement Units: seconds 1157 Issues: None 1159 See Also: 1161 Convergence Packet Loss, Connectivity Packet Loss 1163 3.8. Miscellaneous Terms 1165 3.8.1. Stale Forwarding 1167 Definition: 1169 Forwarding of traffic to route entries that no longer exist or to 1170 route entries with next-hops that are no longer preferred. 1172 Discussion: 1174 Stale Forwarding can be caused by a Convergence Event and can 1175 manifest as a "black-hole" or microloop that produces packet loss, or 1176 out-of-order packets, or delayed packets. Stale Forwarding can exist 1177 until Network Convergence is completed. 1179 Measurement Units: N/A 1181 Issues: None 1183 See Also: Network Convergence 1185 3.8.2. Nested Convergence Event 1187 Definition: 1189 The occurrence of a Convergence Event while the route table is 1190 converging from a prior Convergence Event. 1192 Discussion: 1194 The Convergence Events for a Nested Convergence Event MUST occur with 1195 different neighbors. A possible observation from a Nested 1196 Convergence Event will be the withdrawal of routes from one neighbor 1197 while the routes of another neighbor are being installed. 1199 Measurement Units: N/A 1201 Issues: None 1203 See Also: Convergence Event 1205 4. Security Considerations 1207 Benchmarking activities as described in this memo are limited to 1208 technology characterization using controlled stimuli in a laboratory 1209 environment, with dedicated address space and the constraints 1210 specified in the sections above. 1212 The benchmarking network topology will be an independent test setup 1213 and MUST NOT be connected to devices that may forward the test 1214 traffic into a production network, or misroute traffic to the test 1215 management network. 1217 Further, benchmarking is performed on a "black-box" basis, relying 1218 solely on measurements observable external to the DUT/SUT. 1220 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 1221 benchmarking purposes. Any implications for network security arising 1222 from the DUT/SUT SHOULD be identical in the lab and in production 1223 networks. 1225 5. IANA Considerations 1227 This document requires no IANA considerations. 1229 6. Acknowledgements 1231 Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, 1232 Peter De Vriendt, Anuj Dewagan and the BMWG for their contributions 1233 to this work. 1235 7. References 1237 7.1. Normative References 1239 [Br91] Bradner, S., "Benchmarking terminology for network 1240 interconnection devices", RFC 1242, July 1991. 1242 [Br97] Bradner, S., "Key words for use in RFCs to Indicate 1243 Requirement Levels", BCP 14, RFC 2119, March 1997. 1245 [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1246 Network Interconnect Devices", RFC 2544, March 1999. 1248 [Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual 1249 environments", RFC 1195, December 1990. 1251 [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for 1252 IPv6", RFC 5340, July 2008. 1254 [De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1255 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1256 November 2002. 1258 [Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, 1259 October 2008. 1261 [Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample 1262 Metrics", RFC 3357, August 2002. 1264 [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching 1265 Devices", RFC 2285, February 1998. 1267 [Mo06] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., 1268 and J. Perser, "Packet Reordering Metrics", RFC 4737, 1269 November 2006. 1271 [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 1273 [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1274 "Terminology for Benchmarking Network-layer Traffic Control 1275 Mechanisms", RFC 4689, October 2006. 1277 [Po09a] Poretsky, S., "Considerations for Benchmarking Link-State 1278 IGP Data Plane Route Convergence", 1279 draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in 1280 progress), March 2009. 1282 [Po10m] Poretsky, S., Imhoff, B., and K. Michielsen, "Benchmarking 1283 Methodology for Link-State IGP Data Plane Route 1284 Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-20 1285 (work in progress), March 2010. 1287 7.2. Informative References 1289 [Ca01] Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained 1290 View of High Performance Networking", NANOG 22, June 2001. 1292 [Ci03] Ciavattone, L., Morton, A., and G. Ramachandran, 1293 "Standardized Active Measurements on a Tier 1 IP Backbone", 1294 IEEE Communications Magazine p90-97, May 2003. 1296 Authors' Addresses 1298 Scott Poretsky 1299 Allot Communications 1300 67 South Bedford Street, Suite 400 1301 Burlington, MA 01803 1302 USA 1304 Phone: + 1 508 309 2179 1305 Email: sporetsky@allot.com 1307 Brent Imhoff 1308 Juniper Networks 1309 1194 North Mathilda Ave 1310 Sunnyvale, CA 94089 1311 USA 1313 Phone: + 1 314 378 2571 1314 Email: bimhoff@planetspork.com 1316 Kris Michielsen 1317 Cisco Systems 1318 6A De Kleetlaan 1319 Diegem, BRABANT 1831 1320 Belgium 1322 Email: kmichiel@cisco.com