Network Working Group INTERNET-DRAFT Expires in: December 2003 Scott Poretsky Avici Systems Brent Imhoff Wiltel Communications June 2003 Benchmarking Methodology for IGP Data Plane Route Convergence Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Table of Contents 1. Introduction ...............................................2 2. Existing definitions .......................................2 3. Test Setup..................................................2 3.1 Test Topologies............................................2 3.2 Test Considerations........................................3 4. Test Cases..................................................4 4.1 Local Events...............................................4 4.1.1 Convergence Due to SONET Link Failure....................4 4.1.2 Convergence Due to PPP Session Failure...................5 4.1.3 Convergence Due to IGP Adjacency Failure.................5 4.1.4 Convergence Due to Route Withdrawal......................6 4.1.5 Convergence Due to Cost Change...........................7 4.2 Remote Events..............................................7 4.2.1 Convergence Due to Remote SONET Link Failure.............7 Poretsky, Imhoff [Page 1] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence 5. Measuring Convergence Times.................................8 5.1 Measuring Peak-to-Peak Convergence Time....................8 5.2 Measuring Impact of Components for Convergence.............8 6. Security Considerations.....................................9 7. Acknowledgements............................................9 8. References..................................................9 9. Author's Address............................................9 10. Full Copyright Statement...................................10 1. Introduction This draft describes the methodology for benchmarking IGP Route Convergence. The applicability of this testing is described in [1] and the new terminology that it introduces is defined in [2]. Service Providers use IGP Convergence time as a key metric of router design and architecture. Customers of Service Providers observe convergence time by packet loss. IGP Route Convergence is a Direct Measure of Quality (DMOQ) when benchmarking the data plane and not the control plane. The test cases in this document are black-box tests that emulate the network events that cause route convergence, as described in [1]. Black-box test design accounts for all of the factors for route convergence time, as provided in [1]. The methodology and terminology is to be used for benchmarking route convergence and can be applied to any link-state IGP such as ISIS [3] and OSPF [4]. 2. Existing definitions For the sake of clarity and continuity this RFC adopts the template for definitions set out in Section 2 of RFC 1242. Definitions are indexed and grouped together in sections for ease of reference. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3. Test Setup 3.1 Test Topologies --------- Ingress Traffic Path --------- | |<------------------------------| | | | | | | | Preferred Egress Path | | | DUT |------------------------------>|Tester | | | | | | |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| | | | Backup Egress Path | | --------- --------- Figure 1. IGP Route Convergence Test Topology for Local Changes Poretsky, Imhoff [Page 2] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence Figure 1 shows the test topology to measure IGP Route Convergence due to local changes such as SONET Link Failure, PPP Session Failure, IGP Adjacency Failure, Route Withdrawal, and Route cost change. These test cases are described in section 4.1. These test cases provide IGP Route Convergence times that consider the Event Detection time, SPF Processing time, and FIB Update time. These times are measured by observing packet loss in the data plane. Physical Links may be of any type, such as Sonet or Ethernet, and any speed. Figure 2 shows the test topology to measure IGP Route Convergence time due to remote changes in the network topology. These times are measured by observing packet loss in the data plane. Physical Links may be of any type, such as Sonet or Ethernet, and any speed. In this topology, the three routers are considered a System Under Test (SUT). Application of this topology and test cases described in section 4.2 account for the impact of IGP Advertisement on Route Convergence, as described in [1]. ----- ----------- | | Preferred | | ----- |R2 |------------->| | | |---->| | Egress Path | | | | ----- | | |R1 | | Tester | | | ----- | | | |---->| | Backup | | ----- |R3 |~~~~~~~~~~~~~>| | ^ | | Egress Path | | | ----- ----------- | | |-------------------------------- Ingress Traffic Path Figure 2. IGP Route Convergence Test Topology for Remote Changes 3.2 Test Considerations 3.2.1 IGP Selection The test cases described in section 4 can be used for ISIS or OSPF. The Route Convergence test methodology for both is identical. The IGP adjacencies are established on the Preferred Egress Path and Backup Egress Path. 3.2.2 BGP Configuration The obtained results for IGP Route Convergence may vary if BGP routes are installed. For results similar to those that would be observed in an operational network it is recommended that a BGP session be established on the Ingress Traffic Path with routes installed. Poretsky, Imhoff [Page 3] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence 3.2.3 IGP Route Scaling The number of IGP routes will impact the measured IGP Route Convergence because convergence for the entire IGP route table is measured. For results similar to those that would be observed in an operational network it is recommended that the number of installed routes closely approximate that for routers in the network. 3.2.4 BGP Route Scaling The number of installed BGP routes may impact the IGP Convergence time. For results similar to those that would be observed in an operational Network it is recommended that the number of installed routes closely approximate that for routers in the network. 3.2.5 Timers There are some timers that will impact the measured IGP Convergence time. The following timers should be configured to the minimum value prior to beginning execution of the test cases: SONET Failure Indication Delay IGP Hello Timer IGP Dead-Interval LSA Generation Delay LSA Flood Packet Pacing LSA Retransmission Packet Pacing SPF Delay 4. Test Cases 4.1 Local Events The test cases in this section use the test topology shown in Figure 1. 4.1.1 Convergence Due to Local SONET Link Failure Objective To obtain the IGP Route Convergence due to a Local SONET Link failure event. Procedure 1. Advertise IGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Remove SONET on Tester Interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time [2] as DUT detects the link down event and converges all IGP routes and traffic over the Backup Egress Path. Poretsky, Imhoff [Page 4] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence Results The measured IGP Convergence time is influenced by the Local SONET indication, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.2 Convergence Due to PPP Session Failure Objective To obtain the IGP Route Convergence due to a Local PPP Session failure event. Procedure 1. Advertise IGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Remove PPP session from Tester Interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time as DUT detects the PPP session down event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the Local PPP failure indication, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.3 Convergence Due to IGP Adjacency Failure Objective To obtain the IGP Route Convergence due to a Local IGP Adjacency failure event. Procedure 1. Advertise IGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Remove IGP adjacency from Tester interface connected to Preferred Egress Path. Poretsky, Imhoff [Page 5] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence 6. Measure Peak-to-Peak Convergence Time as DUT detects the IGP session failure event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the IGP Hello Interval, IGP Dead Interval, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.4 Convergence Due to Route Withdrawal Objective To obtain the IGP Route Convergence due to Route Withdrawal. Procedure 1. Advertise IGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Tester withdraws all IGP routes from DUT's Local Interface on Preferred Egress Path. 6. Measure Peak-to-Peak Convergence Time as DUT processes the route withdrawal event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is the SPF Processing and FIB Update time as influenced by the SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. 4.1.5 Convergence Due to Cost Change Objective To obtain the IGP Route Convergence due to route cost change. Procedure 1. Advertise IGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. Poretsky, Imhoff [Page 6] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence 4. Verify traffic routed over Preferred Egress Path. 5. Tester increases cost for all IGP routes at DUT's Local Interface on Preferred Egress Path so that Backup Egress Path have lower cost and becomes preferred path. 6. Measure Reroute Convergence Time [2] as DUT detects the cost change event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is the SPF Processing and FIB Update time as influenced by the SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. There should be no packet loss for this case. 4.2 Remote Events The test cases in this section use the test topology shown in Figure 2. 4.2.1 Convergence Due to Remote SONET Link Failure Objective To obtain the IGP Route Convergence due to a Remote SONET Link failure event. Procedure 1. Advertise IGP routes from Tester to DUT on Ingress Traffic Path. 2. Advertise matching IGP routes from Tester to DUT on Preferred Egress Path and Backup Egress Path. Set the cost of the routes so that the IGP routes along the Preferred Egress Path is the preferred next-hop. 3. Send traffic at maximum forwarding rate to destinations matching all IGP routes from Tester to DUT on Ingress Traffic Path. 4. Verify traffic routed over Preferred Egress Path. 5. Remove SONET on Neighbor Interface connected to Preferred Egress Path. 6. Measure Peak-to-Peak Convergence time as DUT detects the link down event and converges all IGP routes and traffic over the Backup Egress Path. Results The measured IGP Convergence time is influenced by the SONET failure indication, LSA/LSP Flood Packet Pacing, LSA/LSP Retransmission Packet Pacing, LSA/LSP Generation time, SPF delay, SPF Holdtime, SPF Execution Time, Tree Build Time, and Hardware Update Time. Poretsky, Imhoff [Page 7] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence 5. Measuring Convergence Times 5.1 Measuring Full Convergence Time Figure 3 shows a graph model of Convergence Time as measured from the data plane. Refer to [2] for definitions of the terms used. IGP Route Convergence Time is the amount of time for the Forwarding Rate to begin its downward slope upon occurrence of a network event and then fully recover to the Maximum Forwarding Rate. Forwarding Rate versus Time Time=Recovery Time=Network Event Time = 0sec Maximum ^ ^ ^ Forwarding Rate--> ----\ /----------- \ /<----Route Convergence Route Convergence------->\ / Event Slope Recovery Slope \_______/<------100% Packet Loss X-axis = Time Y-axis = Forwarding Rate Figure 3. Convergence Graph Maximum forwarding rate at a fixed packet size without packet loss is required for accurate measurement. The test duration must be greater than the convergence time. Full Convergence Time is obtained directly from the graph in Figure 3 using equation 1. (eq 1) Convergence Time(Full)=Time(Recovery)-Time(Network Event). Given a known constant rate of offered load in units packet per second (pps), the Average Convergence Time can be obtained using equation 2 or equation 3. (eq 2) Convergence Time(Average)=Number Packets Lost/pps(Offered) (eq 3) Convergence Time(Average)= (Number Packets Offered - Number of Packets Received)/pps(Offered) As discussed in [1], Full Convergence Time is the more accurate measurement. Average Convergence Time does not account for the angle of the Route Convergence Recovery Slope, so Full Convergence Time > Average Convergence Time. Ideally, the Recovery Slope has no angle so that it is vertical and Average Convergence Time = Full Convergence Time. 5.2 Measuring Impact of Components for Convergence The factors for IGP Route Convergence Time are provided in [1]. The results of the test cases in section 4 above can be used to calculate the impact each factor has on the Convergence Poretsky, Imhoff [Page 8] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence results, as follow: SPF Processing and FIB Update time = Result (4.1.4) SONET failure indication time = Result(4.1.1) - Result(4.1.4) PPP failure indication time = Result(4.1.2) - Result(4.1.4) IGP failure indication time = Result(4.1.3) - Result(4.1.4) IGP Advertisement time = Result(4.1.1) - Result(4.2.1) 6. Security Considerations Documents of this type do not directly effect the security of the Internet or of corporate networks as long as benchmarking is not performed on devices or systems connected to operating networks. 7. Acknowledgements Thanks to Jayant Kulkarni for doing as most Test Engineers do - working beyond the call of duty to help advance technology. Especially thanks to the many Network Engineers and Network Architects at the Service Providers who are always eager to discuss Route Convergence. 8. References [1] Poretsky, S., "Benchmarking Applicability for IGP Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-00, work in progress, June 2003. [2] Poretsky, S., "Benchmarking Terminology for IGP Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-00, work in progress, June 2003. [3] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual Environments", RFC 1195, December 1990. [4] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998. 9. Author's Address Scott Poretsky Avici Systems, Inc. 101 Billerica Avenue N. Billerica, MA 01862 USA Phone: + 1 978 964 2287 EMail: sporetsky@avici.com Poretsky, Imhoff [Page 9] INTERNET-DRAFT Benchmarking Methodology for June 2003 IGP Data Plane Route Convergence Brent Imhoff WilTel Communications 3180 Rider Trail South Bridgeton, MO 63045 USA Phone: +1 314 595 6853 EMail: brent.imhoff@wcg.com 10. Full Copyright Statement Copyright (C) The Internet Society (1998). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Poretsky, Imhoff [Page 10]