idnits 2.17.1 draft-kishjac-bmwg-evpnvpwstest-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 11 instances of too long lines in the document, the longest one being 14 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 203 has weird spacing: '...nerator sende...' == Line 204 has weird spacing: '...eceiver of la...' -- The document date (May 6, 2020) is 1450 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2544' is defined on line 527, but no explicit reference was found in the text == Unused Reference: 'RFC2899' is defined on line 532, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 538, but no explicit reference was found in the text == Unused Reference: 'RFC8214' is defined on line 543, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. Jacob, Ed. 3 Internet-Draft K. Tiruveedhula 4 Intended status: Informational Juniper Networks 5 Expires: November 7, 2020 May 6, 2020 7 Benchmarking Methodology for EVPN VPWS 8 draft-kishjac-bmwg-evpnvpwstest-04 10 Abstract 12 This document defines methodologies for benchmarking EVPN-VPWS 13 performance. EVPN-VPWS is defined in RFC 8214, and is being deployed 14 in Service Provider networks. Specifically this document defines the 15 methodologies for benchmarking EVPN-VPWS Scale convergence, Fail 16 over,Core isolation,high availability and longevity. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on November 7, 2020. 35 Copyright Notice 37 Copyright (c) 2020 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 2 54 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 2 55 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 56 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 6 57 3.1. Local Failure Scenario 1 . . . . . . . . . . . . . . . . 7 58 3.2. Local Failure Scenario 2 . . . . . . . . . . . . . . . . 7 59 3.3. Core Failure . . . . . . . . . . . . . . . . . . . . . . 8 60 3.4. Link Flap . . . . . . . . . . . . . . . . . . . . . . . . 9 61 4. Scale Convergence . . . . . . . . . . . . . . . . . . . . . . 9 62 4.1. To measure the packet loss during the core link failure. 9 63 5. High Availability . . . . . . . . . . . . . . . . . . . . . . 10 64 5.1. To Record the whether there is traffic loss due to 65 routing engine failover for redundancy test. . . . . . . 10 66 6. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 11 67 6.1. To Measure the stability of the DUT with scale and 68 traffic. . . . . . . . . . . . . . . . . . . . . . . . . 11 69 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12 70 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 71 9. Security Considerations . . . . . . . . . . . . . . . . . . . 12 72 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 73 10.1. Normative References . . . . . . . . . . . . . . . . . . 12 74 10.2. Informative References . . . . . . . . . . . . . . . . . 12 75 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 13 76 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 78 1. Introduction 80 EVPN-VPWS is defined in RFC 8214,discusses how VPWS can be combined 81 with EVPNs to provide a new/combined solution. This draft defines 82 methodologies that can be used to benchmark RFC 8214 solutions. 83 Further, this draft provides methodologies for benchmarking the 84 performance of EVPN VPWS Scale,Scale Convergence, Core isolation, 85 longevity, high availability. 87 1.1. Requirements Language 89 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 90 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 91 document are to be interpreted as described in RFC 2119 [RFC2119]. 93 1.2. Terminologies 95 All-Active Redundancy Mode: When all PEs attached to an Ethernet 96 segment are allowed to forward known unicast traffic to/from that 97 Ethernet segment for a given VLAN, then the Ethernet segment is 98 defined to be operating in All-Active redundancy mode. 100 AA: All Active mode 102 AC: Attachment Circuits 104 CE: Customer Router/Devices/Switch. 106 DF: Designated Forwarder 108 DUT: Device under test. 110 Ethernet Segment (ES): When a customer site (device or network) is 111 connected to one or more PEs via a set of Ethernet links, then that 112 set of links is referred to as an 'Ethernet segment'. 114 EVI: An EVPN instance spanning the Provider Edge (PE) devices 115 participating in that EVPN. 117 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 118 identifies an Ethernet segment is called an 'Ethernet Segment 119 Identifier'. 121 Ethernet Tag: An Ethernet tag identifies a particular broadcast 122 domain, e.g., a VLAN. An EVPN instance consists of one or more 123 broadcast domains. 125 Interface: Physical interface of a router/switch. 127 IRB: Integrated routing and bridging interface 129 MAC: Media Access Control addresses on a PE. 131 MHPE2: Multi homed Provider Edge router 2. 133 MHPE1: Multi homed Provider Edge router 1. 135 SHPE3: Single homed Provider Edge Router 3. 137 PE: Provider Edge device. 139 P: Provider Router. 141 RR: Route Reflector. 143 RT: Traffic Generator. 145 Sub Interface: Each physical Interfaces is subdivided into Logical 146 units. 148 SA: Single Active 150 Single-Active Redundancy Mode: When only a single PE, among all the 151 PEs attached to an Ethernet segment, is allowed to forward traffic 152 to/from that Ethernet segment for a given VLAN, then the Ethernet 153 segment is defined to be operating in Single-Active redundancy mode. 155 VPWS: Virtual private wire service. 157 2. Test Topology 159 There are five routers in the Test setup. SHPE3, RR/P, MHPE1 and 160 MHPE2 emulating a service provider network. CE is a customer device 161 connected to MHPE1 and MHPE2, it is configured with bridge domains in 162 multiple vlans. The traffic generator is connected to CE and 163 SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as 164 sender and receiver of traffic.The DUT will be the reference point 165 for all the test cases. MHPE1 and MHPE2 are multi home routers 166 connected to CE running single active mode. The traffic generator 167 will be generating traffic at 10% of the line rate. 169 Topology Diagram 171 +----------------+ +-------------------------+ 172 | | | | 173 | | |Traffic Generator sender/| 174 | SHPE3 | |receiver of layer 2 traffic| 175 | +-----------------+ with multiple Vlans | 176 | | +-------------------------+ 177 +---------+------+ 178 | Core Link 179 | 180 +--------+-----+ 181 | | 182 | RR/P | 183 | | Core link 184 | +----------------+ 185 +--+-----------+ | 186 | | 187 | core link | 188 | | 189 +-------------+---+ ++------------------+ 190 | | | | 191 | | | MHPE2 | 192 |MHPE1(DUT) | | | 193 | | | | 194 | | | | 195 +-----------------+------+ +-----+-------------------+ 196 | | 197 PE-CE link | | PE-CE link 198 | | 199 | | 200 | | 201 | | 202 +-----+----------+----+ +----------------------------+ 203 | CE/Layer 2 bridge +-----------| Traffic Generator sender/ | 204 | | |receiver of layer 2 traffic| 205 | | | with multiple Vlans | 206 | | +----------------------------+ 207 | | 208 +---------------------+ 210 Topology 1 212 Topology Diagram 214 Figure 1 216 Test Setup Configurations: 218 SHPE3 is configured with Interior Gateway protocols like OPSF or IS- 219 IS for underlay, LDP for MPLS support,Interior Border Gateway with 220 EVPN address family for overlay support. This router must be 221 configured with N EVPN-VPWS instances for testing. Traffic generator 222 is connected to this router for sending and receiving traffic. 224 RR is configured with Interior Gateway protocols like OPSF or IS-IS 225 for underlay, LDP for MPLS support,Interior Border Gateway with EVPN 226 address family for overlay support. This router acts as a provider 227 router and as a route reflector. 229 MHPE1 is configured with Interior Gateway protocols like OPSF or IS- 230 IS for underlay, LDP for MPLS support,Interior Border Gateway with 231 EVPN address family for overlay support.This router must be 232 configured with N EVPN-VPWS instances for testing.This router is 233 configured with ESI per vlan or ESI per interface. It is functioning 234 as multi homing PE working on Single Active EVPN mode.This router 235 serves as the DUT and it is connected to CE. MHPE1 is acting as DUT 236 for all the test cases. 238 MHPE2 is configured with Interior Gateway protocols like OPSF or IS- 239 IS for underlay, LDP for MPLS support,Interior Border Gateway with 240 EVPN address family for overlay support.This router must be 241 configured with N EVPN-VPWS instances for testing.This router is 242 configured with ESI per vlan or ESI per interface. It is functioning 243 as multi homing PE working on Single Active EVPN mode.It is 244 connected to CE. 246 CE is acting as bridge configured with multiple vlans,the same vlans 247 are configured on MHPE1,MHPE2,SHPE3. traffic generator is connected 248 to CE. The traffic generator acts as sender or receiver of traffic. 250 Depending up on the test scenarios the traffic generators will be 251 used to generate uni directional or bi directional flows. 253 The above configuration will be serving as the base configuration for 254 all test cases. 256 3. Test Cases 258 The following tests are conducted to measure the packet loss during 259 the local link and core failure in DUT with Scaled AC's. 261 3.1. Local Failure Scenario 1 263 Objective: 265 Measure the time taken to switch from primary to backup during local 266 link failure. 268 Topology : Topology 1 270 Procedure: 272 Confirm the DUT is up and running with EVPN-VPWS. The AC must be up 273 and running. "N" AC's in MHPE1,MHPE2, working in SA mode.Ensure DUT 274 is active and MHPE2 is backup PE.Send unicast packets to CE from 275 traffic generator. The traffic is uni directional and it flows from 276 CE to DUT working as Active router. Then shut the DUT-CE link, so 277 that traffic from CE switches to MHPE2.Traffic must be tested with 278 various line rate that from 10% to 98%. 280 Measurement : 282 Measure the time taken by the traffic to switch from Active router to 283 the backup. The test is repeated for "N" times and the values are 284 collected. The AC's local switch over time is calculated by 285 averaging the values obtained by "N" samples. "N" is an arbitrary 286 number to get a sufficient sample. The time measured for each sample 287 is denoted by T1,T2...Tn.The measurement is carried out using 288 external server which polls the DUT using automated scripts. Fail 289 over time must be measured for various line rate. 291 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 293 3.2. Local Failure Scenario 2 295 Objective: 297 Measure time taken by remote PE to switch traffic from primary to 298 backup during CE link failure. 300 Topology : Topology 1 302 Procedure: 304 Confirm the DUT is up and running with EVPN-VPWS. The AC must be up 305 and running. "N" AC's in MHPE1,MHPE2, working in SA mode.Ensure DUT 306 is active and MHPE2 is backup PE.Send unicast packets to SHPE3 from 307 traffic generator. The traffic is uni directional and it flows from 308 SHPE3 to DUT working as Active router. Then shut the DUT-CE link, so 309 the remote traffic flow switches from DUT to MHPE2.Traffic must be 310 tested with various line rate that from 10% to 98%. 312 Measurement : 314 Measure the time taken by the traffic to switch from Active router to 315 the backup.The test is repeated for "N" times and the values are 316 collected. The AC's switch over time for the remote traffic is 317 calculated by averaging the values obtained by "N" samples. "N" is 318 an arbitrary number to get a sufficient sample. The time measured 319 for each sample is denoted by T1,T2...Tn.The measurement is carried 320 out using external server which polls the DUT using automated 321 scripts. Fail over time must be measured for various line rate. 323 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 325 3.3. Core Failure 327 Objective: 329 Measure the time taken by remote PE to switch traffic from primary to 330 backup during core link failure. 332 Topology : Topology 1 334 Procedure: 336 Confirm the DUT is up and running with EVPN-VPWS. The AC must be up 337 and running. "N" AC's in MHPE1,MHPE2, working in SA mode.Ensure DUT 338 is active and MHPE2 is backup PE.Send unicast packets to SHPE3 from 339 traffic generator. The traffic is uni directional and it flows from 340 SHPE3 to DUT working as Active router. Then shut the DUT core link, 341 so the remote traffic flow switches from DUT to MHPE2. Traffic must 342 be tested with various line rate that from 10% to 98%. 344 Measurement : 346 Measure the time taken by the traffic to switch from Active router to 347 the backup. The test is repeated for "N" times and the values are 348 collected. The AC's switch over time for the remote traffic is 349 calculated by averaging the values obtained by "N" samples. "N" is 350 an arbitrary number to get a sufficient sample. The time measured 351 for each sample is denoted by T1,T2...Tn.The measurement is carried 352 out using external server which polls the DUT using automated 353 scripts. Fail over time must be measured for various line rate. 355 AC's core Failure fail over time = (T1+T2+..Tn/N) 357 3.4. Link Flap 359 Objective: 361 Measure time taken by primary PE to regain control after the local 362 PE-CE link flap. 364 Topology : Topology 1 366 Procedure: 368 Confirm the DUT is up and running with EVPN-VPWS. The AC must be up 369 and running. "N" AC's in MHPE1,MHPE2, working in SA mode.Ensure DUT 370 is active and MHPE2 is backup PE.Send unicast packets to CE from 371 traffic generator. The traffic is uni directional and it flows from 372 CE to DUT working as Active router. Then shut the DUT core link, so 373 the local traffic flow switches from DUT to MHPE2. Once the fail 374 over is performed. Bring the link up. Now the DUT becomes the 375 Active router. Measure time taken by the DUT to regain the 376 traffic.Traffic must be tested with various line rate that from 10% 377 to 98%. 379 Measurement : 381 Measure the time taken by the traffic to switch back to the DUT. The 382 test is repeated for "N" times and the values are collected. The 383 AC's switch over time for the remote traffic is calculated by 384 averaging the values obtained by "N" samples. "N" is an arbitrary 385 number to get a sufficient sample. The time measured for each sample 386 is denoted by T1,T2...Tn.The measurement is carried out using 387 external server which polls the DUT using automated scripts. Fail 388 over time must be measured for various line rate. 390 Time taken to switch back to primary(DUT) once the link is restored = 391 (T1+T2+..Tn/N) 393 4. Scale Convergence 395 4.1. To measure the packet loss during the core link failure. 397 Objective: 399 Measure the convergence at a higher number of AC's 401 Topology : Topology 1 402 Procedure: 404 Confirm the DUT is up and running with EVPN-VPWS. The AC must be up 405 and running. "N*100" AC's in MHPE1,MHPE2, working in SA mode.Ensure 406 DUT is active and MHPE2 is backup PE.Send unicast packets to CE from 407 traffic generator and send traffic from traffic generator to 408 SHPE3.The traffic is directional and it flows from CE to DUT and from 409 DUT to CE, working as Active router. Then shut the DUT core link, so 410 the traffic flow switches from DUT to MHPE2. Measure traffic 411 switching time.Traffic must be tested with various line rate that 412 from 10% to 98%. 414 Measurement : 416 Measure the time taken by the traffic to switch from DUT to MHPE2. 417 The test is repeated for "N" times and the values are collected. The 418 AC's switch over time for the traffic is calculated by averaging the 419 values obtained by "N" samples. "N" is an arbitrary number to get a 420 sufficient sample. The time measured for each sample is denoted by 421 T1,T2...Tn.The measurement is carried out using external server which 422 polls the DUT using automated scripts. Fail over time must be 423 measured for various line rate. 425 Packet loss in sec = (T1+T2+..Tn/N) 427 5. High Availability 429 5.1. To Record the whether there is traffic loss due to routing engine 430 failover for redundancy test. 432 Objective: 434 Measure the traffic loss during routing engine fail over. 436 Topology : Topology 1 438 Procedure: 440 Confirm the DUT is up and running with EVPN-VPWS. The AC must be up 441 and running. "N*100" AC's in MHPE1,MHPE2, working in SA mode.Ensure 442 DUT is active and MHPE2 is backup PE.Send unicast packets to CE and 443 SHPE3 from traffic generator. The traffic is directional and it 444 flows from CE to DUT and from DUT to CE, working as Active router. 445 Do a routing engine fail over once the traffic is stabilized in DUT. 446 Traffic must be tested with various line rate that from 10% to 98%. 447 The expectation is 0 packet loss, no role change in AC's. 449 Measurement : 451 The expectation of the test is 0 traffic loss with no change in the 452 DF role. DUT should not withdraw any routes.But in cases where the 453 DUT is not property synchronized between master and standby,due to 454 that packet loss are observed. In that scenario the packet loss is 455 measured.The test is repeated for "N" times and the values are 456 collected.The packet loss is calculated by averaging the values 457 obtained by "N" samples. 459 Packet loss in sec = (T1+T2+..Tn/N) 461 6. SOAK Test 463 This test is carried out to measure the stability of the DUT in a 464 scaled environment with traffic over a period of time "T'". In each 465 interval "t1" the DUT CPU usage, memory usage are measured. The DUT 466 is checked for any crashes during this time period. 468 6.1. To Measure the stability of the DUT with scale and traffic. 470 Objective: 472 To measure the stability of the DUT in a scaled environment with 473 traffic. 475 Topology : Topology 1 477 Procedure: 479 Scale N AC's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE 480 using traffic generator with different X SA and DA for N EVI's. Send 481 F frames from traffic generator to SHPE3 with X different SA and DA. 482 There is a bi directional traffic flow with F pps in each direction. 483 The DUT must run with traffic for 24 hours, every hour check for 484 memory leak, crash. 486 Measurement : 488 Take the hourly reading of CPU, process memory.There should not be 489 any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU 490 usage which shoots at 40 to 50 percent of the average usage. The 491 average value vary from device to device. Memory leak is determined 492 by increase usage of the memory for EVPN-VPWS process. The 493 expectation is under steady state the memory usage for EVPN-VPWS 494 process should not increase. 496 7. Acknowledgements 498 We would like to thank Al and Sarah for the support. 500 8. IANA Considerations 502 This memo includes no request to IANA. 504 9. Security Considerations 506 The benchmarking tests described in this document are limited to the 507 performance characterization of controllers in a lab environment with 508 isolated networks. The benchmarking network topology will be an 509 independent test setup and MUST NOT be connected to devices that may 510 forward the test traffic into a production network or misroute 511 traffic to the test management network. Further, benchmarking is 512 performed on a "black-box" basis, relying solely on measurements 513 observable external to the controller. Special capabilities SHOULD 514 NOT exist in the controller specifically for benchmarking purposes. 515 Any implications for network security arising from the controller 516 SHOULD be identical in the lab and in production networks. 518 10. References 520 10.1. Normative References 522 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 523 Requirement Levels", BCP 14, RFC 2119, 524 DOI 10.17487/RFC2119, March 1997, 525 . 527 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 528 Network Interconnect Devices", RFC 2544, 529 DOI 10.17487/RFC2544, March 1999, 530 . 532 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 533 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, 534 . 536 10.2. Informative References 538 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 539 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 540 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 541 2015, . 543 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 544 Rabadan, "Virtual Private Wire Service Support in Ethernet 545 VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017, 546 . 548 Appendix A. Appendix 550 Authors' Addresses 552 Sudhin Jacob (editor) 553 Juniper Networks 554 Bangalore 555 India 557 Phone: +91 8061212543 558 Email: sjacob@juniper.net 560 Kishore Tiruveedhula 561 Juniper Networks 562 10 Technology Park Dr 563 Westford, MA 01886 564 USA 566 Phone: +1 9785898861 567 Email: kishoret@juniper.net