idnits 2.17.1 draft-kishjac-bmwg-evpnvpwstest-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 2 instances of too long lines in the document, the longest one being 35 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 12, 2019) is 1596 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2544' is defined on line 507, but no explicit reference was found in the text == Unused Reference: 'RFC2899' is defined on line 512, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 518, but no explicit reference was found in the text == Unused Reference: 'RFC8214' is defined on line 523, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. Jacob, Ed. 3 Internet-Draft K. Tiruveedhula 4 Intended status: Informational Juniper Networks 5 Expires: June 14, 2020 December 12, 2019 7 Benchmarking Methodology for EVPN VPWS 8 draft-kishjac-bmwg-evpnvpwstest-03 10 Abstract 12 This document defines methodologies for benchmarking EVPN-VPWS 13 performance.EVPN-VPWS is defined in RFC 8214, and is being deployed 14 in Service Provider networks.Specifically this document defines the 15 methodologies for benchmarking EVPN-VPWS Scale convergence, Fail 16 over,Core isolation,high availability and longevity. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on June 14, 2020. 35 Copyright Notice 37 Copyright (c) 2019 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 2 54 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 56 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 6 57 3.1. Local Failure . . . . . . . . . . . . . . . . . . . . . . 6 58 3.2. Fail over Test in Remote PE . . . . . . . . . . . . . . . 7 59 3.3. Core Failure . . . . . . . . . . . . . . . . . . . . . . 7 60 3.4. Link Flap . . . . . . . . . . . . . . . . . . . . . . . . 8 61 4. Activate/deactivate AC's . . . . . . . . . . . . . . . . . . 9 62 4.1. Deactivate/Activate M number of attachment circuits. . . 9 63 5. Scale Convergence . . . . . . . . . . . . . . . . . . . . . . 9 64 5.1. To measure the packet loss during the core link failure. 9 65 6. High Availability . . . . . . . . . . . . . . . . . . . . . . 10 66 6.1. To Record the whether there is traffic loss due to 67 routing engine failover for redundancy test. . . . . . . 10 68 7. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 11 69 7.1. To Measure the stability of the DUT with scale and 70 traffic. . . . . . . . . . . . . . . . . . . . . . . . . 11 71 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 11 72 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 73 10. Security Considerations . . . . . . . . . . . . . . . . . . . 12 74 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 75 11.1. Normative References . . . . . . . . . . . . . . . . . . 12 76 11.2. Informative References . . . . . . . . . . . . . . . . . 12 77 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 13 78 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 80 1. Introduction 82 EVPN-VPWS is defined in RFC 8214,discusses how VPWS can be combined 83 with EVPNs to provide a new/combined solution. This draft defines 84 methodologies that can be used to benchmark RFC 8214 solutions. 85 Further, this draft provides methodologies for benchmarking the 86 performance of EVPN VPWS Scale,Scale Convergence, Core isolation, 87 longevity, high availability. 89 1.1. Requirements Language 91 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 92 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 93 document are to be interpreted as described in RFC 2119 [RFC2119]. 95 1.2. Terminologies 97 All-Active Redundancy Mode: When all PEs attached to an Ethernet 98 segment are allowed to forward known unicast traffic to/from that 99 Ethernet segment for a given VLAN, then the Ethernet segment is 100 defined to be operating in All-Active redundancy mode. 102 AA All Active mode 104 AC Attachment Circuits 106 CE Customer Router/Devices/Switch. 108 DF Designated Forwarder 110 DUT Device under test. 112 Ethernet Segment (ES): When a customer site (device or network) is 113 connected to one or more PEs via a set of Ethernet links, then that 114 set of links is referred to as an 'Ethernet segment'. 116 EVI: An EVPN instance spanning the Provider Edge (PE) devices 117 participating in that EVPN. 119 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 120 identifies an Ethernet segment is called an 'Ethernet Segment 121 Identifier'. 123 Ethernet Tag: An Ethernet tag identifies a particular broadcast 124 domain, e.g., a VLAN. An EVPN instance consists of one or more 125 broadcast domains. 127 Interface Physical interface of a router/switch. 129 IRB Integrated routing and bridging interface 131 MAC Media Access Control addresses on a PE. 133 MHPE2 Multi homed Provider Edge router 2. 135 MHPE1 Multi homed Provider Edge router 1. 137 SHPE3 Single homed Provider Edge Router 3. 139 PE: Provider Edge device. 141 P Provider Router. 143 RR Route Reflector. 145 RT Traffic Generator. 147 Sub Interface Each physical Interfaces is subdivided into Logical 148 units. 150 SA Single Active 152 Single-Active Redundancy Mode: When only a single PE, among all the 153 PEs attached to an Ethernet segment, is allowed to forward traffic 154 to/from that Ethernet segment for a given VLAN, then the Ethernet 155 segment is defined to be operating in Single-Active redundancy mode. 157 2. Test Topology 159 EVPN-VPWS Services running on SHPE3, MHPE1 and MHPE2 in Single Active 160 Mode: 162 Topology Diagram 164 | Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. 165 +----------+ 166 | | 167 | SHPE3 | 168 | SHPE3 | 169 +----------+ 170 | 171 |Core link 172 +----------+ 173 | | 174 | RR | 175 | | Route Reflector/Provider router 176 +----------+-------------| 177 | | 178 | Core links | 179 +----------+ +-----------+ 180 | | | MHPE2 | 181 | DUT | | | 182 | MHPE1 | | | 183 +----------+ +-----------+ 184 | PE-CE link | 185 +----------+------------ 186 | | 187 | CE | 188 | layer2 | 189 |bridge | 190 +----------+------------ Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. 192 Topology 1 194 Topology Diagram 196 Figure 1 198 There are five routers in the topology. SHPE3, RR/P, MHPE1 and MHPE2 199 emulating a service provider network. CE is a customer device 200 connected to MHPE1 and MHPE2, it is configured with bridge domains in 201 different vlans. The traffic generator is connected to CE and 202 SHPE3.The MHPE1 acts as DUT.The traffic generator will act as sender 203 and receiver.The measurement will be taken in DUT. 205 All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with 206 EVPN address family. 208 All routers except CE must have Interior Border Gateway protocol 209 configured,RR acting as route reflector. 211 MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN-VPWS instances 212 depends up on the cases. 214 MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on 215 Interface. 217 MHPE1 and MHEPE2 are running Single Active mode of EVPN-VPWS. 219 CE is acting as bridge configured with vlans that is configured on 220 MHPE1,MHPE2,SHPE3. 222 Depends up on the test traffic will be flowing uni directional or bi 223 directional depends on the topology mentioned above. 225 The above configuration will serve as base configuration for all the 226 test cases. 228 3. Test Cases 230 The following tests are conducted to measure the packet loss during 231 the local link and core failure in DUT with Scaled AC's. 233 3.1. Local Failure 235 Objective: 237 To Record the time taken to switch from primary to backup during 238 local link failure. 240 Topology : Topology 1 242 Procedure: 244 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 245 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 246 packets to CE from traffic generator to MHPE2 AC's working in SA. 247 Then shut the MHPE2-CE link, so that traffic from CE switches to DUT. 249 Measurement : 251 Measure the time taken to switch the traffic from active to backup, 252 the traffic will flow from MHPE1 to SHPE3. Measure the time taken to 253 switch the traffic. 255 The test is repeated for "N" times and the values are collected. The 256 switching time is calculated by averaging the values obtained from 257 "N" samples. 259 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 261 3.2. Fail over Test in Remote PE 263 Objective: 265 To Record the time taken by remote PE to switch traffic from primary 266 to backup during CE link failure. 268 Topology : Topology 1 270 Procedure: 272 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 273 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 274 packets from traffic generator to SHPE3 Ac's.Then shut the MHPE2-CE 275 link, this failure will be notified to remote PE and traffic switch 276 to backup path. 278 Measurement : 280 Measure the time taken to switch the traffic from active to backup, 281 the traffic will flow from SHPE3 to MHPE1. Measure the time taken to 282 switch the traffic. 284 The test is repeated for "N" times and the values are collected. The 285 switching time is calculated by averaging the values obtained from 286 "N" samples. 288 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 290 3.3. Core Failure 292 Objective: 294 To Record the time taken by remote PE to switch traffic from primary 295 to backup during core link failure. 297 Topology : Topology 1 298 Procedure: 300 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 301 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 302 packets from traffic generator to SHPE3 Ac's.Then shut the core link 303 of MHPE2,this failure will be notified to remote PE and traffic 304 switch to backup path. 306 Measurement : 308 Measure the time taken to switch the traffic from active to backup, 309 the traffic will flow from SHPE3 to MHPE1. Measure the time taken to 310 switch the traffic. 312 The test is repeated for "N" times and the values are collected. The 313 switching time is calculated by averaging the values obtained from 314 "N" samples. 316 AC's in remote PE switches from primary to backup PE in sec due to 317 core failure = (T1+T2+..Tn/N) 319 3.4. Link Flap 321 Objective: 323 To Record the time taken by primary PE to regain control after the 324 local PE-CE link flap. 326 Topology : Topology 1 328 Procedure: 330 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 331 mode.Ensure MHPE2 is standby and DUT is primary PE.Send "X" unicast 332 packets from traffic generator connected to CE to all Ac's in 333 MHPE1(DUT).Then shut the link of MHPE1-CE,this failure will be 334 notified to remote PE and traffic switch to backup path. Then bring 335 up the link of MHPE1-CE.Now the traffic switches to DUT. 337 Measurement : 339 Measure the time taken to switch the traffic from MHPE2 to DUT, the 340 traffic will flow from MHPE1 to SHPE3. Measure the time taken to 341 switch the traffic. 343 The test is repeated for "N" times and the values are collected. The 344 switching time is calculated by averaging the values obtained from 345 "N" samples. 347 Time taken to switch back to primary(DUT) once the link is restored = 348 (T1+T2+..Tn/N) 350 4. Activate/deactivate AC's 352 4.1. Deactivate/Activate M number of attachment circuits. 354 Objective: 356 To measure the performance of the DUT while deactivating/activating 357 AC's. 359 Topology : Topology 1 361 Procedure: 363 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 364 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 365 packets from traffic generator to SHPE3 to all Ac's and send "X" 366 unicast packets from CE to MHPE1(DUT),let the DUT is the active and 367 the MHPE2 must be standby.DUT will be forwarding the traffic to CE 368 and from CE to SHPE3.Then deactivate "M" AC's on SHPE1,DUT and MHPE2 369 on the fly. these AC' must be removed from forwarding plane. Stop 370 the traffic for these AC's. Activate the AC's in all PE's. then 371 start the traffic, measure the time taken by "M" AC's to forward the 372 traffic. 374 Measurement : 376 Measure the packet loss in sec during this deactivating/activating 377 AC's.Repeat the test "N" times and the packet loss is calculated by 378 averaging the values obtained from "N" samples. 380 packet loss in sec = (T1+T2+..Tn/N) 382 5. Scale Convergence 384 5.1. To measure the packet loss during the core link failure. 386 Objective: 388 To Measure the convergence at a higher number of AC's 389 Topology : Topology 1 391 Procedure: 393 Configure "N'" AC's in SHPE3 and MHPE1,MHPE2, working in SA mode.DF 394 election must be priority based not on the default RFC 7432, it 395 should not be MOD based DF election.Send "X" unicast packets from 396 traffic generator to SHPE3 to all Ac's and send "X" unicast packets 397 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 must be 398 standby. DUT will be forwarding the traffic to CE from SHPE3 and 399 from CE to SHPE3.Then flap the core link of the DUT. 401 Measurement : 403 Measure the packet loss in seconds once the core link is 404 restored.Repeat the test "N" times and the packet loss is calculated 405 by averaging the values obtained from "N" samples. 407 Packet loss in sec = (T1+T2+..Tn/N) 409 6. High Availability 411 6.1. To Record the whether there is traffic loss due to routing engine 412 fail over for redundancy test. 414 Objective: 416 To record traffic loss during routing engine failover. 418 Topology : Topology 1 420 Procedure: 422 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 423 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 424 packets from traffic generator to SHPE3 to all Ac's and send "X" 425 unicast packets from CE to MHPE1(DUT),let the DUT is the active and 426 the MHPE2 must be standby. DUT will be forwarding the traffic to CE 427 and from CE to SHPE3.Then do a routing engine fail-over. 429 Measurement : 431 The expectation of the test is 0 traffic loss with no change in the 432 DF role. DUT should not withdraw any routes.But in cases where the 433 DUT is not property synchronized between master and standby,due to 434 that packet loss are observed. In that scenario the packet loss is 435 measured.The test is repeated for "N" times and the values are 436 collected.The packet loss is calculated by averaging the values 437 obtained by "N" samples. 439 Packet loss in sec = (T1+T2+..Tn/N) 441 7. SOAK Test 443 This test is carried out to measure the stability of the DUT in a 444 scaled environment with traffic over a period of time "T'". In each 445 interval "t1" the DUT CPU usage, memory usage are measured. The DUT 446 is checked for any crashes during this time period. 448 7.1. To Measure the stability of the DUT with scale and traffic. 450 Objective: 452 To measure the stability of the DUT in a scaled environment with 453 traffic. 455 Topology : Topology 1 457 Procedure: 459 Scale N AC's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE 460 using traffic generator with different X SA and DA for N EVI's. Send 461 F frames from traffic generator to SHPE3 with X different SA and DA. 462 There is a bi directional traffic flow with F pps in each direction. 463 The DUT must run with traffic for 24 hours, every hour check for 464 memory leak, crash. 466 Measurement : 468 Take the hourly reading of CPU, process memory.There should not be 469 any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU 470 usage which shoots at 40 to 50 percent of the average usage. The 471 average value vary from device to device. Memory leak is determined 472 by increase usage of the memory for EVPN-VPWS process. The 473 expectation is under steady state the memory usage for EVPN-VPWS 474 process should not increase. 476 8. Acknowledgments 478 We would like to thank Al and Sarah for the support. 480 9. IANA Considerations 482 This memo includes no request to IANA. 484 10. Security Considerations 486 The benchmarking tests described in this document are limited to the 487 performance characterization of controllers in a lab environment with 488 isolated networks. The benchmarking network topology will be an 489 independent test setup and MUST NOT be connected to devices that may 490 forward the test traffic into a production network or misroute 491 traffic to the test management network. Further, benchmarking is 492 performed on a "black-box" basis, relying solely on measurements 493 observable external to the controller. Special capabilities SHOULD 494 NOT exist in the controller specifically for benchmarking purposes. 495 Any implications for network security arising from the controller 496 SHOULD be identical in the lab and in production networks. 498 11. References 500 11.1. Normative References 502 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 503 Requirement Levels", BCP 14, RFC 2119, 504 DOI 10.17487/RFC2119, March 1997, 505 . 507 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 508 Network Interconnect Devices", RFC 2544, 509 DOI 10.17487/RFC2544, March 1999, 510 . 512 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 513 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, 514 . 516 11.2. Informative References 518 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 519 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 520 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 521 2015, . 523 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 524 Rabadan, "Virtual Private Wire Service Support in Ethernet 525 VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017, 526 . 528 Appendix A. Appendix 530 Authors' Addresses 532 Sudhin Jacob (editor) 533 Juniper Networks 534 Bangalore 535 India 537 Phone: +91 8061212543 538 Email: sjacob@juniper.net 540 Kishore Tiruveedhula 541 Juniper Networks 542 10 Technology Park Dr 543 Westford, MA 01886 544 USA 546 Phone: +1 9785898861 547 Email: kishoret@juniper.net