idnits 2.17.1 draft-kishjac-bmwg-evpnvpwstest-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 7 instances of too long lines in the document, the longest one being 62 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 203 has weird spacing: '... Tester sendi...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 8, 2018) is 2027 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2544' is defined on line 556, but no explicit reference was found in the text == Unused Reference: 'RFC2899' is defined on line 561, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 567, but no explicit reference was found in the text == Unused Reference: 'RFC8214' is defined on line 572, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. Jacob, Ed. 3 Internet-Draft K. Tiruveedhula 4 Intended status:Informational Juniper Networks 5 Expires: April 11, 2019 October 8, 2018 7 Benchmarking Methodology for EVPN VPWS 8 draft-kishjac-bmwg-evpnvpwstest-00 10 Abstract 12 This document defines methodologies for benchmarking EVPN-VPWS 13 performance. EVPN-VPWS is defined in RFC 8214, and is being deployed 14 in Service Provider networks. Specifically this document defines the 15 methodologies for benchmarking EVPN-VPWS Scale convergence, 16 Scale,Core isolation, high availability and longevity. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on April 11, 2019. 35 Copyright Notice 37 Copyright (c) 2018 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 54 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3 56 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 7 57 3.1. How long it takes to switch from primary to backup during 58 local link failure . . . . . . . . . . . . . . . . . . . 7 59 3.2. How long it takes to remote PE to switch traffic from 60 primary to back up path during link failure in CE . . . . 8 61 3.3. How long it takes to remote PE to switch traffic from 62 primary to back up path during core failure . . . . . . . 8 63 3.4. How long it takes to primary PE to regain control after 64 the local link flap . . . . . . . . . . . . . . . . . . . 9 65 4. Activate/deactivate AC's . . . . . . . . . . . . . . . . . . 10 66 4.1. To Add M number of attachment circuits. . . . . . . . . 10 67 4.2. Deactivate/Activate M number of attachment circuits. . . 10 68 5. Scale Convergence . . . . . . . . . . . . . . . . . . . . . . 11 69 5.1. To Record the whether there is traffic loss due to 70 routing engine failover for redundancy test. . . . . . . 11 71 6. High Availability . . . . . . . . . . . . . . . . . . . . . . 11 72 6.1. To Record the whether there is traffic loss due to 73 routing engine failover for redundancy test. . . . . . . 12 74 7. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 12 75 7.1. To Measure the stability of the DUT with scale and 76 traffic. . . . . . . . . . . . . . . . . . . . . . . . . 12 77 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13 78 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 79 10. Security Considerations . . . . . . . . . . . . . . . . . . . 13 80 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 81 11.1. Normative References . . . . . . . . . . . . . . . . . . 13 82 11.2. Informative References . . . . . . . . . . . . . . . . . 13 83 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 14 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 86 1. Introduction 88 EVPN-VPWS is defined in RFC 8214,discusses how VPWS can be combined 89 with EVPNs to provide a new/combined solution. This draft defines 90 methodologies that can be used to benchmark RFC 8214 solutions. 91 Further, this draft provides methodologies for benchmarking the 92 performance of EVPN VPWS Scale,Scale Convergence, Core isolation, 93 longevity, high availability. 95 1.1. Requirements Language 97 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 98 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 99 document are to be interpreted as described in RFC 2119 [RFC2119]. 101 1.2. Terminologies 103 MHPE Multi homed Provide Edge router. 105 RR Route Reflector. 107 P Provider Router. 109 CE Customer Router/Devices/Switch. 111 MHPE2 Multi homed Provider Edge router 2. 113 MHPE1 Multi homed Provider Edge router 1. 115 SHPE3 Single homed Provider Edge Router 3. 117 AA EVPN Terminologies AA All-Active. 119 AC Attachment Circuit( customer EVPN-VPWS Service over the Provider 120 network 122 SA EVPN Terminologies SA Single-Active. 124 RT Router Tester. 126 Sub Interface Each physical Interfaces is subdivided in to Logical 127 units. 129 EVI EVPN Instances which will be running on sub interface or physical 130 port of the provider Edge routers. 132 DF Designated Forwarder. 134 ESI Ethernet Segment Identifier. 136 2. Test Topology 138 EVPN-VPWS Services running on SHPE3, MHPE1 and MHPE2 in Single Active 139 Mode: 141 Topology Diagram 143 | [Traffic Generator ] Router Tester traffic receiver for layer 2 traffic from CE 144 +----------+ 145 | | 146 | SHPE3 | 147 | SHPE3 | 148 +----------+ 149 | 150 |Core link 151 +----------+ 152 | | 153 | RR | 154 | | Route Reflector/Core router 155 +----------+-------------| 156 | | 157 | Core links | 158 +----------+ +-----------+ 159 | | | MHPE2 | 160 | DUT | | | 161 | MHPE1 | | | 162 +----------+ +-----------+ 163 | PE-CE link | 164 +----------+------------ 165 | | 166 | CE | 167 | layer2 | 168 |bridge | 169 +----------+------------ [Traffic Generator](Router Tester sending layer 2 traffic with different VLAN ) 171 Topology 1 173 | [Traffic Generator ] Router Tester sending layer 2 traffic. 175 +----------+ 176 | | 177 | SHPE3 | 178 | SHPE3 | 179 +----------+ 180 | 181 |Core link 182 +----------+ 183 | | 184 | RR | 185 | | Route Reflector/Core router 186 +----------+-------------| 187 | | 188 | Core links | 189 +----------+ +-----------+ 190 | | | MHPE2 | 191 | DUT | | | 192 | MHPE1 | | | 193 +----------+ +-----------+ 194 | PE-CE link | 195 +----------+------------ 196 | | 197 | CE | 198 | layer2 | 199 |bridge | 200 +----------+------------ [Traffic Generator](Router Tester receiver for layer 2 traffic with different vlans.) 202 Topology 2 203 | [Traffic Generator ] Router Tester sending layer 2 bi directional traffic sender/receiver 204 +----------+ 205 | | 206 | SHPE3 | 207 | SHPE3 | 208 +----------+ 209 | 210 |Core link 211 +----------+ 212 | | 213 | RR | 214 | | Route Reflector/Core router 215 +----------+-------------| 216 | | 217 | Core links | 218 +----------+ +-----------+ 219 | | | MHPE2 | 220 | DUT | | | 221 | MHPE1 | | | 222 +----------+ +-----------+ 223 | PE-CE link | 224 +----------+------------ 225 | | 226 | CE | 227 | layer2 | 228 |bridge | 229 +----------+------------ [Traffic Generator](Router Tester sending bi directional layer 2 traffic with different VLAN sender/receiver) 231 Topology 3 233 Topology Diagram 235 Figure 1 237 There are five routers in the topology. SHPE3, RR/P, MHPE1 and MHPE2 238 emulating a service provider network. CE is a customer device 239 connected to MHPE1 and MHPE2, it is configured with bridge domains in 240 different vlans. The router tester is connected to CE and SHPE3.The 241 MHPE1 acts as DUT.The RT will act as sender and receiver.The measurement 242 will be taken in DUT. 244 All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with 245 EVPN address family. 247 All routers except CE must have IBGP configured with RR acting as 248 route reflector. 250 MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN-VPWS instances 251 depends up on the cases. 253 MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD. 255 MHPE1 and MHEPE2 are running Single Active mode of EVPN-VPWS. 257 CE is acting as bridge configured with vlans that is configured on 258 MHPE1,MHPE2,SHPE3. 260 Depends up on the test traffic will be flowing uni directional or bi 261 directional depends on the topology mentioned above. 263 The above configuration will serve as base configuration for all the 264 test cases. 266 3. Test Cases 268 The following tests are conducted to measure the packet loss during 269 the local link and core failure in DUT with Scaled AC's. 271 3.1. How long it takes to switch from primary to backup during local 272 link failure 274 Objective: 276 To Record the time taken to switch from primary to backup during 277 local link failure. 279 Topology : Topology 1 281 Procedure: 283 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 284 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 285 packets from CE to MHPE2 AC's working in SA.Then shut the 286 MHPE2-CE link, so that traffic from CE switches to DUT. 288 Measurement : 290 Measure the time taken to switch the traffic from active to backup, 291 the traffic will flow from MHPE1 to SHPE3. Measure the time taken to 292 switch the traffic. 294 Repeat these test and plot the data. The test is repeated for "N" 295 times and the values are collected. The switching time is calculated 296 by averaging the values obtained from "N" samples. 298 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 300 3.2. How long it takes to remote PE to switch traffic from primary to 301 back up path during link failure in CE 303 Objective: 305 To Record the time taken by remote PE to switch traffic from primary 306 to backup during CE link failure. 308 Topology : Topology 2 310 Procedure: 312 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 313 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 314 packets from RT to SHPE3 Ac's.Then shut the MHPE2-CE link, 315 this failure will be notified to remote PE and traffic switch to 316 backup path. 318 Measurement : 320 Measure the time taken to switch the traffic from active to backup, 321 the traffic will flow from SHPE3 to MHPE1. Measure the time taken to 322 switch the traffic. 324 Repeat these test and plot the data. The test is repeated for "N" 325 times and the values are collected. The switching time is calculated 326 by averaging the values obtained from "N" samples. 328 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 330 3.3. How long it takes to remote PE to switch traffic from primary to 331 back up path during core failure 333 Objective: 335 To Record the time taken by remote PE to switch traffic from primary 336 to backup during core link failure. 338 Topology : Topology 2 340 Procedure: 342 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 343 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 344 packets from RT to SHPE3 Ac's.Then shut the core link of 345 MHPE2,this failure will be notified to remote PE and traffic switch 346 to backup path. 348 Measurement : 350 Measure the time taken to switch the traffic from active to backup, 351 the traffic will flow from SHPE3 to MHPE1. Measure the time taken to 352 switch the traffic. 354 Repeat these test and plot the data. The test is repeated for "N" 355 times and the values are collected. The switching time is calculated 356 by averaging the values obtained from "N" samples. 358 AC's in remote PE switches from primary to backup PE in sec due to 359 core failure = (T1+T2+..Tn/N) 361 3.4. How long it takes to primary PE to regain control after the local 362 link flap 364 Objective: 366 To Record the time taken by primary PE to regain control after the 367 local PE-CE link flap. 369 Topology : Topology 1 371 Procedure: 373 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 374 mode.Ensure MHPE2 is standby and DUT is primary PE.Send "X" unicast 375 packets from CE to all Ac's in MHPE1(DUT).Then shut the link of 376 MHPE1-CE,this failure will be notified to remote PE and traffic 377 switch to backup path. Then bring up the link of MHPE1-CE.Now the 378 traffic switches to DUT. 380 Measurement : 382 Measure the time taken to switch the traffic from MHPE2 to DUT, the 383 traffic will flow from MHPE1 to SHPE3. Measure the time taken to 384 switch the traffic. 386 Repeat these test and plot the data. The test is repeated for "N" 387 times and the values are collected. The switching time is calculated 388 by averaging the values obtained from "N" samples. 390 Time taken to switch back to primary(DUT) once the link is restored = 391 (T1+T2+..Tn/N) 393 4. Activate/deactivate AC's 395 4.1. To Add M number of attachment circuits. 397 Objective: 399 To measure the performance of the DUT while adding M AC's on the fly. 401 Topology : Topology 3 403 Procedure: 405 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 406 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 407 packets from RT to SHPE3 Ac's and send "X" unicast packets 408 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 must be 409 standby. DUT will be forwarding the traffic to CE from SHPE3 and the 410 traffic from CE to SHPE3.Then add "M" AC's on SHPE1,DUT and MHPE2 on 411 the fly. these AC' must be in SA mode. 413 Measurement : 415 There should be 0 traffic loss in existing services while addition of 416 these ACs. 418 4.2. Deactivate/Activate M number of attachment circuits. 420 Objective: 422 To measure the performance of the DUT while deactivating/activating 423 AC's. 425 Topology : Topology 3 427 Procedure: 429 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 430 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 431 packets from RT to SHPE3 to all Ac's and send "X" unicast packets 432 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 must be 433 standby.DUT will be forwarding the traffic to CE and from CE to 434 SHPE3.Then deactivate "M" AC's on SHPE1,DUT and MHPE2 on the fly. 435 these AC' must be removed from forwarding plane. Stop the traffic 436 for these AC's. Activate the AC's in all PE's. then start the 437 traffic, measure the time taken by "M" AC's to forward the traffic. 439 Measurement : 441 Measure the time taken to forward the traffic.Repeat the test "N" 442 times and plot the data.The packet loss is calculated by averaging 443 the values obtained from "N" samples. 445 Time taken by the "M" AC's to forward the traffic = (T1+T2+..Tn/N) 447 5. Scale Convergence 449 5.1. To Record the whether there is traffic loss due to routing engine 450 failover for redundancy test. 452 Objective: 454 To Measure the convergence at a higher number of AC's 456 Topology : Topology 3 458 Procedure: 460 Configure "N'" AC's in SHPE3 and MHPE1,MHPE2, working in SA mode.The 461 scale factor must be in the multiples of thousands. DF election must 462 be priority based. It should not be MOD based DF election. 463 Send "X" unicast packets from RT to SHPE3 to all Ac's and send "X" 464 unicast packets from CE to MHPE1(DUT), let the DUT be the active 465 and the MHPE2 is standby. DUT will be forwarding 466 the traffic to CE and from the SHPE3 and from the CE to SHPE3. 467 Then flap the core link of the DUT. 469 Measurement : 471 Measure the packet loss in seconds once the core link is 472 restored.Repeat the test "N" times and plot the data.The packet loss 473 is calculated by averaging the values obtained from "N" samples. 475 Packet loss in sec = (T1+T2+..Tn/N) 477 6. High Availability 478 6.1. To Record the whether there is traffic loss due to routing engine 479 failover for redundancy test. 481 Objective: 483 To record traffic loss during routing engine failover. 485 Topology : Topology 3 487 Procedure: 489 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 490 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 491 packets from RT to SHPE3 to Ac's and send "X" unicast packets 492 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 is the 493 standby. DUT will be forwarding the traffic to CE and from CE to 494 SHPE3.Then do a routing engine fail-over. 496 Measurement : 498 There should be 0 traffic loss which is the ideal case, No change in 499 the DF role. DUT should not withdraw any routes.Repeat the test "N" 500 times and plot the data.The packet loss is calculated by averaging 501 the values obtained from "N" samples. 503 Packet loss in sec = (T1+T2+..Tn/N) 505 7. SOAK Test 507 This is measuring the performance of DUT running with scaled 508 configuration with traffic over a peroid of time "T'". In each 509 interval "t1" the parameters measured are CPU usage, memory usage, 510 crashes. 512 7.1. To Measure the stability of the DUT with scale and traffic. 514 Objective: 516 To measure the stability of the DUT in a scaled environment with 517 traffic. 519 Topology : Topology 3 521 Procedure: 523 Scale N AC's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE 524 using traffic generator with different X SA and DA for N EVI's. Send 525 F frames from traffic generator to SHPE3 with X different SA and DA. 526 There is a bi directional traffic flow with F pps in each direction. 527 The DUT must run with traffic for 24 hours, every hour check for 528 memory leak, crash. 530 Measurement : 532 Take the hourly reading of CPU, process memory. There should not be 533 any leak, crashes, CPU spikes. 535 8. Acknowledgements 537 We would like to thank Al and Sarah for the support. 539 9. IANA Considerations 541 This memo includes no request to IANA. 543 10. Security Considerations 545 There is no additional consideration from RFC 6192. 547 11. References 549 11.1. Normative References 551 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 552 Requirement Levels", BCP 14, RFC 2119, 553 DOI 10.17487/RFC2119, March 1997, 554 . 556 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 557 Network Interconnect Devices", RFC 2544, 558 DOI 10.17487/RFC2544, March 1999, 559 . 561 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 562 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, 563 . 565 11.2. Informative References 567 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 568 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 569 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 570 2015, . 572 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 573 Rabadan, "Virtual Private Wire Service Support in Ethernet 574 VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017, 575 . 577 Appendix A. Appendix 579 Authors' Addresses 581 Sudhin Jacob (editor) 582 Juniper Networks 583 Bangalore 584 India 586 Phone: +91 8061212543 587 Email: sjacob@juniper.net 589 Kishore Tiruveedhula 590 Juniper Networks 591 10 Technology Park Dr 592 Westford, MA 01886 593 USA 595 Phone: +1 9785898861 596 Email: kishoret@juniper.net