idnits 2.17.1 draft-kishjac-bmwg-evpnvpwstest-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 5 instances of too long lines in the document, the longest one being 62 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 202 has weird spacing: '... Tester sendi...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (June 17, 2019) is 1774 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC2544' is defined on line 554, but no explicit reference was found in the text == Unused Reference: 'RFC2899' is defined on line 559, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 565, but no explicit reference was found in the text == Unused Reference: 'RFC8214' is defined on line 570, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 2544 ** Downref: Normative reference to an Informational RFC: RFC 2899 Summary: 3 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. Jacob, Ed. 3 Internet-Draft K. Tiruveedhula 4 Intended status: Standards Track Juniper Networks 5 Expires: December 19, 2019 June 17, 2019 7 Benchmarking Methodology for EVPN VPWS 8 draft-kishjac-bmwg-evpnvpwstest-02 10 Abstract 12 This document defines methodologies for benchmarking EVPN-VPWS 13 performance. EVPN-VPWS is defined in RFC 8214, and is being deployed 14 in Service Provider networks. Specifically this document defines the 15 methodologies for benchmarking EVPN-VPWS Scale convergence, 16 Scale,Core isolation, high availability and longevity. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on December 19, 2019. 35 Copyright Notice 37 Copyright (c) 2019 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 54 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3 56 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 7 57 3.1. How long it takes to switch from primary to backup during 58 local link failure . . . . . . . . . . . . . . . . . . . 7 59 3.2. How long it takes to remote PE to switch traffic from 60 primary to back up path during link failure in CE . . . . 8 61 3.3. How long it takes to remote PE to switch traffic from 62 primary to back up path during core failure . . . . . . . 8 63 3.4. How long it takes to primary PE to regain control after 64 the local link flap . . . . . . . . . . . . . . . . . . . 9 65 4. Activate/deactivate AC's . . . . . . . . . . . . . . . . . . 10 66 4.1. To Add M number of attachment circuits. . . . . . . . . 10 67 4.2. Deactivate/Activate M number of attachment circuits. . . 10 68 5. Scale Convergence . . . . . . . . . . . . . . . . . . . . . . 11 69 5.1. To measure the packet loss during the core link failure. 11 70 6. High Availability . . . . . . . . . . . . . . . . . . . . . . 11 71 6.1. To Record the whether there is traffic loss due to 72 routing engine failover for redundancy test. . . . . . . 12 73 7. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 12 74 7.1. To Measure the stability of the DUT with scale and 75 traffic. . . . . . . . . . . . . . . . . . . . . . . . . 12 76 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13 77 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 78 10. Security Considerations . . . . . . . . . . . . . . . . . . . 13 79 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 80 11.1. Normative References . . . . . . . . . . . . . . . . . . 13 81 11.2. Informative References . . . . . . . . . . . . . . . . . 13 82 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 14 83 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 85 1. Introduction 87 EVPN-VPWS is defined in RFC 8214,discusses how VPWS can be combined 88 with EVPNs to provide a new/combined solution. This draft defines 89 methodologies that can be used to benchmark RFC 8214 solutions. 90 Further, this draft provides methodologies for benchmarking the 91 performance of EVPN VPWS Scale,Scale Convergence, Core isolation, 92 longevity, high availability. 94 1.1. Requirements Language 96 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 97 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 98 document are to be interpreted as described in RFC 2119 [RFC2119]. 100 1.2. Terminologies 102 MHPE Multi homed Provide Edge router. 104 RR Route Reflector. 106 P Provider Router. 108 CE Customer Router/Devices/Switch. 110 MHPE2 Multi homed Provider Edge router 2. 112 MHPE1 Multi homed Provider Edge router 1. 114 SHPE3 Single homed Provider Edge Router 3. 116 AA EVPN Terminologies AA All-Active. 118 AC Attachment Circuit( customer EVPN-VPWS Service over the Provider 119 network 121 SA EVPN Terminologies SA Single-Active. 123 RT Router Tester. 125 Sub Interface Each physical Interfaces is subdivided in to Logical 126 units. 128 EVI EVPN Instances which will be running on sub interface or physical 129 port of the provider Edge routers. 131 DF Designated Forwarder. 133 ESI Ethernet Segment Identifier. 135 2. Test Topology 137 EVPN-VPWS Services running on SHPE3, MHPE1 and MHPE2 in Single Active 138 Mode: 140 Topology Diagram 142 | [Traffic Generator ] Router Tester traffic receiver for layer 2 traffic from CE 143 +----------+ 144 | | 145 | SHPE3 | 146 | SHPE3 | 147 +----------+ 148 | 149 |Core link 150 +----------+ 151 | | 152 | RR | 153 | | Route Reflector/Core router 154 +----------+-------------| 155 | | 156 | Core links | 157 +----------+ +-----------+ 158 | | | MHPE2 | 159 | DUT | | | 160 | MHPE1 | | | 161 +----------+ +-----------+ 162 | PE-CE link | 163 +----------+------------ 164 | | 165 | CE | 166 | layer2 | 167 |bridge | 168 +----------+------------ [Traffic Generator](Router Tester sending layer 2 traffic with different VLAN ) 170 Topology 1 172 | [Traffic Generator ] Router Tester sending layer 2 traffic. 174 +----------+ 175 | | 176 | SHPE3 | 177 | SHPE3 | 178 +----------+ 179 | 180 |Core link 181 +----------+ 182 | | 183 | RR | 184 | | Route Reflector/Core router 185 +----------+-------------| 186 | | 187 | Core links | 188 +----------+ +-----------+ 189 | | | MHPE2 | 190 | DUT | | | 191 | MHPE1 | | | 192 +----------+ +-----------+ 193 | PE-CE link | 194 +----------+------------ 195 | | 196 | CE | 197 | layer2 | 198 |bridge | 199 +----------+------------ [Traffic Generator](Router Tester receiver for layer 2 traffic with different vlans.) 201 Topology 2 202 | [Traffic Generator ] Router Tester sending layer 2 bi directional traffic sender/receiver 203 +----------+ 204 | | 205 | SHPE3 | 206 | SHPE3 | 207 +----------+ 208 | 209 |Core link 210 +----------+ 211 | | 212 | RR | 213 | | Route Reflector/Core router 214 +----------+-------------| 215 | | 216 | Core links | 217 +----------+ +-----------+ 218 | | | MHPE2 | 219 | DUT | | | 220 | MHPE1 | | | 221 +----------+ +-----------+ 222 | PE-CE link | 223 +----------+------------ 224 | | 225 | CE | 226 | layer2 | 227 |bridge | 228 +----------+------------ [Traffic Generator](Router Tester sending bi directional layer 2 traffic with different VLAN sender/receiver) 230 Topology 3 232 Topology Diagram 234 Figure 1 236 There are five routers in the topology. SHPE3, RR/P, MHPE1 and MHPE2 237 emulating a service provider network. CE is a customer device 238 connected to MHPE1 and MHPE2, it is configured with bridge domains in 239 different vlans. The router tester is connected to CE and SHPE3.The 240 MHPE1 acts as DUT.The RT will act as sender and receiver.The 241 measurement will be taken in DUT. 243 All routers except CE is configured with OSPF/IS-IS,LDP,MPLS,BGP with 244 EVPN address family. 246 All routers except CE must have IBGP configured with RR acting as 247 route reflector. 249 MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN-VPWS instances 250 depends up on the cases. 252 MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on IFD. 254 MHPE1 and MHEPE2 are running Single Active mode of EVPN-VPWS. 256 CE is acting as bridge configured with vlans that is configured on 257 MHPE1,MHPE2,SHPE3. 259 Depends up on the test traffic will be flowing uni directional or bi 260 directional depends on the topology mentioned above. 262 The above configuration will serve as base configuration for all the 263 test cases. 265 3. Test Cases 267 The following tests are conducted to measure the packet loss during 268 the local link and core failure in DUT with Scaled AC's. 270 3.1. How long it takes to switch from primary to backup during local 271 link failure 273 Objective: 275 To Record the time taken to switch from primary to backup during 276 local link failure. 278 Topology : Topology 1 280 Procedure: 282 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 283 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 284 packets from CE to MHPE2 AC's working in SA. Then shut the MHPE2-CE 285 link, so that traffic from CE switches to DUT. 287 Measurement : 289 Measure the time taken to switch the traffic from active to backup, 290 the traffic will flow from MHPE1 to SHPE3. Measure the time taken to 291 switch the traffic. 293 Repeat these test and plot the data. The test is repeated for "N" 294 times and the values are collected. The switching time is calculated 295 by averaging the values obtained from "N" samples. 297 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 299 3.2. How long it takes to remote PE to switch traffic from primary to 300 back up path during link failure in CE 302 Objective: 304 To Record the time taken by remote PE to switch traffic from primary 305 to backup during CE link failure. 307 Topology : Topology 2 309 Procedure: 311 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 312 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 313 packets from RT to SHPE3 Ac's.Then shut the MHPE2-CE link, this 314 failure will be notified to remote PE and traffic switch to backup 315 path. 317 Measurement : 319 Measure the time taken to switch the traffic from active to backup, 320 the traffic will flow from SHPE3 to MHPE1. Measure the time taken to 321 switch the traffic. 323 Repeat these test and plot the data. The test is repeated for "N" 324 times and the values are collected. The switching time is calculated 325 by averaging the values obtained from "N" samples. 327 AC's switch over from primary to backup PE in sec = (T1+T2+..Tn/N) 329 3.3. How long it takes to remote PE to switch traffic from primary to 330 back up path during core failure 332 Objective: 334 To Record the time taken by remote PE to switch traffic from primary 335 to backup during core link failure. 337 Topology : Topology 2 339 Procedure: 341 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 342 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 343 packets from RT to SHPE3 Ac's.Then shut the core link of MHPE2,this 344 failure will be notified to remote PE and traffic switch to backup 345 path. 347 Measurement : 349 Measure the time taken to switch the traffic from active to backup, 350 the traffic will flow from SHPE3 to MHPE1. Measure the time taken to 351 switch the traffic. 353 Repeat these test and plot the data. The test is repeated for "N" 354 times and the values are collected. The switching time is calculated 355 by averaging the values obtained from "N" samples. 357 AC's in remote PE switches from primary to backup PE in sec due to 358 core failure = (T1+T2+..Tn/N) 360 3.4. How long it takes to primary PE to regain control after the local 361 link flap 363 Objective: 365 To Record the time taken by primary PE to regain control after the 366 local PE-CE link flap. 368 Topology : Topology 1 370 Procedure: 372 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 373 mode.Ensure MHPE2 is standby and DUT is primary PE.Send "X" unicast 374 packets from CE to all Ac's in MHPE1(DUT).Then shut the link of 375 MHPE1-CE,this failure will be notified to remote PE and traffic 376 switch to backup path. Then bring up the link of MHPE1-CE.Now the 377 traffic switches to DUT. 379 Measurement : 381 Measure the time taken to switch the traffic from MHPE2 to DUT, the 382 traffic will flow from MHPE1 to SHPE3. Measure the time taken to 383 switch the traffic. 385 Repeat these test and plot the data. The test is repeated for "N" 386 times and the values are collected. The switching time is calculated 387 by averaging the values obtained from "N" samples. 389 Time taken to switch back to primary(DUT) once the link is restored = 390 (T1+T2+..Tn/N) 392 4. Activate/deactivate AC's 394 4.1. To Add M number of attachment circuits. 396 Objective: 398 To measure the performance of the DUT while adding M AC's on the fly. 400 Topology : Topology 3 402 Procedure: 404 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 405 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 406 packets from RT to SHPE3 to all Ac's and send "X" unicast packets 407 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 must be 408 standby. DUT will be forwarding the traffic to CE from SHPE3 and the 409 traffic from CE to SHPE3.Then add "M" AC's on SHPE1,DUT and MHPE2 on 410 the fly. these AC' must be in SA mode. 412 Measurement : 414 There should be 0 traffic loss in existing services while addition of 415 these ACs. 417 4.2. Deactivate/Activate M number of attachment circuits. 419 Objective: 421 To measure the performance of the DUT while deactivating/activating 422 AC's. 424 Topology : Topology 3 426 Procedure: 428 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 429 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 430 packets from RT to SHPE3 to all Ac's and send "X" unicast packets 431 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 must be 432 standby.DUT will be forwarding the traffic to CE and from CE to 433 SHPE3.Then deactivate "M" AC's on SHPE1,DUT and MHPE2 on the fly. 434 these AC' must be removed from forwarding plane. Stop the traffic 435 for these AC's. Activate the AC's in all PE's. then start the 436 traffic, measure the time taken by "M" AC's to forward the traffic. 438 Measurement : 440 Measure the packet loss in sec during this deactivating/activating 441 AC's.Repeat the test "N" times and plot the data.The packet loss is 442 calculated by averaging the values obtained from "N" samples. 444 packet loss in sec = (T1+T2+..Tn/N) 446 5. Scale Convergence 448 5.1. To measure the packet loss during the core link failure. 450 Objective: 452 To Measure the convergence at a higher number of AC's 454 Topology : Topology 3 456 Procedure: 458 Configure "N'" AC's in SHPE3 and MHPE1,MHPE2, working in SA mode.The 459 scale factor must be in the multiples of thousands. DF election must 460 be priority based not on the default RFC 7432, it should not be MOD 461 based DF election. Send "X" unicast packets from RT to SHPE3 to all 462 Ac's and send "X" unicast packets from CE to MHPE1(DUT), let the DUT 463 is the active and the MHPE2 must be standby. DUT will be forwarding 464 the traffic to CE from SHPE3 and from CE to SHPE3.Then flap the core 465 link of the DUT. 467 Measurement : 469 Measure the packet loss in seconds once the core link is 470 restored.Repeat the test "N" times and plot the data.The packet loss 471 is calculated by averaging the values obtained from "N" samples. 473 Packet loss in sec = (T1+T2+..Tn/N) 475 6. High Availability 476 6.1. To Record the whether there is traffic loss due to routing engine 477 failover for redundancy test. 479 Objective: 481 To record traffic loss during routing engine failover. 483 Topology : Topology 3 485 Procedure: 487 Configure "N" AC's in SHPE3 and MHPE1,MHPE2, working in SA 488 mode.Ensure MHPE2 is active and DUT is backup PE.Send "X" unicast 489 packets from RT to SHPE3 to all Ac's and send "X" unicast packets 490 from CE to MHPE1(DUT),let the DUT is the active and the MHPE2 must be 491 standby. DUT will be forwarding the traffic to CE and from CE to 492 SHPE3.Then do a routing engine fail-over. 494 Measurement : 496 There should be 0 traffic loss which is the ideal case, No change in 497 the DF role. DUT should not withdraw any routes.Repeat the test "N" 498 times and plot the data.The packet loss is calculated by averaging 499 the values obtained from "N" samples. 501 Packet loss in sec = (T1+T2+..Tn/N) 503 7. SOAK Test 505 This is measuring the performance of DUT running with scaled 506 configuration with traffic over a peroid of time "T'". In each 507 interval "t1" the parameters measured are CPU usage, memory usage, 508 crashes. 510 7.1. To Measure the stability of the DUT with scale and traffic. 512 Objective: 514 To measure the stability of the DUT in a scaled environment with 515 traffic. 517 Topology : Topology 3 519 Procedure: 521 Scale N AC's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE 522 using traffic generator with different X SA and DA for N EVI's. Send 523 F frames from traffic generator to SHPE3 with X different SA and DA. 524 There is a bi directional traffic flow with F pps in each direction. 525 The DUT must run with traffic for 24 hours, every hour check for 526 memory leak, crash. 528 Measurement : 530 Take the hourly reading of CPU, process memory. There should not be 531 any leak, crashes, CPU spikes. 533 8. Acknowledgements 535 We would like to thank Al and Sarah for the support. 537 9. IANA Considerations 539 This memo includes no request to IANA. 541 10. Security Considerations 543 There is no additional consideration from RFC 6192. 545 11. References 547 11.1. Normative References 549 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 550 Requirement Levels", BCP 14, RFC 2119, 551 DOI 10.17487/RFC2119, March 1997, 552 . 554 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 555 Network Interconnect Devices", RFC 2544, 556 DOI 10.17487/RFC2544, March 1999, 557 . 559 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 560 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, 561 . 563 11.2. Informative References 565 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 566 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 567 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 568 2015, . 570 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 571 Rabadan, "Virtual Private Wire Service Support in Ethernet 572 VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017, 573 . 575 Appendix A. Appendix 577 Authors' Addresses 579 Sudhin Jacob (editor) 580 Juniper Networks 581 Bangalore 582 India 584 Phone: +91 8061212543 585 Email: sjacob@juniper.net 587 Kishore Tiruveedhula 588 Juniper Networks 589 10 Technology Park Dr 590 Westford, MA 01886 591 USA 593 Phone: +1 9785898861 594 Email: kishoret@juniper.net