idnits 2.17.1 draft-vikjac-bmwg-evpnmultest-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 20 instances of too long lines in the document, the longest one being 15 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 27, 2020) is 1271 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2544' is defined on line 694, but no explicit reference was found in the text == Unused Reference: 'RFC2899' is defined on line 699, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 705, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. Jacob, Ed. 3 Internet-Draft V. Nagarajan 4 Intended status: Informational Juniper Networks 5 Expires: April 30, 2021 October 27, 2020 7 Benchmarking Methodology for EVPN Multicasting 8 draft-vikjac-bmwg-evpnmultest-05 10 Abstract 12 This document defines methodologies for benchmarking IGMP proxy 13 performance over EVPN-VXLAN. IGMP proxy over EVPN is defined in 14 draft-ietf-bess-evpn-IGMP-mld-proxy-02, and is being deployed in data 15 center networks. Specifically this document defines the 16 methodologies for benchmarking IGMP proxy convergence, leave latency 17 Scale,Core isolation, high availability and longevity. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on April 30, 2021. 36 Copyright Notice 38 Copyright (c) 2020 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 2 55 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 56 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 57 3. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 6 58 3.1. Learning Rate . . . . . . . . . . . . . . . . . . . . . . 6 59 3.2. Flush Rate . . . . . . . . . . . . . . . . . . . . . . . 7 60 3.3. Leave Latency . . . . . . . . . . . . . . . . . . . . . . 7 61 3.4. Join Latency . . . . . . . . . . . . . . . . . . . . . . 8 62 3.5. Leave Latency of N Vlans in DUT . . . . . . . . . . . . . 9 63 3.6. Join Latency of N vlans in DUT working EVPN AA mode . . . 9 64 3.7. Leave Latency of DUT operating in EVPN AA . . . . . . . . 10 65 3.8. Join Latency with reception of Type 6 route . . . . . . . 11 66 4. Link Flap . . . . . . . . . . . . . . . . . . . . . . . . . . 11 67 4.1. Packet Loss measurement in DUT due to CE link Failure . . 12 68 4.2. Core Link Failure in EVPN AA . . . . . . . . . . . . . . 12 69 4.3. Routing Failure in DUT operating in EVPN-VXLAN AA . . . . 13 70 5. High Availability . . . . . . . . . . . . . . . . . . . . . . 14 71 5.1. Routing Engine Fail over. . . . . . . . . . . . . . . . . 14 72 6. SOAK Test . . . . . . . . . . . . . . . . . . . . . . . . . . 14 73 6.1. Stability of the DUT with traffic. . . . . . . . . . . . 15 74 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15 75 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 76 9. Security Considerations . . . . . . . . . . . . . . . . . . . 15 77 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 78 10.1. Normative References . . . . . . . . . . . . . . . . . . 16 79 10.2. Informative References . . . . . . . . . . . . . . . . . 16 80 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 16 81 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 83 1. Introduction 85 IGMP proxy over EVPN-VXLAN is defined in draft-ietf-bess-evpn-IGMP- 86 mld-proxy-02,and is being deployed in data center networks. 87 Specifically this document defines the methodologies for benchmarking 88 IGMP proxy convergence,leave latency Scale,Core isolation, high 89 availability and longevity. 91 1.1. Requirements Language 93 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 94 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 95 document are to be interpreted as described in RFC 2119 [RFC2119]. 97 1.2. Terminologies 99 All-Active Redundancy Mode: When all PEs attached to an Ethernet 100 segment are allowed to forward known unicast traffic to/from that 101 Ethernet segment for a given VLAN, then the Ethernet segment is 102 defined to be operating in All-Active redundancy mode. 104 AA: All Active mode 106 CE: Customer Router/Devices/Switch. 108 DF: Designated Forwarder 110 DUT: Device under test. 112 EBGP: Exterior Border Gateway Protocol. 114 Ethernet Segment (ES): When a customer site (device or network) is 115 connected to one or more PEs via a set of Ethernet links, then that 116 set of links is referred to as an 'Ethernet segment'. 118 EVI: An EVPN instance spanning the leaf,spine devices participating 119 in that EVPN. 121 EVPN: Ethernet Virtual Private Network 123 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 124 identifies an Ethernet segment is called an 'Ethernet Segment 125 Identifier'. 127 Ethernet Tag: An Ethernet tag identifies a particular broadcast 128 domain, e.g., a VLAN. An EVPN instance consists of one or more 129 broadcast domains. 131 Interface: Physical interface of a router/switch. 133 IGMP: Internet Group Management Protocol 135 IBGP: Interior Border Gateway Protocol 137 IRB: Integrated routing and bridging interface 139 MAC: Media Access Control addresses on a PE. 141 MLD: Multicast Listener Discovery 143 NVO: Network Visualization Overlay 144 RT Traffic Generator. 146 Sub Interface Each physical Interfaces is subdivided into Logical 147 units. 149 SA Single Active 151 Single-Active Redundancy Mode: When only a single PE, among all the 152 PEs attached to an Ethernet segment, is allowed to forward traffic 153 to/from that Ethernet segment for a given VLAN, then the Ethernet 154 segment is defined to be operating in Single-Active redundancy mode. 156 VXLAN: Virtual Extensible LAN 158 2. Test Topology 160 There are six routers in the topology. Leaf1,leaf2, 161 leaf3,spine1,spine2 emulating a data center network. CE is a 162 customer device connected to leaf1 and leaf2,it is configured with 163 bridge domains in different vlans. The traffic generator is 164 connected to CE,leaf1,leaf2,leaf3,spine1 and spine 2 to emulate 165 multicast source and host generating IGMP join/leave. 167 Topology Diagram 168 RT 169 +--------------+ +--------------+ 170 RT | | | +-------------------+ 171 +---------+ spine1 +----------------------------------+ 172 | | | spine2 | | 173 | | | | | 174 +----+---------+---------+ +----+----+----+------------------+ 175 | | | | | | 176 | | | | | | 177 | | | | | | 178 | | | | | | 179 | | | | | | 180 RT +---------+-----+---------+---+-------+----++ +--+-----------+----+ 181 +-------------+ | | | | leaf3 | 182 ++ leaf1 | | leaf2 | | | 183 | DUT | | | | | 184 +----+----------+ +---+--------+----+ +--------+----------+ 185 | | | | 186 | | | | 187 | | | | 188 | | | | 189 +-+-----------------+------+ | | 190 | | +--+ RT ++ RT 191 | CE | RT 192 | +-------+ 193 +-------------------+ 194 CE connected to leaf1 and leaf2 in EVPN-VXLAN Active-Active mode. 196 Topology 1 198 Topology Diagram 200 Figure 1 202 Test Setup Configurations: 204 Leaf1, Leaf2,Leaf3 are configured with Exterior Border Gateway 205 protocol as the underlay protocol. The routes are advertised over 206 it. The EVPN signaling is enabled on it in order to have the overlay 207 reachability. Leaves are configured with "N" EVPN-VXLAN EVI's. CE 208 is multi homed to leaf1 and leaf2. The Interface connecting to the 209 CE is configured with ESI per interface or ESI per vlan. Leaf1 and 210 leaf2 are running EVPN-VXLAN AA mode to CE. 212 Spine1,spine2 are configured with Exterior Border Gateway protocol as 213 the underlay protocol. The routes are advertised over it. The EVPN 214 signaling is enabled over it to have the overlay reachability. 215 Spines are configured with "N" EVPN-VXLAN EVI's. Traffic generators 216 are connected spine1,spine2. Spine1 and Spine2 work as single home 217 EVPN-VXLAN EVI's. 219 CE is acting as bridge configured with multiple vlans,the same vlans 220 are configured on leaf1 and leaf2. Traffic generator is connected to 221 CE. The traffic generator acts as sender or receiver of traffic. 223 Depending up on the test scenarios the traffic generators will be 224 used to generate igmp membership report or multicast traffic. 226 The above configuration will be serving as the base configuration for 227 all test cases. 229 3. Test Cases 231 The following tests are conducted to measure the learning rate,leave 232 rate,leave latency of IGMP messages which propagates in leaf and 233 spine. 235 3.1. Learning Rate 237 Objective: 239 Measure the time taken to learn X1...Xn IGMP join generated by host/ 240 hosts. 242 Topology : Topology 1 244 Procedure: 246 Confirm the DUT is up and running with EVPN-VXLAN.Traffic generator 247 connected to leaf1 must send IGMP membership report for groups X1... 248 Xn to a vlan present in leaf1,leaf2 which is a part of EVPN-VLXAN 249 EVI.Measure the time taken to learn X1..Xn (*,G) entries in the DUT. 251 Measurement : 253 Measure the time taken by the DUT to learn the "X" IGMP membership 254 report. The test is repeated for "N" times and the values are 255 collected. The IGMP membership report learning rate is calculated by 256 averaging the values obtained from "N" samples. "N" is an arbitrary 257 number to get a sufficient sample. The time measured for each sample 258 is denoted by T1,T2...Tn.The measurement is carried out using 259 external server which polls the DUT using automated scripts. 261 Learning Rate = (T1+T2+..Tn)/N 263 3.2. Flush Rate 265 Objective: 267 Measure the time taken to Flush the X1... Xn (*,G) entries in DUT. 269 Topology : Topology 1 271 Procedure: 273 Confirm the DUT is up and running with EVPN-VXLAN.Traffic generator 274 connected to the leaf1 must send IGMP membership report for groups 275 X1... Xn to a vlan present in leaf1 which is a part of EVPN-VLXAN 276 EVI. Stop the membership report from traffic generator. Measure the 277 time taken to Flush X1..Xn (*,G) entries in the DUT. 279 Measurement : 281 Measure the time taken by the DUT to flush the "X" (*,G) entries The 282 test is repeated for "N" times and the values are collected. The 283 flush rate is calculated by averaging the values obtained from "N" 284 samples. "N" is an arbitrary number to get a sufficient sample. The 285 time measured for each sample is denoted by T1,T2...Tn.The 286 measurement is carried out using external server which polls the DUT 287 using automated scripts. 289 Flush Rate = (T1+T2+..Tn)/N 291 3.3. Leave Latency 293 Objective: 295 Measure the time taken by the DUT to stop forwarding the multicast 296 traffic during the receipt of IGMP leave from RT. 298 Topology : Topology 1 300 Procedure: 302 Confirm the DUT is up and running with EVPN-VXLAN.Traffic generator 303 connected to the leaf1 must send IGMP membership report for groups 304 X1... Xn to a vlan present in leaf1,leaf2 which is a part of EVPN- 305 VLXAN EVI. Send multicast traffic from the RT port connected to 306 spine1 to these groups requested by the leaf1. The leaf1 must 307 receives multicast traffic.Send the IGMP leave message from the 308 traffic generator to the leaf1. Measure the time taken by leaf1 to 309 Flush X1..Xn (*,G) entries and stop forwarding the multicast traffic 310 to RT. 312 Measurement : 314 Measure the time taken by the DUT to stop forwarding the multicast 315 traffic. The test is repeated for "N" times and the values are 316 collected. The leave latency is calculated by averaging the values 317 obtained from "N" samples. "N" is an arbitrary number to get a 318 sufficient sample.The time measured for each sample is denoted by 319 T1,T2...Tn.The measurement is carried out using external server which 320 polls the DUT using automated scripts. 322 Leave Latency = (T1+T2+..Tn)/N 324 3.4. Join Latency 326 Objective: 328 Measure the time taken by the DUT to create IGMP entries for N vlans. 330 Topology : Topology 1 332 Procedure: 334 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 335 reachability. The RT port connected to spine1 acts the source of the 336 multicast traffic. The RT port connected to leaf1 acts as receiver 337 of multicast traffic. Send IGMP membership report for groups X1...Xn 338 from RT port connected to leaf1. The leaf1 has N vlans subscribed to 339 these groups. Send multicast traffic from source.Measure the time 340 taken to forward the multicast traffic to the receiver. 342 Measurement : 344 Measure the time taken by the DUT to forward the multicast traffic to 345 these "N" vlans. The test is repeated for "N" times and the values 346 are collected. The join latency is calculated by averaging the 347 values obtained from "N" samples. "N" is an arbitrary number to get 348 a sufficient sample.The time measured for each sample is denoted by 349 T1,T2...Tn.The measurement is carried out using external server which 350 polls the DUT using automated scripts. 352 Join Latency = (T1+T2+..Tn)/N 354 3.5. Leave Latency of N Vlans in DUT 356 Objective: 358 To Record the time taken by the DUT to stop forwarding the multicast 359 traffic to N vlans during the receipt of IGMP leave messages from RT. 361 Topology : Topology 1 363 Procedure: 365 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 366 reachability. The RT port connected to spine1 acts the source of the 367 multicast traffic. The RT port connected to leaf1 acts as receiver 368 of multicast traffic. Send IGMP membership report for groups X1...Xn 369 from RT port connected to leaf1. The leaf1 has N vlans subscribed to 370 these groups. Send multicast traffic from source. Once the traffic 371 is in steady state, send IGMP leave message to these groups.Once the 372 leaf1 receiver the leave messages. it will flush the entries and 373 stop forwarding the traffic to the receiver. 375 Measurement : 377 Measure the time taken by the DUT to stop forwaring the multicast 378 traffic to these "N" vlans. The test is repeated for "N" times and 379 the values are collected. The leave latency is calculated by 380 averaging the values obtained from "N" samples. "N" is an arbitrary 381 number to get a sufficient sample.The time measured for each sample 382 is denoted by T1,T2...Tn.The measurement is carried out using 383 external server which polls the DUT using automated scripts. 385 Leave Latency = (T1+T2+..Tn)/N 387 3.6. Join Latency of N vlans in DUT working EVPN AA mode 389 Objective: 391 Measure the time taken to learn X1...Xn IGMP join generated by host/ 392 hosts located in N vlans in DUT operating in EVPN AA mode. 394 Topology : Topology 1 396 Procedure: 398 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 399 reachability. The RT port connected to spine1 acts the source of the 400 multicast traffic. The RT port connected to CE acts as receiver of 401 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 402 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 403 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 404 from X1...Xn. Send IGMP membership report to these groups from RT 405 connected to CE for these "N" vlans. Send multicast traffic from 406 source to these groups. Measure time taken by the EVPN DF to forward 407 the multicast traffic to the CE. 409 Measurement : 411 Measure the time taken by the EVPN DF to forward the multicast 412 traffic for "N" vlans. The test is repeated for "N" times and the 413 values are collected. The join latency is calculated by averaging 414 the values obtained from "N" samples. "N" is an arbitrary number to 415 get a sufficient sample.The time measured for each sample is denoted 416 by T1,T2...Tn.The measurement is carried out using external server 417 which polls the DUT using automated scripts. 419 Join Latency = (T1+T2+..Tn)/N 421 3.7. Leave Latency of DUT operating in EVPN AA 423 Objective: 425 Measure the time taken by the DUT to stop forwarding the multicast 426 traffic to N vlans during the receipt of IGMP leave messages from RT. 428 Topology : Topology 1 430 Procedure: 432 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 433 reachability. The RT port connected to spine1 acts the source of the 434 multicast traffic. The RT port connected to CE acts as receiver of 435 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 436 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 437 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 438 from X1...Xn. Send IGMP membership report to these groups from RT 439 connected to CE for these "N" vlans. Send multicast traffic from 440 source to these groups. Once traffic reaches steady state, send IGMP 441 leave from RT connected to CE. Measure the time taken by the EVPN DF 442 to stop forward the multicast traffic to the CE. 444 Measurement : 446 Measure the time taken by the EVPN DF to stop forward the multicast 447 traffic for "N" vlans. The test is repeated for "N" times and the 448 values are collected. The leave latency is calculated by averaging 449 the values obtained from "N" samples. "N" is an arbitrary number to 450 get a sufficient sample.The time measured for each sample is denoted 451 by T1,T2...Tn.The measurement is carried out using external server 452 which polls the DUT using automated scripts. 454 Leave Latency = (T1+T2+..Tn/N) 456 3.8. Join Latency with reception of Type 6 route 458 Objective: 460 Measure the time takes to forward the traffic by DUT after the 461 receipt of type 6 join from peer MHPE in same ESI. 463 Topology : Topology 1 465 Procedure: 467 Configure "N" EVPN-VXLAN in leaf1,leaf2,leaf3,spine1 and spine2.Leaf1 468 and leaf2 are connected to CE which are working in EVPN AA mode. 469 Configure N vlans in RT which are present in leaf1, then send IGMP 470 join messages from RT connected to CE for groups ranging from X1...Xn 471 to these vlans. The CE in turn forwards the IGMP messages to leaf2 472 operating in EVPN AA mode. leaf2 and leaf1 are working EVPN AA mode. 473 Leaf 2 will send the type 6 join to the DUT(leaf 1).Then send traffic 474 to these groups from spine1. Traffic flows from spine1 to CE. 475 Measure the time taken by DUT to forward the traffic after the 476 receipt of type 6 join from leaf1. 478 Measurement : 480 Measure the time taken by DUT to forward the multicast traffic 481 flowing towards RT. 483 Repeat these test and plot the data. The test is repeated for "N" 484 times and the values are collected. The time is calculated by 485 averaging the values obtained from "N" samples. 487 Time taken by DUT to forward the traffic towards RT in sec = 488 (T1+T2+..Tn/N) 490 4. Link Flap 491 4.1. Packet Loss measurement in DUT due to CE link Failure 493 Objective: 495 Measure the packet loss during the CE to DF(DUT) link failure. 497 Topology : Topology 1 499 Procedure: 501 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 502 reachability. The RT port connected to spine1 acts the source of the 503 multicast traffic. The RT port connected to CE acts as receiver of 504 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 505 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 506 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 507 from X1...Xn. Send IGMP membership report to these groups from RT 508 connected to CE for these "N" vlans. Send multicast traffic from 509 source to these groups. The DF is the leaf1(DUT).Disable the link 510 between DF and CE. Traffic switch to the new DF. Measure the loss 511 of the traffic. 513 Measurement : 515 Measure the packet loss duration during the link disable. The test 516 is repeated for "N" times and the values are collected. The packet 517 loss duration is calculated by averaging the values obtained from "N" 518 samples. "N" is an arbitrary number to get a sufficient sample.The 519 time measured for each sample is denoted by T1,T2...Tn. 521 Packet loss in sec = (T1+T2+..Tn)/N 523 4.2. Core Link Failure in EVPN AA 525 Objective: 527 Measure the packet loss during the DF core failure 529 Topology : Topology 1 531 Procedure: 533 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 534 reachability. The RT port connected to spine1 acts the source of the 535 multicast traffic. The RT port connected to CE acts as receiver of 536 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 537 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 538 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 539 from X1...Xn. Send IGMP membership report to these groups from RT 540 connected to CE for these "N" vlans. Send multicast traffic from 541 source to these groups. The DF is the leaf1(DUT).Disable all the 542 core links of DUT. Traffic switch to the new DF. Measure the loss 543 of the traffic. 545 Measurement : 547 Measure the packet loss duration during the core link disable. The 548 test is repeated for "N" times and the values are collected. The 549 packet loss duration is calculated by averaging the values obtained 550 from "N" samples. "N" is an arbitrary number to get a sufficient 551 sample.The time measured for each sample is denoted by T1,T2...Tn. 553 Packet loss in sec = (T1+T2+..Tn)/N 555 4.3. Routing Failure in DUT operating in EVPN-VXLAN AA 557 Objective: 559 Measure the packet loss during the DF routing failure 561 Topology : Topology 1 563 Procedure: 565 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 566 reachability. The RT port connected to spine1 acts the source of the 567 multicast traffic. The RT port connected to CE acts as receiver of 568 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 569 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 570 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 571 from X1...Xn. Send IGMP membership report to these groups from RT 572 connected to CE for these "N" vlans. Send multicast traffic from 573 source to these groups. The DF is the leaf1(DUT). Perform restart 574 routing DUT. Traffic switch to the new DF. Measure the loss of the 575 traffic. 577 Measurement : 579 Measure the packet loss duration during the routing failure in DUT. 580 The test is repeated for "N" times and the values are collected. The 581 packet loss duration is calculated by averaging the values obtained 582 from "N" samples. "N" is an arbitrary number to get a sufficient 583 sample.The time measured for each sample is denoted by T1,T2...Tn. 585 Packet loss in sec = (T1+T2+..Tn)/N 587 5. High Availability 589 5.1. Routing Engine Fail over. 591 Objective: 593 Measure traffic loss during routing engine failover. 595 Topology : Topology 3 597 Procedure: 599 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 600 reachability. The RT port connected to spine1 acts the source of the 601 multicast traffic. The RT port connected to CE acts as receiver of 602 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 603 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 604 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 605 from X1...Xn. Send IGMP membership report to these groups from RT 606 connected to CE for these "N" vlans. Send multicast traffic from 607 source to these groups. The DF is the leaf1(DUT). Perform routing 608 engine failover in DUT. Traffic switch to the new DF. Measure the 609 loss of the traffic. 611 Measurement : 613 The expectation of the test is 0 traffic loss with no change in the 614 DF role. DUT should not withdraw any routes.But in cases where the 615 DUT is not property synchronized between master and standby,due to 616 that packet loss are observed. In that scenario the packet loss is 617 measured.The test is repeated for "N" times and the values are 618 collected.The packet loss is calculated by averaging the values 619 obtained by "N" samples. 621 Packet loss in sec = (T1+T2+..Tn)/N 623 6. SOAK Test 625 This is measuring the performance of DUT running with scaled 626 configuration with traffic over a peroid of time "T'". In each 627 interval "t1" the parameters measured are CPU usage, memory usage, 628 crashes. 630 6.1. Stability of the DUT with traffic. 632 Objective: 634 Measure the stability of the DUT in a scaled environment with 635 traffic. 637 Topology : Topology 3 639 Procedure: 641 Confirm the DUT is up and running with EVPN-VXLAN. Ensure the route 642 reachability. The RT port connected to spine1 acts the source of the 643 multicast traffic. The RT port connected to CE acts as receiver of 644 multicast traffic. leaf1 and leaf2 are multi homed EVPN-VXLAN EVI's 645 running AA mode.The leaf1 and leaf2 have "N" vlans configured in 646 EVPN-VXLAN EVI's, these vlans subscribe to multicast groups ranging 647 from X1...Xn. Send IGMP membership report to these groups from RT 648 connected to CE for these "N" vlans. Send multicast traffic from 649 source to these groups. The DF is the leaf1(DUT). Traffic will be 650 forwarded to the CE by the DF. Run the traffic for "T" time 651 interval. 653 Measurement : 655 Take the hourly reading of CPU, process memory.There should not be 656 any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU 657 usage which shoots at 40 to 50 percent of the average usage. The 658 average value vary from device to device. Memory leak is determined 659 by increase usage of the memory for EVPN-VPWS process. The 660 expectation is under steady state the memory usage for EVPN- 661 VXLAN,IGMP processes should not increase. 663 7. Acknowledgments 665 We would like to thank Al and Sarah for the support. 667 8. IANA Considerations 669 This memo includes no request to IANA. 671 9. Security Considerations 673 The benchmarking tests described in this document are limited to the 674 performance characterization of controllers in a lab environment with 675 isolated networks. The benchmarking network topology will be an 676 independent test setup and MUST NOT be connected to devices that may 677 forward the test traffic into a production network or misroute 678 traffic to the test management network. Further, benchmarking is 679 performed on a "black-box" basis, relying solely on measurements 680 observable external to the controller. Special capabilities SHOULD 681 NOT exist in the controller specifically for benchmarking purposes. 682 Any implications for network security arising from the controller 683 SHOULD be identical in the lab and in production networks. 685 10. References 687 10.1. Normative References 689 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 690 Requirement Levels", BCP 14, RFC 2119, 691 DOI 10.17487/RFC2119, March 1997, 692 . 694 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 695 Network Interconnect Devices", RFC 2544, 696 DOI 10.17487/RFC2544, March 1999, 697 . 699 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 700 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, 701 . 703 10.2. Informative References 705 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 706 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 707 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 708 2015, . 710 Appendix A. Appendix 712 Authors' Addresses 714 Sudhin Jacob (editor) 715 Juniper Networks 716 Bangalore, Karnataka 560103 717 India 719 Phone: +91 8061212543 720 Email: sjacob@juniper.net 721 Vikram Nagarajan 722 Juniper Networks 723 Bangalore, Karnataka 560103 724 India 726 Phone: +91 8061212543 727 Email: vikramna@juniper.net