idnits 2.17.1 draft-ietf-bmwg-evpntest-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 35 instances of too long lines in the document, the longest one being 61 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (December 18, 2019) is 1581 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC8174' is mentioned on line 106, but not defined == Unused Reference: 'RFC2119' is defined on line 934, but no explicit reference was found in the text == Unused Reference: 'RFC2544' is defined on line 939, but no explicit reference was found in the text == Unused Reference: 'RFC2899' is defined on line 944, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 950, but no explicit reference was found in the text == Unused Reference: 'RFC7623' is defined on line 955, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force S. Jacob, Ed. 3 Internet-Draft K. Tiruveedhula 4 Intended status: Informational Juniper Networks 5 Expires: June 20, 2020 December 18, 2019 7 Benchmarking Methodology for EVPN and PBB-EVPN 8 draft-ietf-bmwg-evpntest-04 10 Abstract 12 This document defines methodologies for benchmarking EVPN and PBB- 13 EVPN performance.EVPN is defined in RFC 7432, and is being deployed 14 in Service Provider networks.Specifically, this document defines 15 the methodologies for benchmarking EVPN/PBB-EVPN convergence, data 16 plane and control plane performance. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on June 20, 2020. 35 Copyright Notice 37 Copyright (c) 2019 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 54 1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 4 56 3. Test Cases for EVPN Benchmarking . . . . . . . . . . . . . . 7 57 3.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 7 58 3.2. Remote MAC Learning . . . . . . . . . . . . . . . . . . . 8 59 3.3. MAC Flush due to local link failure and Relearning . . . 8 60 3.4. MAC Flush due to remote link failure. . . . . . . . . . . 9 61 3.5. MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . 9 62 3.6. Remote Mac Aging . . . . . . . . . . . . . . . . . . . . 10 63 3.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 10 64 3.8. High Availability. . . . . . . . . . . . . . . . . . . . 11 65 3.9. ARP/ND Scale . . . . . . . . . . . . . . . . . . . . . . 12 66 3.10. Scaling of Services . . . . . . . . . . . . . . . . . . . 12 67 3.11. Scale Convergence . . . . . . . . . . . . . . . . . . . . 13 68 3.12. SOAK Test. . . . . . . . . . . . . . . . . . . . . . . . 14 69 4. Test Cases for PBB-EVPN Benchmarking . . . . . . . . . . . . 14 70 4.1. Local MAC Learning . . . . . . . . . . . . . . . . . . . 14 71 4.2. Remote Mac Learning . . . . . . . . . . . . . . . . . . . 15 72 4.3. MAC Flush due to link failure . . . . . . . . . . . . . . 15 73 4.4. MAC Flush due to remote Failure . . . . . . . . . . . . . 16 74 4.5. MAC aging . . . . . . . . . . . . . . . . . . . . . . . . 17 75 4.6. Remote MAC Aging. . . . . . . . . . . . . . . . . . . . . 17 76 4.7. Local and Remote MAC Learning . . . . . . . . . . . . . . 18 77 4.8. High Availability . . . . . . . . . . . . . . . . . . . . 18 78 4.9. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 19 79 4.10. Scale Convergence . . . . . . . . . . . . . . . . . . . . 19 80 4.11. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 20 81 5. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 21 82 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 83 7. Security Considerations . . . . . . . . . . . . . . . . . . . 21 84 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 85 8.1. Normative References . . . . . . . . . . . . . . . . . . 21 86 8.2. Informative References . . . . . . . . . . . . . . . . . 22 87 Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 22 88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 90 1. Introduction 92 EVPN is defined in RFC 7432, and describes BGP MPLS- based Ethernet 93 VPNs (EVPN).PBB-EVPN is defined in RFC 7623, discusses how Ethernet 94 Provider backbone Bridging can be combined with EVPNs to provide a 95 new/combined solution.This draft defines methodologies that can be 96 used to benchmark both RFC 7432 and RFC 7623 solutions.Further, 97 this draft provides methodologies for benchmarking the performance of 98 EVPN data and control planes, MAC learning, MAC flushing, MAC aging, 99 convergence, high availability, and scale. 101 1.1. Requirements Language 103 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 104 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 105 document are to be interpreted as described in RFC 8174 [RFC8174]. 107 1.2. Terminologies 109 All-Active Redundancy Mode: When all PEs attached to an Ethernet 110 segment are allowed to forward known unicast traffic to/from that 111 Ethernet segment for a given VLAN, then the Ethernet segment is 112 defined to be operating in All-Active redundancy mode. 114 AA All Active mode 116 CE Customer Router/Devices/Switch. 118 DF Designated Forwarder 120 DUT Device under test. 122 Ethernet Segment (ES): When a customer site (device or network) is 123 connected to one or more PEs via a set of Ethernet links, then that 124 set of links is referred to as an 'Ethernet segment'. 126 EVI: An EVPN instance spanning the Provider Edge (PE) devices 127 participating in that EVPN. 129 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 130 identifies an Ethernet segment is called an 'Ethernet Segment 131 Identifier'. 133 Ethernet Tag: An Ethernet tag identifies a particular broadcast 134 domain, e.g., a VLAN. An EVPN instance consists of one or more 135 broadcast domains. 137 Interface Physical interface of a router/switch. 139 IRB Integrated routing and bridging interface 141 MAC Media Access Control addresses on a PE. 143 MHPE2 Multi homed Provider Edge router 2. 145 MHPE1 Multi homed Provider Edge router 1. 147 SHPE3 Single homed Provider Edge Router 3. 149 PE: Provider Edge device. 151 P Provider Router. 153 RR Route Reflector. 155 RT Traffic Generator. 157 Sub Interface Each physical Interfaces is subdivided into Logical 158 units. 160 SA Single Active 162 Single-Active Redundancy Mode: When only a single PE, among all the 163 PEs attached to an Ethernet segment, is allowed to forward traffic 164 to/from that Ethernet segment for a given VLAN, then the Ethernet 165 segment is defined to be operating in Single-Active redundancy mode. 167 2. Test Topology 169 EVPN/PBB-EVPN Services running on SHPE3, MHPE1 and MHPE2 in Single 170 Active Mode: 172 | Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. 173 +----------+ 174 | | 175 | SHPE3 | 176 | | 177 +----------+ 178 | 179 |Core link 180 +----------+ 181 | | 182 | RR | 183 | | Route Reflector /Provider router 184 +----------+-------------| 185 | | 186 | Core links | 187 +----------+ +-----------+ 188 | | | MHPE2 | 189 | DUT | | | 190 | MHPE1 | | | 191 +----------+ +-----------+ 192 | PE-CE link | 193 +----------+------------ 194 | | 195 | CE | 196 | layer2 | 197 |bridge | 198 +----------+------------ Traffic Generator acts as a sender/receiver of layer 2 traffic with multiple vlan. 200 Topology 1 202 Test Setup 204 Figure 1 206 +-----------------+---------------------+---------------------+---------------------+----------------------+-----------------------+ 207 | | | | | | | 208 | | | | | | | 209 | | | | | | | 210 | | | | | | | 211 | Mode | | | |Receiver | | 212 | | Test |Traffic Direction |Sender | | | 213 | | | | | | | 214 | | | | | | | 215 | | | | | | | 216 +----------------------------------------------------------------------------------------------------------------------------------+ 217 | | | | | | | 218 | | | | | SHPE3 | | 219 |Single Active | Local Mac | |CE | |Layer 2 traffic | 220 | | Learning | Uni | | | | 221 | | | | | | multiple MAC | 222 | | | | | | | 223 +-----------------------------------------------------------------------------------------------------------------------------------+ 224 | | | | | | | 225 |Single Active | Remote MAC | | | CE |Layer 2 traffic | 226 | | Learning | uni | SHPE3 | | | 227 | | | | | |multiple MAC | 228 | | | | | | ++ 229 +----------------------------------------------------------------------------------------------------------------------------------+ 230 | | | | | | | 231 |Single Active | Scale Convergence | Bi | | CE/SHPE3 | | 232 | | | | CE/SHPE3 | |Layer 2 traffic | 233 | | Local& Remote | | | |multiple mac& vlans | 234 | | Learning | | | | | 235 +-----------------+---------------------+---------------------+--------------------------------------------+-----------------------+ 236 | 237 ++ 239 Table showing Traffic pattern for various test 241 Figure 2 243 Test Setup Configurations: 245 There are five routers in the Test setup.SHPE3, RR/P, MHPE1 and 246 MHPE2 emulating a service provider network. CE is a customer device 247 connected to MHPE1 and MHPE2, it is configured with bridge domains in 248 multiple vlans.The traffic generator is connected to CE and 249 SHPE3.The MHPE1 acts as DUT.The traffic generator will be used as 250 sender and receiver of traffic.The measurement will be taken in DUT. 252 All routers except CE are configured with OSPF/IS-IS,LDP,MPLS,BGP 253 with EVPN address family. 255 All routers except CE are configured with Interior border gateway 256 protocol,RR acting as route reflector. 258 MHPE1,MHPE2,SHPE3 must be configured with "N" EVPN/PBB-EVPN instances 259 depends up on the cases. 261 MHPE1 and MHEPE2 must be configured with ESI per vlan or ESI on 262 interface. 264 MHPE1 and MHEPE2 are running Single Active mode of EVPN. 266 CE is acting as bridge configured with multiple vlans,the same vlans 267 are configured on MHPE1,MHPE2,SHPE3. 269 Depending up on the test scenarios the traffic generators will be 270 used to generate uni directional or bi directional flows. 272 The above configuration will be serving as the base configuration for 273 all test cases. 275 3. Test Cases for EVPN Benchmarking 277 3.1. Local MAC Learning 279 Objective: 281 To Record the time taken to learn the MAC address locally in DUT. 283 Topology : Topology 1 285 Procedure: 287 The data plane MAC learning can be measured using the parameters 288 defined in RFC 2889 section 5.8.Send "X" unicast frames from CE to 289 MHPE1(DUT) working in SA mode.Traffic generator connected to CE must 290 send frames with "X" different source and destination MAC address. 291 The DUT must learn these "X" macs in data plane. 293 Measurement : 295 Measure the time taken to learn "X" MACs locally in DUT evpn mac 296 table. The data plane measurement is taken by considering DUT as 297 black box. The range of MACs are known from traffic generator,the 298 same must be learned in DUT, the time taken to learn "X" macs is 299 measured. 301 The test is repeated for "N" times and the values are collected.The 302 mac learning rate is calculated by averaging the values obtained from 303 "N" samples. 305 Mac learning rate = (T1+T2+..Tn)/N 307 3.2. Remote MAC Learning 309 Objective: 311 To Record the time taken to learn the remote macs. 313 Topology : Topology 1 315 Procedure: 317 Send X frames with X different source and destination mac addresses 318 to SHPE3 from traffic generator.SHPE3 will advertise these locally 319 learned macs to MHPE1 and MHPE2 via control plane.Measure the time 320 taken to learn these X MACs from remote peer in DUT EVPN MAC address 321 table.The DUT and MHPE2 are running SA mode. 323 Measurement : 325 Measure the time taken by the DUT to learn the "X" MACs in the data 326 plane.The test is repeated for "N" times and the values are 327 collected. The remote mac learning rate is calculated by averaging 328 the values obtained from "N" samples. 330 Remote mac learning rate = (T1+T2+..Tn)/N 332 3.3. MAC Flush due to local link failure and Relearning 334 Objective: 336 To record the time taken to flush the locally learned mac and the 337 time taken to relearn the same amount of macs. 339 Topology : Topology 1 341 Procedure: 343 Send X frames with X different source and destination mac addresses 344 to DUT from CE using traffic generator. Wait till the MHPE1 learns 345 all X MAC addresses. Then fail the MHPE1 CE link and measure the 346 time taken to flush these X MACs from the EVPN MAC table. Bring up 347 the link which was made Down(the link between MHPE1 and CE).Measure 348 time taken to relearn it.The DUT and MHPE2 are running SA mode. 350 Measurement : 352 Measure the time taken for flushing these X MAC addresses.Measure 353 the time taken to relearn the X MACs in DUT.The test is repeated for 354 "N" times and the values are collected.The flush and the relearning 355 time is calculated by averaging the values obtained by "N" samples. 357 Flush rate = (T1+T2+..Tn)/N 359 Relearning rate = (T1+T2+..Tn)/N 361 3.4. MAC Flush due to remote link failure. 363 Objective: 365 To record the time taken to flush the remote mac learned in DUT 366 during remote link failure. 368 Topology : Topology 1 370 Procedure: 372 Send X frames with X different source and destination mac addresses 373 to DUT from SHPE3 using traffic generator. Bring down the link 374 between SHPE3 and traffic generator.Then measure the time taken to 375 flush the DUT EVPN MAC table.The DUT and MHPE2 are running SA mode. 377 Measurement : 379 Measure the time taken to flush X remote MACs from EVPN MAC table of 380 the DUT.The test is repeated for "N" times and the values are 381 collected.The flush rate is calculated by averaging the values 382 obtained by "N" samples. 384 Flush rate = (T1+T2+..Tn)/N 386 3.5. MAC Aging 388 Objective: 390 To measure the mac aging time. 392 Topology : Topology 1 393 Procedure: 395 Send X frames with X different source and destination mac addresses 396 to DUT from CE using traffic generator.Wait till X MAC addresses 397 are learned. Then stop the traffic.Record the time taken to flush 398 X MACS from DUT EVPN MAC table due to aging.The DUT and MHPE2 are 399 running SA mode. 401 Measurement : 403 Measure the time taken to flush X MAC addresses due to aging.The 404 test is repeated for "N" times and the values are collected.The 405 aging is calculated averaging the values obtained by "N" samples. 407 Aging time for X Macs in sec = (T1+T2+..Tn)/N 409 3.6. Remote Mac Aging 411 Objective: 413 To measure the remote mac aging time. 415 Topology : Topology 1 417 Procedure: 419 Send X frames with X different source and destination mac addresses 420 to DUT from SHPE3 using traffic generator. Stop the traffic at 421 remote PE SHPE3.Due to MAC aging SHPE3 will withdraw its routes from 422 DUT and MHPE2.Measure the time taken to remove these MACs from DUT 423 EVPN MAC table.DUT and MHPE2 are running in SA mode 425 Measurement : 427 Measure the time taken to flush X remote MACs learned in DUT EVPN MAC 428 table due to aging.The test is repeated for "N" times and the values 429 are collected.The aging is calculated by averaging the values 430 obtained by "N" samples. 432 Aging time for X Macs in sec = (T1+T2+..Tn)/N 434 3.7. Local and Remote MAC Learning 436 Objective: 438 To record the time taken to learn both local and remote macs. 440 Topology : Topology 1 442 Procedure: 444 Send X frames with X different source and destination mac addresses 445 to DUT from SHPE3 using traffic generator.Send X frames with 446 different source and destination mac addresses from traffic generator 447 connected to CE.The source and destination addresses of flows must 448 be complimentary to have unicast flows.Measure the time taken by 449 the DUT to learn 2X in EVPN MAC.DUT and MHPE2 are running in SA 450 mode. 452 Measurement : 454 Measure the time taken to learn 2X MAC addresses in DUT EVPN MAC 455 table. The test is repeated for "N" times and the values are 456 collected. The mac learning time is calculated by averaging the 457 values obtained by "N" samples. 459 Mac learning rate = (T1+T2+..Tn)/N 461 3.8. High Availability. 463 Objective: 465 To record traffic loss during routing engine fail over. 467 Topology : Topology 1 469 Procedure: 471 Send X frames from CE to DUT from traffic generator with X different 472 source and destination mac addresses.Send X frames from traffic 473 generator to SHPE3 with X different source and destination mac 474 addresses, so that 2X MAC address will be learned in the DUT.There 475 is a bi directional traffic flow with X pps in each direction.Then 476 do a routing engine fail-over. 478 Measurement : 480 The expectation of the test is 0 traffic loss with no change in the 481 DF role.DUT should not withdraw any routes.But in cases where the 482 DUT is not property synchronized between master and standby,due to 483 that packet loss are observed.In that scenario the packet loss is 484 measured.The test is repeated for "N" times and the values are 485 collected.The packet loss is calculated by averaging the values 486 obtained by "N" samples. 488 Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N 490 3.9. ARP/ND Scale 492 To Record the DUT scaling limit of ARP/ND. 494 Objective: 496 To Record the ARP/ND scale of the DUT. 498 Topology : Topology 1 500 Procedure: 502 Send X arp/neighbor discovery(ND) from the traffic generator to DUT 503 with different sender ip/ipv6,mac addresses to the target IRB address 504 configured in EVPN instance. The EVPN instance learns the mac+ip and 505 mac+ipv6 addresses from these request and advertise as type 2 mac+ip/ 506 mac+ipv6 route to remote provide edge routers which have same EVPN 507 configurations.The value of X must be increased at a incremental 508 value of 5% of X, till the limit is reached.The limit is where the 509 DUT cant learn any more type 2 mac+ip/mac+ipv6.The test must be 510 separately conducted for arp and ND. 512 Measurement : 514 Measure the scale limit of type 2 mac+ip/mac+ipv6 route which DUT can 515 learn.The test is repeated for "N" times and the values are 516 collected.The scale limit is calculated by averaging the values 517 obtained by "N" samples for both mac+ip and mac+ipv6. 519 DUT scale limit for mac+ip = (v1+v2+..vn)/N 521 DUT scale limit for mac+ipv6 = (v1+v2+..vn)/N 523 3.10. Scaling of Services 525 Objective: 527 This test is to measure the scale of EVPN instances that a DUT can 528 hold. 530 Topology : Topology 1 531 Procedure: 533 The DUT,MHPE2 and SHPE3 are scaled to "N" EVI.Clear BGP neighbors of 534 the DUT.Once adjacency is established in the DUT, check the 535 routes received from MHPE2 and SHPE3 for "N" EVI in the DUT.Then 536 increment the scale of N by 5% of N till the limit is reached.The 537 limit is where the DUT cant learn any EVPN routes from peers. 539 Measurement : 541 There should not be any loss of route types 1,2,3 and 4 in DUT.DUT 542 must relearn all type 1,2,3 and 4 from remote routers. The DUT must 543 be subjected to various values of N to find the optimal scale limit. 544 The scope of the test is find out the maximum evpn instance that a 545 DUT can hold. 547 3.11. Scale Convergence 549 Objective: 551 To measure the convergence time of DUT when the DUT is scaled with 552 EVPN instance along with traffic. 554 Topology : Topology 1 556 Procedure: 558 Scale N EVIs in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE 559 using traffic generator with X different source and destination mac 560 addresses for N EVI's.Send F frames from traffic generator to SHPE3 561 with X different source and destination mac addresses.There will be 562 2X number of MAC addresses will be learned in DUT EVPN MAC table. 563 There is a bi directional traffic flow with F pps in each direction. 564 Then clear the BGP neighbors in the DUT.Once the adjacency is 565 restored in DUT.Measure the time taken to learn 2X MAC address in 566 DUT MAC table. 568 Measurement : 570 The DUT must learn 2X MAC addresses. Measure the time taken to learn 571 2X MAC in DUT.The test is repeated for "N" times and the values are 572 collected. The convergence time is calculated by averaging the 573 values obtained by "N" samples. 575 Time taken to learn 2X macs in DUT = (T1+T2+..Tn)/N 577 3.12. SOAK Test. 579 Objective: 581 This test is carried out to measure the stability of the DUT in a 582 scaled environment with traffic over a period of time "T'". In each 583 interval "t1" the DUT CPU usage, memory usage are measured.The DUT 584 is checked for any crashes during this time period. 586 Topology : Topology 1 588 Procedure: 590 Scale N EVI's in DUT,SHPE3 and MHPE2.Send F frames to DUT from CE 591 using traffic generator with different X source and destination mac 592 addresses for N EVI's.Send F frames from traffic generator to SHPE3 593 with X different source and destination mac addresses.There will be 594 2X number of MAC addresses will be learned in DUT EVPN MAC table. 595 There is a bi directional traffic flow with F pps in each direction. 596 The DUT must run with traffic for 24 hours, every hour check for 597 memory leak, CPU usage and crash. 599 Measurement : 601 Take the hourly reading of CPU, process memory. There should not be 602 any leak, crashes, CPU spikes. Th CPU spike is determined as the CPU 603 usage which shoots at 40 to 50 percent of the average usage.The 604 average value vary from device to device. Memory leak is determined 605 by increase usage of the memory for EVPN process. The expectation is 606 under steady state the memory usage for EVPN process should not 607 increase. 609 4. Test Cases for PBB-EVPN Benchmarking 611 4.1. Local MAC Learning 613 Objective: 615 To Record the time taken to learn the MAC address locally. 617 Topology : Topology 1 619 Procedure: 621 Send "X" unicast frames from CE to MHPE1(DUT) working in SA mode with 622 "X" different source and destination mac addresses from traffic 623 generator.The DUT must learn "X" macs in data plane. 625 Measurement : 627 Measure the time taken by the DUT to learn the "X" MACs in the data 628 plane. The data plane measurement is taken by considering DUT as 629 black box the range of "X" MAC is known from traffic generator and 630 the same must be learned in DUT, the time taken to learn "X" MAC is 631 measured.The test is repeated for "N" times and the values are 632 collected.The mac learning time is calculated by averaging the 633 values obtained from "N" samples. 635 Mac learning rate = (T1+T2+..Tn)/N 637 4.2. Remote Mac Learning 639 Objective: 641 To Record the time taken to learn the remote macs. 643 Topology : Topology 1 645 Procedure: 647 Send X frames with X different source and destination mac addresses 648 to SHPE3 from traffic generator.These macs will be flooded to MHPE1 649 and MHPE2 by SHPE3.The DUT and MHPE2 are running SA mode. 651 Measurement : 653 Measure the time taken to learn X mac addresses in DUT mac table. 654 The test is repeated for "N" times and the values are collected.The 655 mac learning time is calculated by averaging the values obtained by 656 "N" samples. 658 Remote Mac learning rate = (T1+T2+..Tn)/N 660 4.3. MAC Flush due to link failure 662 Objective: 664 To record the time taken to flush the locally learned mac and the 665 time taken to relearn the same amount of macs. 667 Topology : Topology 1 668 Procedure: 670 Send X frames with X different source and destination mac addresses 671 to DUT from CE using traffic generator.Wait till the MHPE1 learn 672 all X MAC address.Then fail the MHPE1 CE link and measure the time 673 taken to flush these X MACs from the PBB-EVPN MAC table.Then bring 674 up the link.Measure the time taken to relearn X MACS.The DUT and 675 MHPE2 are running SA mode. 677 Measurement : 679 Measure the time taken for flushing these X MAC addresses.Measure 680 the time taken to relearn the X MACs in DUT.The test is repeated for 681 "N" times and the values are collected. The flush and the relearning 682 time is calculated by averaging the values obtained by "N" samples. 684 Flush rate = (T1+T2+..Tn)/N 686 Relearning rate = (T1+T2+..Tn)/N 688 4.4. MAC Flush due to remote Failure 690 Objective: 692 To record the time taken to flush the remote mac learned in DUT 693 during remote link failure. 695 Topology : Topology 1 697 Procedure: 699 Send X frames with X different source and destination mac addresses 700 to DUT from SHPE3 using traffic generator.Bring down the link 701 between SHPE3 and traffic generator. Then measure the time taken to 702 flush the DUT PBB-EVPN MAC address table.The remote MACs will be 703 learned by Data plane, but the B-MAC will be learned by control 704 plane.The DUT and MHPE2 are running SA mode. 706 Measurement : 708 Measure the time taken to flush X remote MACs from PBB-EVPN MAC table 709 of DUT.Repeat these test and plot the data.The test is repeated 710 for "N" times and the values are collected. The flush rate is 711 calculated by averaging the values obtained by "N" samples. 713 Flush rate = (T1+T2+..Tn)/N 715 4.5. MAC aging 717 Objective: 719 To measure the mac aging time. 721 Topology : Topology 1 723 Procedure: 725 Send X frames with X different source and destination mac addresses 726 to DUT from CE using traffic generator.Wait till X MAC address are 727 learned in DUT PBB-EVPN MAC table.Then stop the traffic.Record 728 the time taken to flush X MAC entries due to aging. The DUT and 729 MHPE2 running in SA mode 731 Measurement : 733 Measure the time taken to flush X MAC addresses due to aging.Repeat 734 these test and plot the data.The test is repeated for "N" times and 735 the values are collected.The aging is calculated by averaging the 736 values obtained by "N" samples. 738 Aging time for X Macs in sec = (T1+T2+..Tn)/N 740 4.6. Remote MAC Aging. 742 Objective: 744 To measure the remote mac aging time. 746 Topology : Topology 1 748 Procedure: 750 Send X frames with X different source and destination mac addresses 751 to DUT from SHPE3 using traffic generator.Stop the traffic at 752 remote PE(SHPE3).Measure the time taken to remove these remote MACs 753 from DUT PBB-EVPN MAC table.The DUT and MHPE2 are running in SA 754 mode. 756 Measurement : 758 Measure the time taken to flush the X remote MACs from DUT PBB-EVPN 759 MAC table due to aging .The test is repeated for "N" times and the 760 values are collected. The aging is calculated by averaging the 761 values obtained by "N" samples. 763 Aging time for X Macs in sec = (T1+T2+..Tn)/N 765 4.7. Local and Remote MAC Learning 767 Objective: 769 To record the time taken to learn both local and remote macs. 771 Topology : Topology 1 773 Procedure: 775 Send X frames with X different source and destination mac addresses 776 to DUT from SHPE3 using traffic generator.Send X frames with 777 different source and destination mac addresses from traffic generator 778 connected to CE.The source and destination mac addresses of flows 779 must be complimentary to have unicast flows.Measure the time taken 780 by the DUT to learn 2X in MAC table.DUT and MHPE2 are running in SA 781 mode. 783 Measurement : 785 Measure the time taken to learn 2X MAC addresses table in DUT PBB- 786 EVPN MAC table.The test is repeated for "N" times and the values 787 are collected.The mac learning time is calculated by averaging 788 the values obtained by "N" samples. 790 Mac learning rate = (T1+T2+..Tn)/N 792 4.8. High Availability 794 Objective: 796 To record traffic loss during routing engine failover. 798 Topology : Topology 1 800 Procedure: 802 Send X frames to DUT with X different source and destination mac 803 addresses from CE using the traffic generator.Send X frames from 804 traffic generator to SHPE3 with X different source and destination 805 mac addresses, so that 2X MAC address will be Learned in DUT. There 806 is a bi directional traffic flow with X pps in each direction. Then 807 do a routing engine fail-over. 809 Measurement : 811 The expectation of the test is 0 traffic loss with no change in the 812 DF role.DUT should not withdraw any routes.But in cases where the 813 DUT is not property synchronized between master and standby,due to 814 that packet loss are observed. In that scenario the packet loss is 815 measured.The test is repeated for "N" times and the values are 816 collected.The packet loss is calculated by averaging the values 817 obtained by "N" samples. 819 Packet loss in sec with 2X mac addresses = (T1+T2+..Tn)/N 821 4.9. Scale 823 Objective: 825 To measure the scale limit of DUT for PBB-EVPN. 827 Topology : Topology 1 829 Procedure: 831 The DUT,MHPE2 and SHPE3 are scaled to "N" PBB-EVN instances.Clear BGP 832 neighbors of the DUT. Once adjacency is established in the DUT. 833 check the routes received from MHPE2 and SHPE3 for "N" PBB-EVPN 834 instances in the DUT.Then increment the scale of N by 5% of N till 835 the limit is reached.The limit is where the DUT cant learn any EVPN 836 routes from peers. 838 Measurement : 840 There should not be any loss of route types 2,3 and 4 in DUT.DUT 841 must relearn all type 2,3 and 4 from remote routers.The DUT must be 842 subjected to various values of N to find the optimal scale limit. 843 The scope of the test is find out the maximum evpn instance that a 844 DUT can hold. 846 4.10. Scale Convergence 848 Objective: 850 To measure the convergence time of DUT when the DUT is scaled with 851 EVPN instance along with traffic. 853 Topology : Topology 1 855 Procedure: 857 Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to 858 DUT from CE using traffic generator with X different source and 859 destination mac addresses for N PBB-EVPN instances. Send F frames 860 from traffic generator to SHPE3 with X different source and 861 destination mac addresses.There will be 2X MAC addresses will be 862 learned in DUT PBB-EVPN MAC table.There is a bi directional traffic 863 flow with F pps in each direction.Then clear the BGP neighbors in 864 the DUT. Once the adjacency is restored in DUT. Measure the time 865 taken to learn 2X MAC address in DUT MAC table. 867 Measurement : 869 The DUT must learn 2X MAC addresses.Measure the time taken to learn 870 2X MAC in DUT.The test is repeated for "N" times and the values are 871 collected.The convergence time is calculated by averaging the values 872 obtained by "N" samples. 874 Convergence time for 2X in sec = (T1+T2+..Tn)/N 876 4.11. Soak Test 878 Objective: 880 To measure the stability of the DUT in a scaled environment with 881 traffic. 883 Topology : Topology 1 885 Procedure: 887 Scale N PBB-EVPN instances in DUT,SHPE3 and MHPE2.Send F frames to 888 DUT from CE using traffic generator with X different source and 889 destination mac addresses for N PBB-EVPN instances. Send F frames 890 from traffic generator to SHPE3 with X different source and 891 destination mac addresses.There will be 2X MAC addresses will be 892 learned in DUT PBB- EVPN MAC table.There is a bi directional 893 traffic flow with F pps in Each direction.The DUT must run with 894 traffic for 24 hours, every hour check the memory leak, crashes. 896 Measurement : 898 Take the hourly reading of CPU, process memory. There should not be 899 any leak, crashes, CPU spikes.The CPU spike is determined as the CPU 900 usage which shoots at 40 to 50 percent of the average usage.The 901 average value vary from device to device.Memory leak is determined 902 by increase usage of the memory for PBB-EVPN process.The 903 expectation is under steady state the memory usage for PBB-EVPN process 904 should not increase. 906 5. Acknowledgments 908 We would like to thank Fioccola Giuseppe of Telecom Italia reviewing 909 our draft and commenting it.We would like to thank Sarah Banks for 910 guiding and mentoring us. 912 6. IANA Considerations 914 This memo includes no request to IANA. 916 7. Security Considerations 918 The benchmarking tests described in this document are limited to the 919 performance characterization of controllers in a lab environment with 920 isolated networks. The benchmarking network topology will be an 921 independent test setup and MUST NOT be connected to devices that may 922 forward the test traffic into a production network or misroute 923 traffic to the test management network. Further, benchmarking is 924 performed on a "black-box" basis, relying solely on measurements 925 observable external to the controller. Special capabilities SHOULD 926 NOT exist in the controller specifically for benchmarking purposes. 927 Any implications for network security arising from the controller 928 SHOULD be identical in the lab and in production networks. 930 8. References 932 8.1. Normative References 934 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 935 Requirement Levels", BCP 14, RFC 2119, 936 DOI 10.17487/RFC2119, March 1997, 937 . 939 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 940 Network Interconnect Devices", RFC 2544, 941 DOI 10.17487/RFC2544, March 1999, 942 . 944 [RFC2899] Ginoza, S., "Request for Comments Summary RFC Numbers 945 2800-2899", RFC 2899, DOI 10.17487/RFC2899, May 2001, 946 . 948 8.2. Informative References 950 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 951 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 952 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 953 2015, . 955 [RFC7623] Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W. 956 Henderickx, "Provider Backbone Bridging Combined with 957 Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623, 958 September 2015, . 960 Appendix A. Appendix 962 Authors' Addresses 964 Sudhin Jacob (editor) 965 Juniper Networks 966 Bangalore 967 India 969 Phone: +91 8061212543 970 Email: sjacob@juniper.net 972 Kishore Tiruveedhula 973 Juniper Networks 974 10 Technology Park Dr 975 Westford, MA 01886 976 USA 978 Phone: +1 9785898861 979 Email: kishoret@juniper.net