idnits 2.17.1 draft-ietf-bmwg-mswitch-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 14 longer pages, the longest (page 5) being 65 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 22 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack an Authors' Addresses Section. == There are 3 instances of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 81 has weird spacing: '...erfaces are g...' == Line 618 has weird spacing: '...erfaces are g...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: In these tests, a port SHOULD transmit and receive the same amount of packets. Each port MUST count the packets received with a valid address. Any packet received which does not have a valid address MUST not be counted as a received packet and can be counted as part of a flood count as described in 3.8.3 in RFC 2285. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: In these tests, a port SHOULD transmit and receive the same amount of packets. Each port MUST count the packets received with a valid address. Any packet received which does not have a valid address MUST not be counted as a received packet and can be counted as part of a flood count as described in 3.8.3 in RFC 2285. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (August 1998) is 9385 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 9 errors (**), 0 flaws (~~), 8 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group R. Mandeville 2 Internet-Draft European Network Laboratories 3 Expiration Date: February 1999 J. Keene 4 Netcom Systems 5 August 1998 7 Benchmarking Methodology for LAN Switching Devices 8 10 Status of this Memo 12 This document is an Internet-Draft. Internet-Drafts are working 13 documents of the Internet Engineering Task Force (IETF),its areas, 14 and its working groups. Note that other groups may also distribute 15 working documents as Internet-Drafts. 17 Internet-Drafts are draft documents valid for a maximum of six months 18 and may be updated, replaced, or obsoleted by other documents at any 19 time. It is inappropriate to use Internet-Drafts as reference 20 material or to cite them other than as "work in progress." 22 To view the entire list of current Internet-Drafts, please check 23 the "1id-abstracts.txt" listing contained in the Internet-Drafts 24 Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net 25 (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au 26 (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu 27 (US West Coast). 29 This memo provides information for the Internet community. This memo 30 does not specify an Internet standard of any kind. Distribution of 31 this memo is unlimited. 33 Abstract 35 This document is intended to provide methodology for the benchmarking 36 of local area network (LAN) switching devices. It extends the 37 methodology already defined for benchmarking network interconnecting 38 devices in RFC 1944 to switching devices. 40 This RFC primarily deals with devices which switch frames at the 41 Medium Access Control (MAC) layer. It provides a methodology for 42 benchmarking switching devices, forwarding performance, congestion 43 control, latency, address handling and filtering. In addition to 44 defining the tests, this document also describes specific formats for 45 reporting the results of the tests. 47 1. Introduction 48 This document defines a specific set of tests to measure and report 49 the performance characteristics of network switching devices. 51 A previous document, "Benchmarking Terminology for LAN Switching 52 Devices" (RFC 2285), defined many of the terms that are used in this 53 document. The terminology document SHOULD be consulted before 54 attempting to make use of this document. 56 2. Requirements 58 The following RFCs SHOULD be consulted before attempting to make use 59 of this document: 61 * RFC 1242 "Benchmarking Terminology for Network Interconnect 62 Devices" 64 * RFC 1944 "Benchmarking Methodology for Network Interconnect 65 Devices" 67 * RFC 2285 "Benchmarking Terminology for LAN Switching Devices" 69 For the sake of clarity and continuity, this RFC adopts the template 70 for benchmarking tests set out in Section 26 of RFC 1944. 72 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 73 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 74 this document are to be interpreted as described in RFC 2119. 76 3. Test setup 78 This document extends the general test setup described in section 6 79 of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944 80 primarily describes non-meshed traffic where input and output 81 interfaces are grouped in mutually exclusive sending and 82 receiving pairs. In fully meshed traffic, each interface of a 83 DUT/SUT is set up to both receive and transmit frames to all 84 the other interfaces under test. 86 Prior to each test run, the DUT/SUT MUST learn the MAC addresses used 87 in 88 the test and the address learning MUST be verified to avoid flooded 89 frames being counted as correctly received frames. The forwarding 90 rate, namely the rate at which address learning frames are offered 91 may 92 have to be adjusted to be as low as 50 frames per second or even 93 less, 94 to guarantee successful learning. The DUT/SUT address aging time 95 SHOULD be 96 configured to be greater than the period between the learning phase 97 of 98 the test and the test run; in non-meshed and partially meshed tests, 99 the aging time SHOULD at a minimum be set to at least the length of 100 the test period. More than one trial may be needed for the 101 association 102 of the address to the port to occur. 104 If a DUT/SUT uses a hashing algorithm with address learning, the 105 DUT/SUT may not learn the necessary addresses to perform the tests. 106 The format of the MAC addresses MUST be adjustable so that the 107 address 108 mapping may be re-arranged to make a DUT/SUT learn addresses 109 without confusion. 111 It is recommended that SNMP and Spanning Tree be disabled when bench- 112 marking switching devices unless investigating overhead behavior. If 113 such protocols cannot be turned off, it is recommended that the 114 levels of offered load be reduced (less than 100%) to allow for 115 the additional management frames. 117 4. Frame formats and sizes 119 For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and 120 Appendix C. 122 There are three frame formats for layer 2 Ethernet switches: 123 standard MAC Ethernet frames, standard MAC Ethernet frames 124 with vendor-specific tags added to them, and IEEE 802.3ab 125 frames tagged to accommodate 802.3p,q. The two types of tagged 126 frames may exceed the standard maximum length frame of 1518 bytes, 127 and 128 may not be accepted by the interface controllers of some DUT/SUTs. 129 It is recommended to check the compatibility of the DUT/SUT 130 with tagged frames before testing. 132 Devices switching tagged frames of over 1518 bytes will have a lower 133 maximum forwarding rate than standard untagged frames. 135 5. Benchmarking Tests 137 The following tests offer objectives, procedures, and reporting 138 formats for benchmarking LAN switching devices. 140 5.1 Fully meshed throughput, frame loss and forwarding rates 141 5.2 Partially meshed overloading 142 5.3 Head of line blocking 143 5.4 Partially meshed multiple devices 144 5.5 Multiple streams of unidirectional traffic 145 5.6 Filter illegal frames 146 5.7 Broadcast frame handling and latency 147 5.8 Maximum forwarding rate and minimum interframe gap 148 5.9 Address caching capacity 149 5.10 Address learning rate 151 5.1 Fully meshed throughput, frame loss and forwarding rates 153 Objective: 155 To determine the throughput, frame loss and forwarding rates of 156 DUT/SUTs 157 offered fully meshed traffic as defined in RFC 2285. 159 Procedure: 161 RFC 2285 points out that when offering bursty meshed traffic, the 162 variables which MUST be defined are frame size, bust size, 163 interframe gap, interburst gap, and load. Each variable is 164 configured with the following considerations. 166 Interframe Gap (IFG) - The IFG between frames inside a burst 167 MUST 168 be at the minimum specified by the standard (9.6 us for 10Mbps 169 Ethernet and 0.96 us for 100Mbps Ethernet). 171 Interburst Gap (IBG) - This is the interval between bursts of 172 traffic. Refer to Appendix A, Calculating Interburst Gap, for 173 the formula used to compute IBG. 175 Load / Port - The test SHOULD be run multiple times with a 176 different load per port in each case. The 100% load translates to 177 a transmit load of 50% for half duplex. This type of test SHOULD 178 also be run at higher than 100% loads. 180 In half duplex mode, exactly half of the target load SHOULD be 181 sent to each of the ports under test. For example, with a 100% 182 load of 64-byte frames, the target load for each port under test 183 is 7440 frames received per second and 7440 frames transmitted per 184 second (for 10Mbps Ethernet). 186 Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024, 187 1280 and 1518 bytes, per RFC 1944 section 9. 189 Burst Size - The burst size defines the number of packets sent 190 back-to-back at the minimum legal IFG (96 bit times) before 191 pausing transmission to receive frames. Burst sizes 192 SHOULD vary between 1 and 930 frames. 194 To avoid truncating bursts, a burst size SHOULD be an even 195 multiple into 7440. 7440 is the maximum frames per second for half 196 duplex mode; 14880 is the maximum frames per second for full 197 duplex mode. Refer to Appendix A for a table of recommended burst 198 values. 200 Each port in the test sends frames to all other ports in a round- 201 robin type fashion. The following table shows how each port in a test 202 SHOULD transmit frames to all other ports in the test. In this 203 example, there are six ports with 1 address per port: 205 Source Port, then Destination Ports (in order of transmission) 207 Port #1 2 3 4 5 6 2... 208 Port #2 3 4 5 6 1 3... 209 Port #3 4 5 6 1 2 4... 210 Port #4 5 6 1 2 3 5... 211 Port #5 6 1 2 3 4 6... 212 Port #6 1 2 3 4 5 1... 214 As shown in the table, there is an equal distribution of destination 215 addresses for each transmit opportunity. This keeps the test balanced 216 so that one destination port is not overloaded by the test algorithm 217 and all ports are equally and fully loaded throughout the test. For 218 tests using multiple addresses per port, the actual port destinations 219 are the same as described above and the actual source/destination 220 address 221 pairs are chosen randomly to exercise the DUT/SUT's ability to 222 perform 223 address lookups. 225 For every address, the testing device sends learning packets to allow 226 the DUT/SUT to load its address tables properly. 228 To measure the DUT/SUT's ability to switch traffic while performing 229 many 230 different address lookups, the number of addresses per port SHOULD be 231 increased in a series of tests. 233 Reporting format: 235 In these tests, a port SHOULD transmit and receive the same amount of 236 packets. Each port MUST count the packets received with a valid 237 address. 238 Any packet received which does not have a valid address MUST not be 239 counted as a received packet and can be counted as part of a flood 240 count as described in 3.8.3 in RFC 2285. 242 The results for these tests SHOULD be reported in the form of 243 numerical 244 data or a graph with text to indicate the data type. The data types 245 for 246 the throughput test and for the frame loss rate test are described in 247 RFC 1944. 249 Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number 250 of 251 frames per second that the device is observed to successfully 252 transmit 253 to the correct destination interface in response to a specified 254 offered 255 load. The offered load MUST also be cited. 257 Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported 258 as 259 the number of frames per second that a device can successfully 260 transmit 261 to the correct destination interface in response to the maximum 262 offered 263 load as defined in RFC 2285, section 3.6. The maximum offered load 264 MUST 265 also be cited. 267 Maximum forwarding rate (MFR) SHOULD be reported as the highest 268 forwarding 269 rate of a DUT/SUT taken from an iterative set of forwarding rate 270 measurements. The load applied to the device MUST also be cited. 272 5. 2 Partially meshed overloading 274 To be done. 276 5. 3 Head-of-line blocking 278 To be done. 280 5.4 Partially Meshed Multiple Devices 282 To be done. 284 5.5 Multiple streams of unidirectional traffic 286 To be done. 288 5.6 Filter illegal frames 290 To be done. 292 5.7 Broadcast frame handling and latency test 294 To be done. 296 5.8 Maximum forwarding rate and minimum interframe gap 298 To be done. 300 5.9 Address Caching Capacity 302 Objective: 304 To determine the address caching capacity of a LAN switching device 305 as defined in RFC 2285, section 3.8.1. 307 Procedure: 309 This test SHOULD at a minimum be performed in a three-port 310 configuration as described below. 312 The first port (port 1) of the testing device is connected to the 313 DUT/SUT 314 and is the port from which a testing device sends frames with varying 315 source addresses and a fixed destination address corresponding to the 316 MAC 317 address of the receiving port. By receiving frames with varying 318 source 319 addresses, the DUT/SUT will learn these new addresses from the 320 sending port 321 of the test device. 323 A second port (port 2) of the testing device is connected to the 324 DUT/SUT 325 and acts as the receiving port for the address learning frames. This 326 port also sends "control" frames back to the addresses learned 327 on the first port. The algorithm for this is explained below. 329 -- A third port (port 3) on the testing device MUST be connected to 330 a 331 port on the DUT/SUT and act as a monitoring port to listen for 332 flooded frames. 334 The algorithm for the test is as follows: 336 BEGIN 337 Set Initial Value of N to user specified number, where N is the 338 number of addresses to be verified in each iteration 339 WHILE NOT Finished DO 340 PAUSE for the aging time specified 341 Address learning: Port 1 sends N frames with varying 342 source addresses to Port 2 to attempt to fill the DUT/SUT 343 address table for Port #1. 344 Controlling: Port 2 sends N frames with varying 345 destination addresses corresponding to Port 1. 346 IF (Port 3 received a frame during Control phase) OR 347 (Port 1 did not receive the correct # of frames) 348 THEN 349 Address Table of DUT/SUT was full 350 Set N to lower number (in binary search method) 351 ELSE 352 Address Table of DUT/SUT was NOT full 353 Set N to higher number (in binary search method) 354 IF High and Low Values of N Meet 355 THEN 356 Test is Finished, Value of N equals number of 357 addresses supported by DUT/SUT 358 ELSE 359 Continue Test 360 END WHILE 361 DONE 362 Using a binary search approach, the test targets the exact number 363 of addresses supported per port with minimal test iterations. Due 364 to the aging time of DUT/SUT address tables, each iteration may take 365 some time during the waiting period for the addresses to clear. If 366 possible, configure the DUT/SUT for a low value for the aging time. 368 Once the high and low values of N meet, then the threshold of address 369 handling has been found. 371 For this test, the aging time of the DUT/SUT MUST be known. Use a 372 short 373 aging time if possible to reduce the time needed to run the test. 374 The aging time MUST be longer than the time necessary to produce 375 packets at the specified rate. If a low frame rate is used for 376 the test, then it may be possible that sending a large amount of 377 frames may actually take longer than the aging time. Keep in mind 378 that the test actually sends twice as many frames as the number 379 of addresses being tested due to the learning phase and the 380 controlling phase. 382 Set the initial value of addresses per port to a number 383 slightly higher than the number of addresses that the DUT/SUT can 384 handle. 385 This step primes the binary search and can reduce the number of 386 iterations required to determine the exact number supported. 388 Set the forwarding rate to a number that is reasonable to be handled 389 by the DUT/SUT and one that is high enough so that the test 390 iterations are 391 not too long. 393 Reporting format: 395 After the test is run, results for each iteration SHOULD be displayed 396 in a table to include: 398 -- the number of addresses used for each test iteration. 400 -- the frame rate used for each test iteration. 402 -- number of control frames that were transmitted by test port 403 number 2. Control frames are the frames sent with varying destination 404 addresses to confirm that the DUT/SUT has learned all of the 405 addresses 406 for each test iteration. 408 -- the number of frames received by test port 2 during the control 409 portion of each test. If the number is non-zero, this is an 410 indication 411 of the DUT/SUT flooding a frame in which the destination address is 412 not 413 in the address table. 415 -- the number of frames that test port 1 received during the control 416 portion of the test. This number will include those frames which are 417 Spanning tree frames. For a normal test iteration, this number SHOULD 418 be equal to the number of frames transmitted by port 2 during the 419 control phase. 421 -- the number of frames that test port 1 received during the control 422 phase not including Spanning Tree or other non test generated frames. 423 In all cases, this number SHOULD be equal to the number of frames 424 transmitted by port 2 during the control phase of the test. 426 -- the number of frames test port 3 received during the control 427 phase 428 of the test. If the value is not zero, then this indicates that for 429 that test iteration, the DUT/SUT could not determine the proper 430 destination 431 port for that many frames. In other words, the DUT/SUT flooded the 432 frame to 433 all ports since its address table was full. 435 5.10 Address Learning Rate 437 Objective: 439 Once the maximum number of addresses supported per port by the 440 DUT/SUT is 441 known, the addressing learning rate (the rate at which the DUT/SUT 442 learns 443 these addresses) can be determined. 445 Procedure: 447 An algorithm similar to the one used to determine address caching 448 capacity can be used to determine the address learning rate. This 449 test iterates the rate at which address learning frames are offered 450 by the test device connected to the DUT/SUT. It is recommended to 451 set 452 the number of addresses offered to the DUT/SUT in this test to the 453 maximum caching capacity. However, the address learning rate might 454 be 455 determined for different numbers of addresses but in each test run, 456 the number MUST remain constant. 458 Initializing the forwarding rate primes the binary search algorithm 459 and can help to shorten the overall test duration. 461 A third port on the DUT/SUT MUST listen for flooded frames. 463 In this test, the forwarding rate SHOULD be varied until a rate is 464 found that is the threshold for the rate supported by the DUT/SUT. It 465 may be useful to pick a value slightly higher than the advertised 466 forwarding rate. 468 Reporting format: 470 To be done. 472 Section 6. Security Considerations 474 This document does not yet address Security Considerations. 476 Section 7. Authors' Address 478 Robert Mandeville 479 European Network Laboratories (ENL) 480 2, rue Helene Boucher 481 87286 Guyancourt Cedex 482 France 484 Phone: + 33 1 39 44 12 05 485 EMail: bob@enl.net 487 Judy Keene 488 Netcom Systems 489 20550 Nordhoff St. 490 Chatsworth, CA 91311 491 USA 493 Phone: +1 818 700 5100 494 Email: judy_keene@netcomsystems.com 496 Appendix A: Calculating the Interburst Gap 498 IBG is defined in RFC 2285 as a function of maximum media rate (also 499 known as line rate), the length of the frames in the bursts with the 500 preamble and the interframe gap (IFG), the number of frames in the 501 bursts, 502 and the intended load. 504 Using the burst size, frame size and the load per port, the IBG can 505 be 506 calculated: 508 Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit 509 times)) 510 * 1 (for full duplex) or .5 (for half duplex) 511 (10 bit times equal 1 microsecond) 513 Example: 515 Burst size = 24 516 Frame size = 64 518 24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times = 519 1612.8 us IBG 521 In half duplex mode, exactly half of the target load SHOULD be sent 522 to each of the ports under test. For example, with a 100% load of 523 64-byte frames, the target load for each port under test is 7440 524 frames received per second and 7440 frames transmitted per second 525 (for 10Mbps Ethernet). 527 Recommended characteristic values for burst size are: 529 2 15 60 240 930 530 3 16 62 248 531 4 20 80 310 532 5 24 93 372 533 6 30 120 465 534 8 31 124 496 535 10 40 155 620 536 12 48 186 744 538 Network Working Group R. Mandeville 539 Internet-Draft European Network Laboratories 540 Expiration Date: February 1999 J. Keene 541 Netcom Systems 542 August 1998 544 Benchmarking Methodology for LAN Switching Devices 545 547 Status of this Memo 549 This document is an Internet-Draft. Internet-Drafts are working 550 documents of the Internet Engineering Task Force (IETF),its areas, 551 and its working groups. Note that other groups may also distribute 552 working documents as Internet-Drafts. 554 Internet-Drafts are draft documents valid for a maximum of six months 555 and may be updated, replaced, or obsoleted by other documents at any 556 time. It is inappropriate to use Internet-Drafts as reference 557 material or to cite them other than as "work in progress." 559 To view the entire list of current Internet-Drafts, please check 560 the "1id-abstracts.txt" listing contained in the Internet-Drafts 561 Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net 562 (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au 563 (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu 564 (US West Coast). 566 This memo provides information for the Internet community. This memo 567 does not specify an Internet standard of any kind. Distribution of 568 this memo is unlimited. 570 Abstract 572 This document is intended to provide methodology for the benchmarking 573 of local area network (LAN) switching devices. It extends the 574 methodology already defined for benchmarking network interconnecting 575 devices in RFC 1944 to switching devices. 577 This RFC primarily deals with devices which switch frames at the 578 Medium Access Control (MAC) layer. Itprovides a methodology for 579 benchmarking switching devices, forwarding performance, congestion 580 control, latency, address handling and filtering. In addition to 581 defining the tests, this document also describes specific formats for 582 reporting the results of the tests. 584 1. Introduction 585 This document defines a specific set of tests to measure and report 586 the performance characteristics of network switching devices. 588 A previous document, "Benchmarking Terminology for LAN Switching 589 Devices" (RFC 2285), defined many of the terms that are used in this 590 document. The terminology document SHOULD be consulted before 591 attempting to make use of this document. 593 2. Requirements 595 The following RFCs SHOULD be consulted before attempting to make use 596 of this document: 598 * RFC 1242 "Benchmarking Terminology for Network Interconnect 599 Devices" 601 * RFC 1944 "Benchmarking Methodology for Network Interconnect 602 Devices" 604 * RFC 2285 "Benchmarking Terminology for LAN Switching Devices" 606 For the sake of clarity and continuity, this RFC adopts the template 607 for benchmarking tests set out in Section 26 of RFC 1944. 609 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 610 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 611 this document are to be interpreted as described in RFC 2119. 613 3. Test setup 615 This document extends the general test setup described in section 6 616 of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944 617 primarily describes non-meshed traffic where input and output 618 interfaces are grouped in mutually exclusive sending and 619 receiving pairs. In fully meshed traffic, each interface of a 620 DUT/SUT is set up to both receive and transmit frames to all 621 the other interfaces under test. 623 Prior to each test run, the DUT/SUT MUST learn the MAC addresses used 624 in 625 the test and the address learning MUST be verified to avoid flooded 626 frames being counted as correctly received frames. The forwarding 627 rate, namely the rate at which address learning frames are offered 628 may 629 have to be adjusted to be as low as 50 frames per second or even 630 less, 631 to guarantee successful learning. The DUT/SUT address aging time 632 SHOULD be 633 configured to be greater than the period between the learning phase 634 of 635 the test and the test run; in non-meshed and partially meshed tests, 636 the aging time SHOULD at a minimum be set to at least the length of 637 the test period. More than one trial may be needed for the 638 association 639 of the address to the port to occur. 641 If a DUT/SUT uses a hashing algorithm with address learning, the 642 DUT/SUT may not learn the necessary addresses to perform the tests. 643 The format of the MAC addresses MUST be adjustable so that the 644 address 645 mapping may be re-arranged to make a DUT/SUT learn addresses 646 without confusion. 648 It is recommended that SNMP and Spanning Tree be disabled when bench- 649 marking switching devices unless investigating overhead behavior. If 650 such protocols cannot be turned off, it is recommended that the 651 levels of offered load be reduced (less than 100%) to allow for 652 the additional management frames. 654 4. Frame formats and sizes 656 For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and 657 Appendix C. 659 There are three frame formats for layer 2 Ethernet switches: 660 standard MAC Ethernet frames, standard MAC Ethernet frames 661 with vendor-specific tags added to them, and IEEE 802.3ab 662 frames tagged to accommodate 802.3p,q. The two types of tagged 663 frames may exceed the standard maximum length frame of 1518 bytes, 664 and 665 may not be accepted by the interface controllers of some DUT/SUTs. 666 It is recommended to check the compatibility of the DUT/SUT 667 with tagged frames before testing. 669 Devices switching tagged frames of over 1518 bytes will have a lower 670 maximum forwarding rate than standard untagged frames. 672 5. Benchmarking Tests 674 The following tests offer objectives, procedures, and reporting 675 formats for benchmarking LAN switching devices. 677 5.1 Fully meshed throughput, frame loss and forwarding rates 678 5.2 Partially meshed overloading 679 5.3 Head of line blocking 680 5.4 Partially meshed multiple devices 681 5.5 Multiple streams of unidirectional traffic 682 5.6 Filter illegal frames 683 5.7 Broadcast frame handling and latency 684 5.8 Maximum forwarding rate and minimum interframe gap 685 5.9 Address caching capacity 686 5.10 Address learning rate 688 5.1 Fully meshed throughput, frame loss and forwarding rates 690 Objective: 692 To determine the throughput, frame loss and forwarding rates of 693 DUT/SUTs 694 offered fully meshed traffic as defined in RFC 2285. 696 Procedure: 698 RFC 2285 points out that when offering bursty meshed traffic, the 699 variables which MUST be defined are frame size, bust size, 700 interframe gap, interburst gap, and load. Each variable is 701 configured with the following considerations. 703 Interframe Gap (IFG) - The IFG between frames inside a burst 704 MUST 705 be at the minimum specified by the standard (9.6 us for 10Mbps 706 Ethernet and 0.96 us for 100Mbps Ethernet). 708 Interburst Gap (IBG) - This is the interval between bursts of 709 traffic. Refer to Appendix A, Calculating Interburst Gap, for 710 the formula used to compute IBG. 712 Load / Port - The test SHOULD be run multiple times with a 713 different load per port in each case. The 100% load translates to 714 a transmit load of 50% for half duplex. This type of test SHOULD 715 also be run at higher than 100% loads. 717 In half duplex mode, exactly half of the target load SHOULD be 718 sent to each of the ports under test. For example, with a 100% 719 load of 64-byte frames, the target load for each port under test 720 is 7440 frames received per second and 7440 frames transmitted per 721 second (for 10Mbps Ethernet). 723 Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024, 724 1280 and 1518 bytes, per RFC 1944 section 9. 726 Burst Size - The burst size defines the number of packets sent 727 back-to-back at the minimum legal IFG (96 bit times) before 728 pausing transmission to receive frames. Burst sizes 729 SHOULD vary between 1 and 930 frames. 731 To avoid truncating bursts, a burst size SHOULD be an even 732 multiple into 7440. 7440 is the maximum frames per second for half 733 duplex mode; 14880 is the maximum frames per second for full 734 duplex mode. Refer to Appendix A for a table of recommended burst 735 values. 737 Each port in the test sends frames to all other ports in a round- 738 robin type fashion. The following table shows how each port in a test 739 SHOULD transmit frames to all other ports in the test. In this 740 example, there are six ports with 1 address per port: 742 Source Port, then Destination Ports (in order of transmission) 744 Port #1 2 3 4 5 6 2... 745 Port #2 3 4 5 6 1 3... 746 Port #3 4 5 6 1 2 4... 747 Port #4 5 6 1 2 3 5... 748 Port #5 6 1 2 3 4 6... 749 Port #6 1 2 3 4 5 1... 751 As shown in the table, there is an equal distribution of destination 752 addresses for each transmit opportunity. This keeps the test balanced 753 so that one destination port is not overloaded by the test algorithm 754 and all ports are equally and fully loaded throughout the test. For 755 tests using multiple addresses per port, the actual port destinations 756 are the same as described above and the actual source/destination 757 address 758 pairs are chosen randomly to exercise the DUT/SUT's ability to 759 perform 760 address lookups. 762 For every address, the testing device sends learning packets to allow 763 the DUT/SUT to load its address tables properly. 765 To measure the DUT/SUT's ability to switch traffic while performing 766 many 767 different address lookups, the number of addresses per port SHOULD be 768 increased in a series of tests. 770 Reporting format: 772 In these tests, a port SHOULD transmit and receive the same amount of 773 packets. Each port MUST count the packets received with a valid 774 address. 775 Any packet received which does not have a valid address MUST not be 776 counted as a received packet and can be counted as part of a flood 777 count as described in 3.8.3 in RFC 2285. 779 The results for these tests SHOULD be reported in the form of 780 numerical 781 data or a graph with text to indicate the data type. The data types 782 for 783 the throughput test and for the frame loss rate test are described in 784 RFC 1944. 786 Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number 787 of 788 frames per second that the device is observed to successfully 789 transmit 790 to the correct destination interface in response to a specified 791 offered 792 load. The offered load MUST also be cited. 794 Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported 795 as 796 the number of frames per second that a device can successfully 797 transmit 798 to the correct destination interface in response to the maximum 799 offered 800 load as defined in RFC 2285, section 3.6. The maximum offered load 801 MUST 802 also be cited. 804 Maximum forwarding rate (MFR) SHOULD be reported as the highest 805 forwarding 806 rate of a DUT/SUT taken from an iterative set of forwarding rate 807 measurements. The load applied to the device MUST also be cited. 809 5. 2 Partially meshed overloading 811 To be done. 813 5. 3 Head-of-line blocking 815 To be done. 817 5.4 Partially Meshed Multiple Devices 819 To be done. 821 5.5 Multiple streams of unidirectional traffic 823 To be done. 825 5.6 Filter illegal frames 827 To be done. 829 5.7 Broadcast frame handling and latency test 831 To be done. 833 5.8 Maximum forwarding rate and minimum interframe gap 835 To be done. 837 5.9 Address Caching Capacity 839 Objective: 841 To determine the address caching capacity of a LAN switching device 842 as defined in RFC 2285, section 3.8.1. 844 Procedure: 846 This test SHOULD at a minimum be performed in a three-port 847 configuration as described below. 849 The first port (port 1) of the testing device is connected to the 850 DUT/SUT 851 and is the port from which a testing device sends frames with varying 852 source addresses and a fixed destination address corresponding to the 853 MAC 854 address of the receiving port. By receiving frames with varying 855 source 856 addresses, the DUT/SUT will learn these new addresses from the 857 sending port 858 of the test device. 860 A second port (port 2) of the testing device is connected to the 861 DUT/SUT 862 and acts as the receiving port for the address learning frames. This 863 port also sends "control" frames back to the addresses learned 864 on the first port. The algorithm for this is explained below. 866 -- A third port (port 3) on the testing device MUST be connected to 867 a 868 port on the DUT/SUT and act as a monitoring port to listen for 869 flooded frames. 871 The algorithm for the test is as follows: 873 BEGIN 874 Set Initial Value of N to user specified number, where N is the 875 number of addresses to be verified in each iteration 876 WHILE NOT Finished DO 877 PAUSE for the aging time specified 878 Address learning: Port 1 sends N frames with varying 879 source addresses to Port 2 to attempt to fill the DUT/SUT 880 address table for Port #1. 881 Controlling: Port 2 sends N frames with varying 882 destination addresses corresponding to Port 1. 883 IF (Port 3 received a frame during Control phase) OR 884 (Port 1 did not receive the correct # of frames) 885 THEN 886 Address Table of DUT/SUT was full 887 Set N to lower number (in binary search method) 888 ELSE 889 Address Table of DUT/SUT was NOT full 890 Set N to higher number (in binary search method) 891 IF High and Low Values of N Meet 892 THEN 893 Test is Finished, Value of N equals number of 894 addresses supported by DUT/SUT 895 ELSE 896 Continue Test 897 END WHILE 898 DONE 899 Using a binary search approach, the test targets the exact number 900 of addresses supported per port with minimal test iterations. Due 901 to the aging time of DUT/SUT address tables, each iteration may take 902 some time during the waiting period for the addresses to clear. If 903 possible, configure the DUT/SUT for a low value for the aging time. 905 Once the high and low values of N meet, then the threshold of address 906 handling has been found. 908 For this test, the aging time of the DUT/SUT MUST be known. Use a 909 short 910 aging time if possible to reduce the time needed to run the test. 911 The aging time MUST be longer than the time necessary to produce 912 packets at the specified rate. If a low frame rate is used for 913 the test, then it may be possible that sending a large amount of 914 frames may actually take longer than the aging time. Keep in mind 915 that the test actually sends twice as many frames as the number 916 of addresses being tested due to the learning phase and the 917 controlling phase. 919 Set the initial value of addresses per port to a number 920 slightly higher than the number of addresses that the DUT/SUT can 921 handle. 922 This step primes the binary search and can reduce the number of 923 iterations required to determine the exact number supported. 925 Set the forwarding rate to a number that is reasonable to be handled 926 by the DUT/SUT and one that is high enough so that the test 927 iterations are 928 not too long. 930 Reporting format: 932 After the test is run, results for each iteration SHOULD be displayed 933 in a table to include: 935 -- the number of addresses used for each test iteration. 937 -- the frame rate used for each test iteration. 939 -- number of control frames that were transmitted by test port 940 number 2. Control frames are the frames sent with varying destination 941 addresses to confirm that the DUT/SUT has learned all of the 942 addresses 943 for each test iteration. 945 -- the number of frames received by test port 2 during the control 946 portion of each test. If the number is non-zero, this is an 947 indication 948 of the DUT/SUT flooding a frame in which the destination address is 949 not 950 in the address table. 952 -- the number of frames that test port 1 received during the control 953 portion of the test. This number will include those frames which are 954 Spanning tree frames. For a normal test iteration, this number SHOULD 955 be equal to the number of frames transmitted by port 2 during the 956 control phase. 958 -- the number of frames that test port 1 received during the control 959 phase not including Spanning Tree or other non test generated frames. 960 In all cases, this number SHOULD be equal to the number of frames 961 transmitted by port 2 during the control phase of the test. 963 -- the number of frames test port 3 received during the control 964 phase 965 of the test. If the value is not zero, then this indicates that for 966 that test iteration, the DUT/SUT could not determine the proper 967 destination 968 port for that many frames. In other words, the DUT/SUT flooded the 969 frame to 970 all ports since its address table was full. 972 5.10 Address Learning Rate 974 Objective: 976 Once the maximum number of addresses supported per port by the 977 DUT/SUT is 978 known, the addressing learning rate (the rate at which the DUT/SUT 979 learns 980 these addresses) can be determined. 982 Procedure: 984 An algorithm similar to the one used to determine address caching 985 capacity can be used to determine the address learning rate. This 986 test iterates the rate at which address learning frames are offered 987 by the test device connected to the DUT/SUT. It is recommended to 988 set 989 the number of addresses offered to the DUT/SUT in this test to the 990 maximum caching capacity. However, the address learning rate might 991 be 992 determined for different numbers of addresses but in each test run, 993 the number MUST remain constant. 995 Initializing the forwarding rate primes the binary search algorithm 996 and can help to shorten the overall test duration. 998 A third port on the DUT/SUT MUST listen for flooded frames. 1000 In this test, the forwarding rate SHOULD be varied until a rate is 1001 found that is the threshold for the rate supported by the DUT/SUT. It 1002 may be useful to pick a value slightly higher than the advertised 1003 forwarding rate. 1005 Reporting format: 1007 To be done. 1009 Section 6. Security Considerations 1011 This document does not yet address Security Considerations. 1013 Section 7. Authors' Address 1015 Robert Mandeville 1016 European Network Laboratories (ENL) 1017 2, rue Helene Boucher 1018 87286 Guyancourt Cedex 1019 France 1021 Phone: + 33 1 39 44 12 05 1022 EMail: bob@enl.net 1024 Judy Keene 1025 Netcom Systems 1026 20550 Nordhoff St. 1027 Chatsworth, CA 91311 1028 USA 1030 Phone: +1 818 700 5100 1031 Email: judy_keene@netcomsystems.com 1033 Appendix A: Calculating the Interburst Gap 1035 IBG is defined in RFC 2285 as a function of maximum media rate (also 1036 known as line rate), the length of the frames in the bursts with the 1037 preamble and the interframe gap (IFG), the number of frames in the 1038 bursts, 1039 and the intended load. 1041 Using the burst size, frame size and the load per port, the IBG can 1042 be 1043 calculated: 1045 Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit 1046 times)) 1047 * 1 (for full duplex) or .5 (for half duplex) 1048 (10 bit times equal 1 microsecond) 1050 Example: 1052 Burst size = 24 1053 Frame size = 64 1055 24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times = 1056 1612.8 us IBG 1058 In half duplex mode, exactly half of the target load SHOULD be sent 1059 to each of the ports under test. For example, with a 100% load of 1060 64-byte frames, the target load for each port under test is 7440 1061 frames received per second and 7440 frames transmitted per second 1062 (for 10Mbps Ethernet). 1064 Recommended characteristic values for burst size are: 1066 2 15 60 240 930 1067 3 16 62 248 1068 4 20 80 310 1069 5 24 93 372 1070 6 30 120 465 1071 8 31 124 496 1072 10 40 155 620 1073 12 48 186 744