idnits 2.17.1 draft-bb-2544like-production-tests-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to use 'NOT RECOMMENDED' as an RFC 2119 keyword, but does not include the phrase in its RFC 2119 key words list. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: The 24 input and output protocol addresses SHOULD not be any that are represented in the test data stream. The last filter SHOULD permit the forwarding of the test data stream. By "first" and "last" we mean to ensure that in the second case, 25 conditions must be checked before the data frames will match the conditions that permit the forwarding of the frame. Of course, if the DUT reorders the filters or does not use a linear scan of the filter rules the effect of the sequence in which the filters are input is properly lost. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: This appendix discusses certain issues in the benchmarking methodology where experience or judgment may play a role in the tests selected to be run or in the approach to constructing the test with a particular DUT. As such, this appendix MUST not be read as an amendment to the methodology described in the body of this document but as a guide to testing practice. -- The document date (October 22, 2012) is 4175 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group R. Bonica 3 Internet-Draft Juniper Networks 4 Intended status: Informational S. Bryant 5 Expires: April 25, 2013 Cisco Systems 6 October 22, 2012 8 RFC2544 Testing in Production Networks 9 draft-bb-2544like-production-tests-00 11 Abstract 13 This document considers the use of RFC2544 type tests in an 14 production network. 16 Status of this Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at http://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on April 25, 2013. 33 Copyright Notice 35 Copyright (c) 2012 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Conventions used in the document . . . . . . . . . . . . . . . 3 51 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 52 3. Cautious of RFC2544 in Production Networks . . . . . . . . . . 3 53 4. Real World . . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 5. Test To Be Run . . . . . . . . . . . . . . . . . . . . . . . . 4 55 6. Evaluating the Results . . . . . . . . . . . . . . . . . . . . 4 56 7. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 4 57 8. The Test Set Up . . . . . . . . . . . . . . . . . . . . . . . 5 58 9. DUT set up . . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 10. Frame Formats . . . . . . . . . . . . . . . . . . . . . . . . 5 60 11. Frame Sizes. . . . . . . . . . . . . . . . . . . . . . . . . . 5 61 12. Verifying received frames . . . . . . . . . . . . . . . . . . 6 62 13. Modifiers . . . . . . . . . . . . . . . . . . . . . . . . . . 6 63 14. Protocol Addresses . . . . . . . . . . . . . . . . . . . . . . 9 64 15. Route Set up . . . . . . . . . . . . . . . . . . . . . . . . . 9 65 16. Bidirectional traffic . . . . . . . . . . . . . . . . . . . . 9 66 17. Single Stream Path . . . . . . . . . . . . . . . . . . . . . . 9 67 18. Multi-port . . . . . . . . . . . . . . . . . . . . . . . . . . 10 68 19. Multiple Protocols . . . . . . . . . . . . . . . . . . . . . . 10 69 20. Multiple frame sizes . . . . . . . . . . . . . . . . . . . . . 10 70 21. Testing performance beyond a single DUT. . . . . . . . . . . . 10 71 22. Maximum frame rate . . . . . . . . . . . . . . . . . . . . . . 10 72 23. Bursty Traffic . . . . . . . . . . . . . . . . . . . . . . . . 11 73 24. Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 74 25. Trial description . . . . . . . . . . . . . . . . . . . . . . 11 75 26. Trial duration . . . . . . . . . . . . . . . . . . . . . . . . 11 76 27. Address resolution . . . . . . . . . . . . . . . . . . . . . . 11 77 28. Benchmarking tests . . . . . . . . . . . . . . . . . . . . . . 12 78 28.1. Throughput . . . . . . . . . . . . . . . . . . . . . . . 12 79 28.2. Latency . . . . . . . . . . . . . . . . . . . . . . . . . 12 80 28.3. Frame loss rate . . . . . . . . . . . . . . . . . . . . . 12 81 28.4. Back-to-back frames . . . . . . . . . . . . . . . . . . . 12 82 28.5. System Recovery . . . . . . . . . . . . . . . . . . . . . 12 83 28.6. Reset . . . . . . . . . . . . . . . . . . . . . . . . . . 12 84 29. IANA considerations . . . . . . . . . . . . . . . . . . . . . 13 85 30. Security considerations . . . . . . . . . . . . . . . . . . . 13 86 31. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 13 87 32. Informative References . . . . . . . . . . . . . . . . . . . . 13 88 Appendix A. Testing Considerations . . . . . . . . . . . . . . . 13 89 Appendix B. Maximum frame rates refer . . . . . . . . . . . . . . 14 90 Appendix C. Test Frame Formats . . . . . . . . . . . . . . . . . 15 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 15 93 1. Conventions used in the document 95 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 96 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 97 document are to be interpreted as described in RFC2119 [RFC2119]. 99 2. Introduction 101 This document considers the issues related to conducting tests 102 similar to RFC2544 [RFC2544] in a prodcution network. 104 3. Cautious of RFC2544 in Production Networks 106 Some types of production network offer a high degree of assured 107 issolation between the traffic of different users. An example of a 108 network technique that provides this issolation is a pseudowire 109 [RFC3985]. Some types of network, known as transport networks 110 [RFC5921] are traffic engineered to the point where there is normally 111 no resource contention betwen user traffic. The incautous use of 112 unmodified RFC2544 testing is NOT RECOMMENDED [RFC2544]. However 113 provided suitable approprate caution is applied the existing RFC2544 114 tests may provide some useful information on the behaviour of such a 115 network. 117 When using unmodified RFC2544 tests on such a network the tester 118 needs to be aware that the latency and latency variation will be 119 significantly higher than in a laboratory environment, and that error 120 rate and hence packet loss may be higher than in a laboratory 121 environment. It is therefore necessary to repeat tests a number of 122 times to be confident of the results and to present the outcome in a 123 statistical rather than definitive form. It is also necessary to 124 note that the results represent the behaviour of the system and 125 network at the time of measurement and may not reperesent the 126 behaviour at some other time in the past or in the future. 128 Unmodified RFC2544 tests MUST NOT be conducted on a production 129 network unless the tester is confident that the required degree of 130 resource issolation is in place such that the tests will not be 131 harmful to the network itself, other user traffic, or the performance 132 applications using the network as a data path. 134 4. Real World 136 The following text does not seem useful as it is self evident and 137 thus I propose to delete the section. 139 [2. Real world 141 In producing this document the authors attempted to keep in mind the 142 requirement that apparatus to perform the described tests must 143 actually be built. We do not know of "off the shelf" equipment 144 available to implement all of the tests but it is our opinion that 145 such equipment can be constructed.] 147 5. Test To Be Run 149 The advice in [RFC2544] Section 3 regarding the applicability and 150 usefulness of the tests applies to the situation considered in this 151 document 153 6. Evaluating the Results 155 As noted in RFC2544 [RFC2544] Section 4 the test will produce a great 156 deal of data, more so in the case when the tests are conducted on a 157 production network, since it is advisable to repeat the tests to 158 ensure that the results are statically representative of the 159 behaviour of the devices and network path at the time of the test. 160 In particular behavior such as latency variation or packet loss needs 161 to be properly characterised. Automation of the instumentation and 162 improvements in display and visualization since RFC2544 was written 163 should assist with this. 165 As noted in RFC2544 the selection of the to be run and evaluation of 166 the test data must therefore be done with an understanding of 167 generally accepted testing practices regarding repeatability, 168 variance, the tatistical significance of small numbers of trials and 169 the nature of the production network. 171 7. Requirements 173 An implementation is not compliant if it fails to satisfy one or more 174 of the MUST requirements for the protocols it implements. An 175 implementation that satisfies all the MUST and all the SHOULD 176 requirements for its protocols is said to be "unconditionally 177 compliant"; one that satisfies all the MUST requirements but not all 178 the SHOULD requirements for its protocols is said to be 179 "conditionally compliant". 181 8. The Test Set Up 183 There are two cases to be considered when setting up a test. The 184 first case is where the production network is providing an emulated 185 physical or datalink serice such as a pseudowir, and this service is 186 the subject of the test. This is similar to the case shown in Figure 187 2 of [RFC2544] , noting that either the service emulation will need 188 to be put in loopback, or a second service will need to be provsioned 189 to return the test traffic to the tester. 191 The second case concerns where it is desired to determine the 192 performance of a system that is handing off packets a packet 193 equipment such as a Router (or a lable switched router (LSR)), such 194 as when testing a virtual private network (VPN) service. This is 195 similar to the case shown in Figure 3 of RFC2544. 197 In both cases it should be noted that measurements taken will be 198 subject to imparement on both the outward path and the return path, 199 and that it is not possible to easily determine the degree of 200 imparement attributable to each path. It will be the case that the 201 path that is the subject of the test will, other than under 202 pathalogical conditions, exhibit a performance that is at least as 203 good as the measurements taken. 205 9. DUT set up 207 The advice in section 7 of [RFC2544] applies, with the exception that 208 protocols and protocol modes not supported by the production network 209 MUST NOT be configured or tested. The protocol profile tested MUST 210 be included or referenced in ant test report. 212 10. Frame Formats 214 The advice in Section 8 of [RFC2544]regarding Frame Formats applies. 215 Given the evolution of network technology since 1999, some additional 216 frame formats will be required, but these are out of scope for this 217 document. 219 11. Frame Sizes. 221 The advice in Section 9 of [RFC2544] regarding frame size should be 222 followed. 224 Since the publication of RFC2544 the maximum packet size supported on 225 Ethernet [anything else???] has increased with the introduction of 226 jumbo frames. The set of frame sizes tested SHOULD include the 227 maxium size supported by the network. The maximum size tested MUST 228 be included in the test report. This maximum size may be set by 229 configuration, or may be determined by the tester automatically using 230 a discovery technique. Such a discovery technique should preferable 231 be log n efficient such as a binary chop search. The discovered 232 maximum packet size should be verified a number of times (at least 233 five times is RECOMMENDED) until a consistant result is achieved. 235 12. Verifying received frames 237 The advice in Section 10 of [RFC2544] SHOULD be followed regarding 238 verification of frames 240 13. Modifiers 242 The advice in Section 10 of [RFC2544] SHOULD be followed regarding 243 verification of frames 245 [ We might want to update the following, although that really needs 246 to be aligned with more modern advice on the lab version of this 247 specification 249 The rest of section 13 to be deleted 251 It might be useful to know the DUT performance under a number of 252 conditions; some of these conditions are noted below. The reported 253 results SHOULD include as many of these conditions as the test 254 equipment is able to generate. The suite of tests SHOULD be first 255 run without any modifying conditions and then repeated under each of 256 the conditions separately. To preserve the ability to compare the 257 results of these tests any frames that are required to generate the 258 modifying conditions (management queries for example) will be 259 included in the same data stream as the normal test frames in place 260 of one of the test frames and not be supplied to the DUT on a 261 separate network port. 263 11.1 Broadcast frames 265 In most router designs special processing is required when frames 266 addressed to the hardware broadcast address are received. In bridges 267 (or in bridge mode on routers) these broadcast frames must be flooded 268 to a number of ports. The stream of test frames SHOULD be augmented 269 with 1% frames addressed to the hardware broadcast address. The 270 frames sent to the broadcast address should be of a type that the 271 router will not need to process. The aim of this test is to 272 determine if there is any effect on the forwarding rate of the other 273 data in the stream. The specific frames that should be used are 274 included in the test frame format document. The broadcast frames 275 SHOULD be evenly distributed throughout the data stream, for example, 276 every 100th frame. 278 The same test SHOULD be performed on bridge-like DUTs but in this 279 case the broadcast packets will be processed and flooded to all 280 outputs. 282 It is understood that a level of broadcast frames of 1% is much 283 higher than many networks experience but, as in drug toxicity 284 evaluations, the higher level is required to be able to gage the 285 effect which would otherwise often fall within the normal variability 286 of the system performance. Due to design factors some test equipment 287 will not be able to generate a level of alternate frames this low. 288 In these cases the percentage SHOULD be as small as the equipment can 289 provide and that the actual level be described in the report of the 290 test results. 292 11.2 Management frames 294 Most data networks now make use of management protocols such as SNMP. 295 In many environments there can be a number of management stations 296 sending queries to the same DUT at the same time. 298 The stream of test frames SHOULD be augmented with one management 299 query as the first frame sent each second during the duration of the 300 trial. The result of the query must fit into one response frame. 301 The response frame SHOULD be verified by the test equipment. One 302 example of the specific query frame that should be used is shown in 303 Appendix C. 305 11.3 Routing update frames 307 The processing of dynamic routing protocol updates could have a 308 significant impact on the ability of a router to forward data frames. 309 The stream of test frames SHOULD be augmented with one routing update 310 frame transmitted as the first frame transmitted during the trial. 312 Routing update frames SHOULD be sent at the rate specified in 313 Appendix C for the specific routing protocol being used in the test. 314 Two routing update frames are defined in Appendix C for the TCP/IP 315 over Ethernet example. The routing frames are designed to change the 316 routing to a number of networks that are not involved in the 317 forwarding of the test data. The first frame sets the routing table 318 state to "A", the second one changes the state to "B". The frames 319 MUST be alternated during the trial. 321 The test SHOULD verify that the routing update was processed by the 322 DUT. 324 11.4 Filters 326 Filters are added to routers and bridges to selectively inhibit the 327 forwarding of frames that would normally be forwarded. This is 328 usually done to implement security controls on the data that is 329 accepted between one area and another. Different products have 330 different capabilities to implement filters. 332 The DUT SHOULD be first configured to add one filter condition and 333 the tests performed. This filter SHOULD permit the forwarding of the 334 test data stream. In routers this filter SHOULD be of the form: 336 forward input_protocol_address to output_protocol_address 338 In bridges the filter SHOULD be of the form: 340 forward destination_hardware_address 342 The DUT SHOULD be then reconfigured to implement a total of 25 343 filters. The first 24 of these filters SHOULD be of the form: 345 block input_protocol_address to output_protocol_address 347 The 24 input and output protocol addresses SHOULD not be any that are 348 represented in the test data stream. The last filter SHOULD permit 349 the forwarding of the test data stream. By "first" and "last" we 350 mean to ensure that in the second case, 25 conditions must be checked 351 before the data frames will match the conditions that permit the 352 forwarding of the frame. Of course, if the DUT reorders the filters 353 or does not use a linear scan of the filter rules the effect of the 354 sequence in which the filters are input is properly lost. 356 The exact filters configuration command lines used SHOULD be included 357 with the report of the results. 359 11.4.1 Filter Addresses 361 Two sets of filter addresses are required, one for the single filter 362 case and one for the 25 filter case. 364 The single filter case should permit traffic from IP address 365 198.18.1.2 to IP address 198.19.65.2 and deny all other traffic. 367 The 25 filter case should follow the following sequence. 369 deny aa.ba.1.1 to aa.ba.100.1 deny aa.ba.2.2 to aa.ba.101.2 deny 370 aa.ba.3.3 to aa.ba.103.3 ... deny aa.ba.12.12 to aa.ba.112.12 allow 371 aa.bc.1.2 to aa.bc.65.1 deny aa.ba.13.13 to aa.ba.113.13 deny 372 aa.ba.14.14 to aa.ba.114.14 ... deny aa.ba.24.24 to aa.ba.124.24 deny 373 all else 375 All previous filter conditions should be cleared from the router 376 before this sequence is entered. The sequence is selected to test to 377 see if the router sorts the filter conditions or accepts them in the 378 order that they were entered. Both of these procedures will result 379 in a greater impact on performance than will some form of hash 380 coding.] 382 14. Protocol Addresses 384 Where the test traffic is fully issolated from production traffic, 385 for example when running over a PW, the advice in Section 12 of 386 [RFC2544] MAY be followed. 388 Where the test traffic shares the network with the production 389 traffic, the addresses used MUST be those that are correctly routed 390 to the designated test traffic receiver. Correct routing of this 391 traffic at a low data rate MUST be verified prior to running tests 392 that subject the test receiver to a significant load. 394 15. Route Set up 396 Where the test traffic is fully issolated from production traffic, 397 for example when running over a PW, the advice in Section 12 of 398 [RFC2544] MAY be followed. 400 Where the test traffic shares the network with production traffic, 401 the existing routing protocols SHOULD be used to set up the routed 402 path between the test traffic source and the test traffic 403 destination. 405 16. Bidirectional traffic 407 The advice on traffic bidirectionality in Section 14 of [RFC2544] 408 SHOULD be followed. 410 17. Single Stream Path 412 The advice on stream selection in Section 15 of [RFC2544] SHOULD be 413 followed. 415 18. Multi-port 417 The advice on exercising the ability of the DUT to concurrently 418 receive packets for the same destination port giveb in Section 16 of 419 [RFC2544] SHOULD be followed. 421 19. Multiple Protocols 423 The advice on multiple protocols given in Section 17 of [RFC2544] 424 SHOULD be followed. 426 20. Multiple frame sizes 428 The advice on multiple frame sizes given in Section 18 of [RFC2544] 429 SHOULD be followed. 431 21. Testing performance beyond a single DUT. 433 Section 19 of [RFC2544] discusses testing of multiple systems. This 434 is the normal case in a production network. As noted in RFC2544 such 435 tests require care in care in interpretation. Unlike the laboratory 436 benchmrk case however, the test will be exercising the network in the 437 expected configuration, and thus is representive of the opertion of 438 the system with the background traffic load of the time. It is 439 RECOMMENDED that the test be repeated a number of times to guage the 440 effect of variation of background traffic over both short and long 441 term time frames. The times of the test SHOULD be logged, and the 442 results presented in such a way that the statistical nature of the 443 test be clear to the reviewer. It should be noted whether the type 444 of production network was such that it would or not it would be 445 anticipated that the test would be repeatable within the statistical 446 significance of the measurements. 448 22. Maximum frame rate 450 The advice on maximum frame rate given in Section 20 of RFC2544 451 [RFC2544] SHOULD be followed. 453 23. Bursty Traffic 455 The advice in Section 21 of [RFC2544] on bursty traffic SHOULD be 456 followed. 458 Editor's Note Do we need to recomemnd larger burst sizes? 460 24. Rings 462 [This is probably obselete now - suggest we delete the section 464 Although it is possible to configure some token ring and FDDI 465 interfaces to transmit more than one frame each time that the token 466 is received, most of the network devices currently available transmit 467 only one frame per token. These tests SHOULD first be performed 468 while transmitting only one frame per token. 470 Some current high-performance workstation servers do transmit more 471 than one frame per token on FDDI to maximize throughput. Since this 472 may be a common feature in future workstations and servers, 473 interconnect devices with FDDI interfaces SHOULD be tested with 1, 4, 474 8, and 16 frames per token. The reported frame rate SHOULD be the 475 average rate of frame transmission over the total trial period.] 477 25. Trial description 479 If the prodcution network is providing a layer 1 or layer 2 service, 480 then the test may be conducted as described in Section 23 of 481 [RFC2544]. If the production network is providing a layer 3 service 482 care MUST be taken to ensure that any routes intruduced may be safely 483 announced by the DUT without causing disruption to production 484 traffic, and at such a volume that the route processors in the 485 production network are not overloaded. 487 26. Trial duration 489 The advice in Section 24 of [RFC2544] on trial duration is 490 applicable. 492 27. Address resolution 494 As stated in Section 25 of [RFC2544], the DUT SHOULD be able to 495 respond to address resolution requests sent by the DUT wherever the 496 protocol requires such a process. 498 28. Benchmarking tests 500 28.1. Throughput 502 The throughput test described in Section 26.1 of [RFC2544]applies. 503 However the reporting of a single number is NOT RECOMMENDED. If a 504 single is reported, it must be measured at a time during the busy 505 period, and the value of the lower decile SHOULD be reported. 507 28.2. Latency 509 The latency testing described in Section 26.1 of [RFC2544] applies. 510 Measurements SHOULD be carried out during the busy period. The 511 statistical averaging approach is for further study. 513 28.3. Frame loss rate 515 The frame loss proceedure described in Section 26.3 of [RFC2544] is 516 modified as follows: 518 The test should be repeated a number of times at each frame rate. 519 The number of repeats SHOULD be configured by the tester and reported 520 with the results. The default value is 10 (pulled out of a hat). 521 The exit criteria of loss free transmissions SHOULD be configured by 522 the tester and reported with the results. The default value is 10 523 (also pulled out of a hat). 525 28.4. Back-to-back frames 527 The advice on back-to-back frames provided in Section 26.4 of 528 [RFC2544] applies. 530 28.5. System Recovery 532 The advice on system recovery frames provided in Section 26.5 of 533 [RFC2544] applies. 535 28.6. Reset 537 The advice provided in Section 26.6 of [RFC2544] on reset SHOULD be 538 followed, except that the test MUST NOT be carried out on a DUT that 539 is being used for production traffic unless a specific decision is 540 made that disruption of the user traffic is acceptable. This test 541 MAY be carried out during the quiet period. 543 29. IANA considerations 545 There are no IANA considerations which arise from this document. 547 30. Security considerations 549 To be provided in a future version. 551 31. Acknowledgments 553 The Authors of [RFC2544] are acknowledged. 555 32. Informative References 557 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 558 Requirement Levels", BCP 14, RFC 2119, March 1997. 560 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 561 Network Interconnect Devices", RFC 2544, March 1999. 563 [RFC3985] Bryant, S. and P. Pate, "Pseudo Wire Emulation Edge-to- 564 Edge (PWE3) Architecture", RFC 3985, March 2005. 566 [RFC5921] Bocci, M., Bryant, S., Frost, D., Levrau, L., and L. 567 Berger, "A Framework for MPLS in Transport Networks", 568 RFC 5921, July 2010. 570 Appendix A. Testing Considerations 572 This Appendix will be deleted or updated in next version. Text is a 573 copy of Appendix A of [RFC2544] 575 A.1 Scope Of This Appendix 577 This appendix discusses certain issues in the benchmarking 578 methodology where experience or judgment may play a role in the tests 579 selected to be run or in the approach to constructing the test with a 580 particular DUT. As such, this appendix MUST not be read as an 581 amendment to the methodology described in the body of this document 582 but as a guide to testing practice. 584 1. Typical testing practice has been to enable all protocols to be 585 tested and conduct all testing with no further configuration of 586 protocols, even though a given set of trials may exercise only one 587 protocol at a time. This minimizes the opportunities to "tune" a DUT 588 for a single protocol. 590 2. The least common denominator of the available filter functions 591 should be used to ensure that there is a basis for comparison between 592 vendors. Because of product differences, those conducting and 593 evaluating tests must make a judgment about this issue. 595 3. Architectural considerations may need to be considered. For 596 example, first perform the tests with the stream going between ports 597 on the same interface card and the repeat the tests with the stream 598 going into a port on one interface card and out of a port on a second 599 interface card. There will almost always be a best case and worst 600 case configuration for a given DUT architecture. 602 4. Testing done using traffic streams consisting of mixed protocols 603 has not shown much difference between testing with individual 604 protocols. That is, if protocol A testing and protocol B testing 605 give two different performance results, mixed protocol testing 606 appears to give a result which is the average of the two. 608 5. Wide Area Network (WAN) performance may be tested by setting up 609 two identical devices connected by the appropriate short- haul 610 versions of the WAN modems. Performance is then measured between a 611 LAN interface on one DUT to a LAN interface on the other DUT. 613 The maximum frame rate to be used for LAN-WAN-LAN configurations is a 614 judgment that can be based on known characteristics of the overall 615 system including compression effects, fragmentation, and gross link 616 speeds. Practice suggests that the rate should be at least 110% of 617 the slowest link speed. Substantive issues of testing compression 618 itself are beyond the scope of this document.] 620 Appendix B. Maximum frame rates refer 622 This Appendix will be updated looks updated in next version. Text is 623 a copy of Appendix B of [RFC2544] 625 (Provided by Roger Beeman, Cisco Systems) 627 Size Ethernet 16Mb Token Ring FDDI (bytes) (pps) (pps) (pps) 629 64 14880 24691 152439 128 8445 13793 85616 256 4528 7326 45620 512 630 2349 3780 23585 768 1586 2547 15903 1024 1197 1921 11996 1280 961 631 1542 9630 1518 812 1302 8138 633 Ethernet size Preamble 64 bits Frame 8 x N bits Gap 96 bits 634 16Mb Token Ring size SD 8 bits AC 8 bits FC 8 bits DA 48 bits SA 48 635 bits RI 48 bits ( 06 30 00 12 00 30 ) SNAP DSAP 8 bits SSAP 8 bits 636 Control 8 bits Vendor 24 bits Type 16 bits Data 8 x ( N - 18) bits 637 FCS 32 bits ED 8 bits FS 8 bits 639 Tokens or idles between packets are not included 641 FDDI size Preamble 64 bits SD 8 bits FC 8 bits DA 48 bits SA 48 bits 642 SNAP 644 DSAP 8 bits SSAP 8 bits Control 8 bits Vendor 24 bits Type 16 bits 645 Data 8 x ( N - 18) bits FCS 32 bits ED 4 bits FS 12 bits 647 Appendix C. Test Frame Formats 649 The considerations provided in Apendix C of [RFC2544] apply. 651 The requirement for any new frame formats will be considered in a 652 future version. 654 Authors' Addresses 656 Ronald Bonica 657 Juniper Networks 659 Email: rbonica@juniper.net 661 Stewart Bryant 662 Cisco Systems 663 Green Park, 250, Longwater Avenue, 664 Reading RG2 6GB 665 UK 667 Email: stbryant@cisco.com