idnits 2.17.1 draft-ietf-bmwg-ca-bench-meth-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 14, 2011) is 4608 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 3511 (ref. '4') (Obsoleted by RFC 9411) -- Obsolete informational reference (is this intentional?): RFC 4566 (ref. '11') (Obsoleted by RFC 8866) -- Obsolete informational reference (is this intentional?): RFC 5226 (ref. '12') (Obsoleted by RFC 8126) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force M. Hamilton 3 Internet-Draft BreakingPoint Systems 4 Intended status: Informational S. Banks 5 Expires: March 17, 2012 Cisco Systems 6 September 14, 2011 8 Benchmarking Methodology for Content-Aware Network Devices 9 draft-ietf-bmwg-ca-bench-meth-00 11 Abstract 13 This document defines a set of test scenarios and metrics that can be 14 used to benchmark content-aware network devices. More specifically, 15 these scenarios are designed to most accurately predict performance 16 of these devices when subjected to relevant traffic patterns. This 17 document will operate within the constraints of the Benchmarking 18 Working Group charter, namely black box characterization in a 19 laboratory environment. 21 Status of this Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on March 17, 2012. 38 Copyright Notice 40 Copyright (c) 2011 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 56 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5 57 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 58 3. Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 3.1. Test Considerations . . . . . . . . . . . . . . . . . . . 6 60 3.2. Clients and Servers . . . . . . . . . . . . . . . . . . . 6 61 3.3. Traffic Generation Requirements . . . . . . . . . . . . . 6 62 3.4. Discussion of Network Mathematics . . . . . . . . . . . . 6 63 3.5. Framework for Traffic Specification . . . . . . . . . . . 8 64 3.6. Multiple Client/Server Testing . . . . . . . . . . . . . . 8 65 3.7. Device Configuration Considerations . . . . . . . . . . . 8 66 3.7.1. Network Addressing . . . . . . . . . . . . . . . . . . 8 67 3.7.2. Network Address Translation . . . . . . . . . . . . . 9 68 3.7.3. TCP Stack Considerations . . . . . . . . . . . . . . . 9 69 3.7.4. Other Considerations . . . . . . . . . . . . . . . . . 9 70 4. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . . . 9 71 4.1. Maximum Application Flow Rate . . . . . . . . . . . . . . 9 72 4.1.1. Objective . . . . . . . . . . . . . . . . . . . . . . 10 73 4.1.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 10 74 4.1.2.1. Application-Layer Parameters . . . . . . . . . . . 10 75 4.1.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 10 76 4.1.4. Measurement . . . . . . . . . . . . . . . . . . . . . 10 77 4.1.4.1. Maximum Application Flow Rate . . . . . . . . . . 10 78 4.1.4.2. Application Flow Duration . . . . . . . . . . . . 10 79 4.1.4.3. Packet Loss . . . . . . . . . . . . . . . . . . . 11 80 4.1.4.4. Application Flow Latency . . . . . . . . . . . . . 11 81 4.2. Application Throughput . . . . . . . . . . . . . . . . . . 11 82 4.2.1. Objective . . . . . . . . . . . . . . . . . . . . . . 11 83 4.2.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 11 84 4.2.2.1. Parameters . . . . . . . . . . . . . . . . . . . . 11 85 4.2.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 11 86 4.2.4. Measurement . . . . . . . . . . . . . . . . . . . . . 11 87 4.2.4.1. Maximum Throughput . . . . . . . . . . . . . . . . 11 88 4.2.4.2. Packet Loss . . . . . . . . . . . . . . . . . . . 12 89 4.2.4.3. Maximum Application Flow Rate . . . . . . . . . . 12 90 4.2.4.4. Application Flow Duration . . . . . . . . . . . . 12 91 4.2.4.5. Packet Loss . . . . . . . . . . . . . . . . . . . 12 92 4.2.4.6. Application Flow Latency . . . . . . . . . . . . . 12 93 4.3. Malicious Traffic Handling . . . . . . . . . . . . . . . . 12 94 4.3.1. Objective . . . . . . . . . . . . . . . . . . . . . . 12 95 4.3.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 12 96 4.3.2.1. Parameters . . . . . . . . . . . . . . . . . . . . 12 97 4.3.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 13 98 4.3.4. Measurement . . . . . . . . . . . . . . . . . . . . . 13 99 4.4. Malformed Traffic Handling . . . . . . . . . . . . . . . . 13 100 4.4.1. Objective . . . . . . . . . . . . . . . . . . . . . . 13 101 4.4.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 13 102 4.4.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 13 103 4.4.4. Measurement . . . . . . . . . . . . . . . . . . . . . 14 104 5. Appendix A: Example Test Case . . . . . . . . . . . . . . . . 14 105 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 106 7. Security Considerations . . . . . . . . . . . . . . . . . . . 16 107 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16 108 8.1. Normative References . . . . . . . . . . . . . . . . . . . 16 109 8.2. Informative References . . . . . . . . . . . . . . . . . . 17 110 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 112 1. Introduction 114 Content-aware and deep packet inspection (DPI) device deployments 115 have grown significantly in recent years. No longer are devices 116 simply using Ethernet and IP headers to make forwarding decisions. 117 This class of device now uses application-specific data to make these 118 decisions. For example, a web-application firewall (WAF) may use 119 search criteria upon the HTTP uniform resource indicator (URI)[1] to 120 decide whether a HTTP GET method may traverse the network. In the 121 case of lawful/legal intercept technology, a device could use the 122 phone number within the Session Description Protocol[11] to determine 123 whether a voice-over-IP phone may be allowed to connect. In addition 124 to the development of entirely new classes of devices, devices that 125 could historically be classified as 'stateless' or raw forwarding 126 devices are now performing DPI functionality. Devices such as core 127 and edge routers are now being developed with DPI functionality to 128 make more intelligent routing and forwarding decisions. 130 The Benchmarking Working Group (BMWG) has historically produced 131 Internet Drafts and Requests for Comment that are focused 132 specifically on creating output metrics that are derived from a very 133 specific and well-defined set of input parameters that are completely 134 and unequivocally reproducible from test bed to test bed. The end 135 goal of such methodologies is to, in the words of the RFC 2544 [2], 136 reduce "specsmanship" in the industry. Existing BMWG work has 137 certainly met this stated goal. 139 The BMWG has historically avoided the use of the term "realistic" 140 throughout all of its drafts and RFCs. While this document will not 141 explicitly use this term, the end goal of the terminology and 142 methodology is to generate performance metrics that will be as close 143 as possible to equivalent metrics in a production environment. It 144 should be further noted than any metrics acquired from a production 145 network SHOULD be captured according to the policies and procedures 146 of the IPPM or PMOL working groups. 148 An explicit non-goal of this document is to replace existing 149 methodology/terminology pairs such as RFC 2544 [2]/RFC 1242 [3] or 150 RFC 3511 [4]/RFC 2647 [5]. The explicit goal of this document is to 151 create a methodology and terminology pair that is more suited for 152 modern devices while complementing the data acquired using existing 153 BMWG methodologies. Existing BMWG work generally relies on 154 completely repeatable input stimulus, expecting fully repeatable 155 output. For unicast UDP streams, this makes complete sense. This 156 document does not assume completely repeatable input stimulus. The 157 nature of application-driven networks is such that a single dropped 158 packet inherently changes the input stimulus from a network 159 perspective. While application flows will be specified in great 160 detail, it simply is not practical to require totally repeatable 161 input stimulus. 163 1.1. Requirements Language 165 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 166 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 167 document are to be interpreted as described in RFC 2119 [6]. 169 2. Scope 171 Content-aware devices take many forms, shapes and architectures. 172 These devices are advanced network interconnect devices that inspect 173 deep into the application payload of network data packets to do 174 classification. They may be as simple as a firewall that uses 175 application data inspection for rule set enforcement, or they may 176 have advanced functionality such as performing protocol decoding and 177 validation, anti-virus, anti-spam and even application exploit 178 filtering. 180 This document is strictly focused on examining performance and 181 robustness across a focused set of metrics: throughput(min/max/avg/ 182 sample std dev), transaction rates(successful/failed), application 183 response times, concurrent flows, and unidirectional packet latency. 184 None of the metrics captured through this methodology are specific to 185 a device, nor do they characterize the functional behavior of those 186 devices. The metrics are implementation independent. Functional 187 testing of the DUT is outside the scope of this methodology. 189 Devices such as firewalls, intrusion detection and prevention 190 devices, application delivery controllers, deep packet inspection 191 devices, wide-area network(WAN) optimization devices, and unified 192 threat management systems generally fall into the content-aware 193 category. While this list may become obsolete, these are a subset of 194 devices that fall under this scope of testing. 196 3. Test Setup 198 This document will be applicable to most test configurations and will 199 not be confined to a discussion on specific test configurations. 200 Since each DUT/SUT will have their own unique configuration, users 201 SHOULD configure their device with the same parameters that would be 202 used in the actual deployment of the device or a typical deployment, 203 if the actual deployment is unknown. In order to improve 204 repeatability, the DUT configuration SHOULD be published with the 205 final benchmarking results. If available, command-line scripts used 206 to configured the DUT and any configuration information for the 207 tester SHOULD be published with the final results 209 3.1. Test Considerations 211 3.2. Clients and Servers 213 Content-aware device testing SHOULD involve multiple clients and 214 multiple servers. As with RFC 3511 [4], this methodology will use 215 the terms virtual clients/servers because both the client and server 216 will be represented by the tester and not actual clients/servers. 217 Similarly defined in RFC 3511 [4], a data source may emulate multiple 218 clients and/or servers within the context of the same test scenario. 219 The test report SHOULD indicate the number of virtual clients/servers 220 used during the test. IANA has reserved address ranges for 221 laboratory characterization. These are defined for IPv4 and IPv6 by 222 RFC 2544 Appendix C [2] and RFC 5180 Section 5.2 [7] respectively and 223 SHOULD be consulted prior to testing. 225 3.3. Traffic Generation Requirements 227 The explicit purposes of content-aware devices vary widely, but these 228 devices use information deeper inside the application flow to make 229 decisions and classify traffic. This methodology will utilize 230 traffic flows that resemble real application traffic without 231 utilizing captures from live production networks. Application Flows, 232 as defined in RFC 2722 [8] are able to be well-defined without simply 233 referring to a network capture. An example traffic template is 234 defined and listed in Section 5 of this document. A user of this 235 methodology is free to utilize the example mix as provided in the 236 appendix. If a user of this methodology understands the traffic 237 patterns in their production network, that user SHOULD use the 238 template provided in Section 5 to describe a traffic mix appropriate 239 for their environment. 241 The test tool SHOULD be able to create application flows between 242 every client and server, regardless of direction. The tester SHOULD 243 be able to open TCP connections on multiple destination ports and 244 SHOULD be able to direct UDP traffic to multiple destination ports. 246 3.4. Discussion of Network Mathematics 248 Prior to executing the methodology as outlined in the following 249 sections, it is imperative to understand the implications of 250 utilizing representative application flows for the traffic content of 251 the benchmarking effort. One interesting aspect of utilizing 252 application flows is that each flow is inherently different from 253 every other application flow. The content of each flow will vary 254 from application to application, and in most cases, even varies 255 within the same type of application flow. The following description 256 of the methodology will individually benchmark every individual type 257 and subset of application flow, prior to performing similar tests 258 with a traffic mix as specified either by the example mix in 259 Section 5, or as defined by the user of this methodology. 261 The purpose of this process is to ensure that any performance 262 implications that are discovered during the mixed testing aren't due 263 to the inherent physical network limitations. As an example of this 264 phenomena, it is useful to examine a network device inserted into a 265 single path, as illustrated in the following diagram. 267 +----------+ 268 +---+ 1gE | DUT/ | 1gE +---+ 269 |C/S|------| SUT |------|C/S| 270 +---+ +----------+ +---+ 272 Simple Inline DUT Configuration 274 Figure 1: Single Path Example 276 For the purpose of this discussion, let's take a theoretical 277 application flow that utilizes UDP for the transport layer. Assume 278 that the sample transaction we will be using to model this particular 279 flow requires 10 UDP datagrams to complete the transaction. For 280 simplicity, each datagram within the flow is exactly 64 bytes, 281 including associated Ethernet, IP, and UDP overhead. With any 282 network device,there are always three metrics which interact with 283 each other: number of concurrent application flows, number of 284 application flows per second, and layer-7 throughput. 286 Our example test bed is a single-path device connected with 1 gigabit 287 Ethernet links. The purpose of this benchmark effort is to quantify 288 the number of application flows per second that may be processed 289 through our device under test. Let's assume that the result from our 290 scenario is that the DUT is able to process 10,000 application flows 291 per second. The question is whether that ceiling is the actual 292 ceiling of the device, or if it is actually being limited by one of 293 the other metrics. If we do the appropriate math, 10000 flows per 294 second, with each flow at 640 total bytes means that we are achieving 295 a throughput of roughly 49 Mbps. This is dramatically less than the 296 1 gigabit physical link we are using. We can conclude that 10,000 297 flows per second is in fact the performance limit of the device. 299 If we change the example slightly and increase the size of each 300 datagram to 1312 bytes, then it becomes necessary to recompute the 301 math. Assuming the same observed DUT limitation of 10,000 flows per 302 second, it must be ensured that this is an artifact of the DUT, and 303 not of physical limitations. For each flow, we'll require 104,960 304 bits. 10,000 flows per second implies a throughput of roughly 1 Gbps. 305 At this point, we cannot definitively answer whether the DUT is 306 actually limited to 10,000 flows per second. If we are able to 307 modify the scenario, and utilize 10 Gigabit interfaces, then perhaps 308 the flow per second ceiling will be reached at a higher number than 309 10,000. 311 This example illustrates why a user of this methodology SHOULD 312 benchmark each application variant individually to ensure that the 313 cause of a measured limit is fully understood 315 3.5. Framework for Traffic Specification 317 The following table SHOULD be specified for each application flow 318 variant. 320 o Flow Size in Bits 322 o Percentage of Aggregate Flows: 25% 324 o Transport Protocol(s): TCP,UDP 326 o Destination Port(s): 80 328 3.6. Multiple Client/Server Testing 330 In actual network deployments, connections are being established 331 between multiple clients and multiple servers simultaneously. Device 332 vendors have been known to optimize the operation of their devices 333 for easily defined patterns. The connection sequence ordering 334 scenarios a device will see on a network will likely be much less 335 deterministic. In fact, many application flows have multiple layer 4 336 connections within a single flow, with client and server reversing 337 roles. This methodology makes no assumptions about flow initiation 338 sequence across multiple ports. 340 3.7. Device Configuration Considerations 342 The configuration of the DUT may have an effect on the observed 343 results of the following methodology. A comprehensive, but certainly 344 not exhaustive, list of potential considerations is listed below. 346 3.7.1. Network Addressing 348 The IANA has issued a range of IP addresses to the BMWG for purposes 349 of benchmarking. Please refer to RFC 2544 [2] and RFC 5180 [7] for 350 more details. 352 3.7.2. Network Address Translation 354 Many content-aware devices are capable of performing Network Address 355 Translation (NAT)[5]. If the final deployment of the DUT will have 356 this functionality enabled, then the DUT SHOULD also have it enabled 357 during the execution of this methodology. It MAY be beneficial to 358 perform the test series in both modes in order to determine the 359 performance differential when using NAT. The test report SHOULD 360 indicate whether NAT was enabled during the testing process. 362 3.7.3. TCP Stack Considerations 364 The IETF has historically provided guidance and information on TCP 365 stack considerations. This methodology is strictly focused on 366 performance metrics at layers above 4, thus does not specifically 367 define any TCP stack configuration parameters of either the tester or 368 the DUTs. The TCP configuration of the tester SHOULD remain constant 369 across all DUTs in order to ensure comparable results. While the 370 following list of references is not exhaustive, each document 371 contains a relevant discussion on TCP stack considerations. 373 Congestion control algorithms are discussed in Section 2 of RFC 3148 374 [9] with even more detailed references. TCP receive and congestion 375 window sizes are discussed in detail in RFC 6349 [10]. 377 3.7.4. Other Considerations 379 Various content-aware devices will have widely varying feature sets. 380 In the interest of representative test results, the DUT features that 381 will likely be enabled in the final deployment SHOULD be used. This 382 methodology is not intended to advise on which features should be 383 enabled, but to suggest using actual deployment configurations. 385 4. Benchmarking Tests 387 Each of the following benchmark scenarios SHOULD be run with each of 388 the single application flow templates. Upon completion of all 389 iterations, the mixed test SHOULD be completed, subject to the 390 traffic mix as defined by the user. 392 4.1. Maximum Application Flow Rate 393 4.1.1. Objective 395 To determine the maximum rate through which a device is able to 396 establish and complete application flows as defined by 397 draft-ietf-bmwg-ca-bench-term-00. 399 4.1.2. Setup Parameters 401 The following parameters SHOULD be used and reported for all tests: 403 4.1.2.1. Application-Layer Parameters 405 For each application protocol in use during the test run, the table 406 provided in Section 3.5 SHOULD be published. 408 4.1.3. Procedure 410 The test SHOULD generate application network traffic that meets the 411 conditions of Section 3.3. The traffic pattern SHOULD begin with an 412 application flow rate of 10% of expected maximum. The test SHOULD be 413 configured to increase the attempt rate in units of 10% up through 414 110% of expected maximum. The duration of each loading phase SHOULD 415 be at least 30 seconds. This test MAY be repeated, each subsequent 416 iteration beginning at 5% of expected maximum and increasing session 417 establishment rate to 10% more than the maximum observed from the 418 previous test run. 420 This procedure MAY be repeated any number of times with the results 421 being averaged together. 423 4.1.4. Measurement 425 The following metrics MAY be determined from this test, and SHOULD be 426 observed for each application protocol within the traffic mix: 428 4.1.4.1. Maximum Application Flow Rate 430 The test tool SHOULD report the maximum rate at which application 431 flows were completed, as defined by RFC 2647 [5], Section 3.7. This 432 rate SHOULD be reported individually for each application protocol 433 present within the traffic mix. 435 4.1.4.2. Application Flow Duration 437 The test tool SHOULD report the minimum, maximum and average 438 application duration, as defined by RFC 2647 [5], Section 3.9. This 439 duration SHOULD be reported individually for each application 440 protocol present within the traffic mix. 442 4.1.4.3. Packet Loss 444 The test tool SHOULD report the number of flow packets lost or 445 dropped from source to destination. 447 4.1.4.4. Application Flow Latency 449 The test tool SHOULD report the minimum, maximum and average amount 450 of time an application flow member takes to traverse the DUT, as 451 defined by RFC 1242 [3], Section 3.13. This rate SHOULD be reported 452 individually for each application protocol present within the traffic 453 mix. 455 4.2. Application Throughput 457 4.2.1. Objective 459 To determine the maximum rate through which a device is able to 460 forward bits when using application flows as defined in the previous 461 sections. 463 4.2.2. Setup Parameters 465 The following parameters SHOULD be used and reported for all tests: 467 4.2.2.1. Parameters 469 The same parameters as described in Section 4.1.2 SHOULD be used. 471 4.2.3. Procedure 473 This test will attempt to send application flows through the device 474 at a flow rate of 30% of the maximum, as observed in Section 4.1. 475 This procedure MAY be repeated with the results from each iteration 476 averaged together. 478 4.2.4. Measurement 480 The following metrics MAY be determined from this test, and SHOULD be 481 observed for each application protocol within the traffic mix: 483 4.2.4.1. Maximum Throughput 485 The test tool SHOULD report the minimum, maximum and average 486 application throughput. 488 4.2.4.2. Packet Loss 490 The test tool SHOULD report the number of network packets lost or 491 dropped from source to destination. 493 4.2.4.3. Maximum Application Flow Rate 495 The test tool SHOULD report the maximum rate at which application 496 flows were completed, as defined by RFC 2647 [5], Section 3.7. This 497 rate SHOULD be reported individually for each application protocol 498 present within the traffic mix. 500 4.2.4.4. Application Flow Duration 502 The test tool SHOULD report the minimum, maximum and average 503 application duration, as defined by RFC 2647 [5], Section 3.9. This 504 duration SHOULD be reported individually for each application 505 protocol present within the traffic mix. 507 4.2.4.5. Packet Loss 509 The test tool SHOULD report the number of flow packets lost or 510 dropped from source to destination. 512 4.2.4.6. Application Flow Latency 514 The test tool SHOULD report the minimum, maximum and average amount 515 of time an application flow member takes to traverse the DUT, as 516 defined by RFC 1242 [3], Section 3.13. This rate SHOULD be reported 517 individually for each application protocol present within the traffic 518 mix. 520 4.3. Malicious Traffic Handling 522 4.3.1. Objective 524 To determine the effects on performance that malicious traffic may 525 have on the DUT. While this test is not designed to characterize 526 accuracy of detection or classification, it MAY be useful to record 527 these measurements as specified below. 529 4.3.2. Setup Parameters 531 4.3.2.1. Parameters 533 The same parameters as described in Section 4.1.2 SHOULD be used. 535 Additionally, the following parameters SHOULD be used and reported 536 for all tests: 538 o Attack List: A listing of the malicious traffic that was generated 539 by the test. 541 4.3.3. Procedure 543 This test will utilize the procedures specified previously in 544 Section 4.1.3 and Section 4.2.3. When performing the procedures 545 listed previously, the tester should generate malicious traffic 546 representative of the final network deployment. The mix of attacks 547 MAY include software vulnerability exploits, network worms, back-door 548 access attempts, network probes and other malicious traffic. 550 If a DUT can be run with and without the attack mitigation, both 551 procedures SHOULD be run with and without the feature enabled on the 552 DUT to determine the affects of the malicious traffic on the baseline 553 metrics previously derived. If a DUT does not have active attack 554 mitigation capabilities, this procedure SHOULD be run regardless. 555 Certain malicious traffic could affect device performance even if the 556 DUT does not actively inspect packet data for malicious traffic. 558 4.3.4. Measurement 560 The metrics specified by Section 4.1.4 and Section 4.2.4 SHOULD be 561 determined from this test. 563 4.4. Malformed Traffic Handling 565 4.4.1. Objective 567 To determine the effects on performance and stability that malformed 568 traffic may have on the DUT. 570 4.4.2. Setup Parameters 572 The same parameters SHOULD be used for Transport-Layer and 573 Application Layer Parameters previously specified in Section 4.1.2 574 and Section 4.2.2. 576 4.4.3. Procedure 578 This test will utilize the procedures specified previously in 579 Section 4.1.3 and Section 4.2.3. When performing the procedures 580 listed previously, the tester should generate malformed traffic at 581 all protocol layers. This is commonly known as fuzzed traffic. 582 Fuzzing techniques generally modify portions of packets, including 583 checksum errors, invalid protocol options, and improper protocol 584 conformance. This test SHOULD be run on a DUT regardless of whether 585 it has built-in mitigation capabilities. 587 4.4.4. Measurement 589 For each protocol present in the traffic mix, the metrics specified 590 by Section 4.1.4 and Section 4.2.4 MAY be determined. This data may 591 be used to ascertain the effects of fuzzed traffic on the DUT. 593 5. Appendix A: Example Test Case 595 This appendix shows an example case of a protocol mix that may be 596 used with this methodology. 598 +---------------------------+-----------------------+-------------+ 599 | Application Flow | Options | Value | 600 +---------------------------+-----------------------+-------------+ 601 | Web 1kB | | | 602 | | Flow Size (L7) | 1kB | 603 | | Flow Percentage | 15% | 604 | | Transport Protocol(s) | TCP | 605 | | Destination Port(s) | 80 | 606 | Web 10kB | | | 607 | | Flow Size (L7) | 10kB | 608 | | Flow Percentage | 15% | 609 | | Transport Protocol(s) | TCP | 610 | | Destination Port(s) | 80 | 611 | Web 100kB | | | 612 | | Flow Size (L7) | 100kB | 613 | | Flow Percentage | 15% | 614 | | Transport Protocol(s) | TCP | 615 | | Destination Port(s) | 80 | 616 | BitTorrent Movie Download | | | 617 | | Flow Size (L7) | 500 MB | 618 | | Flow Percentage | 5% | 619 | | Transport Protocol(s) | TCP | 620 | | Destination Port(s) | 6881-6889 | 621 | SMTP Email | | | 622 | | Flow Size (L7) | 50 kB | 623 | | Flow Percentage | 10% | 624 | | Transport Protocol(s) | TCP | 625 | | Destination Port(s) | 25 | 626 | IMAP Email | | | 627 | | Flow Size (L7) | 100 kB | 628 | | Flow Percentage | 15% | 629 | | Transport Protocol(s) | TCP | 630 | | Destination Port(s) | 143 | 631 | DNS | | | 632 | | Flow Size (L7) | 2 kB | 633 | | Flow Percentage | 10% | 634 | | Transport Protocol(s) | UDP | 635 | | Destination Port(s) | 53 | 636 | RTP | | | 637 | | Flow Size (L7) | 100 MB | 638 | | Flow Percentage | 10% | 639 | | Transport Protocol(s) | UDP | 640 | | Destination Port(s) | 20000-65535 | 641 +---------------------------+-----------------------+-------------+ 643 Table 1: Sample Traffic Pattern 645 6. IANA Considerations 647 This memo includes no request to IANA. 649 All drafts are required to have an IANA considerations section (see 650 the update of RFC 2434 [12] for a guide). If the draft does not 651 require IANA to do anything, the section contains an explicit 652 statement that this is the case (as above). If there are no 653 requirements for IANA, the section will be removed during conversion 654 into an RFC by the RFC Editor. 656 7. Security Considerations 658 Benchmarking activities as described in this memo are limited to 659 technology characterization using controlled stimuli in a laboratory 660 environment, with dedicated address space and the other constraints 661 RFC 2544 [2]. 663 The benchmarking network topology will be an independent test setup 664 and MUST NOT be connected to devices that may forward the test 665 traffic into a production network, or misroute traffic to the test 666 management network 668 8. References 670 8.1. Normative References 672 [1] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 673 Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, 674 January 2005. 676 [2] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 677 Network Interconnect Devices", RFC 2544, March 1999. 679 [3] Bradner, S., "Benchmarking terminology for network 680 interconnection devices", RFC 1242, July 1991. 682 [4] Hickman, B., Newman, D., Tadjudin, S., and T. Martin, 683 "Benchmarking Methodology for Firewall Performance", RFC 3511, 684 April 2003. 686 [5] Newman, D., "Benchmarking Terminology for Firewall 687 Performance", RFC 2647, August 1999. 689 [6] Bradner, S., "Key words for use in RFCs to Indicate Requirement 690 Levels", BCP 14, RFC 2119, March 1997. 692 [7] Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin, 693 "IPv6 Benchmarking Methodology for Network Interconnect 694 Devices", RFC 5180, May 2008. 696 [8] Brownlee, N., Mills, C., and G. Ruth, "Traffic Flow 697 Measurement: Architecture", RFC 2722, October 1999. 699 [9] Mathis, M. and M. Allman, "A Framework for Defining Empirical 700 Bulk Transfer Capacity Metrics", RFC 3148, July 2001. 702 [10] Constantine, B., Forget, G., Geib, R., and R. Schrage, 703 "Framework for TCP Throughput Testing", RFC 6349, August 2011. 705 8.2. Informative References 707 [11] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 708 Description Protocol", RFC 4566, July 2006. 710 [12] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA 711 Considerations Section in RFCs", BCP 26, RFC 5226, May 2008. 713 Authors' Addresses 715 Mike Hamilton 716 BreakingPoint Systems 717 Austin, TX 78717 718 US 720 Phone: +1 512 636 2303 721 Email: mhamilton@breakingpoint.com 723 Sarah Banks 724 Cisco Systems 725 San Jose, CA 95134 726 US 728 Email: sabanks@cisco.com