idnits 2.17.1 draft-morton-bmwg-imix-genome-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 7, 2011) is 4675 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational July 7, 2011 5 Expires: January 8, 2012 7 IMIX Genome: Specification of variable packet sizes for additional 8 testing 9 draft-morton-bmwg-imix-genome-02 11 Abstract 13 Benchmarking Methodologies have always relied on test conditions with 14 constant packet sizes, with the goal of understanding what network 15 device capability has been tested. Tests with constant packet size 16 reveal device capabilities but differ significantly from the 17 conditions encountered in operational deployment, and so additional 18 tests are sometimes conducted with a mixture of packet sizes, or 19 "IMIX". The mixture of sizes a networking device will encounter is 20 highly variable and depends on many factors. An IMIX suited for one 21 networking device and deployment will not be appropriate for another. 22 However, the mix of sizes may be known and the tester may be asked to 23 augment the fixed size tests. To address this need, and the 24 perpetual goal of specifying repeatable test conditions, this draft 25 defines a way to specify the exact repeating sequence of packet sizes 26 from the usual set of fixed sizes, and other forms of mixed size 27 specification. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in RFC 2119 [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 50 This Internet-Draft will expire on January 8, 2012. 52 Copyright Notice 54 Copyright (c) 2011 IETF Trust and the persons identified as the 55 document authors. All rights reserved. 57 This document is subject to BCP 78 and the IETF Trust's Legal 58 Provisions Relating to IETF Documents 59 (http://trustee.ietf.org/license-info) in effect on the date of 60 publication of this document. Please review these documents 61 carefully, as they describe your rights and restrictions with respect 62 to this document. Code Components extracted from this document must 63 include Simplified BSD License text as described in Section 4.e of 64 the Trust Legal Provisions and are provided without warranty as 65 described in the Simplified BSD License. 67 Table of Contents 69 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . . 4 71 3. Specification of the IMIX Genome . . . . . . . . . . . . . . . 5 72 4. Specification of a Custom IMIX . . . . . . . . . . . . . . . . 6 73 5. Reporting Long or Pseudo-Random Packet Sequences . . . . . . . 7 74 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 7 75 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8 76 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 8 77 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 78 9.1. Normative References . . . . . . . . . . . . . . . . . . . 8 79 9.2. Informative References . . . . . . . . . . . . . . . . . . 8 80 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 9 82 1. Introduction 84 This memo defines a method to unambiguously specify the sequence of 85 packet sizes used in a load test. 87 Benchmarking Methodologies [RFC2544] have always relied on test 88 conditions with constant packet sizes, with the goal of understanding 89 what network device capability has been tested. Tests with the 90 smallest size stress the header processing capacity, and tests with 91 the largest size stress the overall bit processing capacity. Tests 92 with sizes in-between may determine the transition between these two 93 capacities. 95 Streams of constant packet size differ significantly from the 96 conditions encountered in operational deployment, and so additional 97 tests are sometimes conducted with a mixture of packet sizes. The 98 set of sizes used is often called an Internet Mix, or "IMIX" 99 [Spirent], [IXIA], [Agilent]. 101 The mixture of sizes a networking device will encounter is highly 102 variable and depends on many factors. An IMIX suited for one 103 networking device and deployment will not be appropriate for another. 104 However, the mix of sizes may be known and the tester may be asked to 105 augment the fixed size tests. The references above cite the original 106 studies and their methodologies - similar methods can be used to 107 determine new size mixes. 109 To address this need, and the perpetual goal of specifying repeatable 110 test conditions, this draft proposes a way to specify the exact 111 repeating sequence of packet sizes from the usual set of fixed sizes: 112 the IMIX Genome. Other, less exact forms of size specification are 113 also recommended for extremely complicated or customized size mixes. 115 This memo takes the position that it cannot be proven for all 116 circumstances that the sequence of packet sizes does not affect the 117 test result, thus a standardized specification of sequence is 118 valuable. 120 2. Scope and Goals 122 This memo defines a method to unambiguously specify the sequence of 123 packet sizes that have been used in a load test, assuming that a 124 relevant mix of sizes is known to the tester and the length of the 125 repeating sequence is not very long (<100 packets). 127 The IMIX Genome will allow an exact sequence of packet sizes to be 128 communicated as a single-line name, resolving the current ambiguity 129 with results that simply refer to "IMIX". 131 While documentation of the exact sequence is ideal, the memo also 132 covers the case where the sequence of sizes is very long or may be 133 generated by a pseudo-random process. 135 It is a colossal non-goal to standardize one or more versions of the 136 IMIX. This topic has been discussed on many occasions on the bmwg- 137 list[IMIXonList]. The goal is to enable customization with minimal 138 constraints while fostering repeatable testing once the fixed size 139 testing is complete. 141 3. Specification of the IMIX Genome 143 The IMIX Genome is specified in the following format: 145 IMIX - 123456...x 147 where each number is replaced by the letter corresponding to the size 148 of the packet at that position in the sequence. The following table 149 gives the letter encoding for the [RFC2544] standard sizes (64, 128, 150 256, 512, 1024, 1280, and 1518 bytes) and "jumbo" sizes (2112, 9000, 151 16000). Note that the 4 octet Ethernet frame check sequence may fail 152 to detect bit errors in the larger jumbo frames, see [jumbo]. 154 +-------------+--------------------+ 155 | Size, bytes | Genome Code Letter | 156 +-------------+--------------------+ 157 | 64 | a | 158 | 128 | b | 159 | 256 | c | 160 | 512 | d | 161 | 1024 | e | 162 | 1280 | f | 163 | 1518 | g | 164 | 2112 | h | 165 | 9000 | i | 166 | 16000 | j | 167 | MTU | z | 168 +-------------+--------------------+ 170 For example: a five packet sequence with sizes 64,64,64,1280,1518 171 would be designated: 173 IMIX - aaafg 175 While this approach allows some flexibility, there are also 176 constraints. 178 o Non-RFC2544 packet sizes would need to be approximated by those 179 available in the table. 181 o The Genome for very long sequences can become undecipherable by 182 humans. 184 o z=MTU is seen as valuable (so far). 186 o Whether more tabulated packet sizes would be useful is TBD, and 187 "jumbo" sizes were added in this version. 189 Some questions testers must ask and answer when using the IMIX Genome 190 are: 192 1. Multiple Source-Destination Address Pairs: is the IMIX sequence 193 applicable to each pair, across multiple pairs in sets, or across 194 all pairs? 196 2. Multiple Tester Ports:is the IMIX sequence applicable to each 197 port, across multiple ports in sets, or across all ports? 199 The chosen configuration would be expressed the following general 200 form: 202 +-----------------------+-------------------------+-----------------+ 203 | Source | Destination | Corresponding | 204 | Address/Port/Blade | Address/Port/Blade | IMIX | 205 +-----------------------+-------------------------+-----------------+ 206 | x.x.x.x Blade2 | y.y.y.y Blade3 | IMIX - aaafg | 207 +-----------------------+-------------------------+-----------------+ 209 where testers can specify the IMIX used between any two entities in 210 the test architecture. 212 4. Specification of a Custom IMIX 214 The Custom IMIX is specified in the following format: 216 CUSTOM IMIX - 123456...x 218 where each number is replaced by the letter corresponding to the size 219 of the packet at that position in the sequence. The tester MUST 220 complete the following table, giving the letter encoding for each 221 size used, where each set of three lower-case letters would be 222 replaced by the integer size in octets. 224 +-------------+--------------------+ 225 | Size, bytes | Custom Code Letter | 226 +-------------+--------------------+ 227 | aaa | A | 228 | bbb | B | 229 | ccc | C | 230 | ddd | D | 231 | eee | E | 232 | fff | F | 233 | ggg | G | 234 | etc. | up to Z | 235 +-------------+--------------------+ 237 For example: a five packet sequence with sizes aaa,aaa,aaa,ggg,ggg 238 would be designated: 240 CUSTOM IMIX - AAAGG 242 5. Reporting Long or Pseudo-Random Packet Sequences 244 When the IMIX-Genome cannot be used (when the sheer length of the 245 sequence would make the genome unmanageable) or when the sequence is 246 designed to vary within some proportional constraints, a table is 247 necessary. 249 +-----------+---------------------+-----------------+ 250 | IP Length | Percentage of Total | Other Length(s) | 251 +-----------+---------------------+-----------------+ 252 | 64 | 23 | 82 | 253 | 128 | 67 | 146 | 254 | 1000 | 10 | 1018 | 255 +-----------+---------------------+-----------------+ 257 Note that this approach also allows non-standard packet sizes, but 258 trades the short genome specification and ability to specify the 259 exact sequence for other flexibilities. 261 6. Security Considerations 263 Benchmarking activities as described in this memo are limited to 264 technology characterization using controlled stimuli in a laboratory 265 environment, with dedicated address space and the other constraints 266 [RFC2544]. 268 The benchmarking network topology will be an independent test setup 269 and MUST NOT be connected to devices that may forward the test 270 traffic into a production network, or misroute traffic to the test 271 management network. 273 Further, benchmarking is performed on a "black-box" basis, relying 274 solely on measurements observable external to the DUT/SUT. 276 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 277 benchmarking purposes. Any implications for network security arising 278 from the DUT/SUT SHOULD be identical in the lab and in production 279 networks. 281 7. IANA Considerations 283 This memo makes no requests of IANA, and hopes that IANA will leave 284 it alone as well. 286 8. Acknowledgements 288 Thanks to Sarah Banks, Aamer Akhter, and Steve Maxwell for their 289 review and comments. 291 9. References 293 9.1. Normative References 295 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 296 Requirement Levels", BCP 14, RFC 2119, March 1997. 298 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 299 Network Interconnect Devices", RFC 2544, March 1999. 301 9.2. Informative References 303 [Agilent] http://www.ixiacom.com/pdfs/test_plans/ 304 agilent_journal_of_internet_test_methodologies.pdf, "The 305 Journal of Internet Test Methodologies", 2007. 307 [IMIXonList] 308 http://www.ietf.org/mail-archive/web/bmwg/current/ 309 msg00691.html, "Discussion on IMIX", 2003. 311 [IXIA] http://www.ixiacom.com/library/test_plans/ 312 display?skey=testing_pppox, "Library: Test Plans", 2010. 314 [Spirent] http://gospirent.com/whitepaper/ 315 IMIX%20Test%20Methodolgy%20Journal.pdf, "Test Methodology 316 Journal: IMIX (Internet Mix) Journal", 2006. 318 [jumbo] http://www.ietf.org/mail-archive/web/bmwg/current/ 319 msg00691.html, "Discussion of Jumbo Packets and FCS 320 Failure, see http://sd.wareonearth.com/~phil/jumbo.html 321 and http://staff.psc.edu/mathis/MTU/arguments.html#crc". 323 Author's Address 325 Al Morton 326 AT&T Labs 327 200 Laurel Avenue South 328 Middletown,, NJ 07748 329 USA 331 Phone: +1 732 420 1571 332 Fax: +1 732 368 1192 333 Email: acmorton@att.com 334 URI: http://home.comcast.net/~acmacm/