idnits 2.17.1 draft-ietf-bmwg-imix-genome-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 3, 2013) is 3973 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational June 3, 2013 5 Expires: December 5, 2013 7 IMIX Genome: Specification of variable packet sizes for additional 8 testing 9 draft-ietf-bmwg-imix-genome-05 11 Abstract 13 Benchmarking Methodologies have always relied on test conditions with 14 constant packet sizes, with the goal of understanding what network 15 device capability has been tested. Tests with constant packet size 16 reveal device capabilities but differ significantly from the 17 conditions encountered in operational deployment, and so additional 18 tests are sometimes conducted with a mixture of packet sizes, or 19 "IMIX". The mixture of sizes a networking device will encounter is 20 highly variable and depends on many factors. An IMIX suited for one 21 networking device and deployment will not be appropriate for another. 22 However, the mix of sizes may be known and the tester may be asked to 23 augment the fixed size tests. To address this need, and the 24 perpetual goal of specifying repeatable test conditions, this draft 25 defines a way to specify the exact repeating sequence of packet sizes 26 from the usual set of fixed sizes, and other forms of mixed size 27 specification. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in RFC 2119 [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 50 This Internet-Draft will expire on December 5, 2013. 52 Copyright Notice 54 Copyright (c) 2013 IETF Trust and the persons identified as the 55 document authors. All rights reserved. 57 This document is subject to BCP 78 and the IETF Trust's Legal 58 Provisions Relating to IETF Documents 59 (http://trustee.ietf.org/license-info) in effect on the date of 60 publication of this document. Please review these documents 61 carefully, as they describe your rights and restrictions with respect 62 to this document. Code Components extracted from this document must 63 include Simplified BSD License text as described in Section 4.e of 64 the Trust Legal Provisions and are provided without warranty as 65 described in the Simplified BSD License. 67 Table of Contents 69 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . 4 71 3. Specification of the IMIX Genome . . . . . . . . . . . . . . . 5 72 4. Specification of a Custom IMIX . . . . . . . . . . . . . . . . 7 73 5. Reporting Long or Pseudo-Random Packet Sequences . . . . . . . 8 74 6. Security Considerations . . . . . . . . . . . . . . . . . . . 9 75 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 76 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 9 77 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 78 9.1. Normative References . . . . . . . . . . . . . . . . . . . 9 79 9.2. Informative References . . . . . . . . . . . . . . . . . . 10 80 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 10 82 1. Introduction 84 This memo defines a method to unambiguously specify the sequence of 85 packet sizes used in a load test. 87 Benchmarking Methodologies [RFC2544] have always relied on test 88 conditions with constant packet sizes, with the goal of understanding 89 what network device capability has been tested. Tests with the 90 smallest size stress the header processing capacity, and tests with 91 the largest size stress the overall bit processing capacity. Tests 92 with sizes in-between may determine the transition between these two 93 capacities. 95 Streams of constant packet size differ significantly from the 96 conditions encountered in operational deployment, and so additional 97 tests are sometimes conducted with a mixture of packet sizes. The 98 set of sizes used is often called an Internet Mix, or "IMIX" 99 [Spirent], [IXIA], [Agilent]. 101 The mixture of sizes a networking device will encounter is highly 102 variable and depends on many factors. An IMIX suited for one 103 networking device and deployment will not be appropriate for another. 104 However, the mix of sizes may be known and the tester may be asked to 105 augment the fixed size tests. The references above cite the original 106 studies and their methodologies. Similar methods can be used to 107 determine new size mixes present on a link or network. We note that 108 the architecture for IP Flow Information Export [RFC5470] provides 109 one method to gather packet size information on private networks. 111 To address this need, and the perpetual goal of specifying repeatable 112 test conditions, this memo proposes a way to specify the exact 113 repeating sequence of packet sizes from the usual set of fixed sizes: 114 the IMIX Genome. Other, less exact forms of size specification are 115 also recommended for extremely complicated or customized size mixes. 116 We apply the term "genome" to infer that the entire test packet size 117 sequence can be replicated if this information is known, a parallel 118 to the information needed for biological replication. 120 This memo takes the position that it cannot be proven for all 121 circumstances that the sequence of packet sizes does not affect the 122 test result, thus a standardized specification of sequence is 123 valuable. 125 2. Scope and Goals 127 This memo defines a method to unambiguously specify the sequence of 128 packet sizes that have been used in a load test, assuming that a 129 relevant mix of sizes is known to the tester and the length of the 130 repeating sequence is not very long (<100 packets). 132 The IMIX Genome will allow an exact sequence of packet sizes to be 133 communicated as a single-line name, resolving the current ambiguity 134 with results that simply refer to "IMIX". This aspect is critical 135 because no ability has been demonstrated to extrapolate results from 136 one IMIX to another IMIX, even when the mix varies only slightly from 137 another IMIX, and certainly no ability to extrapolate results to 138 other circumstances. 140 While documentation of the exact sequence is ideal, the memo also 141 covers the case where the sequence of sizes is very long or may be 142 generated by a pseudo-random process. 144 It is a colossal non-goal to standardize one or more versions of the 145 IMIX. This topic has been discussed on many occasions on the bmwg- 146 list[IMIXonList]. The goal is to enable customization with minimal 147 constraints while fostering repeatable testing once the fixed size 148 testing is complete. Thus, the requirements presented in this 149 specification, expressed in [RFC2119] terms, are intended for those 150 performing/reporting laboratory tests to improve clarity and 151 repeatability. 153 3. Specification of the IMIX Genome 155 The IMIX Genome is specified in the following format: 157 IMIX - 123456...x 159 where each number is replaced by the letter corresponding to the size 160 of the packet at that position in the sequence. The following table 161 gives the letter encoding for the [RFC2544] standard sizes (64, 128, 162 256, 512, 1024, 1280, and 1518 bytes) and "jumbo" sizes (2112, 9000, 163 16000). Note that the 4 octet Ethernet frame check sequence may fail 164 to detect bit errors in the larger jumbo frames, see [jumbo]. 166 +-------------+--------------------+ 167 | Size, bytes | Genome Code Letter | 168 +-------------+--------------------+ 169 | 64 | a | 170 | 128 | b | 171 | 256 | c | 172 | 512 | d | 173 | 1024 | e | 174 | 1280 | f | 175 | 1518 | g | 176 | 2112 | h | 177 | 9000 | i | 178 | 16000 | j | 179 | MTU | z | 180 +-------------+--------------------+ 182 For example: a five packet sequence with sizes 64,64,64,1280,1518 183 would be designated: 185 IMIX - aaafg 187 If z (MTU) is used, the tester MUST specify the length of the MTU in 188 the report. 190 While this approach allows some flexibility, there are also 191 constraints. 193 o Non-RFC2544 packet sizes would need to be approximated by those 194 available in the table. 196 o The Genome for very long sequences can become undecipherable by 197 humans. 199 Some questions testers must ask and answer when using the IMIX Genome 200 are: 202 1. Multiple Source-Destination Address Pairs: is the IMIX sequence 203 applicable to each pair, across multiple pairs in sets, or across 204 all pairs? 206 2. Multiple Tester Ports: is the IMIX sequence applicable to each 207 port, across multiple ports in sets, or across all ports? 209 The chosen configuration would be expressed in the following general 210 form: 212 +------------------------+--------------------------+---------------+ 213 | Source Address + Port | Destination Address + | Corresponding | 214 | AND/OR Blade | Port AND/OR Blade | IMIX | 215 +------------------------+--------------------------+---------------+ 216 | x.x.x.x Blade2 | y.y.y.y Blade3 | IMIX - aaafg | 217 +------------------------+--------------------------+---------------+ 219 where testers can specify the IMIX used between any two entities in 220 the test architecture (and Blade is a component in a multi-component 221 device chassis). 223 4. Specification of a Custom IMIX 225 This section describes how to specify an IMIX with locally-selected 226 packet sizes 228 The Custom IMIX is specified in the following format: 230 CUSTOM IMIX - 123456...x 232 where each number is replaced by the letter corresponding to the size 233 of the packet at that position in the sequence. The tester MUST 234 complete the following table, giving the letter encoding for each 235 size used, where each set of three lower-case letters would be 236 replaced by the integer size in octets. 238 +-------------+--------------------+ 239 | Size, bytes | Custom Code Letter | 240 +-------------+--------------------+ 241 | aaa | A | 242 | bbb | B | 243 | ccc | C | 244 | ddd | D | 245 | eee | E | 246 | fff | F | 247 | ggg | G | 248 | etc. | up to Z | 249 +-------------+--------------------+ 251 For example: a five packet sequence with sizes 252 aaa=64,aaa=64,aaa=64,ggg=1020,ggg=1020 would be designated: 254 CUSTOM IMIX - AAAGG 256 5. Reporting Long or Pseudo-Random Packet Sequences 258 When the IMIX-Genome cannot be used (when the sheer length of the 259 sequence would make the Genome unmanageable), two options are 260 possible. When a sequence can be decomposed into a series of short 261 repeating sequences, then a run-length encoding approach MAY be 262 specified as shown in the table below (using the single lower-case 263 letter Genome Codes from section 3): 265 +------------------------------+----------------------+ 266 | Count of Repeating Sequences | Packet Size Sequence | 267 +------------------------------+----------------------+ 268 | 20 | abcd | 269 | 5 | ggga | 270 | 10 | dcba | 271 +------------------------------+----------------------+ 273 The run-length encoding approach is also applicable to custom IMIX 274 described in section 4 (where the single upper-case letter Genome 275 Codes would be used instead). 277 When the sequence is designed to vary within some proportional 278 constraints, a table simply giving the proportions of each size MAY 279 be used instead. 281 +-----------+---------------------+---------------------------+ 282 | IP Length | Percentage of Total | Length(s) at other layers | 283 +-----------+---------------------+---------------------------+ 284 | 64 | 23 | 82 | 285 | 128 | 67 | 146 | 286 | 1000 | 10 | 1018 | 287 +-----------+---------------------+---------------------------+ 289 Note that the table of proportions also allows non-standard packet 290 sizes, but trades the short Genome specification and ability to 291 specify the exact sequence for other flexibilities. 293 If a deterministic packet size generation method is used (such as 294 monotonic increase by one octet from start value to MTU), then the 295 generation algorithm SHOULD be reported. 297 If a pseudo-random length generation capability is used, then the 298 generation algorithm SHOULD be reported with the results along with 299 the seed value used. We also recognize the opportunity to randomize 300 inter-packet spacing from a test sender as well as the size, and both 301 spacing and length pseudo-random generation algorithms and seeds 302 SHOULD be reported when used. 304 Finally, we note another possibility: a pseudo-random sequence 305 generates an index to the table of packet lengths, and the generation 306 algorithm SHOULD be reported with the results along with the seed 307 value if used. 309 6. Security Considerations 311 Benchmarking activities as described in this memo are limited to 312 technology characterization using controlled stimuli in a laboratory 313 environment, with dedicated address space and the other constraints 314 [RFC2544]. 316 The benchmarking network topology will be an independent test setup 317 and MUST NOT be connected to devices that may forward the test 318 traffic into a production network, or misroute traffic to the test 319 management network. 321 Further, benchmarking is performed on a "black-box" basis, relying 322 solely on measurements observable external to the DUT/SUT. 324 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 325 benchmarking purposes. Any implications for network security arising 326 from the DUT/SUT SHOULD be identical in the lab and in production 327 networks. 329 7. IANA Considerations 331 This memo makes no requests of IANA, and hopes that IANA will leave 332 it alone as well. 334 8. Acknowledgements 336 Thanks to Sarah Banks, Aamer Akhter, Steve Maxwell, and Scott Bradner 337 for their reviews and comments. Ilya Varlashkin suggested the run- 338 length coding approach in Section 5. 340 9. References 342 9.1. Normative References 344 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 345 Requirement Levels", BCP 14, RFC 2119, March 1997. 347 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 348 Network Interconnect Devices", RFC 2544, March 1999. 350 9.2. Informative References 352 [Agilent] http://www.ixiacom.com/pdfs/test_plans/ 353 agilent_journal_of_internet_test_methodologies.pdf, "The 354 Journal of Internet Test Methodologies", 2007. 356 [IMIXonList] 357 http://www.ietf.org/mail-archive/web/bmwg/current/ 358 msg00691.html, "Discussion on IMIX", 2003. 360 [IXIA] http://www.ixiacom.com/library/test_plans/ 361 display?skey=testing_pppox, "Library: Test Plans", 2010. 363 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 364 "Architecture for IP Flow Information Export", RFC 5470, 365 March 2009. 367 [Spirent] http://gospirent.com/whitepaper/ 368 IMIX%20Test%20Methodolgy%20Journal.pdf, "Test Methodology 369 Journal: IMIX (Internet Mix) Journal", 2006. 371 [jumbo] http://sd.wareonearth.com/~phil/jumbo.html and 372 http://staff.psc.edu/mathis/MTU/arguments.html#crc, 373 "Discussion of Jumbo Packets and FCS Failure". 375 Author's Address 377 Al Morton 378 AT&T Labs 379 200 Laurel Avenue South 380 Middletown,, NJ 07748 381 USA 383 Phone: +1 732 420 1571 384 Fax: +1 732 368 1192 385 Email: acmorton@att.com 386 URI: http://home.comcast.net/~acmacm/