idnits 2.17.1 draft-ietf-bmwg-imix-genome-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 12, 2012) is 4125 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational December 12, 2012 5 Expires: June 15, 2013 7 IMIX Genome: Specification of variable packet sizes for additional 8 testing 9 draft-ietf-bmwg-imix-genome-04 11 Abstract 13 Benchmarking Methodologies have always relied on test conditions with 14 constant packet sizes, with the goal of understanding what network 15 device capability has been tested. Tests with constant packet size 16 reveal device capabilities but differ significantly from the 17 conditions encountered in operational deployment, and so additional 18 tests are sometimes conducted with a mixture of packet sizes, or 19 "IMIX". The mixture of sizes a networking device will encounter is 20 highly variable and depends on many factors. An IMIX suited for one 21 networking device and deployment will not be appropriate for another. 22 However, the mix of sizes may be known and the tester may be asked to 23 augment the fixed size tests. To address this need, and the 24 perpetual goal of specifying repeatable test conditions, this draft 25 defines a way to specify the exact repeating sequence of packet sizes 26 from the usual set of fixed sizes, and other forms of mixed size 27 specification. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in RFC 2119 [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 50 This Internet-Draft will expire on June 15, 2013. 52 Copyright Notice 54 Copyright (c) 2012 IETF Trust and the persons identified as the 55 document authors. All rights reserved. 57 This document is subject to BCP 78 and the IETF Trust's Legal 58 Provisions Relating to IETF Documents 59 (http://trustee.ietf.org/license-info) in effect on the date of 60 publication of this document. Please review these documents 61 carefully, as they describe your rights and restrictions with respect 62 to this document. Code Components extracted from this document must 63 include Simplified BSD License text as described in Section 4.e of 64 the Trust Legal Provisions and are provided without warranty as 65 described in the Simplified BSD License. 67 Table of Contents 69 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . 4 71 3. Specification of the IMIX Genome . . . . . . . . . . . . . . . 5 72 4. Specification of a Custom IMIX . . . . . . . . . . . . . . . . 7 73 5. Reporting Long or Pseudo-Random Packet Sequences . . . . . . . 7 74 6. Security Considerations . . . . . . . . . . . . . . . . . . . 8 75 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 76 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 9 77 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 78 9.1. Normative References . . . . . . . . . . . . . . . . . . . 9 79 9.2. Informative References . . . . . . . . . . . . . . . . . . 9 80 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 10 82 1. Introduction 84 This memo defines a method to unambiguously specify the sequence of 85 packet sizes used in a load test. 87 Benchmarking Methodologies [RFC2544] have always relied on test 88 conditions with constant packet sizes, with the goal of understanding 89 what network device capability has been tested. Tests with the 90 smallest size stress the header processing capacity, and tests with 91 the largest size stress the overall bit processing capacity. Tests 92 with sizes in-between may determine the transition between these two 93 capacities. 95 Streams of constant packet size differ significantly from the 96 conditions encountered in operational deployment, and so additional 97 tests are sometimes conducted with a mixture of packet sizes. The 98 set of sizes used is often called an Internet Mix, or "IMIX" 99 [Spirent], [IXIA], [Agilent]. 101 The mixture of sizes a networking device will encounter is highly 102 variable and depends on many factors. An IMIX suited for one 103 networking device and deployment will not be appropriate for another. 104 However, the mix of sizes may be known and the tester may be asked to 105 augment the fixed size tests. The references above cite the original 106 studies and their methodologies. Similar methods can be used to 107 determine new size mixes present on a link or network. We note that 108 the architecture for IP Flow Information Export [RFC5470] provides 109 one method to gather packet size information on private networks. 111 To address this need, and the perpetual goal of specifying repeatable 112 test conditions, this draft proposes a way to specify the exact 113 repeating sequence of packet sizes from the usual set of fixed sizes: 114 the IMIX Genome. Other, less exact forms of size specification are 115 also recommended for extremely complicated or customized size mixes. 117 This memo takes the position that it cannot be proven for all 118 circumstances that the sequence of packet sizes does not affect the 119 test result, thus a standardized specification of sequence is 120 valuable. 122 2. Scope and Goals 124 This memo defines a method to unambiguously specify the sequence of 125 packet sizes that have been used in a load test, assuming that a 126 relevant mix of sizes is known to the tester and the length of the 127 repeating sequence is not very long (<100 packets). 129 The IMIX Genome will allow an exact sequence of packet sizes to be 130 communicated as a single-line name, resolving the current ambiguity 131 with results that simply refer to "IMIX". This aspect is critical 132 because no ability has been demonstrated to extrapolate results from 133 one IMIX to another IMIX, even when the mix varies only slightly from 134 another IMIX, and certainly no ability to extrapolate results to 135 other circumstances. 137 While documentation of the exact sequence is ideal, the memo also 138 covers the case where the sequence of sizes is very long or may be 139 generated by a pseudo-random process. 141 It is a colossal non-goal to standardize one or more versions of the 142 IMIX. This topic has been discussed on many occasions on the bmwg- 143 list[IMIXonList]. The goal is to enable customization with minimal 144 constraints while fostering repeatable testing once the fixed size 145 testing is complete. 147 3. Specification of the IMIX Genome 149 The IMIX Genome is specified in the following format: 151 IMIX - 123456...x 153 where each number is replaced by the letter corresponding to the size 154 of the packet at that position in the sequence. The following table 155 gives the letter encoding for the [RFC2544] standard sizes (64, 128, 156 256, 512, 1024, 1280, and 1518 bytes) and "jumbo" sizes (2112, 9000, 157 16000). Note that the 4 octet Ethernet frame check sequence may fail 158 to detect bit errors in the larger jumbo frames, see [jumbo]. 160 +-------------+--------------------+ 161 | Size, bytes | Genome Code Letter | 162 +-------------+--------------------+ 163 | 64 | a | 164 | 128 | b | 165 | 256 | c | 166 | 512 | d | 167 | 1024 | e | 168 | 1280 | f | 169 | 1518 | g | 170 | 2112 | h | 171 | 9000 | i | 172 | 16000 | j | 173 | MTU | z | 174 +-------------+--------------------+ 176 For example: a five packet sequence with sizes 64,64,64,1280,1518 177 would be designated: 179 IMIX - aaafg 181 While this approach allows some flexibility, there are also 182 constraints. 184 o Non-RFC2544 packet sizes would need to be approximated by those 185 available in the table. 187 o The Genome for very long sequences can become undecipherable by 188 humans. 190 o z=MTU is seen as valuable, so MTU MUST be specified if used. 192 o "jumbo" sizes are included. 194 Some questions testers must ask and answer when using the IMIX Genome 195 are: 197 1. Multiple Source-Destination Address Pairs: is the IMIX sequence 198 applicable to each pair, across multiple pairs in sets, or across 199 all pairs? 201 2. Multiple Tester Ports: is the IMIX sequence applicable to each 202 port, across multiple ports in sets, or across all ports? 204 The chosen configuration would be expressed the following general 205 form: 207 +-----------------------+-------------------------+-----------------+ 208 | Source | Destination | Corresponding | 209 | Address/Port/Blade | Address/Port/Blade | IMIX | 210 +-----------------------+-------------------------+-----------------+ 211 | x.x.x.x Blade2 | y.y.y.y Blade3 | IMIX - aaafg | 212 +-----------------------+-------------------------+-----------------+ 214 where testers can specify the IMIX used between any two entities in 215 the test architecture (and Blade is a component in a multi-component 216 device chassis). 218 4. Specification of a Custom IMIX 220 The Custom IMIX is specified in the following format: 222 CUSTOM IMIX - 123456...x 224 where each number is replaced by the letter corresponding to the size 225 of the packet at that position in the sequence. The tester MUST 226 complete the following table, giving the letter encoding for each 227 size used, where each set of three lower-case letters would be 228 replaced by the integer size in octets. 230 +-------------+--------------------+ 231 | Size, bytes | Custom Code Letter | 232 +-------------+--------------------+ 233 | aaa | A | 234 | bbb | B | 235 | ccc | C | 236 | ddd | D | 237 | eee | E | 238 | fff | F | 239 | ggg | G | 240 | etc. | up to Z | 241 +-------------+--------------------+ 243 For example: a five packet sequence with sizes 244 aaa=64,aaa=64,aaa=64,ggg=1020,ggg=1020 would be designated: 246 CUSTOM IMIX - AAAGG 248 5. Reporting Long or Pseudo-Random Packet Sequences 250 When the IMIX-Genome cannot be used (when the sheer length of the 251 sequence would make the genome unmanageable), two options are 252 possible. When a sequence can be decomposed into a series of short 253 repeating sequences, then a run-length encoding approach MAY be used 254 as shown below: 256 +------------------------------+----------------------+ 257 | Count of Repeating Sequences | Packet Size Sequence | 258 +------------------------------+----------------------+ 259 | 20 | abcd | 260 | 5 | ggga | 261 | 10 | dcba | 262 +------------------------------+----------------------+ 264 When the sequence is designed to vary within some proportional 265 constraints, a table simply giving the proportions of each size MAY 266 be used instead. 268 +-----------+---------------------+---------------------------+ 269 | IP Length | Percentage of Total | Length(s) at other layers | 270 +-----------+---------------------+---------------------------+ 271 | 64 | 23 | 82 | 272 | 128 | 67 | 146 | 273 | 1000 | 10 | 1018 | 274 +-----------+---------------------+---------------------------+ 276 Note that the table of proportions also allows non-standard packet 277 sizes, but trades the short genome specification and ability to 278 specify the exact sequence for other flexibilities. 280 If a pseudo-random length generation capability is used, then the 281 generation algorithm SHOULD be reported with the results along with 282 the seed value used. We also recognize the opportunity to randomize 283 inter-packet spacing from a test sender as well as the size, and both 284 spacing and length pseudo-random generation algorithms and seeds 285 SHOULD be reported when used. 287 6. Security Considerations 289 Benchmarking activities as described in this memo are limited to 290 technology characterization using controlled stimuli in a laboratory 291 environment, with dedicated address space and the other constraints 292 [RFC2544]. 294 The benchmarking network topology will be an independent test setup 295 and MUST NOT be connected to devices that may forward the test 296 traffic into a production network, or misroute traffic to the test 297 management network. 299 Further, benchmarking is performed on a "black-box" basis, relying 300 solely on measurements observable external to the DUT/SUT. 302 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 303 benchmarking purposes. Any implications for network security arising 304 from the DUT/SUT SHOULD be identical in the lab and in production 305 networks. 307 7. IANA Considerations 309 This memo makes no requests of IANA, and hopes that IANA will leave 310 it alone as well. 312 8. Acknowledgements 314 Thanks to Sarah Banks, Aamer Akhter, Steve Maxwell, and Scott Bradner 315 for their reviews and comments. Ilya Varlashkin suggested the run- 316 length coding approach in Section 5. 318 9. References 320 9.1. Normative References 322 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 323 Requirement Levels", BCP 14, RFC 2119, March 1997. 325 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 326 Network Interconnect Devices", RFC 2544, March 1999. 328 9.2. Informative References 330 [Agilent] http://www.ixiacom.com/pdfs/test_plans/ 331 agilent_journal_of_internet_test_methodologies.pdf, "The 332 Journal of Internet Test Methodologies", 2007. 334 [IMIXonList] 335 http://www.ietf.org/mail-archive/web/bmwg/current/ 336 msg00691.html, "Discussion on IMIX", 2003. 338 [IXIA] http://www.ixiacom.com/library/test_plans/ 339 display?skey=testing_pppox, "Library: Test Plans", 2010. 341 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, 342 "Architecture for IP Flow Information Export", RFC 5470, 343 March 2009. 345 [Spirent] http://gospirent.com/whitepaper/ 346 IMIX%20Test%20Methodolgy%20Journal.pdf, "Test Methodology 347 Journal: IMIX (Internet Mix) Journal", 2006. 349 [jumbo] http://www.ietf.org/mail-archive/web/bmwg/current/ 350 msg00691.html, "Discussion of Jumbo Packets and FCS 351 Failure, see http://sd.wareonearth.com/~phil/jumbo.html 352 and http://staff.psc.edu/mathis/MTU/arguments.html#crc". 354 Author's Address 356 Al Morton 357 AT&T Labs 358 200 Laurel Avenue South 359 Middletown,, NJ 07748 360 USA 362 Phone: +1 732 420 1571 363 Fax: +1 732 368 1192 364 Email: acmorton@att.com 365 URI: http://home.comcast.net/~acmacm/