idnits 2.17.1 draft-ietf-bmwg-imix-genome-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 8, 2012) is 4492 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Informational January 8, 2012 5 Expires: July 11, 2012 7 IMIX Genome: Specification of variable packet sizes for additional 8 testing 9 draft-ietf-bmwg-imix-genome-01 11 Abstract 13 Benchmarking Methodologies have always relied on test conditions with 14 constant packet sizes, with the goal of understanding what network 15 device capability has been tested. Tests with constant packet size 16 reveal device capabilities but differ significantly from the 17 conditions encountered in operational deployment, and so additional 18 tests are sometimes conducted with a mixture of packet sizes, or 19 "IMIX". The mixture of sizes a networking device will encounter is 20 highly variable and depends on many factors. An IMIX suited for one 21 networking device and deployment will not be appropriate for another. 22 However, the mix of sizes may be known and the tester may be asked to 23 augment the fixed size tests. To address this need, and the 24 perpetual goal of specifying repeatable test conditions, this draft 25 defines a way to specify the exact repeating sequence of packet sizes 26 from the usual set of fixed sizes, and other forms of mixed size 27 specification. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in RFC 2119 [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 50 This Internet-Draft will expire on July 11, 2012. 52 Copyright Notice 54 Copyright (c) 2012 IETF Trust and the persons identified as the 55 document authors. All rights reserved. 57 This document is subject to BCP 78 and the IETF Trust's Legal 58 Provisions Relating to IETF Documents 59 (http://trustee.ietf.org/license-info) in effect on the date of 60 publication of this document. Please review these documents 61 carefully, as they describe your rights and restrictions with respect 62 to this document. Code Components extracted from this document must 63 include Simplified BSD License text as described in Section 4.e of 64 the Trust Legal Provisions and are provided without warranty as 65 described in the Simplified BSD License. 67 Table of Contents 69 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . . 4 71 3. Specification of the IMIX Genome . . . . . . . . . . . . . . . 5 72 4. Specification of a Custom IMIX . . . . . . . . . . . . . . . . 6 73 5. Reporting Long or Pseudo-Random Packet Sequences . . . . . . . 7 74 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 8 75 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8 76 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 8 77 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 78 9.1. Normative References . . . . . . . . . . . . . . . . . . . 9 79 9.2. Informative References . . . . . . . . . . . . . . . . . . 9 80 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 9 82 1. Introduction 84 This memo defines a method to unambiguously specify the sequence of 85 packet sizes used in a load test. 87 Benchmarking Methodologies [RFC2544] have always relied on test 88 conditions with constant packet sizes, with the goal of understanding 89 what network device capability has been tested. Tests with the 90 smallest size stress the header processing capacity, and tests with 91 the largest size stress the overall bit processing capacity. Tests 92 with sizes in-between may determine the transition between these two 93 capacities. 95 Streams of constant packet size differ significantly from the 96 conditions encountered in operational deployment, and so additional 97 tests are sometimes conducted with a mixture of packet sizes. The 98 set of sizes used is often called an Internet Mix, or "IMIX" 99 [Spirent], [IXIA], [Agilent]. 101 The mixture of sizes a networking device will encounter is highly 102 variable and depends on many factors. An IMIX suited for one 103 networking device and deployment will not be appropriate for another. 104 However, the mix of sizes may be known and the tester may be asked to 105 augment the fixed size tests. The references above cite the original 106 studies and their methodologies - similar methods can be used to 107 determine new size mixes. 109 To address this need, and the perpetual goal of specifying repeatable 110 test conditions, this draft proposes a way to specify the exact 111 repeating sequence of packet sizes from the usual set of fixed sizes: 112 the IMIX Genome. Other, less exact forms of size specification are 113 also recommended for extremely complicated or customized size mixes. 115 This memo takes the position that it cannot be proven for all 116 circumstances that the sequence of packet sizes does not affect the 117 test result, thus a standardized specification of sequence is 118 valuable. 120 2. Scope and Goals 122 This memo defines a method to unambiguously specify the sequence of 123 packet sizes that have been used in a load test, assuming that a 124 relevant mix of sizes is known to the tester and the length of the 125 repeating sequence is not very long (<100 packets). 127 The IMIX Genome will allow an exact sequence of packet sizes to be 128 communicated as a single-line name, resolving the current ambiguity 129 with results that simply refer to "IMIX". 131 While documentation of the exact sequence is ideal, the memo also 132 covers the case where the sequence of sizes is very long or may be 133 generated by a pseudo-random process. 135 It is a colossal non-goal to standardize one or more versions of the 136 IMIX. This topic has been discussed on many occasions on the bmwg- 137 list[IMIXonList]. The goal is to enable customization with minimal 138 constraints while fostering repeatable testing once the fixed size 139 testing is complete. 141 3. Specification of the IMIX Genome 143 The IMIX Genome is specified in the following format: 145 IMIX - 123456...x 147 where each number is replaced by the letter corresponding to the size 148 of the packet at that position in the sequence. The following table 149 gives the letter encoding for the [RFC2544] standard sizes (64, 128, 150 256, 512, 1024, 1280, and 1518 bytes) and "jumbo" sizes (2112, 9000, 151 16000). Note that the 4 octet Ethernet frame check sequence may fail 152 to detect bit errors in the larger jumbo frames, see [jumbo]. 154 +-------------+--------------------+ 155 | Size, bytes | Genome Code Letter | 156 +-------------+--------------------+ 157 | 64 | a | 158 | 128 | b | 159 | 256 | c | 160 | 512 | d | 161 | 1024 | e | 162 | 1280 | f | 163 | 1518 | g | 164 | 2112 | h | 165 | 9000 | i | 166 | 16000 | j | 167 | MTU | z | 168 +-------------+--------------------+ 170 For example: a five packet sequence with sizes 64,64,64,1280,1518 171 would be designated: 173 IMIX - aaafg 175 While this approach allows some flexibility, there are also 176 constraints. 178 o Non-RFC2544 packet sizes would need to be approximated by those 179 available in the table. 181 o The Genome for very long sequences can become undecipherable by 182 humans. 184 o z=MTU is seen as valuable, so MTU MUST be specified if used. 186 o Whether more tabulated packet sizes would be useful is TBD, and 187 "jumbo" sizes were added in this version. 189 Some questions testers must ask and answer when using the IMIX Genome 190 are: 192 1. Multiple Source-Destination Address Pairs: is the IMIX sequence 193 applicable to each pair, across multiple pairs in sets, or across 194 all pairs? 196 2. Multiple Tester Ports: is the IMIX sequence applicable to each 197 port, across multiple ports in sets, or across all ports? 199 The chosen configuration would be expressed the following general 200 form: 202 +-----------------------+-------------------------+-----------------+ 203 | Source | Destination | Corresponding | 204 | Address/Port/Blade | Address/Port/Blade | IMIX | 205 +-----------------------+-------------------------+-----------------+ 206 | x.x.x.x Blade2 | y.y.y.y Blade3 | IMIX - aaafg | 207 +-----------------------+-------------------------+-----------------+ 209 where testers can specify the IMIX used between any two entities in 210 the test architecture. 212 4. Specification of a Custom IMIX 214 The Custom IMIX is specified in the following format: 216 CUSTOM IMIX - 123456...x 218 where each number is replaced by the letter corresponding to the size 219 of the packet at that position in the sequence. The tester MUST 220 complete the following table, giving the letter encoding for each 221 size used, where each set of three lower-case letters would be 222 replaced by the integer size in octets. 224 +-------------+--------------------+ 225 | Size, bytes | Custom Code Letter | 226 +-------------+--------------------+ 227 | aaa | A | 228 | bbb | B | 229 | ccc | C | 230 | ddd | D | 231 | eee | E | 232 | fff | F | 233 | ggg | G | 234 | etc. | up to Z | 235 +-------------+--------------------+ 237 For example: a five packet sequence with sizes aaa,aaa,aaa,ggg,ggg 238 would be designated: 240 CUSTOM IMIX - AAAGG 242 5. Reporting Long or Pseudo-Random Packet Sequences 244 When the IMIX-Genome cannot be used (when the sheer length of the 245 sequence would make the genome unmanageable), two options are 246 possible. When a sequence can be decomposed into a series of short 247 repeating sequences, then a run-length encoding approach MAY be used 248 as shown below: 250 +------------------------------+----------------------+ 251 | Count of Repeating Sequences | Packet Size Sequence | 252 +------------------------------+----------------------+ 253 | 20 | abcd | 254 | 5 | ggga | 255 | 10 | dcba | 256 +------------------------------+----------------------+ 258 When the sequence is designed to vary within some proportional 259 constraints, a table simply giving the proportions of each size MAY 260 be used instead. 262 +-----------+---------------------+-----------------+ 263 | IP Length | Percentage of Total | Other Length(s) | 264 +-----------+---------------------+-----------------+ 265 | 64 | 23 | 82 | 266 | 128 | 67 | 146 | 267 | 1000 | 10 | 1018 | 268 +-----------+---------------------+-----------------+ 270 Note that the table of proportions also allows non-standard packet 271 sizes, but trades the short genome specification and ability to 272 specify the exact sequence for other flexibilities. 274 6. Security Considerations 276 Benchmarking activities as described in this memo are limited to 277 technology characterization using controlled stimuli in a laboratory 278 environment, with dedicated address space and the other constraints 279 [RFC2544]. 281 The benchmarking network topology will be an independent test setup 282 and MUST NOT be connected to devices that may forward the test 283 traffic into a production network, or misroute traffic to the test 284 management network. 286 Further, benchmarking is performed on a "black-box" basis, relying 287 solely on measurements observable external to the DUT/SUT. 289 Special capabilities SHOULD NOT exist in the DUT/SUT specifically for 290 benchmarking purposes. Any implications for network security arising 291 from the DUT/SUT SHOULD be identical in the lab and in production 292 networks. 294 7. IANA Considerations 296 This memo makes no requests of IANA, and hopes that IANA will leave 297 it alone as well. 299 8. Acknowledgements 301 Thanks to Sarah Banks, Aamer Akhter, and Steve Maxwell for their 302 review and comments. Ilya Varlashkin suggested the run-length coding 303 approach in Section 5. 305 9. References 306 9.1. Normative References 308 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 309 Requirement Levels", BCP 14, RFC 2119, March 1997. 311 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 312 Network Interconnect Devices", RFC 2544, March 1999. 314 9.2. Informative References 316 [Agilent] http://www.ixiacom.com/pdfs/test_plans/ 317 agilent_journal_of_internet_test_methodologies.pdf, "The 318 Journal of Internet Test Methodologies", 2007. 320 [IMIXonList] 321 http://www.ietf.org/mail-archive/web/bmwg/current/ 322 msg00691.html, "Discussion on IMIX", 2003. 324 [IXIA] http://www.ixiacom.com/library/test_plans/ 325 display?skey=testing_pppox, "Library: Test Plans", 2010. 327 [Spirent] http://gospirent.com/whitepaper/ 328 IMIX%20Test%20Methodolgy%20Journal.pdf, "Test Methodology 329 Journal: IMIX (Internet Mix) Journal", 2006. 331 [jumbo] http://www.ietf.org/mail-archive/web/bmwg/current/ 332 msg00691.html, "Discussion of Jumbo Packets and FCS 333 Failure, see http://sd.wareonearth.com/~phil/jumbo.html 334 and http://staff.psc.edu/mathis/MTU/arguments.html#crc". 336 Author's Address 338 Al Morton 339 AT&T Labs 340 200 Laurel Avenue South 341 Middletown,, NJ 07748 342 USA 344 Phone: +1 732 420 1571 345 Fax: +1 732 368 1192 346 Email: acmorton@att.com 347 URI: http://home.comcast.net/~acmacm/