idnits 2.17.1 draft-morton-ippm-capcity-metric-method-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (July 8, 2019) is 1753 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC7680' is mentioned on line 220, but not defined == Unused Reference: 'RFC1242' is defined on line 451, but no explicit reference was found in the text == Unused Reference: 'RFC2544' is defined on line 465, but no explicit reference was found in the text == Unused Reference: 'RFC2889' is defined on line 474, but no explicit reference was found in the text == Unused Reference: 'RFC5136' is defined on line 479, but no explicit reference was found in the text == Unused Reference: 'RFC5180' is defined on line 483, but no explicit reference was found in the text == Unused Reference: 'RFC6201' is defined on line 488, but no explicit reference was found in the text == Unused Reference: 'RFC6412' is defined on line 493, but no explicit reference was found in the text == Unused Reference: 'RFC6815' is defined on line 507, but no explicit reference was found in the text == Unused Reference: 'RFC6985' is defined on line 513, but no explicit reference was found in the text == Unused Reference: 'RFC8468' is defined on line 535, but no explicit reference was found in the text == Unused Reference: 'RFC8239' is defined on line 548, but no explicit reference was found in the text == Unused Reference: 'TST009' is defined on line 552, but no explicit reference was found in the text == Unused Reference: 'VSPERF-b2b' is defined on line 559, but no explicit reference was found in the text == Unused Reference: 'VSPERF-BSLV' is defined on line 565, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 1242 ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Downref: Normative reference to an Informational RFC: RFC 2544 ** Downref: Normative reference to an Informational RFC: RFC 2889 ** Downref: Normative reference to an Informational RFC: RFC 5136 ** Downref: Normative reference to an Informational RFC: RFC 5180 ** Downref: Normative reference to an Informational RFC: RFC 6201 ** Downref: Normative reference to an Informational RFC: RFC 6412 ** Downref: Normative reference to an Informational RFC: RFC 6815 ** Downref: Normative reference to an Informational RFC: RFC 6985 ** Downref: Normative reference to an Informational RFC: RFC 7312 ** Downref: Normative reference to an Informational RFC: RFC 7799 ** Downref: Normative reference to an Experimental RFC: RFC 8337 ** Downref: Normative reference to an Informational RFC: RFC 8468 Summary: 14 errors (**), 0 flaws (~~), 17 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Updates: ???? (if approved) R. Geib 5 Intended status: Standards Track Deutsche Telekom 6 Expires: January 9, 2020 L. Ciavattone 7 AT&T Labs 8 July 8, 2019 10 Metrics and Methods for IP Capacity 11 draft-morton-ippm-capcity-metric-method-00 13 Abstract 15 This memo revisits the problem of Network Capacity metrics first 16 examined in RFC 5136. The memo specifies a more practical Maximum 17 IP-layer Capacity metric definition catering for measurement 18 purposes, and outlines the corresponding methods of measurement. 20 Requirements Language 22 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 23 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 24 "OPTIONAL" in this document are to be interpreted as described in BCP 25 14[RFC2119] [RFC8174] when, and only when, they appear in all 26 capitals, as shown here. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on January 9, 2020. 45 Copyright Notice 47 Copyright (c) 2019 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 63 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . 3 64 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 3 65 4. Metric Definitions . . . . . . . . . . . . . . . . . . . . . 4 66 4.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 4 67 4.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 4 68 4.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 5 69 4.4. Related Round-Trip Delay and Loss Definitions . . . . . . 7 70 4.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 7 71 4.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 8 72 5. Method of Measurement . . . . . . . . . . . . . . . . . . . . 8 73 5.1. Load Rate Adjustment Algorithm (from udpst) . . . . . . . 8 74 6. Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . 9 75 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 76 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 77 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 78 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 79 10.1. Normative References . . . . . . . . . . . . . . . . . . 10 80 10.2. Informative References . . . . . . . . . . . . . . . . . 12 81 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 83 1. Introduction 85 The IETF's efforts to define Network and Bulk Transport Capacity have 86 been chartered and progressed for over twenty years. Over that time, 87 the performance community has seen development of Informative 88 definitions in RFC 3148 for Framework for Bulk Transport Capacity 89 (BTC), RFC 5136 for Network Capacity and Maximum IP-layer Capacity, 90 and the Experimental metric definitions and methods in [RFC8337], 91 Model-Based Metrics for BTC. 93 This memo recognizes the importance of a definition of a Maximum IP- 94 layer Capacity Metric at this time, a definition that is both 95 practical and effective for the performance community's needs, 96 including Internet users. The metric definition is intended for 97 Active Methods of Measurement [RFC7799], and a method of measurement 98 is included here. 100 The most direct active measurement of IP-layer Capacity would use IP 101 packets, but in practice a transport header is needed to traverse 102 address and port translators. UDP offeres the most direct assessment 103 possibility, and in the [copycat][copycat] measurement study to 104 investigate whether UDP is viable as a general Internet transport 105 protocol, the authors found that a high percentage of paths tested 106 support UDP transport. A number of liaisons have been exchanged on 107 this topic [ refs to ITU-T SG12, ETSI STQ, BBF liaisons ], discussing 108 the laboratory and field tests that support the UDP-based approach to 109 IP-layer Capacity measurement. 111 This memo also recognizes the many updates to the IP Performance 112 Metrics Framework [RFC2330] published over twenty years, and makes 113 use of [RFC7312] for Advanced Stream and Sampling Framework, and RFC 114 8468 [RFC8468]IPv4, IPv6, and IPv4-IPv6 Coexistence Updates. 116 NOTE: The text contains Author comments, in brackets [RG: , acm: ] 118 2. Scope and Goals 120 The scope of this memo is to define a metric and corresponding method 121 to unambiguously perform Active measurements of Maximum IP-Layer 122 Capacity. 124 The main goal is to harmonize the specified metric and method across 125 the industry, and this memo is the vehicle through which working 126 group (and eventually, IETF) consensus will be captured and 127 communicated to achieve broad agreement, and possibly changes in 128 other Standards Development Organizations (SDO). 130 A local goal is provide efficient test procedures where possible, and 131 to recommend reporting with additional interpretation of the results. 132 Also, to foster the development of protocol support for this metric 133 and method of measurement. 135 3. Motivation 137 As with any problem that has been worked for many years in various 138 SDOs without any special attempts at coordination, various solutions 139 for metrics and methods have emerged. 141 There are five factors that have changed (or begun to change) in the 142 last 5 years, and the presence of any one of them on the path 143 requires features in the measurement design to account for the 144 changes: 146 1. Internet access is no longer the bottleneck for many users 148 2. Both speed and latency are important to user's satisfaction 150 3. UDP's growing role in Transport, in areas where TCP once 151 dominated. 153 4. Content and applications moving closer to users. 155 5. Possibly less traffic crossing ISP gateways in future. 157 4. Metric Definitions 159 This section sets requirements for the following components to 160 support the Maximum IP-layer Capacity Metric. 162 4.1. Formal Name 164 Type-P-Max-IP-Capacity, or informally called Maximum IP-layer 165 Capacity. 167 Note that Type-P depends on the chosen method. 169 4.2. Parameters 171 This section lists the REQUIRED input factors to specify a Route 172 metric. 174 o Src, the address of a host (such as the globally routable IP 175 address). 177 o Dst, the address of a host (such as the globally routable IP 178 address). 180 o i, the limit on the number of Hops a specific packet may visit as 181 it traverses from the host at Src to the host at Dst (such as the 182 TTL or Hop Limit). 184 o MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops). 186 o T, the time at the start of measurement interval 188 o I, the duration of measurement interval 189 o dt, the duration of N equal sub-intervals in I 191 o Ts, the host time of a transmitted test packet as measured at 192 MP(Src), meaning Measurement Point at the Source. 194 o Ta, the host time of a test packet's *arrival* as measured at 195 MP(Dst), assigned to packets that arrive within a "reasonable" 196 time (see parameter below). 198 o Tmax, a maximum waiting time for test packets to arrive at the 199 destination, set sufficiently long to disambiguate packets with 200 long delays from packets that are discarded (lost), such that the 201 distribution of one-way delay is not truncated. 203 o F, the number of different flows synthesized by the method. 205 o flow, the stream of packets with the same n-tuple of designated 206 header fields that (when held constant) result in identical 207 treatment in a multi-path decision (such as the decision taken in 208 load balancing). Note: The IPv6 flow label MAY be included in the 209 flow definition when routers have complied with [RFC6438] 210 guidelines at the Tunnel End Points (TEP), and the source of the 211 measurement is a TEP. 213 o Type-P, the complete description of the packets for which this 214 assessment applies (including the flow-defining fields). Note 215 that the UDP transport layer is one requirement specified below. 217 o PM, a list of fundamental metrics, such as loss, delay, and 218 reordering, and corresponding Target performance threshold. At 219 least one fundamental metric and Target performance threshold MUST 220 be supplied (such as One-way IP Packet Loss [RFC7680] equal to 221 zero). 223 4.3. Metric Definitions 225 This section defines the REQUIRED aspects of the measureable Maximum 226 IP-layer Capacity metric (unless otherwise indicated) for 227 measurements between specified Source and Destination hosts: 229 Define the IP-layer capacity, Maximum_C(T,I,PM), to be the maximum 230 number of IP-layer bits (including header and data fields) that can 231 be transmitted from the Src host and correctly received by the Dst 232 host during one contiguous sub-interval, dt in length, during the 233 interval [T, T+I], and where the packet count of that single sub- 234 interval dtn in [T, T+I] indicates the maximum number of IP-layer 235 bits n0[dtn-1,dtn] which was captured as part of all packet counts n 236 for all dt in [T, T+I]. The interval dt SHOULD be set to a natural 237 number m so that T+I = T + m*dt with dtn - dtn-1 = dt and with 0 < n 238 <= m. Parameter PM represents the other performance metrics [see 239 section Related Round-Trip Delay and Loss Definitions below] and 240 their measurement results for the maximum IP-layer capacity. At 241 least one target performance threshold MUST be defined. If more than 242 one target performance threshold is defined, then the sub-interval 243 with maximum number of bits transmitted MUST meet all the target 244 performance thresholds. 246 [RG: this requires more explanation. Do you mean that all results 247 must hit the target performance level? Or is a one / two / k times 248 hit out of x trials [T, T+I] a criterium indicating that a target has 249 been reached? Or do you look for the maximum capacity without packet 250 loss and queuing or added RTD latency, respectively? acm: I think 251 I've clarified this in the text above... ] 253 Mathematically, this definition can be represented as: 255 max ( n0[dtn-1,dtn] ) 256 [T,T+I] 257 Maximum_C(T,I,PM) = ------------------------- 258 dt 259 where: 260 T T+I 261 _________________________________________ 262 | | | | | | | | | | | 263 dtn=0 1 2 3 4 5 6 7 8 9 m=10 265 Equation for Maximum Capacity 267 and: 269 o n0 is the total number of IP-layer header and payload bits that 270 can be transmitted in Standard Formed packets from the Src host 271 and correctly received by the Dst host during one contiguous sub- 272 interval, dt in length, during the interval [T, T+I], 274 o Maximum _C(T,I,PM) corresponds to the maximum value of n0 measured 275 in any sub-interval ending at dtn (meaning T + n*dt), divided by 276 the constant length of all sub-intervals, dt. 278 o all sub-intervals SHOULD be of equal duration. Choosing dt as 279 non-overlapping consecutive time intervals allows for a simple 280 implementation. [RG: a sliding window of dt has its charme too, 281 why exclude it? acm: seems less practical for real-time feedback.] 283 o [acm: I think this is a discussion point, not essential to the 284 definition] If traffic conditioning applies along a path for which 285 Maximum _C(T,I,PM) is to be determined, different values for dt 286 SHOULD be picked and measurements be executed during multiple 287 intervals [T, T+I]. Any single interval dt SHOULD be chosen so 288 that is an integer multiple of increasing values k times 289 serialisation delay of a path MTU at the physical interface speed 290 where traffic conditioning is expected. This should avoid taking 291 configured burst tolerance singletons as a valid Maximum 292 _C(T,I,PM) result. 294 o The bandwidth of the physical interface of the measurement device 295 must be higher than that of the interface whose Maximum _C(T,I,PM) 296 is to be measured. 298 In this definition, the m sub-intervals can be viewed as trials when 299 the Src host varies the transmitted packet rate, searching for the 300 maximum n0 that meets the PM criteria measured at the Dst host in a 301 test of duration, I. When the transmitted packet rate is held 302 constant at the Src host, the m sub-intervals may also be viewed as 303 trials to evaluate the stability of n0 and metric(s) in the PM list 304 over all dt-length intervals in I. 306 Measurements according to these definitions SHALL use UDP transport 307 layer. [RG: don't we need loss free transmission without added 308 latency as criteria and add that UDP without closed loop flow control 309 needs to be applied ? acm: I don't think we should require loss-free 310 transmission, because most networks allow a small loss ratio which 311 would likely appear to be zero loss in most trials. Methods may use 312 feedback, let's talk about how to diffentiate from flow-control] 314 4.4. Related Round-Trip Delay and Loss Definitions 316 RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip 317 Delay between the Src host and the Dst host over the interval 318 [T,T+I]. The statistics used to to summarize RTD[dtn-1,dtn] MAY 319 include the minimum, maximum, and mean. 321 RTL[dtn-1,dtn] is defined as a sample of the [RFC6673] Round-trip 322 Loss between the Src host and the Dst host over the interval [T,T+I]. 323 The statistics used to to summarize RTL[dtn-1,dtn] MAY include the 324 lost packet count and the lost packet ratio. 326 4.5. Discussion 328 Depending on the 330 4.6. Reporting the Metric 332 @@@@ not yet addressed, 334 5. Method of Measurement 336 This section needs development, based on Annex A of Y.1540. 338 The duration of a test, I, MUST be constrained in a production 339 network, since this is an active test method and it will likely cause 340 congestion on the Src to Dst host path during a test. 342 Additional Test methods and configurations may be provided in this 343 section, after review. 345 5.1. Load Rate Adjustment Algorithm (from udpst) 347 A table is pre-built defining all the offered load rates that will be 348 supported (R1 - Rn, in ascending order). Each rate is defined as 349 datagrams of size S, sent as a burst of count C, every time interval 350 T. While it is advantageous to use datagrams of as large a size as 351 possible, it may be prudent to use a slightly smaller maximum that 352 allows for secondary protocol headers and/or tunneling without 353 resulting in IP-layer fragmentation. 355 At the beginning of a test, the sender begins sending at rate R1 and 356 the receiver starts a feedback timer at interval F (while awaiting 357 inbound datagrams). As datagrams are received they are checked for 358 sequence number anomalies (loss, out-of-order, duplication, etc.) and 359 the delay variation is measured (one-way or round-trip). This 360 information is accumulated until the feedback timer F expires and a 361 status feedback message is sent from the receiver back to the sender, 362 to communicate this information. The accumulated statistics are then 363 reset by the receiver for the next feedback interval. As feedback 364 messages are received back at the sender, they are evaluated to 365 determine how to adjust the current offered load rate (Rx). 367 If the feedback indicates that there were no sequence number 368 anomalies AND the delay variation was below the lower threshold, the 369 offered load rate is increased. If congestion has not been confirmed 370 up to this point, the offered load rate is increased by more than one 371 rate (e.g., Rx+10). This allows the offered load to quickly reach a 372 near-maximum rate. Conversely, if congestion has been previously 373 confirmed, the offered load rate is only increased by one (Rx+1). 375 If the feedback indicates that sequence number anomalies were 376 detected OR the delay variation was above the upper threshold, the 377 offered load rate is decreased. If congestion has not been confirmed 378 up to this point, the offered load rate is decreased by more than one 379 rate (e.g., Rx-30). This allows the offered load to back off enough 380 to compensate for the fast initial ramp-up. Conversely, if 381 congestion has been previously confirmed, the offered load rate is 382 only decreased by one (Rx-1). 384 If the feedback indicates that there were no sequence number 385 anomalies AND the delay variation was above the lower threshold, but 386 below the upper threshold, the offered load rate is not changed. 387 This allows time for recent changes in the offered load rate to 388 stabilize, and the feedback to represent current conditions more 389 accurately. 391 Lastly, the method for confirming congestion is that there were 392 sequence number anomalies OR the delay variation was above the upper 393 threshold for two consecutive feedback intervals. 395 6. Reporting 397 This section needs development... 399 The following text TO BE REPLACED !!!! 401 ======================================= 403 The results SHOULD be reported in the format of a table with a row 404 for each of the tested frame sizes and Number of Flows. There SHOULD 405 be columns for the frame size with number of flows, and for the 406 resultant average frame count (or time) for each type of data stream 407 tested. 409 The number of tests Averaged for the Benchmark, N, MUST be reported. 411 The Minimum, Maximum, and Standard Deviation across all complete 412 tests SHOULD also be reported. 414 The Corrected DUT Restoration Time SHOULD also be reported, as 415 applicable. 417 +-------------------+-------------------+----------------+----------+ 418 | Frame Size, | Max IP-Layer | Min,Ave,StdDev | Time dt, | 419 | octets + # Flows | Capacity, bps | | Sec | 420 +-------------------+-------------------+----------------+----------+ 421 | 64,100 | 26000 | 25500,27000,20 | 0.00004 | 422 +-------------------+-------------------+----------------+----------+ 424 Maximum IP-layer Capacity Results 426 Static and configuration parameters: 428 Number of test repetitions, N 430 Minimum Step Size (during searches), in frames. 432 7. Security Considerations 434 Active metrics and measurements have a long history of security 435 considerations [add references to LMAP Framework, etc.]. 437 439 8. IANA Considerations 441 This memo makes no requests of IANA. 443 9. Acknowledgements 445 Thanks to 447 10. References 449 10.1. Normative References 451 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 452 Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242, 453 July 1991, . 455 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 456 Requirement Levels", BCP 14, RFC 2119, 457 DOI 10.17487/RFC2119, March 1997, 458 . 460 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 461 "Framework for IP Performance Metrics", RFC 2330, 462 DOI 10.17487/RFC2330, May 1998, 463 . 465 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 466 Network Interconnect Devices", RFC 2544, 467 DOI 10.17487/RFC2544, March 1999, 468 . 470 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 471 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 472 September 1999, . 474 [RFC2889] Mandeville, R. and J. Perser, "Benchmarking Methodology 475 for LAN Switching Devices", RFC 2889, 476 DOI 10.17487/RFC2889, August 2000, 477 . 479 [RFC5136] Chimento, P. and J. Ishac, "Defining Network Capacity", 480 RFC 5136, DOI 10.17487/RFC5136, February 2008, 481 . 483 [RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D. 484 Dugatkin, "IPv6 Benchmarking Methodology for Network 485 Interconnect Devices", RFC 5180, DOI 10.17487/RFC5180, May 486 2008, . 488 [RFC6201] Asati, R., Pignataro, C., Calabria, F., and C. Olvera, 489 "Device Reset Characterization", RFC 6201, 490 DOI 10.17487/RFC6201, March 2011, 491 . 493 [RFC6412] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology 494 for Benchmarking Link-State IGP Data-Plane Route 495 Convergence", RFC 6412, DOI 10.17487/RFC6412, November 496 2011, . 498 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 499 for Equal Cost Multipath Routing and Link Aggregation in 500 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 501 . 503 [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, 504 DOI 10.17487/RFC6673, August 2012, 505 . 507 [RFC6815] Bradner, S., Dubray, K., McQuaid, J., and A. Morton, 508 "Applicability Statement for RFC 2544: Use on Production 509 Networks Considered Harmful", RFC 6815, 510 DOI 10.17487/RFC6815, November 2012, 511 . 513 [RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet 514 Sizes for Additional Testing", RFC 6985, 515 DOI 10.17487/RFC6985, July 2013, 516 . 518 [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling 519 Framework for IP Performance Metrics (IPPM)", RFC 7312, 520 DOI 10.17487/RFC7312, August 2014, 521 . 523 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 524 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 525 May 2016, . 527 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 528 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 529 May 2017, . 531 [RFC8337] Mathis, M. and A. Morton, "Model-Based Metrics for Bulk 532 Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March 533 2018, . 535 [RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V. 536 Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for 537 the IP Performance Metrics (IPPM) Framework", RFC 8468, 538 DOI 10.17487/RFC8468, November 2018, 539 . 541 10.2. Informative References 543 [copycat] Edleine, K., Kuhlewind, K., Trammell, B., and B. Donnet, 544 "copycat: Testing Differential Treatment of New Transport 545 Protocols in the Wild (ANRW '17)", July 2017, 546 . 548 [RFC8239] Avramov, L. and J. Rapp, "Data Center Benchmarking 549 Methodology", RFC 8239, DOI 10.17487/RFC8239, August 2017, 550 . 552 [TST009] Morton, R. A., "ETSI GS NFV-TST 009 V3.1.1 (2018-10), 553 "Network Functions Virtualisation (NFV) Release 3; 554 Testing; Specification of Networking Benchmarks and 555 Measurement Methods for NFVI"", October 2018, 556 . 559 [VSPERF-b2b] 560 Morton, A., "Back2Back Testing Time Series (from CI)", 561 June 2017, . 565 [VSPERF-BSLV] 566 Morton, A. and S. Rao, "Evolution of Repeatability in 567 Benchmarking: Fraser Plugfest (Summary for IETF BMWG)", 568 July 2018, 569 . 573 Authors' Addresses 575 Al Morton 576 AT&T Labs 577 200 Laurel Avenue South 578 Middletown,, NJ 07748 579 USA 581 Phone: +1 732 420 1571 582 Fax: +1 732 368 1192 583 Email: acm@research.att.com 585 Ruediger Geib 586 Deutsche Telekom 587 Heinrich Hertz Str. 3-7 588 Darmstadt 64295 589 Germany 591 Phone: +49 6151 5812747 592 Email: Ruediger.Geib@telekom.de 594 Len Ciavattone 595 AT&T Labs 596 200 Laurel Avenue South 597 Middletown,, NJ 07748 598 USA 600 Email: lencia@att.com