idnits 2.17.1 draft-ietf-ippm-capacity-metric-method-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (June 29, 2020) is 1396 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC7680' is mentioned on line 240, but not defined == Unused Reference: 'RFC1242' is defined on line 880, but no explicit reference was found in the text == Unused Reference: 'RFC2889' is defined on line 903, but no explicit reference was found in the text == Unused Reference: 'RFC5180' is defined on line 917, but no explicit reference was found in the text == Unused Reference: 'RFC6201' is defined on line 922, but no explicit reference was found in the text == Unused Reference: 'RFC6412' is defined on line 927, but no explicit reference was found in the text == Unused Reference: 'RFC6985' is defined on line 947, but no explicit reference was found in the text == Unused Reference: 'RFC8468' is defined on line 969, but no explicit reference was found in the text == Unused Reference: 'RFC8239' is defined on line 982, but no explicit reference was found in the text == Unused Reference: 'TST009' is defined on line 990, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 1242 ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Downref: Normative reference to an Informational RFC: RFC 2544 ** Downref: Normative reference to an Informational RFC: RFC 2889 ** Downref: Normative reference to an Informational RFC: RFC 3148 ** Downref: Normative reference to an Informational RFC: RFC 5136 ** Downref: Normative reference to an Informational RFC: RFC 5180 ** Downref: Normative reference to an Informational RFC: RFC 6201 ** Downref: Normative reference to an Informational RFC: RFC 6412 ** Downref: Normative reference to an Informational RFC: RFC 6815 ** Downref: Normative reference to an Informational RFC: RFC 6985 ** Downref: Normative reference to an Informational RFC: RFC 7312 ** Downref: Normative reference to an Informational RFC: RFC 7799 ** Downref: Normative reference to an Experimental RFC: RFC 8337 ** Downref: Normative reference to an Informational RFC: RFC 8468 Summary: 15 errors (**), 0 flaws (~~), 12 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Updates: ???? (if approved) R. Geib 5 Intended status: Standards Track Deutsche Telekom 6 Expires: December 31, 2020 L. Ciavattone 7 AT&T Labs 8 June 29, 2020 10 Metrics and Methods for IP Capacity 11 draft-ietf-ippm-capacity-metric-method-02 13 Abstract 15 This memo revisits the problem of Network Capacity metrics first 16 examined in RFC 5136. The memo specifies a more practical Maximum 17 IP-layer Capacity metric definition catering for measurement 18 purposes, and outlines the corresponding methods of measurement. 20 Requirements Language 22 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 23 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 24 "OPTIONAL" in this document are to be interpreted as described in BCP 25 14[RFC2119] [RFC8174] when, and only when, they appear in all 26 capitals, as shown here. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on December 31, 2020. 45 Copyright Notice 47 Copyright (c) 2020 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . 4 64 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 4 65 4. General Parameters and Definitions . . . . . . . . . . . . . 5 66 5. IP-Layer Capacity Singleton Metric Definitions . . . . . . . 6 67 5.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6 68 5.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 69 5.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 6 70 5.4. Related Round-Trip Delay and Loss Definitions . . . . . . 8 71 5.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 8 72 5.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 8 73 6. Maximum IP-Layer Capacity Metric Definitions (Statistic) . . 8 74 6.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 8 75 6.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 8 76 6.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 9 77 6.4. Related Round-Trip Delay and Loss Definitions . . . . . . 10 78 6.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 10 79 6.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 11 80 7. IP-Layer Sender Bit Rate Singleton Metric Definitions . . . . 11 81 7.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 11 82 7.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 12 83 7.3. Metric Definition . . . . . . . . . . . . . . . . . . . . 12 84 7.4. Discussion . . . . . . . . . . . . . . . . . . . . . . . 12 85 7.5. Reporting the Metric . . . . . . . . . . . . . . . . . . 12 86 8. Method of Measurement . . . . . . . . . . . . . . . . . . . . 13 87 8.1. Load Rate Adjustment Algorithm . . . . . . . . . . . . . 13 88 8.2. Measurement Qualification or Verification . . . . . . . . 14 89 8.3. Measurement Considerations . . . . . . . . . . . . . . . 15 90 8.4. Running Code . . . . . . . . . . . . . . . . . . . . . . 17 91 9. Reporting Formats . . . . . . . . . . . . . . . . . . . . . . 17 92 9.1. Configuration and Reporting Data Formats . . . . . . . . 19 94 10. Security Considerations . . . . . . . . . . . . . . . . . . . 19 95 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 96 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 97 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 98 13.1. Normative References . . . . . . . . . . . . . . . . . . 19 99 13.2. Informative References . . . . . . . . . . . . . . . . . 21 100 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 102 1. Introduction 104 The IETF's efforts to define Network and Bulk Transport Capacity have 105 been chartered and progressed for over twenty years. Over that time, 106 the performance community has seen development of Informative 107 definitions in [RFC3148] for Framework for Bulk Transport Capacity 108 (BTC), RFC 5136 for Network Capacity and Maximum IP-layer Capacity, 109 and the Experimental metric definitions and methods in [RFC8337], 110 Model-Based Metrics for BTC. 112 This memo revisits the problem of Network Capacity metrics examined 113 first in [RFC3148] and later in [RFC5136]. Maximum IP-Layer Capacity 114 and [RFC3148] Bulk Transfer Capacity (goodput) are different metrics. 115 Max IP-layer Capacity is like the theoretical goal for goodput. 116 There are many metrics in [RFC5136], such as Available Capacity. 117 Measurements depend on the network path under test and the use case. 118 Here, the main use case is to assess the maximum capacity of the 119 access network, with specific performance criteria used in the 120 measurement. 122 This memo recognizes the importance of a definition of a Maximum IP- 123 layer Capacity Metric at a time when access speeds have increased 124 dramatically; a definition that is both practical and effective for 125 the performance community's needs, including Internet users. The 126 metric definition is intended to use Active Methods of Measurement 127 [RFC7799], and a method of measurement is included. 129 The most direct active measurement of IP-layer Capacity would use IP 130 packets, but in practice a transport header is needed to traverse 131 address and port translators. UDP offers the most direct assessment 132 possibility, and in the [copycat][copycat] measurement study to 133 investigate whether UDP is viable as a general Internet transport 134 protocol, the authors found that a high percentage of paths tested 135 support UDP transport. A number of liaisons have been exchanged on 136 this topic [ refs to ITU-T SG12, ETSI STQ, BBF liaisons ], discussing 137 the laboratory and field tests that support the UDP-based approach to 138 IP-layer Capacity measurement. 140 This memo also recognizes the many updates to the IP Performance 141 Metrics Framework [RFC2330] published over twenty years, and makes 142 use of [RFC7312] for Advanced Stream and Sampling Framework, and RFC 143 8468 [RFC8468]IPv4, IPv6, and IPv4-IPv6 Coexistence Updates. 145 2. Scope and Goals 147 The scope of this memo is to define a metric and corresponding method 148 to unambiguously perform Active measurements of Maximum IP-Layer 149 Capacity, along with related metrics and methods. 151 The main goal is to harmonize the specified metric and method across 152 the industry, and this memo is the vehicle through which working 153 group (and eventually, IETF) consensus will be captured and 154 communicated to achieve broad agreement, and possibly result in 155 changes in the specifications of other Standards Development 156 Organizations (SDO) (through the SDO's normal contribution process, 157 or through liaison exchange). 159 A local goal is to aid efficient test procedures where possible, and 160 to recommend reporting with additional interpretation of the results. 161 Also, to foster the development of protocol support for this metric 162 and method of measurement (all active testing protocols currently 163 defined by the IPPM WG are UDP-based, meeting a key requirement of 164 these methods). The supporting protocol development to measure this 165 metric according to the specified method is a key future contribution 166 to Internet measurement. 168 3. Motivation 170 As with any problem that has been worked for many years in various 171 SDOs without any special attempts at coordination, various solutions 172 for metrics and methods have emerged. 174 There are five factors that have changed (or begun to change) in the 175 2013-2019 time frame, and the presence of any one of them on the path 176 requires features in the measurement design to account for the 177 changes: 179 1. Internet access is no longer the bottleneck for many users. 181 2. Both speed and latency are important to user's satisfaction. 183 3. UDP's growing role in Transport, in areas where TCP once 184 dominated. 186 4. Content and applications moving physically closer to users. 188 5. Less emphasis on ISP gateway measurements, possibly due to less 189 traffic crossing ISP gateways in future. 191 4. General Parameters and Definitions 193 This section lists the REQUIRED input factors to specify a Sender or 194 Receiver metric. 196 o Src, the address of a host (such as the globally routable IP 197 address). 199 o Dst, the address of a host (such as the globally routable IP 200 address). 202 o i, the limit on the number of Hops a specific packet may visit as 203 it traverses from the host at Src to the host at Dst (such as the 204 TTL or Hop Limit). 206 o MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops). 208 o T0, the time at the start of measurement interval, when packets 209 are first transmitted from the Source. 211 o I, the duration of a measurement interval (default 10 sec) 213 o dt, the duration of N equal sub-intervals in I (default 1 sec) 215 o Tmax, a maximum waiting time for test packets to arrive at the 216 destination, set sufficiently long to disambiguate packets with 217 long delays from packets that are discarded (lost), such that the 218 distribution of one-way delay is not truncated. 220 o F, the number of different flows synthesized by the method 221 (default 1 flow) 223 o flow, the stream of packets with the same n-tuple of designated 224 header fields that (when held constant) result in identical 225 treatment in a multi-path decision (such as the decision taken in 226 load balancing). Note: The IPv6 flow label MAY be included in the 227 flow definition when routers have complied with [RFC6438] 228 guidelines at the Tunnel End Points (TEP), and the source of the 229 measurement is a TEP. 231 o Type-P, the complete description of the packets for which this 232 assessment applies (including the flow-defining fields). Note 233 that the UDP transport layer is one requirement specified below. 234 Type-P is a parallel concept to "population of interest" defined 235 in ITU-T Rec. Y.1540. 237 o PM, a list of fundamental metrics, such as loss, delay, and 238 reordering, and corresponding Target performance threshold. At 239 least one fundamental metric and Target performance threshold MUST 240 be supplied (such as One-way IP Packet Loss [RFC7680] equal to 241 zero). 243 A non-Parameter which is required for several metrics is defined 244 below: 246 o T, the host time of the *first* test packet's *arrival* as 247 measured at MP(Dst). There may be other packets sent between 248 source and destination hosts that are excluded, so this is the 249 time of arrival of the first packet used for measurement of the 250 metric. 252 Note that time stamps, sequnce numbers, etc. will be established by 253 the test protocol. 255 5. IP-Layer Capacity Singleton Metric Definitions 257 This section sets requirements for the following components to 258 support the Maximum IP-layer Capacity Metric. 260 5.1. Formal Name 262 Type-P-IP-Capacity, or informally called IP-layer Capacity. 264 Note that Type-P depends on the chosen method. 266 5.2. Parameters 268 This section lists the REQUIRED input factors to specify the metric, 269 beyond those listed in Section 4. 271 No additional Parameters are needed. 273 5.3. Metric Definitions 275 This section defines the REQUIRED aspects of the measureable IP-layer 276 Capacity metric (unless otherwise indicated) for measurements between 277 specified Source and Destination hosts: 279 Define the IP-layer capacity, C(T,I,PM), to be the number of IP-layer 280 bits (including header and data fields) in packets that can be 281 transmitted from the Src host and correctly received by the Dst host 282 during one contiguous sub-interval, dt. 284 The number of these IP-layer bits is designated n0[dtn,dtn+1] for a 285 specific dt. 287 When the packet size is known and of fixed size, the packet count 288 during a single sub-interval dt multiplied by the total bits in IP 289 header and data fields is equal to n0[dtn,dtn+1]. 291 Anticipating a Sample of Singletons, the interval dt SHOULD be set to 292 a natural number m so that T+I = T + m*dt with dtn+1 - dtn = dt and 293 with 1 <= n <= m. 295 Parameter PM represents other performance metrics [see section 296 Related Round-Trip Delay and Loss Definitions below]; their 297 measurement results SHALL be collected during measurement of IP-layer 298 Capacity and associated with the corresponding dtn for further 299 evaluation and reporting. 301 Mathematically, this definition can be represented as: 303 ( n0[dtn,dtn+1] ) 304 C(T,I,PM) = ------------------------- 305 dt 307 Equation for IP-Layer Capacity 309 and: 311 o n0 is the total number of IP-layer header and payload bits that 312 can be transmitted in Standard Formed packets from the Src host 313 and correctly received by the Dst host during one contiguous sub- 314 interval, dt in length, during the interval [T, T+I], 316 o C(T,I,PM) the IP-Layer Capacity, corresponds to the value of n0 317 measured in any sub-interval ending at dtn (meaning T + n*dt), 318 divided by the length of sub-interval, dt. 320 o all sub-intervals SHOULD be of equal duration. Choosing dt as 321 non-overlapping consecutive time intervals allows for a simple 322 implementation. 324 o The bit rate of the physical interface of the measurement device 325 must be higher than that of the link whose C(T,I,PM) is to be 326 measured. 328 Measurements according to these definitions SHALL use UDP transport 329 layer. 331 5.4. Related Round-Trip Delay and Loss Definitions 333 RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip 334 Delay between the Src host and the Dst host over the interval 335 [T,T+I]. The statistics used to to summarize RTD[dtn-1,dtn] MAY 336 include the minimum, maximum, and mean. 338 RTL[dtn-1,dtn] is defined as a sample of the [RFC6673] Round-trip 339 Loss between the Src host and the Dst host over the interval [T,T+I]. 340 The statistics used to to summarize RTL[dtn-1,dtn] MAY include the 341 lost packet count and the lost packet ratio. 343 5.5. Discussion 345 See the corresponding section for Maximum IP-Layer Capacity. 347 5.6. Reporting the Metric 349 The IP-Layer Capacity SHALL be reported with meaningful resolution, 350 in units of Megabits per second (Mbps). 352 The Related Round Trip Delay and/or Loss metric measurements for the 353 same Singleton SHALL be reported, also with meaningful resolution for 354 the values measured. 356 Individual Capacity measurements MAY be reported in a manner 357 consistent with the Maximum IP-Layer Capacity, see Section 9. 359 6. Maximum IP-Layer Capacity Metric Definitions (Statistic) 361 This section sets requirements for the following components to 362 support the Maximum IP-layer Capacity Metric. 364 6.1. Formal Name 366 Type-P-Max-IP-Capacity, or informally called Maximum IP-layer 367 Capacity. 369 Note that Type-P depends on the chosen method. 371 6.2. Parameters 373 This section lists the REQUIRED input factors to specify the metric, 374 beyond those listed in Section 4. 376 No additional Parameters or definitions are needed. 378 6.3. Metric Definitions 380 This section defines the REQUIRED aspects of the Maximum IP-layer 381 Capacity metric (unless otherwise indicated) for measurements between 382 specified Source and Destination hosts: 384 Define the Maximum IP-layer capacity, Maximum_C(T,I,PM), to be the 385 maximum number of IP-layer bits n0[dtn,dtn+1] that can be transmitted 386 in packets from the Src host and correctly received by the Dst host, 387 over all dt length intervals in [T, T+I], and meeting the PM 388 criteria. Equivalently the Maximum of a Sample of size m of 389 C(T,I,PM) collected during the interval [T, T+I] and meeting the PM 390 criteria. 392 The interval dt SHOULD be set to a natural number m so that T+I = T + 393 m*dt with dtn+1 - dtn = dt and with 1 <= n <= m. 395 Parameter PM represents the other performance metrics [see section 396 Related Round-Trip Delay and Loss Definitions below] and their 397 measurement results for the maximum IP-layer capacity. At least one 398 target performance threshold (PM criterion) MUST be defined. If more 399 than one target performance threshold is defined, then the sub- 400 interval with maximum number of bits transmitted MUST meet all the 401 target performance thresholds. 403 Mathematically, this definition can be represented as: 405 max ( n0[dtn,dtn+1] ) 406 [T,T+I] 407 Maximum_C(T,I,PM) = ------------------------- 408 dt 409 where: 410 T T+I 411 _________________________________________ 412 | | | | | | | | | | | 413 dtn=1 2 3 4 5 6 7 8 9 10 n+1 414 n=m 416 Equation for Maximum Capacity 418 and: 420 o n0 is the total number of IP-layer header and payload bits that 421 can be transmitted in Standard Formed packets from the Src host 422 and correctly received by the Dst host during one contiguous sub- 423 interval, dt in length, during the interval [T, T+I], 425 o Maximum _C(T,I,PM) the Maximum IP-Layer Capacity, corresponds to 426 the maximum value of n0 measured in any sub-interval ending at dtn 427 (meaning T + n*dt), divided by the constant length of all sub- 428 intervals, dt. 430 o all sub-intervals SHOULD be of equal duration. Choosing dt as 431 non-overlapping consecutive time intervals allows for a simple 432 implementation. 434 o The bit rate of the physical interface of the measurement systems 435 must be higher than that of the link whose Maximum _C(T,I,PM) is 436 to be measured (the bottleneck link). 438 In this definition, the m sub-intervals can be viewed as trials when 439 the Src host varies the transmitted packet rate, searching for the 440 maximum n0 that meets the PM criteria measured at the Dst host in a 441 test of duration, I. When the transmitted packet rate is held 442 constant at the Src host, the m sub-intervals may also be viewed as 443 trials to evaluate the stability of n0 and metric(s) in the PM list 444 over all dt-length intervals in I. 446 Measurements according to these definitions SHALL use UDP transport 447 layer. 449 6.4. Related Round-Trip Delay and Loss Definitions 451 RTD[dtn,dtn+1] is defined as a sample of the [RFC2681] Round-trip 452 Delay between the Src host and the Dst host over the interval 453 [T,T+I], and corresponds to the dt interval containing 454 Maximum_C(T,I,PM). The statistics used to to summarize 455 RTD[dtn,dtn+1] MAY include the minimum, maximum, and mean. 457 RTL[dtn,dtn+1] is defined as a sample of the [RFC6673] Round-trip 458 Loss between the Src host and the Dst host over the interval [T,T+I] 459 and corresponds to the dt interval containing Maximum_C(T,I,PM). The 460 statistics used to to summarize RTL[dtn-1,dtn] MAY include the lost 461 packet count and the lost packet ratio. 463 6.5. Discussion 465 If traffic conditioning applies along a path for which Maximum 466 _C(T,I,PM) is to be determined, different values for dt SHOULD be 467 picked and measurements be executed during multiple intervals [T, 468 T+I]. Any single interval dt SHOULD be chosen so that is an integer 469 multiple of increasing values k times serialisation delay of a path 470 MTU at the physical interface speed where traffic conditioning is 471 expected. This should avoid taking configured burst tolerance 472 singletons as a valid Maximum _C(T,I,PM) result. 474 A Maximum_C(T,I,PM) without any indication of bottleneck congestion, 475 be that an increasing latency, packet loss or ECN marks during a 476 measurement interval I, is likely to underestimate Maximum_C(T,I,PM). 478 6.6. Reporting the Metric 480 The Maximum IP-Layer Capacity SHALL be reported with meaningful 481 resolution, in units of Megabits per second. 483 The Related Round Trip Delay and/or Loss metric measurements for the 484 same Singleton SHALL be reported, also with meaningful resolution for 485 the values measured. 487 When there are demonstrated and repeatable Capacity modes in the 488 Sample, then the Maximum IP-Layer Capacity SHALL be reported for each 489 mode, along with the relative time from the beginning of the stream 490 that the mode was observed to be present. Bimodal Maxima have been 491 observed with some services, sometimes called a "turbo mode" 492 intending to deliver short transfers more quickly, or reduce the 493 initial buffering time for some video streams. Note that modes 494 lasting less than dt duration will not be detected. 496 Some transmission technologies have multiple methods of operation 497 that may be activated when channel conditions degrade or improve, and 498 these transmission methods may determine the Maximum IP-Layer 499 Capacity. Examples include line-of-sight microwave modulator 500 constellations, or cellular modem technologies where the changes may 501 be initiated by a user moving from one coverage area to another. 502 Operation in the different transmission methods may be observed over 503 time, but the modes of Maximum IP-Layer Capacity will not be 504 activated deterministically as with the "turbo mode" described in the 505 paragraph above. 507 7. IP-Layer Sender Bit Rate Singleton Metric Definitions 509 This section sets requirements for the following components to 510 support the IP-layer Sender Bitrate Metric. 512 7.1. Formal Name 514 Type-P-IP-Sender-Bit-Rate, or informally called IP-layer Sender 515 Bitrate. 517 Note that Type-P depends on the chosen method. 519 7.2. Parameters 521 This section lists the REQUIRED input factors to specify the metric, 522 beyond those listed in Section 4. 524 o S, the duration of the measurement interval at the Source 526 o st, the nominal duration of N sub-intervals in S (default = 0.05 527 seconds) 529 S SHALL be longer than I, primarily to account for on-demand 530 activation of the path, or any preamble to testing required. 532 st SHOULD be much smaller than the sub-interval dt. The st parameter 533 does not have relevance when the Source is transmitting at a fixed 534 rate throughout S. 536 7.3. Metric Definition 538 This section defines the REQUIRED aspects of the IP-layer Sender 539 Bitrate metric (unless otherwise indicated) for measurements at the 540 specified Source on packets addressed for the intended Destination 541 host and matching the required Type-P: 543 Define the IP-layer Sender Bit Rate, B(S,st), to be the number of IP- 544 layer bits (including header and data fields) that are transmitted 545 from the Source during one contiguous sub-interval, st, during the 546 test interval S (where S SHALL be longer than I), and where the 547 fixed-size packet count during that single sub-interval st also 548 provides the number of IP-layer bits in any interval: n0[stn-1,stn]. 550 Measurements according to these definitions SHALL use UDP transport 551 layer. Any feedback from Dst host to Src host received by Src host 552 during an interval [stn-1,stn] MUST NOT result in an adaptation of 553 the Src host traffic conditioning during this interval. 555 7.4. Discussion 557 Both the Sender and Receiver or (source and destination) bit rates 558 SHOULD be assessed as part of a measurement. 560 7.5. Reporting the Metric 562 The IP-Layer Sender Bit Rate SHALL be reported with meaningful 563 resolution, in units of Megabits per second. 565 Individual IP-Layer Sender Bit Rate measurements are discussed 566 further in Section 9. 568 8. Method of Measurement 570 The duration of a test, I, MUST be constrained in a production 571 network, since this is an active test method and it will likely cause 572 congestion on the Src to Dst host path during a test. 574 Additional Test methods and configurations may be provided in this 575 section, after review and further testing. 577 8.1. Load Rate Adjustment Algorithm 579 A table is pre-built defining all the offered load rates that will be 580 supported (R1 - Rn, in ascending order). Each rate is defined as 581 datagrams of size S, sent as a burst of count C, every time interval 582 T. While it is advantageous to use datagrams of as large a size as 583 possible, it may be prudent to use a slightly smaller maximum that 584 allows for secondary protocol headers and/or tunneling without 585 resulting in IP-layer fragmentation. 587 At the beginning of a test, the sender begins sending at rate R1 and 588 the receiver starts a feedback timer at interval F (while awaiting 589 inbound datagrams). As datagrams are received they are checked for 590 sequence number anomalies (loss, out-of-order, duplication, etc.) and 591 the delay variation is measured (one-way or round-trip). This 592 information is accumulated until the feedback timer F expires and a 593 status feedback message is sent from the receiver back to the sender, 594 to communicate this information. The accumulated statistics are then 595 reset by the receiver for the next feedback interval. As feedback 596 messages are received back at the sender, they are evaluated to 597 determine how to adjust the current offered load rate (Rx). 599 If the feedback indicates that there were no sequence number 600 anomalies AND the delay variation was below the lower threshold, the 601 offered load rate is increased. If congestion has not been confirmed 602 up to this point, the offered load rate is increased by more than one 603 rate (e.g., Rx+10). This allows the offered load to quickly reach a 604 near-maximum rate. Conversely, if congestion has been previously 605 confirmed, the offered load rate is only increased by one (Rx+1). 607 If the feedback indicates that sequence number anomalies were 608 detected OR the delay variation was above the upper threshold, the 609 offered load rate is decreased. If congestion is confirmed by the 610 current feedback message being processed, the offered load rate is 611 decreased by more than one rate (e.g., Rx-30). This one-time 612 reduction is intended to compensate for the fast initial ramp-up. In 613 all other cases, the offered load rate is only decreased by one (Rx- 614 1). 616 If the feedback indicates that there were no sequence number 617 anomalies AND the delay variation was above the lower threshold, but 618 below the upper threshold, the offered load rate is not changed. 619 This allows time for recent changes in the offered load rate to 620 stabilize, and the feedback to represent current conditions more 621 accurately. 623 Lastly, the method for confirming congestion is that there were 624 sequence number anomalies OR the delay variation was above the upper 625 threshold for two consecutive feedback intervals. The algorithm 626 described above is also presented in ITU-T Rec. Y.1540, 2020 627 version[Y.1540], in Annexes A and B, and implemented in the reference 628 for Section 8.4, Running Code. 630 8.2. Measurement Qualification or Verification 632 When assessing a Maximum rate as the metric specifies, artificially 633 high (optimistic) values might be measured until some buffer on the 634 path is filled. Other causes include bursts of back-to-back packets 635 with idle intervals delivered by a path, while the measurement 636 interval (dt) is small and aligned with the bursts. The artificial 637 values might result in an un-sustainable Maximum Capacity observed 638 when the method of measurement is searching for the Maximum, and that 639 would not do. This situation is different from the bi-modal service 640 rates (discussed under Reporting), which are characterized by a 641 multi-second duration (much longer than the measured RTT) and 642 repeatable behavior. 644 There are many ways that the Method of Measurement could handle this 645 false-max issue. The default value for measurement of singletons (dt 646 = 1 second) has proven to a be of practical value during tests of 647 this method, allows the bimodal service rates to be characterized, 648 and it has an obvious alignment with the reporting units (Mbps). 650 Another approach comes from Section 24 of RFC 2544[RFC2544] and its 651 discussion of Trial duration, where relatively short trials conducted 652 as part of the search are followed by longer trials to make the final 653 determination. In the production network, measurements of singletons 654 and samples (the terms for trials and tests of Lab Benchmarking) must 655 be limited in duration because they may be service-affecting. But 656 there is sufficient value in repeating a sample with a fixed sending 657 rate determined by the previous search for the Max IP-layer Capacity, 658 to qualify the result in terms of the other performance metrics 659 measured at the same time. 661 A qualification measurement for the search result is a subsequent 662 measurement, sending at a fixed 99.x % of the Max IP-layer Capacity 663 for I, or an indefinite period. The same Max Capacity Metric is 664 applied, and the Qualification for the result is a sample without 665 packet loss or a growing minimum delay trend in subsequent singletons 666 (or each dt of the measurement interval, I). Samples exhibiting 667 losses or increasing queue occupation require a repeated search and/ 668 or test at reduced fixed sender rate for qualification. 670 Here, as with any Active Capacity test, the test duration must be 671 kept short. 10 second tests for each direction of transmission are 672 common today. The default measurement interval specified here is I = 673 10 seconds). In combination with a fast search method and user- 674 network coordination, the concerns raised in RFC 6815[RFC6815] are 675 alleviated. The method for assessing Max IP Capacity is different 676 from classic [RFC2544] methods: they use short term load adjustment 677 and are sensitive to loss and delay, like other congestion control 678 algorithms used on the Internet every day. 680 8.3. Measurement Considerations 682 In general, the wide-spread measurements that this memo encourages 683 will encounter wide-spread behaviors. The bimodal IP Capacity 684 behaviors already discussed in Section 6.6 are good examples. 686 In general, it is RECOMMENDED to locate test endpoints as close to 687 the intended measured link(s) as practical (this is not always 688 possible for reasons of scale; there is a limit on number of test 689 endpoints coming from many perspecitves, management and measurement 690 traffic for example). 692 The path measured may be state-full based on many factors, and the 693 Parameter "Time of day" when a test starts may not be enough 694 information. Repeatable testing may require the time from the 695 beginning of a measured flow, and how the flow is constructed 696 including how much traffic has already been sent on that flow when a 697 state-change is observed, because the state-change may be based on 698 time or bytes sent or both. 700 Many different traffic shapers and on-demand access technologies may 701 be encountered, as anticipated in [RFC7312], and play a key role in 702 measurement results. Methods MUST be prepared to provide a short 703 preamble transmission to activate on-demand access, and to discard 704 the preamble from subsequent test results. 706 Conditions which might be encountered during measurement, where 707 packet losses may occur independently from the measurement sending 708 rate: 710 1. Congestion of an interconnection or backbone interface may appear 711 as packet losses distributed over time in the test stream, due to 712 much higher rate interfaces in the backbone. 714 2. Packet loss due to use of Random Early Detection (RED) or other 715 active queue management. 717 3. There may be only small delay variation independent of sending 718 rate under these conditions, too. 720 4. Persistent competing traffic on measurement paths that include 721 shared media may cause random packet losses in the test stream. 723 It is possible to mitigate these conditions using the flexibility of 724 the load-rate adjusting algorithm described in Section 8.1 above 725 (tuning specific parameters). 727 In general, results depend on the sending stream characteristics; the 728 measurement community has known this for a long time, and needs to 729 keep it front of mind. Although the default is a single flow (F=1) 730 for testing, use of multiple flows may be advantageous for the 731 following reasons: 733 1. the test hosts may be able to create higher load than with a 734 single flow, or parallel test hosts may be used to generate 1 735 flow each. 737 2. there may be link aggregation present (flow-based load balancing) 738 and multiple flows are needed to occupy each member of the 739 aggregate. 741 3. access policies may limit the IP-Layer Capacity depending on the 742 Type-P of packets, possibly reserving capacity for various stream 743 types. 745 Each flow would be controlled using its own implementation of the 746 Load Adjustment (Search) Algorithm. 748 As testing continues, implementers should expect some evolution in 749 the methods. The ITU-T has published a Supplement (60) to the 750 Y-series of Recommendations, "Interpreting ITU-T Y.1540 maximum IP- 751 layer capacity measurements", [Y.Sup60], which is the result of 752 continued testing with the metric and method described here. 754 8.4. Running Code 756 Much of the development of the method and comparisons with existing 757 methods conducted at IETF Hackathons and elsewhere have been based on 758 the example udpst Linux measurement tool (which is a working 759 reference for further development). The current project: 761 o is a utility that can function as a client or server daemon 763 o is written in C, and built with gcc (release 9.3) and its standard 764 run-time libraries 766 o allows configuration of most of the parameters described in 767 Sections 4 and 7. 769 Watch this space for the URL to the opensource project. 771 9. Reporting Formats 773 The singleton IP-Layer Capacity results SHOULD be accompanied by the 774 context under which they were measured. 776 o timestamp (especially the time when the maximum was observed in 777 dtn) 779 o source and destination (by IP or other meaningful ID) 781 o other inner parameters of the test case (Section 4) 783 o outer parameters, such as "done in motion" or other factors 784 belonging to the context of the measurement 786 o result validity (indicating cases where the process was somehow 787 interrupted or the attempt failed) 789 o a field where unusual circumstances could be documented, and 790 another one for "ignore/mask out" purposes in further processing 792 The Maximum IP-Layer Capacity results SHOULD be reported in the 793 format of a table with a row for each of the test Phases and Number 794 of Flows. There SHOULD be columns for the phases with number of 795 flows, and for the resultant Maximum IP-Layer Capacity results for 796 the aggregate and each flow tested. 798 As mentioned in Section 6.6, bi-modal (or multi-modal) maxima SHALL 799 be reported for each mode separately. 801 +--------------+----------------------+-----------+-----------------+ 802 | Phase, # | Max IP-Layer | Loss | RTT min, max, | 803 | Flows | Capacity, Mbps | Ratio | msec | 804 +--------------+----------------------+-----------+-----------------+ 805 | Search,1 | 967.31 | 0.0002 | 30, 58 | 806 | Verify,1 | 966.00 | 0.0000 | 30, 38 | 807 +--------------+----------------------+-----------+-----------------+ 809 Maximum IP-layer Capacity Results 811 Static and configuration parameters: 813 The sub-interval time, dt, MUST accompany a report of Maximum IP- 814 Layer Capacity results, and the remaining Parameters from Section 4, 815 General Parameters. 817 The PM list metrics corresponding to the sub-interval where the 818 Maximum Capacity occurred MUST accompany a report of Maximum IP-Layer 819 Capacity results, for each test phase. 821 The IP-Layer Sender Bit rate results SHOULD be reported in the format 822 of a table with a row for each of the test Phases, sub-intervals (st) 823 and Number of Flows. There SHOULD be columns for the phases with 824 number of flows, and for the resultant IP-Layer Sender Bit rate 825 results for the aggregate and each flow tested. 827 +------------------------+-------------+-----------------------+----+ 828 | Phase, Flow or | st, sec | Sender Bit Rate, Mbps | ?? | 829 | Aggregate | | | | 830 +------------------------+-------------+-----------------------+----+ 831 | Search,1 | 0.00 - 0.05 | 345 | __ | 832 | Search,2 | 0.00 - 0.05 | 289 | __ | 833 | Search,Agg | 0.00 - 0.05 | 634 | __ | 834 +------------------------+-------------+-----------------------+----+ 836 IP-layer Sender Bit Rate Results 838 Static and configuration parameters: 840 The subinterval time, st, MUST accompany a report of Sender IP-Layer 841 Bit Rate results. 843 Also, the values of the remaining Parameters from Section 4, General 844 Parameters, MUST be reported. 846 9.1. Configuration and Reporting Data Formats 848 As a part of the multi-Standards Development Organization (SDO) 849 harmonization of this metric and method of measurement, one of the 850 areas where the Broadband Forum (BBF) contributed its expertise was 851 in the definition of an information model and data model for 852 configuration and reporting. These models are consistent with the 853 metric parameters and default values specified as lists is this memo. 854 [TR-471] provides the Information model that was used to prepare a 855 full data model in related BBF work. The BBF has als carefully 856 considered topics within its purvue, such as placement of measurement 857 systems within the access archtecture. 859 10. Security Considerations 861 Active metrics and measurements have a long history of security 862 considerations [add references to LMAP Framework, etc.]. 864 866 11. IANA Considerations 868 This memo makes no requests of IANA. 870 12. Acknowledgements 872 Thanks to Joachim Fabini, Matt Mathis, Ignacio Alvarez-Hamelin, and 873 Wolfgang Balzer for their extensive comments on the memo and related 874 topics. 876 13. References 878 13.1. Normative References 880 [RFC1242] Bradner, S., "Benchmarking Terminology for Network 881 Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242, 882 July 1991, . 884 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 885 Requirement Levels", BCP 14, RFC 2119, 886 DOI 10.17487/RFC2119, March 1997, 887 . 889 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 890 "Framework for IP Performance Metrics", RFC 2330, 891 DOI 10.17487/RFC2330, May 1998, 892 . 894 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 895 Network Interconnect Devices", RFC 2544, 896 DOI 10.17487/RFC2544, March 1999, 897 . 899 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 900 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 901 September 1999, . 903 [RFC2889] Mandeville, R. and J. Perser, "Benchmarking Methodology 904 for LAN Switching Devices", RFC 2889, 905 DOI 10.17487/RFC2889, August 2000, 906 . 908 [RFC3148] Mathis, M. and M. Allman, "A Framework for Defining 909 Empirical Bulk Transfer Capacity Metrics", RFC 3148, 910 DOI 10.17487/RFC3148, July 2001, 911 . 913 [RFC5136] Chimento, P. and J. Ishac, "Defining Network Capacity", 914 RFC 5136, DOI 10.17487/RFC5136, February 2008, 915 . 917 [RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D. 918 Dugatkin, "IPv6 Benchmarking Methodology for Network 919 Interconnect Devices", RFC 5180, DOI 10.17487/RFC5180, May 920 2008, . 922 [RFC6201] Asati, R., Pignataro, C., Calabria, F., and C. Olvera, 923 "Device Reset Characterization", RFC 6201, 924 DOI 10.17487/RFC6201, March 2011, 925 . 927 [RFC6412] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology 928 for Benchmarking Link-State IGP Data-Plane Route 929 Convergence", RFC 6412, DOI 10.17487/RFC6412, November 930 2011, . 932 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 933 for Equal Cost Multipath Routing and Link Aggregation in 934 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 935 . 937 [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, 938 DOI 10.17487/RFC6673, August 2012, 939 . 941 [RFC6815] Bradner, S., Dubray, K., McQuaid, J., and A. Morton, 942 "Applicability Statement for RFC 2544: Use on Production 943 Networks Considered Harmful", RFC 6815, 944 DOI 10.17487/RFC6815, November 2012, 945 . 947 [RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet 948 Sizes for Additional Testing", RFC 6985, 949 DOI 10.17487/RFC6985, July 2013, 950 . 952 [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling 953 Framework for IP Performance Metrics (IPPM)", RFC 7312, 954 DOI 10.17487/RFC7312, August 2014, 955 . 957 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 958 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 959 May 2016, . 961 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 962 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 963 May 2017, . 965 [RFC8337] Mathis, M. and A. Morton, "Model-Based Metrics for Bulk 966 Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March 967 2018, . 969 [RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V. 970 Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for 971 the IP Performance Metrics (IPPM) Framework", RFC 8468, 972 DOI 10.17487/RFC8468, November 2018, 973 . 975 13.2. Informative References 977 [copycat] Edleine, K., Kuhlewind, K., Trammell, B., and B. Donnet, 978 "copycat: Testing Differential Treatment of New Transport 979 Protocols in the Wild (ANRW '17)", July 2017, 980 . 982 [RFC8239] Avramov, L. and J. Rapp, "Data Center Benchmarking 983 Methodology", RFC 8239, DOI 10.17487/RFC8239, August 2017, 984 . 986 [TR-471] Morton, A., "Broadband Forum TR-471: IP Layer Capacity 987 Metrics and Measurement", July 2020, 988 . 990 [TST009] Morton, R. A., "ETSI GS NFV-TST 009 V3.1.1 (2018-10), 991 "Network Functions Virtualisation (NFV) Release 3; 992 Testing; Specification of Networking Benchmarks and 993 Measurement Methods for NFVI"", October 2018, 994 . 997 [Y.1540] Y.1540, I. R., "Internet protocol data communication 998 service - IP packet transfer and availability performance 999 parameters", January 2020, 1000 . 1002 [Y.Sup60] Morton, A., "Recommendation Y.Sup60, (04/20) Interpreting 1003 ITU-T Y.1540 maximum IP-layer capacity measurements", June 1004 2020, . 1006 Authors' Addresses 1008 Al Morton 1009 AT&T Labs 1010 200 Laurel Avenue South 1011 Middletown,, NJ 07748 1012 USA 1014 Phone: +1 732 420 1571 1015 Fax: +1 732 368 1192 1016 Email: acm@research.att.com 1018 Ruediger Geib 1019 Deutsche Telekom 1020 Heinrich Hertz Str. 3-7 1021 Darmstadt 64295 1022 Germany 1024 Phone: +49 6151 5812747 1025 Email: Ruediger.Geib@telekom.de 1027 Len Ciavattone 1028 AT&T Labs 1029 200 Laurel Avenue South 1030 Middletown,, NJ 07748 1031 USA 1033 Email: lencia@att.com