idnits 2.17.1 draft-ietf-ippm-capacity-metric-method-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (September 10, 2020) is 1324 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Downref: Normative reference to an Informational RFC: RFC 2544 ** Downref: Normative reference to an Informational RFC: RFC 3148 ** Downref: Normative reference to an Informational RFC: RFC 5136 ** Downref: Normative reference to an Informational RFC: RFC 6815 ** Downref: Normative reference to an Informational RFC: RFC 7312 ** Downref: Normative reference to an Informational RFC: RFC 7594 ** Downref: Normative reference to an Informational RFC: RFC 7799 ** Downref: Normative reference to an Experimental RFC: RFC 8337 ** Downref: Normative reference to an Informational RFC: RFC 8468 Summary: 10 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Updates: ???? (if approved) R. Geib 5 Intended status: Standards Track Deutsche Telekom 6 Expires: March 14, 2021 L. Ciavattone 7 AT&T Labs 8 September 10, 2020 10 Metrics and Methods for One-way IP Capacity 11 draft-ietf-ippm-capacity-metric-method-04 13 Abstract 15 This memo revisits the problem of Network Capacity metrics first 16 examined in RFC 5136. The memo specifies a more practical Maximum 17 IP-layer Capacity metric definition catering for measurement 18 purposes, and outlines the corresponding methods of measurement. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at https://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on March 14, 2021. 37 Copyright Notice 39 Copyright (c) 2020 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (https://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 55 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 56 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . 4 57 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 4 58 4. General Parameters and Definitions . . . . . . . . . . . . . 5 59 5. IP-Layer Capacity Singleton Metric Definitions . . . . . . . 6 60 5.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6 61 5.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 62 5.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 6 63 5.4. Related Round-Trip Delay and One-way Loss Definitions . . 8 64 5.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 8 65 5.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 8 66 6. Maximum IP-Layer Capacity Metric Definitions (Statistic) . . 8 67 6.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 9 68 6.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 9 69 6.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 9 70 6.4. Related Round-Trip Delay and One-way Loss Definitions . . 11 71 6.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 11 72 6.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 11 73 7. IP-Layer Sender Bit Rate Singleton Metric Definitions . . . . 12 74 7.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 12 75 7.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 12 76 7.3. Metric Definition . . . . . . . . . . . . . . . . . . . . 12 77 7.4. Discussion . . . . . . . . . . . . . . . . . . . . . . . 13 78 7.5. Reporting the Metric . . . . . . . . . . . . . . . . . . 13 79 8. Method of Measurement . . . . . . . . . . . . . . . . . . . . 13 80 8.1. Load Rate Adjustment Algorithm . . . . . . . . . . . . . 13 81 8.2. Measurement Qualification or Verification . . . . . . . . 15 82 8.3. Measurement Considerations . . . . . . . . . . . . . . . 16 83 8.4. Running Code . . . . . . . . . . . . . . . . . . . . . . 17 84 9. Reporting Formats . . . . . . . . . . . . . . . . . . . . . . 18 85 9.1. Configuration and Reporting Data Formats . . . . . . . . 20 86 10. Security Considerations . . . . . . . . . . . . . . . . . . . 20 87 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 88 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 89 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 90 13.1. Normative References . . . . . . . . . . . . . . . . . . 21 91 13.2. Informative References . . . . . . . . . . . . . . . . . 23 92 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 94 1. Introduction 96 The IETF's efforts to define Network and Bulk Transport Capacity have 97 been chartered and progressed for over twenty years. Over that time, 98 the performance community has seen development of Informative 99 definitions in [RFC3148] for Framework for Bulk Transport Capacity 100 (BTC), RFC 5136 for Network Capacity and Maximum IP-layer Capacity, 101 and the Experimental metric definitions and methods in [RFC8337], 102 Model-Based Metrics for BTC. 104 This memo revisits the problem of Network Capacity metrics examined 105 first in [RFC3148] and later in [RFC5136]. Maximum IP-Layer Capacity 106 and [RFC3148] Bulk Transfer Capacity (goodput) are different metrics. 107 Max IP-layer Capacity is like the theoretical goal for goodput. 108 There are many metrics in [RFC5136], such as Available Capacity. 109 Measurements depend on the network path under test and the use case. 110 Here, the main use case is to assess the maximum capacity of the 111 access network, with specific performance criteria used in the 112 measurement. 114 This memo recognizes the importance of a definition of a Maximum IP- 115 layer Capacity Metric at a time when access speeds have increased 116 dramatically; a definition that is both practical and effective for 117 the performance community's needs, including Internet users. The 118 metric definition is intended to use Active Methods of Measurement 119 [RFC7799], and a method of measurement is included. 121 The most direct active measurement of IP-layer Capacity would use IP 122 packets, but in practice a transport header is needed to traverse 123 address and port translators. UDP offers the most direct assessment 124 possibility, and in the [copycat] measurement study to investigate 125 whether UDP is viable as a general Internet transport protocol, the 126 authors found that a high percentage of paths tested support UDP 127 transport. A number of liaisons have been exchanged on this topic 128 [LS-SG12-A] [LS-SG12-B], discussing the laboratory and field tests 129 that support the UDP-based approach to IP-layer Capacity measurement. 131 This memo also recognizes the many updates to the IP Performance 132 Metrics Framework [RFC2330] published over twenty years, and makes 133 use of [RFC7312] for Advanced Stream and Sampling Framework, and 134 [RFC8468] with IPv4, IPv6, and IPv4-IPv6 Coexistence Updates. 136 1.1. Requirements Language 138 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 139 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 140 "OPTIONAL" in this document are to be interpreted as described in BCP 141 14[RFC2119] [RFC8174] when, and only when, they appear in all 142 capitals, as shown here. 144 2. Scope and Goals 146 The scope of this memo is to define a metric and corresponding method 147 to unambiguously perform Active measurements of Maximum IP-Layer 148 Capacity, along with related metrics and methods. 150 The main goal is to harmonize the specified metric and method across 151 the industry, and this memo is the vehicle through which working 152 group (and eventually, IETF) consensus will be captured and 153 communicated to achieve broad agreement, and possibly result in 154 changes in the specifications of other Standards Development 155 Organizations (SDO) (through the SDO's normal contribution process, 156 or through liaison exchange). 158 A local goal is to aid efficient test procedures where possible, and 159 to recommend reporting with additional interpretation of the results. 160 Also, to foster the development of protocol support for this metric 161 and method of measurement (all active testing protocols currently 162 defined by the IPPM WG are UDP-based, meeting a key requirement of 163 these methods). The supporting protocol development to measure this 164 metric according to the specified method is a key future contribution 165 to Internet measurement. 167 3. Motivation 169 As with any problem that has been worked for many years in various 170 SDOs without any special attempts at coordination, various solutions 171 for metrics and methods have emerged. 173 There are five factors that have changed (or begun to change) in the 174 2013-2019 time frame, and the presence of any one of them on the path 175 requires features in the measurement design to account for the 176 changes: 178 1. Internet access is no longer the bottleneck for many users. 180 2. Both speed and latency are important to user's satisfaction. 182 3. UDP's growing role in Transport, in areas where TCP once 183 dominated. 185 4. Content and applications moving physically closer to users. 187 5. Less emphasis on ISP gateway measurements, possibly due to less 188 traffic crossing ISP gateways in future. 190 4. General Parameters and Definitions 192 This section lists the REQUIRED input factors to specify a Sender or 193 Receiver metric. 195 o Src, the address of a host (such as the globally routable IP 196 address). 198 o Dst, the address of a host (such as the globally routable IP 199 address). 201 o i, the limit on the number of Hops a specific packet may visit as 202 it traverses from the host at Src to the host at Dst (such as the 203 TTL or Hop Limit). 205 o MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops). 207 o T0, the time at the start of measurement interval, when packets 208 are first transmitted from the Source. 210 o I, the nominal duration of a measurement interval at the 211 destination (default 10 sec) 213 o dt, the nominal duration of m equal sub-intervals in I at the 214 destination (default 1 sec) 216 o dtn, a specific sub-interval, n, one of m sub-intervals in I 218 o Tmax, a maximum waiting time for test packets to arrive at the 219 destination, set sufficiently long to disambiguate packets with 220 long delays from packets that are discarded (lost), such that the 221 distribution of one-way delay is not truncated. 223 o F, the number of different flows synthesized by the method 224 (default 1 flow) 226 o flow, the stream of packets with the same n-tuple of designated 227 header fields that (when held constant) result in identical 228 treatment in a multi-path decision (such as the decision taken in 229 load balancing). Note: The IPv6 flow label MAY be included in the 230 flow definition when routers have complied with [RFC6438] 231 guidelines at the Tunnel End Points (TEP), and the source of the 232 measurement is a TEP. 234 o Type-P, the complete description of the test packets for which 235 this assessment applies (including the flow-defining fields). 236 Note that the UDP transport layer is one requirement for test 237 packets specified below. Type-P is a parallel concept to 238 "population of interest" defined in clause 6.1.1 of[Y.1540]. 240 o PM, a list of fundamental metrics, such as loss, delay, and 241 reordering, and corresponding target performance threshold. At 242 least one fundamental metric and target performance threshold MUST 243 be supplied (such as One-way IP Packet Loss [RFC7680] equal to 244 zero). 246 A non-Parameter which is required for several metrics is defined 247 below: 249 o T, the host time of the *first* test packet's *arrival* as 250 measured at the destination Measurement Point, or MP(Dst). There 251 may be other packets sent between source and destination hosts 252 that are excluded, so this is the time of arrival of the first 253 packet used for measurement of the metric. 255 Note that time stamp format and resolution, sequence numbers, etc. 256 will be established by the chosen test protocol standard or 257 implementation. 259 5. IP-Layer Capacity Singleton Metric Definitions 261 This section sets requirements for the following components to 262 support the Maximum IP-layer Capacity Metric. 264 5.1. Formal Name 266 Type-P-One-way-IP-Capacity, or informally called IP-layer Capacity. 268 Note that Type-P depends on the chosen method. 270 5.2. Parameters 272 This section lists the REQUIRED input factors to specify the metric, 273 beyond those listed in Section 4. 275 No additional Parameters are needed. 277 5.3. Metric Definitions 279 This section defines the REQUIRED aspects of the measureable IP-layer 280 Capacity metric (unless otherwise indicated) for measurements between 281 specified Source and Destination hosts: 283 Define the IP-layer capacity, C(T,I,PM), to be the number of IP-layer 284 bits (including header and data fields) in packets that can be 285 transmitted from the Src host and correctly received by the Dst host 286 during one contiguous sub-interval, dt in length. The IP-layer 287 capacity depends on the Src and Dst hosts, the host addresses, and 288 the path between the hosts. 290 The number of these IP-layer bits is designated n0[dtn,dtn+1] for a 291 specific dt. 293 When the packet size is known and of fixed size, the packet count 294 during a single sub-interval dt multiplied by the total bits in IP 295 header and data fields is equal to n0[dtn,dtn+1]. 297 Anticipating a Sample of Singletons, the interval dt SHOULD be set to 298 a natural number m so that T+I = T + m*dt with dtn+1 - dtn = dt and 299 with 1 <= n <= m. 301 Parameter PM represents other performance metrics [see section 302 Related Round-Trip Delay and Loss Definitions below]; their 303 measurement results SHALL be collected during measurement of IP-layer 304 Capacity and associated with the corresponding dtn for further 305 evaluation and reporting. 307 Mathematically, this definition can be represented as: 309 ( n0[dtn,dtn+1] ) 310 C(T,I,PM) = ------------------------- 311 dt 313 Equation for IP-Layer Capacity 315 and: 317 o n0 is the total number of IP-layer header and payload bits that 318 can be transmitted in Standard Formed packets [RFC8468] from the 319 Src host and correctly received by the Dst host during one 320 contiguous sub-interval, dt in length, during the interval [T, 321 T+I], 323 o C(T,I,PM) the IP-Layer Capacity, corresponds to the value of n0 324 measured in any sub-interval ending at dtn (meaning T + n*dt), 325 divided by the length of sub-interval, dt. 327 o all sub-intervals MUST be of equal duration. Choosing dt as non- 328 overlapping consecutive time intervals allows for a simple 329 implementation. 331 o The bit rate of the physical interface of the measurement device 332 must be higher than that of the link whose C(T,I,PM) is to be 333 measured. 335 Measurements according to these definitions SHALL use UDP transport 336 layer. Standard Formed packets are specified in Section 5 of 337 [RFC8468]. Some compression affects on measurement are discussed in 338 Section 6 of [RFC8468], as well. 340 5.4. Related Round-Trip Delay and One-way Loss Definitions 342 RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip 343 Delay between the Src host and the Dst host over the interval 344 [T,T+I]. The statistics used to to summarize RTD[dtn-1,dtn] MAY 345 include the minimum, maximum, median, and mean. 347 OWL[dtn-1,dtn] is defined as a sample of the [RFC7680] One-way Loss 348 between the Src host and the Dst host over the interval [T,T+I]. The 349 statistics used to to summarize OWL[dtn-1,dtn] MAY include the lost 350 packet count and the lost packet ratio. 352 Other metrics MAY be measured: one-way reordering, duplication, and 353 delay variation. 355 5.5. Discussion 357 See the corresponding section for Maximum IP-Layer Capacity. 359 5.6. Reporting the Metric 361 The IP-Layer Capacity SHALL be reported with meaningful resolution, 362 in units of Megabits per second (Mbps). 364 The Related Round Trip Delay and/or Loss metric measurements for the 365 same Singleton SHALL be reported, also with meaningful resolution for 366 the values measured. 368 Individual Capacity measurements MAY be reported in a manner 369 consistent with the Maximum IP-Layer Capacity, see Section 9. 371 6. Maximum IP-Layer Capacity Metric Definitions (Statistic) 373 This section sets requirements for the following components to 374 support the Maximum IP-layer Capacity Metric. 376 6.1. Formal Name 378 Type-P-One-way-Max-IP-Capacity, or informally called Maximum IP-layer 379 Capacity. 381 Note that Type-P depends on the chosen method. 383 6.2. Parameters 385 This section lists the REQUIRED input factors to specify the metric, 386 beyond those listed in Section 4. 388 No additional Parameters or definitions are needed. 390 6.3. Metric Definitions 392 This section defines the REQUIRED aspects of the Maximum IP-layer 393 Capacity metric (unless otherwise indicated) for measurements between 394 specified Source and Destination hosts: 396 Define the Maximum IP-layer capacity, Maximum_C(T,I,PM), to be the 397 maximum number of IP-layer bits n0[dtn,dtn+1] that can be transmitted 398 in packets from the Src host and correctly received by the Dst host, 399 over all dt length intervals in [T, T+I], and meeting the PM 400 criteria. Equivalently the Maximum of a Sample of size m of 401 C(T,I,PM) collected during the interval [T, T+I] and meeting the PM 402 criteria. 404 The interval dt SHOULD be set to a natural number m so that T+I = T + 405 m*dt with dtn+1 - dtn = dt and with 1 <= n <= m. 407 Parameter PM represents the other performance metrics (see 408 Section 6.4 below) and their measurement results for the maximum IP- 409 layer capacity. At least one target performance threshold (PM 410 criterion) MUST be defined. If more than one target performance 411 threshold is defined, then the sub-interval with maximum number of 412 bits transmitted MUST meet all the target performance thresholds. 414 Mathematically, this definition can be represented as: 416 max ( n0[dtn,dtn+1] ) 417 [T,T+I] 418 Maximum_C(T,I,PM) = ------------------------- 419 dt 420 where: 421 T T+I 422 _________________________________________ 423 | | | | | | | | | | | 424 dtn=1 2 3 4 5 6 7 8 9 10 n+1 425 n=m 427 Equation for Maximum Capacity 429 and: 431 o n0 is the total number of IP-layer header and payload bits that 432 can be transmitted in Standard Formed packets from the Src host 433 and correctly received by the Dst host during one contiguous sub- 434 interval, dt in length, during the interval [T, T+I], 436 o Maximum _C(T,I,PM) the Maximum IP-Layer Capacity, corresponds to 437 the maximum value of n0 measured in any sub-interval ending at dtn 438 (meaning T + n*dt), divided by the constant length of all sub- 439 intervals, dt. 441 o all sub-intervals MUST be of equal duration. Choosing dt as non- 442 overlapping consecutive time intervals allows for a simple 443 implementation. 445 o The bit rate of the physical interface of the measurement systems 446 must be higher than that of the link whose Maximum _C(T,I,PM) is 447 to be measured (the bottleneck link). 449 In this definition, the m sub-intervals can be viewed as trials when 450 the Src host varies the transmitted packet rate, searching for the 451 maximum n0 that meets the PM criteria measured at the Dst host in a 452 test of duration, I. When the transmitted packet rate is held 453 constant at the Src host, the m sub-intervals may also be viewed as 454 trials to evaluate the stability of n0 and metric(s) in the PM list 455 over all dt-length intervals in I. 457 Measurements according to these definitions SHALL use UDP transport 458 layer. 460 6.4. Related Round-Trip Delay and One-way Loss Definitions 462 RTD[dtn,dtn+1] is defined as a sample of the [RFC2681] Round-trip 463 Delay between the Src host and the Dst host over the interval 464 [T,T+I], and corresponds to the dt interval containing 465 Maximum_C(T,I,PM). The statistics used to to summarize 466 RTD[dtn,dtn+1] MAY include the minimum, maximum, median, and mean. 468 OWL[dtn,dtn+1] is defined as a sample of the [RFC7680] One-way Loss 469 between the Src host and the Dst host over the interval [T,T+I] and 470 corresponds to the dt interval containing Maximum_C(T,I,PM). The 471 statistics used to to summarize OWL[dtn-1,dtn] MAY include the lost 472 packet count and the lost packet ratio. 474 Other metrics MAY be measured: one-way reordering, duplication, and 475 delay variation. 477 6.5. Discussion 479 If traffic conditioning applies along a path for which 480 Maximum_C(T,I,PM) is to be determined, different values for dt SHOULD 481 be picked and measurements be executed during multiple intervals [T, 482 T+I]. Any single interval dt SHOULD be chosen so that is an integer 483 multiple of increasing values k times serialisation delay of a path 484 MTU at the physical interface speed where traffic conditioning is 485 expected. This should avoid taking configured burst tolerance 486 singletons as a valid Maximum _C(T,I,PM) result. 488 A Maximum_C(T,I,PM) without any indication of bottleneck congestion, 489 be that an increasing latency, packet loss or ECN marks during a 490 measurement interval I, is likely to underestimate Maximum_C(T,I,PM). 492 6.6. Reporting the Metric 494 The Maximum IP-Layer Capacity SHALL be reported with meaningful 495 resolution, in units of Megabits per second (which is 1,000,000 bits 496 per second to avoid any confusion). 498 The Related Round Trip Delay and/or Loss metric measurements for the 499 same Singleton SHALL be reported, also with meaningful resolution for 500 the values measured. 502 When there are demonstrated and repeatable Capacity modes in the 503 Sample, then the Maximum IP-Layer Capacity SHALL be reported for each 504 mode, along with the relative time from the beginning of the stream 505 that the mode was observed to be present. Bimodal Maxima have been 506 observed with some services, sometimes called a "turbo mode" 507 intending to deliver short transfers more quickly, or reduce the 508 initial buffering time for some video streams. Note that modes 509 lasting less than dt duration will not be detected. 511 Some transmission technologies have multiple methods of operation 512 that may be activated when channel conditions degrade or improve, and 513 these transmission methods may determine the Maximum IP-Layer 514 Capacity. Examples include line-of-sight microwave modulator 515 constellations, or cellular modem technologies where the changes may 516 be initiated by a user moving from one coverage area to another. 517 Operation in the different transmission methods may be observed over 518 time, but the modes of Maximum IP-Layer Capacity will not be 519 activated deterministically as with the "turbo mode" described in the 520 paragraph above. 522 7. IP-Layer Sender Bit Rate Singleton Metric Definitions 524 This section sets requirements for the following components to 525 support the IP-layer Sender Bitrate Metric. 527 7.1. Formal Name 529 Type-P-IP-Sender-Bit-Rate, or informally called IP-layer Sender 530 Bitrate. 532 Note that Type-P depends on the chosen method. 534 7.2. Parameters 536 This section lists the REQUIRED input factors to specify the metric, 537 beyond those listed in Section 4. 539 o S, the duration of the measurement interval at the Source 541 o st, the nominal duration of N sub-intervals in S (default = 0.05 542 seconds) 544 S SHALL be longer than I, primarily to account for on-demand 545 activation of the path, or any preamble to testing required. 547 st SHOULD be much smaller than the sub-interval dt. The st parameter 548 does not have relevance when the Source is transmitting at a fixed 549 rate throughout S. 551 7.3. Metric Definition 553 This section defines the REQUIRED aspects of the IP-layer Sender 554 Bitrate metric (unless otherwise indicated) for measurements at the 555 specified Source on packets addressed for the intended Destination 556 host and matching the required Type-P: 558 Define the IP-layer Sender Bit Rate, B(S,st), to be the number of IP- 559 layer bits (including header and data fields) that are transmitted 560 from the Source during one contiguous sub-interval, st, during the 561 test interval S (where S SHALL be longer than I), and where the 562 fixed-size packet count during that single sub-interval st also 563 provides the number of IP-layer bits in any interval: n0[stn-1,stn]. 565 Measurements according to these definitions SHALL use UDP transport 566 layer. Any feedback from Dst host to Src host received by Src host 567 during an interval [stn-1,stn] MUST NOT result in an adaptation of 568 the Src host traffic conditioning during this interval. 570 7.4. Discussion 572 Both the Sender and Receiver or (source and destination) bit rates 573 SHOULD be assessed as part of a measurement. 575 7.5. Reporting the Metric 577 The IP-Layer Sender Bit Rate SHALL be reported with meaningful 578 resolution, in units of Megabits per second. 580 Individual IP-Layer Sender Bit Rate measurements are discussed 581 further in Section 9. 583 8. Method of Measurement 585 The architecture of the method REQUIRES two cooperating hosts 586 operating in the roles of Src (test packet sender) and Dst 587 (receiver), with a measured path and return path between them. 589 The duration of a test, parameter I, MUST be constrained in a 590 production network, since this is an active test method and it will 591 likely cause congestion on the Src to Dst host path during a test. 593 Additional Test methods and configurations may be provided in this 594 section, after review and further testing. 596 8.1. Load Rate Adjustment Algorithm 598 A table SHALL be pre-built defining all the offered load rates that 599 will be supported (R1 - Rn, in ascending order). Each rate is 600 defined as datagrams of size S, sent as a burst of count C, every 601 time interval T. While it is advantageous to use datagrams of as 602 large a size as possible, it may be prudent to use a slightly smaller 603 maximum that allows for secondary protocol headers and/or tunneling 604 without resulting in IP-layer fragmentation. 606 At the beginning of a test, the sender begins sending at rate R1 and 607 the receiver starts a feedback timer at interval F (while awaiting 608 inbound datagrams). As datagrams are received they are checked for 609 sequence number anomalies (loss, out-of-order, duplication, etc.) and 610 the delay variation is measured (one-way or round-trip). This 611 information is accumulated until the feedback timer F expires and a 612 status feedback message is sent from the receiver back to the sender, 613 to communicate this information. The accumulated statistics are then 614 reset by the receiver for the next feedback interval. As feedback 615 messages are received back at the sender, they are evaluated to 616 determine how to adjust the current offered load rate (Rx). 618 If the feedback indicates that there were no sequence number 619 anomalies AND the delay variation was below the lower threshold, the 620 offered load rate is increased. If congestion has not been confirmed 621 up to this point, the offered load rate is increased by more than one 622 rate (e.g., Rx+10). This allows the offered load to quickly reach a 623 near-maximum rate. Conversely, if congestion has been previously 624 confirmed, the offered load rate is only increased by one (Rx+1). 626 If the feedback indicates that sequence number anomalies were 627 detected OR the delay variation was above the upper threshold, the 628 offered load rate is decreased. If congestion is confirmed by the 629 current feedback message being processed, the offered load rate is 630 decreased by more than one rate (e.g., Rx-30). This one-time 631 reduction is intended to compensate for the fast initial ramp-up. In 632 all other cases, the offered load rate is only decreased by one (Rx- 633 1). 635 If the feedback indicates that there were no sequence number 636 anomalies AND the delay variation was above the lower threshold, but 637 below the upper threshold, the offered load rate is not changed. 638 This allows time for recent changes in the offered load rate to 639 stabilize, and the feedback to represent current conditions more 640 accurately. 642 Lastly, the method for confirming congestion is that there were 643 sequence number anomalies OR the delay variation was above the upper 644 threshold for two consecutive feedback intervals. The algorithm 645 described above is also presented in ITU-T Rec. Y.1540, 2020 646 version[Y.1540], in Annexes A and B, and implemented in the reference 647 for Section 8.4, Running Code. 649 All the values used above are relevant to searches in the 1 Mbps to 650 10 Gbps capacity range. Searches can accommodate higher rate 651 capacities by changing the rates in the pre-built table. 653 8.2. Measurement Qualification or Verification 655 It is of course necessary to calibrate the equipment performing the 656 IP-layer Capacity measurement, to ensure that the expected capacity 657 can be measured accurately, and that equipment choices (processing 658 speed, interface bandwidth, etc.) are suitably matched to the 659 measurement range. 661 When assessing a Maximum rate as the metric specifies, artificially 662 high (optimistic) values might be measured until some buffer on the 663 path is filled. Other causes include bursts of back-to-back packets 664 with idle intervals delivered by a path, while the measurement 665 interval (dt) is small and aligned with the bursts. The artificial 666 values might result in an un-sustainable Maximum Capacity observed 667 when the method of measurement is searching for the Maximum, and that 668 would not do. This situation is different from the bi-modal service 669 rates (discussed under Reporting), which are characterized by a 670 multi-second duration (much longer than the measured RTT) and 671 repeatable behavior. 673 There are many ways that the Method of Measurement could handle this 674 false-max issue. The default value for measurement of singletons (dt 675 = 1 second) has proven to a be of practical value during tests of 676 this method, allows the bimodal service rates to be characterized, 677 and it has an obvious alignment with the reporting units (Mbps). 679 Another approach comes from Section 24 of RFC 2544[RFC2544] and its 680 discussion of Trial duration, where relatively short trials conducted 681 as part of the search are followed by longer trials to make the final 682 determination. In the production network, measurements of singletons 683 and samples (the terms for trials and tests of Lab Benchmarking) must 684 be limited in duration because they may be service-affecting. But 685 there is sufficient value in repeating a sample with a fixed sending 686 rate determined by the previous search for the Max IP-layer Capacity, 687 to qualify the result in terms of the other performance metrics 688 measured at the same time. 690 A qualification measurement for the search result is a subsequent 691 measurement, sending at a fixed 99.x % of the Max IP-layer Capacity 692 for I, or an indefinite period. The same Max Capacity Metric is 693 applied, and the Qualification for the result is a sample without 694 packet loss or a growing minimum delay trend in subsequent singletons 695 (or each dt of the measurement interval, I). Samples exhibiting 696 losses or increasing queue occupation require a repeated search and/ 697 or test at reduced fixed sender rate for qualification. 699 Here, as with any Active Capacity test, the test duration must be 700 kept short. 10 second tests for each direction of transmission are 701 common today. The default measurement interval specified here is I = 702 10 seconds). In combination with a fast search method and user- 703 network coordination, the concerns raised in RFC 6815[RFC6815] are 704 alleviated. The method for assessing Max IP Capacity is different 705 from classic [RFC2544] methods: they use short term load adjustment 706 and are sensitive to loss and delay, like other congestion control 707 algorithms used on the Internet every day. 709 8.3. Measurement Considerations 711 In general, the wide-spread measurements that this memo encourages 712 will encounter wide-spread behaviors. The bimodal IP Capacity 713 behaviors already discussed in Section 6.6 are good examples. 715 In general, it is RECOMMENDED to locate test endpoints as close to 716 the intended measured link(s) as practical (this is not always 717 possible for reasons of scale; there is a limit on number of test 718 endpoints coming from many perspecitves, management and measurement 719 traffic for example). 721 The path measured may be state-full based on many factors, and the 722 Parameter "Time of day" when a test starts may not be enough 723 information. Repeatable testing may require the time from the 724 beginning of a measured flow, and how the flow is constructed 725 including how much traffic has already been sent on that flow when a 726 state-change is observed, because the state-change may be based on 727 time or bytes sent or both. 729 Many different traffic shapers and on-demand access technologies may 730 be encountered, as anticipated in [RFC7312], and play a key role in 731 measurement results. Methods MUST be prepared to provide a short 732 preamble transmission to activate on-demand access, and to discard 733 the preamble from subsequent test results. 735 Conditions which might be encountered during measurement, where 736 packet losses may occur independently from the measurement sending 737 rate: 739 1. Congestion of an interconnection or backbone interface may appear 740 as packet losses distributed over time in the test stream, due to 741 much higher rate interfaces in the backbone. 743 2. Packet loss due to use of Random Early Detection (RED) or other 744 active queue management. 746 3. There may be only small delay variation independent of sending 747 rate under these conditions, too. 749 4. Persistent competing traffic on measurement paths that include 750 shared media may cause random packet losses in the test stream. 752 It is possible to mitigate these conditions using the flexibility of 753 the load-rate adjusting algorithm described in Section 8.1 above 754 (tuning specific parameters). 756 In general, results depend on the sending stream characteristics; the 757 measurement community has known this for a long time, and needs to 758 keep it front of mind. Although the default is a single flow (F=1) 759 for testing, use of multiple flows may be advantageous for the 760 following reasons: 762 1. the test hosts may be able to create higher load than with a 763 single flow, or parallel test hosts may be used to generate 1 764 flow each. 766 2. there may be link aggregation present (flow-based load balancing) 767 and multiple flows are needed to occupy each member of the 768 aggregate. 770 3. access policies may limit the IP-Layer Capacity depending on the 771 Type-P of packets, possibly reserving capacity for various stream 772 types. 774 Each flow would be controlled using its own implementation of the 775 Load Adjustment (Search) Algorithm. 777 As testing continues, implementers should expect some evolution in 778 the methods. The ITU-T has published a Supplement (60) to the 779 Y-series of Recommendations, "Interpreting ITU-T Y.1540 maximum IP- 780 layer capacity measurements", [Y.Sup60], which is the result of 781 continued testing with the metric and method described here. 783 8.4. Running Code 785 Much of the development of the method and comparisons with existing 786 methods conducted at IETF Hackathons and elsewhere have been based on 787 the example udpst Linux measurement tool (which is a working 788 reference for further development) [udpst]. The current project: 790 o is a utility that can function as a client or server daemon 791 o requires a successful client-initiated setup handshake between 792 cooperating hosts and allows firewalls to control inbound 793 unsolicited UDP which either go to a control port [expected and w/ 794 authentication] or to ephemeral ports that are only created as 795 needed. Firewalls protecting each host can both continue to do 796 their job normally. This aspect is similar to many other test 797 utilities available. 799 o is written in C, and built with gcc (release 9.3) and its standard 800 run-time libraries 802 o allows configuration of most of the parameters described in 803 Sections 4 and 7. 805 o supports IPv4 and IPv6 address families. 807 o supports IP-layer packet marking. 809 9. Reporting Formats 811 The singleton IP-Layer Capacity results SHOULD be accompanied by the 812 context under which they were measured. 814 o timestamp (especially the time when the maximum was observed in 815 dtn) 817 o source and destination (by IP or other meaningful ID) 819 o other inner parameters of the test case (Section 4) 821 o outer parameters, such as "done in motion" or other factors 822 belonging to the context of the measurement 824 o result validity (indicating cases where the process was somehow 825 interrupted or the attempt failed) 827 o a field where unusual circumstances could be documented, and 828 another one for "ignore/mask out" purposes in further processing 830 The Maximum IP-Layer Capacity results SHOULD be reported in the 831 format of a table with a row for each of the test Phases and Number 832 of Flows. There SHOULD be columns for the phases with number of 833 flows, and for the resultant Maximum IP-Layer Capacity results for 834 the aggregate and each flow tested. 836 As mentioned in Section 6.6, bi-modal (or multi-modal) maxima SHALL 837 be reported for each mode separately. 839 +--------------+----------------------+-----------+-----------------+ 840 | Phase, # | Max IP-Layer | Loss | RTT min, max, | 841 | Flows | Capacity, Mbps | Ratio | msec | 842 +--------------+----------------------+-----------+-----------------+ 843 | Search,1 | 967.31 | 0.0002 | 30, 58 | 844 | Verify,1 | 966.00 | 0.0000 | 30, 38 | 845 +--------------+----------------------+-----------+-----------------+ 847 Maximum IP-layer Capacity Results 849 Static and configuration parameters: 851 The sub-interval time, dt, MUST accompany a report of Maximum IP- 852 Layer Capacity results, and the remaining Parameters from Section 4, 853 General Parameters. 855 The PM list metrics corresponding to the sub-interval where the 856 Maximum Capacity occurred MUST accompany a report of Maximum IP-Layer 857 Capacity results, for each test phase. 859 The IP-Layer Sender Bit rate results SHOULD be reported in the format 860 of a table with a row for each of the test Phases, sub-intervals (st) 861 and Number of Flows. There SHOULD be columns for the phases with 862 number of flows, and for the resultant IP-Layer Sender Bit rate 863 results for the aggregate and each flow tested. 865 +------------------------+-------------+-----------------------+----+ 866 | Phase, Flow or | st, sec | Sender Bit Rate, Mbps | ?? | 867 | Aggregate | | | | 868 +------------------------+-------------+-----------------------+----+ 869 | Search,1 | 0.00 - 0.05 | 345 | __ | 870 | Search,2 | 0.00 - 0.05 | 289 | __ | 871 | Search,Agg | 0.00 - 0.05 | 634 | __ | 872 +------------------------+-------------+-----------------------+----+ 874 IP-layer Sender Bit Rate Results 876 Static and configuration parameters: 878 The subinterval time, st, MUST accompany a report of Sender IP-Layer 879 Bit Rate results. 881 Also, the values of the remaining Parameters from Section 4, General 882 Parameters, MUST be reported. 884 9.1. Configuration and Reporting Data Formats 886 As a part of the multi-Standards Development Organization (SDO) 887 harmonization of this metric and method of measurement, one of the 888 areas where the Broadband Forum (BBF) contributed its expertise was 889 in the definition of an information model and data model for 890 configuration and reporting. These models are consistent with the 891 metric parameters and default values specified as lists is this memo. 892 [TR-471] provides the Information model that was used to prepare a 893 full data model in related BBF work. The BBF has als carefully 894 considered topics within its purvue, such as placement of measurement 895 systems within the access archtecture. For example, timestamp 896 resolution requirements that influence the choice of the test 897 protocol are provided in Table 2 of [TR-471]. 899 10. Security Considerations 901 Active metrics and measurements have a long history of security 902 considerations. The security considerations that apply to any active 903 measurement of live paths are relevant here. See [RFC4656] and 904 [RFC5357]. 906 When considering privacy of those involved in measurement or those 907 whose traffic is measured, the sensitive information available to 908 potential observers is greatly reduced when using active techniques 909 which are within this scope of work. Passive observations of user 910 traffic for measurement purposes raise many privacy issues. We refer 911 the reader to the privacy considerations described in the Large Scale 912 Measurement of Broadband Performance (LMAP) Framework [RFC7594], 913 which covers active and passive techniques. 915 There are some new considerations for Capacity measurement as 916 described in this memo. 918 1. Cooperating source and destination hosts and agreements to test 919 the path between the hosts are REQUIRED. Hosts perform in either 920 the Src or Dst roles. 922 2. A REQUIRED user client-initiated setup handshake between 923 cooperating hosts and allows firewalls to control inbound 924 unsolicited UDP which either go to a control port [expected and 925 w/authentication] or to ephemeral ports that are only created as 926 needed. Firewalls protecting each host can both continue to do 927 their job normally. 929 3. Integrity protection for feedback messages conveying measurements 930 is RECOMMENDED. 932 4. Hosts MUST limit the number of simultaneous tests to avoid 933 resource exhaust and inaccuate results. 935 5. Senders MUST be rate-limited. This can be accomplished using the 936 pre-built table defining all the offered load rates that will be 937 supported (Section 8.1). The recommended load-control search 938 algorithm results in "ramp up" from the lowest rate in the table. 940 6. Service subscribers with limited data volumes who conduct 941 extensive capacity testing might experience the effects of 942 Service Provider controls on their service. Testing with the 943 Service Provider's measurement hosts SHOULD be limited in 944 frequency and/or overall volume of test traffic. 946 The exact specification of these features is left for the future 947 protocol development. 949 11. IANA Considerations 951 This memo makes no requests of IANA. 953 12. Acknowledgements 955 Thanks to Joachim Fabini, Matt Mathis, J.Ignacio Alvarez-Hamelin, 956 Wolfgang Balzer, Frank Brockners and Greg Mirsky for their extensive 957 comments on the memo and related topics. 959 13. References 961 13.1. Normative References 963 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 964 Requirement Levels", BCP 14, RFC 2119, 965 DOI 10.17487/RFC2119, March 1997, 966 . 968 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 969 "Framework for IP Performance Metrics", RFC 2330, 970 DOI 10.17487/RFC2330, May 1998, 971 . 973 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 974 Network Interconnect Devices", RFC 2544, 975 DOI 10.17487/RFC2544, March 1999, 976 . 978 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 979 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 980 September 1999, . 982 [RFC3148] Mathis, M. and M. Allman, "A Framework for Defining 983 Empirical Bulk Transfer Capacity Metrics", RFC 3148, 984 DOI 10.17487/RFC3148, July 2001, 985 . 987 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 988 Zekauskas, "A One-way Active Measurement Protocol 989 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 990 . 992 [RFC5136] Chimento, P. and J. Ishac, "Defining Network Capacity", 993 RFC 5136, DOI 10.17487/RFC5136, February 2008, 994 . 996 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 997 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 998 RFC 5357, DOI 10.17487/RFC5357, October 2008, 999 . 1001 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 1002 for Equal Cost Multipath Routing and Link Aggregation in 1003 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 1004 . 1006 [RFC6815] Bradner, S., Dubray, K., McQuaid, J., and A. Morton, 1007 "Applicability Statement for RFC 2544: Use on Production 1008 Networks Considered Harmful", RFC 6815, 1009 DOI 10.17487/RFC6815, November 2012, 1010 . 1012 [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling 1013 Framework for IP Performance Metrics (IPPM)", RFC 7312, 1014 DOI 10.17487/RFC7312, August 2014, 1015 . 1017 [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 1018 Aitken, P., and A. Akhter, "A Framework for Large-Scale 1019 Measurement of Broadband Performance (LMAP)", RFC 7594, 1020 DOI 10.17487/RFC7594, September 2015, 1021 . 1023 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1024 Ed., "A One-Way Loss Metric for IP Performance Metrics 1025 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1026 2016, . 1028 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1029 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1030 May 2016, . 1032 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1033 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1034 May 2017, . 1036 [RFC8337] Mathis, M. and A. Morton, "Model-Based Metrics for Bulk 1037 Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March 1038 2018, . 1040 [RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V. 1041 Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for 1042 the IP Performance Metrics (IPPM) Framework", RFC 8468, 1043 DOI 10.17487/RFC8468, November 2018, 1044 . 1046 13.2. Informative References 1048 [copycat] Edleine, K., Kuhlewind, K., Trammell, B., and B. Donnet, 1049 "copycat: Testing Differential Treatment of New Transport 1050 Protocols in the Wild (ANRW '17)", July 2017, 1051 . 1053 [LS-SG12-A] 1054 12, I. S., "LS - Harmonization of IP Capacity and Latency 1055 Parameters: Revision of Draft Rec. Y.1540 on IP packet 1056 transfer performance parameters and New Annex A with Lab 1057 Evaluation Plan", May 2019, 1058 . 1060 [LS-SG12-B] 1061 12, I. S., "LS on harmonization of IP Capacity and Latency 1062 Parameters: Consent of Draft Rec. Y.1540 on IP packet 1063 transfer performance parameters and New Annex A with Lab & 1064 Field Evaluation Plans", March 2019, 1065 . 1067 [TR-471] Morton, A., "Broadband Forum TR-471: IP Layer Capacity 1068 Metrics and Measurement", July 2020, 1069 . 1072 [udpst] AT&T, "UDP Speed Test Open Broadband project", August 1073 2020, >. 1075 [Y.1540] Y.1540, I. R., "Internet protocol data communication 1076 service - IP packet transfer and availability performance 1077 parameters", December 2019, 1078 . 1080 [Y.Sup60] Morton, A., "Recommendation Y.Sup60, (04/20) Interpreting 1081 ITU-T Y.1540 maximum IP-layer capacity measurements", June 1082 2020, . 1084 Authors' Addresses 1086 Al Morton 1087 AT&T Labs 1088 200 Laurel Avenue South 1089 Middletown,, NJ 07748 1090 USA 1092 Phone: +1 732 420 1571 1093 Fax: +1 732 368 1192 1094 Email: acm@research.att.com 1096 Ruediger Geib 1097 Deutsche Telekom 1098 Heinrich Hertz Str. 3-7 1099 Darmstadt 64295 1100 Germany 1102 Phone: +49 6151 5812747 1103 Email: Ruediger.Geib@telekom.de 1105 Len Ciavattone 1106 AT&T Labs 1107 200 Laurel Avenue South 1108 Middletown,, NJ 07748 1109 USA 1111 Email: lencia@att.com