idnits 2.17.1 draft-ietf-ippm-capacity-metric-method-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (February 1, 2021) is 1172 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'T' is mentioned on line 485, but not defined == Missing Reference: 'I' is mentioned on line 485, but not defined == Missing Reference: 'PM' is mentioned on line 484, but not defined ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Downref: Normative reference to an Informational RFC: RFC 2544 ** Downref: Normative reference to an Informational RFC: RFC 3148 ** Downref: Normative reference to an Informational RFC: RFC 5136 ** Downref: Normative reference to an Informational RFC: RFC 6815 ** Downref: Normative reference to an Informational RFC: RFC 7312 ** Downref: Normative reference to an Informational RFC: RFC 7594 ** Downref: Normative reference to an Informational RFC: RFC 7799 ** Downref: Normative reference to an Experimental RFC: RFC 8337 ** Downref: Normative reference to an Informational RFC: RFC 8468 Summary: 10 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Morton 3 Internet-Draft AT&T Labs 4 Intended status: Standards Track R. Geib 5 Expires: August 5, 2021 Deutsche Telekom 6 L. Ciavattone 7 AT&T Labs 8 February 1, 2021 10 Metrics and Methods for One-way IP Capacity 11 draft-ietf-ippm-capacity-metric-method-05 13 Abstract 15 This memo revisits the problem of Network Capacity metrics first 16 examined in RFC 5136. The memo specifies a more practical Maximum 17 IP-layer Capacity metric definition catering for measurement 18 purposes, and outlines the corresponding methods of measurement. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at https://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on August 5, 2021. 37 Copyright Notice 39 Copyright (c) 2021 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (https://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 55 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 56 2. Scope and Goals . . . . . . . . . . . . . . . . . . . . . . . 4 57 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 4 58 4. General Parameters and Definitions . . . . . . . . . . . . . 5 59 5. IP-Layer Capacity Singleton Metric Definitions . . . . . . . 6 60 5.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6 61 5.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 62 5.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 6 63 5.4. Related Round-Trip Delay and One-way Loss Definitions . . 8 64 5.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 8 65 5.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 8 66 6. Maximum IP-Layer Capacity Metric Definitions (Statistic) . . 9 67 6.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 9 68 6.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 9 69 6.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 9 70 6.4. Related Round-Trip Delay and One-way Loss Definitions . . 11 71 6.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 11 72 6.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 11 73 7. IP-Layer Sender Bit Rate Singleton Metric Definitions . . . . 12 74 7.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 12 75 7.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 12 76 7.3. Metric Definition . . . . . . . . . . . . . . . . . . . . 13 77 7.4. Discussion . . . . . . . . . . . . . . . . . . . . . . . 13 78 7.5. Reporting the Metric . . . . . . . . . . . . . . . . . . 13 79 8. Method of Measurement . . . . . . . . . . . . . . . . . . . . 13 80 8.1. Load Rate Adjustment Algorithm . . . . . . . . . . . . . 13 81 8.2. Measurement Qualification or Verification . . . . . . . . 15 82 8.3. Measurement Considerations . . . . . . . . . . . . . . . 16 83 8.4. Running Code . . . . . . . . . . . . . . . . . . . . . . 18 84 9. Reporting Formats . . . . . . . . . . . . . . . . . . . . . . 18 85 9.1. Configuration and Reporting Data Formats . . . . . . . . 20 86 10. Security Considerations . . . . . . . . . . . . . . . . . . . 20 87 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 88 12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 89 13. Appendix - Load Rate Adjustment Pseudo Code . . . . . . . . . 22 90 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 91 14.1. Normative References . . . . . . . . . . . . . . . . . . 23 92 14.2. Informative References . . . . . . . . . . . . . . . . . 24 93 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 95 1. Introduction 97 The IETF's efforts to define Network and Bulk Transport Capacity have 98 been chartered and progressed for over twenty years. Over that time, 99 the performance community has seen development of Informative 100 definitions in [RFC3148] for Framework for Bulk Transport Capacity 101 (BTC), RFC 5136 for Network Capacity and Maximum IP-layer Capacity, 102 and the Experimental metric definitions and methods in [RFC8337], 103 Model-Based Metrics for BTC. 105 This memo revisits the problem of Network Capacity metrics examined 106 first in [RFC3148] and later in [RFC5136]. Maximum IP-Layer Capacity 107 and [RFC3148] Bulk Transfer Capacity (goodput) are different metrics. 108 Maximum IP-layer Capacity is like the theoretical goal for goodput. 109 There are many metrics in [RFC5136], such as Available Capacity. 110 Measurements depend on the network path under test and the use case. 111 Here, the main use case is to assess the maximum capacity of the 112 access network, with specific performance criteria used in the 113 measurement. 115 This memo recognizes the importance of a definition of a Maximum IP- 116 layer Capacity Metric at a time when access speeds have increased 117 dramatically; a definition that is both practical and effective for 118 the performance community's needs, including Internet users. The 119 metric definition is intended to use Active Methods of Measurement 120 [RFC7799], and a method of measurement is included. 122 The most direct active measurement of IP-layer Capacity would use IP 123 packets, but in practice a transport header is needed to traverse 124 address and port translators. UDP offers the most direct assessment 125 possibility, and in the [copycat] measurement study to investigate 126 whether UDP is viable as a general Internet transport protocol, the 127 authors found that a high percentage of paths tested support UDP 128 transport. A number of liaisons have been exchanged on this topic 129 [LS-SG12-A] [LS-SG12-B], discussing the laboratory and field tests 130 that support the UDP-based approach to IP-layer Capacity measurement. 132 This memo also recognizes the many updates to the IP Performance 133 Metrics Framework [RFC2330] published over twenty years, and makes 134 use of [RFC7312] for Advanced Stream and Sampling Framework, and 135 [RFC8468] with IPv4, IPv6, and IPv4-IPv6 Coexistence Updates. 137 1.1. Requirements Language 139 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 140 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 141 "OPTIONAL" in this document are to be interpreted as described in BCP 142 14[RFC2119] [RFC8174] when, and only when, they appear in all 143 capitals, as shown here. 145 2. Scope and Goals 147 The scope of this memo is to define a metric and corresponding method 148 to unambiguously perform Active measurements of Maximum IP-Layer 149 Capacity, along with related metrics and methods. 151 The main goal is to harmonize the specified metric and method across 152 the industry, and this memo is the vehicle through which working 153 group (and eventually, IETF) consensus will be captured and 154 communicated to achieve broad agreement, and possibly result in 155 changes in the specifications of other Standards Development 156 Organizations (SDO) (through the SDO's normal contribution process, 157 or through liaison exchange). 159 A local goal is to aid efficient test procedures where possible, and 160 to recommend reporting with additional interpretation of the results. 161 Also, to foster the development of protocol support for this metric 162 and method of measurement (all active testing protocols currently 163 defined by the IPPM WG are UDP-based, meeting a key requirement of 164 these methods). The supporting protocol development to measure this 165 metric according to the specified method is a key future contribution 166 to Internet measurement. 168 3. Motivation 170 As with any problem that has been worked for many years in various 171 SDOs without any special attempts at coordination, various solutions 172 for metrics and methods have emerged. 174 There are five factors that have changed (or begun to change) in the 175 2013-2019 time frame, and the presence of any one of them on the path 176 requires features in the measurement design to account for the 177 changes: 179 1. Internet access is no longer the bottleneck for many users. 181 2. Both transfer rate and latency are important to user's 182 satisfaction. 184 3. UDP's growing role in Transport, in areas where TCP once 185 dominated. 187 4. Content and applications moving physically closer to users. 189 5. Less emphasis on ISP gateway measurements, possibly due to less 190 traffic crossing ISP gateways in future. 192 4. General Parameters and Definitions 194 This section lists the REQUIRED input factors to specify a Sender or 195 Receiver metric. 197 o Src, the address of a host (such as the globally routable IP 198 address). 200 o Dst, the address of a host (such as the globally routable IP 201 address). 203 o MaxHops, the limit on the number of Hops a specific packet may 204 visit as it traverses from the host at Src to the host at Dst 205 (implemented in the TTL or Hop Limit). 207 o T, the time at the start of measurement interval, when packets are 208 first transmitted from the Source. 210 o I, the nominal duration of a measurement interval at the 211 destination (default 10 sec) 213 o dt, the nominal duration of m equal sub-intervals in I at the 214 destination (default 1 sec) 216 o dtn, a specific sub-interval, n, one of m sub-intervals in I 218 o Tmax, a maximum waiting time for test packets to arrive at the 219 destination, set sufficiently long to disambiguate packets with 220 long delays from packets that are discarded (lost), such that the 221 distribution of one-way delay is not truncated. 223 o F, the number of different flows synthesized by the method 224 (default 1 flow) 226 o flow, the stream of packets with the same n-tuple of designated 227 header fields that (when held constant) result in identical 228 treatment in a multi-path decision (such as the decision taken in 229 load balancing). Note: The IPv6 flow label MAY be included in the 230 flow definition when routers have complied with [RFC6438] 231 guidelines at the Tunnel End Points (TEP), and the source of the 232 measurement is a TEP. 234 o Type-P, the complete description of the test packets for which 235 this assessment applies (including the flow-defining fields). 236 Note that the UDP transport layer is one requirement for test 237 packets specified below. Type-P is a parallel concept to 238 "population of interest" defined in clause 6.1.1 of[Y.1540]. 240 o PM, a list of fundamental metrics, such as loss, delay, and 241 reordering, and corresponding target performance threshold. At 242 least one fundamental metric and target performance threshold MUST 243 be supplied (such as One-way IP Packet Loss [RFC7680] equal to 244 zero). 246 A non-Parameter which is required for several metrics is defined 247 below: 249 o T, the host time of the *first* test packet's *arrival* as 250 measured at the destination Measurement Point, or MP(Dst). There 251 may be other packets sent between source and destination hosts 252 that are excluded, so this is the time of arrival of the first 253 packet used for measurement of the metric. 255 Note that time stamp format and resolution, sequence numbers, etc. 256 will be established by the chosen test protocol standard or 257 implementation. 259 5. IP-Layer Capacity Singleton Metric Definitions 261 This section sets requirements for the following components to 262 support the Maximum IP-layer Capacity Metric. 264 5.1. Formal Name 266 Type-P-One-way-IP-Capacity, or informally called IP-layer Capacity. 268 Note that Type-P depends on the chosen method. 270 5.2. Parameters 272 This section lists the REQUIRED input factors to specify the metric, 273 beyond those listed in Section 4. 275 No additional Parameters are needed. 277 5.3. Metric Definitions 279 This section defines the REQUIRED aspects of the measurable IP-layer 280 Capacity metric (unless otherwise indicated) for measurements between 281 specified Source and Destination hosts: 283 Define the IP-layer capacity, C(T,dt,PM), to be the number of IP- 284 layer bits (including header and data fields) in packets that can be 285 transmitted from the Src host and correctly received by the Dst host 286 during one contiguous sub-interval, dt in length. The IP-layer 287 capacity depends on the Src and Dst hosts, the host addresses, and 288 the path between the hosts. 290 The number of these IP-layer bits is designated n0[dtn,dtn+1] for a 291 specific dt. 293 When the packet size is known and of fixed size, the packet count 294 during a single sub-interval dt multiplied by the total bits in IP 295 header and data fields is equal to n0[dtn,dtn+1]. 297 Anticipating a Sample of Singletons, the interval dt SHOULD be set to 298 a natural number m so that T+I = T + m*dt with dtn+1 - dtn = dt and 299 with 1 <= n <= m. 301 Parameter PM represents other performance metrics [see section 5.4 302 below]; their measurement results SHALL be collected during 303 measurement of IP-layer Capacity and associated with the 304 corresponding dtn for further evaluation and reporting. Users SHALL 305 specify the parameter Tmax as required by each metric's reference 306 definition. 308 Mathematically, this definition can be represented as: 310 ( n0[dtn,dtn+1] ) 311 C(T,dt,PM) = ------------------------- 312 dt 314 Equation for IP-Layer Capacity 316 and: 318 o n0 is the total number of IP-layer header and payload bits that 319 can be transmitted in Standard Formed packets [RFC8468] from the 320 Src host and correctly received by the Dst host during one 321 contiguous sub-interval, dt in length, during the interval [T, 322 T+I], 324 o C(T,dt,PM) the IP-Layer Capacity, corresponds to the value of n0 325 measured in any sub-interval ending at dtn (meaning T + n*dt), 326 divided by the length of sub-interval, dt. 328 o PM represents other performance metrics [see section 5.4 below]; 329 their measurement results SHALL be collected during measurement of 330 IP-layer Capacity and associated with the corresponding dtn for 331 further evaluation and reporting. 333 o all sub-intervals MUST be of equal duration. Choosing dt as non- 334 overlapping consecutive time intervals allows for a simple 335 implementation. 337 o The bit rate of the physical interface of the measurement device 338 must be higher than that of the link whose C(T,I,PM) is to be 339 measured. 341 Measurements according to these definitions SHALL use the UDP 342 transport layer. Standard Formed packets are specified in Section 5 343 of [RFC8468]. Some compression affects on measurement are discussed 344 in Section 6 of [RFC8468], as well. 346 5.4. Related Round-Trip Delay and One-way Loss Definitions 348 RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip 349 Delay between the Src host and the Dst host over the interval [T,T+I] 350 (that contains equal non-overlapping intervals of dt). The 351 "reasonable period of time" in [RFC2681] is the parameter Tmax in 352 this memo. The statistics used to to summarize RTD[dtn-1,dtn] MAY 353 include the minimum, maximum, median, and mean, and the range = 354 (maximum - minimum) is referred to below in Section 8.1 for load 355 adjustment purposes. 357 OWL[dtn-1,dtn] is defined as a sample of the [RFC7680] One-way Loss 358 between the Src host and the Dst host over the interval [T,T+I] (that 359 contains equal non-overlapping intervals of dt). The statistics used 360 to to summarize OWL[dtn-1,dtn] MAY include the lost packet count and 361 the lost packet ratio. 363 Other metrics MAY be measured: one-way reordering, duplication, and 364 delay variation. 366 5.5. Discussion 368 See the corresponding section for Maximum IP-Layer Capacity. 370 5.6. Reporting the Metric 372 The IP-Layer Capacity SHOULD be reported with at least single Megabit 373 resolution, in units of Megabits per second (Mbps). 375 The Related Round Trip Delay and/or Loss metric measurements for the 376 same Singleton SHALL be reported, also with meaningful resolution for 377 the values measured. 379 Individual Capacity measurements MAY be reported in a manner 380 consistent with the Maximum IP-Layer Capacity, see Section 9. 382 6. Maximum IP-Layer Capacity Metric Definitions (Statistic) 384 This section sets requirements for the following components to 385 support the Maximum IP-layer Capacity Metric. 387 6.1. Formal Name 389 Type-P-One-way-Max-IP-Capacity, or informally called Maximum IP-layer 390 Capacity. 392 Note that Type-P depends on the chosen method. 394 6.2. Parameters 396 This section lists the REQUIRED input factors to specify the metric, 397 beyond those listed in Section 4. 399 No additional Parameters or definitions are needed. 401 6.3. Metric Definitions 403 This section defines the REQUIRED aspects of the Maximum IP-layer 404 Capacity metric (unless otherwise indicated) for measurements between 405 specified Source and Destination hosts: 407 Define the Maximum IP-layer capacity, Maximum_C(T,I,PM), to be the 408 maximum number of IP-layer bits n0[dtn,dtn+1] that can be transmitted 409 in packets from the Src host and correctly received by the Dst host, 410 over all dt length intervals in [T, T+I], and meeting the PM 411 criteria. Equivalently the Maximum of a Sample of size m of 412 C(T,I,PM) collected during the interval [T, T+I] and meeting the PM 413 criteria. 415 The interval dt SHOULD be set to a natural number m so that T+I = T + 416 m*dt with dtn+1 - dtn = dt and with 1 <= n <= m. 418 Parameter PM represents the other performance metrics (see 419 Section 6.4 below) and their measurement results for the maximum IP- 420 layer capacity. At least one target performance threshold (PM 421 criterion) MUST be defined. If more than one metric and target 422 performance threshold are defined, then the sub-interval with maximum 423 number of bits transmitted MUST meet all the target performance 424 thresholds. Users SHALL specify the parameter Tmax as required by 425 each metric's reference definition. 427 Mathematically, this definition can be represented as: 429 max ( n0[dtn,dtn+1] ) 430 [T,T+I] 431 Maximum_C(T,I,PM) = ------------------------- 432 dt 433 where: 434 T T+I 435 _________________________________________ 436 | | | | | | | | | | | 437 dtn=1 2 3 4 5 6 7 8 9 10 n+1 438 n=m 440 Equation for Maximum Capacity 442 and: 444 o n0 is the total number of IP-layer header and payload bits that 445 can be transmitted in Standard Formed packets from the Src host 446 and correctly received by the Dst host during one contiguous sub- 447 interval, dt in length, during the interval [T, T+I], 449 o Maximum_C(T,I,PM) the Maximum IP-Layer Capacity, corresponds to 450 the maximum value of n0 measured in any sub-interval ending at dtn 451 (meaning T + n*dt), divided by the constant length of all sub- 452 intervals, dt. 454 o PM represents the other performance metrics (see Section 5.4) and 455 their measurement results for the maximum IP-layer capacity. At 456 least one target performance threshold (PM criterion) MUST be 457 defined. 459 o all sub-intervals MUST be of equal duration. Choosing dt as non- 460 overlapping consecutive time intervals allows for a simple 461 implementation. 463 o The bit rate of the physical interface of the measurement systems 464 must be higher than that of the link whose Maximum_C(T,I,PM) is to 465 be measured (the bottleneck link). 467 In this definition, the m sub-intervals can be viewed as trials when 468 the Src host varies the transmitted packet rate, searching for the 469 maximum n0 that meets the PM criteria measured at the Dst host in a 470 test of duration, I. When the transmitted packet rate is held 471 constant at the Src host, the m sub-intervals may also be viewed as 472 trials to evaluate the stability of n0 and metric(s) in the PM list 473 over all dt-length intervals in I. 475 Measurements according to these definitions SHALL use the UDP 476 transport layer. 478 6.4. Related Round-Trip Delay and One-way Loss Definitions 480 RTD[dtn,dtn+1] and OWL[dtn,dtn+1] are defined in Section 5.4. Here, 481 the test intervals are increased to match the capacity samples, 482 RTD[T,I] and OWL[T,I]. 484 The interval dtn,dtn+1 where Maximum_C[T,I,PM] occurs is the 485 reporting sub-interval for RTD[T,I] and OWL[T,I]. 487 Other metrics MAY be measured: one-way reordering, duplication, and 488 delay variation. 490 6.5. Discussion 492 If traffic conditioning (e.g., shaping, policing) applies along a 493 path for which Maximum_C(T,I,PM) is to be determined, different 494 values for dt SHOULD be picked and measurements be executed during 495 multiple intervals [T, T+I]. A single constant interval dt SHOULD be 496 chosen so that is an integer multiple of increasing values k times 497 serialisation delay of a path MTU at the physical interface speed 498 where traffic conditioning is expected. This should avoid taking 499 configured burst tolerance singletons as a valid Maximum_C(T,I,PM) 500 result. 502 A Maximum_C(T,I,PM) without any indication of bottleneck congestion, 503 be that an increasing latency, packet loss or ECN marks during a 504 measurement interval I, is likely to underestimate Maximum_C(T,I,PM). 506 6.6. Reporting the Metric 508 The IP-Layer Capacity SHOULD be reported with at least single Megabit 509 resolution, in units of Megabits per second (Mbps) (which is 510 1,000,000 bits per second to avoid any confusion). 512 The Related Round Trip Delay and/or Loss metric measurements for the 513 same Singleton SHALL be reported, also with meaningful resolution for 514 the values measured. 516 When there are demonstrated and repeatable Capacity modes in the 517 Sample, then the Maximum IP-Layer Capacity SHALL be reported for each 518 mode, along with the relative time from the beginning of the stream 519 that the mode was observed to be present. Bimodal Maxima have been 520 observed with some services, sometimes called a "turbo mode" 521 intending to deliver short transfers more quickly, or reduce the 522 initial buffering time for some video streams. Note that modes 523 lasting less than dt duration will not be detected. 525 Some transmission technologies have multiple methods of operation 526 that may be activated when channel conditions degrade or improve, and 527 these transmission methods may determine the Maximum IP-Layer 528 Capacity. Examples include line-of-sight microwave modulator 529 constellations, or cellular modem technologies where the changes may 530 be initiated by a user moving from one coverage area to another. 531 Operation in the different transmission methods may be observed over 532 time, but the modes of Maximum IP-Layer Capacity will not be 533 activated deterministically as with the "turbo mode" described in the 534 paragraph above. 536 7. IP-Layer Sender Bit Rate Singleton Metric Definitions 538 This section sets requirements for the following components to 539 support the IP-layer Sender Bitrate Metric. This metric helps to 540 check that the sender actually generated the desired rates during a 541 test, and measurement takes place at the Src host to network path 542 interface (or as close as practical). It is not a metric for path 543 performance. 545 7.1. Formal Name 547 Type-P-IP-Sender-Bit-Rate, or informally called IP-layer Sender 548 Bitrate. 550 Note that Type-P depends on the chosen method. 552 7.2. Parameters 554 This section lists the REQUIRED input factors to specify the metric, 555 beyond those listed in Section 4. 557 o S, the duration of the measurement interval at the Source 559 o st, the nominal duration of N sub-intervals in S (default = 0.05 560 seconds) 562 S SHALL be longer than I, primarily to account for on-demand 563 activation of the path, or any preamble to testing required, and the 564 delay of the path. 566 st SHOULD be much smaller than the sub-interval dt. The st parameter 567 does not have relevance when the Source is transmitting at a fixed 568 rate throughout S. 570 7.3. Metric Definition 572 This section defines the REQUIRED aspects of the IP-layer Sender 573 Bitrate metric (unless otherwise indicated) for measurements at the 574 specified Source on packets addressed for the intended Destination 575 host and matching the required Type-P: 577 Define the IP-layer Sender Bit Rate, B(S,st), to be the number of IP- 578 layer bits (including header and data fields) that are transmitted 579 from the Source during one contiguous sub-interval, st, during the 580 test interval S (where S SHALL be longer than I), and where the 581 fixed-size packet count during that single sub-interval st also 582 provides the number of IP-layer bits in any interval: n0[stn-1,stn]. 584 Measurements according to these definitions SHALL use the UDP 585 transport layer. Any feedback from Dst host to Src host received by 586 Src host during an interval [stn-1,stn] MUST NOT result in an 587 adaptation of the Src host traffic conditioning during this interval 588 (rate adjustment occurs on st boundaries). 590 7.4. Discussion 592 Both the Sender and Receiver or (source and destination) bit rates 593 SHOULD be assessed as part of an IP-layer Capacity measurement. 595 7.5. Reporting the Metric 597 The IP-Layer Sender Bit Rate SHALL be reported with meaningful 598 resolution, in units of Megabits per second. 600 Individual IP-Layer Sender Bit Rate measurements are discussed 601 further in Section 9. 603 8. Method of Measurement 605 The architecture of the method REQUIRES two cooperating hosts 606 operating in the roles of Src (test packet sender) and Dst 607 (receiver), with a measured path and return path between them. 609 The duration of a test, parameter I, MUST be constrained in a 610 production network, since this is an active test method and it will 611 likely cause congestion on the Src to Dst host path during a test. 613 8.1. Load Rate Adjustment Algorithm 615 A table SHALL be pre-built defining all the offered load rates that 616 will be supported (R1 through Rn, in ascending order, corresponding 617 to indexed rows in the table). Each rate is defined as datagrams of 618 size ss, sent as a burst of count cc, every time interval tt (default 619 for tt is 1ms, a likely system tick-interval). While it is 620 advantageous to use datagrams of as large a size as possible, it may 621 be prudent to use a slightly smaller maximum that allows for 622 secondary protocol headers and/or tunneling without resulting in IP- 623 layer fragmentation. Selection of a new rate is indicated by a 624 calculation on the current row, Rx. For example: 626 "Rx+1": the sender uses the next higher rate in the table. 628 "Rx-10": the sender uses the rate 10 rows lower in the table. 630 At the beginning of a test, the sender begins sending at rate R1 and 631 the receiver starts a feedback timer at interval F (while awaiting 632 inbound datagrams). As datagrams are received they are checked for 633 sequence number anomalies (loss, out-of-order, duplication, etc.) and 634 the delay range is measured (one-way or round-trip). This 635 information is accumulated until the feedback timer F expires and a 636 status feedback message is sent from the receiver back to the sender, 637 to communicate this information. The accumulated statistics are then 638 reset by the receiver for the next feedback interval. As feedback 639 messages are received back at the sender, they are evaluated to 640 determine how to adjust the current offered load rate (Rx). 642 If the feedback indicates that no sequence number anomalies were 643 detected AND the delay range was below the lower threshold, the 644 offered load rate is increased. If congestion has not been confirmed 645 up to this point, the offered load rate is increased by more than one 646 rate (e.g., Rx+10). This allows the offered load to quickly reach a 647 near-maximum rate. Conversely, if congestion has been previously 648 confirmed, the offered load rate is only increased by one (Rx+1). 650 If the feedback indicates that sequence number anomalies were 651 detected OR the delay range was above the upper threshold, the 652 offered load rate is decreased. Also, if congestion is now confirmed 653 by the current feedback message being processed, then the offered 654 load rate is decreased by more than one rate (e.g., Rx-30). This 655 one-time reduction is intended to compensate for the fast initial 656 ramp-up. In all other cases, the offered load rate is only decreased 657 by one (Rx-1). 659 If the feedback indicates that there were no sequence number 660 anomalies AND the delay range was above the lower threshold, but 661 below the upper threshold, the offered load rate is not changed. 662 This allows time for recent changes in the offered load rate to 663 stabilize, and the feedback to represent current conditions more 664 accurately. 666 Lastly, the method for inferring congestion is that there were 667 sequence number anomalies AND/OR the delay range was above the upper 668 threshold for two consecutive feedback intervals. The algorithm 669 described above is also illustrated in ITU-T Rec. Y.1540, 2020 670 version[Y.1540], in Annex B, and implemented in the Appendix on Load 671 Rate Adjustment Pseudo Code in this memo. 673 All the values used above are relevant to searches in the 1 Mbps to 674 10 Gbps capacity range. Searches can accommodate higher rate 675 capacities by changing the rates in the pre-built table. 677 8.2. Measurement Qualification or Verification 679 It is of course necessary to calibrate the equipment performing the 680 IP-layer Capacity measurement, to ensure that the expected capacity 681 can be measured accurately, and that equipment choices (processing 682 speed, interface bandwidth, etc.) are suitably matched to the 683 measurement range. 685 When assessing a Maximum rate as the metric specifies, artificially 686 high (optimistic) values might be measured until some buffer on the 687 path is filled. Other causes include bursts of back-to-back packets 688 with idle intervals delivered by a path, while the measurement 689 interval (dt) is small and aligned with the bursts. The artificial 690 values might result in an un-sustainable Maximum Capacity observed 691 when the method of measurement is searching for the Maximum, and that 692 would not do. This situation is different from the bi-modal service 693 rates (discussed under Reporting), which are characterized by a 694 multi-second duration (much longer than the measured RTT) and 695 repeatable behavior. 697 There are many ways that the Method of Measurement could handle this 698 false-max issue. The default value for measurement of singletons (dt 699 = 1 second) has proven to a be of practical value during tests of 700 this method, allows the bimodal service rates to be characterized, 701 and it has an obvious alignment with the reporting units (Mbps). 703 Another approach comes from Section 24 of RFC 2544[RFC2544] and its 704 discussion of Trial duration, where relatively short trials conducted 705 as part of the search are followed by longer trials to make the final 706 determination. In the production network, measurements of singletons 707 and samples (the terms for trials and tests of Lab Benchmarking) must 708 be limited in duration because they may be service-affecting. But 709 there is sufficient value in repeating a sample with a fixed sending 710 rate determined by the previous search for the Max IP-layer Capacity, 711 to qualify the result in terms of the other performance metrics 712 measured at the same time. 714 A qualification measurement for the search result is a subsequent 715 measurement, sending at a fixed 99.x % of the Max IP-layer Capacity 716 for I, or an indefinite period. The same Max Capacity Metric is 717 applied, and the Qualification for the result is a sample without 718 packet loss or a growing minimum delay trend in subsequent singletons 719 (or each dt of the measurement interval, I). Samples exhibiting 720 losses or increasing queue occupation require a repeated search and/ 721 or test at reduced fixed sender rate for qualification. 723 Here, as with any Active Capacity test, the test duration must be 724 kept short. 10 second tests for each direction of transmission are 725 common today. The default measurement interval specified here is I = 726 10 seconds). In combination with a fast search method and user- 727 network coordination, the concerns raised in RFC 6815[RFC6815] are 728 alleviated. The method for assessing Max IP Capacity is different 729 from classic [RFC2544] methods: they use short term load adjustment 730 and are sensitive to loss and delay, like other congestion control 731 algorithms used on the Internet every day. 733 8.3. Measurement Considerations 735 In general, the wide-spread measurements that this memo encourages 736 will encounter wide-spread behaviors. The bimodal IP Capacity 737 behaviors already discussed in Section 6.6 are good examples. 739 In general, it is RECOMMENDED to locate test endpoints as close to 740 the intended measured link(s) as practical (this is not always 741 possible for reasons of scale; there is a limit on number of test 742 endpoints coming from many perspectives, management and measurement 743 traffic for example). The testing operator MUST set a value for the 744 MaxHops parameter, based on the expected path length. This parameter 745 can keep measurement traffic from straying too far beyond the 746 intended path. 748 The path measured may be state-full based on many factors, and the 749 Parameter "Time of day" when a test starts may not be enough 750 information. Repeatable testing may require the time from the 751 beginning of a measured flow, and how the flow is constructed 752 including how much traffic has already been sent on that flow when a 753 state-change is observed, because the state-change may be based on 754 time or bytes sent or both. 756 Many different traffic shapers and on-demand access technologies may 757 be encountered, as anticipated in [RFC7312], and play a key role in 758 measurement results. Methods MUST be prepared to provide a short 759 preamble transmission to activate on-demand access, and to discard 760 the preamble from subsequent test results. 762 Conditions which might be encountered during measurement, where 763 packet losses may occur independently from the measurement sending 764 rate: 766 1. Congestion of an interconnection or backbone interface may appear 767 as packet losses distributed over time in the test stream, due to 768 much higher rate interfaces in the backbone. 770 2. Packet loss due to use of Random Early Detection (RED) or other 771 active queue management may or may not affect the measurement 772 flow if competing background traffic (other flows) are 773 simultaneously present. 775 3. There may be only small delay variation independent of sending 776 rate under these conditions, too. 778 4. Persistent competing traffic on measurement paths that include 779 shared transmission media may cause random packet losses in the 780 test stream. 782 It is possible to mitigate these conditions using the flexibility of 783 the load-rate adjusting algorithm described in Section 8.1 above 784 (tuning specific parameters). 786 If the measurement flow burst duration happens to be on the order of 787 or smaller than the burst size of a shaper or a policer in the path, 788 then the line rate might be measured rather than the bandwidth limit 789 imposed by the shaper or policer. If this condition is suspected, 790 alternate configurations SHOULD be used. 792 In general, results depend on the sending stream characteristics; the 793 measurement community has known this for a long time, and needs to 794 keep it front of mind. Although the default is a single flow (F=1) 795 for testing, use of multiple flows may be advantageous for the 796 following reasons: 798 1. the test hosts may be able to create higher load than with a 799 single flow, or parallel test hosts may be used to generate 1 800 flow each. 802 2. there may be link aggregation present (flow-based load balancing) 803 and multiple flows are needed to occupy each member of the 804 aggregate. 806 3. access policies may limit the IP-Layer Capacity depending on the 807 Type-P of packets, possibly reserving capacity for various stream 808 types. 810 Each flow would be controlled using its own implementation of the 811 Load Adjustment (Search) Algorithm. 813 As testing continues, implementers should expect some evolution in 814 the methods. The ITU-T has published a Supplement (60) to the 815 Y-series of Recommendations, "Interpreting ITU-T Y.1540 maximum IP- 816 layer capacity measurements", [Y.Sup60], which is the result of 817 continued testing with the metric and method described here. 819 8.4. Running Code 821 This section is for the benefit of the Document Shepherd's form, and 822 will be deleted prior to final review. 824 Much of the development of the method and comparisons with existing 825 methods conducted at IETF Hackathons and elsewhere have been based on 826 the example udpst Linux measurement tool (which is a working 827 reference for further development) [udpst]. The current project: 829 o is a utility that can function as a client or server daemon 831 o requires a successful client-initiated setup handshake between 832 cooperating hosts and allows firewalls to control inbound 833 unsolicited UDP which either go to a control port [expected and w/ 834 authentication] or to ephemeral ports that are only created as 835 needed. Firewalls protecting each host can both continue to do 836 their job normally. This aspect is similar to many other test 837 utilities available. 839 o is written in C, and built with gcc (release 9.3) and its standard 840 run-time libraries 842 o allows configuration of most of the parameters described in 843 Sections 4 and 7. 845 o supports IPv4 and IPv6 address families. 847 o supports IP-layer packet marking. 849 9. Reporting Formats 851 The singleton IP-Layer Capacity results SHOULD be accompanied by the 852 context under which they were measured. 854 o timestamp (especially the time when the maximum was observed in 855 dtn) 857 o source and destination (by IP or other meaningful ID) 858 o other inner parameters of the test case (Section 4) 860 o outer parameters, such as "test conducted in motion" or other 861 factors belonging to the context of the measurement 863 o result validity (indicating cases where the process was somehow 864 interrupted or the attempt failed) 866 o a field where unusual circumstances could be documented, and 867 another one for "ignore/mask out" purposes in further processing 869 The Maximum IP-Layer Capacity results SHOULD be reported in the 870 format of a table with a row for each of the test Phases and Number 871 of Flows. There SHOULD be columns for the phases with number of 872 flows, and for the resultant Maximum IP-Layer Capacity results for 873 the aggregate and each flow tested. 875 As mentioned in Section 6.6, bi-modal (or multi-modal) maxima SHALL 876 be reported for each mode separately. 878 +--------------+----------------------+-----------+-----------------+ 879 | Phase, # | Max IP-Layer | Loss | RTT min, max, | 880 | Flows | Capacity, Mbps | Ratio | msec | 881 +--------------+----------------------+-----------+-----------------+ 882 | Search,1 | 967.31 | 0.0002 | 30, 58 | 883 | Verify,1 | 966.00 | 0.0000 | 30, 38 | 884 +--------------+----------------------+-----------+-----------------+ 886 Maximum IP-layer Capacity Results 888 Static and configuration parameters: 890 The sub-interval time, dt, MUST accompany a report of Maximum IP- 891 Layer Capacity results, and the remaining Parameters from Section 4, 892 General Parameters. 894 The PM list metrics corresponding to the sub-interval where the 895 Maximum Capacity occurred MUST accompany a report of Maximum IP-Layer 896 Capacity results, for each test phase. 898 The IP-Layer Sender Bit rate results SHOULD be reported in the format 899 of a table with a row for each of the test Phases, sub-intervals (st) 900 and Number of Flows. There SHOULD be columns for the phases with 901 number of flows, and for the resultant IP-Layer Sender Bit rate 902 results for the aggregate and each flow tested. 904 +------------------------+-------------+-----------------------+----+ 905 | Phase, Flow or | st, sec | Sender Bit Rate, Mbps | ?? | 906 | Aggregate | | | | 907 +------------------------+-------------+-----------------------+----+ 908 | Search,1 | 0.00 - 0.05 | 345 | __ | 909 | Search,2 | 0.00 - 0.05 | 289 | __ | 910 | Search,Agg | 0.00 - 0.05 | 634 | __ | 911 +------------------------+-------------+-----------------------+----+ 913 IP-layer Sender Bit Rate Results 915 Static and configuration parameters: 917 The subinterval time, st, MUST accompany a report of Sender IP-Layer 918 Bit Rate results. 920 Also, the values of the remaining Parameters from Section 4, General 921 Parameters, MUST be reported. 923 9.1. Configuration and Reporting Data Formats 925 As a part of the multi-Standards Development Organization (SDO) 926 harmonization of this metric and method of measurement, one of the 927 areas where the Broadband Forum (BBF) contributed its expertise was 928 in the definition of an information model and data model for 929 configuration and reporting. These models are consistent with the 930 metric parameters and default values specified as lists is this memo. 931 [TR-471] provides the Information model that was used to prepare a 932 full data model in related BBF work. The BBF has also carefully 933 considered topics within its purview, such as placement of 934 measurement systems within the access architecture. For example, 935 timestamp resolution requirements that influence the choice of the 936 test protocol are provided in Table 2 of [TR-471]. 938 10. Security Considerations 940 Active metrics and measurements have a long history of security 941 considerations. The security considerations that apply to any active 942 measurement of live paths are relevant here. See [RFC4656] and 943 [RFC5357]. 945 When considering privacy of those involved in measurement or those 946 whose traffic is measured, the sensitive information available to 947 potential observers is greatly reduced when using active techniques 948 which are within this scope of work. Passive observations of user 949 traffic for measurement purposes raise many privacy issues. We refer 950 the reader to the privacy considerations described in the Large Scale 951 Measurement of Broadband Performance (LMAP) Framework [RFC7594], 952 which covers active and passive techniques. 954 There are some new considerations for Capacity measurement as 955 described in this memo. 957 1. Cooperating source and destination hosts and agreements to test 958 the path between the hosts are REQUIRED. Hosts perform in either 959 the Src or Dst roles. 961 2. A REQUIRED user client-initiated setup handshake between 962 cooperating hosts and allows firewalls to control inbound 963 unsolicited UDP which either go to a control port [expected and 964 w/authentication] or to ephemeral ports that are only created as 965 needed. Firewalls protecting each host can both continue to do 966 their job normally. 968 3. Integrity protection for feedback messages conveying measurements 969 is RECOMMENDED. 971 4. Hosts MUST limit the number of simultaneous tests to avoid 972 resource exhaustion and inaccurate results. 974 5. Senders MUST be rate-limited. This can be accomplished using the 975 pre-built table defining all the offered load rates that will be 976 supported (Section 8.1). The recommended load-control search 977 algorithm results in "ramp up" from the lowest rate in the table. 979 6. Service subscribers with limited data volumes who conduct 980 extensive capacity testing might experience the effects of 981 Service Provider controls on their service. Testing with the 982 Service Provider's measurement hosts SHOULD be limited in 983 frequency and/or overall volume of test traffic. 985 The exact specification of these features is left for the future 986 protocol development. 988 11. IANA Considerations 990 This memo makes no requests of IANA. 992 12. Acknowledgments 994 Thanks to Joachim Fabini, Matt Mathis, J.Ignacio Alvarez-Hamelin, 995 Wolfgang Balzer, Frank Brockners, Greg Mirsky and Martin Duke for 996 their extensive comments on the memo and related topics. 998 13. Appendix - Load Rate Adjustment Pseudo Code 1000 The following is a pseudo-code implementation of the algorithm 1001 described in Section 8.1. 1003 Rx = 1 # The current sending rate (equivalent to a row of the table) 1004 seqErr = 0 # Measured count of any of Loss or Reordering impairments 1005 delay = 0 # Measured Range of Round Trip Time, RTT, ms 1006 lowThresh = 30 # Low threshold on the Range of RTT, ms 1007 upperThresh = 90 # Upper threshold on the Range of RTT, ms 1008 hSpeedTresh = 1Gbps # Threshold for transition between sending rate step 1009 sizes (such as 1 Mbps and 100 Mbps) 1010 slowAdjCount = 0 # Measured Number of consecutive status reports 1011 indicating loss and/or delay variation above upperThresh 1012 slowAdjThresh = 2 # Threshold on slowAdjCount used to infer congestion. 1013 Use values >1 to avoid misinterpreting transient loss 1014 highSpeedDelta = 10 # The number of rows to move in a single adjustment 1015 when initially increasing offered load (to ramp-up quickly) 1016 maxLoadRates = 2000 # Maximum table index (rows) 1018 if ( seqErr == 0 && delay < lowThresh ) { 1019 if ( Rx < hSpeedTresh && slowAdjCount < slowAdjThresh ) { 1020 Rx += highSpeedDelta; 1021 slowAdjCount = 0; 1022 } else { 1023 if ( Rx < maxLoadRates - 1 ) 1024 Rx++; 1025 } 1026 } else if ( seqErr > 0 || delay > upperThresh ) { 1027 slowAdjCount++; 1028 if ( Rx < hSpeedTresh && c->slowAdjCount == slowAdjThresh ) { 1029 if ( Rx > highSpeedDelta * 3 ) 1030 Rx -= highSpeedDelta * 3; 1031 else 1032 Rx = 0; 1033 } else { 1034 if ( Rx > 0 ) 1035 Rx--; 1036 } 1037 } 1039 14. References 1040 14.1. Normative References 1042 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1043 Requirement Levels", BCP 14, RFC 2119, 1044 DOI 10.17487/RFC2119, March 1997, 1045 . 1047 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1048 "Framework for IP Performance Metrics", RFC 2330, 1049 DOI 10.17487/RFC2330, May 1998, 1050 . 1052 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1053 Network Interconnect Devices", RFC 2544, 1054 DOI 10.17487/RFC2544, March 1999, 1055 . 1057 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1058 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 1059 September 1999, . 1061 [RFC3148] Mathis, M. and M. Allman, "A Framework for Defining 1062 Empirical Bulk Transfer Capacity Metrics", RFC 3148, 1063 DOI 10.17487/RFC3148, July 2001, 1064 . 1066 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1067 Zekauskas, "A One-way Active Measurement Protocol 1068 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 1069 . 1071 [RFC5136] Chimento, P. and J. Ishac, "Defining Network Capacity", 1072 RFC 5136, DOI 10.17487/RFC5136, February 2008, 1073 . 1075 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1076 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1077 RFC 5357, DOI 10.17487/RFC5357, October 2008, 1078 . 1080 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 1081 for Equal Cost Multipath Routing and Link Aggregation in 1082 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 1083 . 1085 [RFC6815] Bradner, S., Dubray, K., McQuaid, J., and A. Morton, 1086 "Applicability Statement for RFC 2544: Use on Production 1087 Networks Considered Harmful", RFC 6815, 1088 DOI 10.17487/RFC6815, November 2012, 1089 . 1091 [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling 1092 Framework for IP Performance Metrics (IPPM)", RFC 7312, 1093 DOI 10.17487/RFC7312, August 2014, 1094 . 1096 [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 1097 Aitken, P., and A. Akhter, "A Framework for Large-Scale 1098 Measurement of Broadband Performance (LMAP)", RFC 7594, 1099 DOI 10.17487/RFC7594, September 2015, 1100 . 1102 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1103 Ed., "A One-Way Loss Metric for IP Performance Metrics 1104 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1105 2016, . 1107 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1108 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1109 May 2016, . 1111 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1112 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1113 May 2017, . 1115 [RFC8337] Mathis, M. and A. Morton, "Model-Based Metrics for Bulk 1116 Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March 1117 2018, . 1119 [RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V. 1120 Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for 1121 the IP Performance Metrics (IPPM) Framework", RFC 8468, 1122 DOI 10.17487/RFC8468, November 2018, 1123 . 1125 14.2. Informative References 1127 [copycat] Edleine, K., Kuhlewind, K., Trammell, B., and B. Donnet, 1128 "copycat: Testing Differential Treatment of New Transport 1129 Protocols in the Wild (ANRW '17)", July 2017, 1130 . 1132 [LS-SG12-A] 1133 12, I. S., "LS - Harmonization of IP Capacity and Latency 1134 Parameters: Revision of Draft Rec. Y.1540 on IP packet 1135 transfer performance parameters and New Annex A with Lab 1136 Evaluation Plan", May 2019, 1137 . 1139 [LS-SG12-B] 1140 12, I. S., "LS on harmonization of IP Capacity and Latency 1141 Parameters: Consent of Draft Rec. Y.1540 on IP packet 1142 transfer performance parameters and New Annex A with Lab & 1143 Field Evaluation Plans", March 2019, 1144 . 1146 [TR-471] Morton, A., "Broadband Forum TR-471: IP Layer Capacity 1147 Metrics and Measurement", July 2020, 1148 . 1151 [udpst] AT&T, "UDP Speed Test Open Broadband project", August 1152 2020, >. 1154 [Y.1540] Y.1540, I. R., "Internet protocol data communication 1155 service - IP packet transfer and availability performance 1156 parameters", December 2019, 1157 . 1159 [Y.Sup60] Morton, A., "Recommendation Y.Sup60, (04/20) Interpreting 1160 ITU-T Y.1540 maximum IP-layer capacity measurements", June 1161 2020, . 1163 Authors' Addresses 1165 Al Morton 1166 AT&T Labs 1167 200 Laurel Avenue South 1168 Middletown,, NJ 07748 1169 USA 1171 Phone: +1 732 420 1571 1172 Fax: +1 732 368 1192 1173 Email: acm@research.att.com 1174 Ruediger Geib 1175 Deutsche Telekom 1176 Heinrich Hertz Str. 3-7 1177 Darmstadt 64295 1178 Germany 1180 Phone: +49 6151 5812747 1181 Email: Ruediger.Geib@telekom.de 1183 Len Ciavattone 1184 AT&T Labs 1185 200 Laurel Avenue South 1186 Middletown,, NJ 07748 1187 USA 1189 Email: lencia@att.com