idnits 2.17.1 draft-ietf-ippm-metric-registry-21.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 21, 2019) is 1618 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 2330 == Outdated reference: A later version (-16) exists of draft-ietf-ippm-initial-registry-12 -- Obsolete informational reference (is this intentional?): RFC 4148 (Obsoleted by RFC 6248) Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Bagnulo 3 Internet-Draft UC3M 4 Intended status: Best Current Practice B. Claise 5 Expires: May 24, 2020 Cisco Systems, Inc. 6 P. Eardley 7 BT 8 A. Morton 9 AT&T Labs 10 A. Akhter 11 Consultant 12 November 21, 2019 14 Registry for Performance Metrics 15 draft-ietf-ippm-metric-registry-21 17 Abstract 19 This document defines the format for the IANA Performance Metrics 20 Registry. This document also gives a set of guidelines for 21 Registered Performance Metric requesters and reviewers. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on May 24, 2020. 40 Copyright Notice 42 Copyright (c) 2019 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (https://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 58 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 60 4. Motivation for a Performance Metrics Registry . . . . . . . . 8 61 4.1. Interoperability . . . . . . . . . . . . . . . . . . . . 8 62 4.2. Single point of reference for Performance Metrics . . . . 9 63 4.3. Side benefits . . . . . . . . . . . . . . . . . . . . . . 9 64 5. Criteria for Performance Metrics Registration . . . . . . . . 9 65 6. Performance Metric Registry: Prior attempt . . . . . . . . . 10 66 6.1. Why this Attempt Will Succeed . . . . . . . . . . . . . . 11 67 7. Definition of the Performance Metric Registry . . . . . . . . 11 68 7.1. Summary Category . . . . . . . . . . . . . . . . . . . . 13 69 7.1.1. Identifier . . . . . . . . . . . . . . . . . . . . . 13 70 7.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 13 71 7.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 17 72 7.1.4. Description . . . . . . . . . . . . . . . . . . . . . 17 73 7.1.5. Reference . . . . . . . . . . . . . . . . . . . . . . 17 74 7.1.6. Change Controller . . . . . . . . . . . . . . . . . . 17 75 7.1.7. Version (of Registry Format) . . . . . . . . . . . . 17 76 7.2. Metric Definition Category . . . . . . . . . . . . . . . 18 77 7.2.1. Reference Definition . . . . . . . . . . . . . . . . 18 78 7.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 18 79 7.3. Method of Measurement Category . . . . . . . . . . . . . 19 80 7.3.1. Reference Method . . . . . . . . . . . . . . . . . . 19 81 7.3.2. Packet Stream Generation . . . . . . . . . . . . . . 19 82 7.3.3. Traffic Filter . . . . . . . . . . . . . . . . . . . 20 83 7.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 20 84 7.3.5. Run-time Parameters . . . . . . . . . . . . . . . . . 21 85 7.3.6. Role . . . . . . . . . . . . . . . . . . . . . . . . 21 86 7.4. Output Category . . . . . . . . . . . . . . . . . . . . . 22 87 7.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 22 88 7.4.2. Reference Definition . . . . . . . . . . . . . . . . 22 89 7.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 22 90 7.4.4. Calibration . . . . . . . . . . . . . . . . . . . . . 23 91 7.5. Administrative information . . . . . . . . . . . . . . . 23 92 7.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 23 93 7.5.2. Requester . . . . . . . . . . . . . . . . . . . . . . 23 94 7.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 24 95 7.5.4. Revision Date . . . . . . . . . . . . . . . . . . . . 24 96 7.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 24 98 8. The Life-Cycle of Registered Performance Metrics . . . . . . 24 99 8.1. Adding new Performance Metrics to the Performance Metrics 100 Registry . . . . . . . . . . . . . . . . . . . . . . . . 24 101 8.2. Revising Registered Performance Metrics . . . . . . . . . 25 102 8.3. Deprecating Registered Performance Metrics . . . . . . . 27 103 9. Security considerations . . . . . . . . . . . . . . . . . . . 28 104 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28 105 10.1. Registry Group . . . . . . . . . . . . . . . . . . . . . 28 106 10.2. Performance Metric Name Elements . . . . . . . . . . . . 28 107 10.3. New Performance Metrics Registry . . . . . . . . . . . . 29 108 11. Blank Registry Template . . . . . . . . . . . . . . . . . . . 31 109 11.1. Summary . . . . . . . . . . . . . . . . . . . . . . . . 31 110 11.1.1. ID (Identifier) . . . . . . . . . . . . . . . . . . 31 111 11.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 31 112 11.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 31 113 11.1.4. Description . . . . . . . . . . . . . . . . . . . . 31 114 11.1.5. Change Controller . . . . . . . . . . . . . . . . . 31 115 11.1.6. Version (of Registry Format) . . . . . . . . . . . . 31 116 11.2. Metric Definition . . . . . . . . . . . . . . . . . . . 31 117 11.2.1. Reference Definition . . . . . . . . . . . . . . . . 31 118 11.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 32 119 11.3. Method of Measurement . . . . . . . . . . . . . . . . . 32 120 11.3.1. Reference Method . . . . . . . . . . . . . . . . . . 32 121 11.3.2. Packet Stream Generation . . . . . . . . . . . . . . 32 122 11.3.3. Traffic Filtering (observation) Details . . . . . . 32 123 11.3.4. Sampling Distribution . . . . . . . . . . . . . . . 32 124 11.3.5. Run-time Parameters and Data Format . . . . . . . . 32 125 11.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . 32 126 11.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 33 127 11.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 33 128 11.4.2. Reference Definition . . . . . . . . . . . . . . . . 33 129 11.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 33 130 11.4.4. Calibration . . . . . . . . . . . . . . . . . . . . 33 131 11.5. Administrative items . . . . . . . . . . . . . . . . . . 33 132 11.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 33 133 11.5.2. Requestor . . . . . . . . . . . . . . . . . . . . . 33 134 11.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 33 135 11.5.4. Revision Date . . . . . . . . . . . . . . . . . . . 33 136 11.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 33 137 12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 138 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 34 139 13.1. Normative References . . . . . . . . . . . . . . . . . . 34 140 13.2. Informative References . . . . . . . . . . . . . . . . . 35 141 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37 143 1. Introduction 145 The IETF specifies and uses Performance Metrics of protocols and 146 applications transported over its protocols. Performance metrics are 147 such an important part of the operations of IETF protocols that 148 [RFC6390] specifies guidelines for their development. 150 The definition and use of Performance Metrics in the IETF happens in 151 various working groups (WG), most notably: 153 The "IP Performance Metrics" (IPPM) WG is the WG primarily 154 focusing on Performance Metrics definition at the IETF. 156 The "Metric Blocks for use with RTCP's Extended Report Framework" 157 (XRBLOCK) WG recently specified many Performance Metrics related 158 to "RTP Control Protocol Extended Reports (RTCP XR)" [RFC3611], 159 which establishes a framework to allow new information to be 160 conveyed in RTCP, supplementing the original report blocks defined 161 in "RTP: A Transport Protocol for Real-Time Applications", 162 [RFC3550]. 164 The "Benchmarking Methodology" WG (BMWG) defined many Performance 165 Metrics for use in laboratory benchmarking of inter-networking 166 technologies. 168 The "IP Flow Information eXport" (IPFIX) concluded WG specified an 169 IANA process for new Information Elements. Some Performance 170 Metrics related Information Elements are proposed on regular 171 basis. 173 The "Performance Metrics for Other Layers" (PMOL) a concluded WG, 174 defined some Performance Metrics related to Session Initiation 175 Protocol (SIP) voice quality [RFC6035]. 177 It is expected that more Performance Metrics will be defined in the 178 future, not only IP-based metrics, but also metrics which are 179 protocol-specific and application-specific. 181 Despite the importance of Performance Metrics, there are two related 182 problems for the industry. First, ensuring that when one party 183 requests another party to measure (or report or in some way act on) a 184 particular Performance Metric, then both parties have exactly the 185 same understanding of what Performance Metric is being referred to. 186 Second, discovering which Performance Metrics have been specified, to 187 avoid developing a new Performance Metric that is very similar, but 188 not quite inter-operable. These problems can be addressed by 189 creating a registry of performance metrics. The usual way in which 190 the IETF organizes registries is with Internet Assigned Numbers 191 Authority (IANA), and there is currently no Performance Metrics 192 Registry maintained by the IANA. 194 This document requests that IANA create and maintain a Performance 195 Metrics Registry, according to the maintenance procedures and the 196 Performance Metrics Registry format defined in this memo. The 197 resulting Performance Metrics Registry is for use by the IETF and 198 others. Although the Registry formatting specifications herein are 199 primarily for registry creation by IANA, any other organization that 200 wishes to create a Performance Metrics Registry MAY use the same 201 formatting specifications for their purposes. The authors make no 202 guarantee of the registry format's applicability to any possible set 203 of Performance Metrics envisaged by other organizations, but 204 encourage others to apply it. In the remainder of this document, 205 unless we explicitly say otherwise, we will refer to the IANA- 206 maintained Performance Metrics Registry as simply the Performance 207 Metrics Registry. 209 2. Terminology 211 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 212 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 213 "OPTIONAL" in this document are to be interpreted as described in BCP 214 14 [RFC2119] [RFC8174] when, and only when, they appear in all 215 capitals, as shown here. 217 Performance Metric: A Performance Metric is a quantitative measure 218 of performance, targeted to an IETF-specified protocol or targeted 219 to an application transported over an IETF-specified protocol. 220 Examples of Performance Metrics are the FTP response time for a 221 complete file download, the DNS response time to resolve the IP 222 address, a database logging time, etc. This definition is 223 consistent with the definition of metric in [RFC2330] and broader 224 than the definition of performance metric in [RFC6390]. 226 Registered Performance Metric: A Registered Performance Metric is a 227 Performance Metric expressed as an entry in the Performance 228 Metrics Registry, administered by IANA. Such a performance metric 229 has met all the registry review criteria defined in this document 230 in order to included in the registry. 232 Performance Metrics Registry: The IANA registry containing 233 Registered Performance Metrics. 235 Proprietary Registry: A set of metrics that are registered in a 236 proprietary registry, as opposed to Performance Metrics Registry. 238 Performance Metrics Experts: The Performance Metrics Experts is a 239 group of designated experts [RFC8126] selected by the IESG to 240 validate the Performance Metrics before updating the Performance 241 Metrics Registry. The Performance Metrics Experts work closely 242 with IANA. 244 Parameter: A Parameter is an input factor defined as a variable in 245 the definition of a Performance Metric. A Parameter is a 246 numerical or other specified factor forming one of a set that 247 defines a metric or sets the conditions of its operation. All 248 Parameters must be known to measure using a metric and interpret 249 the results. There are two types of Parameters: Fixed and Run- 250 time parameters. For the Fixed Parameters, the value of the 251 variable is specified in the Performance Metrics Registry entry 252 and different Fixed Parameter values results in different 253 Registered Performance Metrics. For the Run-time Parameters, the 254 value of the variable is defined when the metric measurement 255 method is executed and a given Registered Performance Metric 256 supports multiple values for the parameter. Although Run-time 257 Parameters do not change the fundamental nature of the Performance 258 Metric's definition, some have substantial influence on the 259 network property being assessed and interpretation of the results. 261 Note: Consider the case of packet loss in the following two 262 Active Measurement Method cases. The first case is packet loss 263 as background loss where the Run-time Parameter set includes a 264 very sparse Poisson stream, and only characterizes the times 265 when packets were lost. Actual user streams likely see much 266 higher loss at these times, due to tail drop or radio errors. 267 The second case is packet loss as inverse of throughput where 268 the Run-time Parameter set includes a very dense, bursty 269 stream, and characterizes the loss experienced by a stream that 270 approximates a user stream. These are both "loss metrics", but 271 the difference in interpretation of the results is highly 272 dependent on the Run-time Parameters (at least), to the extreme 273 where we are actually using loss to infer its compliment: 274 delivered throughput. 276 Active Measurement Method: Methods of Measurement conducted on 277 traffic which serves only the purpose of measurement and is 278 generated for that reason alone, and whose traffic characteristics 279 are known a priori. The complete definition of Active Methods is 280 specified in section 3.4 of[RFC7799]. Examples of Active 281 Measurement Methods are the measurement methods for the One way 282 delay metric defined in [RFC7679] and the one for round trip delay 283 defined in [RFC2681]. 285 Passive Measurement Method: Methods of Measurement conducted on 286 network traffic, generated either from the end users or from 287 network elements that would exist regardless whether the 288 measurement was being conducted or not. The complete definition 289 of Passive Methods is specified in section 3.6 of [RFC7799]. One 290 characteristic of Passive Measurement Methods is that sensitive 291 information may be observed, and as a consequence, stored in the 292 measurement system. 294 Hybrid Measurement Method: Hybrid Methods are Methods of Measurement 295 that use a combination of Active Methods and Passive Methods, to 296 assess Active Metrics, Passive Metrics, or new metrics derived 297 from the a priori knowledge and observations of the stream of 298 interest. The complete definition of Hybrid Methods is specified 299 in section 3.8 of [RFC7799]. 301 3. Scope 303 This document is intended for two different audiences: 305 1. For those defining new Registered Performance Metrics, it 306 provides specifications and best practices to be used in deciding 307 which Registered Performance Metrics are useful for a measurement 308 study, instructions for writing the text for each column of the 309 Registered Performance Metrics, and information on the supporting 310 documentation required for the new Performance Metrics Registry 311 entry (up to and including the publication of one or more RFCs or 312 I-Ds describing it). 314 2. For the appointed Performance Metrics Experts and for IANA 315 personnel administering the new IANA Performance Metrics 316 Registry, it defines a set of acceptance criteria against which 317 these proposed Registered Performance Metrics should be 318 evaluated. 320 In addition, this document may be useful for other organizations who 321 are defining a Performance Metric registry of their own, and may re- 322 use the features of the Performance Metrics Registry defined in this 323 document. 325 This Performance Metrics Registry is applicable to Performance 326 Metrics issued from Active Measurement, Passive Measurement, and any 327 other form of Performance Metric. This registry is designed to 328 encompass Performance Metrics developed throughout the IETF and 329 especially for the technologies specified in the following working 330 groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes an 331 prior attempt to set up a Performance Metrics Registry, and the 332 reasons why this design was inadequate [RFC6248]. Finally, this 333 document gives a set of guidelines for requesters and expert 334 reviewers of candidate Registered Performance Metrics. 336 This document makes no attempt to populate the Performance Metrics 337 Registry with initial entries. 339 Based on [RFC8126] Section 4.3, this document is processed as Best 340 Current Practice (BCP) [RFC2026]. 342 4. Motivation for a Performance Metrics Registry 344 In this section, we detail several motivations for the Performance 345 Metrics Registry. 347 4.1. Interoperability 349 As any IETF registry, the primary use for a registry is to manage a 350 registry for its use within one or more protocols. In the particular 351 case of the Performance Metrics Registry, there are two types of 352 protocols that will use the Performance Metrics in the Performance 353 Metrics Registry during their operation (by referring to the Index 354 values): 356 o Control protocol: This type of protocol used to allow one entity 357 to request another entity to perform a measurement using a 358 specific metric defined by the Performance Metrics Registry. One 359 particular example is the LMAP framework [RFC7594]. Using the 360 LMAP terminology, the Performance Metrics Registry is used in the 361 LMAP Control protocol to allow a Controller to request a 362 measurement task to one or more Measurement Agents. In order to 363 enable this use case, the entries of the Performance Metrics 364 Registry must be sufficiently defined to allow a Measurement Agent 365 implementation to trigger a specific measurement task upon the 366 reception of a control protocol message. This requirement heavily 367 constrains the type of entries that are acceptable for the 368 Performance Metrics Registry. 370 o Report protocol: This type of protocol is used to allow an entity 371 to report measurement results to another entity. By referencing 372 to a specific Performance Metrics Registry, it is possible to 373 properly characterize the measurement result data being reported. 374 Using the LMAP terminology, the Performance Metrics Registry is 375 used in the Report protocol to allow a Measurement Agent to report 376 measurement results to a Collector. 378 It should be noted that the LMAP framework explicitly allows for 379 using not only the IANA-maintained Performance Metrics Registry but 380 also other registries containing Performance Metrics, either defined 381 by other organizations or private ones. However, others who are 382 creating Registries to be used in the context of an LMAP framework 383 are encouraged to use the Registry format defined in this document, 384 because this makes it easier for developers of LMAP Measurement 385 Agents (MAs) to programmatically use information found in those other 386 Registries' entries. 388 4.2. Single point of reference for Performance Metrics 390 A Performance Metrics Registry serves as a single point of reference 391 for Performance Metrics defined in different working groups in the 392 IETF. As we mentioned earlier, there are several WGs that define 393 Performance Metrics in the IETF and it is hard to keep track of all 394 them. This results in multiple definitions of similar Performance 395 Metrics that attempt to measure the same phenomena but in slightly 396 different (and incompatible) ways. Having a registry would allow 397 both the IETF community and external people to have a single list of 398 relevant Performance Metrics defined by the IETF (and others, where 399 appropriate). The single list is also an essential aspect of 400 communication about Performance Metrics, where different entities 401 that request measurements, execute measurements, and report the 402 results can benefit from a common understanding of the referenced 403 Performance Metric. 405 4.3. Side benefits 407 There are a couple of side benefits of having such a registry. 408 First, the Performance Metrics Registry could serve as an inventory 409 of useful and used Performance Metrics, that are normally supported 410 by different implementations of measurement agents. Second, the 411 results of measurements using the Performance Metrics should be 412 comparable even if they are performed by different implementations 413 and in different networks, as the Performance Metric is properly 414 defined. BCP 176 [RFC6576] examines whether the results produced by 415 independent implementations are equivalent in the context of 416 evaluating the completeness and clarity of metric specifications. 417 This BCP defines the standards track advancement testing for (active) 418 IPPM metrics, and the same process will likely suffice to determine 419 whether Registered Performance Metrics are sufficiently well 420 specified to result in comparable (or equivalent) results. 421 Registered Performance Metrics which have undergone such testing 422 SHOULD be noted, with a reference to the test results. 424 5. Criteria for Performance Metrics Registration 426 It is neither possible nor desirable to populate the Performance 427 Metrics Registry with all combinations of Parameters of all 428 Performance Metrics. The Registered Performance Metrics SHOULD be: 430 1. interpretable by the user. 432 2. implementable by the software designer, 434 3. deployable by network operators, 436 4. accurate, for interoperability and deployment across vendors, 438 5. Operationally useful, so that it has significant industry 439 interest and/or has seen deployment, 441 6. Sufficiently tightly defined, so that different values for the 442 Run-time Parameters does not change the fundamental nature of the 443 measurement, nor change the practicality of its implementation. 445 In essence, there needs to be evidence that a candidate Registered 446 Performance Metric has significant industry interest, or has seen 447 deployment, and there is agreement that the candidate Registered 448 Performance Metric serves its intended purpose. 450 6. Performance Metric Registry: Prior attempt 452 There was a previous attempt to define a metric registry RFC 4148 453 [RFC4148]. However, it was obsoleted by RFC 6248 [RFC6248] because 454 it was "found to be insufficiently detailed to uniquely identify IPPM 455 metrics... [there was too much] variability possible when 456 characterizing a metric exactly" which led to the RFC4148 registry 457 having "very few users, if any". 459 A couple of interesting additional quotes from RFC 6248 [RFC6248] 460 might help to understand the issues related to that registry. 462 1. "It is not believed to be feasible or even useful to register 463 every possible combination of Type P, metric parameters, and 464 Stream parameters using the current structure of the IPPM Metrics 465 Registry." 467 2. "The registry structure has been found to be insufficiently 468 detailed to uniquely identify IPPM metrics." 470 3. "Despite apparent efforts to find current or even future users, 471 no one responded to the call for interest in the RFC 4148 472 registry during the second half of 2010." 474 The current approach learns from this by tightly defining each 475 Registered Performance Metric with only a few variable (Run-time) 476 Parameters to be specified by the measurement designer, if any. The 477 idea is that entries in the Performance Metrics Registry stem from 478 different measurement methods which require input (Run-time) 479 parameters to set factors like source and destination addresses 480 (which do not change the fundamental nature of the measurement). The 481 downside of this approach is that it could result in a large number 482 of entries in the Performance Metrics Registry. There is agreement 483 that less is more in this context - it is better to have a reduced 484 set of useful metrics rather than a large set of metrics, some with 485 with questionable usefulness. 487 6.1. Why this Attempt Will Succeed 489 As mentioned in the previous section, one of the main issues with the 490 previous registry was that the metrics contained in the registry were 491 too generic to be useful. This document specifies stricter criteria 492 for performance metric registration (see section 5), and imposes a 493 group of Performance Metrics Experts that will provide guidelines to 494 assess if a Performance Metric is properly specified. 496 Another key difference between this attempt and the previous one is 497 that in this case there is at least one clear user for the 498 Performance Metrics Registry: the LMAP framework and protocol. 499 Because the LMAP protocol will use the Performance Metrics Registry 500 values in its operation, this actually helps to determine if a metric 501 is properly defined. 503 In particular, since we expect that the LMAP control protocol will 504 enable a controller to request a measurement agent to perform a 505 measurement using a given metric by embedding the Performance Metrics 506 Registry identifier in the protocol. Such a metric and method are 507 properly specified if they are defined well-enough so that it is 508 possible (and practical) to implement them in the measurement agent. 509 This was the failure of the previous attempt: a registry entry with 510 an undefined Type-P (section 13 of RFC 2330 [RFC2330]) allows 511 implementation to be ambiguous. 513 7. Definition of the Performance Metric Registry 515 This Performance Metrics Registry is applicable to Performance 516 Metrics used for Active Measurement, Passive Measurement, and any 517 other form of Performance Metric. Each category of measurement has 518 unique properties, so some of the columns defined below are not 519 applicable for a given metric category. In this case, the column(s) 520 SHOULD be populated with the "NA" value (Non Applicable). However, 521 the "NA" value MUST NOT be used by any metric in the following 522 columns: Identifier, Name, URI, Status, Requester, Revision, Revision 523 Date, Description. In the future, a new category of metrics could 524 require additional columns, and adding new columns is a recognized 525 form of registry extension. The specification defining the new 526 column(s) MUST give guidelines to populate the new column(s) for 527 existing entries (in general). 529 The columns of the Performance Metrics Registry are defined below. 530 The columns are grouped into "Categories" to facilitate the use of 531 the registry. Categories are described at the 7.x heading level, and 532 columns are at the 7.x.y heading level. The Figure below illustrates 533 this organization. An entry (row) therefore gives a complete 534 description of a Registered Performance Metric. 536 Each column serves as a check-list item and helps to avoid omissions 537 during registration and expert review. 539 Registry Categories and Columns, shown as 541 Category 542 ------------------ 543 Column | Column | 545 Summary 546 ------------------------------------------------------------------------ 547 Identifier | Name | URIs | Desc. | Reference | Change Controller | Ver | 549 Metric Definition 550 ----------------------------------------- 551 Reference Definition | Fixed Parameters | 553 Method of Measurement 554 --------------------------------------------------------------------- 555 Reference | Packet | Traffic | Sampling | Run-time | Role | 556 Method | Stream | Filter | Distribution | Parameters | | 557 | Generation | 558 Output 559 ----------------------------------------- 560 Type | Reference | Units | Calibration | 561 | Definition | | | 563 Administrative Information 564 ------------------------------------ 565 Status |Requester | Rev | Rev.Date | 567 Comments and Remarks 568 -------------------- 569 7.1. Summary Category 571 7.1.1. Identifier 573 A numeric identifier for the Registered Performance Metric. This 574 identifier MUST be unique within the Performance Metrics Registry. 576 The Registered Performance Metric unique identifier is an unbounded 577 integer (range 0 to infinity). 579 The Identifier 0 should be Reserved. The Identifier values from 580 64512 to 65536 are reserved for private or experimental use, and the 581 user may encounter overlapping uses. 583 When adding newly Registered Performance Metrics to the Performance 584 Metrics Registry, IANA SHOULD assign the lowest available identifier 585 to the new Registered Performance Metric. 587 If a Performance Metrics Expert providing review determines that 588 there is a reason to assign a specific numeric identifier, possibly 589 leaving a temporary gap in the numbering, then the Performance Expert 590 SHALL inform IANA of this decision. 592 7.1.2. Name 594 As the name of a Registered Performance Metric is the first thing a 595 potential human implementor will use when determining whether it is 596 suitable for their measurement study, it is important to be as 597 precise and descriptive as possible. In future, users will review 598 the names to determine if the metric they want to measure has already 599 been registered, or if a similar entry is available as a basis for 600 creating a new entry. 602 Names are composed of the following elements, separated by an 603 underscore character "_": 605 MetricType_Method_SubTypeMethod_... Spec_Units_Output 607 o MetricType: a combination of the directional properties and the 608 metric measured, such as: 610 RTDelay (Round Trip Delay) 612 RTDNS (Response Time Domain Name Service) 614 RLDNS (Response Loss Domain Name Service) 616 OWDelay (One Way Delay) 617 RTLoss (Round Trip Loss) 619 OWLoss (One Way Loss) 621 OWPDV (One Way Packet Delay Variation) 623 OWIPDV (One Way Inter-Packet Delay Variation) 625 OWReorder (One Way Packet Reordering) 627 OWDuplic (One Way Packet Duplication) 629 OWBTC (One Way Bulk Transport Capacity) 631 OWMBM (One Way Model Based Metric) 633 SPMonitor (Single Point Monitor) 635 MPMonitor (Multi-Point Monitor) 637 o Method: One of the methods defined in [RFC7799], such as: 639 Active (depends on a dedicated measurement packet stream and 640 observations of the stream) 642 Passive (depends *solely* on observation of one or more 643 existing packet streams) 645 HybridType1 (obervations on one stream that combine both active 646 and passive methods) 648 HybridType2 (obervations on two or more streams that combine 649 both active and passive methods) 651 Spatial (Spatial Metric of RFC5644) 653 o SubTypeMethod: One or more sub-types to further describe the 654 features of the entry, such as: 656 ICMP (Internet Control Message Protocol) 658 IP (Internet Protocol) 660 DSCPxx (where xx is replaced by a Diffserv code point) 662 UDP (User Datagram Protocol) 664 TCP (Transport Control Protocol) 665 QUIC (QUIC transport protocol) 667 HS (Hand-Shake, such as TCP's 3-way HS) 669 Poisson (Packet generation using Poisson distribution) 671 Periodic (Periodic packet generation) 673 SendOnRcv (Sender keeps one packet in-transit by sending when 674 previous packet arrives) 676 PayloadxxxxB (where xxxx is replaced by an integer, the number 677 of octets in the Payload)) 679 SustainedBurst (Capacity test, worst case) 681 StandingQueue (test of bottleneck queue behavior) 683 SubTypeMethod values are separated by a hyphen "-" character, 684 which indicates that they belong to this element, and that their 685 order is unimportant when considering name uniqueness. 687 o Spec: RFC number and major section number that specifies this 688 Registry entry in the form RFCXXXXsecY, such as RFC7799sec3. 689 Note: the RFC number is not the Primary Reference specification 690 for the metric definition, such as [RFC7679] for One-way Delay; it 691 will contain the placeholder "RFCXXXXsecY" until the RFC number is 692 assigned to the specifying document, and would remain blank in 693 private registry entries without a corresponding RFC. 694 Anticipating the "RFC10K" problem, the number of the RFC continues 695 to replace RFCXXXX regardless of the number of digits in the RFC 696 number. Anticipating Registry Entries from other standards 697 bodies, the form of this Name Element MUST be proposed and 698 reviewed for consistency and uniqueness by the Expert Reviewer. 700 o Units: The units of measurement for the output, such as: 702 Seconds 704 Ratio (unitless) 706 Percent (value multiplied by 100) 708 Logical (1 or 0) 710 Packets 711 BPS (Bits per Second) 713 PPS (Packets per Second) 715 EventTotal (for unit-less counts) 717 Multiple (more than one type of unit) 719 Enumerated (a list of outcomes) 721 Unitless 723 o Output: The type of output resulting from measurement, such as: 725 Singleton 727 Raw (multiple Singletons) 729 Count 731 Minimum 733 Maximum 735 Median 737 Mean 739 95Percentile (95th Percentile) 741 99Percentile (99th Percentile) 743 StdDev (Standard Deviation) 745 Variance 747 PFI (Pass, Fail, Inconclusive) 749 FlowRecords (descriptions of flows observed) 751 LossRatio (lost packets to total packets, <=1) 753 An example is: 755 RTDelay_Active_IP-UDP-Periodic_RFCXXXXsecY_Seconds_95Percentile 757 as described in section 4 of [I-D.ietf-ippm-initial-registry]. 759 Note that private registries following the format described here 760 SHOULD use the prefix "Priv_" on any name to avoid unintended 761 conflicts (further considerations are described in section 10). 762 Private registry entries usually have no specifying RFC, thus the 763 Spec: element has no clear interpretation. 765 7.1.3. URIs 767 The URIs column MUST contain a URL [RFC3986] that uniquely identifies 768 and locates the metric entry so it is accessible through the 769 Internet. The URL points to a file containing all the human-readable 770 information for one registry entry. The URL SHALL reference a target 771 file that is HTML-formated and contains URLs to referenced sections 772 of HTML-ized RFCs. These target files for different entries can be 773 more easily edited and re-used when preparing new entries. The exact 774 form of the URL for each target file will be determined by IANA and 775 reside on "iana.org". The major sections of 776 [I-D.ietf-ippm-initial-registry] provide an example of a target file 777 in HTML form (sections 4 and higher). 779 7.1.4. Description 781 A Registered Performance Metric description is a written 782 representation of a particular Performance Metrics Registry entry. 783 It supplements the Registered Performance Metric name to help 784 Performance Metrics Registry users select relevant Registered 785 Performance Metrics. 787 7.1.5. Reference 789 This entry gives the specification containing the candidate registry 790 entry which was reviewed and agreed, if such an RFC or other 791 specification exists. 793 7.1.6. Change Controller 795 This entry names the entity responsible for approving revisions to 796 the registry entry, and SHALL provide contact information (for an 797 individual, where appropriate). 799 7.1.7. Version (of Registry Format) 801 This entry gives the version number for the registry format used. 802 Formats complying with this memo MUST use 1.0. The version number 803 SHALL NOT change unless a new RFC is published that changes the 804 registry format. 806 7.2. Metric Definition Category 808 This category includes columns to prompt all necessary details 809 related to the metric definition, including the RFC reference and 810 values of input factors, called fixed parameters, which are left open 811 in the RFC but have a particular value defined by the performance 812 metric. 814 7.2.1. Reference Definition 816 This entry provides a reference (or references) to the relevant 817 section(s) of the document(s) that define the metric, as well as any 818 supplemental information needed to ensure an unambiguous definition 819 for implementations. The reference needs to be an immutable 820 document, such as an RFC; for other standards bodies, it is likely to 821 be necessary to reference a specific, dated version of a 822 specification. 824 7.2.2. Fixed Parameters 826 Fixed Parameters are Parameters whose value must be specified in the 827 Performance Metrics Registry. The measurement system uses these 828 values. 830 Where referenced metrics supply a list of Parameters as part of their 831 descriptive template, a sub-set of the Parameters will be designated 832 as Fixed Parameters. As an example for active metrics, Fixed 833 Parameters determine most or all of the IPPM Framework convention 834 "packets of Type-P" as described in [RFC2330], such as transport 835 protocol, payload length, TTL, etc. An example for passive metrics 836 is for RTP packet loss calculation that relies on the validation of a 837 packet as RTP which is a multi-packet validation controlled by 838 MIN_SEQUENTIAL as defined by [RFC3550]. Varying MIN_SEQUENTIAL 839 values can alter the loss report and this value could be set as a 840 Fixed Parameter. 842 Parameters MUST have well-defined names. For human readers, the 843 hanging indent style is preferred, and any Parameter names and 844 definitions that do not appear in the Reference Method Specification 845 MUST appear in this column (or Run-time Parameters column). 847 Parameters MUST have a well-specified data format. 849 A Parameter which is a Fixed Parameter for one Performance Metrics 850 Registry entry may be designated as a Run-time Parameter for another 851 Performance Metrics Registry entry. 853 7.3. Method of Measurement Category 855 This category includes columns for references to relevant sections of 856 the immutable document(s) and any supplemental information needed to 857 ensure an unambiguous method for implementations. 859 7.3.1. Reference Method 861 This entry provides references to relevant sections of immutable 862 documents, such as RFC(s) (for other standards bodies, it is likely 863 to be necessary to reference a specific, dated version of a 864 specification) describing the method of measurement, as well as any 865 supplemental information needed to ensure unambiguous interpretation 866 for implementations referring to the RFC text. 868 Specifically, this section should include pointers to pseudocode or 869 actual code that could be used for an unambigious implementation. 871 7.3.2. Packet Stream Generation 873 This column applies to Performance Metrics that generate traffic as 874 part of their Measurement Method, including but not necessarily 875 limited to Active metrics. The generated traffic is referred as a 876 stream and this column describes its characteristics. 878 Each entry for this column contains the following information: 880 o Value: The name of the packet stream scheduling discipline 882 o Reference: the specification where the parameters of the stream 883 are defined 885 The packet generation stream may require parameters such as the 886 average packet rate and distribution truncation value for streams 887 with Poisson-distributed inter-packet sending times. In case such 888 parameters are needed, they should be included either in the Fixed 889 parameter column or in the run time parameter column, depending on 890 wether they will be fixed or will be an input for the metric. 892 The simplest example of stream specification is Singleton scheduling 893 (see [RFC2330]), where a single atomic measurement is conducted. 894 Each atomic measurement could consist of sending a single packet 895 (such as a DNS request) or sending several packets (for example, to 896 request a webpage). Other streams support a series of atomic 897 measurements in a "sample", with a schedule defining the timing 898 between each transmitted packet and subsequent measurement. 899 Principally, two different streams are used in IPPM metrics, Poisson 900 distributed as described in [RFC2330] and Periodic as described in 902 [RFC3432]. Both Poisson and Periodic have their own unique 903 parameters, and the relevant set of parameters names and values 904 should be included either in the Fixed Parameters column or in the 905 Run-time parameter column. 907 7.3.3. Traffic Filter 909 This column applies to Performance Metrics that observe packets 910 flowing through (the device with) the measurement agent i.e. that is 911 not necessarily addressed to the measurement agent. This includes 912 but is not limited to Passive Metrics. The filter specifies the 913 traffic that is measured. This includes protocol field values/ 914 ranges, such as address ranges, and flow or session identifiers. 916 The traffic filter itself depends on needs of the metric itself and a 917 balance of an operator's measurement needs and a user's need for 918 privacy. Mechanics for conveying the filter criteria might be the 919 BPF (Berkley Packet Filter) or PSAMP [RFC5475] Property Match 920 Filtering which reuses IPFIX [RFC7012]. An example BPF string for 921 matching TCP/80 traffic to remote destination net 192.0.2.0/24 would 922 be "dst net 192.0.2.0/24 and tcp dst port 80". More complex filter 923 engines might be supported by the implementation that might allow for 924 matching using Deep Packet Inspection (DPI) technology. 926 The traffic filter includes the following information: 928 Type: the type of traffic filter used, e.g. BPF, PSAMP, OpenFlow 929 rule, etc. as defined by a normative reference 931 Value: the actual set of rules expressed 933 7.3.4. Sampling Distribution 935 The sampling distribution defines out of all the packets that match 936 the traffic filter, which one of those are actually used for the 937 measurement. One possibility is "all" which implies that all packets 938 matching the Traffic filter are considered, but there may be other 939 sampling strategies. It includes the following information: 941 Value: the name of the sampling distribution 943 Reference definition: pointer to the specification where the 944 sampling distribution is properly defined. 946 The sampling distribution may require parameters. In case such 947 parameters are needed, they should be included either in the Fixed 948 parameter column or in the run time parameter column, depending on 949 whether they will be fixed or will be an input for the metric. 951 Sampling and Filtering Techniques for IP Packet Selection are 952 documented in the PSAMP (Packet Sampling) [RFC5475], while the 953 Framework for Packet Selection and Reporting, [RFC5474] provides more 954 background information. The sampling distribution parameters might 955 be expressed in terms of the Information Model for Packet Sampling 956 Exports, [RFC5477], and the Flow Selection Techniques, [RFC7014]. 958 7.3.5. Run-time Parameters 960 Run-Time Parameters are Parameters that must be determined, 961 configured into the measurement system, and reported with the results 962 for the context to be complete. However, the values of these 963 parameters is not specified in the Performance Metrics Registry (like 964 the Fixed Parameters), rather these parameters are listed as an aid 965 to the measurement system implementer or user (they must be left as 966 variables, and supplied on execution). 968 Where metrics supply a list of Parameters as part of their 969 descriptive template, a sub-set of the Parameters will be designated 970 as Run-Time Parameters. 972 Parameters MUST have well defined names. For human readers, the 973 hanging indent style is preferred, and the names and definitions that 974 do not appear in the Reference Method Specification MUST appear in 975 this column. 977 A Data Format for each Run-time Parameter MUST be specified in this 978 column, to simplify the control and implementation of measurement 979 devices. For example, parameters that include an IPv4 address can be 980 encoded as a 32 bit integer (i.e. binary base64 encoded value) or ip- 981 address as defined in [RFC6991]. The actual encoding(s) used must be 982 explicitly defined for each Run-time parameter. IPv6 addresses and 983 options MUST be accomodated, allowing Registered Metrics to be used 984 in either address family. 986 Examples of Run-time Parameters include IP addresses, measurement 987 point designations, start times and end times for measurement, and 988 other information essential to the method of measurement. 990 7.3.6. Role 992 In some methods of measurement, there may be several roles defined, 993 e.g., for a one-way packet delay active measurement there is one 994 measurement agent that generates the packets and another agent that 995 receives the packets. This column contains the name of the Role(s) 996 for this particular entry. In the one-way delay example above, there 997 should be two entries in the Role registry column, one for each Role 998 (Source and Destination). When a measurement agent is instructed to 999 perform the "Source" Role for one-way delay metric, the agent knows 1000 that it is required to generate packets. The values for this field 1001 are defined in the reference method of measurement (and this 1002 frequently results in abbreviated role names such as "Src"). 1004 When the Role column of a registry entry defines more than one Role, 1005 then the Role SHALL be treated as a Run-time Parameter and supplied 1006 for execution. It should be noted that the LMAP framework [RFC7594] 1007 distinguishes the Role from other Run-time Parameters, and defines a 1008 special parameter "Roles" inside the registry-grouping function list 1009 in the LMAP YANG model[RFC8194]. 1011 7.4. Output Category 1013 For entries which involve a stream and many singleton measurements, a 1014 statistic may be specified in this column to summarize the results to 1015 a single value. If the complete set of measured singletons is 1016 output, this will be specified here. 1018 Some metrics embed one specific statistic in the reference metric 1019 definition, while others allow several output types or statistics. 1021 7.4.1. Type 1023 This column contains the name of the output type. The output type 1024 defines a single type of result that the metric produces. It can be 1025 the raw results (packet send times and singleton metrics), or it can 1026 be a summary statistic. The specification of the output type MUST 1027 define the format of the output. In some systems, format 1028 specifications will simplify both measurement implementation and 1029 collection/storage tasks. Note that if two different statistics are 1030 required from a single measurement (for example, both "Xth percentile 1031 mean" and "Raw"), then a new output type must be defined ("Xth 1032 percentile mean AND Raw"). See the Naming section above for a list 1033 of Output Types. 1035 7.4.2. Reference Definition 1037 This column contains a pointer to the specification(s) where the 1038 output type and format are defined. 1040 7.4.3. Metric Units 1042 The measured results must be expressed using some standard dimension 1043 or units of measure. This column provides the units. 1045 When a sample of singletons (see Section 11 of[RFC2330] for 1046 definitions of these terms) is collected, this entry will specify the 1047 units for each measured value. 1049 7.4.4. Calibration 1051 Some specifications for Methods of Measurement include the 1052 possibility to perform an error calibration. Section 3.7.3 of 1053 [RFC7679] is one example. In the registry entry, this field will 1054 identify a method of calibration for the metric, and when available, 1055 the measurement system SHOULD perform the calibration when requested 1056 and produce the output with an indication that it is the result of a 1057 calbration method. In-situ calibration could be enabled with an 1058 internal loopback that includes as much of the measurement system as 1059 possible, performs address manipulation as needed, and provides some 1060 form of isolation (e.g., deterministic delay) to avoid send-receive 1061 interface contention. Some portion of the random and systematic 1062 error can be characterized this way. 1064 For one-way delay measurements, the error calibration must include an 1065 assessment of the internal clock synchronization with its external 1066 reference (this internal clock is supplying timestamps for 1067 measurement). In practice, the time offsets of clocks at both the 1068 source and destination are needed to estimate the systematic error 1069 due to imperfect clock synchronization (the time offsets are 1070 smoothed, thus the random variation is not usually represented in the 1071 results). 1073 Both internal loopback calibration and clock synchronization can be 1074 used to estimate the *available accuracy* of the Output Metric Units. 1075 For example, repeated loopback delay measurements will reveal the 1076 portion of the Output result resolution which is the result of system 1077 noise, and thus inaccurate. 1079 7.5. Administrative information 1081 7.5.1. Status 1083 The status of the specification of this Registered Performance 1084 Metric. Allowed values are 'current' and 'deprecated'. All newly 1085 defined Information Elements have 'current' status. 1087 7.5.2. Requester 1089 The requester for the Registered Performance Metric. The requester 1090 MAY be a document, such as RFC, or person. 1092 7.5.3. Revision 1094 The revision number of a Registered Performance Metric, starting at 0 1095 for Registered Performance Metrics at time of definition and 1096 incremented by one for each revision. 1098 7.5.4. Revision Date 1100 The date of acceptance or the most recent revision for the Registered 1101 Performance Metric. The date SHALL be determined by IANA and the 1102 reviewing Performance Metrics Expert. 1104 7.6. Comments and Remarks 1106 Besides providing additional details which do not appear in other 1107 categories, this open Category (single column) allows for unforeseen 1108 issues to be addressed by simply updating this informational entry. 1110 8. The Life-Cycle of Registered Performance Metrics 1112 Once a Performance Metric or set of Performance Metrics has been 1113 identified for a given application, candidate Performance Metrics 1114 Registry entry specifications prepared in accordance with Section 7 1115 should be submitted to IANA to follow the process for review by the 1116 Performance Metric Experts, as defined below. This process is also 1117 used for other changes to the Performance Metrics Registry, such as 1118 deprecation or revision, as described later in this section. 1120 It is desirable that the author(s) of a candidate Performance Metrics 1121 Registry entry seek review in the relevant IETF working group, or 1122 offer the opportunity for review on the working group mailing list. 1124 8.1. Adding new Performance Metrics to the Performance Metrics Registry 1126 Requests to add Registered Performance Metrics in the Performance 1127 Metrics Registry SHALL be submitted to IANA, which forwards the 1128 request to a designated group of experts (Performance Metric Experts) 1129 appointed by the IESG; these are the reviewers called for by the 1130 Expert Review [RFC8126]policy defined for the Performance Metrics 1131 Registry. The Performance Metric Experts review the request for such 1132 things as compliance with this document, compliance with other 1133 applicable Performance Metric-related RFCs, and consistency with the 1134 currently defined set of Registered Performance Metrics. 1136 Submission to IANA MAY be during IESG review (leading to IETF 1137 Standards Action), where an Internet Draft proposes one or more 1138 Registered Performance Metrics to be added to the Performance Metrics 1139 Registry, including the text of the proposed Registered Performance 1140 Metric(s). 1142 If the proposed registry entry is defined in an RFC but is not yet 1143 widely deployed, there SHOULD be a statement in the RFC that says the 1144 proposed registry entry is not ready for registration, and use SHOULD 1145 employ a private/experimental ID. It is the responsibility of the 1146 document authors to submit the request to IANA when the proposed 1147 registry entry is ready for official registration. 1149 Authors of proposed Registered Performance Metrics SHOULD review 1150 compliance with the specifications in this document to check their 1151 submissions before sending them to IANA. 1153 At least one Performance Metric Expert should endeavor to complete 1154 referred reviews in a timely manner. If the request is acceptable, 1155 the Performance Metric Experts signify their approval to IANA, and 1156 IANA updates the Performance Metrics Registry. If the request is not 1157 acceptable, the Performance Metric Experts MAY coordinate with the 1158 requester to change the request to be compliant, otherwise IANA SHALL 1159 coordinate resolution of issues on behalf of the expert. The 1160 Performance Metric Experts MAY choose to reject clearly frivolous or 1161 inappropriate change requests outright, but such exceptional 1162 circumstances should be rare. 1164 This process should not in any way be construed as allowing the 1165 Performance Metric Experts to overrule IETF consensus. Specifically, 1166 any Registered Performance Metrics that were added to the Performance 1167 Metrics Registry with IETF consensus require IETF consensus for 1168 revision or deprecation. 1170 Decisions by the Performance Metric Experts may be appealed as in 1171 Section 7 of [RFC8126]. 1173 8.2. Revising Registered Performance Metrics 1175 A request for Revision is only permitted when the requested changes 1176 maintain backward-compatibility with implementations of the prior 1177 Performance Metrics Registry entry describing a Registered 1178 Performance Metric (entries with lower revision numbers, but the same 1179 Identifier and Name). 1181 The purpose of the Status field in the Performance Metrics Registry 1182 is to indicate whether the entry for a Registered Performance Metric 1183 is 'current' or 'deprecated'. 1185 In addition, no policy is defined for revising the Performance Metric 1186 entries in the IANA Regsirty or addressing errors therein. To be 1187 clear, changes and deprecations within the Performance Metrics 1188 Registry are not encouraged, and should be avoided to the extent 1189 possible. However, in recognition that change is inevitable, the 1190 provisions of this section address the need for revisions. 1192 Revisions are initiated by sending a candidate Registered Performance 1193 Metric definition to IANA, as in Section 8.1, identifying the 1194 existing Performance Metrics Registry entry, and explaining how and 1195 why the existing entry should be revised. 1197 The primary requirement in the definition of procedures for managing 1198 changes to existing Registered Performance Metrics is avoidance of 1199 measurement interoperability problems; the Performance Metric Experts 1200 must work to maintain interoperability above all else. Changes to 1201 Registered Performance Metrics may only be done in an interoperable 1202 way; necessary changes that cannot be done in a way to allow 1203 interoperability with unchanged implementations MUST result in the 1204 creation of a new Registered Performance Metric (with a new Name, 1205 replacing the RFCXXXXsecY portion of the name) and possibly the 1206 deprecation of the earlier metric. 1208 A change to a Registered Performance Metric SHALL be determined to be 1209 backward-compatible only when: 1211 1. it involves the correction of an error that is obviously only 1212 editorial; or 1214 2. it corrects an ambiguity in the Registered Performance Metric's 1215 definition, which itself leads to issues severe enough to prevent 1216 the Registered Performance Metric's usage as originally defined; 1217 or 1219 3. it corrects missing information in the metric definition without 1220 changing its meaning (e.g., the explicit definition of 'quantity' 1221 semantics for numeric fields without a Data Type Semantics 1222 value); or 1224 4. it harmonizes with an external reference that was itself 1225 corrected. 1227 If a Performance Metric revision is deemed permissible and backward- 1228 compatible by the Performance Metric Experts, according to the rules 1229 in this document, IANA SHOULD execute the change(s) in the 1230 Performance Metrics Registry. The requester of the change is 1231 appended to the original requester in the Performance Metrics 1232 Registry. The Name of the revised Registered Performance Metric, 1233 including the RFCXXXXsecY portion of the name, SHALL remain unchamged 1234 (even when the change is the result of IETF Standards Action; the 1235 revised registry entry SHOULD reference the new immutable document, 1236 such as an RFC or for other standards bodies, it is likely to be 1237 necessary to reference a specific, dated version of a specification, 1238 in an appropriate category and column). 1240 Each Registered Performance Metric in the Performance Metrics 1241 Registry has a revision number, starting at zero. Each change to a 1242 Registered Performance Metric following this process increments the 1243 revision number by one. 1245 When a revised Registered Performance Metric is accepted into the 1246 Performance Metrics Registry, the date of acceptance of the most 1247 recent revision is placed into the revision Date column of the 1248 registry for that Registered Performance Metric. 1250 Where applicable, additions to Registered Performance Metrics in the 1251 form of text Comments or Remarks should include the date, but such 1252 additions may not constitute a revision according to this process. 1254 Older version(s) of the updated metric entries are kept in the 1255 registry for archival purposes. The older entries are kept with all 1256 fields unmodified (version, revision date) except for the status 1257 field that SHALL be changed to "Deprecated". 1259 8.3. Deprecating Registered Performance Metrics 1261 Changes that are not permissible by the above criteria for Registered 1262 Performance Metric's revision may only be handled by deprecation. A 1263 Registered Performance Metric MAY be deprecated and replaced when: 1265 1. the Registered Performance Metric definition has an error or 1266 shortcoming that cannot be permissibly changed as in Section 8.2 1267 Revising Registered Performance Metrics; or 1269 2. the deprecation harmonizes with an external reference that was 1270 itself deprecated through that reference's accepted deprecation 1271 method. 1273 A request for deprecation is sent to IANA, which passes it to the 1274 Performance Metric Experts for review. When deprecating an 1275 Performance Metric, the Performance Metric description in the 1276 Performance Metrics Registry must be updated to explain the 1277 deprecation, as well as to refer to any new Performance Metrics 1278 created to replace the deprecated Performance Metric. 1280 The revision number of a Registered Performance Metric is incremented 1281 upon deprecation, and the revision Date updated, as with any 1282 revision. 1284 The use of deprecated Registered Performance Metrics should result in 1285 a log entry or human-readable warning by the respective application. 1287 Names and Metric IDs of deprecated Registered Performance Metrics 1288 must not be reused. 1290 The deprecated entries are kept with all fields unmodified, except 1291 the version, revision date, and the status field (changed to 1292 "Deprecated"). 1294 9. Security considerations 1296 This draft defines a registry structure, and does not itself 1297 introduce any new security considerations for the Internet. The 1298 definition of Performance Metrics for this registry may introduce 1299 some security concerns, but the mandatory references should have 1300 their own considerations for secuity, and such definitions should be 1301 reviewed with security in mind if the security considerations are not 1302 covered by one or more reference standards. 1304 10. IANA Considerations 1306 With the background and processes described in earlier sections, this 1307 document requests the following IANA Actions. 1309 Editor's Note: Mock-ups of the implementation of this set of requests 1310 have been prepared with IANA's help during development of this memo, 1311 and have been captured in the Proceedings of IPPM working group 1312 sessions. IANA is currently preparing a mock-up. A recent version 1313 is available here: http://encrypted.net/IETFMetricsRegistry-106.html 1315 10.1. Registry Group 1317 The new registry group SHALL be named, "PERFORMANCE METRICS Group". 1319 Registration Procedure: Specification Required 1321 Reference: 1323 Experts: Performance Metrics Experts 1325 Note: TBD 1327 10.2. Performance Metric Name Elements 1329 This document specifies the procedure for Performance Metrics Name 1330 Element Registry setup. IANA is requested to create a new set of 1331 registries for Performance Metric Name Elements called "Registered 1332 Performance Metric Name Elements". Each Registry, whose names are 1333 listed below: 1335 MetricType: 1337 Method: 1339 SubTypeMethod: 1341 Spec: 1343 Units: 1345 Output: 1347 will contain the current set of possibilities for Performance Metrics 1348 Registry Entry Names. 1350 To populate the Registered Performance Metric Name Elements at 1351 creation, the IANA is asked to use the lists of values for each name 1352 element listed in Section 7.1.2. The Name Elements in each registry 1353 are case-sensitive. 1355 When preparing a Metric entry for Registration, the developer SHOULD 1356 choose Name elements from among the registered elements. However, if 1357 the proposed metric is unique in a significant way, it may be 1358 necessary to propose a new Name element to properly describe the 1359 metric, as described below. 1361 A candidate Metric Entry RFC or immutable document for IANA and 1362 Expert Review would propose one or more new element values required 1363 to describe the unique entry, and the new name element(s) would be 1364 reviewed along with the metric entry. New assignments for Registered 1365 Performance Metric Name Elements will be administered by IANA through 1366 Expert Review [RFC8126], i.e., review by one of a group of experts, 1367 the Performance Metric Experts, who are appointed by the IESG upon 1368 recommendation of the Transport Area Directors. 1370 10.3. New Performance Metrics Registry 1372 This document specifies the procedure for Performance Metrics 1373 Registry setup. IANA is requested to create a new registry for 1374 Performance Metrics called "Performance Metrics Registry". This 1375 Registry will contain the following Summary columns: 1377 Identifier: 1379 Name: 1381 URIs: 1383 Description: 1385 Reference: 1387 Change Controller: 1389 Version: 1391 Descriptions of these columns and additional information found in the 1392 template for registry entries (categories and columns) are further 1393 defined in section Section 7. 1395 The "Identifier" 0 should be Reserved. "The Identifier" values from 1396 64512 to 65536 are reserved for private use. 1398 Names starting with the prefix Priv_ are reserved for private use, 1399 and are not considered for registration. The "Name" column entries 1400 are further defined in section Section 7. 1402 The "URIs" column will have a URL to the full template of each 1403 registry entry. The Registry Entry text SHALL be HTML-ized to aid 1404 the reader, with links to reference RFCs (similar to the way that 1405 Internet Drafts are HTML-ized, the same tool can perform the 1406 function) or immutable document. 1408 The "Reference" column will include an RFC number, an approved 1409 specification designator from another standards body, other immutable 1410 document, or the contact person @@@@ acm thinks this list needs to 1411 exclude contact person now. 1413 New assignments for Performance Metrics Registry will be administered 1414 by IANA through Expert Review [RFC8126], i.e., review by one of a 1415 group of experts, the Performance Metric Experts, who are appointed 1416 by the IESG upon recommendation of the Transport Area Directors, or 1417 by Standards Action. The experts can be initially drawn from the 1418 Working Group Chairs, document editors, and members of the 1419 Performance Metrics Directorate, among other sources of experts. 1421 Extensions of the Performance Metrics Registry require IETF Standards 1422 Action. Only one form of registry extension is envisaged: 1424 1. Adding columns, or both categories and columns, to accommodate 1425 unanticipated aspects of new measurements and metric categories. 1427 If the Performance Metrics Registry is extended in this way, the 1428 Version number of future entries complying with the extension SHALL 1429 be incremented (either in the unit or tenths digit, depending on the 1430 degree of extension. 1432 11. Blank Registry Template 1434 This section provides a blank template to help IANA and registry 1435 entry writers. 1437 11.1. Summary 1439 This category includes multiple indexes to the registry entry: the 1440 element ID and metric name. 1442 11.1.1. ID (Identifier) 1444 1446 11.1.2. Name 1448 1450 11.1.3. URIs 1452 URL: http:/// 1454 11.1.4. Description 1456 1458 11.1.5. Change Controller 1460 11.1.6. Version (of Registry Format) 1462 11.2. Metric Definition 1464 This category includes columns to prompt the entry of all necessary 1465 details related to the metric definition, including the RFC reference 1466 and values of input factors, called fixed parameters. 1468 11.2.1. Reference Definition 1470 1472 1474 11.2.2. Fixed Parameters 1476 1480 11.3. Method of Measurement 1482 This category includes columns for references to relevant sections of 1483 the RFC(s) and any supplemental information needed to ensure an 1484 unambiguous methods for implementations. 1486 11.3.1. Reference Method 1488 1491 11.3.2. Packet Stream Generation 1493 1495 11.3.3. Traffic Filtering (observation) Details 1497 The measured results based on a filtered version of the packets 1498 observed, and this section provides the filter details (when 1499 present). 1501
. 1503 11.3.4. Sampling Distribution 1505 1508 11.3.5. Run-time Parameters and Data Format 1510 Run-time Parameters are input factors that must be determined, 1511 configured into the measurement system, and reported with the results 1512 for the context to be complete. 1514 1516 11.3.6. Roles 1518 1520 11.4. Output 1522 This category specifies all details of the Output of measurements 1523 using the metric. 1525 11.4.1. Type 1527 1529 11.4.2. Reference Definition 1531 1533 11.4.3. Metric Units 1535 . 1538 11.4.4. Calibration 1540 1542 11.5. Administrative items 1544 11.5.1. Status 1546 1548 11.5.2. Requestor 1550 1552 11.5.3. Revision 1554 <1.0> 1556 11.5.4. Revision Date 1558 1560 11.6. Comments and Remarks 1562 1564 12. Acknowledgments 1566 Thanks to Brian Trammell and Bill Cerveny, IPPM chairs, for leading 1567 some brainstorming sessions on this topic. Thanks to Barbara Stark 1568 and Juergen Schoenwaelder for the detailed feedback and suggestions. 1569 Thanks to Andrew McGregor for suggestions on metric naming. Thanks 1570 to Michelle Cotton for her early IANA review, and to Amanda Barber 1571 for answering questions related to the presentation of the registry 1572 and accessibility of the complete template via URL. 1574 13. References 1576 13.1. Normative References 1578 [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 1579 3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996, 1580 . 1582 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1583 Requirement Levels", BCP 14, RFC 2119, 1584 DOI 10.17487/RFC2119, March 1997, 1585 . 1587 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1588 "Framework for IP Performance Metrics", RFC 2330, 1589 DOI 10.17487/RFC2330, May 1998, 1590 . 1592 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 1593 Resource Identifier (URI): Generic Syntax", STD 66, 1594 RFC 3986, DOI 10.17487/RFC3986, January 2005, 1595 . 1597 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1598 Performance Metric Development", BCP 170, RFC 6390, 1599 DOI 10.17487/RFC6390, October 2011, 1600 . 1602 [RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz, 1603 "IP Performance Metrics (IPPM) Standard Advancement 1604 Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March 1605 2012, . 1607 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 1608 Writing an IANA Considerations Section in RFCs", BCP 26, 1609 RFC 8126, DOI 10.17487/RFC8126, June 2017, 1610 . 1612 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1613 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1614 May 2017, . 1616 13.2. Informative References 1618 [I-D.ietf-ippm-initial-registry] 1619 Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza, 1620 "Initial Performance Metrics Registry Entries", draft- 1621 ietf-ippm-initial-registry-12 (work in progress), 1622 September 2019. 1624 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1625 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 1626 September 1999, . 1628 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1629 performance measurement with periodic streams", RFC 3432, 1630 DOI 10.17487/RFC3432, November 2002, 1631 . 1633 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1634 Jacobson, "RTP: A Transport Protocol for Real-Time 1635 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1636 July 2003, . 1638 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 1639 "RTP Control Protocol Extended Reports (RTCP XR)", 1640 RFC 3611, DOI 10.17487/RFC3611, November 2003, 1641 . 1643 [RFC4148] Stephan, E., "IP Performance Metrics (IPPM) Metrics 1644 Registry", BCP 108, RFC 4148, DOI 10.17487/RFC4148, August 1645 2005, . 1647 [RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., 1648 Grossglauser, M., and J. Rexford, "A Framework for Packet 1649 Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474, 1650 March 2009, . 1652 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. 1653 Raspall, "Sampling and Filtering Techniques for IP Packet 1654 Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, 1655 . 1657 [RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. 1658 Carle, "Information Model for Packet Sampling Exports", 1659 RFC 5477, DOI 10.17487/RFC5477, March 2009, 1660 . 1662 [RFC6035] Pendleton, A., Clark, A., Johnston, A., and H. Sinnreich, 1663 "Session Initiation Protocol Event Package for Voice 1664 Quality Reporting", RFC 6035, DOI 10.17487/RFC6035, 1665 November 2010, . 1667 [RFC6248] Morton, A., "RFC 4148 and the IP Performance Metrics 1668 (IPPM) Registry of Metrics Are Obsolete", RFC 6248, 1669 DOI 10.17487/RFC6248, April 2011, 1670 . 1672 [RFC6991] Schoenwaelder, J., Ed., "Common YANG Data Types", 1673 RFC 6991, DOI 10.17487/RFC6991, July 2013, 1674 . 1676 [RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model 1677 for IP Flow Information Export (IPFIX)", RFC 7012, 1678 DOI 10.17487/RFC7012, September 2013, 1679 . 1681 [RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1682 Selection Techniques", RFC 7014, DOI 10.17487/RFC7014, 1683 September 2013, . 1685 [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 1686 Aitken, P., and A. Akhter, "A Framework for Large-Scale 1687 Measurement of Broadband Performance (LMAP)", RFC 7594, 1688 DOI 10.17487/RFC7594, September 2015, 1689 . 1691 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1692 Ed., "A One-Way Delay Metric for IP Performance Metrics 1693 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1694 2016, . 1696 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1697 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1698 May 2016, . 1700 [RFC8194] Schoenwaelder, J. and V. Bajpai, "A YANG Data Model for 1701 LMAP Measurement Agents", RFC 8194, DOI 10.17487/RFC8194, 1702 August 2017, . 1704 Authors' Addresses 1706 Marcelo Bagnulo 1707 Universidad Carlos III de 1708 Madrid 1709 Av. Universidad 30 1710 Leganes, Madrid 28911 1711 SPAIN 1713 Phone: 34 91 6249500 1714 Email: marcelo@it.uc3m.es 1715 URI: http://www.it.uc3m.es 1717 Benoit Claise 1718 Cisco Systems, 1719 Inc. 1720 De Kleetlaan 6a b1 1721 1831 Diegem 1722 Belgium 1724 Email: bclaise@cisco.com 1726 Philip Eardley 1727 BT 1728 Adastral Park, Martlesham Heath 1729 Ipswich 1730 ENGLAND 1732 Email: philip.eardley@bt.com 1734 Al Morton 1735 AT&T Labs 1736 200 Laurel Avenue South 1737 Middletown, NJ 1738 USA 1740 Email: acmorton@att.com 1741 Aamer Akhter 1742 Consultant 1743 118 Timber Hitch 1744 Cary, NC 1745 USA 1747 Email: aakhter@gmail.com