< draft-ietf-ippm-metric-registry-22.txt   draft-ietf-ippm-metric-registry-23.txt >
Network Working Group M. Bagnulo Network Working Group M. Bagnulo
Internet-Draft UC3M Internet-Draft UC3M
Intended status: Best Current Practice B. Claise Intended status: Standards Track B. Claise
Expires: May 30, 2020 Cisco Systems, Inc. Expires: June 13, 2020 Cisco Systems, Inc.
P. Eardley P. Eardley
BT BT
A. Morton A. Morton
AT&T Labs AT&T Labs
A. Akhter A. Akhter
Consultant Consultant
November 27, 2019 December 11, 2019
Registry for Performance Metrics Registry for Performance Metrics
draft-ietf-ippm-metric-registry-22 draft-ietf-ippm-metric-registry-23
Abstract Abstract
This document defines the format for the IANA Performance Metrics This document defines the format for the IANA Performance Metrics
Registry. This document also gives a set of guidelines for Registry. This document also gives a set of guidelines for
Registered Performance Metric requesters and reviewers. Registered Performance Metric requesters and reviewers.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
skipping to change at page 1, line 39 skipping to change at page 1, line 39
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 30, 2020. This Internet-Draft will expire on June 13, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 21 skipping to change at page 2, line 21
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4. Motivation for a Performance Metrics Registry . . . . . . . . 8 4. Motivation for a Performance Metrics Registry . . . . . . . . 8
4.1. Interoperability . . . . . . . . . . . . . . . . . . . . 8 4.1. Interoperability . . . . . . . . . . . . . . . . . . . . 8
4.2. Single point of reference for Performance Metrics . . . . 9 4.2. Single point of reference for Performance Metrics . . . . 9
4.3. Side benefits . . . . . . . . . . . . . . . . . . . . . . 9 4.3. Side benefits . . . . . . . . . . . . . . . . . . . . . . 9
5. Criteria for Performance Metrics Registration . . . . . . . . 9 5. Criteria for Performance Metrics Registration . . . . . . . . 9
6. Performance Metric Registry: Prior attempt . . . . . . . . . 10 6. Performance Metric Registry: Prior attempt . . . . . . . . . 10
6.1. Why this Attempt Will Succeed . . . . . . . . . . . . . . 11 6.1. Why this Attempt Should Succeed . . . . . . . . . . . . . 11
7. Definition of the Performance Metric Registry . . . . . . . . 11 7. Definition of the Performance Metric Registry . . . . . . . . 11
7.1. Summary Category . . . . . . . . . . . . . . . . . . . . 13 7.1. Summary Category . . . . . . . . . . . . . . . . . . . . 13
7.1.1. Identifier . . . . . . . . . . . . . . . . . . . . . 13 7.1.1. Identifier . . . . . . . . . . . . . . . . . . . . . 13
7.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 13 7.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 13
7.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 17 7.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 17
7.1.4. Description . . . . . . . . . . . . . . . . . . . . . 17 7.1.4. Description . . . . . . . . . . . . . . . . . . . . . 17
7.1.5. Reference . . . . . . . . . . . . . . . . . . . . . . 17 7.1.5. Reference . . . . . . . . . . . . . . . . . . . . . . 17
7.1.6. Change Controller . . . . . . . . . . . . . . . . . . 17 7.1.6. Change Controller . . . . . . . . . . . . . . . . . . 17
7.1.7. Version (of Registry Format) . . . . . . . . . . . . 17 7.1.7. Version (of Registry Format) . . . . . . . . . . . . 18
7.2. Metric Definition Category . . . . . . . . . . . . . . . 18 7.2. Metric Definition Category . . . . . . . . . . . . . . . 18
7.2.1. Reference Definition . . . . . . . . . . . . . . . . 18 7.2.1. Reference Definition . . . . . . . . . . . . . . . . 18
7.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 18 7.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 18
7.3. Method of Measurement Category . . . . . . . . . . . . . 19 7.3. Method of Measurement Category . . . . . . . . . . . . . 19
7.3.1. Reference Method . . . . . . . . . . . . . . . . . . 19 7.3.1. Reference Method . . . . . . . . . . . . . . . . . . 19
7.3.2. Packet Stream Generation . . . . . . . . . . . . . . 19 7.3.2. Packet Stream Generation . . . . . . . . . . . . . . 19
7.3.3. Traffic Filter . . . . . . . . . . . . . . . . . . . 20 7.3.3. Traffic Filter . . . . . . . . . . . . . . . . . . . 20
7.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 20 7.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 20
7.3.5. Run-time Parameters . . . . . . . . . . . . . . . . . 21 7.3.5. Run-time Parameters . . . . . . . . . . . . . . . . . 21
7.3.6. Role . . . . . . . . . . . . . . . . . . . . . . . . 21 7.3.6. Role . . . . . . . . . . . . . . . . . . . . . . . . 22
7.4. Output Category . . . . . . . . . . . . . . . . . . . . . 22 7.4. Output Category . . . . . . . . . . . . . . . . . . . . . 22
7.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 22 7.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 22
7.4.2. Reference Definition . . . . . . . . . . . . . . . . 22 7.4.2. Reference Definition . . . . . . . . . . . . . . . . 23
7.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 22 7.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 23
7.4.4. Calibration . . . . . . . . . . . . . . . . . . . . . 23 7.4.4. Calibration . . . . . . . . . . . . . . . . . . . . . 23
7.5. Administrative information . . . . . . . . . . . . . . . 23 7.5. Administrative information . . . . . . . . . . . . . . . 24
7.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 23 7.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 24
7.5.2. Requester . . . . . . . . . . . . . . . . . . . . . . 23 7.5.2. Requester . . . . . . . . . . . . . . . . . . . . . . 24
7.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 24 7.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 24
7.5.4. Revision Date . . . . . . . . . . . . . . . . . . . . 24 7.5.4. Revision Date . . . . . . . . . . . . . . . . . . . . 24
7.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 24 7.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 24
8. The Life-Cycle of Registered Performance Metrics . . . . . . 24 8. Processes for Managing the Performance Metric Registry Group 24
8.1. Adding new Performance Metrics to the Performance Metrics 8.1. Adding new Performance Metrics to the Performance Metrics
Registry . . . . . . . . . . . . . . . . . . . . . . . . 24 Registry . . . . . . . . . . . . . . . . . . . . . . . . 25
8.2. Revising Registered Performance Metrics . . . . . . . . . 25 8.2. Revising Registered Performance Metrics . . . . . . . . . 26
8.3. Deprecating Registered Performance Metrics . . . . . . . 27 8.3. Deprecating Registered Performance Metrics . . . . . . . 27
9. Security considerations . . . . . . . . . . . . . . . . . . . 28 9. Security considerations . . . . . . . . . . . . . . . . . . . 28
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29
10.1. Registry Group . . . . . . . . . . . . . . . . . . . . . 28 10.1. Registry Group . . . . . . . . . . . . . . . . . . . . . 29
10.2. Performance Metric Name Elements . . . . . . . . . . . . 28 10.2. Performance Metric Name Elements . . . . . . . . . . . . 29
10.3. New Performance Metrics Registry . . . . . . . . . . . . 29 10.3. New Performance Metrics Registry . . . . . . . . . . . . 30
11. Blank Registry Template . . . . . . . . . . . . . . . . . . . 31 11. Blank Registry Template . . . . . . . . . . . . . . . . . . . 31
11.1. Summary . . . . . . . . . . . . . . . . . . . . . . . . 31 11.1. Summary . . . . . . . . . . . . . . . . . . . . . . . . 32
11.1.1. ID (Identifier) . . . . . . . . . . . . . . . . . . 31 11.1.1. ID (Identifier) . . . . . . . . . . . . . . . . . . 32
11.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 31 11.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 32
11.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 31 11.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 32
11.1.4. Description . . . . . . . . . . . . . . . . . . . . 31 11.1.4. Description . . . . . . . . . . . . . . . . . . . . 32
11.1.5. Change Controller . . . . . . . . . . . . . . . . . 31 11.1.5. Change Controller . . . . . . . . . . . . . . . . . 32
11.1.6. Version (of Registry Format) . . . . . . . . . . . . 31 11.1.6. Version (of Registry Format) . . . . . . . . . . . . 32
11.2. Metric Definition . . . . . . . . . . . . . . . . . . . 31 11.2. Metric Definition . . . . . . . . . . . . . . . . . . . 32
11.2.1. Reference Definition . . . . . . . . . . . . . . . . 31 11.2.1. Reference Definition . . . . . . . . . . . . . . . . 32
11.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 32 11.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 32
11.3. Method of Measurement . . . . . . . . . . . . . . . . . 32 11.3. Method of Measurement . . . . . . . . . . . . . . . . . 33
11.3.1. Reference Method . . . . . . . . . . . . . . . . . . 32 11.3.1. Reference Method . . . . . . . . . . . . . . . . . . 33
11.3.2. Packet Stream Generation . . . . . . . . . . . . . . 32 11.3.2. Packet Stream Generation . . . . . . . . . . . . . . 33
11.3.3. Traffic Filtering (observation) Details . . . . . . 32 11.3.3. Traffic Filtering (observation) Details . . . . . . 33
11.3.4. Sampling Distribution . . . . . . . . . . . . . . . 32 11.3.4. Sampling Distribution . . . . . . . . . . . . . . . 33
11.3.5. Run-time Parameters and Data Format . . . . . . . . 32 11.3.5. Run-time Parameters and Data Format . . . . . . . . 33
11.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . 32 11.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . 33
11.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 33 11.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 33 11.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 34
11.4.2. Reference Definition . . . . . . . . . . . . . . . . 33 11.4.2. Reference Definition . . . . . . . . . . . . . . . . 34
11.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 33 11.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 34
11.4.4. Calibration . . . . . . . . . . . . . . . . . . . . 33 11.4.4. Calibration . . . . . . . . . . . . . . . . . . . . 34
11.5. Administrative items . . . . . . . . . . . . . . . . . . 33 11.5. Administrative items . . . . . . . . . . . . . . . . . . 34
11.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 33 11.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 34
11.5.2. Requestor . . . . . . . . . . . . . . . . . . . . . 33 11.5.2. Requester . . . . . . . . . . . . . . . . . . . . . 34
11.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 33 11.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 34
11.5.4. Revision Date . . . . . . . . . . . . . . . . . . . 33 11.5.4. Revision Date . . . . . . . . . . . . . . . . . . . 34
11.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 33 11.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 34
12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 34 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 35
13.1. Normative References . . . . . . . . . . . . . . . . . . 34 13.1. Normative References . . . . . . . . . . . . . . . . . . 35
13.2. Informative References . . . . . . . . . . . . . . . . . 35 13.2. Informative References . . . . . . . . . . . . . . . . . 36
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37
1. Introduction 1. Introduction
The IETF specifies and uses Performance Metrics of protocols and The IETF specifies and uses Performance Metrics of protocols and
applications transported over its protocols. Performance metrics are applications transported over its protocols. Performance metrics are
such an important part of the operations of IETF protocols that important part of network operations using IETF protocols, and
[RFC6390] specifies guidelines for their development. [RFC6390] specifies guidelines for their development.
The definition and use of Performance Metrics in the IETF happens in The definition and use of Performance Metrics in the IETF has been
various working groups (WG), most notably: fostered in various working groups (WG), most notably:
The "IP Performance Metrics" (IPPM) WG is the WG primarily The "IP Performance Metrics" (IPPM) WG is the WG primarily
focusing on Performance Metrics definition at the IETF. focusing on Performance Metrics definition at the IETF.
The "Metric Blocks for use with RTCP's Extended Report Framework" The "Benchmarking Methodology" WG (BMWG) defines many Performance
(XRBLOCK) WG recently specified many Performance Metrics related
to "RTP Control Protocol Extended Reports (RTCP XR)" [RFC3611],
which establishes a framework to allow new information to be
conveyed in RTCP, supplementing the original report blocks defined
in "RTP: A Transport Protocol for Real-Time Applications",
[RFC3550].
The "Benchmarking Methodology" WG (BMWG) defined many Performance
Metrics for use in laboratory benchmarking of inter-networking Metrics for use in laboratory benchmarking of inter-networking
technologies. technologies.
The "Metric Blocks for use with RTCP's Extended Report Framework"
(XRBLOCK) WG (concluded) specified many Performance Metrics
related to "RTP Control Protocol Extended Reports (RTCP XR)"
[RFC3611], which establishes a framework to allow new information
to be conveyed in RTCP, supplementing the original report blocks
defined in "RTP: A Transport Protocol for Real-Time Applications",
[RFC3550].
The "IP Flow Information eXport" (IPFIX) concluded WG specified an The "IP Flow Information eXport" (IPFIX) concluded WG specified an
IANA process for new Information Elements. Some Performance IANA process for new Information Elements. Some Performance
Metrics related Information Elements are proposed on regular Metrics related Information Elements are proposed on regular
basis. basis.
The "Performance Metrics for Other Layers" (PMOL) a concluded WG, The "Performance Metrics for Other Layers" (PMOL) a concluded WG
defined some Performance Metrics related to Session Initiation defined some Performance Metrics related to Session Initiation
Protocol (SIP) voice quality [RFC6035]. Protocol (SIP) voice quality [RFC6035].
It is expected that more Performance Metrics will be defined in the It is expected that more Performance Metrics will be defined in the
future, not only IP-based metrics, but also metrics which are future, not only IP-based metrics, but also metrics which are
protocol-specific and application-specific. protocol-specific and application-specific.
Despite the importance of Performance Metrics, there are two related Despite the importance of Performance Metrics, there are two related
problems for the industry. First, ensuring that when one party problems for the industry. First, ensuring that when one party
requests another party to measure (or report or in some way act on) a requests another party to measure (or report or in some way act on) a
skipping to change at page 5, line 13 skipping to change at page 5, line 13
the IETF organizes registries is with Internet Assigned Numbers the IETF organizes registries is with Internet Assigned Numbers
Authority (IANA), and there is currently no Performance Metrics Authority (IANA), and there is currently no Performance Metrics
Registry maintained by the IANA. Registry maintained by the IANA.
This document requests that IANA create and maintain a Performance This document requests that IANA create and maintain a Performance
Metrics Registry, according to the maintenance procedures and the Metrics Registry, according to the maintenance procedures and the
Performance Metrics Registry format defined in this memo. The Performance Metrics Registry format defined in this memo. The
resulting Performance Metrics Registry is for use by the IETF and resulting Performance Metrics Registry is for use by the IETF and
others. Although the Registry formatting specifications herein are others. Although the Registry formatting specifications herein are
primarily for registry creation by IANA, any other organization that primarily for registry creation by IANA, any other organization that
wishes to create a Performance Metrics Registry MAY use the same wishes to create a performance metrics registry may use the same
formatting specifications for their purposes. The authors make no formatting specifications for their purposes. The authors make no
guarantee of the registry format's applicability to any possible set guarantee of the registry format's applicability to any possible set
of Performance Metrics envisaged by other organizations, but of Performance Metrics envisaged by other organizations, but
encourage others to apply it. In the remainder of this document, encourage others to apply it. In the remainder of this document,
unless we explicitly say otherwise, we will refer to the IANA- unless we explicitly say otherwise, we will refer to the IANA-
maintained Performance Metrics Registry as simply the Performance maintained Performance Metrics Registry as simply the Performance
Metrics Registry. Metrics Registry.
2. Terminology 2. Terminology
skipping to change at page 5, line 35 skipping to change at page 5, line 35
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
Performance Metric: A Performance Metric is a quantitative measure Performance Metric: A Performance Metric is a quantitative measure
of performance, targeted to an IETF-specified protocol or targeted of performance, targeted to an IETF-specified protocol or targeted
to an application transported over an IETF-specified protocol. to an application transported over an IETF-specified protocol.
Examples of Performance Metrics are the FTP response time for a Examples of Performance Metrics are the FTP response time for a
complete file download, the DNS response time to resolve the IP complete file download, the DNS response time to resolve the IP
address, a database logging time, etc. This definition is address(es), a database logging time, etc. This definition is
consistent with the definition of metric in [RFC2330] and broader consistent with the definition of metric in [RFC2330] and broader
than the definition of performance metric in [RFC6390]. than the definition of performance metric in [RFC6390].
Registered Performance Metric: A Registered Performance Metric is a Registered Performance Metric: A Registered Performance Metric is a
Performance Metric expressed as an entry in the Performance Performance Metric expressed as an entry in the Performance
Metrics Registry, administered by IANA. Such a performance metric Metrics Registry, administered by IANA. Such a performance metric
has met all the registry review criteria defined in this document has met all the registry review criteria defined in this document
in order to included in the registry. in order to be included in the registry.
Performance Metrics Registry: The IANA registry containing Performance Metrics Registry: The IANA registry containing
Registered Performance Metrics. Registered Performance Metrics.
Proprietary Registry: A set of metrics that are registered in a Proprietary Registry: A set of metrics that are registered in a
proprietary registry, as opposed to Performance Metrics Registry. proprietary registry, as opposed to Performance Metrics Registry.
Performance Metrics Experts: The Performance Metrics Experts is a Performance Metrics Experts: The Performance Metrics Experts is a
group of designated experts [RFC8126] selected by the IESG to group of designated experts [RFC8126] selected by the IESG to
validate the Performance Metrics before updating the Performance validate the Performance Metrics before updating the Performance
Metrics Registry. The Performance Metrics Experts work closely Metrics Registry. The Performance Metrics Experts work closely
with IANA. with IANA.
Parameter: A Parameter is an input factor defined as a variable in Parameter: A Parameter is an input factor defined as a variable in
the definition of a Performance Metric. A Parameter is a the definition of a Performance Metric. A Parameter is a
numerical or other specified factor forming one of a set that numerical or other specified factor forming one of a set that
defines a metric or sets the conditions of its operation. All defines a metric or sets the conditions of its operation. All
Parameters must be known to measure using a metric and interpret Parameters must be known in order to make a measurement using a
the results. There are two types of Parameters: Fixed and Run- metric and interpret the results. There are two types of
time parameters. For the Fixed Parameters, the value of the Parameters: Fixed and Run-time parameters. For the Fixed
variable is specified in the Performance Metrics Registry entry Parameters, the value of the variable is specified in the
and different Fixed Parameter values results in different Performance Metrics Registry entry and different Fixed Parameter
Registered Performance Metrics. For the Run-time Parameters, the values results in different Registered Performance Metrics. For
value of the variable is defined when the metric measurement the Run-time Parameters, the value of the variable is defined when
method is executed and a given Registered Performance Metric the metric measurement method is executed and a given Registered
supports multiple values for the parameter. Although Run-time Performance Metric supports multiple values for the parameter.
Parameters do not change the fundamental nature of the Performance Although Run-time Parameters do not change the fundamental nature
Metric's definition, some have substantial influence on the of the Performance Metric's definition, some have substantial
network property being assessed and interpretation of the results. influence on the network property being assessed and
interpretation of the results.
Note: Consider the case of packet loss in the following two Note: Consider the case of packet loss in the following two
Active Measurement Method cases. The first case is packet loss Active Measurement Method cases. The first case is packet loss
as background loss where the Run-time Parameter set includes a as background loss where the Run-time Parameter set includes a
very sparse Poisson stream, and only characterizes the times very sparse Poisson stream, and only characterizes the times
when packets were lost. Actual user streams likely see much when packets were lost. Actual user streams likely see much
higher loss at these times, due to tail drop or radio errors. higher loss at these times, due to tail drop or radio errors.
The second case is packet loss as inverse of throughput where The second case is packet loss as inverse of throughput where
the Run-time Parameter set includes a very dense, bursty the Run-time Parameter set includes a very dense, bursty
stream, and characterizes the loss experienced by a stream that stream, and characterizes the loss experienced by a stream that
skipping to change at page 7, line 31 skipping to change at page 7, line 31
3. Scope 3. Scope
This document is intended for two different audiences: This document is intended for two different audiences:
1. For those defining new Registered Performance Metrics, it 1. For those defining new Registered Performance Metrics, it
provides specifications and best practices to be used in deciding provides specifications and best practices to be used in deciding
which Registered Performance Metrics are useful for a measurement which Registered Performance Metrics are useful for a measurement
study, instructions for writing the text for each column of the study, instructions for writing the text for each column of the
Registered Performance Metrics, and information on the supporting Registered Performance Metrics, and information on the supporting
documentation required for the new Performance Metrics Registry documentation required for the new Performance Metrics Registry
entry (up to and including the publication of one or more RFCs or entry (up to and including the publication of one or more
I-Ds describing it). immutable documents such as an RFC).
2. For the appointed Performance Metrics Experts and for IANA 2. For the appointed Performance Metrics Experts and for IANA
personnel administering the new IANA Performance Metrics personnel administering the new IANA Performance Metrics
Registry, it defines a set of acceptance criteria against which Registry, it defines a set of acceptance criteria against which
these proposed Registered Performance Metrics should be these proposed Registered Performance Metrics should be
evaluated. evaluated.
In addition, this document may be useful for other organizations who In addition, this document may be useful for other organizations who
are defining a Performance Metric registry of their own, and may re- are defining a Performance Metric registry of their own, and may re-
use the features of the Performance Metrics Registry defined in this use the features of the Performance Metrics Registry defined in this
document. document.
This Performance Metrics Registry is applicable to Performance This Performance Metrics Registry is applicable to Performance
Metrics issued from Active Measurement, Passive Measurement, and any Metrics issued from Active Measurement, Passive Measurement, and any
other form of Performance Metric. This registry is designed to other form of Performance Metric. This registry is designed to
encompass Performance Metrics developed throughout the IETF and encompass Performance Metrics developed throughout the IETF and
especially for the technologies specified in the following working especially for the technologies specified in the following working
groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes an groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes a
prior attempt to set up a Performance Metrics Registry, and the prior attempt to set up a Performance Metrics Registry, and the
reasons why this design was inadequate [RFC6248]. Finally, this reasons why this design was inadequate [RFC6248]. Finally, this
document gives a set of guidelines for requesters and expert document gives a set of guidelines for requesters and expert
reviewers of candidate Registered Performance Metrics. reviewers of candidate Registered Performance Metrics.
This document makes no attempt to populate the Performance Metrics This document makes no attempt to populate the Performance Metrics
Registry with initial entries. Registry with initial entries; the related memo
[I-D.ietf-ippm-initial-registry] proposes the initial set of regsitry
Based on [RFC8126] Section 4.3, this document is processed as Best entries.
Current Practice (BCP) [RFC2026].
4. Motivation for a Performance Metrics Registry 4. Motivation for a Performance Metrics Registry
In this section, we detail several motivations for the Performance In this section, we detail several motivations for the Performance
Metrics Registry. Metrics Registry.
4.1. Interoperability 4.1. Interoperability
As any IETF registry, the primary use for a registry is to manage a As with any IETF registry, the primary intention is to manage
registry for its use within one or more protocols. In the particular registration of identifiers for use within one or more protocols. In
case of the Performance Metrics Registry, there are two types of the particular case of the Performance Metrics Registry, there are
protocols that will use the Performance Metrics in the Performance two types of protocols that will use the Performance Metrics in the
Metrics Registry during their operation (by referring to the Index Performance Metrics Registry during their operation (by referring to
values): the Index values):
o Control protocol: This type of protocol used to allow one entity o Control protocol: This type of protocol used to allow one entity
to request another entity to perform a measurement using a to request another entity to perform a measurement using a
specific metric defined by the Performance Metrics Registry. One specific metric defined by the Performance Metrics Registry. One
particular example is the LMAP framework [RFC7594]. Using the particular example is the LMAP framework [RFC7594]. Using the
LMAP terminology, the Performance Metrics Registry is used in the LMAP terminology, the Performance Metrics Registry is used in the
LMAP Control protocol to allow a Controller to request a LMAP Control protocol to allow a Controller to request a
measurement task to one or more Measurement Agents. In order to measurement task to one or more Measurement Agents. In order to
enable this use case, the entries of the Performance Metrics enable this use case, the entries of the Performance Metrics
Registry must be sufficiently defined to allow a Measurement Agent Registry must be sufficiently defined to allow a Measurement Agent
skipping to change at page 9, line 19 skipping to change at page 9, line 18
Registries' entries. Registries' entries.
4.2. Single point of reference for Performance Metrics 4.2. Single point of reference for Performance Metrics
A Performance Metrics Registry serves as a single point of reference A Performance Metrics Registry serves as a single point of reference
for Performance Metrics defined in different working groups in the for Performance Metrics defined in different working groups in the
IETF. As we mentioned earlier, there are several WGs that define IETF. As we mentioned earlier, there are several WGs that define
Performance Metrics in the IETF and it is hard to keep track of all Performance Metrics in the IETF and it is hard to keep track of all
them. This results in multiple definitions of similar Performance them. This results in multiple definitions of similar Performance
Metrics that attempt to measure the same phenomena but in slightly Metrics that attempt to measure the same phenomena but in slightly
different (and incompatible) ways. Having a registry would allow different (and incompatible) ways. Having a registry would allow the
both the IETF community and external people to have a single list of IETF community and others to have a single list of relevant
relevant Performance Metrics defined by the IETF (and others, where Performance Metrics defined by the IETF (and others, where
appropriate). The single list is also an essential aspect of appropriate). The single list is also an essential aspect of
communication about Performance Metrics, where different entities communication about Performance Metrics, where different entities
that request measurements, execute measurements, and report the that request measurements, execute measurements, and report the
results can benefit from a common understanding of the referenced results can benefit from a common understanding of the referenced
Performance Metric. Performance Metric.
4.3. Side benefits 4.3. Side benefits
There are a couple of side benefits of having such a registry. There are a couple of side benefits of having such a registry.
First, the Performance Metrics Registry could serve as an inventory First, the Performance Metrics Registry could serve as an inventory
skipping to change at page 10, line 7 skipping to change at page 10, line 7
SHOULD be noted, with a reference to the test results. SHOULD be noted, with a reference to the test results.
5. Criteria for Performance Metrics Registration 5. Criteria for Performance Metrics Registration
It is neither possible nor desirable to populate the Performance It is neither possible nor desirable to populate the Performance
Metrics Registry with all combinations of Parameters of all Metrics Registry with all combinations of Parameters of all
Performance Metrics. The Registered Performance Metrics SHOULD be: Performance Metrics. The Registered Performance Metrics SHOULD be:
1. interpretable by the user. 1. interpretable by the user.
2. implementable by the software designer, 2. implementable by the software or hardware designer,
3. deployable by network operators, 3. deployable by network operators,
4. accurate, for interoperability and deployment across vendors, 4. accurate in terms of producing equivalent results, and for
interoperability and deployment across vendors,
5. Operationally useful, so that it has significant industry 5. Operationally useful, so that it has significant industry
interest and/or has seen deployment, interest and/or has seen deployment,
6. Sufficiently tightly defined, so that different values for the 6. Sufficiently tightly defined, so that different values for the
Run-time Parameters does not change the fundamental nature of the Run-time Parameters does not change the fundamental nature of the
measurement, nor change the practicality of its implementation. measurement, nor change the practicality of its implementation.
In essence, there needs to be evidence that a candidate Registered In essence, there needs to be evidence that a candidate Registered
Performance Metric has significant industry interest, or has seen Performance Metric has significant industry interest, or has seen
skipping to change at page 11, line 13 skipping to change at page 11, line 14
idea is that entries in the Performance Metrics Registry stem from idea is that entries in the Performance Metrics Registry stem from
different measurement methods which require input (Run-time) different measurement methods which require input (Run-time)
parameters to set factors like source and destination addresses parameters to set factors like source and destination addresses
(which do not change the fundamental nature of the measurement). The (which do not change the fundamental nature of the measurement). The
downside of this approach is that it could result in a large number downside of this approach is that it could result in a large number
of entries in the Performance Metrics Registry. There is agreement of entries in the Performance Metrics Registry. There is agreement
that less is more in this context - it is better to have a reduced that less is more in this context - it is better to have a reduced
set of useful metrics rather than a large set of metrics, some with set of useful metrics rather than a large set of metrics, some with
with questionable usefulness. with questionable usefulness.
6.1. Why this Attempt Will Succeed 6.1. Why this Attempt Should Succeed
As mentioned in the previous section, one of the main issues with the As mentioned in the previous section, one of the main issues with the
previous registry was that the metrics contained in the registry were previous registry was that the metrics contained in the registry were
too generic to be useful. This document specifies stricter criteria too generic to be useful. This document specifies stricter criteria
for performance metric registration (see section 5), and imposes a for performance metric registration (see section 5), and imposes a
group of Performance Metrics Experts that will provide guidelines to group of Performance Metrics Experts that will provide guidelines to
assess if a Performance Metric is properly specified. assess if a Performance Metric is properly specified.
Another key difference between this attempt and the previous one is Another key difference between this attempt and the previous one is
that in this case there is at least one clear user for the that in this case there is at least one clear user for the
skipping to change at page 11, line 43 skipping to change at page 11, line 44
properly specified if they are defined well-enough so that it is properly specified if they are defined well-enough so that it is
possible (and practical) to implement them in the measurement agent. possible (and practical) to implement them in the measurement agent.
This was the failure of the previous attempt: a registry entry with This was the failure of the previous attempt: a registry entry with
an undefined Type-P (section 13 of RFC 2330 [RFC2330]) allows an undefined Type-P (section 13 of RFC 2330 [RFC2330]) allows
implementation to be ambiguous. implementation to be ambiguous.
7. Definition of the Performance Metric Registry 7. Definition of the Performance Metric Registry
This Performance Metrics Registry is applicable to Performance This Performance Metrics Registry is applicable to Performance
Metrics used for Active Measurement, Passive Measurement, and any Metrics used for Active Measurement, Passive Measurement, and any
other form of Performance Metric. Each category of measurement has other form of Performance Measurement. Each category of measurement
unique properties, so some of the columns defined below are not has unique properties, so some of the columns defined below are not
applicable for a given metric category. In this case, the column(s) applicable for a given metric category. In this case, the column(s)
SHOULD be populated with the "NA" value (Non Applicable). However, SHOULD be populated with the "NA" value (Non Applicable). However,
the "NA" value MUST NOT be used by any metric in the following the "NA" value MUST NOT be used by any metric in the following
columns: Identifier, Name, URI, Status, Requester, Revision, Revision columns: Identifier, Name, URI, Status, Requester, Revision, Revision
Date, Description. In the future, a new category of metrics could Date, Description. In the future, a new category of metrics could
require additional columns, and adding new columns is a recognized require additional columns, and adding new columns is a recognized
form of registry extension. The specification defining the new form of registry extension. The specification defining the new
column(s) MUST give guidelines to populate the new column(s) for column(s) MUST give guidelines to populate the new column(s) for
existing entries (in general). existing entries (in general).
The columns of the Performance Metrics Registry are defined below. The columns of the Performance Metrics Registry are defined below.
The columns are grouped into "Categories" to facilitate the use of The columns are grouped into "Categories" to facilitate the use of
the registry. Categories are described at the 7.x heading level, and the registry. Categories are described at the 7.x heading level, and
columns are at the 7.x.y heading level. The Figure below illustrates columns are at the 7.x.y heading level. The Figure below illustrates
this organization. An entry (row) therefore gives a complete this organization. An entry (row) therefore gives a complete
description of a Registered Performance Metric. description of a Registered Performance Metric.
Each column serves as a check-list item and helps to avoid omissions Each column serves as a check-list item and helps to avoid omissions
during registration and expert review. during registration and expert review.
Registry Categories and Columns, shown as =======================================================================
Legend:
Category Registry Categories and Columns are shown below as:
Column | Column | Category
------------------...
Column | Column |...
=======================================================================
Summary Summary
------------------------------------------------------------------------ ------------------------------------------------------------------------
Identifier | Name | URIs | Desc. | Reference | Change Controller | Ver | Identifier | Name | URIs | Desc. | Reference | Change Controller | Ver |
Metric Definition Metric Definition
----------------------------------------- -----------------------------------------
Reference Definition | Fixed Parameters | Reference Definition | Fixed Parameters |
Method of Measurement Method of Measurement
--------------------------------------------------------------------- ---------------------------------------------------------------------
skipping to change at page 13, line 43 skipping to change at page 13, line 43
the names to determine if the metric they want to measure has already the names to determine if the metric they want to measure has already
been registered, or if a similar entry is available as a basis for been registered, or if a similar entry is available as a basis for
creating a new entry. creating a new entry.
Names are composed of the following elements, separated by an Names are composed of the following elements, separated by an
underscore character "_": underscore character "_":
MetricType_Method_SubTypeMethod_... Spec_Units_Output MetricType_Method_SubTypeMethod_... Spec_Units_Output
o MetricType: a combination of the directional properties and the o MetricType: a combination of the directional properties and the
metric measured, such as: metric measured, such as and not limited to:
RTDelay (Round Trip Delay) RTDelay (Round Trip Delay)
RTDNS (Response Time Domain Name Service) RTDNS (Response Time Domain Name Service)
RLDNS (Response Loss Domain Name Service) RLDNS (Response Loss Domain Name Service)
OWDelay (One Way Delay) OWDelay (One Way Delay)
RTLoss (Round Trip Loss) RTLoss (Round Trip Loss)
skipping to change at page 14, line 24 skipping to change at page 14, line 24
OWDuplic (One Way Packet Duplication) OWDuplic (One Way Packet Duplication)
OWBTC (One Way Bulk Transport Capacity) OWBTC (One Way Bulk Transport Capacity)
OWMBM (One Way Model Based Metric) OWMBM (One Way Model Based Metric)
SPMonitor (Single Point Monitor) SPMonitor (Single Point Monitor)
MPMonitor (Multi-Point Monitor) MPMonitor (Multi-Point Monitor)
o Method: One of the methods defined in [RFC7799], such as: o Method: One of the methods defined in [RFC7799], such as and not
limited to:
Active (depends on a dedicated measurement packet stream and Active (depends on a dedicated measurement packet stream and
observations of the stream) observations of the stream)
Passive (depends *solely* on observation of one or more Passive (depends *solely* on observation of one or more
existing packet streams) existing packet streams)
HybridType1 (obervations on one stream that combine both active HybridType1 (observations on one stream that combine both
and passive methods) active and passive methods)
HybridType2 (obervations on two or more streams that combine HybridType2 (observations on two or more streams that combine
both active and passive methods) both active and passive methods)
Spatial (Spatial Metric of RFC5644) Spatial (Spatial Metric of RFC5644)
o SubTypeMethod: One or more sub-types to further describe the o SubTypeMethod: One or more sub-types to further describe the
features of the entry, such as: features of the entry, such as and not limited to:
ICMP (Internet Control Message Protocol) ICMP (Internet Control Message Protocol)
IP (Internet Protocol) IP (Internet Protocol)
DSCPxx (where xx is replaced by a Diffserv code point) DSCPxx (where xx is replaced by a Diffserv code point)
UDP (User Datagram Protocol) UDP (User Datagram Protocol)
TCP (Transport Control Protocol) TCP (Transport Control Protocol)
QUIC (QUIC transport protocol) QUIC (QUIC transport protocol)
HS (Hand-Shake, such as TCP's 3-way HS) HS (Hand-Shake, such as TCP's 3-way HS)
Poisson (Packet generation using Poisson distribution) Poisson (Packet generation using Poisson distribution)
Periodic (Periodic packet generation) Periodic (Periodic packet generation)
SendOnRcv (Sender keeps one packet in-transit by sending when SendOnRcv (Sender keeps one packet in-transit by sending when
previous packet arrives) previous packet arrives)
skipping to change at page 15, line 26 skipping to change at page 15, line 28
of octets in the Payload)) of octets in the Payload))
SustainedBurst (Capacity test, worst case) SustainedBurst (Capacity test, worst case)
StandingQueue (test of bottleneck queue behavior) StandingQueue (test of bottleneck queue behavior)
SubTypeMethod values are separated by a hyphen "-" character, SubTypeMethod values are separated by a hyphen "-" character,
which indicates that they belong to this element, and that their which indicates that they belong to this element, and that their
order is unimportant when considering name uniqueness. order is unimportant when considering name uniqueness.
o Spec: RFC number and major section number that specifies this o Spec: an immutable document, for RFCs, the RFC number and major
Registry entry in the form RFCXXXXsecY, such as RFC7799sec3. section number that specifies this Registry entry in the form
Note: the RFC number is not the Primary Reference specification RFCXXXXsecY, such as RFC7799sec3. Note: the RFC number is not the
for the metric definition, such as [RFC7679] for One-way Delay; it Primary Reference specification for the metric definition, such as
will contain the placeholder "RFCXXXXsecY" until the RFC number is [RFC7679] for One-way Delay; it will contain the placeholder
assigned to the specifying document, and would remain blank in "RFCXXXXsecY" until the RFC number is assigned to the specifying
private registry entries without a corresponding RFC. document, and would remain blank in private registry entries
Anticipating the "RFC10K" problem, the number of the RFC continues without a corresponding RFC. Anticipating the "RFC10K" problem,
to replace RFCXXXX regardless of the number of digits in the RFC the number of the RFC continues to replace RFCXXXX regardless of
number. Anticipating Registry Entries from other standards the number of digits in the RFC number. Anticipating Registry
bodies, the form of this Name Element MUST be proposed and Entries from other standards bodies, the form of this Name Element
reviewed for consistency and uniqueness by the Expert Reviewer. MUST be proposed and reviewed for consistency and uniqueness by
the Expert Reviewer.
o Units: The units of measurement for the output, such as: o Units: The units of measurement for the output, such as and not
limited to:
Seconds Seconds
Ratio (unitless) Ratio (unitless)
Percent (value multiplied by 100) Percent (value multiplied by 100)
Logical (1 or 0) Logical (1 or 0)
Packets Packets
BPS (Bits per Second) BPS (Bits per Second)
PPS (Packets per Second) PPS (Packets per Second)
EventTotal (for unit-less counts) EventTotal (for unit-less counts)
Multiple (more than one type of unit) Multiple (more than one type of unit)
Enumerated (a list of outcomes) Enumerated (a list of outcomes)
Unitless Unitless
o Output: The type of output resulting from measurement, such as: o Output: The type of output resulting from measurement, such as and
not limited to:
Singleton Singleton
Raw (multiple Singletons) Raw (multiple Singletons)
Count Count
Minimum Minimum
Maximum Maximum
skipping to change at page 17, line 17 skipping to change at page 17, line 21
conflicts (further considerations are described in section 10). conflicts (further considerations are described in section 10).
Private registry entries usually have no specifying RFC, thus the Private registry entries usually have no specifying RFC, thus the
Spec: element has no clear interpretation. Spec: element has no clear interpretation.
7.1.3. URIs 7.1.3. URIs
The URIs column MUST contain a URL [RFC3986] that uniquely identifies The URIs column MUST contain a URL [RFC3986] that uniquely identifies
and locates the metric entry so it is accessible through the and locates the metric entry so it is accessible through the
Internet. The URL points to a file containing all the human-readable Internet. The URL points to a file containing all the human-readable
information for one registry entry. The URL SHALL reference a target information for one registry entry. The URL SHALL reference a target
file that is HTML-formated and contains URLs to referenced sections file that is HTML-formatted and contains URLs to referenced sections
of HTML-ized RFCs. These target files for different entries can be of HTML-ized RFCs, or other reference specifications. These target
more easily edited and re-used when preparing new entries. The exact files for different entries can be more easily edited and re-used
form of the URL for each target file will be determined by IANA and when preparing new entries. The exact form of the URL for each
reside on "iana.org". The major sections of target file will be determined by IANA and reside on "iana.org". The
[I-D.ietf-ippm-initial-registry] provide an example of a target file major sections of [I-D.ietf-ippm-initial-registry] provide an example
in HTML form (sections 4 and higher). of a target file in HTML form (sections 4 and higher).
7.1.4. Description 7.1.4. Description
A Registered Performance Metric description is a written A Registered Performance Metric description is a written
representation of a particular Performance Metrics Registry entry. representation of a particular Performance Metrics Registry entry.
It supplements the Registered Performance Metric name to help It supplements the Registered Performance Metric name to help
Performance Metrics Registry users select relevant Registered Performance Metrics Registry users select relevant Registered
Performance Metrics. Performance Metrics.
7.1.5. Reference 7.1.5. Reference
skipping to change at page 17, line 50 skipping to change at page 18, line 10
This entry names the entity responsible for approving revisions to This entry names the entity responsible for approving revisions to
the registry entry, and SHALL provide contact information (for an the registry entry, and SHALL provide contact information (for an
individual, where appropriate). individual, where appropriate).
7.1.7. Version (of Registry Format) 7.1.7. Version (of Registry Format)
This entry gives the version number for the registry format used. This entry gives the version number for the registry format used.
Formats complying with this memo MUST use 1.0. The version number Formats complying with this memo MUST use 1.0. The version number
SHALL NOT change unless a new RFC is published that changes the SHALL NOT change unless a new RFC is published that changes the
registry format. registry format. The version number of registry entries SHALL NOT
change unless the registry entry is updated (following procedures in
section 8).
7.2. Metric Definition Category 7.2. Metric Definition Category
This category includes columns to prompt all necessary details This category includes columns to prompt all necessary details
related to the metric definition, including the RFC reference and related to the metric definition, including the immutable document
values of input factors, called fixed parameters, which are left open reference and values of input factors, called fixed parameters, which
in the RFC but have a particular value defined by the performance are left open in the immutable document, but have a particular value
metric. defined by the performance metric.
7.2.1. Reference Definition 7.2.1. Reference Definition
This entry provides a reference (or references) to the relevant This entry provides a reference (or references) to the relevant
section(s) of the document(s) that define the metric, as well as any section(s) of the document(s) that define the metric, as well as any
supplemental information needed to ensure an unambiguous definition supplemental information needed to ensure an unambiguous definition
for implementations. The reference needs to be an immutable for implementations. The reference needs to be an immutable
document, such as an RFC; for other standards bodies, it is likely to document, such as an RFC; for other standards bodies, it is likely to
be necessary to reference a specific, dated version of a be necessary to reference a specific, dated version of a
specification. specification.
skipping to change at page 19, line 18 skipping to change at page 19, line 26
the immutable document(s) and any supplemental information needed to the immutable document(s) and any supplemental information needed to
ensure an unambiguous method for implementations. ensure an unambiguous method for implementations.
7.3.1. Reference Method 7.3.1. Reference Method
This entry provides references to relevant sections of immutable This entry provides references to relevant sections of immutable
documents, such as RFC(s) (for other standards bodies, it is likely documents, such as RFC(s) (for other standards bodies, it is likely
to be necessary to reference a specific, dated version of a to be necessary to reference a specific, dated version of a
specification) describing the method of measurement, as well as any specification) describing the method of measurement, as well as any
supplemental information needed to ensure unambiguous interpretation supplemental information needed to ensure unambiguous interpretation
for implementations referring to the RFC text. for implementations referring to the immutable document text.
Specifically, this section should include pointers to pseudocode or Specifically, this section should include pointers to pseudocode or
actual code that could be used for an unambigious implementation. actual code that could be used for an unambiguous implementation.
7.3.2. Packet Stream Generation 7.3.2. Packet Stream Generation
This column applies to Performance Metrics that generate traffic as This column applies to Performance Metrics that generate traffic as
part of their Measurement Method, including but not necessarily part of their Measurement Method, including but not necessarily
limited to Active metrics. The generated traffic is referred as a limited to Active metrics. The generated traffic is referred as a
stream and this column describes its characteristics. stream and this column describes its characteristics.
Each entry for this column contains the following information: Each entry for this column contains the following information:
o Value: The name of the packet stream scheduling discipline o Value: The name of the packet stream scheduling discipline
o Reference: the specification where the parameters of the stream o Reference: the specification where the parameters of the stream
are defined are defined
The packet generation stream may require parameters such as the The packet generation stream may require parameters such as the
average packet rate and distribution truncation value for streams average packet rate and distribution truncation value for streams
with Poisson-distributed inter-packet sending times. In case such with Poisson-distributed inter-packet sending times. In case such
parameters are needed, they should be included either in the Fixed parameters are needed, they should be included either in the Fixed
parameter column or in the run time parameter column, depending on parameter column or in the run time parameter column, depending on
wether they will be fixed or will be an input for the metric. whether they will be fixed or will be an input for the metric.
The simplest example of stream specification is Singleton scheduling The simplest example of stream specification is Singleton scheduling
(see [RFC2330]), where a single atomic measurement is conducted. (see [RFC2330]), where a single atomic measurement is conducted.
Each atomic measurement could consist of sending a single packet Each atomic measurement could consist of sending a single packet
(such as a DNS request) or sending several packets (for example, to (such as a DNS request) or sending several packets (for example, to
request a webpage). Other streams support a series of atomic request a webpage). Other streams support a series of atomic
measurements in a "sample", with a schedule defining the timing measurements in a "sample", with a schedule defining the timing
between each transmitted packet and subsequent measurement. between each transmitted packet and subsequent measurement.
Principally, two different streams are used in IPPM metrics, Poisson Principally, two different streams are used in IPPM metrics, Poisson
distributed as described in [RFC2330] and Periodic as described in distributed as described in [RFC2330] and Periodic as described in
skipping to change at page 21, line 37 skipping to change at page 21, line 47
hanging indent style is preferred, and the names and definitions that hanging indent style is preferred, and the names and definitions that
do not appear in the Reference Method Specification MUST appear in do not appear in the Reference Method Specification MUST appear in
this column. this column.
A Data Format for each Run-time Parameter MUST be specified in this A Data Format for each Run-time Parameter MUST be specified in this
column, to simplify the control and implementation of measurement column, to simplify the control and implementation of measurement
devices. For example, parameters that include an IPv4 address can be devices. For example, parameters that include an IPv4 address can be
encoded as a 32 bit integer (i.e. binary base64 encoded value) or ip- encoded as a 32 bit integer (i.e. binary base64 encoded value) or ip-
address as defined in [RFC6991]. The actual encoding(s) used must be address as defined in [RFC6991]. The actual encoding(s) used must be
explicitly defined for each Run-time parameter. IPv6 addresses and explicitly defined for each Run-time parameter. IPv6 addresses and
options MUST be accomodated, allowing Registered Metrics to be used options MUST be accommodated, allowing Registered Metrics to be used
in either address family. in either address family.
Examples of Run-time Parameters include IP addresses, measurement Examples of Run-time Parameters include IP addresses, measurement
point designations, start times and end times for measurement, and point designations, start times and end times for measurement, and
other information essential to the method of measurement. other information essential to the method of measurement.
7.3.6. Role 7.3.6. Role
In some methods of measurement, there may be several roles defined, In some methods of measurement, there may be several roles defined,
e.g., for a one-way packet delay active measurement there is one e.g., for a one-way packet delay active measurement there is one
skipping to change at page 23, line 17 skipping to change at page 23, line 27
units for each measured value. units for each measured value.
7.4.4. Calibration 7.4.4. Calibration
Some specifications for Methods of Measurement include the Some specifications for Methods of Measurement include the
possibility to perform an error calibration. Section 3.7.3 of possibility to perform an error calibration. Section 3.7.3 of
[RFC7679] is one example. In the registry entry, this field will [RFC7679] is one example. In the registry entry, this field will
identify a method of calibration for the metric, and when available, identify a method of calibration for the metric, and when available,
the measurement system SHOULD perform the calibration when requested the measurement system SHOULD perform the calibration when requested
and produce the output with an indication that it is the result of a and produce the output with an indication that it is the result of a
calbration method. In-situ calibration could be enabled with an calibration method. In-situ calibration could be enabled with an
internal loopback that includes as much of the measurement system as internal loopback that includes as much of the measurement system as
possible, performs address manipulation as needed, and provides some possible, performs address manipulation as needed, and provides some
form of isolation (e.g., deterministic delay) to avoid send-receive form of isolation (e.g., deterministic delay) to avoid send-receive
interface contention. Some portion of the random and systematic interface contention. Some portion of the random and systematic
error can be characterized this way. error can be characterized this way.
For one-way delay measurements, the error calibration must include an For one-way delay measurements, the error calibration must include an
assessment of the internal clock synchronization with its external assessment of the internal clock synchronization with its external
reference (this internal clock is supplying timestamps for reference (this internal clock is supplying timestamps for
measurement). In practice, the time offsets of clocks at both the measurement). In practice, the time offsets of clocks at both the
skipping to change at page 24, line 23 skipping to change at page 24, line 36
The date of acceptance or the most recent revision for the Registered The date of acceptance or the most recent revision for the Registered
Performance Metric. The date SHALL be determined by IANA and the Performance Metric. The date SHALL be determined by IANA and the
reviewing Performance Metrics Expert. reviewing Performance Metrics Expert.
7.6. Comments and Remarks 7.6. Comments and Remarks
Besides providing additional details which do not appear in other Besides providing additional details which do not appear in other
categories, this open Category (single column) allows for unforeseen categories, this open Category (single column) allows for unforeseen
issues to be addressed by simply updating this informational entry. issues to be addressed by simply updating this informational entry.
8. The Life-Cycle of Registered Performance Metrics 8. Processes for Managing the Performance Metric Registry Group
Once a Performance Metric or set of Performance Metrics has been Once a Performance Metric or set of Performance Metrics has been
identified for a given application, candidate Performance Metrics identified for a given application, candidate Performance Metrics
Registry entry specifications prepared in accordance with Section 7 Registry entry specifications prepared in accordance with Section 7
should be submitted to IANA to follow the process for review by the should be submitted to IANA to follow the process for review by the
Performance Metric Experts, as defined below. This process is also Performance Metric Experts, as defined below. This process is also
used for other changes to the Performance Metrics Registry, such as used for other changes to the Performance Metrics Registry, such as
deprecation or revision, as described later in this section. deprecation or revision, as described later in this section.
It is desirable that the author(s) of a candidate Performance Metrics It is desirable that the author(s) of a candidate Performance Metrics
Registry entry seek review in the relevant IETF working group, or Registry entry seek review in the relevant IETF working group, or
offer the opportunity for review on the working group mailing list. offer the opportunity for review on the working group mailing list.
8.1. Adding new Performance Metrics to the Performance Metrics Registry 8.1. Adding new Performance Metrics to the Performance Metrics Registry
Requests to add Registered Performance Metrics in the Performance Requests to add Registered Performance Metrics in the Performance
Metrics Registry SHALL be submitted to IANA, which forwards the Metrics Registry SHALL be submitted to IANA, which forwards the
request to a designated group of experts (Performance Metric Experts) request to a designated group of experts (Performance Metric Experts)
appointed by the IESG; these are the reviewers called for by the appointed by the IESG; these are the reviewers called for by the
Expert Review [RFC8126]policy defined for the Performance Metrics Specification Required [RFC8126] policy defined for the Performance
Registry. The Performance Metric Experts review the request for such Metrics Registry. The Performance Metric Experts review the request
things as compliance with this document, compliance with other for such things as compliance with this document, compliance with
applicable Performance Metric-related RFCs, and consistency with the other applicable Performance Metric-related RFCs, and consistency
currently defined set of Registered Performance Metrics. with the currently defined set of Registered Performance Metrics.
Submission to IANA MAY be during IESG review (leading to IETF Submission to IANA may be during IESG review (leading to IETF
Standards Action), where an Internet Draft proposes one or more Standards Action), where an Internet Draft proposes one or more
Registered Performance Metrics to be added to the Performance Metrics Registered Performance Metrics to be added to the Performance Metrics
Registry, including the text of the proposed Registered Performance Registry, including the text of the proposed Registered Performance
Metric(s). Metric(s).
If the proposed registry entry is defined in an RFC but is not yet If an RFC-to-be includes a Performance Metric and a proposed
widely deployed, there SHOULD be a statement in the RFC that says the Performance Metrics Registry entry, but the IANA and Performance
proposed registry entry is not ready for registration, and use SHOULD Metric Expert review determines that one or more of the Section 5
employ a private/experimental ID. It is the responsibility of the criteria have not been met, then the proposed Performance Metrics
document authors to submit the request to IANA when the proposed Registry entry MUST be removed from the text. When the RFC-to-be
registry entry is ready for official registration. authors are ready to show evidence of meeting the criteria in section
5, they SHOULD re-submit the proposed Performance Metrics Registry
entry to IANA to be evaluated in consultation with the Performance
Metric Experts for registration at that time.
Authors of proposed Registered Performance Metrics SHOULD review Authors of proposed Registered Performance Metrics SHOULD review
compliance with the specifications in this document to check their compliance with the specifications in this document to check their
submissions before sending them to IANA. submissions before sending them to IANA.
At least one Performance Metric Expert should endeavor to complete At least one Performance Metric Expert should endeavor to complete
referred reviews in a timely manner. If the request is acceptable, referred reviews in a timely manner. If the request is acceptable,
the Performance Metric Experts signify their approval to IANA, and the Performance Metric Experts signify their approval to IANA, and
IANA updates the Performance Metrics Registry. If the request is not IANA updates the Performance Metrics Registry. If the request is not
acceptable, the Performance Metric Experts MAY coordinate with the acceptable, the Performance Metric Experts MAY coordinate with the
skipping to change at page 25, line 51 skipping to change at page 26, line 21
maintain backward-compatibility with implementations of the prior maintain backward-compatibility with implementations of the prior
Performance Metrics Registry entry describing a Registered Performance Metrics Registry entry describing a Registered
Performance Metric (entries with lower revision numbers, but the same Performance Metric (entries with lower revision numbers, but the same
Identifier and Name). Identifier and Name).
The purpose of the Status field in the Performance Metrics Registry The purpose of the Status field in the Performance Metrics Registry
is to indicate whether the entry for a Registered Performance Metric is to indicate whether the entry for a Registered Performance Metric
is 'current' or 'deprecated'. is 'current' or 'deprecated'.
In addition, no policy is defined for revising the Performance Metric In addition, no policy is defined for revising the Performance Metric
entries in the IANA Regsirty or addressing errors therein. To be entries in the IANA Registry or addressing errors therein. To be
clear, changes and deprecations within the Performance Metrics clear, changes and deprecations within the Performance Metrics
Registry are not encouraged, and should be avoided to the extent Registry are not encouraged, and should be avoided to the extent
possible. However, in recognition that change is inevitable, the possible. However, in recognition that change is inevitable, the
provisions of this section address the need for revisions. provisions of this section address the need for revisions.
Revisions are initiated by sending a candidate Registered Performance Revisions are initiated by sending a candidate Registered Performance
Metric definition to IANA, as in Section 8.1, identifying the Metric definition to IANA, as in Section 8.1, identifying the
existing Performance Metrics Registry entry, and explaining how and existing Performance Metrics Registry entry, and explaining how and
why the existing entry should be revised. why the existing entry should be revised.
skipping to change at page 26, line 26 skipping to change at page 26, line 44
measurement interoperability problems; the Performance Metric Experts measurement interoperability problems; the Performance Metric Experts
must work to maintain interoperability above all else. Changes to must work to maintain interoperability above all else. Changes to
Registered Performance Metrics may only be done in an interoperable Registered Performance Metrics may only be done in an interoperable
way; necessary changes that cannot be done in a way to allow way; necessary changes that cannot be done in a way to allow
interoperability with unchanged implementations MUST result in the interoperability with unchanged implementations MUST result in the
creation of a new Registered Performance Metric (with a new Name, creation of a new Registered Performance Metric (with a new Name,
replacing the RFCXXXXsecY portion of the name) and possibly the replacing the RFCXXXXsecY portion of the name) and possibly the
deprecation of the earlier metric. deprecation of the earlier metric.
A change to a Registered Performance Metric SHALL be determined to be A change to a Registered Performance Metric SHALL be determined to be
backward-compatible only when: backward-compatible when:
1. it involves the correction of an error that is obviously only 1. it involves the correction of an error that is obviously only
editorial; or editorial; or
2. it corrects an ambiguity in the Registered Performance Metric's 2. it corrects an ambiguity in the Registered Performance Metric's
definition, which itself leads to issues severe enough to prevent definition, which itself leads to issues severe enough to prevent
the Registered Performance Metric's usage as originally defined; the Registered Performance Metric's usage as originally defined;
or or
3. it corrects missing information in the metric definition without 3. it corrects missing information in the metric definition without
skipping to change at page 26, line 50 skipping to change at page 27, line 19
4. it harmonizes with an external reference that was itself 4. it harmonizes with an external reference that was itself
corrected. corrected.
If a Performance Metric revision is deemed permissible and backward- If a Performance Metric revision is deemed permissible and backward-
compatible by the Performance Metric Experts, according to the rules compatible by the Performance Metric Experts, according to the rules
in this document, IANA SHOULD execute the change(s) in the in this document, IANA SHOULD execute the change(s) in the
Performance Metrics Registry. The requester of the change is Performance Metrics Registry. The requester of the change is
appended to the original requester in the Performance Metrics appended to the original requester in the Performance Metrics
Registry. The Name of the revised Registered Performance Metric, Registry. The Name of the revised Registered Performance Metric,
including the RFCXXXXsecY portion of the name, SHALL remain unchamged including the RFCXXXXsecY portion of the name, SHALL remain unchanged
(even when the change is the result of IETF Standards Action; the (even when the change is the result of IETF Standards Action; the
revised registry entry SHOULD reference the new immutable document, revised registry entry SHOULD reference the new immutable document,
such as an RFC or for other standards bodies, it is likely to be such as an RFC or for other standards bodies, it is likely to be
necessary to reference a specific, dated version of a specification, necessary to reference a specific, dated version of a specification,
in an appropriate category and column). in an appropriate category and column).
Each Registered Performance Metric in the Performance Metrics Each Registered Performance Metric in the Performance Metrics
Registry has a revision number, starting at zero. Each change to a Registry has a revision number, starting at zero. Each change to a
Registered Performance Metric following this process increments the Registered Performance Metric following this process increments the
revision number by one. revision number by one.
skipping to change at page 28, line 5 skipping to change at page 28, line 24
Performance Metric Experts for review. When deprecating an Performance Metric Experts for review. When deprecating an
Performance Metric, the Performance Metric description in the Performance Metric, the Performance Metric description in the
Performance Metrics Registry must be updated to explain the Performance Metrics Registry must be updated to explain the
deprecation, as well as to refer to any new Performance Metrics deprecation, as well as to refer to any new Performance Metrics
created to replace the deprecated Performance Metric. created to replace the deprecated Performance Metric.
The revision number of a Registered Performance Metric is incremented The revision number of a Registered Performance Metric is incremented
upon deprecation, and the revision Date updated, as with any upon deprecation, and the revision Date updated, as with any
revision. revision.
The use of deprecated Registered Performance Metrics should result in The intentional use of deprecated Registered Performance Metrics
a log entry or human-readable warning by the respective application. should result in a log entry or human-readable warning by the
respective application.
Names and Metric IDs of deprecated Registered Performance Metrics Names and Metric IDs of deprecated Registered Performance Metrics
must not be reused. must not be reused.
The deprecated entries are kept with all fields unmodified, except The deprecated entries are kept with all fields unmodified, except
the version, revision date, and the status field (changed to the version, revision date, and the status field (changed to
"Deprecated"). "Deprecated").
9. Security considerations 9. Security considerations
This draft defines a registry structure, and does not itself This draft defines a registry structure, and does not itself
introduce any new security considerations for the Internet. The introduce any new security considerations for the Internet. The
definition of Performance Metrics for this registry may introduce definition of Performance Metrics for this registry may introduce
some security concerns, but the mandatory references should have some security concerns, but the mandatory references should have
their own considerations for secuity, and such definitions should be their own considerations for security, and such definitions should be
reviewed with security in mind if the security considerations are not reviewed with security in mind if the security considerations are not
covered by one or more reference standards. covered by one or more reference standards.
The aggregated results of the performance metrics described in this
registry might reveal network topology information that may be
considered sensitive. If such cases are found, then access control
mechanisms should be applied.
10. IANA Considerations 10. IANA Considerations
With the background and processes described in earlier sections, this With the background and processes described in earlier sections, this
document requests the following IANA Actions. document requests the following IANA Actions.
Editor's Note: Mock-ups of the implementation of this set of requests Editor's Note: Mock-ups of the implementation of this set of requests
have been prepared with IANA's help during development of this memo, have been prepared with IANA's help during development of this memo,
and have been captured in the Proceedings of IPPM working group and have been captured in the Proceedings of IPPM working group
sessions. IANA is currently preparing a mock-up. A recent version sessions. IANA is currently preparing a mock-up. A recent version
is available here: http://encrypted.net/IETFMetricsRegistry-106.html is available here: http://encrypted.net/IETFMetricsRegistry-106.html
skipping to change at page 29, line 38 skipping to change at page 30, line 18
choose Name elements from among the registered elements. However, if choose Name elements from among the registered elements. However, if
the proposed metric is unique in a significant way, it may be the proposed metric is unique in a significant way, it may be
necessary to propose a new Name element to properly describe the necessary to propose a new Name element to properly describe the
metric, as described below. metric, as described below.
A candidate Metric Entry RFC or immutable document for IANA and A candidate Metric Entry RFC or immutable document for IANA and
Expert Review would propose one or more new element values required Expert Review would propose one or more new element values required
to describe the unique entry, and the new name element(s) would be to describe the unique entry, and the new name element(s) would be
reviewed along with the metric entry. New assignments for Registered reviewed along with the metric entry. New assignments for Registered
Performance Metric Name Elements will be administered by IANA through Performance Metric Name Elements will be administered by IANA through
Expert Review [RFC8126], i.e., review by one of a group of experts, Specification Required policy (which includes Expert Review)
the Performance Metric Experts, who are appointed by the IESG upon [RFC8126], i.e., review by one of a group of experts, the Performance
recommendation of the Transport Area Directors. Metric Experts, who are appointed by the IESG upon recommendation of
the Transport Area Directors.
10.3. New Performance Metrics Registry 10.3. New Performance Metrics Registry
This document specifies the procedure for Performance Metrics This document specifies the procedure for Performance Metrics
Registry setup. IANA is requested to create a new registry for Registry setup. IANA is requested to create a new registry for
Performance Metrics called "Performance Metrics Registry". This Performance Metrics called "Performance Metrics Registry". This
Registry will contain the following Summary columns: Registry will contain the following Summary columns:
Identifier: Identifier:
skipping to change at page 30, line 19 skipping to change at page 30, line 48
Reference: Reference:
Change Controller: Change Controller:
Version: Version:
Descriptions of these columns and additional information found in the Descriptions of these columns and additional information found in the
template for registry entries (categories and columns) are further template for registry entries (categories and columns) are further
defined in section Section 7. defined in section Section 7.
The "Identifier" 0 should be Reserved. "The Identifier" values from The Identifier 0 should be Reserved. The Registered Performance
64512 to 65536 are reserved for private use. Metric unique identifier is an unbounded integer (range 0 to
infinity). The Identifier values from 64512 to 65536 are reserved
for private or experimental use, and the user may encounter
overlapping uses. When adding newly Registered Performance Metrics
to the Performance Metrics Registry, IANA SHOULD assign the lowest
available identifier to the new Registered Performance Metric. If a
Performance Metrics Expert providing review determines that there is
a reason to assign a specific numeric identifier, possibly leaving a
temporary gap in the numbering, then the Performance Expert SHALL
inform IANA of this decision.
Names starting with the prefix Priv_ are reserved for private use, Names starting with the prefix Priv_ are reserved for private use,
and are not considered for registration. The "Name" column entries and are not considered for registration. The "Name" column entries
are further defined in section Section 7. are further defined in section Section 7.
The "URIs" column will have a URL to the full template of each The "URIs" column will have a URL to the full template of each
registry entry. The Registry Entry text SHALL be HTML-ized to aid registry entry. The Registry Entry text SHALL be HTML-ized to aid
the reader, with links to reference RFCs (similar to the way that the reader, with links to reference RFCs (similar to the way that
Internet Drafts are HTML-ized, the same tool can perform the Internet Drafts are HTML-ized, the same tool can perform the
function) or immutable document. function) or immutable document.
The "Reference" column will include an RFC number, an approved The "Reference" column will include an RFC number, an approved
specification designator from another standards body, other immutable specification designator from another standards body, or other
document, or the contact person. immutable document.
New assignments for Performance Metrics Registry will be administered New assignments for Performance Metrics Registry will be administered
by IANA through Expert Review [RFC8126], i.e., review by one of a by IANA through Specification Required policty (which includes Expert
group of experts, the Performance Metric Experts, who are appointed Review) [RFC8126], i.e., review by one of a group of experts, the
by the IESG upon recommendation of the Transport Area Directors, or Performance Metric Experts, who are appointed by the IESG upon
by Standards Action. The experts can be initially drawn from the recommendation of the Transport Area Directors, or by Standards
Working Group Chairs, document editors, and members of the Action. The experts can be initially drawn from the Working Group
Performance Metrics Directorate, among other sources of experts. Chairs, document editors, and members of the Performance Metrics
Directorate, among other sources of experts.
Extensions of the Performance Metrics Registry require IETF Standards Extensions of the Performance Metrics Registry require IETF Standards
Action. Only one form of registry extension is envisaged: Action. Only one form of registry extension is envisaged:
1. Adding columns, or both categories and columns, to accommodate 1. Adding columns, or both categories and columns, to accommodate
unanticipated aspects of new measurements and metric categories. unanticipated aspects of new measurements and metric categories.
If the Performance Metrics Registry is extended in this way, the If the Performance Metrics Registry is extended in this way, the
Version number of future entries complying with the extension SHALL Version number of future entries complying with the extension SHALL
be incremented (either in the unit or tenths digit, depending on the be incremented (either in the unit or tenths digit, depending on the
skipping to change at page 31, line 27 skipping to change at page 32, line 20
11.1.1. ID (Identifier) 11.1.1. ID (Identifier)
<insert a numeric identifier, an integer, TBD> <insert a numeric identifier, an integer, TBD>
11.1.2. Name 11.1.2. Name
<insert name according to metric naming convention> <insert name according to metric naming convention>
11.1.3. URIs 11.1.3. URIs
URL: http://<TBD by IANA>/<name> URL: https://www.iana.org/ ... <name>
11.1.4. Description 11.1.4. Description
<provide a description> <provide a description>
11.1.5. Change Controller 11.1.5. Change Controller
11.1.6. Version (of Registry Format) 11.1.6. Version (of Registry Format)
11.2. Metric Definition 11.2. Metric Definition
This category includes columns to prompt the entry of all necessary This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the RFC reference details related to the metric definition, including the immutable
and values of input factors, called fixed parameters. document reference and values of input factors, called fixed
parameters.
11.2.1. Reference Definition 11.2.1. Reference Definition
<Full bibliographic reference to an immutable doc.> <Full bibliographic reference to an immutable doc.>
<specific section reference and additional clarifications, if needed> <specific section reference and additional clarifications, if needed>
11.2.2. Fixed Parameters 11.2.2. Fixed Parameters
<list and specify Fixed Parameters, input factors that must be <list and specify Fixed Parameters, input factors that must be
determined and embedded in the measurement system for use when determined and embedded in the measurement system for use when
needed> needed>
11.3. Method of Measurement 11.3. Method of Measurement
This category includes columns for references to relevant sections of This category includes columns for references to relevant sections of
the RFC(s) and any supplemental information needed to ensure an the immutable documents(s) and any supplemental information needed to
unambiguous methods for implementations. ensure an unambiguous methods for implementations.
11.3.1. Reference Method 11.3.1. Reference Method
<for metric, insert relevant section references and supplemental <for metric, insert relevant section references and supplemental
info> info>
11.3.2. Packet Stream Generation 11.3.2. Packet Stream Generation
<list of generation parameters and section/spec references if needed> <list of generation parameters and section/spec references if needed>
skipping to change at page 33, line 33 skipping to change at page 34, line 28
11.4.4. Calibration 11.4.4. Calibration
<insert information on calibration> <insert information on calibration>
11.5. Administrative items 11.5. Administrative items
11.5.1. Status 11.5.1. Status
<current or deprecated> <current or deprecated>
11.5.2. Requestor 11.5.2. Requester
<name or RFC, etc.> <name or RFC, etc.>
11.5.3. Revision 11.5.3. Revision
<1.0> <1.0>
11.5.4. Revision Date 11.5.4. Revision Date
<format YYYY-MM-DD> <format YYYY-MM-DD>
skipping to change at page 34, line 13 skipping to change at page 34, line 52
<Additional (Informational) details for this entry> <Additional (Informational) details for this entry>
12. Acknowledgments 12. Acknowledgments
Thanks to Brian Trammell and Bill Cerveny, IPPM chairs, for leading Thanks to Brian Trammell and Bill Cerveny, IPPM chairs, for leading
some brainstorming sessions on this topic. Thanks to Barbara Stark some brainstorming sessions on this topic. Thanks to Barbara Stark
and Juergen Schoenwaelder for the detailed feedback and suggestions. and Juergen Schoenwaelder for the detailed feedback and suggestions.
Thanks to Andrew McGregor for suggestions on metric naming. Thanks Thanks to Andrew McGregor for suggestions on metric naming. Thanks
to Michelle Cotton for her early IANA review, and to Amanda Barber to Michelle Cotton for her early IANA review, and to Amanda Barber
for answering questions related to the presentation of the registry for answering questions related to the presentation of the registry
and accessibility of the complete template via URL. and accessibility of the complete template via URL. Thanks to Roni
Even for his review and suggestions to generalize the procedures.
Thanks to ~all the Area Directors for their reviews.
13. References 13. References
13.1. Normative References 13.1. Normative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision [RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996, 3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996,
<https://www.rfc-editor.org/info/rfc2026>. <https://www.rfc-editor.org/info/rfc2026>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
skipping to change at page 34, line 48 skipping to change at page 35, line 40
[RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New
Performance Metric Development", BCP 170, RFC 6390, Performance Metric Development", BCP 170, RFC 6390,
DOI 10.17487/RFC6390, October 2011, DOI 10.17487/RFC6390, October 2011,
<https://www.rfc-editor.org/info/rfc6390>. <https://www.rfc-editor.org/info/rfc6390>.
[RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz, [RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz,
"IP Performance Metrics (IPPM) Standard Advancement "IP Performance Metrics (IPPM) Standard Advancement
Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March
2012, <https://www.rfc-editor.org/info/rfc6576>. 2012, <https://www.rfc-editor.org/info/rfc6576>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26, Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017, RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>. <https://www.rfc-editor.org/info/rfc8126>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
13.2. Informative References 13.2. Informative References
[I-D.ietf-ippm-initial-registry] [I-D.ietf-ippm-initial-registry]
Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza, Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza,
"Initial Performance Metrics Registry Entries", draft- "Initial Performance Metrics Registry Entries", draft-
ietf-ippm-initial-registry-12 (work in progress), ietf-ippm-initial-registry-14 (work in progress), November
September 2019. 2019.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
September 1999, <https://www.rfc-editor.org/info/rfc2681>. September 1999, <https://www.rfc-editor.org/info/rfc2681>.
[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
performance measurement with periodic streams", RFC 3432, performance measurement with periodic streams", RFC 3432,
DOI 10.17487/RFC3432, November 2002, DOI 10.17487/RFC3432, November 2002,
<https://www.rfc-editor.org/info/rfc3432>. <https://www.rfc-editor.org/info/rfc3432>.
skipping to change at page 36, line 44 skipping to change at page 37, line 39
Aitken, P., and A. Akhter, "A Framework for Large-Scale Aitken, P., and A. Akhter, "A Framework for Large-Scale
Measurement of Broadband Performance (LMAP)", RFC 7594, Measurement of Broadband Performance (LMAP)", RFC 7594,
DOI 10.17487/RFC7594, September 2015, DOI 10.17487/RFC7594, September 2015,
<https://www.rfc-editor.org/info/rfc7594>. <https://www.rfc-editor.org/info/rfc7594>.
[RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Ed., "A One-Way Delay Metric for IP Performance Metrics Ed., "A One-Way Delay Metric for IP Performance Metrics
(IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
2016, <https://www.rfc-editor.org/info/rfc7679>. 2016, <https://www.rfc-editor.org/info/rfc7679>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8194] Schoenwaelder, J. and V. Bajpai, "A YANG Data Model for [RFC8194] Schoenwaelder, J. and V. Bajpai, "A YANG Data Model for
LMAP Measurement Agents", RFC 8194, DOI 10.17487/RFC8194, LMAP Measurement Agents", RFC 8194, DOI 10.17487/RFC8194,
August 2017, <https://www.rfc-editor.org/info/rfc8194>. August 2017, <https://www.rfc-editor.org/info/rfc8194>.
Authors' Addresses Authors' Addresses
Marcelo Bagnulo Marcelo Bagnulo
Universidad Carlos III de Universidad Carlos III de
Madrid Madrid
Av. Universidad 30 Av. Universidad 30
Leganes, Madrid 28911 Leganes, Madrid 28911
SPAIN SPAIN
Phone: 34 91 6249500 Phone: 34 91 6249500
Email: marcelo@it.uc3m.es Email: marcelo@it.uc3m.es
URI: http://www.it.uc3m.es URI: http://www.it.uc3m.es
 End of changes. 78 change blocks. 
188 lines changed or deleted 219 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/