< draft-ietf-ippm-metric-registry-19.txt		draft-ietf-ippm-metric-registry-20.txt >
	Network Working Group M. Bagnulo		Network Working Group M. Bagnulo
	Internet-Draft UC3M		Internet-Draft UC3M
	Intended status: Best Current Practice B. Claise		Intended status: Best Current Practice B. Claise
	Expires: September 29, 2019 Cisco Systems, Inc.		Expires: March 14, 2020 Cisco Systems, Inc.
	P. Eardley		P. Eardley
	BT		BT
	A. Morton		A. Morton
	AT&T Labs		AT&T Labs
	A. Akhter		A. Akhter
	Consultant		Consultant
	March 28, 2019		September 11, 2019
	Registry for Performance Metrics		Registry for Performance Metrics
	draft-ietf-ippm-metric-registry-19		draft-ietf-ippm-metric-registry-20
	Abstract		Abstract
	This document defines the format for the IANA Performance Metrics		This document defines the format for the IANA Performance Metrics
	Registry. This document also gives a set of guidelines for		Registry. This document also gives a set of guidelines for
	Registered Performance Metric requesters and reviewers.		Registered Performance Metric requesters and reviewers.
	Status of This Memo		Status of This Memo
	This Internet-Draft is submitted in full conformance with the		This Internet-Draft is submitted in full conformance with the
	skipping to change at page 1, line 39 ¶		skipping to change at page 1, line 39 ¶
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF). Note that other groups may also distribute		Task Force (IETF). Note that other groups may also distribute
	working documents as Internet-Drafts. The list of current Internet-		working documents as Internet-Drafts. The list of current Internet-
	Drafts is at https://datatracker.ietf.org/drafts/current/.		Drafts is at https://datatracker.ietf.org/drafts/current/.
	Internet-Drafts are draft documents valid for a maximum of six months		Internet-Drafts are draft documents valid for a maximum of six months
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	This Internet-Draft will expire on September 29, 2019.		This Internet-Draft will expire on March 14, 2020.
	Copyright Notice		Copyright Notice
	Copyright (c) 2019 IETF Trust and the persons identified as the		Copyright (c) 2019 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(https://trustee.ietf.org/license-info) in effect on the date of		(https://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	skipping to change at page 3, line 19 ¶		skipping to change at page 3, line 19 ¶
	8.3. Deprecating Registered Performance Metrics . . . . . . . 27		8.3. Deprecating Registered Performance Metrics . . . . . . . 27
	9. Security considerations . . . . . . . . . . . . . . . . . . . 27		9. Security considerations . . . . . . . . . . . . . . . . . . . 27
	10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28		10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28
	10.1. Registry Group . . . . . . . . . . . . . . . . . . . . . 28		10.1. Registry Group . . . . . . . . . . . . . . . . . . . . . 28
	10.2. Performance Metric Name Elements . . . . . . . . . . . . 28		10.2. Performance Metric Name Elements . . . . . . . . . . . . 28
	10.3. New Performance Metrics Registry . . . . . . . . . . . . 29		10.3. New Performance Metrics Registry . . . . . . . . . . . . 29
	11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 30		11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 30
	12. References . . . . . . . . . . . . . . . . . . . . . . . . . 30		12. References . . . . . . . . . . . . . . . . . . . . . . . . . 30
	12.1. Normative References . . . . . . . . . . . . . . . . . . 30		12.1. Normative References . . . . . . . . . . . . . . . . . . 30
	12.2. Informative References . . . . . . . . . . . . . . . . . 31		12.2. Informative References . . . . . . . . . . . . . . . . . 31
	Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 34		Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 33
	1. Introduction		1. Introduction
	The IETF specifies and uses Performance Metrics of protocols and		The IETF specifies and uses Performance Metrics of protocols and
	applications transported over its protocols. Performance metrics are		applications transported over its protocols. Performance metrics are
	such an important part of the operations of IETF protocols that		such an important part of the operations of IETF protocols that
	[RFC6390] specifies guidelines for their development.		[RFC6390] specifies guidelines for their development.
	The definition and use of Performance Metrics in the IETF happens in		The definition and use of Performance Metrics in the IETF happens in
	various working groups (WG), most notably:		various working groups (WG), most notably:
	skipping to change at page 4, line 13 ¶		skipping to change at page 4, line 13 ¶
	basis.		basis.
	The "Performance Metrics for Other Layers" (PMOL) a concluded WG,		The "Performance Metrics for Other Layers" (PMOL) a concluded WG,
	defined some Performance Metrics related to Session Initiation		defined some Performance Metrics related to Session Initiation
	Protocol (SIP) voice quality [RFC6035].		Protocol (SIP) voice quality [RFC6035].
	It is expected that more Performance Metrics will be defined in the		It is expected that more Performance Metrics will be defined in the
	future, not only IP-based metrics, but also metrics which are		future, not only IP-based metrics, but also metrics which are
	protocol-specific and application-specific.		protocol-specific and application-specific.
	However, despite the importance of Performance Metrics, there are two		Despite the importance of Performance Metrics, there are two related
	related problems for the industry. First, how to ensure that when		problems for the industry. First, ensuring that when one party
	one party requests another party to measure (or report or in some way		requests another party to measure (or report or in some way act on) a
	act on) a particular Performance Metric, then both parties have		particular Performance Metric, then both parties have exactly the
	exactly the same understanding of what Performance Metric is being		same understanding of what Performance Metric is being referred to.
	referred to. Second, how to discover which Performance Metrics have		Second, discovering which Performance Metrics have been specified, to
	been specified, so as to avoid developing a new Performance Metric		avoid developing a new Performance Metric that is very similar, but
	that is very similar, but not quite inter-operable. The problems can		not quite inter-operable. These problems can be addressed by
	be addressed by creating a registry of performance metrics. The		creating a registry of performance metrics. The usual way in which
	usual way in which IETF organizes namespaces is with Internet		the IETF organizes registries is with Internet Assigned Numbers
	Assigned Numbers Authority (IANA) registries, and there is currently		Authority (IANA), and there is currently no Performance Metrics
	no Performance Metrics Registry maintained by the IANA.		Registry maintained by the IANA.
	This document therefore requests that IANA create and maintain a		This document requests that IANA create and maintain a Performance
	Performance Metrics Registry, according to the maintenance procedures		Metrics Registry, according to the maintenance procedures and the
	and the Performance Metrics Registry format defined in this memo.		Performance Metrics Registry format defined in this memo. The
	The resulting Performance Metrics Registry is for use by the IETF and		resulting Performance Metrics Registry is for use by the IETF and
	others. Although the Registry formatting specifications herein are		others. Although the Registry formatting specifications herein are
	primarily for registry creation by IANA, any other organization that		primarily for registry creation by IANA, any other organization that
	wishes to create a Performance Metrics Registry MAY use the same		wishes to create a Performance Metrics Registry MAY use the same
	formatting specifications for their purposes. The authors make no		formatting specifications for their purposes. The authors make no
	guarantee of the registry format's applicability to any possible set		guarantee of the registry format's applicability to any possible set
	of Performance Metrics envisaged by other organizations, but		of Performance Metrics envisaged by other organizations, but
	encourage others to apply it. In the remainder of this document,		encourage others to apply it. In the remainder of this document,
	unless we explicitly say otherwise, we will refer to the IANA-		unless we explicitly say otherwise, we will refer to the IANA-
	maintained Performance Metrics Registry as simply the Performance		maintained Performance Metrics Registry as simply the Performance
	Metrics Registry.		Metrics Registry.
	2. Terminology		2. Terminology
	The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",		The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
	"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and		"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
	"OPTIONAL" in this document are to be interpreted as described in BCP		"OPTIONAL" in this document are to be interpreted as described in BCP
	14[RFC2119] [RFC8174] when, and only when, they appear in all		14 [RFC2119] [RFC8174] when, and only when, they appear in all
	capitals, as shown here.		capitals, as shown here.
	Performance Metric: A Performance Metric is a quantitative measure		Performance Metric: A Performance Metric is a quantitative measure
	of performance, targeted to an IETF-specified protocol or targeted		of performance, targeted to an IETF-specified protocol or targeted
	to an application transported over an IETF-specified protocol.		to an application transported over an IETF-specified protocol.
	Examples of Performance Metrics are the FTP response time for a		Examples of Performance Metrics are the FTP response time for a
	complete file download, the DNS response time to resolve the IP		complete file download, the DNS response time to resolve the IP
	address, a database logging time, etc. This definition is		address, a database logging time, etc. This definition is
	consistent with the definition of metric in [RFC2330] and broader		consistent with the definition of metric in [RFC2330] and broader
	than the definition of performance metric in [RFC6390].		than the definition of performance metric in [RFC6390].
	skipping to change at page 5, line 27 ¶		skipping to change at page 5, line 27 ¶
	Proprietary Registry: A set of metrics that are registered in a		Proprietary Registry: A set of metrics that are registered in a
	proprietary registry, as opposed to Performance Metrics Registry.		proprietary registry, as opposed to Performance Metrics Registry.
	Performance Metrics Experts: The Performance Metrics Experts is a		Performance Metrics Experts: The Performance Metrics Experts is a
	group of designated experts [RFC8126] selected by the IESG to		group of designated experts [RFC8126] selected by the IESG to
	validate the Performance Metrics before updating the Performance		validate the Performance Metrics before updating the Performance
	Metrics Registry. The Performance Metrics Experts work closely		Metrics Registry. The Performance Metrics Experts work closely
	with IANA.		with IANA.
	Parameter: An input factor defined as a variable in the definition		Parameter: A Parameter is an input factor defined as a variable in
	of a Performance Metric. A numerical or other specified factor		the definition of a Performance Metric. A Parameter is a
	forming one of a set that defines a metric or sets the conditions		numerical or other specified factor forming one of a set that
	of its operation. All Parameters must be known to measure using a		defines a metric or sets the conditions of its operation. All
	metric and interpret the results. There are two types of		Parameters must be known to measure using a metric and interpret
	Parameters, Fixed and Run-time parameters. For the Fixed		the results. There are two types of Parameters: Fixed and Run-
	Parameters, the value of the variable is specified in the		time parameters. For the Fixed Parameters, the value of the
	Performance Metrics Registry entry and different Fixed Parameter		variable is specified in the Performance Metrics Registry entry
	values results in different Registered Performance Metrics. For		and different Fixed Parameter values results in different
	the Run-time Parameters, the value of the variable is defined when		Registered Performance Metrics. For the Run-time Parameters, the
	the metric measurement method is executed and a given Registered		value of the variable is defined when the metric measurement
	Performance Metric supports multiple values for the parameter.		method is executed and a given Registered Performance Metric
	Although Run-time Parameters do not change the fundamental nature		supports multiple values for the parameter. Although Run-time
	of the Performance Metric's definition, some have substantial		Parameters do not change the fundamental nature of the Performance
	influence on the network property being assessed and		Metric's definition, some have substantial influence on the
	interpretation of the results.		network property being assessed and interpretation of the results.
	Note: Consider the case of packet loss in the following two		Note: Consider the case of packet loss in the following two
	Active Measurement Method cases. The first case is packet loss		Active Measurement Method cases. The first case is packet loss
	as background loss where the Run-time Parameter set includes a		as background loss where the Run-time Parameter set includes a
	very sparse Poisson stream, and only characterizes the times		very sparse Poisson stream, and only characterizes the times
	when packets were lost. Actual user streams likely see much		when packets were lost. Actual user streams likely see much
	higher loss at these times, due to tail drop or radio errors.		higher loss at these times, due to tail drop or radio errors.
	The second case is packet loss as inverse of throughput where		The second case is packet loss as inverse of throughput where
	the Run-time Parameter set includes a very dense, bursty		the Run-time Parameter set includes a very dense, bursty
	stream, and characterizes the loss experienced by a stream that		stream, and characterizes the loss experienced by a stream that
	skipping to change at page 6, line 38 ¶		skipping to change at page 6, line 38 ¶
	Hybrid Measurement Method: Hybrid Methods are Methods of Measurement		Hybrid Measurement Method: Hybrid Methods are Methods of Measurement
	that use a combination of Active Methods and Passive Methods, to		that use a combination of Active Methods and Passive Methods, to
	assess Active Metrics, Passive Metrics, or new metrics derived		assess Active Metrics, Passive Metrics, or new metrics derived
	from the a priori knowledge and observations of the stream of		from the a priori knowledge and observations of the stream of
	interest. The complete definition of Hybrid Methods is specified		interest. The complete definition of Hybrid Methods is specified
	in section 3.8 of [RFC7799].		in section 3.8 of [RFC7799].
	3. Scope		3. Scope
	This document is meant mainly for two different audiences. For those		This document is intended for two different audiences:
	defining new Registered Performance Metrics, it provides
	specifications and best practices to be used in deciding which		1. For those defining new Registered Performance Metrics, it
	Registered Performance Metrics are useful for a measurement study,		provides specifications and best practices to be used in deciding
	instructions for writing the text for each column of the Registered		which Registered Performance Metrics are useful for a measurement
	Performance Metrics, and information on the supporting documentation		study, instructions for writing the text for each column of the
	required for the new Performance Metrics Registry entry (up to and		Registered Performance Metrics, and information on the supporting
	including the publication of one or more RFCs or I-Ds describing it).		documentation required for the new Performance Metrics Registry
	For the appointed Performance Metrics Experts and for IANA personnel		entry (up to and including the publication of one or more RFCs or
	administering the new IANA Performance Metrics Registry, it defines a		I-Ds describing it).
	set of acceptance criteria against which these proposed Registered
	Performance Metrics should be evaluated. In addition, this document		2. For the appointed Performance Metrics Experts and for IANA
	may be useful for other organizations who are defining a Performance		personnel administering the new IANA Performance Metrics
	Metric registry of their own, and may re-use the features of the		Registry, it defines a set of acceptance criteria against which
	Performance Metrics Registry defined in this document.		these proposed Registered Performance Metrics should be
			evaluated.
			In addition, this document may be useful for other organizations who
			are defining a Performance Metric registry of their own, and may re-
			use the features of the Performance Metrics Registry defined in this
			document.
	This Performance Metrics Registry is applicable to Performance		This Performance Metrics Registry is applicable to Performance
	Metrics issued from Active Measurement, Passive Measurement, and any		Metrics issued from Active Measurement, Passive Measurement, and any
	other form of Performance Metric. This registry is designed to		other form of Performance Metric. This registry is designed to
	encompass Performance Metrics developed throughout the IETF and		encompass Performance Metrics developed throughout the IETF and
	especially for the technologies specified in the following working		especially for the technologies specified in the following working
	groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes an		groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document analyzes an
	prior attempt to set up a Performance Metrics Registry, and the		prior attempt to set up a Performance Metrics Registry, and the
	reasons why this design was inadequate [RFC6248]. Finally, this		reasons why this design was inadequate [RFC6248]. Finally, this
	document gives a set of guidelines for requesters and expert		document gives a set of guidelines for requesters and expert
	skipping to change at page 7, line 32 ¶		skipping to change at page 7, line 37 ¶
	Current Practice (BCP) [RFC2026].		Current Practice (BCP) [RFC2026].
	4. Motivation for a Performance Metrics Registry		4. Motivation for a Performance Metrics Registry
	In this section, we detail several motivations for the Performance		In this section, we detail several motivations for the Performance
	Metrics Registry.		Metrics Registry.
	4.1. Interoperability		4.1. Interoperability
	As any IETF registry, the primary use for a registry is to manage a		As any IETF registry, the primary use for a registry is to manage a
	namespace for its use within one or more protocols. In the		registry for its use within one or more protocols. In the particular
	particular case of the Performance Metrics Registry, there are two		case of the Performance Metrics Registry, there are two types of
	types of protocols that will use the Performance Metrics in the		protocols that will use the Performance Metrics in the Performance
	Performance Metrics Registry during their operation (by referring to		Metrics Registry during their operation (by referring to the Index
	the Index values):		values):
	o Control protocol: this type of protocols is used to allow one		o Control protocol: This type of protocol used to allow one entity
	entity to request another entity to perform a measurement using a		to request another entity to perform a measurement using a
	specific metric defined by the Performance Metrics Registry. One		specific metric defined by the Performance Metrics Registry. One
	particular example is the LMAP framework [RFC7594]. Using the		particular example is the LMAP framework [RFC7594]. Using the
	LMAP terminology, the Performance Metrics Registry is used in the		LMAP terminology, the Performance Metrics Registry is used in the
	LMAP Control protocol to allow a Controller to request a		LMAP Control protocol to allow a Controller to request a
	measurement task to one or more Measurement Agents. In order to		measurement task to one or more Measurement Agents. In order to
	enable this use case, the entries of the Performance Metrics		enable this use case, the entries of the Performance Metrics
	Registry must be well enough defined to allow a Measurement Agent		Registry must be sufficiently defined to allow a Measurement Agent
	implementation to trigger a specific measurement task upon the		implementation to trigger a specific measurement task upon the
	reception of a control protocol message. This requirement heavily		reception of a control protocol message. This requirement heavily
	constrains the type of entries that are acceptable for the		constrains the type of entries that are acceptable for the
	Performance Metrics Registry.		Performance Metrics Registry.
	o Report protocol: This type of protocols is used to allow an entity		o Report protocol: This type of protocol is used to allow an entity
	to report measurement results to another entity. By referencing		to report measurement results to another entity. By referencing
	to a specific Performance Metrics Registry, it is possible to		to a specific Performance Metrics Registry, it is possible to
	properly characterize the measurement result data being reported.		properly characterize the measurement result data being reported.
	Using the LMAP terminology, the Performance Metrics Registry is		Using the LMAP terminology, the Performance Metrics Registry is
	used in the Report protocol to allow a Measurement Agent to report		used in the Report protocol to allow a Measurement Agent to report
	measurement results to a Collector.		measurement results to a Collector.
	It should be noted that the LMAP framework explicitly allows for		It should be noted that the LMAP framework explicitly allows for
	using not only the IANA-maintained Performance Metrics Registry but		using not only the IANA-maintained Performance Metrics Registry but
	also other registries containing Performance Metrics, either defined		also other registries containing Performance Metrics, either defined
	skipping to change at page 8, line 46 ¶		skipping to change at page 8, line 50 ¶
	that request measurements, execute measurements, and report the		that request measurements, execute measurements, and report the
	results can benefit from a common understanding of the referenced		results can benefit from a common understanding of the referenced
	Performance Metric.		Performance Metric.
	4.3. Side benefits		4.3. Side benefits
	There are a couple of side benefits of having such a registry.		There are a couple of side benefits of having such a registry.
	First, the Performance Metrics Registry could serve as an inventory		First, the Performance Metrics Registry could serve as an inventory
	of useful and used Performance Metrics, that are normally supported		of useful and used Performance Metrics, that are normally supported
	by different implementations of measurement agents. Second, the		by different implementations of measurement agents. Second, the
	results of measurements using the Performance Metrics would be		results of measurements using the Performance Metrics should be
	comparable even if they are performed by different implementations		comparable even if they are performed by different implementations
	and in different networks, as the Performance Metric is properly		and in different networks, as the Performance Metric is properly
	defined. BCP 176 [RFC6576] examines whether the results produced by		defined. BCP 176 [RFC6576] examines whether the results produced by
	independent implementations are equivalent in the context of		independent implementations are equivalent in the context of
	evaluating the completeness and clarity of metric specifications.		evaluating the completeness and clarity of metric specifications.
	This BCP defines the standards track advancement testing for (active)		This BCP defines the standards track advancement testing for (active)
	IPPM metrics, and the same process will likely suffice to determine		IPPM metrics, and the same process will likely suffice to determine
	whether Registered Performance Metrics are sufficiently well		whether Registered Performance Metrics are sufficiently well
	specified to result in comparable (or equivalent) results.		specified to result in comparable (or equivalent) results.
	Registered Performance Metrics which have undergone such testing		Registered Performance Metrics which have undergone such testing
	skipping to change at page 16, line 36 ¶		skipping to change at page 16, line 36 ¶
	as described in section 4 of [I-D.ietf-ippm-initial-registry].		as described in section 4 of [I-D.ietf-ippm-initial-registry].
	Note that private registries following the format described here		Note that private registries following the format described here
	SHOULD use the prefix "Priv_" on any name to avoid unintended		SHOULD use the prefix "Priv_" on any name to avoid unintended
	conflicts (further considerations are described in section 10).		conflicts (further considerations are described in section 10).
	Private registry entries usually have no specifying RFC, thus the		Private registry entries usually have no specifying RFC, thus the
	Spec: element has no clear interpretation.		Spec: element has no clear interpretation.
	7.1.3. URIs		7.1.3. URIs
	The URIs column MUST contain a URL [RFC3986] and uniquely identifies		The URIs column MUST contain a URL [RFC3986] that uniquely identifies
	and locates the metric entry so it is accessible through the		and locates the metric entry so it is accessible through the
	Internet. The URL points to a file containing all the human-readable		Internet. The URL points to a file containing all the human-readable
	information for one registry entry. The URL SHALL reference a target		information for one registry entry. The URL SHALL reference a target
	file that is HTML-formated and contains URLs to referenced sections		file that is HTML-formated and contains URLs to referenced sections
	of HTML-ized RFCs. These target files for different entries can be		of HTML-ized RFCs. These target files for different entries can be
	more easily edited and re-used when preparing new entries. The exact		more easily edited and re-used when preparing new entries. The exact
	form of the URL for each target file will be determined by IANA and		form of the URL for each target file will be determined by IANA and
	reside on "iana.org". The major sections of		reside on "iana.org". The major sections of
	[I-D.ietf-ippm-initial-registry] provide an example of a target file		[I-D.ietf-ippm-initial-registry] provide an example of a target file
	in HTML form (sections 4 and higher).		in HTML form (sections 4 and higher).
	skipping to change at page 17, line 29 ¶		skipping to change at page 17, line 29 ¶
	7.1.6. Change Controller		7.1.6. Change Controller
	This entry names the entity responsible for approving revisions to		This entry names the entity responsible for approving revisions to
	the registry entry, and SHALL provide contact information (for an		the registry entry, and SHALL provide contact information (for an
	individual, where appropriate).		individual, where appropriate).
	7.1.7. Version (of Registry Format)		7.1.7. Version (of Registry Format)
	This entry gives the version number for the registry format used.		This entry gives the version number for the registry format used.
	Formats complying with this memo MUST use 1.0. The version number		Formats complying with this memo MUST use 1.0. The version number
	SHALL not change unless a new RFC is published that changes the		SHALL NOT change unless a new RFC is published that changes the
	registry format.		registry format.
	7.2. Metric Definition Category		7.2. Metric Definition Category
	This category includes columns to prompt all necessary details		This category includes columns to prompt all necessary details
	related to the metric definition, including the RFC reference and		related to the metric definition, including the RFC reference and
	values of input factors, called fixed parameters, which are left open		values of input factors, called fixed parameters, which are left open
	in the RFC but have a particular value defined by the performance		in the RFC but have a particular value defined by the performance
	metric.		metric.
	skipping to change at page 19, line 19 ¶		skipping to change at page 19, line 19 ¶
	limited to Active metrics. The generated traffic is referred as a		limited to Active metrics. The generated traffic is referred as a
	stream and this column describes its characteristics.		stream and this column describes its characteristics.
	Each entry for this column contains the following information:		Each entry for this column contains the following information:
	o Value: The name of the packet stream scheduling discipline		o Value: The name of the packet stream scheduling discipline
	o Reference: the specification where the parameters of the stream		o Reference: the specification where the parameters of the stream
	are defined		are defined
	The packet generation stream may require parameters such as the the		The packet generation stream may require parameters such as the
	average packet rate and distribution truncation value for streams		average packet rate and distribution truncation value for streams
	with Poisson-distributed inter-packet sending times. In case such		with Poisson-distributed inter-packet sending times. In case such
	parameters are needed, they should be included either in the Fixed		parameters are needed, they should be included either in the Fixed
	parameter column or in the run time parameter column, depending on		parameter column or in the run time parameter column, depending on
	wether they will be fixed or will be an input for the metric.		wether they will be fixed or will be an input for the metric.
	The simplest example of stream specification is Singleton scheduling		The simplest example of stream specification is Singleton scheduling
	(see [RFC2330]), where a single atomic measurement is conducted.		(see [RFC2330]), where a single atomic measurement is conducted.
	Each atomic measurement could consist of sending a single packet		Each atomic measurement could consist of sending a single packet
	(such as a DNS request) or sending several packets (for example, to		(such as a DNS request) or sending several packets (for example, to
	skipping to change at page 19, line 50 ¶		skipping to change at page 19, line 50 ¶
	7.3.3. Traffic Filter		7.3.3. Traffic Filter
	This column applies to Performance Metrics that observe packets		This column applies to Performance Metrics that observe packets
	flowing through (the device with) the measurement agent i.e. that is		flowing through (the device with) the measurement agent i.e. that is
	not necessarily addressed to the measurement agent. This includes		not necessarily addressed to the measurement agent. This includes
	but is not limited to Passive Metrics. The filter specifies the		but is not limited to Passive Metrics. The filter specifies the
	traffic that is measured. This includes protocol field values/		traffic that is measured. This includes protocol field values/
	ranges, such as address ranges, and flow or session identifiers.		ranges, such as address ranges, and flow or session identifiers.
	The traffic filter itself depends on needs of the metric itself and a		The traffic filter itself depends on needs of the metric itself and a
	balance of operators measurement needs and user's need for privacy.		balance of an operator's measurement needs and a user's need for
	Mechanics for conveying the filter criteria might be the BPF (Berkley		privacy. Mechanics for conveying the filter criteria might be the
	Packet Filter) or PSAMP [RFC5475] Property Match Filtering which		BPF (Berkley Packet Filter) or PSAMP [RFC5475] Property Match
	reuses IPFIX [RFC7012]. An example BPF string for matching TCP/80		Filtering which reuses IPFIX [RFC7012]. An example BPF string for
	traffic to remote destination net 192.0.2.0/24 would be "dst net		matching TCP/80 traffic to remote destination net 192.0.2.0/24 would
	192.0.2.0/24 and tcp dst port 80". More complex filter engines might		be "dst net 192.0.2.0/24 and tcp dst port 80". More complex filter
	be supported by the implementation that might allow for matching		engines might be supported by the implementation that might allow for
	using Deep Packet Inspection (DPI) technology.		matching using Deep Packet Inspection (DPI) technology.
	The traffic filter includes the following information:		The traffic filter includes the following information:
	Type: the type of traffic filter used, e.g. BPF, PSAMP, OpenFlow		Type: the type of traffic filter used, e.g. BPF, PSAMP, OpenFlow
	rule, etc. as defined by a normative reference		rule, etc. as defined by a normative reference
	Value: the actual set of rules expressed		Value: the actual set of rules expressed
	7.3.4. Sampling Distribution		7.3.4. Sampling Distribution
	skipping to change at page 24, line 15 ¶		skipping to change at page 24, line 15 ¶
	8. The Life-Cycle of Registered Performance Metrics		8. The Life-Cycle of Registered Performance Metrics
	Once a Performance Metric or set of Performance Metrics has been		Once a Performance Metric or set of Performance Metrics has been
	identified for a given application, candidate Performance Metrics		identified for a given application, candidate Performance Metrics
	Registry entry specifications prepared in accordance with Section 7		Registry entry specifications prepared in accordance with Section 7
	should be submitted to IANA to follow the process for review by the		should be submitted to IANA to follow the process for review by the
	Performance Metric Experts, as defined below. This process is also		Performance Metric Experts, as defined below. This process is also
	used for other changes to the Performance Metrics Registry, such as		used for other changes to the Performance Metrics Registry, such as
	deprecation or revision, as described later in this section.		deprecation or revision, as described later in this section.
	It is also desirable that the author(s) of a candidate Performance		It is desirable that the author(s) of a candidate Performance Metrics
	Metrics Registry entry seek review in the relevant IETF working		Registry entry seek review in the relevant IETF working group, or
	group, or offer the opportunity for review on the working group		offer the opportunity for review on the working group mailing list.
	mailing list.
	8.1. Adding new Performance Metrics to the Performance Metrics Registry		8.1. Adding new Performance Metrics to the Performance Metrics Registry
	Requests to add Registered Performance Metrics in the Performance		Requests to add Registered Performance Metrics in the Performance
	Metrics Registry SHALL be submitted to IANA, which forwards the		Metrics Registry SHALL be submitted to IANA, which forwards the
	request to a designated group of experts (Performance Metric Experts)		request to a designated group of experts (Performance Metric Experts)
	appointed by the IESG; these are the reviewers called for by the		appointed by the IESG; these are the reviewers called for by the
	Expert Review [RFC8126]policy defined for the Performance Metrics		Expert Review [RFC8126]policy defined for the Performance Metrics
	Registry. The Performance Metric Experts review the request for such		Registry. The Performance Metric Experts review the request for such
	things as compliance with this document, compliance with other		things as compliance with this document, compliance with other
	applicable Performance Metric-related RFCs, and consistency with the		applicable Performance Metric-related RFCs, and consistency with the
	currently defined set of Registered Performance Metrics.		currently defined set of Registered Performance Metrics.
	Submission to IANA MAY be the result of IETF Standards Action, where		Submission to IANA MAY be during IESG review (leading to IETF
	an approved Internet Draft proposes one or more Registered		Standards Action), where an Internet Draft proposes one or more
	Performance Metrics to be added to the Performance Metrics Registry,		Registered Performance Metrics to be added to the Performance Metrics
	including the text of the proposed Registered Performance Metric(s).		Registry, including the text of the proposed Registered Performance
			Metric(s).
	Authors of proposed Registered Performance Metrics SHOULD review		Authors of proposed Registered Performance Metrics SHOULD review
	compliance with the specifications in this document to check their		compliance with the specifications in this document to check their
	submissions before sending them to IANA.		submissions before sending them to IANA.
	At least one Performance Metric Expert should endeavor to complete		At least one Performance Metric Expert should endeavor to complete
	referred reviews in a timely manner. If the request is acceptable,		referred reviews in a timely manner. If the request is acceptable,
	the Performance Metric Experts signify their approval to IANA, and		the Performance Metric Experts signify their approval to IANA, and
	IANA updates the Performance Metrics Registry. If the request is not		IANA updates the Performance Metrics Registry. If the request is not
	acceptable, the Performance Metric Experts MAY coordinate with the		acceptable, the Performance Metric Experts MAY coordinate with the
	skipping to change at page 25, line 16 ¶		skipping to change at page 25, line 16 ¶
	Performance Metric Experts to overrule IETF consensus. Specifically,		Performance Metric Experts to overrule IETF consensus. Specifically,
	any Registered Performance Metrics that were added to the Performance		any Registered Performance Metrics that were added to the Performance
	Metrics Registry with IETF consensus require IETF consensus for		Metrics Registry with IETF consensus require IETF consensus for
	revision or deprecation.		revision or deprecation.
	Decisions by the Performance Metric Experts may be appealed as in		Decisions by the Performance Metric Experts may be appealed as in
	Section 7 of [RFC8126].		Section 7 of [RFC8126].
	8.2. Revising Registered Performance Metrics		8.2. Revising Registered Performance Metrics
	A request for Revision is only permissible when the changes maintain		A request for Revision is only permitted when the requested changes
	backward-compatibility with implementations of the prior Performance		maintain backward-compatibility with implementations of the prior
	Metrics Registry entry describing a Registered Performance Metric		Performance Metrics Registry entry describing a Registered
	(entries with lower revision numbers, but the same Identifier and		Performance Metric (entries with lower revision numbers, but the same
	Name).		Identifier and Name).
	The purpose of the Status field in the Performance Metrics Registry		The purpose of the Status field in the Performance Metrics Registry
	is to indicate whether the entry for a Registered Performance Metric		is to indicate whether the entry for a Registered Performance Metric
	is 'current' or 'deprecated'.		is 'current' or 'deprecated'.
	In addition, no policy is defined for revising the Performance Metric		In addition, no policy is defined for revising the Performance Metric
	entries in the IANA Regsirty or addressing errors therein. To be		entries in the IANA Regsirty or addressing errors therein. To be
	clear, changes and deprecations within the Performance Metrics		clear, changes and deprecations within the Performance Metrics
	Registry are not encouraged, and should be avoided to the extent		Registry are not encouraged, and should be avoided to the extent
	possible. However, in recognition that change is inevitable, the		possible. However, in recognition that change is inevitable, the
	provisions of this section address the need for revisions.		provisions of this section address the need for revisions.
	Revisions are initiated by sending a candidate Registered Performance		Revisions are initiated by sending a candidate Registered Performance
	Metric definition to IANA, as in Section 8.1, identifying the		Metric definition to IANA, as in Section 8.1, identifying the
	existing Performance Metrics Registry entry, and explaining how and		existing Performance Metrics Registry entry, and explaining how and
	why the existing entry shuold be revised.		why the existing entry should be revised.
	The primary requirement in the definition of procedures for managing		The primary requirement in the definition of procedures for managing
	changes to existing Registered Performance Metrics is avoidance of		changes to existing Registered Performance Metrics is avoidance of
	measurement interoperability problems; the Performance Metric Experts		measurement interoperability problems; the Performance Metric Experts
	must work to maintain interoperability above all else. Changes to		must work to maintain interoperability above all else. Changes to
	Registered Performance Metrics may only be done in an interoperable		Registered Performance Metrics may only be done in an interoperable
	way; necessary changes that cannot be done in a way to allow		way; necessary changes that cannot be done in a way to allow
	interoperability with unchanged implementations MUST result in the		interoperability with unchanged implementations MUST result in the
	creation of a new Registered Performance Metric (with a new Name,		creation of a new Registered Performance Metric (with a new Name,
	replacing the RFCXXXXsecY portion of the name) and possibly the		replacing the RFCXXXXsecY portion of the name) and possibly the
	skipping to change at page 30, line 47 ¶		skipping to change at page 30, line 47 ¶
	[RFC2026] Bradner, S., "The Internet Standards Process -- Revision		[RFC2026] Bradner, S., "The Internet Standards Process -- Revision
	3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996,		3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996,
	<https://www.rfc-editor.org/info/rfc2026>.		<https://www.rfc-editor.org/info/rfc2026>.
	[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate		[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
	Requirement Levels", BCP 14, RFC 2119,		Requirement Levels", BCP 14, RFC 2119,
	DOI 10.17487/RFC2119, March 1997,		DOI 10.17487/RFC2119, March 1997,
	<https://www.rfc-editor.org/info/rfc2119>.		<https://www.rfc-editor.org/info/rfc2119>.
	[RFC2141] Moats, R., "URN Syntax", RFC 2141, DOI 10.17487/RFC2141,
	May 1997, <https://www.rfc-editor.org/info/rfc2141>.
	[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,		[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
	"Framework for IP Performance Metrics", RFC 2330,		"Framework for IP Performance Metrics", RFC 2330,
	DOI 10.17487/RFC2330, May 1998,		DOI 10.17487/RFC2330, May 1998,
	<https://www.rfc-editor.org/info/rfc2330>.		<https://www.rfc-editor.org/info/rfc2330>.
	[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform		[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
	Resource Identifier (URI): Generic Syntax", STD 66,		Resource Identifier (URI): Generic Syntax", STD 66,
	RFC 3986, DOI 10.17487/RFC3986, January 2005,		RFC 3986, DOI 10.17487/RFC3986, January 2005,
	<https://www.rfc-editor.org/info/rfc3986>.		<https://www.rfc-editor.org/info/rfc3986>.
	[RFC4148] Stephan, E., "IP Performance Metrics (IPPM) Metrics
	Registry", BCP 108, RFC 4148, DOI 10.17487/RFC4148, August
	2005, <https://www.rfc-editor.org/info/rfc4148>.
	[RFC6248] Morton, A., "RFC 4148 and the IP Performance Metrics
	(IPPM) Registry of Metrics Are Obsolete", RFC 6248,
	DOI 10.17487/RFC6248, April 2011,
	<https://www.rfc-editor.org/info/rfc6248>.
	[RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New		[RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New
	Performance Metric Development", BCP 170, RFC 6390,		Performance Metric Development", BCP 170, RFC 6390,
	DOI 10.17487/RFC6390, October 2011,		DOI 10.17487/RFC6390, October 2011,
	<https://www.rfc-editor.org/info/rfc6390>.		<https://www.rfc-editor.org/info/rfc6390>.
	[RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz,		[RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz,
	"IP Performance Metrics (IPPM) Standard Advancement		"IP Performance Metrics (IPPM) Standard Advancement
	Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March		Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March
	2012, <https://www.rfc-editor.org/info/rfc6576>.		2012, <https://www.rfc-editor.org/info/rfc6576>.
	skipping to change at page 31, line 48 ¶		skipping to change at page 31, line 34 ¶
	[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC		[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
	2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,		2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
	May 2017, <https://www.rfc-editor.org/info/rfc8174>.		May 2017, <https://www.rfc-editor.org/info/rfc8174>.
	12.2. Informative References		12.2. Informative References
	[I-D.ietf-ippm-initial-registry]		[I-D.ietf-ippm-initial-registry]
	Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza,		Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza,
	"Initial Performance Metrics Registry Entries", draft-		"Initial Performance Metrics Registry Entries", draft-
	ietf-ippm-initial-registry-10 (work in progress), March		ietf-ippm-initial-registry-11 (work in progress), March
	2019.		2019.
	[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
	Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679,
	September 1999, <https://www.rfc-editor.org/info/rfc2679>.
	[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip		[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
	Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,		Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
	September 1999, <https://www.rfc-editor.org/info/rfc2681>.		September 1999, <https://www.rfc-editor.org/info/rfc2681>.
	[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
	Metric for IP Performance Metrics (IPPM)", RFC 3393,
	DOI 10.17487/RFC3393, November 2002,
	<https://www.rfc-editor.org/info/rfc3393>.
	[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network		[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
	performance measurement with periodic streams", RFC 3432,		performance measurement with periodic streams", RFC 3432,
	DOI 10.17487/RFC3432, November 2002,		DOI 10.17487/RFC3432, November 2002,
	<https://www.rfc-editor.org/info/rfc3432>.		<https://www.rfc-editor.org/info/rfc3432>.
	[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.		[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
	Jacobson, "RTP: A Transport Protocol for Real-Time		Jacobson, "RTP: A Transport Protocol for Real-Time
	Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,		Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
	July 2003, <https://www.rfc-editor.org/info/rfc3550>.		July 2003, <https://www.rfc-editor.org/info/rfc3550>.
	[RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,		[RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,
	"RTP Control Protocol Extended Reports (RTCP XR)",		"RTP Control Protocol Extended Reports (RTCP XR)",
	RFC 3611, DOI 10.17487/RFC3611, November 2003,		RFC 3611, DOI 10.17487/RFC3611, November 2003,
	<https://www.rfc-editor.org/info/rfc3611>.		<https://www.rfc-editor.org/info/rfc3611>.
	[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session		[RFC4148] Stephan, E., "IP Performance Metrics (IPPM) Metrics
	Description Protocol", RFC 4566, DOI 10.17487/RFC4566,		Registry", BCP 108, RFC 4148, DOI 10.17487/RFC4148, August
	July 2006, <https://www.rfc-editor.org/info/rfc4566>.		2005, <https://www.rfc-editor.org/info/rfc4148>.
	[RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A.,		[RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A.,
	Grossglauser, M., and J. Rexford, "A Framework for Packet		Grossglauser, M., and J. Rexford, "A Framework for Packet
	Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474,		Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474,
	March 2009, <https://www.rfc-editor.org/info/rfc5474>.		March 2009, <https://www.rfc-editor.org/info/rfc5474>.
	[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.		[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
	Raspall, "Sampling and Filtering Techniques for IP Packet		Raspall, "Sampling and Filtering Techniques for IP Packet
	Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,		Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,
	<https://www.rfc-editor.org/info/rfc5475>.		<https://www.rfc-editor.org/info/rfc5475>.
	[RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G.		[RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G.
	Carle, "Information Model for Packet Sampling Exports",		Carle, "Information Model for Packet Sampling Exports",
	RFC 5477, DOI 10.17487/RFC5477, March 2009,		RFC 5477, DOI 10.17487/RFC5477, March 2009,
	<https://www.rfc-editor.org/info/rfc5477>.		<https://www.rfc-editor.org/info/rfc5477>.
	[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
	Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,
	March 2009, <https://www.rfc-editor.org/info/rfc5481>.
	[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
	"Network Time Protocol Version 4: Protocol and Algorithms
	Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010,
	<https://www.rfc-editor.org/info/rfc5905>.
	[RFC6035] Pendleton, A., Clark, A., Johnston, A., and H. Sinnreich,		[RFC6035] Pendleton, A., Clark, A., Johnston, A., and H. Sinnreich,
	"Session Initiation Protocol Event Package for Voice		"Session Initiation Protocol Event Package for Voice
	Quality Reporting", RFC 6035, DOI 10.17487/RFC6035,		Quality Reporting", RFC 6035, DOI 10.17487/RFC6035,
	November 2010, <https://www.rfc-editor.org/info/rfc6035>.		November 2010, <https://www.rfc-editor.org/info/rfc6035>.
	[RFC6776] Clark, A. and Q. Wu, "Measurement Identity and Information		[RFC6248] Morton, A., "RFC 4148 and the IP Performance Metrics
	Reporting Using a Source Description (SDES) Item and an		(IPPM) Registry of Metrics Are Obsolete", RFC 6248,
	RTCP Extended Report (XR) Block", RFC 6776,		DOI 10.17487/RFC6248, April 2011,
	DOI 10.17487/RFC6776, October 2012,		<https://www.rfc-editor.org/info/rfc6248>.
	<https://www.rfc-editor.org/info/rfc6776>.
	[RFC6792] Wu, Q., Ed., Hunt, G., and P. Arden, "Guidelines for Use
	of the RTP Monitoring Framework", RFC 6792,
	DOI 10.17487/RFC6792, November 2012,
	<https://www.rfc-editor.org/info/rfc6792>.
	[RFC6991] Schoenwaelder, J., Ed., "Common YANG Data Types",		[RFC6991] Schoenwaelder, J., Ed., "Common YANG Data Types",
	RFC 6991, DOI 10.17487/RFC6991, July 2013,		RFC 6991, DOI 10.17487/RFC6991, July 2013,
	<https://www.rfc-editor.org/info/rfc6991>.		<https://www.rfc-editor.org/info/rfc6991>.
	[RFC7003] Clark, A., Huang, R., and Q. Wu, Ed., "RTP Control
	Protocol (RTCP) Extended Report (XR) Block for Burst/Gap
	Discard Metric Reporting", RFC 7003, DOI 10.17487/RFC7003,
	September 2013, <https://www.rfc-editor.org/info/rfc7003>.
	[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model		[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model
	for IP Flow Information Export (IPFIX)", RFC 7012,		for IP Flow Information Export (IPFIX)", RFC 7012,
	DOI 10.17487/RFC7012, September 2013,		DOI 10.17487/RFC7012, September 2013,
	<https://www.rfc-editor.org/info/rfc7012>.		<https://www.rfc-editor.org/info/rfc7012>.
	[RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow		[RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow
	Selection Techniques", RFC 7014, DOI 10.17487/RFC7014,		Selection Techniques", RFC 7014, DOI 10.17487/RFC7014,
	September 2013, <https://www.rfc-editor.org/info/rfc7014>.		September 2013, <https://www.rfc-editor.org/info/rfc7014>.
	[RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,		[RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
End of changes. 32 change blocks.
	137 lines changed or deleted		102 lines changed or added
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/