< draft-ietf-ippm-ioam-data-08.txt   draft-ietf-ippm-ioam-data-09.txt >
ippm F. Brockners ippm F. Brockners
Internet-Draft S. Bhandari Internet-Draft S. Bhandari
Intended status: Standards Track C. Pignataro Intended status: Standards Track C. Pignataro
Expires: April 26, 2020 Cisco Expires: September 9, 2020 Cisco
H. Gredler H. Gredler
RtBrick Inc. RtBrick Inc.
J. Leddy J. Leddy
S. Youell S. Youell
JPMC JPMC
T. Mizrahi T. Mizrahi
Huawei Network.IO Innovation Lab Huawei Network.IO Innovation Lab
D. Mozes D. Mozes
P. Lapukhov P. Lapukhov
Facebook Facebook
R. Chang R. Chang
Barefoot Networks Barefoot Networks
D. Bernier D. Bernier
Bell Canada Bell Canada
J. Lemon J. Lemon
Broadcom Broadcom
October 24, 2019 March 08, 2020
Data Fields for In-situ OAM Data Fields for In-situ OAM
draft-ietf-ippm-ioam-data-08 draft-ietf-ippm-ioam-data-09
Abstract Abstract
In-situ Operations, Administration, and Maintenance (IOAM) records In-situ Operations, Administration, and Maintenance (IOAM) records
operational and telemetry information in the packet while the packet operational and telemetry information in the packet while the packet
traverses a path between two points in the network. This document traverses a path between two points in the network. This document
discusses the data fields and associated data types for in-situ OAM. discusses the data fields and associated data types for in-situ OAM.
In-situ OAM data fields can be embedded into a variety of transports In-situ OAM data fields can be encapsulated into a variety of
such as NSH, Segment Routing, Geneve, native IPv6 (via extension protocols such as NSH, Segment Routing, Geneve, IPv6 (via extension
header), or IPv4. In-situ OAM can be used to complement OAM header), or IPv4. In-situ OAM can be used to complement OAM
mechanisms based on e.g. ICMP or other types of probe packets. mechanisms based on e.g. ICMP or other types of probe packets.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 26, 2020. This Internet-Draft will expire on September 9, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Scope, Applicability, and Assumptions . . . . . . . . . . . . 4 3. Scope, Applicability, and Assumptions . . . . . . . . . . . . 4
4. IOAM Data-Fields, Types, Nodes . . . . . . . . . . . . . . . 5 4. IOAM Data-Fields, Types, Nodes . . . . . . . . . . . . . . . 6
4.1. IOAM Data-Fields and Option-Types . . . . . . . . . . . . 5 4.1. IOAM Data-Fields and Option-Types . . . . . . . . . . . . 6
4.2. IOAM-Domains and types of IOAM Nodes . . . . . . . . . . 6 4.2. IOAM-Domains and types of IOAM Nodes . . . . . . . . . . 6
4.3. IOAM-Namespaces . . . . . . . . . . . . . . . . . . . . . 7 4.3. IOAM-Namespaces . . . . . . . . . . . . . . . . . . . . . 8
4.4. IOAM Trace Option-Types . . . . . . . . . . . . . . . . . 9 4.4. IOAM Trace Option-Types . . . . . . . . . . . . . . . . . 10
4.4.1. Pre-allocated and Incremental Trace Option-Types . . 12 4.4.1. Pre-allocated and Incremental Trace Option-Types . . 12
4.4.2. IOAM node data fields and associated formats . . . . 15 4.4.2. IOAM node data fields and associated formats . . . . 16
4.4.3. Examples of IOAM node data . . . . . . . . . . . . . 21 4.4.3. Examples of IOAM node data . . . . . . . . . . . . . 22
4.5. IOAM Proof of Transit Option-Type . . . . . . . . . . . . 23 4.5. IOAM Proof of Transit Option-Type . . . . . . . . . . . . 24
4.5.1. IOAM Proof of Transit Type 0 . . . . . . . . . . . . 25 4.5.1. IOAM Proof of Transit Type 0 . . . . . . . . . . . . 26
4.6. IOAM Edge-to-Edge Option-Type . . . . . . . . . . . . . . 26 4.6. IOAM Edge-to-Edge Option-Type . . . . . . . . . . . . . . 27
5. Timestamp Formats . . . . . . . . . . . . . . . . . . . . . . 28 5. Timestamp Formats . . . . . . . . . . . . . . . . . . . . . . 29
5.1. PTP Truncated Timestamp Format . . . . . . . . . . . . . 28 5.1. PTP Truncated Timestamp Format . . . . . . . . . . . . . 29
5.2. NTP 64-bit Timestamp Format . . . . . . . . . . . . . . . 29 5.2. NTP 64-bit Timestamp Format . . . . . . . . . . . . . . . 30
5.3. POSIX-based Timestamp Format . . . . . . . . . . . . . . 30 5.3. POSIX-based Timestamp Format . . . . . . . . . . . . . . 32
6. IOAM Data Export . . . . . . . . . . . . . . . . . . . . . . 32 6. IOAM Data Export . . . . . . . . . . . . . . . . . . . . . . 33
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33
7.1. Creation of a new In-Situ OAM Protocol Parameters 7.1. Creation of a new In-Situ OAM Protocol Parameters
Registry (IOAM) Protocol Parameters IANA registry . . . . 32 Registry (IOAM) Protocol Parameters IANA registry . . . . 33
7.2. IOAM Option-Type Registry . . . . . . . . . . . . . . . . 33 7.2. IOAM Option-Type Registry . . . . . . . . . . . . . . . . 34
7.3. IOAM Trace-Type Registry . . . . . . . . . . . . . . . . 33 7.3. IOAM Trace-Type Registry . . . . . . . . . . . . . . . . 34
7.4. IOAM Trace-Flags Registry . . . . . . . . . . . . . . . . 34 7.4. IOAM Trace-Flags Registry . . . . . . . . . . . . . . . . 35
7.5. IOAM POT-Type Registry . . . . . . . . . . . . . . . . . 34 7.5. IOAM POT-Type Registry . . . . . . . . . . . . . . . . . 35
7.6. IOAM POT-Flags Registry . . . . . . . . . . . . . . . . . 34 7.6. IOAM POT-Flags Registry . . . . . . . . . . . . . . . . . 36
7.7. IOAM E2E-Type Registry . . . . . . . . . . . . . . . . . 35 7.7. IOAM E2E-Type Registry . . . . . . . . . . . . . . . . . 36
7.8. IOAM Namespace-ID Registry . . . . . . . . . . . . . . . 35 7.8. IOAM Namespace-ID Registry . . . . . . . . . . . . . . . 36
8. Security Considerations . . . . . . . . . . . . . . . . . . . 35 8. Security Considerations . . . . . . . . . . . . . . . . . . . 37
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 36 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 38
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 37 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 38
10.1. Normative References . . . . . . . . . . . . . . . . . . 37 10.1. Normative References . . . . . . . . . . . . . . . . . . 39
10.2. Informative References . . . . . . . . . . . . . . . . . 37 10.2. Informative References . . . . . . . . . . . . . . . . . 39
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 39 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 41
1. Introduction 1. Introduction
This document defines data fields for "in-situ" Operations, This document defines data fields for "in-situ" Operations,
Administration, and Maintenance (IOAM). In-situ OAM records OAM Administration, and Maintenance (IOAM). In-situ OAM records OAM
information within the packet while the packet traverses a particular information within the packet while the packet traverses a particular
network domain. The term "in-situ" refers to the fact that the OAM network domain. The term "in-situ" refers to the fact that the OAM
data is added to the data packets rather than is being sent within data is added to the data packets rather than is being sent within
packets specifically dedicated to OAM. IOAM is to complement packets specifically dedicated to OAM. IOAM is to complement
mechanisms such as Ping or Traceroute, or more recent active probing mechanisms such as Ping or Traceroute. In terms of "active" or
mechanisms as described in [I-D.lapukhov-dataplane-probe]. In terms "passive" OAM, "in-situ" OAM can be considered a hybrid OAM type.
of "active" or "passive" OAM, "in-situ" OAM can be considered a "In-situ" mechanisms do not require extra packets to be sent. IOAM
hybrid OAM type. "In-situ" mechanisms do not require extra packets adds information to the already available data packets and therefore
to be sent. IOAM adds information to the already available data cannot be considered passive. In terms of the classification given
packets and therefore cannot be considered passive. In terms of the in [RFC7799] IOAM could be portrayed as Hybrid Type 1. IOAM
classification given in [RFC7799] IOAM could be portrayed as Hybrid mechanisms can be leveraged where mechanisms using e.g. ICMP do not
Type 1. IOAM mechanisms can be leveraged where mechanisms using e.g. apply or do not offer the desired results, such as proving that a
ICMP do not apply or do not offer the desired results, such as certain traffic flow takes a pre-defined path, SLA verification for
proving that a certain traffic flow takes a pre-defined path, SLA the live data traffic, detailed statistics on traffic distribution
verification for the live data traffic, detailed statistics on paths in networks that distribute traffic across multiple paths, or
traffic distribution paths in networks that distribute traffic across scenarios in which probe traffic is potentially handled differently
multiple paths, or scenarios in which probe traffic is potentially from regular data traffic by the network devices.
handled differently from regular data traffic by the network devices.
IOAM use cases and mechanisms have expanded as this document matured,
resulting in additional flags and options that may trigger creation
of additional packets dedicated to OAM. The term IOAM continues to
be used for such mechanisms, in addition to the "in-situ" mechanisms
that motivated this terminology.
2. Conventions 2. Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
Abbreviations used in this document: Abbreviations used in this document:
E2E Edge to Edge E2E Edge to Edge
skipping to change at page 4, line 30 skipping to change at page 4, line 35
VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol
Extension [I-D.ietf-nvo3-vxlan-gpe] Extension [I-D.ietf-nvo3-vxlan-gpe]
3. Scope, Applicability, and Assumptions 3. Scope, Applicability, and Assumptions
IOAM deployment assumes a set of constraints, requirements, and IOAM deployment assumes a set of constraints, requirements, and
guiding principles which are described in this section. guiding principles which are described in this section.
Scope: This document defines the data fields and associated data Scope: This document defines the data fields and associated data
types for in-situ OAM. The in-situ OAM data field can be transported types for in-situ OAM. The in-situ OAM data field can be
by a variety of transport protocols, including NSH, Segment Routing, encapsulated in a variety of protocols, including NSH, Segment
Geneve, IPv6, or IPv4. Specification details for these different Routing, Geneve, IPv6, or IPv4. Specification details for these
transport protocols are outside the scope of this document. different protocols are outside the scope of this document.
Deployment domain (or scope) of in-situ OAM deployment: IOAM is a Deployment domain (or scope) of in-situ OAM deployment: IOAM is a
network domain focused feature, with "network domain" being a set of network domain focused feature, with "network domain" being a set of
network devices or entities within a single administration. For network devices or entities within a single administration. For
example, a network domain can include an enterprise campus using example, a network domain can include an enterprise campus using
physical connections between devices or an overlay network using physical connections between devices or an overlay network using
virtual connections / tunnels for connectivity between said devices. virtual connections / tunnels for connectivity between said devices.
A network domain is defined by its perimeter or edge. Designers of A network domain is defined by its perimeter or edge. Designers of
protocol encapsulations for IOAM must specify mechanisms to ensure protocol encapsulations for IOAM must specify mechanisms to ensure
that IOAM data stays within an IOAM domain. In addition, the that IOAM data stays within an IOAM domain. In addition, the
operator of such a domain is expected to put provisions in place to operator of such a domain is expected to put provisions in place to
ensure that IOAM data does not leak beyond the edge of an IOAM ensure that IOAM data does not leak beyond the edge of an IOAM domain
domain, e.g. using for example packet filtering methods. The using for example packet filtering methods. The operator should
operator should consider the potential operational impact of IOAM to consider the potential operational impact of IOAM to mechanisms such
mechanisms such as ECMP processing (e.g. load-balancing schemes as ECMP processing (e.g. load-balancing schemes based on packet
based on packet length could be impacted by the increased packet size length could be impacted by the increased packet size due to IOAM),
due to IOAM), path MTU (i.e. ensure that the MTU of all links within path MTU (i.e. ensure that the MTU of all links within a domain is
a domain is sufficiently large to support the increased packet size sufficiently large to support the increased packet size due to IOAM)
due to IOAM) and ICMP message handling (i.e. in case of a native IPv6 and ICMP message handling (i.e. in case of IPv6, IOAM support for
transport, IOAM support for ICMPv6 Echo Request/Reply could desired ICMPv6 Echo Request/Reply is desired which would translate into
which would translate into ICMPv6 extensions to enable IOAM-Data- ICMPv6 extensions to enable IOAM-Data-Fields to be copied from an
Fields to be copied from an Echo Request message to an Echo Reply Echo Request message to an Echo Reply message).
message).
IOAM control points: IOAM-Data-Fields are added to or removed from IOAM control points: IOAM-Data-Fields are added to or removed from
the live user traffic by the devices which form the edge of a domain. the live user traffic by the devices which form the edge of a domain.
Devices which form an IOAM-Domain can add, update or remove IOAM- Devices which form an IOAM-Domain can add, update or remove IOAM-
Data-Fields. Edge devices of an IOAM-Domain can be hosts or network Data-Fields. Edge devices of an IOAM-Domain can be hosts or network
devices. devices.
Traffic-sets that IOAM is applied to: IOAM can be deployed on all or Traffic-sets that IOAM is applied to: IOAM can be deployed on all or
only on subsets of the live user traffic. It SHOULD be possible to only on subsets of the live user traffic. Using IOAM on a selected
enable IOAM on a selected set of traffic (e.g., per interface, based set of traffic (e.g., per interface, based on an access control list
on an access control list or flow specification defining a specific or flow specification defining a specific set of traffic, etc.) could
set of traffic, etc.) The selected set of traffic can also be all be useful in deployments where the cost of processing IOAM-Data-
traffic. Fields by encapsulating, transit, or decapsulating node(s) might be a
concern from a performance or operational perspective. Thus limiting
the amount of traffic IOAM is applied to could be beneficial in some
deployments.
Encapsulation independence: Data formats for IOAM SHOULD be defined Encapsulation independence: The definition of IOAM-Data-Fields is
in a transport-independent manner. IOAM applies to a variety of independent from the protocols the IOAM-Data-Fields are encapsulated
encapsulating protocols. A definition of how IOAM-Data-Fields are into. IOAM-Data-Fields can be encapsulated into several
encapsulated into "parent" protocols, like NSH or IPv6 is outside the encapsulating protocols. The specification of how IOAM-Data-Fields
scope of this document. are encapsulated into "parent" protocols, like e.g., NSH or IPv6 is
outside the scope of this document.
Layering: If several encapsulation protocols (e.g., in case of Layering: If several encapsulation protocols (e.g., in case of
tunneling) are stacked on top of each other, IOAM-Data-Fields could tunneling) are stacked on top of each other, IOAM-Data-Fields could
be present at multiple layers. The behavior follows the ships-in- be present at multiple layers. The behavior follows the ships-in-
the-night model, i.e. IOAM-Data-Fields in one layer are independent the-night model, i.e. IOAM-Data-Fields in one layer are independent
from IOAM-Data-Fields in another layer. Layering allows operators to from IOAM-Data-Fields in another layer. Layering allows operators to
instrument the protocol layer they want to measure. The different instrument the protocol layer they want to measure. The different
layers could, but do not have to share the same IOAM encapsulation layers could, but do not have to share the same IOAM encapsulation
mechanisms. mechanisms.
skipping to change at page 9, line 37 skipping to change at page 10, line 4
Namespaces, in a way that at average, only every second hop Namespaces, in a way that at average, only every second hop
would be recorded by any device. To retrieve a full view of would be recorded by any device. To retrieve a full view of
the deployment, the captured IOAM-Data-Fields of the two IOAM- the deployment, the captured IOAM-Data-Fields of the two IOAM-
Namespaces need to be correlated. Namespaces need to be correlated.
* Assigning different IOAM Namespace-IDs to different sets of * Assigning different IOAM Namespace-IDs to different sets of
nodes or network partitions and using a separate instance of an nodes or network partitions and using a separate instance of an
IOAM-Option-Type for each Namespace-ID, a full trace for a flow IOAM-Option-Type for each Namespace-ID, a full trace for a flow
could be collected and constructed via partial traces from each could be collected and constructed via partial traces from each
IOAM-Option-Type in each of the packets in the flow. Example: IOAM-Option-Type in each of the packets in the flow. Example:
An operator could choose to group the devices of a domain into An operator could choose to group the devices of a domain into
two IOAM-Namespaces, in a way that each IOAM-Namespace is two IOAM-Namespaces, in a way that each IOAM-Namespace is
represented by one of two IOAM-Option-Types in the packet. represented by one of two IOAM-Option-Types in the packet.
Each node would record data only for the IOAM-Namespace that it Each node would record data only for the IOAM-Namespace that it
belongs to, ignoring the other IOAM-Option-Type with a IOAM- belongs to, ignoring the other IOAM-Option-Type with a IOAM-
Namespace to which it doesn't belong. To retrieve a full view Namespace to which it doesn't belong. To retrieve a full view
of the deployment, the captured IOAM-Data-Fields of the two of the deployment, the captured IOAM-Data-Fields of the two
IOAM-Namespaces need to be correlated. IOAM-Namespaces need to be correlated.
4.4. IOAM Trace Option-Types 4.4. IOAM Trace Option-Types
"IOAM tracing data" is expected to be collected at every IOAM transit "IOAM tracing data" is expected to be either collected at every IOAM
node that a packet traverses to ensure visibility into the entire transit node that a packet traverses to ensure visibility into the
path a packet takes within an IOAM-Domain. I.e., in a typical entire path a packet takes within an IOAM-Domain. I.e., in a typical
deployment all nodes in an IOAM-Domain would participate in IOAM and deployment all nodes in an IOAM-Domain would participate in IOAM and
thus be IOAM transit nodes, IOAM encapsulating or IOAM decapsulating thus be IOAM transit nodes, IOAM encapsulating or IOAM decapsulating
nodes. If not all nodes within a domain are IOAM capable, IOAM nodes. If not all nodes within a domain support IOAM functionality
tracing information (i.e., node data, see below) will only be as defined in this document, IOAM tracing information (i.e., node
collected on those nodes which are IOAM capable. Nodes which are not data, see below) will only be collected on those nodes which support
IOAM capable will forward the packet without any changes to the IOAM- IOAM functionality as defined in this document. Nodes which do not
Data-Fields. The maximum number of hops and the minimum path MTU of support IOAM functionality as defined in this document will forward
the IOAM domain is assumed to be known. the packet without any changes to the IOAM-Data-Fields. The maximum
number of hops and the minimum path MTU of the IOAM domain is assumed
to be known.
To optimize hardware and software implementations IOAM tracing is To optimize hardware and software implementations IOAM tracing is
defined as two separate options. Any deployment MAY choose to defined as two separate options. Any deployment MAY choose to
configure and support one or both of the following options. configure and support one or both of the following options.
Pre-allocated Trace-Option: This trace option is defined as a Pre-allocated Trace-Option: This trace option is defined as a
container of node data fields (see below) with pre-allocated space container of node data fields (see below) with pre-allocated space
for each node to populate its information. This option is useful for each node to populate its information. This option is useful
for implementations where it is efficient to allocate the space for implementations where it is efficient to allocate the space
once and index into the array to populate the data during transit once and index into the array to populate the data during transit
skipping to change at page 12, line 15 skipping to change at page 12, line 29
o Information to detect whether IOAM trace data was added at every o Information to detect whether IOAM trace data was added at every
hop or whether certain hops in the domain weren't IOAM transit hop or whether certain hops in the domain weren't IOAM transit
nodes. nodes.
4.4.1. Pre-allocated and Incremental Trace Option-Types 4.4.1. Pre-allocated and Incremental Trace Option-Types
The IOAM Pre-allocated Trace-Option and the IOAM Incremental Trace- The IOAM Pre-allocated Trace-Option and the IOAM Incremental Trace-
Option have similar formats. Except where noted below, the internal Option have similar formats. Except where noted below, the internal
formats and fields of the two trace options are identical. Both formats and fields of the two trace options are identical. Both
Trace-Options consist of a fixed size "trace option header" and a Trace-Options consist of a fixed size "trace option header" and a
variable data space to store gathered data, the "node data list": variable data space to store gathered data, the "node data list". An
IOAM transit node (that is not an IOAM encapsulating node or IOAM
decapsulating node) MUST NOT modify any of the fields in the fixed
size "trace option header", other than "flags" and "RemainingLen",
i.e. an IOAM transit node MUST NOT modify the Namespace-ID, NodeLen,
IOAM-Trace-Type, or Reserved fields.
Pre-allocated and incremental trace option headers: Pre-allocated and incremental trace option headers:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Namespace-ID |NodeLen | Flags | RemainingLen| | Namespace-ID |NodeLen | Flags | RemainingLen|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IOAM-Trace-Type | Reserved | | IOAM-Trace-Type | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
skipping to change at page 13, line 4 skipping to change at page 13, line 36
~ ... ~ S ~ ... ~ S
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ p +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ p
| | a | | a
| node data list [n-1] | c | node data list [n-1] | c
| | e | | e
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | | | | |
| node data list [n] | | | node data list [n] | |
| | | | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
Namespace-ID: 16-bit identifier of an IOAM-Namespace. The Namespace-ID: 16-bit identifier of an IOAM-Namespace. The
Namespace-ID value of 0x0000 is defined as the default value and Namespace-ID value of 0x0000 is defined as the default value and
MUST be known to all the nodes implementing IOAM. For any other MUST be known to all the nodes implementing IOAM. For any other
Namespace-ID value that does not match any Namespace-ID the node Namespace-ID value that does not match any Namespace-ID the node
is configured to operate on, the node MUST NOT change the contents is configured to operate on, the node MUST NOT change the contents
of the IOAM-Data-Fields. of the IOAM-Data-Fields.
NodeLen: 5-bit unsigned integer. This field specifies the length of NodeLen: 5-bit unsigned integer. This field specifies the length of
data added by each node in multiples of 4-octets, excluding the data added by each node in multiples of 4-octets, excluding the
length of the "Opaque State Snapshot" field. length of the "Opaque State Snapshot" field.
If IOAM-Trace-Type bit 22 is not set, then NodeLen specifies the If IOAM-Trace-Type bit 22 is not set, then NodeLen specifies the
actual length added by each node. If IOAM-Trace-Type bit 22 is actual length added by each node. If IOAM-Trace-Type bit 22 is
set, then the actual length added by a node would be (NodeLen + set, then the actual length added by a node would be (NodeLen +
Opaque Data Length). length of the "Opaque State Snapshot" field) in 4 octet units.
For example, if 3 IOAM-Trace-Type bits are set and none of them For example, if 3 IOAM-Trace-Type bits are set and none of them
are wide, then NodeLen would be 3. If 3 IOAM-Trace-Type bits are are wide, then NodeLen would be 3. If 3 IOAM-Trace-Type bits are
set and 2 of them are wide, then NodeLen would be 5. set and 2 of them are wide, then NodeLen would be 5.
An IOAM encapsulating node must set NodeLen. An IOAM encapsulating node must set NodeLen.
A node receiving an IOAM Pre-allocated or Incremental Trace-Option A node receiving an IOAM Pre-allocated or Incremental Trace-Option
may rely on the NodeLen value, or it may ignore the NodeLen value may rely on the NodeLen value, or it may ignore the NodeLen value
and calculate the node length from the IOAM-Trace-Type bits (see and calculate the node length from the IOAM-Trace-Type bits (see
skipping to change at page 13, line 44 skipping to change at page 14, line 31
Bit 0 "Overflow" (O-bit) (most significant bit). This bit is set Bit 0 "Overflow" (O-bit) (most significant bit). This bit is set
by the network element if there are not enough octets left to by the network element if there are not enough octets left to
record node data, no field is added and the overflow "O-bit" record node data, no field is added and the overflow "O-bit"
must be set to "1" in the IOAM-Trace-Option header. This is must be set to "1" in the IOAM-Trace-Option header. This is
useful for transit nodes to ignore further processing of the useful for transit nodes to ignore further processing of the
option. option.
RemainingLen: 7-bit unsigned integer. This field specifies the data RemainingLen: 7-bit unsigned integer. This field specifies the data
space in multiples of 4-octets remaining for recording the node space in multiples of 4-octets remaining for recording the node
data, before the node data list is considered to have overflowed. data, before the node data list is considered to have overflowed.
When RemainingLen reaches 0, nodes are no longer allowed to add Given that the sender knows the minimum path MTU, the sender MAY
node data. Given that the sender knows the minimum path MTU, the set the initial value of RemainingLen according to the number of
sender MAY set the initial value of RemainingLen according to the node data bytes allowed before exceeding the MTU. Subsequent
number of node data bytes allowed before exceeding the MTU. nodes can carry out a simple comparison between RemainingLen and
Subsequent nodes can carry out a simple comparison between NodeLen, along with the length of the "Opaque State Snapshot" if
RemainingLen and NodeLen, along with the length of the "Opaque applicable, to determine whether or not data can be added by this
State Snapshot" if applicable, to determine whether or not data node. When node data is added, the node MUST decrease
can be added by this node. When node data is added, the node MUST RemainingLen by the amount of data added. In the pre-allocated
decrease RemainingLen by the amount of data added. In the pre- trace option, RemainingLength is used to derive the offset in data
allocated trace option, RemainingLength is used to derive the space to record the node data element. Specifically, the
offset in data space to record the node data element. recording of the node data element would start from RemainingLen -
Specifically, the recording of the node data element would start NodeLen - sizeof(opaque snapshot) in 4 octet units.
from RemainingLen - NodeLen - sizeof(opaque snapshot) in 4 octet
units.
IOAM-Trace-Type: A 24-bit identifier which specifies which data IOAM-Trace-Type: A 24-bit identifier which specifies which data
types are used in this node data list. types are used in this node data list.
The IOAM-Trace-Type value is a bit field. The following bits are The IOAM-Trace-Type value is a bit field. The following bits are
defined in this document, with details on each bit described in defined in this document, with details on each bit described in
the Section 4.4.2. The order of packing the data fields in each the Section 4.4.2. The order of packing the data fields in each
node data element follows the bit order of the IOAM-Trace-Type node data element follows the bit order of the IOAM-Trace-Type
field, as follows: field, as follows:
Bit 0 (Most significant bit) When set indicates presence of Bit 0 (Most significant bit) When set indicates presence of
Hop_Lim and node_id in the node data. Hop_Lim and node_id (short format) in the node data.
Bit 1 When set indicates presence of ingress_if_id and Bit 1 When set indicates presence of ingress_if_id and
egress_if_id (short format) in the node data. egress_if_id (short format) in the node data.
Bit 2 When set indicates presence of timestamp seconds in the Bit 2 When set indicates presence of timestamp seconds in the
node data. node data.
Bit 3 When set indicates presence of timestamp subseconds in Bit 3 When set indicates presence of timestamp subseconds in
the node data. the node data.
skipping to change at page 15, line 32 skipping to change at page 16, line 18
Bit 22 When set indicates presence of variable length Opaque Bit 22 When set indicates presence of variable length Opaque
State Snapshot field. State Snapshot field.
Bit 23 Reserved: Must be set to zero upon transmission and Bit 23 Reserved: Must be set to zero upon transmission and
ignored upon receipt. ignored upon receipt.
Section 4.4.2 describes the IOAM-Data-Types and their formats. Section 4.4.2 describes the IOAM-Data-Types and their formats.
Within an IOAM-Domain possible combinations of these bits making Within an IOAM-Domain possible combinations of these bits making
the IOAM-Trace-Type can be restricted by configuration knobs. the IOAM-Trace-Type can be restricted by configuration knobs.
Reserved: 8-bits. Must be zero. Reserved: 8-bits. An IOAM encapsulating node MUST set the value to
zero upon transmission. IOAM transit nodes must ignore the
received value.
Node data List [n]: Variable-length field. The type of which is Node data List [n]: Variable-length field. This is a list of node
determined by the IOAM-Trace-Type bit representing the n-th node data elements where the content of each node data element is
data in the node data list. The node data list is encoded determined by the IOAM-Trace-Type. The order of packing the data
starting from the last node data of the path. The first element fields in each node data element follows the bit order of the
of the node data list (node data list [0]) contains the last node IOAM-Trace-Type field. Each node MUST prepend its node data
of the path while the last node data of the node data list (node element in front of the node data elements that it received, such
data list[n]) contains the first node data of the path traced. that the transmitted node data list begins with this node's data
Populating the node data list in this way ensures that the order element as the first populated element in the list. The last node
of node data list is the same for incremental and pre-allocated data element in this list is the node data of the first IOAM
trace options. In the pre-allocated trace option, the index capable node in the path. Populating the node data list in this
contained in RemainingLen identifies the offset for current active way ensures that the order of node data list is the same for
node data to be populated. incremental and pre-allocated trace options. In the pre-allocated
trace option, the index contained in RemainingLen identifies the
offset for current active node data to be populated.
4.4.2. IOAM node data fields and associated formats 4.4.2. IOAM node data fields and associated formats
All the IOAM-Data-Fields MUST be 4-octet aligned. If a node which is All the IOAM-Data-Fields MUST be 4-octet aligned. If a node which is
supposed to update an IOAM-Data-Field is not capable of populating supposed to update an IOAM-Data-Field is not capable of populating
the value of a field set in the IOAM-Trace-Type, the field value MUST the value of a field set in the IOAM-Trace-Type, the field value MUST
be set to 0xFFFFFFFF for 4-octet fields or 0xFFFFFFFFFFFFFFFF for be set to 0xFFFFFFFF for 4-octet fields or 0xFFFFFFFFFFFFFFFF for
8-octet fields, indicating that the value is not populated, except 8-octet fields, indicating that the value is not populated, except
when explicitly specified in the field description below. when explicitly specified in the field description below.
skipping to change at page 16, line 18 skipping to change at page 17, line 8
IOAM-Namespace specific data, are defined in both "short format" as IOAM-Namespace specific data, are defined in both "short format" as
well as "wide format". Their use is not exclusive. A deployment well as "wide format". Their use is not exclusive. A deployment
could choose to leverage both. For example, ingress_if_id_(short could choose to leverage both. For example, ingress_if_id_(short
format) could be an identifier for the physical interface, whereas format) could be an identifier for the physical interface, whereas
ingress_if_id_(wide format) could be an identifier for a logical sub- ingress_if_id_(wide format) could be an identifier for a logical sub-
interface of that physical interface. interface of that physical interface.
Data field and associated data type for each of the IOAM-Data-Fields Data field and associated data type for each of the IOAM-Data-Fields
is shown below: is shown below:
Hop_Lim and node_id: 4-octet field defined as follows: Hop_Lim and node_id short format: 4-octet field defined as follows:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Hop_Lim | node_id | | Hop_Lim | node_id |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit
value in the packet at the node that records this data. Hop value in the packet at the node that records this data. Hop
Limit information is used to identify the location of the node Limit information is used to identify the location of the node
in the communication path. This is copied from the lower in the communication path. This is copied from the lower
layer, e.g., TTL value in IPv4 header or hop limit field from layer, e.g., TTL value in IPv4 header or hop limit field from
IPv6 header of the packet when the packet is ready for IPv6 header of the packet when the packet is ready for
transmission. The semantics of the Hop_Lim field depend on the transmission. The semantics of the Hop_Lim field depend on the
lower layer protocol that IOAM is encapsulated over, and lower layer protocol that IOAM is encapsulated into, and
therefore its specific semantics are outside the scope of this therefore its specific semantics are outside the scope of this
memo. memo. The value of this field MUST be set to 0xff when the
lower level does not have a TTL/Hop limit equivalent field.
node_id: 3-octet unsigned integer. Node identifier field to node_id: 3-octet unsigned integer. Node identifier field to
uniquely identify a node within the IOAM-Namespace and uniquely identify a node within the IOAM-Namespace and
associated IOAM-Domain. The procedure to allocate, manage and associated IOAM-Domain. The procedure to allocate, manage and
map the node_ids is beyond the scope of this document. map the node_ids is beyond the scope of this document.
ingress_if_id and egress_if_id: 4-octet field defined as follows: ingress_if_id and egress_if_id: 4-octet field defined as follows:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
skipping to change at page 18, line 48 skipping to change at page 19, line 38
~ node_id (contd) | ~ node_id (contd) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit
value in the packet at the node that records this data. Hop value in the packet at the node that records this data. Hop
Limit information is used to identify the location of the node Limit information is used to identify the location of the node
in the communication path. This is copied from the lower layer in the communication path. This is copied from the lower layer
for e.g. TTL value in IPv4 header or hop limit field from IPv6 for e.g. TTL value in IPv4 header or hop limit field from IPv6
header of the packet. The semantics of the Hop_Lim field header of the packet. The semantics of the Hop_Lim field
depend on the lower layer protocol that IOAM is encapsulated depend on the lower layer protocol that IOAM is encapsulated
over, and therefore its specific semantics are outside the into, and therefore its specific semantics are outside the
scope of this memo. scope of this memo. The value of this field MUST be set to
0xff when the lower level does not have a TTL/Hop limit
equivalent field.
node_id: 7-octet unsigned integer. Node identifier field to node_id: 7-octet unsigned integer. Node identifier field to
uniquely identify a node within the IOAM-Namespace and uniquely identify a node within the IOAM-Namespace and
associated IOAM-Domain. The procedure to allocate, manage and associated IOAM-Domain. The procedure to allocate, manage and
map the node_ids is beyond the scope of this document. map the node_ids is beyond the scope of this document.
ingress_if_id and egress_if_id wide: 8-octet field defined as ingress_if_id and egress_if_id wide: 8-octet field defined as
follows: follows:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
skipping to change at page 19, line 37 skipping to change at page 20, line 32
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| namespace specific data ~ | namespace specific data ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ namespace specific data (contd) | ~ namespace specific data (contd) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
buffer occupancy: 4-octet unsigned integer field. This field buffer occupancy: 4-octet unsigned integer field. This field
indicates the current status of the occupancy of the common buffer indicates the current status of the occupancy of the common buffer
pool used by a set of queues. The units of this field depend on pool used by a set of queues. The units of this field may be
the equipment type and deployment and has to be interpreted within implementation specific. Hence, the units may need to be
the context of an IOAM-Namespace and/or node-id if used. interpreted within the context of an IOAM-Namespace and/or node-id
if used. The authors acknowledge that in some operational cases
there is a need for the units to be consistent across a packet
path through the network, hence recommend the implementations to
use standard unit such as Bytes.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| buffer occupancy | | buffer occupancy |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Checksum Complement: 4-octet node data which contains a 4-octet Checksum Complement: 4-octet node data which contains a 4-octet
Checksum Complement field. The Checksum Complement is useful when Checksum Complement field. The Checksum Complement is useful when
IOAM is transported over encapsulations that make use of a UDP IOAM is transported over encapsulations that make use of a UDP
transport, such as VXLAN-GPE or Geneve. Without the Checksum transport, such as VXLAN-GPE or Geneve. Without the Checksum
skipping to change at page 26, line 7 skipping to change at page 27, line 7
zero upon transmission and ignored upon receipt. zero upon transmission and ignored upon receipt.
Random: 64-bit Per packet Random number. Random: 64-bit Per packet Random number.
Cumulative: 64-bit Cumulative that is updated at specific nodes by Cumulative: 64-bit Cumulative that is updated at specific nodes by
processing per packet Random number field and configured processing per packet Random number field and configured
parameters. parameters.
Note: Larger or smaller sizes of "Random" and "Cumulative" data are Note: Larger or smaller sizes of "Random" and "Cumulative" data are
feasible and could be required for certain deployments (e.g. in case feasible and could be required for certain deployments (e.g. in case
of space constraints in the transport protocol used). Future of space constraints in the encapsulation protocols used). Future
versions of this document will address different sizes of data for documents may address different sizes of data for "proof of transit".
"proof of transit".
4.6. IOAM Edge-to-Edge Option-Type 4.6. IOAM Edge-to-Edge Option-Type
The IOAM Edge-to-Edge Option-Type is to carry data that is added by The IOAM Edge-to-Edge Option-Type is to carry data that is added by
the IOAM encapsulating node and interpreted by IOAM decapsulating the IOAM encapsulating node and interpreted by IOAM decapsulating
node. The IOAM transit nodes MAY process the data but MUST NOT node. The IOAM transit nodes MAY process the data but MUST NOT
modify it. modify it.
The IOAM Edge-to-Edge Option-Type consist of a fixed size "IOAM Edge- The IOAM Edge-to-Edge Option-Type consist of a fixed size "IOAM Edge-
to-Edge Option-Type header" and "IOAM Edge-to-Edge Option-Type data to-Edge Option-Type header" and "IOAM Edge-to-Edge Option-Type data
skipping to change at page 27, line 21 skipping to change at page 28, line 18
of packets. of packets.
Bit 1 When set indicates presence of a 32-bit sequence number Bit 1 When set indicates presence of a 32-bit sequence number
added to a specific "packet group" which is used to added to a specific "packet group" which is used to
detect packet loss, packet reordering, or packet detect packet loss, packet reordering, or packet
duplication within that group. The "packet group" is duplication within that group. The "packet group" is
deployment dependent and defined at the IOAM deployment dependent and defined at the IOAM
encapsulating node e.g. by n-tuple based classification encapsulating node e.g. by n-tuple based classification
of packets. of packets.
Bit 2 When set indicates presence of timestamp seconds for the Bit 2 When set indicates presence of timestamp seconds,
transmission of the frame. This 4-octet field has three representing the time at which the packet entered the
possible formats; based on either PTP [IEEE1588v2], NTP IOAM domain. Within the IOAM encapsulating node, the
[RFC5905], or POSIX [POSIX]. The three timestamp formats time that the timestamp is retrieved can depend on the
are specified in Section 5. In all three cases, the implementation. Some possibilities are: 1) the time at
Timestamp Seconds field contains the 32 most significant which the packet was received by the node, 2) the time at
bits of the timestamp format that is specified in which the packet was transmitted by the node, 3) when a
Section 5. If a node is not capable of populating this tunnel encapsulation is used, the point at which the
field, it assigns the value 0xFFFFFFFF. Note that this packet is encapsulated into the tunnel. Each
is a legitimate value that is valid for 1 second in implementation should document when the E2E timestamp
approximately 136 years; the analyzer should correlate that is going to be put in the packet is retrieved. This
several packets or compare the timestamp value to its own 4-octet field has three possible formats; based on either
time-of-day in order to detect the error indication. PTP [IEEE1588v2], NTP [RFC5905], or POSIX [POSIX]. The
three timestamp formats are specified in Section 5. In
all three cases, the Timestamp Seconds field contains the
32 most significant bits of the timestamp format that is
specified in Section 5. If a node is not capable of
populating this field, it assigns the value 0xFFFFFFFF.
Note that this is a legitimate value that is valid for 1
second in approximately 136 years; the analyzer should
correlate several packets or compare the timestamp value
to its own time-of-day in order to detect the error
indication.
Bit 3 When set indicates presence of timestamp subseconds for Bit 3 When set indicates presence of timestamp subseconds,
the transmission of the frame. This 4-octet field has representing the time at which the packet entered the
three possible formats; based on either PTP [IEEE1588v2], IOAM domain. This 4-octet field has three possible
NTP [RFC5905], or POSIX [POSIX]. The three timestamp formats; based on either PTP [IEEE1588v2], NTP [RFC5905],
formats are specified in Section 5. In all three cases, or POSIX [POSIX]. The three timestamp formats are
the Timestamp Subseconds field contains the 32 least specified in Section 5. In all three cases, the
Timestamp Subseconds field contains the 32 least
significant bits of the timestamp format that is significant bits of the timestamp format that is
specified in Section 5. If a node is not capable of specified in Section 5. If a node is not capable of
populating this field, it assigns the value 0xFFFFFFFF. populating this field, it assigns the value 0xFFFFFFFF.
Note that this is a legitimate value in the NTP format, Note that this is a legitimate value in the NTP format,
valid for approximately 233 picoseconds in every second. valid for approximately 233 picoseconds in every second.
If the NTP format is used the analyzer should correlate If the NTP format is used the analyzer should correlate
several packets in order to detect the error indication. several packets in order to detect the error indication.
Bit 4-15 Undefined. An IOAM encapsulating node Must set the value Bit 4-15 Undefined. An IOAM encapsulating node Must set the value
of these bits to zero upon transmission and ignore upon of these bits to zero upon transmission and ignore upon
receipt. receipt.
E2E Option data: Variable-length field. The type of which is E2E Option data: Variable-length field. The type of which is
skipping to change at page 34, line 27 skipping to change at page 35, line 37
7.4. IOAM Trace-Flags Registry 7.4. IOAM Trace-Flags Registry
This registry defines code points for each bit in the 4 bit flags for This registry defines code points for each bit in the 4 bit flags for
the Pre-allocated trace option and for the Incremental trace option the Pre-allocated trace option and for the Incremental trace option
defined in Section 4.4. The meaning of Bit 0 (the most significant defined in Section 4.4. The meaning of Bit 0 (the most significant
bit) for trace flags is defined in this document in Paragraph 3 of bit) for trace flags is defined in this document in Paragraph 3 of
Section 4.4.1: Section 4.4.1:
Bit 0 "Overflow" (O-bit) Bit 0 "Overflow" (O-bit)
Bit 1 - 3 are available for assignment via RFC Required process as
per [RFC8126].
7.5. IOAM POT-Type Registry 7.5. IOAM POT-Type Registry
This registry defines 256 code points to define IOAM POT Type for This registry defines 256 code points to define IOAM POT Type for
IOAM proof of transit option Section 4.5. The code point value 0 is IOAM proof of transit option Section 4.5. The code point value 0 is
defined in this document: defined in this document:
0: 16 Octet POT data 0: 16 Octet POT data
1 - 255 are available for assignment via RFC Required process as per 1 - 255 are available for assignment via RFC Required process as per
[RFC8126]. [RFC8126].
skipping to change at page 35, line 43 skipping to change at page 37, line 10
0x0001 - 0x7FFF: reserved for private use 0x0001 - 0x7FFF: reserved for private use
0x8000 - 0xFFFF: unassigned 0x8000 - 0xFFFF: unassigned
8. Security Considerations 8. Security Considerations
As discussed in [RFC7276], a successful attack on an OAM protocol in As discussed in [RFC7276], a successful attack on an OAM protocol in
general, and specifically on IOAM, can prevent the detection of general, and specifically on IOAM, can prevent the detection of
failures or anomalies, or create a false illusion of nonexistent failures or anomalies, or create a false illusion of nonexistent
ones. ones. In particular, these threats are applicable by compromising
the integrity of IOAM data, either by maliciously modifying IOAM
options in transit, or by injecting packets with maliciously
generated IOAM options
The Proof of Transit Option-Type (Section Section 4.5) is used for The Proof of Transit Option-Type (Section Section 4.5) is used for
verifying the path of data packets. The security considerations of verifying the path of data packets. The security considerations of
POT are further discussed in [I-D.ietf-sfc-proof-of-transit]. POT are further discussed in [I-D.ietf-sfc-proof-of-transit].
The data elements of IOAM can be used for network reconnaissance, From a confidentiality perspective, although IOAM options do not
contain user data, they can be used for network reconnaissance,
allowing attackers to collect information about network paths, allowing attackers to collect information about network paths,
performance, queue states, buffer occupancy and other information. performance, queue states, buffer occupancy and other information.
Note that in case IOAM is used in "immediate export" mode (reference Moreover, if IOAM data leaks from the IOAM domain it may enable
to be added in a future revision), the IOAM related trace information reconnaissance beyond the scope of the IOAM domain. Note that in
would not be available in the customer data packets, but would case IOAM is used in "Direct Exporting" mode
trigger export of packet related IOAM information at every node. [I-D.ioamteam-ippm-ioam-direct-export], the IOAM related trace
IOAM data export and securing IOAM data export is outside the scope information would not be available in the customer data packets, but
of this document. would trigger export of packet related IOAM information at every
node, thus restricting the potential threat to the management plane
and mitigating the leakage threat. IOAM data exporting and the way
it is secured is outside the scope of this document.
IOAM can be used as a means for implementing Denial of Service (DoS) IOAM can be used as a means for implementing Denial of Service (DoS)
attacks, or for amplifying them. For example, a malicious attacker attacks, or for amplifying them. For example, a malicious attacker
can add an IOAM header to packets in order to consume the resources can add an IOAM header to packets in order to consume the resources
of network devices that take part in IOAM or collectors that analyze of network devices that take part in IOAM or entities that receive,
the IOAM data. Another example is a packet length attack, in which collect or analyze the IOAM data. Another example is a packet length
an attacker pushes headers associated with IOAM Option-Types into attack, in which an attacker pushes headers associated with IOAM
data packets, causing these packets to be increased beyond the MTU Option-Types into data packets, causing these packets to be increased
size, resulting in fragmentation or in packet drops. beyond the MTU size, resulting in fragmentation or in packet drops.
Since IOAM options may include timestamps, if network devices use Since IOAM options may include timestamps, if network devices use
synchronization protocols then any attack on the time protocol synchronization protocols then any attack on the time protocol
[RFC7384] can compromise the integrity of the timestamp-related data [RFC7384] can compromise the integrity of the timestamp-related data
fields. fields.
At the management plane, attacks may be implemented by misconfiguring At the management plane, attacks may be implemented by misconfiguring
or by maliciously configuring IOAM-enabled nodes in a way that or by maliciously configuring IOAM-enabled nodes in a way that
enables other attacks. Thus, IOAM configuration should be secured in enables other attacks. Thus, IOAM configuration should be secured in
a way that authenticates authorized users and verifies the integrity a way that authenticates authorized users and verifies the integrity
of configuration procedures. of configuration procedures.
Notably, IOAM is expected to be deployed in specific network domains, The current document does not define a specific IOAM encapsulation.
thus confining the potential attack vectors to within the network It should be noted that some IOAM encapsulation types may introduce
domain. Indeed, in order to limit the scope of threats to within the specific security considerations. A specification that defines an
current network domain the network operator is expected to enforce IOAM encapsulation is expected to address the respective
policies that prevent IOAM traffic from leaking outside of the IOAM encapsulation-specific security considerations.
domain, and prevent IOAM data from outside the domain to be processed
and used within the domain. Note that the Immediate Export mode Notably, in most cases IOAM is expected to be deployed in specific
(reference to be added in a future revision) can mitigate the network domains, thus confining the potential attack vectors to
potential threat of IOAM data leaking through data packets. within the network domain. A limited administrative domain provides
the operator with the means to select, monitor, and control the
access of all the network devices, making these devices trusted by
the operator. Indeed, in order to limit the scope of threats
mentioned above to within the current network domain the network
operator is expected to enforce policies that prevent IOAM traffic
from leaking outside of the IOAM domain, and prevent IOAM data from
outside the domain to be processed and used within the domain.
The security considerations of a system that deploys IOAM, much like
any system, should be reviewed on a per-deployment-scenario basis,
based on a systems-specific threat analysis, which may lead to
specific security solutions that are beyond the scope of the current
document. For example, in an IOAM deployment that is not confined to
a single LAN, but spans multiple inter-connected sites, the inter-
site links may be secured (e.g., by IPsec) in order to avoid external
threats.
9. Acknowledgements 9. Acknowledgements
The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari
Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya
Nadahalli, LJ Wobker, Erik Nordmark, Vengada Prasad Govindan, Andrew Nadahalli, LJ Wobker, Erik Nordmark, Vengada Prasad Govindan, Andrew
Yourtchenko, Aviv Kfir, Tianran Zhou and Zhenbin (Robin) for the Yourtchenko, Aviv Kfir, Tianran Zhou and Zhenbin (Robin) for the
comments and advice. comments and advice.
This document leverages and builds on top of several concepts This document leverages and builds on top of several concepts
described in [I-D.kitamura-ipv6-record-route]. The authors would described in [I-D.kitamura-ipv6-record-route]. The authors would
like to acknowledge the work done by the author Hiroshi Kitamura and like to acknowledge the work done by the author Hiroshi Kitamura and
people involved in writing it. people involved in writing it.
The authors would like to gracefully acknowledge useful review and The authors would like to gracefully acknowledge useful review and
insightful comments received from Joe Clarke, Al Morton, Tom Herbet, insightful comments received from Joe Clarke, Al Morton, Tom Herbert,
Haoyu song, and Mickey Spiegel. Haoyu Song, Mickey Spiegel and Barak Gafni.
The authors would like to acknowledge the contribution of "Immediate
export" of IOAM trace by Barak Gafni.
10. References 10. References
10.1. Normative References 10.1. Normative References
[IEEE1588v2] [IEEE1588v2]
Institute of Electrical and Electronics Engineers, "IEEE Institute of Electrical and Electronics Engineers, "IEEE
Std 1588-2008 - IEEE Standard for a Precision Clock Std 1588-2008 - IEEE Standard for a Precision Clock
Synchronization Protocol for Networked Measurement and Synchronization Protocol for Networked Measurement and
Control Systems", IEEE Std 1588-2008, 2008, Control Systems", IEEE Std 1588-2008, 2008,
<http://standards.ieee.org/findstds/ <http://standards.ieee.org/findstds/
standard/1588-2008.html>. standard/1588-2008.html>.
skipping to change at page 37, line 51 skipping to change at page 39, line 41
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26, Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017, RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>. <https://www.rfc-editor.org/info/rfc8126>.
10.2. Informative References 10.2. Informative References
[I-D.ietf-ntp-packet-timestamps] [I-D.ietf-ntp-packet-timestamps]
Mizrahi, T., Fabini, J., and A. Morton, "Guidelines for Mizrahi, T., Fabini, J., and A. Morton, "Guidelines for
Defining Packet Timestamps", draft-ietf-ntp-packet- Defining Packet Timestamps", draft-ietf-ntp-packet-
timestamps-07 (work in progress), August 2019. timestamps-08 (work in progress), February 2020.
[I-D.ietf-nvo3-geneve] [I-D.ietf-nvo3-geneve]
Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic
Network Virtualization Encapsulation", draft-ietf- Network Virtualization Encapsulation", draft-ietf-
nvo3-geneve-14 (work in progress), September 2019. nvo3-geneve-15 (work in progress), February 2020.
[I-D.ietf-nvo3-vxlan-gpe] [I-D.ietf-nvo3-vxlan-gpe]
Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol
Extension for VXLAN", draft-ietf-nvo3-vxlan-gpe-07 (work Extension for VXLAN", draft-ietf-nvo3-vxlan-gpe-09 (work
in progress), April 2019. in progress), December 2019.
[I-D.ietf-sfc-proof-of-transit] [I-D.ietf-sfc-proof-of-transit]
Brockners, F., Bhandari, S., Mizrahi, T., Dara, S., and S. Brockners, F., Bhandari, S., Mizrahi, T., Dara, S., and S.
Youell, "Proof of Transit", draft-ietf-sfc-proof-of- Youell, "Proof of Transit", draft-ietf-sfc-proof-of-
transit-03 (work in progress), September 2019. transit-04 (work in progress), November 2019.
[I-D.ioamteam-ippm-ioam-direct-export]
Song, H., Gafni, B., Zhou, T., Li, Z., Brockners, F.,
Bhandari, S., Sivakolundu, R., and T. Mizrahi, "In-situ
OAM Direct Exporting", draft-ioamteam-ippm-ioam-direct-
export-00 (work in progress), October 2019.
[I-D.kitamura-ipv6-record-route] [I-D.kitamura-ipv6-record-route]
Kitamura, H., "Record Route for IPv6 (PR6) Hop-by-Hop Kitamura, H., "Record Route for IPv6 (PR6) Hop-by-Hop
Option Extension", draft-kitamura-ipv6-record-route-00 Option Extension", draft-kitamura-ipv6-record-route-00
(work in progress), November 2000. (work in progress), November 2000.
[I-D.lapukhov-dataplane-probe] [I-D.lapukhov-dataplane-probe]
Lapukhov, P. and r. remy@barefootnetworks.com, "Data-plane Lapukhov, P. and r. remy@barefootnetworks.com, "Data-plane
probe for in-band telemetry collection", draft-lapukhov- probe for in-band telemetry collection", draft-lapukhov-
dataplane-probe-01 (work in progress), June 2016. dataplane-probe-01 (work in progress), June 2016.
 End of changes. 46 change blocks. 
178 lines changed or deleted 243 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/