]> Cisco Systems, Inc.

Canada rgandhi@cisco.com

Cisco Systems, Inc.

zali@cisco.com

Cisco Systems, Inc.

Belgium cf@cisco.com

Cisco Systems, Inc.

Hansaallee 249, 3rd Floor DUESSELDORF, NORDRHEIN-WESTFALEN 40549 Germany fbrockne@cisco.com

Comcast

Bin_Wen@cable.comcast.com

Comcast

Voitek_Kozak@comcast.com

MPLS Working Group In-situ Operations, Administration, and Maintenance (IOAM) records operational and telemetry information in the data packet while the packet traverses a path between two nodes in the network. This document defines how IOAM data fields are transported with MPLS data plane encapsulation using new Generic Associated Channel (G-ACh), including Segment Routing (SR) with MPLS data plane (SR-MPLS).

In-situ Operations, Administration, and Maintenance (IOAM) records operational and telemetry information within the packet while the packet traverses a particular network domain. The term "in-situ" refers to the fact that the IOAM data fields are added to the data packets rather than being sent within the probe packets specifically dedicated to OAM or Performance Measurement (PM). The IOAM data fields are defined in , and can be used for various use-cases for OAM and PM. The IOAM data fields are further updated in for direct export use-cases and in for Loopback and Active flags. This document defines how IOAM data fields are transported with MPLS data plane encapsulations using new Generic Associated Channel (G-ACh), including Segment Routing (SR) with MPLS data plane (SR-MPLS).

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in when, and only when, they appear in all capitals, as shown here.

Abbreviations used in this document: Equal Cost Multi-Path Generic Associated Channel In-situ Operations, Administration, and Maintenance Multiprotocol Label Switching Operations, Administration, and Maintenance Performance Measurement Proof-of-Transit Path Segment Identifier Segment Routing Segment Routing with MPLS Data plane

The IOAM data fields defined in are used. IOAM data fields are carried in the MPLS header as shown in Figure 1. More than one trace options can be present in the IOAM data fields. G-ACh provides a mechanism to transport OAM and other control messages over MPLS data plane. The IOAM G-ACh header with new IOAM G-ACh type is added immediately after the the MPLS label stack in the MPLS header as shown in Figure 1, before the IOAM data field(s).

The fields related to the encapsulation of IOAM data fields in the MPLS header are defined as follows: The first four octets are IP Version Field part of a G-ACh header . The Version field is set to 0, as defined in . Generic Associated Channel (G-ACh) Type (value TBA3) for IOAM . Reserved Bits MUST be set to zero upon transmission and ignored upon receipt. The Block Number can be used to aggregate the IOAM data collected in data plane, e.g. compute measurement metrics for each block of a flow. It is also used to correlate the IOAM data on different nodes. 8-bit field defining the IOAM Option type, as defined in Section 8.1 of . 8-bit unsigned integer. Length of the IOAM HDR in 4-octet units. IOAM option header and data is present as defined by the IOAM-OPT-Type field, and is defined in Section 5 of .

The E2E IOAM Indicator Label is used to indicate the presence of the E2E IOAM data field in the MPLS header as shown in Figure 2. If only edge nodes need to process IOAM data then E2E IOAM Indicator Label is used so that transit nodes can ignore it.

The Edge-to-Edge (E2E) IOAM includes IOAM Option-Type as Edge-to-Edge Option-Type . This section summarizes the procedure for data encapsulation and decapsulation for Edge-to-Edge IOAM in MPLS header. The encapsulating node inserts the E2E IOAM Indicator Label and one or more IOAM data field(s) in the MPLS header. The decapsulating node "punts the timestamped copy" of the received data packet as is including IOAM data fields when the node recognizes the IOAM Indicator Label. It is punted with receive timestamp to the slow path for IOAM processing. The receive timestamp is required by various E2E OAM use-cases, including streaming telemetry. Note that it is not necessarily punted to the control-plane. The decapsulating node processes the IOAM data field(s) using the procedures defined in . An example of IOAM processing is to export the data fields, send data fields via streaming telemetry, etc. The decapsulating node also pops the IOAM Indicator Label and the IOAM data fields from received packet. A copy of the decapsulated data packet is forwarded downstream or terminated locally similar to the regular data packets.

The E2E IOAM Indicator Label is used to indicate the presence of the E2E IOAM data field in the MPLS header. The E2E IOAM Indicator Label can be allocated using one of the following methods: Label assigned by IANA with value TBA1 from the Extended Special-Purpose MPLS Values . Label allocated by a Controller from the global table of the decapsulating node. The Controller provisions the label on both encapsulating and decapsulating nodes. Label allocated by the decapsulating node and signalled or advertised in the network. The signaling and/or advertisement extension for this is outside the scope of this document.

The HbH IOAM Indicator Label is used to indicate the presence of the HbH IOAM data field in the MPLS header as shown in Figure 3. Different IOAM Indicator Labels are used for E2E and HbH IOAM to optimize processing on transit nodes and for checking if IOAM data fields need to be processed on transit nodes. If only edge nodes need to process IOAM data then E2E IOAM Indicator Label is used so that transit nodes can ignore it. If both edge and transit nodes need to process IOAM data then HbH IOAM Indicator Label is used.

The Hop-by-Hop (HbH) IOAM includes IOAM Option-Types IOAM Pre-allocated Trace Option-Type, IOAM Incremental Trace Option-Type and IOAM Proof of Transit (POT) Option-Type . This section summarizes the procedure for data encapsulation and decapsulation for Hop-by-hop IOAM in MPLS header. The encapsulating node inserts the HbH IOAM Indicator Label and one or more IOAM data field(s) in the MPLS header. The intermediate node enabled with HbH IOAM functions processes the data packet including IOAM data fields as defined in when the node recognizes the HbH IOAM Indicator Label present in the MPLS header. The intermediate node may 'punt the timestamped copy' of the received data packet including the IOAM data fields as required by the IOAM data field processing. It is punted with receive timestamp to the slow path for IOAM processing. The intermediate node forwards a copy of the processsed data packet downstream. The decapsulating node "punts the timestamped copy" of the received data packet as is including IOAM data fields when the node recognizes the IOAM Indicator Label. It is punted with receive timestamp to the slow path for IOAM processing. The receive timestamp is required by various E2E OAM use-cases, including streaming telemetry. Note that it is not necessarily punted to the control-plane. The decapsulating node processes the IOAM data field(s) using the procedures defined in . An example of IOAM processing is to export the data fields, send data fields via streaming telemetry, etc. The decapsulating node also pops the IOAM Indicator Label and the IOAM data fields from received packet. A copy of the decapsulated data packet is forwarded downstream or terminated locally similar to the regular data packets.

The HbH IOAM Indicator Label is used to indicate the presence of the HbH IOAM data field in the MPLS header. The HbH IOAM Indicator Label can be allocated using one of the following methods: Label assigned by IANA with value TBA2 from the Extended Special-Purpose MPLS Values . Label allocated by a Controller from the network-wide global table. The Controller provisions the labels on all nodes participating in IOAM functions along the data traffic path.

The encapsulating node needs to make sure the IOAM data field does not start with a well known IP Version Number (e.g. 0x4 for IPv4 and 0x6 for IPv6) as it can alter the hashing function for ECMP that uses the IP header. This is achieved by using the IOAM G-ACh with IP Version Number 0001b after the MPLS label stack . Note that the hashing function for ECMP that uses the labels from the MPLS header may now include the IOAM Indicator Label. When entropy label is used for hashing function for ECMP, the procedure defined in this document does not alter the hashing function.

The decapsulating node that has to pop the IOAM Indicator Label, data fields, and perform the IOAM function may not be capable of supporting it. The encapsulating node needs to know if the decapsulating node can support the IOAM function. The signaling extension for this capability exchange is outside the scope of this document. The intermediate node that is not capable of supporting the IOAM functions defined in this document, can simply skip the IOAM processing of the MPLS header.

The SR path computation needs to know the Maximum SID Depth (MSD) that can be imposed at each node/link of a given SR path . This ensures that the SID stack depth of a computed path does not exceed the number of SIDs the node is capable of imposing. The MSD used for path computation MUST include the IOAM Indicator Label.

The data packets with IOAM data fields carry only one IOAM Indicator Label in the MPLS header. Any intermediate node that adds additional MPLS encapsulation in the MPLS header may further update the IOAM data fields in the header without inserting another IOAM Indicator Label.

Segment Routing (SR) technology leverages the source routing paradigm . A node steers a packet through a controlled set of instructions, called segments, by pre-pending the packet with an SR header. In the SR with MPLS data plane (SR-MPLS), the SR header is instantiated through a label stack. An example of data packet carrying the SR-MPLS header with Path Segment Identifier (PSID) and E2E IOAM encapsulation is shown in Figure 4. The Path Segment Identifier allows to identify the path associated with the data traffic being monitored for IOAM on the decapsulating node.

The security considerations of SR-MPLS are discussed in , and the security considerations of IOAM in general are discussed in . IOAM is considered a "per domain" feature, where one or several operators decide on leveraging and configuring IOAM according to their needs. Still, operators need to properly secure the IOAM domain to avoid malicious configuration and use, which could include injecting malicious IOAM packets into a domain. Routers that support G-ACh are subject to the same security considerations as defined in and .

IANA maintains the "Special-Purpose Multiprotocol Label Switching (MPLS) Label Values" registry (see ). IANA is requested to allocate IOAM Indicator Label value from the "Extended Special-Purpose MPLS Label Values" registry: Value Description Reference TBA1 E2E IOAM Indicator Label This document TBA2 HbH IOAM Indicator Label This document IANA maintains G-ACh Type Registry (see ). IANA is requested to allocate a value for IOAM G-ACh Type from "MPLS Generalized Associated Channel (G-ACh) Types (including Pseudowire Associated Channel Types)" registry. Value Description Reference TBA3 IOAM G-ACh Type This document

&RFC2119; &RFC4385; &RFC5586; &RFC8174; &RFC8660; &RFC8664; &I-D.ietf-ippm-ioam-direct-export; &I-D.ietf-ippm-ioam-data; &I-D.ietf-ippm-ioam-flags; &RFC6790; &I-D.ietf-mpls-spl-terminology; &I-D.ietf-spring-mpls-path-segment;

The authors would like to thank Patrick Khordoc, Shwetha Bhandari and Vengada Prasad Govindan for the discussions on IOAM. The authors would also like to thank Tarek Saad, Loa Andersson, Greg Mirsky, Stewart Bryant, and Cheng Li for providing many useful comments. The authors would also like to thank Mach Chen, Andrew Malis, Matthew Bocci, and Nick Delregno for the MPLS-RT reviews.