SRv6-INT: Protocol Extensions to Segment Routing over IPv6 for In-Band Network Telemetry in Support of Closed-Loop Resource Control

4.1. Problem Statement

An SRv6 packet already carries a Segment List in its SRH. If a separate telemetry header is inserted and each endpoint appends a telemetry record, the packet length increases as the packet traverses the path. This growth increases the risk of exceeding the maximum transmission unit (MTU) and adds data-plane processing overhead.¶

To avoid this growth, each SRv6-INT endpoint replaces its corresponding Segment List entry with the telemetry data it collects.¶

The collector then obtains the resulting telemetry data, which the local and global controllers use as feedback for closed-loop resource control.¶

4.2. Scope and Non-Goals

This document defines:¶

• The SRv6-INT protocol extension and associated control information;¶

• The reuse of Segment List entries to carry INT Metadata;¶

• The SRv6-INT processing behaviors defined in Section 6; and¶

• How the collector obtains telemetry data and how the local controller and global controller consume the data for closed-loop resource control.¶

This document does not define resource-control algorithms used by the local and global controllers, APIs used for resource control, or encapsulation formats and transport protocols for delivering telemetry data to the collector.¶

4.3. Applicability

SRv6-INT applies within an SRv6-INT domain. An SRv6-INT domain is a portion of an SRv6 network in which SRv6-INT is enabled. Only configured SRv6-INT ingress nodes and SRv6-INT endpoints perform SRv6-INT processing. Ordinary IPv6 transit nodes may forward packets between these nodes without examining or modifying SRv6-INT fields.¶

SRv6-INT can be applied to all application traffic or only to traffic selected according to a local policy. This document does not define the selection criteria. The amount of traffic subject to SRv6-INT can be limited according to device capacity, packet overhead, and operational constraints.¶

5.1. SRv6-INT Packet Format

This section specifies the SRv6-INT packet format and the additional semantics assigned to SRH fields. Except as specified in this document, SRH fields are used as defined in [RFC8754]. IPv6 packet processing follows [RFC8200].¶

The format and semantics of the SRv6-INT fields are the same whether the SRH is added during encapsulation or included when an IPv6 packet is originated [RFC8754].¶

Figure 1 presents the encapsulation context and the SRH format. The encapsulation in Figure 1(a) is schematic and is not drawn to scale. Figure 1(b) shows a Segment List entry after it has been reused for INT Metadata as specified in Section 5.2.¶

    +---------------------------------------------------------------+
    |                        Ethernet Header                        |
    +---------------------------------------------------------------+
    |                   IPv6 Header (Next Header = 43)              |
    +---------------------------------------------------------------+
    |            Segment Routing Header (expanded below)            |
    +---------------------------------------------------------------+
    |                         Inner Packet                          |
    +---------------------------------------------------------------+

                      (a) Encapsulation (schematic)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Next Header  |  Hdr Ext Len  | Routing Type  | Segments Left |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Last Entry   |     Flags     |              Tag              |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    |                                                               |
    |            Segment List[0] (128-bit IPv6 address)             |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    |                                                               |
    |            Segment List[1] (128-bit IPv6 address)             |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    |                                                               |
                                   ...
    |                                                               |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    |                       INT Metadata[0]                         |
    |               (replaced Segment List[Last Entry])             |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         SRv6-INT TLV                          |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   (b) Segment Routing Header (SRH)

where:¶

Next Header: Defined in Section 4.4 of [RFC8200].¶

Hdr Ext Len: Defined in Section 4.4 of [RFC8200].¶

Routing Type: 4.¶

Segments Left: Defined in Section 4.4 of [RFC8200].¶

Last Entry: In a complete Segment List, identifies the entry containing the first segment of the SR Policy, as specified in Section 2 of [RFC8754].¶

Flags: An 8-bit field defined in [RFC8754]. Section 5.1.1 defines the use of one bit for SRv6-INT.¶

Tag: A 16-bit field defined in [RFC8754]. Its use for SRv6-INT is defined in Section 5.1.1.¶

Segment List[0..n]: A list of 128-bit IPv6 addresses encoded in reverse SR Policy order, as specified in Section 2 of [RFC8754]. In a complete Segment List, Segment List[0] contains the last segment of the SR Policy.¶

SRv6-INT TLV: An SRH TLV defined in Section 5.1.1. It follows the Segment List and MUST be fully contained within the SRH, as determined by Hdr Ext Len, in accordance with Section 2.1 of [RFC8754].¶

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |     Type      |    Length     |            MapInfo            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                              PF                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

bit 0: Device ID
bit 1: Time Information
bit 2: Queue Information (Queue ID and Queue Length)
bit 3: Port Counter
bits 4-15: Reserved; MUST be zero on transmission and
           ignored on receipt

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Device ID   |                 Time Field 1                  ~
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    ~ Time Field 1  |                 Time Field 2                  ~
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    ~ Time Field 2  |Queue ID |          Queue Length               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         Port Counter                          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5.2. Segment List Entry Reuse

The endpoint associated with the SID in Segment List[0] uses Plain mode.¶

When an SRv6-INT endpoint receives a packet destined to a local SID, it reuses only the Segment List entry corresponding to that SID. Section 5.1.2 defines the INT Metadata format and field semantics.¶

In Plain mode, the endpoint writes INT Metadata into that entry once. In Service mode, the endpoint writes INT Metadata into the entry for transmission to the associated service node and updates the same entry after the packet returns. Section 6 specifies the write sequence.¶

The endpoint MUST NOT modify any other Segment List entry. Once it has completed SRv6-INT processing, the INT Metadata in the corresponding entry MUST NOT be modified or interpreted as a SID.¶

Entry reuse MUST NOT change the number of entries in the Segment List, the 128-bit length of each entry, Last Entry, Hdr Ext Len, or the order or value of any SID that remains to be processed.¶

6.1. SRv6-INT Processing Behaviors

S01. Construct an outer IPv6 header and an SRH containing the
     complete Segment List of the SR Policy applied to the packet.
     Set the outer IPv6 fields, except Payload Length, as specified
     by H.Encaps.
S02. Decrement the inner IPv6 Hop Limit or IPv4 TTL.

S03. Set the SRv6-INT Flag and initialize Tag to indicate Plain mode.
S04. Insert the SRv6-INT TLV, set MapInfo from local configuration,
     and initialize PF to 0.

S05. Update Hdr Ext Len and the outer IPv6 Payload Length.
S06. Submit the packet to IPv6 forwarding.

S01. Apply the checks in lines S05 through S11 of
     the End behavior in Section 4.1 of [RFC8986].

     Apply the checks in Section 6.1.1. Verify that
     0 < Segments Left <= Last Entry.
     If any check fails, handle the error according to
     Section 6.1.5 and stop processing.

S02. If (Tag does not indicate Service mode in the
     service-processing state) {
S03.     If the local SID is associated with a service node:
             Set Tag to indicate Service mode in the
             endpoint-processing state;
         Else:
             Set Tag to indicate Plain mode.
S04.     Initialize M to 0. Obtain Device ID, Queue ID, Queue
         Length, and Port Counter as selected by MapInfo, and store
         them in M.
S05. }

S06. If (Tag indicates Plain mode) {
S07.     If MapInfo selects Time Information:
             Time Field 1 = In_Time;
             Time Field 2 = Out_Time.

S08. } Else If (Tag indicates Service mode in the
                endpoint-processing state) {
S09.     If MapInfo selects Time Information:
             Time Field 1 = In_Time;
             Time Field 2 = Out_Time.
S10.     If a Service_Delay is stored locally for the local SID:
             PF = the Service_Delay stored for that SID.
S11.     Write M to Segment List[Segments Left].
S12.     Set Tag to indicate Service mode in the service-processing
         state; leave Segments Left and the IPv6 Destination Address
         unchanged.
S13.     Forward the packet to the service node associated with the
         local SID and stop processing.

S14. } Else If (Tag indicates Service mode in the
                service-processing state) {
S15.     Read Segment List[Segments Left] into M.
S16.     If MapInfo selects Time Information:
             Service_Delay = return packet arrival time
                             - Time Field 2;
             Hop_Delay = packet departure time from this endpoint
                        - Time Field 1;
             Time Field 1 = Service_Delay; Time Field 2 = Hop_Delay;
             Store Service_Delay locally for the local SID.
S17.     Set Tag to indicate Service mode in the endpoint-processing
         state and set PF to 0.
S18. }

S19. Write M to Segment List[Segments Left].
S20. Decrement the IPv6 Hop Limit and Segments Left.
S21. Set IPv6 Destination Address to Segment List[Segments Left].
S22. Submit the packet to the egress IPv6 FIB lookup using table T.

S01. Apply the checks in Section 6.1.1. Verify that
     Segments Left == 0 and that Tag does not indicate
     Service mode in the service-processing state.
     If any check fails, handle the error according to
     Section 6.1.5 and stop processing.

S02. Set Tag to indicate Plain mode.
S03. Obtain Device ID, In_Time, Out_Time, Queue ID, Queue Length, and
     Port Counter as selected by MapInfo, and store them in M.
S04. Write M to Segment List[0].
S05. Construct a packet containing the complete SRH and transfer it
     to the collector through a deployment-defined local interface.

S06. Remove the outer IPv6 header and its extension headers.
S07. Forward the inner packet according to the instance's local
     decapsulation and delivery context.

6.2. SRv6-INT Domain and Node Roles

Figure 2 shows an SRv6-INT domain. Packets steered by the SR Policy <S1, S2, S3> are processed by N1, N2, and N3 in that order. The nodes have the following roles:¶

• H is the SRv6-INT ingress/headend executing H.Encaps.INT.¶

• N1 executes End.T.INT for SID S1. No service node is associated with S1, so N1 uses Plain mode.¶

• N2 executes End.T.INT for SID S2. S2 is associated with the service node shown, so N2 uses Service mode.¶

• R is an ordinary IPv6 transit node between N2 and N3.¶

• N3 executes End.DT.INT for the final SID S3.¶

         +--------------- SRv6-INT Domain ----------------+
Original |                                                | Forwarded
 Packet  |                                                |  Packet
     --->[H]-->[N1]-->[N2]---->[R]--------->[N3]-------------->
         |              ^                    :^           |
         |              |                    ::           |
         |              v                    ::           |
         |       +--------------+            ::           |
         |       | Service Node |            ::           |
         |       +--------------+            ::           |
         |              :^                   ::           |
         |       Local  ::         Complete  ::           |
         |    Telemetry ::         Telemetry ::           |
         |              v: Config            v: Config    |
         |  +-------------------+  +-------------------+  |
         |  |    Collector /    |  |    Collector /    |  |
         |  | Local Controller  |  | Global Controller |  |
         |  +-------------------+  +-------------------+  |
         |                                                |
         +------------------------------------------------+

Solid arrows show packet forwarding, including delivery from N2 to the service node and return to N2. Dotted arrows show telemetry data provided by N2 and N3 to the corresponding collectors. Arrows labeled Config show resource configuration. A slash indicates that the collector and controller roles may be co-located.¶

The following requirements apply within an SRv6-INT domain:¶

• Each non-final SID in an SR Policy used for SRv6-INT MUST be instantiated at an SRv6-INT endpoint and bound to the End.T.INT behavior. The final SID MUST be instantiated at an SRv6-INT endpoint and bound to the End.DT.INT behavior. For each non-final SID, the deployment configuration MUST identify any associated service node.¶

• The collector MUST interpret the INT Metadata using information associated with the deployed SR Policy.¶

• SRv6-INT endpoints whose In_Time or Out_Time values are compared MUST use clocks synchronized to a common time reference. Service_Delay and Hop_Delay MAY be computed using the local clock of the endpoint that writes them.¶

In the following example, H sets MapInfo to select telemetry data categories. N2 has stored a Service_Delay measurement obtained from an earlier packet. A_H denotes the Source Address in the outer IPv6 header constructed by H. M1 and M3 denote the INT Metadata written by N1 and N3, respectively. T2 denotes the INT Metadata written by N2 before delivery to the service node; M2 denotes that INT Metadata after N2 updates its time fields when the packet returns.¶

The notation follows Section 6.1 of [RFC8754]. (A_H,S1) lists the outer IPv6 Source and Destination Addresses. (S3,S2,S1;SL=2) lists the Segment List in encoding order. Other header fields and the inner packet are omitted.¶

When H receives a selected packet, it executes H.Encaps.INT as specified in Section 6.1.2, sets Tag to indicate Plain mode, and forwards the packet:¶

P1: (A_H,S1)(S3,S2,S1;SL=2)

N1 executes End.T.INT in Plain mode. It sets Tag to indicate Plain mode, writes M1 to Segment List[2], and forwards the packet to N2:¶

P2: (A_H,S2)(S3,S2,M1;SL=1)

N2 executes End.T.INT in Service mode. Before delivering the packet to the service node, N2 sets Tag to indicate the endpoint-processing state, writes the stored Service_Delay into PF, and writes T2 to Segment List[1]. It then sets Tag to indicate the service-processing state. The IPv6 Destination Address and SL remain unchanged:¶

P3: (A_H,S2)(S3,T2,M1;SL=1)

When the packet is delivered to the service node, the collector reads the INT Metadata denoted by T2 as local telemetry data and obtains the Service_Delay measured for the earlier packet from PF.¶

The service node returns the packet to N2 without modifying the SRH. As specified in Section 6.1.3, N2 replaces Time Field 1 and Time Field 2 in T2 with Service_Delay and Hop_Delay, respectively, producing M2. N2 stores Service_Delay locally for S2, restores Tag to indicate the endpoint-processing state, and sets PF to 0. It then writes M2 to Segment List[1], sets SL to 0 and the IPv6 Destination Address to S3, and forwards the packet:¶

P4: (A_H,S3)(S3,M2,M1;SL=0)

R forwards the packet to N3 using ordinary IPv6 forwarding, leaving the fields shown above unchanged. N3 executes End.DT.INT as specified in Section 6.1.4. It sets Tag to indicate Plain mode and writes M3 to Segment List[0]. Before decapsulation, the packet is:¶

P5: (A_H,S3)(M3,M2,M1;SL=0)

N3 provides the collector with a packet containing the complete SRH, then removes the outer IPv6 header and its extension headers and forwards the inner packet. The collector obtains the INT Metadata sequence M1, M2, M3.¶

9.1. SRH Flag Assignment

The requested assignment from the "Segment Routing Header Flags" registry is:¶

Section 5.1.1 defines the flag semantics and identifies the value used for testing.¶

9.2. SRv6-INT TLV Type Assignment

The requested assignment from the mutable range (128-251) of the "Segment Routing Header TLVs" registry is:¶

Section 5.1.1 defines the TLV format and field mutability.¶

9.3. SRv6 Endpoint Behavior Assignments

The requested assignments from the "SRv6 Endpoint Behaviors" registry are:¶

No code point in this registry is requested for H.Encaps.INT.¶

The values used in the SRH Tag field are not SRv6 Endpoint Behavior code points, and this document requests no IANA action for them. Section 5 and Section 6 specify their semantics and when the field is updated.¶

10.1. Normative References

[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/rfc/rfc2119>.

[RFC8174]

Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

[RFC8200]

Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017, <https://www.rfc-editor.org/rfc/rfc8200>.

[RFC8754]

Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, <https://www.rfc-editor.org/rfc/rfc8754>.

[RFC8986]

Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, February 2021, <https://www.rfc-editor.org/rfc/rfc8986>.

10.2. Informative References

[INT]

P4.org Applications Working Group, "In-band Network Telemetry (INT) Dataplane Specification", Version 2.1, November 2020, <https://p4.org/wp-content/uploads/sites/53/p4-spec/docs/INT_v2_1.pdf>.

[RFC9197]

Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi, Ed., "Data Fields for In Situ Operations, Administration, and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197, May 2022, <https://www.rfc-editor.org/rfc/rfc9197>.

[Yan23]

Yan, X., Xu, Z., Chen, B., and Z. Zhu, "SRv6-INT: Runtime Monitoring for Green Service Function Chaining in B5G-MEC", IEEE, ICC 2023 - IEEE International Conference on Communications pp. 3145-3150, DOI 10.1109/ICC45041.2023.10278740, May 2023, <https://doi.org/10.1109/ICC45041.2023.10278740>.

[Yan24]

Yan, X., da Fonseca, N. L. S., and Z. Zhu, "Self-Adaptive SRv6-INT-Driven System Adjustment in Runtime for Reliable Service Function Chaining", Institute of Electrical and Electronics Engineers (IEEE), IEEE Transactions on Network and Service Management vol. 21, no. 5, pp. 4962-4973, DOI 10.1109/TNSM.2024.3404461, October 2024, <https://doi.org/10.1109/TNSM.2024.3404461>.