| < draft-mirsky-ippm-hybrid-two-step-06.txt | draft-mirsky-ippm-hybrid-two-step-07.txt > | |||
|---|---|---|---|---|
| IPPM Working Group G. Mirsky | IPPM Working Group G. Mirsky | |||
| Internet-Draft ZTE Corp. | Internet-Draft ZTE Corp. | |||
| Intended status: Standards Track W. Lingqiang | Intended status: Standards Track W. Lingqiang | |||
| Expires: April 29, 2021 G. Zhui | Expires: June 7, 2021 G. Zhui | |||
| ZTE Corporation | ZTE Corporation | |||
| H. Song | H. Song | |||
| Futurewei Technologies | Futurewei Technologies | |||
| October 26, 2020 | December 4, 2020 | |||
| Hybrid Two-Step Performance Measurement Method | Hybrid Two-Step Performance Measurement Method | |||
| draft-mirsky-ippm-hybrid-two-step-06 | draft-mirsky-ippm-hybrid-two-step-07 | |||
| Abstract | Abstract | |||
| Development of, and advancements in, automation of network operations | Development of, and advancements in, automation of network operations | |||
| brought new requirements for measurement methodology. Among them is | brought new requirements for measurement methodology. Among them is | |||
| the ability to collect instant network state as the packet being | the ability to collect instant network state as the packet being | |||
| processed by the networking elements along its path through the | processed by the networking elements along its path through the | |||
| domain. This document introduces a new hybrid measurement method, | domain. This document introduces a new hybrid measurement method, | |||
| referred to as hybrid two-step, as it separates the act of measuring | referred to as hybrid two-step, as it separates the act of measuring | |||
| and/or calculating the performance metric from the act of collecting | and/or calculating the performance metric from the act of collecting | |||
| skipping to change at page 1, line 41 ¶ | skipping to change at page 1, line 41 ¶ | |||
| Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
| Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
| working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
| Drafts is at https://datatracker.ietf.org/drafts/current/. | Drafts is at https://datatracker.ietf.org/drafts/current/. | |||
| Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
| and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
| time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
| material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
| This Internet-Draft will expire on April 29, 2021. | This Internet-Draft will expire on June 7, 2021. | |||
| Copyright Notice | Copyright Notice | |||
| Copyright (c) 2020 IETF Trust and the persons identified as the | Copyright (c) 2020 IETF Trust and the persons identified as the | |||
| document authors. All rights reserved. | document authors. All rights reserved. | |||
| This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
| Provisions Relating to IETF Documents | Provisions Relating to IETF Documents | |||
| (https://trustee.ietf.org/license-info) in effect on the date of | (https://trustee.ietf.org/license-info) in effect on the date of | |||
| publication of this document. Please review these documents | publication of this document. Please review these documents | |||
| carefully, as they describe your rights and restrictions with respect | carefully, as they describe your rights and restrictions with respect | |||
| to this document. Code Components extracted from this document must | to this document. Code Components extracted from this document must | |||
| include Simplified BSD License text as described in Section 4.e of | include Simplified BSD License text as described in Section 4.e of | |||
| the Trust Legal Provisions and are provided without warranty as | the Trust Legal Provisions and are provided without warranty as | |||
| described in the Simplified BSD License. | described in the Simplified BSD License. | |||
| Table of Contents | Table of Contents | |||
| 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | |||
| 2. Conventions used in this document . . . . . . . . . . . . . . 3 | 2. Conventions used in this document . . . . . . . . . . . . . . 3 | |||
| 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 | 2.1. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . 3 | |||
| 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 | 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 | |||
| 3. Problem Overview . . . . . . . . . . . . . . . . . . . . . . 4 | 3. Problem Overview . . . . . . . . . . . . . . . . . . . . . . 4 | |||
| 4. Theory of Operation . . . . . . . . . . . . . . . . . . . . . 5 | 4. Theory of Operation . . . . . . . . . . . . . . . . . . . . . 5 | |||
| 4.1. Operation of the HTS Ingress Node . . . . . . . . . . . . 6 | 4.1. Operation of the HTS Ingress Node . . . . . . . . . . . . 6 | |||
| 4.2. Operation of the HTS Intermediate Node . . . . . . . . . 8 | 4.2. Operation of the HTS Intermediate Node . . . . . . . . . 8 | |||
| 4.3. Operation of the HTS Egress Node . . . . . . . . . . . . 9 | 4.3. Operation of the HTS Egress Node . . . . . . . . . . . . 9 | |||
| 4.4. Considerations for HTS Timers . . . . . . . . . . . . . . 9 | 4.4. Considerations for HTS Timers . . . . . . . . . . . . . . 10 | |||
| 4.5. Deploying HTS in a Multicast Network . . . . . . . . . . 9 | 4.5. Deploying HTS in a Multicast Network . . . . . . . . . . 10 | |||
| 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 | 5. Authentication in HTS . . . . . . . . . . . . . . . . . . . . 10 | |||
| 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 | 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 | |||
| 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 11 | 6.1. IOAM Option-Type for HTS . . . . . . . . . . . . . . . . 12 | |||
| 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 | 6.2. HTS TLV Registry . . . . . . . . . . . . . . . . . . . . 12 | |||
| 8.1. Normative References . . . . . . . . . . . . . . . . . . 11 | 6.3. HTS Sub-TLV Type Sub-registry . . . . . . . . . . . . . . 12 | |||
| 8.2. Informative References . . . . . . . . . . . . . . . . . 11 | 6.4. HMAC Type Sub-registry . . . . . . . . . . . . . . . . . 13 | |||
| Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 | 7. Security Considerations . . . . . . . . . . . . . . . . . . . 14 | |||
| 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 | ||||
| 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 15 | ||||
| 9.1. Normative References . . . . . . . . . . . . . . . . . . 15 | ||||
| 9.2. Informative References . . . . . . . . . . . . . . . . . 15 | ||||
| Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 | ||||
| 1. Introduction | 1. Introduction | |||
| Successful resolution of challenges of automated network operation, | Successful resolution of challenges of automated network operation, | |||
| as part of, for example, overall service orchestration or data center | as part of, for example, overall service orchestration or data center | |||
| operation, relies on a timely collection of accurate information that | operation, relies on a timely collection of accurate information that | |||
| reflects the state of network elements on an unprecedented scale. | reflects the state of network elements on an unprecedented scale. | |||
| Because performing the analysis and act upon the collected | Because performing the analysis and act upon the collected | |||
| information requires considerable computing and storage resources, | information requires considerable computing and storage resources, | |||
| the network state information is unlikely to be processed by the | the network state information is unlikely to be processed by the | |||
| skipping to change at page 3, line 36 ¶ | skipping to change at page 3, line 41 ¶ | |||
| minimizing the overhead of the generated load in a network. HTS | minimizing the overhead of the generated load in a network. HTS | |||
| method extends the two-step mode of Residence Time Measurement (RTM) | method extends the two-step mode of Residence Time Measurement (RTM) | |||
| defined in [RFC8169] to on-path network state collection and | defined in [RFC8169] to on-path network state collection and | |||
| transport. HTS allows the collection of telemetry information from | transport. HTS allows the collection of telemetry information from | |||
| any arbitrary path, does not change data packets of the monitored | any arbitrary path, does not change data packets of the monitored | |||
| flow and makes the process of attribution of telemetry to the data | flow and makes the process of attribution of telemetry to the data | |||
| flow simple. | flow simple. | |||
| 2. Conventions used in this document | 2. Conventions used in this document | |||
| 2.1. Terminology | 2.1. Acronyms | |||
| RTM Residence Time Measurement | RTM Residence Time Measurement | |||
| ECMP Equal Cost Multipath | ECMP Equal Cost Multipath | |||
| MTU Maximum Transmission Unit | MTU Maximum Transmission Unit | |||
| HTS Hybrid Two-Step | HTS Hybrid Two-Step | |||
| HMAC Hashed Message Authentication Code | ||||
| Network telemetry - the process of collecting and reporting of | Network telemetry - the process of collecting and reporting of | |||
| network state | network state | |||
| 2.2. Requirements Language | 2.2. Requirements Language | |||
| The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", | The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", | |||
| "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and | "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and | |||
| "OPTIONAL" in this document are to be interpreted as described in BCP | "OPTIONAL" in this document are to be interpreted as described in BCP | |||
| 14 [RFC2119] [RFC8174] when, and only when, they appear in all | 14 [RFC2119] [RFC8174] when, and only when, they appear in all | |||
| capitals, as shown here. | capitals, as shown here. | |||
| 3. Problem Overview | 3. Problem Overview | |||
| Performance measurements are meant to provide data that characterize | Performance measurements are meant to provide data that characterize | |||
| conditions experienced by traffic flows in the network and possibly | conditions experienced by traffic flows in the network and possibly | |||
| trigger operational changes (e.g., re-route of flows, or changes in | trigger operational changes (e.g., re-route of flows, or changes in | |||
| resource allocations). Modifications to a network are determined | resource allocations). Modifications to a network are determined | |||
| based on the performance metric information available at the time | based on the performance metric information available when a change | |||
| that a change is to be made. The correctness of this determination | is to be made. The correctness of this determination is based on the | |||
| is based on the quality of the collected metrics data. The quality | quality of the collected metrics data. The quality of collected | |||
| of collected measurement data is defined by: | measurement data is defined by: | |||
| o the resolution and accuracy of each measurement; | o the resolution and accuracy of each measurement; | |||
| o predictability of both the time at which each measurement is made | o predictability of both the time at which each measurement is made | |||
| and the timeliness of measurement collection data delivery for | and the timeliness of measurement collection data delivery for | |||
| use. | use. | |||
| Consider the case of delay measurement that relies on collecting time | Consider the case of delay measurement that relies on collecting time | |||
| of packet arrival at the ingress interface and time of the packet | of packet arrival at the ingress interface and time of the packet | |||
| transmission at the egress interface. The method includes recording | transmission at the egress interface. The method includes recording | |||
| a local clock value on receiving the first octet of an affected | a local clock value on receiving the first octet of an affected | |||
| message at the device ingress, and again recording the clock value on | message at the device ingress, and again recording the clock value on | |||
| transmitting the first byte of the same message at the device egress. | transmitting the first byte of the same message at the device egress. | |||
| In this ideal case, the difference between the two recorded clock | In this ideal case, the difference between the two recorded clock | |||
| times corresponds to the time that the message spent in traversing | times corresponds to the time that the message spent in traversing | |||
| the device. In practice, the time that has been recorded can differ | the device. In practice, the time recorded can differ from the ideal | |||
| from the ideal case by any fixed amount and a correction can be | case by any fixed amount. A correction can be applied to compute the | |||
| applied to compute the same time difference taking into account the | same time difference taking into account the known fixed time | |||
| known fixed time associated with the actual measurement. In this | associated with the actual measurement. In this way, the resulting | |||
| way, the resulting time difference reflects any variable delay | time difference reflects any variable delay associated with queuing. | |||
| associated with queuing. | ||||
| Depending on the implementation, it may be a challenge to compute the | Depending on the implementation, it may be a challenge to compute the | |||
| difference between message arrival and departure times and - on the | difference between message arrival and departure times and - on the | |||
| fly - add the necessary residence time information to the same | fly - add the necessary residence time information to the same | |||
| message. And that task may become even more challenging if the | message. And that task may become even more challenging if the | |||
| packet is encrypted. Recording the departure of a packet time in the | packet is encrypted. Recording the departure of a packet time in the | |||
| same packet may be decremental to the accuracy of the measurement, | same packet may be decremental to the accuracy of the measurement | |||
| because the departure time includes the variable time component (such | because the departure time includes the variable time component (such | |||
| as that associated with buffering and queuing of the packet). A | as that associated with buffering and queuing of the packet). A | |||
| similar problem may lower the quality of, for example, information | similar problem may lower the quality of, for example, information | |||
| that characterizes utilization of the egress interface. If unable to | that characterizes utilization of the egress interface. If unable to | |||
| obtain the data consistently, without variable delays for additional | obtain the data consistently, without variable delays for additional | |||
| processing, information may not accurately reflect the egress | processing, information may not accurately reflect the egress | |||
| interface state. To mitigate this problem [RFC8169] defined an RTM | interface state. To mitigate this problem [RFC8169] defined an RTM | |||
| two-step mode. | two-step mode. | |||
| Another challenge associated with methods that collect network state | Another challenge associated with methods that collect network state | |||
| information into the actual data packet is the risk to exceed the | information into the actual data packet is the risk to exceed the | |||
| Maximum Transmission Unit (MTU) size, especially if the packet | Maximum Transmission Unit (MTU) size, especially if the packet | |||
| traverses overlay domains or VPNs. Since the fragmentation is not | traverses overlay domains or VPNs. Since the fragmentation is not | |||
| available at the transport network, operators may have to reduce MTU | available at the transport network, operators may have to reduce MTU | |||
| size advertised to client layer or risk missing network state data | size advertised to the client layer or risk missing network state | |||
| for the part, most probably the latter part, of the path. | data for the part, most probably the latter part, of the path. | |||
| 4. Theory of Operation | 4. Theory of Operation | |||
| The HTS method consists of the two phases: | The HTS method consists of two phases: | |||
| o performing a measurement or obtaining network state information, | o performing a measurement or obtaining network state information, | |||
| one or more than one type, on a node; | one or more than one type, on a node; | |||
| o collecting and transporting the measurement. | o collecting and transporting the measurement. | |||
| HTS uses HTS Trigger carried in a data packet or a specially | HTS uses HTS Trigger carried in a data packet or a specially | |||
| constructed test packet. For example, an HTS Trigger could be a | constructed test packet. For example, an HTS Trigger could be a | |||
| packet that includes iOAM Namespace-ID and IOAM-Trace-Type | packet that has IOAM Option-Type set to the "IOAM Hybrid Two-Step | |||
| information [I-D.ietf-ippm-ioam-data] or a packet in the flow to | Option-Type" value (TBA1) allocated by IANA (see Section 6.1). The | |||
| which the Alternate-Marking method [RFC8321] is applied. Nature of | HTS Trigger also includes IOAM Namespace-ID and IOAM-Trace-Type | |||
| the HTS Trigger is a transport network layer-specific, and its | information [I-D.ietf-ippm-ioam-data]. A packet in the flow to which | |||
| description is outside the scope of this document. The packet that | the Alternate-Marking method [RFC8321] is applied can be used as an | |||
| includes the HTS Trigger in this document is also referred to as the | HTS Trigger. The nature of the HTS Trigger is a transport network | |||
| trigger packet. | layer-specific, and its description is outside the scope of this | |||
| document. The packet that includes the HTS Trigger in this document | ||||
| is also referred to as the trigger packet. | ||||
| The HTS method uses the HTS Follow-up packet, in this document also | The HTS method uses the HTS Follow-up packet, referred to as the | |||
| referred to as the follow-up packet, to collect measurement and | follow-up packet, to collect measurement and network state data from | |||
| network state data from the nodes. The node that creates the HTS | the nodes. The node that creates the HTS Trigger also generates the | |||
| Trigger also generates the HTS Follow-up packet. The follow-up | HTS Follow-up packet. The follow-up packet contains characteristic | |||
| packet contains characteristic information, copied from the trigger | information, copied from the trigger packet, sufficient for | |||
| packet, sufficient for participating HTS nodes to associate it with | participating HTS nodes to associate it with the original packet. | |||
| the original packet. The exact composition of the characteristic | The exact composition of the characteristic information is specific | |||
| information is specific for each transport network, and its | for each transport network, and its definition is outside the scope | |||
| definition is outside the scope of this document. The follow-up | of this document. The follow-up packet also uses the same | |||
| packet also uses the same encapsulation as the data packet. If not | encapsulation as the data packet. If not payload but only network | |||
| payload but only network information used to load-balance flows in | information used to load-balance flows in equal cost multipath | |||
| equal cost multipath (ECMP), use of the network encapsulation | (ECMP), use of the network encapsulation identical to the trigger | |||
| identical to the trigger packet should guarantee that the follow-up | packet should guarantee that the follow-up packet remains in-band, | |||
| packet remains in-band, i.e., traverses the same set of network | i.e., traverses the same set of network elements, with the original | |||
| elements, with the original data packet with the HTS Trigger. Only | data packet with the HTS Trigger. Only one outstanding follow-up | |||
| one outstanding follow-up packet MUST be on the node for the given | packet MUST be on the node for the given path. That means that if | |||
| path. That means that if the node receives an HTS Trigger for the | the node receives an HTS Trigger for the flow on which it still waits | |||
| flow on which it still waits for the follow-up packet to the previous | for the follow-up packet to the previous HTS Trigger, the node will | |||
| HTS Trigger, the node will originate the follow-up packet to | originate the follow-up packet to transport the former set of the | |||
| transport the former set of the network state data and transmit it | network state data and transmit it before it sends the follow-up | |||
| before it sends the follow-up packet with the latest collection of | packet with the latest collection of network state information. | |||
| network state information. | ||||
| 4.1. Operation of the HTS Ingress Node | 4.1. Operation of the HTS Ingress Node | |||
| A node that originates the HTS Trigger is referred to as HTS ingress | A node that originates the HTS Trigger is referred to as the HTS | |||
| node. As stated, the ingress node originates the follow-up packet. | ingress node. As stated, the ingress node originates the follow-up | |||
| The follow-up packet has the transport network encapsulation | packet. The follow-up packet has the transport network encapsulation | |||
| identical with the trigger packet followed by the HTS shim and one or | identical with the trigger packet followed by the HTS shim and one or | |||
| more telemetry information elements encoded as Type-Length-Value | more telemetry information elements encoded as Type-Length-Value | |||
| {TLV}. Figure 1 displays the example of the follow-up packet format. | {TLV}. Figure 1 displays an example of the follow-up packet format. | |||
| 0 1 2 3 | 0 1 2 3 | |||
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | | | | | | |||
| ~ Transport Network ~ | ~ Transport Network ~ | |||
| | Encapsulation | | | Encapsulation | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| |Ver|HTS Shim Len| Flags | Sequence Number | | |Ver|HTS Shim Len| Flags | Sequence Number | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| skipping to change at page 7, line 13 ¶ | skipping to change at page 7, line 13 ¶ | |||
| bytes. | bytes. | |||
| 0 | 0 | |||
| 0 1 2 3 4 5 6 7 | 0 1 2 3 4 5 6 7 | |||
| +-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+ | |||
| |F| Reserved | | |F| Reserved | | |||
| +-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+ | |||
| Figure 2: Flags Field Format | Figure 2: Flags Field Format | |||
| Flags is eight-bits long field. The format of the Flags field | Flags is eight-bits long. The format of the Flags field displayed | |||
| displayed in Figure 2. | in Figure 2. | |||
| Full (F) flag MUST be set to zero by the node originating the | Full (F) flag MUST be set to zero by the node originating the | |||
| HTS follow-up packet and MUST be set to one by the node that | HTS follow-up packet and MUST be set to one by the node that | |||
| does not add its telemetry data to avoid exceeding MTU size. | does not add its telemetry data to avoid exceeding MTU size. | |||
| The node originating the follow-up packet MUST zero the | The node originating the follow-up packet MUST zero the | |||
| Reserved field and ignore it on the receipt. | Reserved field and ignore it on the receipt. | |||
| Sequence Number is 16 bits-long field. The zero-based value of | Sequence Number is 16 bits-long field. The zero-based value of | |||
| the field reflects the place of the HTS follow-up packet in the | the field reflects the place of the HTS follow-up packet in the | |||
| sequence of the HTS follow-up packets originated in response to | sequence of the HTS follow-up packets that originated in response | |||
| the same HTS trigger. The ingress node MUST set the value of the | to the same HTS trigger. The ingress node MUST set the value of | |||
| field to zero. | the field to zero. | |||
| Telemetry Data Profile is the optional variable length field of | Telemetry Data Profile is the optional variable-length field of | |||
| bit-size flags. Each flag indicates requested type of telemetry | bit-size flags. Each flag indicates the requested type of | |||
| data to be collected at the each HTS node. The increment of the | telemetry data to be collected at each HTS node. The increment of | |||
| field is four bytes with a minimum length of zero. For example, | the field is four bytes with a minimum length of zero. For | |||
| IOAM-Trace-Type information defined in [I-D.ietf-ippm-ioam-data] | example, IOAM-Trace-Type information defined in | |||
| can be used in the Telemetry Data Profile field. | [I-D.ietf-ippm-ioam-data] can be used in the Telemetry Data | |||
| Profile field. | ||||
| 0 1 2 3 | 0 1 2 3 | |||
| 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| | Type | Length | | | Type | Reserved | Length | | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| ~ Value ~ | ~ Value ~ | |||
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| Figure 3: Telemetry Data TLV Format | Figure 3: Telemetry Data TLV Format | |||
| Telemetry Data TLV is a variable-length field. Multiple TLVs MAY | Telemetry Data TLV is a variable-length field. Multiple TLVs MAY | |||
| be placed in an HTS packet. Additional TLVs may be enclosed | be placed in an HTS packet. Additional TLVs may be enclosed | |||
| within a given TLV, subject to the semantics of the (outer) TLV in | within a given TLV, subject to the semantics of the (outer) TLV in | |||
| question. Figure 3 presentes the format of a Telementry Data TLV, | question. Figure 3 presents the format of a Telemetry Data TLV, | |||
| where fields are defined as the following: | where fields are defined as the following: | |||
| Type - two-octet-long field that characterizes the | Type - a one-octet-long field that characterizes the | |||
| interpretation of the Value field. | interpretation of the Value field. | |||
| Reserved - one-octet-long field. | ||||
| Length - two-octet-long field equal to the length of the Value | Length - two-octet-long field equal to the length of the Value | |||
| field in octets. | field in octets. | |||
| Value - a variable-length field. Its interpretation and | Value - a variable-length field. The value of the Type field | |||
| encoding is determined by the value of the Type field. IOAM | determines its interpretation and encoding. IOAM data fields, | |||
| data fields, defined in [I-D.ietf-ippm-ioam-data], MAY be | defined in [I-D.ietf-ippm-ioam-data], MAY be carried in the | |||
| carried in the Value field. | Value field. | |||
| All multibyte fields defined in this specification are in network | All multibyte fields defined in this specification are in network | |||
| byte order. | byte order. | |||
| 4.2. Operation of the HTS Intermediate Node | 4.2. Operation of the HTS Intermediate Node | |||
| Upon receiving the trigger packet the HTS intermediate node MUST: | Upon receiving the trigger packet, the HTS intermediate node MUST: | |||
| o copy the transport information; | o copy the transport information; | |||
| o start the HTS Follow-up Timer for the obtained flow. | o start the HTS Follow-up Timer for the obtained flow. | |||
| Upon receiving the follow-up packet the HTS intermediate node MUST: | Upon receiving the follow-up packet, the HTS intermediate node MUST: | |||
| o verify that the matching transport information exists and the Full | o verify that the matching transport information exists and the Full | |||
| flag is cleared, then stop the associated HTS Follow-up timer; | flag is cleared, then stop the associated HTS Follow-up timer; | |||
| o collect telemetry data requested in the Telemetry Data Profile | o collect telemetry data requested in the Telemetry Data Profile | |||
| field or defined by the local HTS policy; | field or defined by the local HTS policy; | |||
| o if adding the collected telemetry would not exceed MTU, then | o if adding the collected telemetry would not exceed MTU, then | |||
| append data into Telemetry Data TLVs field and transmit the | append data as a new Telemetry Data TLV and transmit the follow-up | |||
| follow-up packet; | packet; | |||
| o otherwise, set the value of the Full flag to one and transmit the | o otherwise, set the value of the Full flag to one and transmit the | |||
| received a follow-up packet; | received a follow-up packet; | |||
| o originate the new follow-up packet using the same transport | o originate the new follow-up packet using the same transport | |||
| information. The value of the Sequence Number field in the HTS | information. The value of the Sequence Number field in the HTS | |||
| shim MUST be set to the value of the field in the received follow- | shim MUST be set to the value of the field in the received follow- | |||
| up packet incremented by one. Copy collected telemetry data and | up packet incremented by one; | |||
| transmit the packet. | ||||
| o copy collected telemetry data into the first Telemetry Data TLV's | ||||
| Value field and then transmit the packet. | ||||
| If the HTS Follow-up Timer expires, the intermediate node MUST: | If the HTS Follow-up Timer expires, the intermediate node MUST: | |||
| o originate the follow-up packet using transport information | o originate the follow-up packet using transport information | |||
| associated with the expired timer; | associated with the expired timer; | |||
| o initialize the HTS shim by setting Version field to 0b00 and | o initialize the HTS shim by setting Version field to 0b00 and | |||
| Sequence Number field to 0. Values of HTS Shim Length and | Sequence Number field to 0. Values of HTS Shim Length and | |||
| Telemetry Data Profile fields MAY be set according to the local | Telemetry Data Profile fields MAY be set according to the local | |||
| policy. | policy. | |||
| o copy telemetry information into Telemetry Data TLVs field and | o copy telemetry information into Telemetry Data TLV's Value field | |||
| transmit the packet. | and transmit the packet. | |||
| If the intermediate node receives a "late" follow-up packet, i.e., a | If the intermediate node receives a "late" follow-up packet, i.e., a | |||
| packet to which the node has no associated HTS Follow-up timer, the | packet to which the node has no associated HTS Follow-up timer, the | |||
| node MUST forward the "late" packet. | node MUST forward the "late" packet. | |||
| 4.3. Operation of the HTS Egress Node | 4.3. Operation of the HTS Egress Node | |||
| Upon receiving the trigger packet the HTS egress node MUST: | Upon receiving the trigger packet, the HTS egress node MUST: | |||
| o copy the transport information; | o copy the transport information; | |||
| o start the HTS Collection timer for the obtained flow. | o start the HTS Collection timer for the obtained flow. | |||
| When the egress node receives the follow-up packet for the known | When the egress node receives the follow-up packet for the known | |||
| flow, i.e., the flow to which the Collection timer is running, the | flow, i.e., the flow to which the Collection timer is running, the | |||
| node MUST: | node for each of Telemetry Data TLVs MUST: | |||
| o copy telemetry information; | o if HTS is used in the authenticated mode, verify the | |||
| authentication of the Telemetry Data TLV using the Authentication | ||||
| sub-TLV (see Section 5); | ||||
| o copy telemetry information from the Value field; | ||||
| o restart the corresponding Collection timer. | o restart the corresponding Collection timer. | |||
| When the Collection timer expires the egress relays the collected | When the Collection timer expires, the egress relays the collected | |||
| telemetry information for processing and analysis to a local or | telemetry information for processing and analysis to a local or | |||
| remote agent. | remote agent. | |||
| 4.4. Considerations for HTS Timers | 4.4. Considerations for HTS Timers | |||
| This specification defines two timers - HTS Follow-up and HTS | This specification defines two timers - HTS Follow-up and HTS | |||
| Collection. Because for the particular flow there MUST be not more | Collection. For the particular flow, there MUST be no more than one | |||
| than one HTS Trigger, values of HTS timers bounded by the rate of the | HTS Trigger, values of HTS timers bounded by the rate of the trigger | |||
| trigger generation for that flow. | generation for that flow. | |||
| 4.5. Deploying HTS in a Multicast Network | 4.5. Deploying HTS in a Multicast Network | |||
| Previous sections discussed the operation of HTS in a unicast | Previous sections discussed the operation of HTS in a unicast | |||
| network. Multicast services are important, and the ability to | network. Multicast services are important, and the ability to | |||
| collect telemetry information is an invaluable component in | collect telemetry information is invaluable in delivering a high | |||
| delivering a high quality of experience. While the replication of | quality of experience. While the replication of data packets is | |||
| data packets is necessary, replication of HTS follow-up packets is | necessary, replication of HTS follow-up packets is not. Replication | |||
| not. Replication of multicast data packets down a multicast tree may | of multicast data packets down a multicast tree may be set based on | |||
| be set based on multicast routing information or explicit information | multicast routing information or explicit information included in the | |||
| included in the special header, as, for example, in Bit-Indexed | special header, as, for example, in Bit-Indexed Explicit Replication | |||
| Explicit Replication [RFC8296]. A replicating node processes HTS | [RFC8296]. A replicating node processes the HTS packet as defined | |||
| packet as defined below: | below: | |||
| o the first transmitted multicast packet MUST be followed by the | o the first transmitted multicast packet MUST be followed by the | |||
| received corresponding HTS packet as described in Section 4.2; | received corresponding HTS packet as described in Section 4.2; | |||
| o each consecutively transmitted copy of the original multicast | o each consecutively transmitted copy of the original multicast | |||
| packet MUST be followed by the new HTS packet originated by the | packet MUST be followed by the new HTS packet originated by the | |||
| replicating node that acts as a intermediate HTS node when the HTS | replicating node that acts as an intermediate HTS node when the | |||
| Follow-up timer expired. | HTS Follow-up timer expired. | |||
| As a result, there are no duplicate copies of Telemetry Data TLV for | As a result, there are no duplicate copies of Telemetry Data TLV for | |||
| the same pair of ingress and egress interfaces. At the same time, | the same pair of ingress and egress interfaces. At the same time, | |||
| all ingress/egress pairs traversed by the given multicast packet | all ingress/egress pairs traversed by the given multicast packet | |||
| reflected in their respective Telemetry Data TLV. Consequently, a | reflected in their respective Telemetry Data TLV. Consequently, a | |||
| centralized controller would be able to reconstruct and analyze the | centralized controller would reconstruct and analyze the state of the | |||
| state of the particular multicast distribution tree based on HTS | particular multicast distribution tree based on HTS packets collected | |||
| packets collected from egress nodes. | from egress nodes. | |||
| 5. IANA Considerations | 5. Authentication in HTS | |||
| TBD | Telemetry information may be used to drive network operation, closing | |||
| the control loop for self-driving, self-healing networks. Thus it is | ||||
| critical to provide a mechanism to protect the telemetry information | ||||
| collected using the HTS method. This document defines an optional | ||||
| authentication of a Telemetry Data TLV that protects the collected | ||||
| information's integrity. | ||||
| 6. Security Considerations | The format of the Authentication sub-TLV is displayed in Figure 4. | |||
| Nodes that practice HTS method are presumed to share a trust model | 0 1 2 3 | |||
| that depends on the existence of a trusted relationship among nodes. | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
| This is necessary as these nodes are expected to correctly modify the | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| specific content of the data in the follow-up packet, and the degree | |Authentic. Type| HMAC Type | Length | | |||
| to which HTS measurement is useful for network operation depends on | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| this ability. In practice, this means either confidentiality or | | | | |||
| integrity protection cannot cover those portions of messages that | | Digest | | |||
| contain the network state data. Though there are methods that make | | | | |||
| it possible in theory to provide either or both such protections and | | | | |||
| still allow for intermediate nodes to make detectable yet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| authenticated modifications, such methods do not seem practical at | ||||
| present, particularly for protocols that used to measure latency and/ | ||||
| or jitter. | ||||
| The ability to potentially authenticate and/or encrypt the network | Figure 4: HMAC sub-TLV | |||
| state data for scenarios both with and without the participation of | ||||
| intermediate nodes that participate in HTS measurement is left for | where fields are defined as follows: | |||
| further study. | ||||
| o Authentication Type - is a one-octet-long field, value TBA2 | ||||
| allocated by IANA Section 6.2. | ||||
| o Length - two-octet-long field, set equal to the length of the | ||||
| Digest field in octets. | ||||
| o HMAC Type - is a one-octet-long field that identifies the type of | ||||
| the HMAC and the length of the digest and the length of the digest | ||||
| according to the HTS HMAC Type sub-registry (see Section 6.4). | ||||
| o Digest - is a variable-length field that carries HMAC digest of | ||||
| the text that includes the encompassing TLV. | ||||
| This specification defines the use of HMAC-SHA-256 truncated to 128 | ||||
| bits ([RFC4868]) in HTS. Future specifications may define the use in | ||||
| HTS of more advanced cryptographic algorithms or the use of digest of | ||||
| a different length. HMAC is calculated as defined in [RFC2104] over | ||||
| text as the concatenation of the Sequence Number field of the follow- | ||||
| up packet (see Figure 1) and the preceding data collected in the | ||||
| Telemetry Data TLV. The digest then MUST be truncated to 128 bits | ||||
| and written into the Digest field. Distribution and management of | ||||
| shared keys are outside the scope of this document. In the HTS | ||||
| authenticated mode, the Authentication sub-TLV MUST be present in | ||||
| each Telemetry Data TLV. HMAC MUST be verified before using any data | ||||
| in the included Telemetry Data TLV. If HMAC verification fails, the | ||||
| system MUST stop processing corresponding Telemetry Data TLV and | ||||
| notify an operator. Specification of the notification mechanism is | ||||
| outside the scope of this document. | ||||
| 6. IANA Considerations | ||||
| 6.1. IOAM Option-Type for HTS | ||||
| The IOAM Option-Type registry is requested in | ||||
| [I-D.ietf-ippm-ioam-data]. IANA is requested to allocate a new code | ||||
| point as listed in Table 1. | ||||
| +-------+----------------------------------+---------------+ | ||||
| | Value | Description | Reference | | ||||
| +-------+----------------------------------+---------------+ | ||||
| | TBA1 | IOAM Hybrid Two-Step Option-Type | This document | | ||||
| +-------+----------------------------------+---------------+ | ||||
| Table 1: IOAM Option-Type for HTS | ||||
| 6.2. HTS TLV Registry | ||||
| IANA is requested to create the HTS TLV Type registry. All code | ||||
| points in the range 1 through 175 in this registry shall be allocated | ||||
| according to the "IETF Review" procedure specified in [RFC8126]. | ||||
| Code points in the range 176 through 239 in this registry shall be | ||||
| allocated according to the "First Come First Served" procedure | ||||
| specified in [RFC8126]. The remaining code points are allocated | ||||
| according to Table 2: | ||||
| +-----------+--------------+---------------+ | ||||
| | Value | Description | Reference | | ||||
| +-----------+--------------+---------------+ | ||||
| | 0 | Reserved | This document | | ||||
| | 1- 175 | Unassigned | This document | | ||||
| | 176 - 239 | Unassigned | This document | | ||||
| | 240 - 251 | Experimental | This document | | ||||
| | 252 - 254 | Private Use | This document | | ||||
| | 255 | Reserved | This document | | ||||
| +-----------+--------------+---------------+ | ||||
| Table 2: HTS TLV Type Registry | ||||
| 6.3. HTS Sub-TLV Type Sub-registry | ||||
| IANA is requested to create the HTS sub-TLV Type sub-registry as part | ||||
| of the HTS TLV Type registry. All code points in the range 1 through | ||||
| 175 in this registry shall be allocated according to the "IETF | ||||
| Review" procedure specified in [RFC8126]. Code points in the range | ||||
| 176 through 239 in this registry shall be allocated according to the | ||||
| "First Come First Served" procedure specified in [RFC8126]. The | ||||
| remaining code points are allocated according to Table 3: | ||||
| +-----------+--------------+---------------+ | ||||
| | Value | Description | Reference | | ||||
| +-----------+--------------+---------------+ | ||||
| | 0 | Reserved | This document | | ||||
| | 1- 175 | Unassigned | This document | | ||||
| | 176 - 239 | Unassigned | This document | | ||||
| | 240 - 251 | Experimental | This document | | ||||
| | 252 - 254 | Private Use | This document | | ||||
| | 255 | Reserved | This document | | ||||
| +-----------+--------------+---------------+ | ||||
| Table 3: HTS Sub-TLV Type Sub-registry | ||||
| This document defines the following new values in the IETF Review | ||||
| range of the HTS sub-TLV Type sub-registry: | ||||
| +-------+-------------+----------+---------------+ | ||||
| | Value | Description | TLV Used | Reference | | ||||
| +-------+-------------+----------+---------------+ | ||||
| | TBA2 | HMAC | Any | This document | | ||||
| +-------+-------------+----------+---------------+ | ||||
| Table 4: HTS sub-TLV Types | ||||
| 6.4. HMAC Type Sub-registry | ||||
| IANA is requested to create the HMAC Type sub-registry as part of the | ||||
| HTS TLV Type registry. All code points in the range 1 through 127 in | ||||
| this registry shall be allocated according to the "IETF Review" | ||||
| procedure specified in [RFC8126]. Code points in the range 128 | ||||
| through 239 in this registry shall be allocated according to the | ||||
| "First Come First Served" procedure specified in [RFC8126]. The | ||||
| remaining code points are allocated according to Table 5: | ||||
| +-----------+--------------+---------------+ | ||||
| | Value | Description | Reference | | ||||
| +-----------+--------------+---------------+ | ||||
| | 0 | Reserved | This document | | ||||
| | 1- 127 | Unassigned | This document | | ||||
| | 128 - 239 | Unassigned | This document | | ||||
| | 240 - 249 | Experimental | This document | | ||||
| | 250 - 254 | Private Use | This document | | ||||
| | 255 | Reserved | This document | | ||||
| +-----------+--------------+---------------+ | ||||
| Table 5: HMAC Type Sub-registry | ||||
| This document defines the following new values in the HMAC Type sub- | ||||
| registry: | ||||
| +-------+-----------------------------+---------------+ | ||||
| | Value | Description | Reference | | ||||
| +-------+-----------------------------+---------------+ | ||||
| | 1 | HMAC-SHA-256 16 octets long | This document | | ||||
| +-------+-----------------------------+---------------+ | ||||
| Table 6: HMAC Types | ||||
| 7. Security Considerations | ||||
| Nodes that practice the HTS method are presumed to share a trust | ||||
| model that depends on the existence of a trusted relationship among | ||||
| nodes. This is necessary as these nodes are expected to correctly | ||||
| modify the specific content of the data in the follow-up packet, and | ||||
| the degree to which HTS measurement is useful for network operation | ||||
| depends on this ability. In practice, this means either | ||||
| confidentiality or integrity protection cannot cover those portions | ||||
| of messages that contain the network state data. Though there are | ||||
| methods that make it possible in theory to provide either or both | ||||
| such protections and still allow for intermediate nodes to make | ||||
| detectable yet authenticated modifications, such methods do not seem | ||||
| practical at present, particularly for protocols that used to measure | ||||
| latency and/or jitter. | ||||
| This document defines the use of authentication (Section 5) to | ||||
| protect the integrity of the telemetry information collected using | ||||
| the HTS method. Privacy protection can be achieved by, for example, | ||||
| sharing the IPsec tunnel with a data flow that generates information | ||||
| that is collected using HTS. | ||||
| While it is possible for a supposed compromised node to intercept and | While it is possible for a supposed compromised node to intercept and | |||
| modify the network state information in the follow-up packet, this is | modify the network state information in the follow-up packet; this is | |||
| an issue that exists for nodes in general - for all data that to be | an issue that exists for nodes in general - for all data that to be | |||
| carried over the particular networking technology - and is therefore | carried over the particular networking technology - and is therefore | |||
| the basis for an additional presumed trust model associated with an | the basis for an additional presumed trust model associated with an | |||
| existing network. | existing network. | |||
| 7. Acknowledgments | 8. Acknowledgments | |||
| Authors express their gratitude and appreciation to Joel Halpern for | Authors express their gratitude and appreciation to Joel Halpern for | |||
| the most helpful and insightful discussion on the applicability of | the most helpful and insightful discussion on the applicability of | |||
| HTS in a Service Function Chaining domain. | HTS in a Service Function Chaining domain. | |||
| 8. References | 9. References | |||
| 8.1. Normative References | 9.1. Normative References | |||
| [RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: Keyed- | ||||
| Hashing for Message Authentication", RFC 2104, | ||||
| DOI 10.17487/RFC2104, February 1997, | ||||
| <https://www.rfc-editor.org/info/rfc2104>. | ||||
| [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate | [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate | |||
| Requirement Levels", BCP 14, RFC 2119, | Requirement Levels", BCP 14, RFC 2119, | |||
| DOI 10.17487/RFC2119, March 1997, | DOI 10.17487/RFC2119, March 1997, | |||
| <https://www.rfc-editor.org/info/rfc2119>. | <https://www.rfc-editor.org/info/rfc2119>. | |||
| [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for | ||||
| Writing an IANA Considerations Section in RFCs", BCP 26, | ||||
| RFC 8126, DOI 10.17487/RFC8126, June 2017, | ||||
| <https://www.rfc-editor.org/info/rfc8126>. | ||||
| [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | |||
| 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | |||
| May 2017, <https://www.rfc-editor.org/info/rfc8174>. | May 2017, <https://www.rfc-editor.org/info/rfc8174>. | |||
| 8.2. Informative References | 9.2. Informative References | |||
| [I-D.ietf-ippm-ioam-data] | [I-D.ietf-ippm-ioam-data] | |||
| Brockners, F., Bhandari, S., and T. Mizrahi, "Data Fields | Brockners, F., Bhandari, S., and T. Mizrahi, "Data Fields | |||
| for In-situ OAM", draft-ietf-ippm-ioam-data-10 (work in | for In-situ OAM", draft-ietf-ippm-ioam-data-11 (work in | |||
| progress), July 2020. | progress), November 2020. | |||
| [I-D.ietf-ippm-ioam-direct-export] | [I-D.ietf-ippm-ioam-direct-export] | |||
| Song, H., Gafni, B., Zhou, T., Li, Z., Brockners, F., | Song, H., Gafni, B., Zhou, T., Li, Z., Brockners, F., | |||
| Bhandari, S., Sivakolundu, R., and T. Mizrahi, "In-situ | Bhandari, S., Sivakolundu, R., and T. Mizrahi, "In-situ | |||
| OAM Direct Exporting", draft-ietf-ippm-ioam-direct- | OAM Direct Exporting", draft-ietf-ippm-ioam-direct- | |||
| export-01 (work in progress), August 2020. | export-02 (work in progress), November 2020. | |||
| [I-D.song-ippm-postcard-based-telemetry] | [I-D.song-ippm-postcard-based-telemetry] | |||
| Song, H., Zhou, T., Li, Z., Shin, J., and K. Lee, | Song, H., Zhou, T., Li, Z., Mirsky, G., Shin, J., and K. | |||
| "Postcard-based On-Path Flow Data Telemetry", draft-song- | Lee, "Postcard-based On-Path Flow Data Telemetry using | |||
| ippm-postcard-based-telemetry-07 (work in progress), April | Packet Marking", draft-song-ippm-postcard-based- | |||
| 2020. | telemetry-08 (work in progress), October 2020. | |||
| [P4.INT] "In-band Network Telemetry (INT)", P4.org Specification, | [P4.INT] "In-band Network Telemetry (INT)", P4.org Specification, | |||
| October 2017. | October 2017. | |||
| [RFC4868] Kelly, S. and S. Frankel, "Using HMAC-SHA-256, HMAC-SHA- | ||||
| 384, and HMAC-SHA-512 with IPsec", RFC 4868, | ||||
| DOI 10.17487/RFC4868, May 2007, | ||||
| <https://www.rfc-editor.org/info/rfc4868>. | ||||
| [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with | [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with | |||
| Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, | Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, | |||
| May 2016, <https://www.rfc-editor.org/info/rfc7799>. | May 2016, <https://www.rfc-editor.org/info/rfc7799>. | |||
| [RFC8169] Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S., | [RFC8169] Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S., | |||
| and A. Vainshtein, "Residence Time Measurement in MPLS | and A. Vainshtein, "Residence Time Measurement in MPLS | |||
| Networks", RFC 8169, DOI 10.17487/RFC8169, May 2017, | Networks", RFC 8169, DOI 10.17487/RFC8169, May 2017, | |||
| <https://www.rfc-editor.org/info/rfc8169>. | <https://www.rfc-editor.org/info/rfc8169>. | |||
| [RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., | [RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., | |||
| End of changes. 53 change blocks. | ||||
| 140 lines changed or deleted | 329 lines changed or added | |||
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ | ||||