idnits 2.17.1 draft-ietf-ippm-ioam-deployment-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 19, 2021) is 919 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-06) exists of draft-ali-spring-ioam-srv6-04 == Outdated reference: A later version (-05) exists of draft-brockners-ippm-ioam-vxlan-gpe-03 == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-15 == Outdated reference: A later version (-11) exists of draft-ietf-ippm-ioam-direct-export-07 == Outdated reference: A later version (-10) exists of draft-ietf-ippm-ioam-flags-07 == Outdated reference: A later version (-12) exists of draft-ietf-ippm-ioam-ipv6-options-06 == Outdated reference: A later version (-13) exists of draft-ietf-nvo3-vxlan-gpe-12 == Outdated reference: A later version (-13) exists of draft-ietf-sfc-ioam-nsh-06 == Outdated reference: A later version (-06) exists of draft-mizrahi-ippm-ioam-profile-05 == Outdated reference: A later version (-07) exists of draft-spiegel-ippm-ioam-rawexport-05 == Outdated reference: A later version (-05) exists of draft-weis-ippm-ioam-eth-04 Summary: 0 errors (**), 0 flaws (~~), 13 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ippm F. Brockners, Ed. 3 Internet-Draft Cisco 4 Intended status: Best Current Practice S. Bhandari, Ed. 5 Expires: April 22, 2022 Thoughtspot 6 D. Bernier 7 Bell Canada 8 T. Mizrahi, Ed. 9 Huawei 10 October 19, 2021 12 In-situ OAM Deployment 13 draft-ietf-ippm-ioam-deployment-00 15 Abstract 17 In-situ Operations, Administration, and Maintenance (IOAM) collects 18 operational and telemetry information in the packet while the packet 19 traverses a path between two points in the network. This document 20 provides a framework for IOAM deployment and provides best current 21 practices. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on April 22, 2022. 40 Copyright Notice 42 Copyright (c) 2021 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (https://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 59 3. IOAM Deployment: Domains And Nodes . . . . . . . . . . . . . 4 60 4. Types of IOAM . . . . . . . . . . . . . . . . . . . . . . . . 5 61 4.1. Per-hop Tracing IOAM . . . . . . . . . . . . . . . . . . 6 62 4.2. Proof of Transit IOAM . . . . . . . . . . . . . . . . . . 8 63 4.3. Edge to Edge IOAM . . . . . . . . . . . . . . . . . . . . 8 64 4.4. Direct Export IOAM . . . . . . . . . . . . . . . . . . . 8 65 5. IOAM Encapsulations . . . . . . . . . . . . . . . . . . . . . 8 66 5.1. IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . 9 67 5.2. NSH . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 68 5.3. GRE . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 69 5.4. Geneve . . . . . . . . . . . . . . . . . . . . . . . . . 10 70 5.5. Segment Routing . . . . . . . . . . . . . . . . . . . . . 10 71 5.6. Segment Routing for IPv6 . . . . . . . . . . . . . . . . 10 72 5.7. VXLAN-GPE . . . . . . . . . . . . . . . . . . . . . . . . 10 73 6. IOAM Data Export . . . . . . . . . . . . . . . . . . . . . . 10 74 7. IOAM Deployment Considerations . . . . . . . . . . . . . . . 11 75 7.1. IOAM Namespaces . . . . . . . . . . . . . . . . . . . . . 12 76 7.2. IOAM Layering . . . . . . . . . . . . . . . . . . . . . . 13 77 7.3. IOAM Trace Option Types . . . . . . . . . . . . . . . . . 14 78 7.4. Traffic-sets That IOAM Is Applied To . . . . . . . . . . 16 79 7.5. IOAM Loopback Mode . . . . . . . . . . . . . . . . . . . 16 80 7.6. IOAM Active Mode . . . . . . . . . . . . . . . . . . . . 16 81 7.7. Brown Field Deployments: IOAM Unaware Nodes . . . . . . . 17 82 8. IOAM Manageability . . . . . . . . . . . . . . . . . . . . . 17 83 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 84 10. Security Considerations . . . . . . . . . . . . . . . . . . . 18 85 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 86 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 87 12.1. Normative References . . . . . . . . . . . . . . . . . . 19 88 12.2. Informative References . . . . . . . . . . . . . . . . . 20 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 91 1. Introduction 93 "In-situ" Operations, Administration, and Maintenance (IOAM) collects 94 OAM information within the packet while the packet traverses a 95 particular network domain. The term "in-situ" refers to the fact 96 that the OAM data is added to the data packets rather than is being 97 sent within packets specifically dedicated to OAM. IOAM is to 98 complement mechanisms such as Ping, Traceroute, or other active 99 probing mechanisms. In terms of "active" or "passive" OAM, "in-situ" 100 OAM can be considered a hybrid OAM type. "In-situ" mechanisms do not 101 require extra packets to be sent. IOAM adds information to the 102 already available data packets and therefore cannot be considered 103 passive. In terms of the classification given in [RFC7799] IOAM 104 could be portrayed as Hybrid Type I. IOAM mechanisms can be 105 leveraged where mechanisms using e.g. ICMP do not apply or do not 106 offer the desired results, such as proving that a certain traffic 107 flow takes a pre-defined path, SLA verification for the live data 108 traffic, detailed statistics on traffic distribution paths in 109 networks that distribute traffic across multiple paths, or scenarios 110 in which probe traffic is potentially handled differently from 111 regular data traffic by the network devices. 113 2. Conventions 115 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 116 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 117 "OPTIONAL" in this document are to be interpreted as described in BCP 118 14 [RFC2119] [RFC8174] when, and only when, they appear in all 119 capitals, as shown here. 121 Abbreviations used in this document: 123 E2E: Edge to Edge 125 Geneve: Generic Network Virtualization Encapsulation 126 [I-D.ietf-nvo3-geneve] 128 GRE: Generic Routing Encapsulation 130 IOAM: In-situ Operations, Administration, and Maintenance 132 MTU: Maximum Transmit Unit 134 NSH: Network Service Header [RFC8300] 136 OAM: Operations, Administration, and Maintenance 138 POT: Proof of Transit 140 SFC: Service Function Chain 142 SID: Segment Identifier 144 SR: Segment Routing 145 VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol 146 Extension [I-D.ietf-nvo3-vxlan-gpe] 148 3. IOAM Deployment: Domains And Nodes 150 IOAM is focused on "limited domains" as defined in [RFC8799]. IOAM 151 is not targeted for a deployment on the global Internet. The part of 152 the network which employs IOAM is referred to as the "IOAM-Domain". 153 For example, an IOAM-domain can include an enterprise campus using 154 physical connections between devices or an overlay network using 155 virtual connections / tunnels for connectivity between said devices. 156 An IOAM-domain is defined by its perimeter or edge. The operator of 157 an IOAM-domain is expected to put provisions in place to ensure that 158 packets which contain IOAM data fields do not leak beyond the edge of 159 an IOAM domain, e.g. using for example packet filtering methods. The 160 operator should consider the potential operational impact of IOAM to 161 mechanisms such as ECMP load-balancing schemes (e.g., load-balancing 162 schemes based on packet length could be impacted by the increased 163 packet size due to IOAM), path MTU (i.e., ensure that the MTU of all 164 links within a domain is sufficiently large to support the increased 165 packet size due to IOAM) and ICMP message handling. 167 An IOAM-Domain consists of "IOAM encapsulating nodes", "IOAM 168 decapsulating nodes" and "IOAM transit nodes". The role of a node 169 (i.e., encapsulating, transit, decapsulating) is defined within an 170 IOAM-Namespace (see below). A node can have different roles in 171 different IOAM-Namespaces. 173 An "IOAM encapsulating node" incorporates one or more IOAM-Option- 174 Types into packets that IOAM is enabled for. If IOAM is enabled for 175 a selected subset of the traffic, the IOAM encapsulating node is 176 responsible for applying the IOAM functionality to the selected 177 subset. 179 An "IOAM transit node" updates one or more of the IOAM-Data-Fields. 180 If both the Pre-allocated and the Incremental Trace Option-Types are 181 present in the packet, each IOAM transit node will update at most one 182 of these Option-Types. A transit node does not add new IOAM-Option- 183 Types to a packet, and does not change the IOAM-Data-Fields of an 184 IOAM Edge-to-Edge Option-Type. 186 An "IOAM decapsulating node" removes IOAM-Option-Type(s) from 187 packets. 189 The role of an IOAM-encapsulating, IOAM-transit or IOAM-decapsulating 190 node is always performed within a specific IOAM-Namespace. This 191 means that an IOAM node which is e.g., an IOAM-decapsulating node for 192 IOAM-Namespace "A" but not for IOAM-Namespace "B" will only remove 193 the IOAM-Option-Types for IOAM-Namespace "A" from the packet. An 194 IOAM decapsulating node situated at the edge of an IOAM domain 195 removes all IOAM-Option-Types and associated encapsulation headers 196 for all IOAM-Namespaces from the packet. 198 IOAM-Namespaces allow for a namespace-specific definition and 199 interpretation of IOAM-Data-Fields. An interface-id could for 200 example point to a physical interface (e.g., to understand which 201 physical interface of an aggregated link is used when receiving or 202 transmitting a packet) whereas in another case it could refer to a 203 logical interface (e.g., in case of tunnels). Please refer to 204 Section 7.1 for a discussion of IOAM-Namespaces. 206 Export of Export of Export of Export of 207 IOAM data IOAM data IOAM data IOAM data 208 (optional) (optional) (optional) (optional) 209 ^ ^ ^ ^ 210 | | | | 211 | | | | 212 User +---+----+ +---+----+ +---+----+ +---+----+ 213 packets |Encapsu-| | Transit| | Transit| |Decapsu-| 214 -------->|lating |====>| Node |====>| Node |====>|lating |--> 215 |Node | | A | | B | |Node | 216 +--------+ +--------+ +--------+ +--------+ 218 Figure 1: Roles of IOAM nodes 220 IOAM nodes which add or remove the IOAM-Data-Fields can also update 221 the IOAM-Data-Fields at the same time. Or in other words, IOAM 222 encapsulating or decapsulating nodes can also serve as IOAM transit 223 nodes at the same time. Note that not every node in an IOAM domain 224 needs to be an IOAM transit node. For example, a deployment might 225 require that packets traverse a set of firewalls which support IOAM. 226 In that case, only the set of firewall nodes would be IOAM transit 227 nodes rather than all nodes. 229 4. Types of IOAM 231 IOAM supports different modes of operation, which are differentiated 232 by the type of IOAM data fields being carried in the packet, the data 233 being collected, the type of nodes which collect or update data as 234 well as whether and how nodes export IOAM information. 236 o Per-hop tracing: OAM information about each IOAM node a packet 237 traverses is collected and stored within the user data packet as 238 the packet progresses through the IOAM domain. Potential uses of 239 IOAM per-hop tracing include: 241 * Understand the different paths different packets traverse 242 between an IOAM encapsulating and an IOAM decapsulating node in 243 a network that uses load balancing such as Equal Cost Multi- 244 Path (ECMP). This information could be used to tune the 245 algorithm for ECMP for optimized network resource usage. 247 * Operations/Troubleshooting: Understand which path a particular 248 packet or set of packets take as well as what amount of jitter 249 and delay different IOAM nodes in the path contribute to the 250 overall delay and jitter between the IOAM encapsulating node 251 and the IOAM decapsulating node. 253 o Proof-of-transit: Information that a verifier node can use to 254 verify whether a packet has traversed all nodes that is supposed 255 to traverse is stored within the user data packet. Proof-of- 256 transit could for example be used to verify that a packet indeed 257 passes through all services of a service function chain (e.g., 258 verify whether a packet indeed traversed the set of firewalls that 259 it is expected to traverse), or whether a packet indeed took the 260 expected path. 262 o Edge-to-edge: OAM information which is specific to the edges of an 263 IOAM domain is collected and stored within the user data packet. 264 Edge-to-Edge OAM could be used to gather operational information 265 about a particular network domain, such as the delay and jitter 266 incurred by that network domain or the traffic matrix of the 267 network domain. 269 o Direct export: OAM information about each IOAM node a packet 270 traverses is collected and immediately exported to a collector. 271 Direct export could be used in situations where per-hop tracing 272 information is desired, but gathering the information within the 273 packet - as with per-hop tracing - is not feasible. Rather than 274 automatically correlating the per-hop tracing information, as done 275 with per-hop tracing, direct export requires a collector to 276 correlate the information from the individual nodes. In addition, 277 all nodes enabled for direct export need to be capable to export 278 the IOAM information to the collector. 280 4.1. Per-hop Tracing IOAM 282 "IOAM tracing data" is expected to be collected at every IOAM transit 283 node that a packet traverses to ensure visibility into the entire 284 path a packet takes within an IOAM-Domain. I.e., in a typical 285 deployment all nodes in an IOAM-Domain would participate in IOAM and 286 thus be IOAM transit nodes, IOAM encapsulating or IOAM decapsulating 287 nodes. If not all nodes within a domain are IOAM capable, IOAM 288 tracing information (i.e., node data, see below) will only be 289 collected on those nodes which are IOAM capable. Nodes which are not 290 IOAM capable will forward the packet without any changes to the IOAM- 291 Data-Fields. The maximum number of hops and the minimum path MTU of 292 the IOAM domain is assumed to be known. 294 IOAM offers two different trace Option-Types, the "incremental" 295 Option-Type as well as the "pre-allocated" Option-Type. For a 296 discussion which of the two option types is the most suitable for an 297 implementation and/or deployment, see Section 7.3. 299 Every node data entry holds information for a particular IOAM transit 300 node that is traversed by a packet. The IOAM decapsulating node 301 removes the IOAM-Option-Type(s) and processes and/or exports the 302 associated data. All IOAM-Data-Fields are defined in the context of 303 an IOAM-Namespace. 305 IOAM tracing can collect the following types of information: 307 o Identification of the IOAM node. An IOAM node identifier can 308 match to a device identifier or a particular control point or 309 subsystem within a device. 311 o Identification of the interface that a packet was received on, 312 i.e. ingress interface. 314 o Identification of the interface that a packet was sent out on, 315 i.e. egress interface. 317 o Time of day when the packet was processed by the node as well as 318 the transit delay. Different definitions of processing time are 319 feasible and expected, though it is important that all devices of 320 an in-situ OAM domain follow the same definition. 322 o Generic data: Format-free information where syntax and semantic of 323 the information is defined by the operator in a specific 324 deployment. For a specific IOAM-Namespace, all IOAM nodes should 325 interpret the generic data the same way. Examples for generic 326 IOAM data include geo-location information (location of the node 327 at the time the packet was processed), buffer queue fill level or 328 cache fill level at the time the packet was processed, or even a 329 battery charge level. 331 o Information to detect whether IOAM trace data was added at every 332 hop or whether certain hops in the domain weren't IOAM transit 333 nodes. 335 o Data that relates to how the packet traversed a node (transit 336 delay, buffer occupancy in case the packet was buffered, queue 337 depth in case the packet was queued) 339 The Option-Types of incremental tracing and pre-allocated tracing are 340 defined in [I-D.ietf-ippm-ioam-data]. 342 4.2. Proof of Transit IOAM 344 IOAM Proof of Transit Option-Type is to support path or service 345 function chain [RFC7665] verification use cases. Proof-of-transit 346 uses methods like nested hashing or nested encryption of the IOAM 347 data or mechanisms such as Shamir's Secret Sharing Schema (SSSS). 349 The IOAM Proof of Transit Option-Type consist of a fixed size "IOAM 350 proof of transit option header" and "IOAM proof of transit option 351 data fields". For details see [I-D.ietf-ippm-ioam-data]. 353 4.3. Edge to Edge IOAM 355 The IOAM Edge-to-Edge Option-Type is to carry data that is added by 356 the IOAM encapsulating node and interpreted by IOAM decapsulating 357 node. The IOAM transit nodes may process the data but must not 358 modify it. 360 The IOAM Edge-to-Edge Option-Type consist of a fixed size "IOAM Edge- 361 to-Edge Option-Type header" and "IOAM Edge-to-Edge Option-Type data 362 fields". For details see [I-D.ietf-ippm-ioam-data]. 364 4.4. Direct Export IOAM 366 Direct Export is an IOAM mode of operation within which IOAM data to 367 be directly exported to a collector rather than being collected 368 within the data packets. The IOAM Direct Export Option-Type consist 369 of a fixed size "IOAM direct export option header". Direct Export 370 for IOAM is defined in [I-D.ietf-ippm-ioam-direct-export]. 372 5. IOAM Encapsulations 374 IOAM data fields and associated data types for in-situ OAM are 375 defined in [I-D.ietf-ippm-ioam-data]. The in-situ OAM data field can 376 be transported by a variety of transport protocols, including NSH, 377 Segment Routing, Geneve, IPv6, etc. 379 5.1. IPv6 381 IOAM encapsulation for IPv6 is defined in 382 [I-D.ietf-ippm-ioam-ipv6-options]. IOAM deployment considerations 383 for IPv6 networks are found in 384 [I-D.ioametal-ippm-6man-ioam-ipv6-deployment]. 386 5.2. NSH 388 IOAM encapsulation for NSH is defined in [I-D.ietf-sfc-ioam-nsh]. 390 5.3. GRE 392 IOAM encapsulation for GRE is outlined as part of the "EtherType 393 Protocol Identification of In-situ OAM Data" in 394 [I-D.weis-ippm-ioam-eth], though no example protocol header stacks 395 are provided in the document. When used with GRE, the IOAM-Option- 396 Types (the below diagram uses "IOAM" as shorthand for IOAM-Option- 397 Types) are sequenced in behind the GRE header that follows the 398 "outer" IP header. Figure 2 shows two example protocol header stacks 399 that use GRE along with IOAM. 401 Example 1 Example 2 403 | ... | | ... | 404 +----------------+ +----------------+ 405 | TCP/UDP header | | IP, ... | 406 +----------------+ +----------------+ 407 | IP header | | Eth. header | 408 +----------------+ +----------------+ 409 | IOAM | | IOAM | 410 +----------------+ +----------------+ 411 | GRE header | | GRE header | 412 +----------------+ +----------------+ 413 | IP header | | IP header | 414 +----------------+ +----------------+ 415 | Layer 2 | | Layer 2 | 416 +----------------+ +----------------+ 417 | Layer 1 | | Layer 1 | 418 +----------------+ +----------------+ 420 Figure 2: GRE with IOAM examples 422 5.4. Geneve 424 IOAM encapsulation for Geneve is defined in 425 [I-D.brockners-ippm-ioam-geneve]. 427 5.5. Segment Routing 429 IOAM encapsulation for Segment Routing is defined in 430 [I-D.gandhi-spring-ioam-sr-mpls]. 432 5.6. Segment Routing for IPv6 434 IOAM encapsulation for Segment Routing over IPv6 is defined in 435 [I-D.ali-spring-ioam-srv6]. 437 5.7. VXLAN-GPE 439 IOAM encapsulation for VXLAN-GPE is defined in 440 [I-D.brockners-ippm-ioam-vxlan-gpe]. 442 6. IOAM Data Export 444 IOAM nodes collect information for packets traversing a domain that 445 supports IOAM. IOAM decapsulating nodes as well as IOAM transit 446 nodes can choose to retrieve IOAM information from the packet, 447 process the information further and export the information using 448 e.g., IPFIX. 450 Raw data export of IOAM data using IPFIX is discussed in 451 [I-D.spiegel-ippm-ioam-rawexport]. "Raw export of IOAM data" refers 452 to a mode of operation where a node exports the IOAM data as it is 453 received in the packet. The exporting node neither interprets, 454 aggregates nor reformats the IOAM data before it is exported. Raw 455 export of IOAM data is to support an operational model where the 456 processing and interpretation of IOAM data is decoupled from the 457 operation of encapsulating/updating/decapsulating IOAM data, which is 458 also referred to as IOAM data-plane operation. The figure below 459 shows the separation of concerns for IOAM export: Exporting IOAM data 460 is performed by the "IOAM node" which performs IOAM data-plane 461 operation, whereas the interpretation of IOAM data is performed by 462 one or several IOAM data processing systems. The separation of 463 concerns is to off-load interpretation, aggregation and formatting of 464 IOAM data from the node which performs data-plane operations. In 465 other words, a node which is focused on data-plane operations, i.e. 466 forwarding of packets and handling IOAM data will not be tasked to 467 also interpret the IOAM data, but can leave this task to another 468 system or a set of systems. For scalability reasons, a single IOAM 469 node could choose to export IOAM data to several IOAM data processing 470 systems. Similarly, there several monitoring systems or analytics 471 systems can be used to further process the data received from the 472 IOAM preprocessing systems. Figure 3 shows an overview of IOAM 473 export, including IOAM data processing systems and monitoring/ 474 analytics systems. 476 +--------------+ 477 +-------------+ | 478 | Monitoring/ | | 479 | Analytics | | 480 | system(s) |-+ 481 +-------------+ 482 ^ 483 | Processed/interpreted/ 484 | aggregated IOAM data 485 | 486 +--------------+ 487 +-------------+ | 488 | IOAM data | | 489 | processing | | 490 | system(s) |-+ 491 +-------------+ 492 ^ 493 | Raw export of 494 | IOAM data 495 | 496 +--------------+-------+------+--------------+ 497 | | | | 498 | | | | 499 User +---+----+ +---+----+ +---+----+ +---+----+ 500 packets |Encapsu-| | Transit| | Transit| |Decapsu-| 501 -------->|lating |====>| Node |====>| Node |====>|lating |--> 502 |Node | | A | | B | |Node | 503 +--------+ +--------+ +--------+ +--------+ 505 Figure 3: IOAM framework with data export 507 7. IOAM Deployment Considerations 509 This section discusses several aspects of an IOAM deployment, 510 including IOAM Namespaces, IOAM Layering, traffic-sets that IOAM is 511 applied to and IOAM loopback mode. 513 7.1. IOAM Namespaces 515 IOAM-Namespaces add further context to IOAM-Option-Types and 516 associated IOAM-Data-Fields. IOAM-Namespaces support several 517 different uses: 519 o IOAM-Namespaces can be used by an operator to distinguish 520 different operational domains. Devices at domain edges can filter 521 on Namespace-IDs to provide for proper IOAM-Domain isolation. 523 o IOAM-Namespaces provide additional context for IOAM-Data-Fields 524 and thus ensure that IOAM-Data-Fields are unique and can be 525 interpreted properly by management stations or network 526 controllers. While, for example, the node identifier field does 527 not need to be unique in a deployment (e.g., an operator may wish 528 to use different node identifiers for different IOAM layers, even 529 within the same device; or node identifiers might not be unique 530 for other organizational reasons, such as after a merger of two 531 formerly separated organizations), the combination of node_id and 532 Namespace-ID should always be unique. Similarly, IOAM-Namespaces 533 can be used to define how certain IOAM-Data-Fields are 534 interpreted: IOAM offers three different timestamp format options. 535 The Namespace-ID can be used to determine the timestamp format. 536 IOAM-Data-Fields (e.g., buffer occupancy) which do not have a unit 537 associated are to be interpreted within the context of a IOAM- 538 Namespace. 540 o IOAM-Namespaces can be used to identify different sets of devices 541 (e.g., different types of devices) in a deployment: If an operator 542 desires to insert different IOAM-Data-Fields based on the device, 543 the devices could be grouped into multiple IOAM-Namespaces. This 544 could be due to the fact that the IOAM feature set differs between 545 different sets of devices, or it could be for reasons of optimized 546 space usage in the packet header. It could also stem from 547 hardware or operational limitations on the size of the trace data 548 that can be added and processed, preventing collection of a full 549 trace for a flow. 551 * Assigning different IOAM Namespace-IDs to different sets of 552 nodes or network partitions and using the Namespace-ID as a 553 selector at the IOAM encapsulating node, a full trace for a 554 flow could be collected and constructed via partial traces in 555 different packets of the same flow. Example: An operator could 556 choose to group the devices of a domain into two IOAM- 557 Namespaces, in a way that at average, only every second hop 558 would be recorded by any device. To retrieve a full view of 559 the deployment, the captured IOAM-Data-Fields of the two IOAM- 560 Namespaces need to be correlated. 562 * Assigning different IOAM Namespace-IDs to different sets of 563 nodes or network partitions and using a separate instance of an 564 IOAM-Option-Type for each Namespace-ID, a full trace for a flow 565 could be collected and constructed via partial traces from each 566 IOAM-Option-Type in each of the packets in the flow. Example: 567 An operator could choose to group the devices of a domain into 568 two IOAM-Namespaces, in a way that each IOAM-Namespace is 569 represented by one of two IOAM-Option-Types in the packet. 570 Each node would record data only for the IOAM-Namespace that it 571 belongs to, ignoring the other IOAM-Option-Type with a IOAM- 572 Namespace to which it doesn't belong. To retrieve a full view 573 of the deployment, the captured IOAM-Data-Fields of the two 574 IOAM-Namespaces need to be correlated. 576 7.2. IOAM Layering 578 If several encapsulation protocols (e.g., in case of tunneling) are 579 stacked on top of each other, IOAM-Data-Fields could be present in 580 different protocol fields at different layers. Layering allows 581 operators to instrument the protocol layer they want to measure. The 582 behavior follows the ships-in-the-night model, i.e. IOAM-Data-Fields 583 in one layer are independent from IOAM-Data-Fields in another layer. 584 Or in other words: Even though the term "layering" often implies some 585 form of hierarchy and relationship, in IOAM, layers are independent 586 from each other and don't assume any relationship among them. The 587 different layers could, but do not have to share the same IOAM 588 encapsulation mechanisms. Similarly, the sematics of the IOAM-Data- 589 Fields, can, do not have to be associated to across different layers. 590 For example, a node which inserts node-id information into two 591 different layers could use "node-id=10" for one layer and "node- 592 id=1000" for the second layer. 594 Figure 4 shows an example of IOAM layering. The figure shows a 595 Geneve tunnel carried over IPv6 which starts at node A and ends at 596 node D. IOAM information is encapsulated in IPv6 as well as in 597 Geneve. At the IPv6 layer, node A is IOAM encapsulating node (into 598 IPv6), node D is the IOAM decapsulating node and node B and node C 599 are IOAM transit nodes. At the Geneve layer, node A is IOAM 600 encapsulating node (into Geneve) and node D is IOAM decapsulating 601 node (from Geneve). The use of IOAM at both layers as shown in the 602 example here could be used to reveal which nodes of an underlay (here 603 the IPv6 network) are traversed by tunneled packet in an overlay 604 (here the Geneve network) - which assumes that the IOAM information 605 encapsulated by nodes A and D into Geneve and IPv6 is associated to 606 each other. 608 +---+----+ +---+----+ 609 User |Geneve | |Geneve | 610 packets |Encapsu-| |Decapsu-| 611 -------->|lating |==================================>|lating |--> 612 | Node | | Node | 613 | A | | D | 614 +--------+ +--------+ 615 ^ ^ 616 | | 617 v v 618 +--------+ +--------+ +--------+ +--------+ 619 |IPv6 | | IPv6 | | IPv6 | |IPv6 | 620 |Encapsu-| | Transit| | Transit| |Decapsu-| 621 |lating |====>| Node |====>| Node |====>|lating | 622 | Node | | | | | | Node | 623 | A | | B | | C | | D | 624 +--------+ +--------+ +--------+ +--------+ 626 Figure 4: IOAM layering example 628 7.3. IOAM Trace Option Types 630 IOAM offers two different IOAM Option-Types for tracing: 631 "Incremental" Trace-Option-Type and "Pre-allocated" Trace-Option- 632 Type. "Incremental" refers to a mode of operation where the packet 633 is expanded at every IOAM node that addes IOAM-Data-Fields. "Pre- 634 Allocated" describes a mode of operation where the IOAM encapsulating 635 node allocates room for all IOAM-Data-Fields in the entire IOAM 636 domain. More specifically: 638 Pre-allocated Trace-Option: This trace option is defined as a 639 container of node data fields with pre-allocated space for each 640 node to populate its information. This option is useful for 641 implementations where it is efficient to allocate the space once 642 and index into the array to populate the data during transit 643 (e.g., software forwarders often fall into this class). 645 Incremental Trace-Option: This trace option is defined as a 646 container of node data fields where each node allocates and pushes 647 its node data immediately following the option header. 649 A deployment can choose to configure and support one or both of the 650 IOAM Trace-Option-Types. The operator decides by means of 651 configuration which Trace-Option-Type(s) will be used for a 652 particular domain. Deployments can mix devices which include either 653 the Incremental Trace-Option-Type or the Pre-allocated Trace-Option- 654 Type, e.g., in case different types of packet forwarders and 655 associated different types of IOAM implementations exist in a 656 deployment. As a result, both Option-Types can be present in a 657 packet. IOAM decapsulating nodes remove both types of Trace-Option- 658 Types from the packet. 660 The two different Option-Types cater to different packet forwarding 661 infrastructures and are to allow an optimized implementation of IOAM 662 tracing: 664 Pre-allocated Trace-Option: For some implementations of packet 665 forwarders it is efficient to allocate the space for the maximum 666 number of nodes that IOAM Trace Data-Fields should be collected 667 from and insert/update information in the packet at flexible 668 locations, based on a pointer retrieved from a field in the 669 packet. The IOAM encapsulating node allocates an array of the 670 size of the maximum number of nodes that IOAM Trace Data-Fields 671 should be collected from. IOAM transit nodes index into the array 672 to populate the data during transit. Software forwarders often 673 fall into this class of packet forwarder implementations. The 674 maximum number of nodes that IOAM information could be collected 675 from is configured by the operator on the IOAM encapsulating node. 676 The operator has to ensure that the packet with the pre-allocated 677 array that carries the IOAM Data-Fields does not exceed the MTU of 678 any of the links in the IOAM domain. 680 Incremental Trace-Option: Looking up a pointer contained in the 681 packet and inserting/updating information at a flexible location 682 in the packet as a result of the pointer lookup is costly for some 683 forwarding infrastructures. Hardware-based packet forwarding 684 infrastructures often fall into this category. Consequently, 685 hardware-based packet forwarders could choose to support the 686 incremental IOAM-Trace-Option-Type. The incremental IOAM-Trace- 687 Option-Type eliminates the need for the IOAM transit nodes to read 688 the full array in the Trace-Option-Type and allows packets to grow 689 to the size of the MTU of the IOAM domain. IOAM transit nodes 690 will expand the packet and insert the IOAM-Data-Fields as long as 691 there is space available in the packet, i.e. as long as the size 692 of the packet stays within the bounds of the MTU of any of the 693 links in the IOAM domain. There is no need for the operator to 694 configure the IOAM encapsulation node with the maximum number of 695 nodes that IOAM information could be collected from. The operator 696 has to ensure that the minimum MTU of any of the links in the IOAM 697 domain is known to all IOAM transit nodes. 699 7.4. Traffic-sets That IOAM Is Applied To 701 IOAM can be deployed on all or only on subsets of the live user 702 traffic, e.g., per interface, based on an access control list or flow 703 specification defining a specific set of traffic, etc. 705 7.5. IOAM Loopback Mode 707 IOAM Loopback is used to trigger each transit device along the path 708 of a packet to send a copy of the data packet back to the source. 709 Loopback allows an IOAM encapsulating node to trace the path to a 710 given destination, and to receive per-hop data about both the forward 711 and the return path. Loopback is enabled by the encapsulating node 712 setting the loopback flag. Looped-back packets use the source 713 address of the original packet as destination address and the address 714 of the node which performs the loopback operation as source address. 715 Nodes which loop back a packet clear the loopback flag before sending 716 the copy back towards the source. Loopack applies to IOAM 717 deployments where the encapsulating node is either a host or the 718 start of a tunnel: For details on IOAM loopback, please refer to 719 [I-D.ietf-ippm-ioam-flags]. 721 7.6. IOAM Active Mode 723 The IOAM active mode flag indicates that a packet is an active OAM 724 packet as opposed to regular user data traffic. Active mode is 725 expected to be used for active measurement using IOAM. For details 726 on IOAM active mode, please refer to [I-D.ietf-ippm-ioam-flags]. 728 Example use-cases for IOAM Active Mode include: 730 o Endpoint detailed active measurement: Synthetic probe packets are 731 sent between the source and destination. These probe packets 732 include a Trace Option-Type (i.e., either incremental or pre- 733 allocated). Since the probe packets are sent between the 734 endpoints, these packets are treated as data packets by the IOAM 735 domain, and do not require special treatment at the IOAM layer. 736 The source, which is also the IOAM encapsulating node can choose 737 to set the Active flag, providing an explicit indication that 738 these probe packets are meant for telemetry collection. 740 o IOAM active measurement using probe packets: Probe packets are 741 generated and transmitted by an IOAM encapsulating node towards a 742 destination which is also the IOAM decapsulating node. Probe 743 packets include a Trace Option-Type (i.e., either incremental or 744 pre-allocated) which has its Active flag set. 746 o IOAM active measurement using replicated data packets: Probe 747 packets are created by an IOAM encapsulating node by selecting 748 some or all of the en route data packets and replicating them. A 749 selected data packet that is replicated, and its (possibly 750 truncated) copy is forwarded with one or more IOAM option, while 751 the original packet is forwarded normally, without IOAM options. 752 To the extent possible, the original data packet and its replica 753 are forwarded through the same path. The replica includes a Trace 754 Option-Type that has its Active flag set, indicating that the IOAM 755 decapsulating node should terminate it. In this case the IOAM 756 Active flag ensures that the replicated traffic is not forwarded 757 beyond the IOAM domain. 759 7.7. Brown Field Deployments: IOAM Unaware Nodes 761 A network can consist of a mix of IOAM aware and IOAM unaware nodes. 762 The encapsulation of IOAM-Data-Fields into different protocols (see 763 also Section 5) are defined such that data packets that include IOAM- 764 Data-Fields do not get dropped by IOAM unaware nodes. For example, 765 packets which contain the IOAM-Trace-Option-Types in IPv6 Hop by Hop 766 extension headers are defined with bits to indicate "00 - skip over 767 this option and continue processing the header". This will ensure 768 that when a node that is IOAM unaware receives a packet with IOAM- 769 Data-Fields included, does not drop the packet. 771 Deployments which leverage the IOAM-Trace-Option-Type(s) could 772 benefit from the ability to detect the presence of IOAM unaware 773 nodes, i.e. nodes which forward the packet but do not update/add 774 IOAM-Data-Fields in IOAM-Trace-Option-Type(s). The node data that is 775 defined as part of the IOAM-Trace-Option-Type(s) includes a Hop_Lim 776 field associated to the node identifier to detect missed nodes, i.e. 777 "holes" in the trace. Monitoring/Analytics system(s) could utilize 778 this information to account for the presence of IOAM unaware nodes in 779 the network. 781 8. IOAM Manageability 783 The YANG model for configuring IOAM in network nodes which support 784 IOAM is defined in [I-D.zhou-ippm-ioam-yang]. 786 A deployment can leverage IOAM profiles is to limit the scope of IOAM 787 features, allowing simpler implementation, verification, and 788 interoperability testing in the context of specific use cases that do 789 not require the full functionality of IOAM. An IOAM profile defines 790 a use case or a set of use cases for IOAM, and an associated set of 791 rules that restrict the scope and features of the IOAM specification, 792 thereby limiting it to a subset of the full functionality. IOAM 793 profiles are defined in [I-D.mizrahi-ippm-ioam-profile]. 795 9. IANA Considerations 797 This document does not request any IANA actions. 799 10. Security Considerations 801 As discussed in [RFC7276], a successful attack on an OAM protocol in 802 general, and specifically on IOAM, can prevent the detection of 803 failures or anomalies, or create a false illusion of nonexistent 804 ones. 806 The Proof of Transit Option-Type (Section Section 4.2) is used for 807 verifying the path of data packets. The security considerations of 808 POT are further discussed in [I-D.ietf-sfc-proof-of-transit]. 810 Security considerations related to the use of IOAM flags, in 811 particular the loopback flag are found in [I-D.ietf-ippm-ioam-flags]. 813 IOAM data can be subject to eavesdropping. Although the 814 confidentiality of the user data is not at risk in this context, the 815 IOAM data elements can be used for network reconnaissance, allowing 816 attackers to collect information about network paths, performance, 817 queue states, buffer occupancy and other information. Recon is an 818 improbable security threat in an IOAM deployment that is within a 819 confined physical domain. However, in deployments that are not 820 confined to a single LAN, but span multiple inter-connected sites 821 (for example, using an overlay network), the inter-site links can be 822 secured (e.g., by IPsec) in order to avoid external eavesdropping. 823 Another possible mitigation approach is to use the "direct exporting" 824 mode [I-D.ietf-ippm-ioam-direct-export]. In this case the IOAM 825 related trace information would not be available in the customer data 826 packets, but would trigger exporting of (secured) packet-related IOAM 827 information at every node. IOAM data export and securing IOAM data 828 export is outside the scope of this document. 830 IOAM can be used as a means for implementing Denial of Service (DoS) 831 attacks, or for amplifying them. For example, a malicious attacker 832 can add an IOAM header to packets or modify an IOAM header in en 833 route packets in order to consume the resources of network devices 834 that take part in IOAM or collectors that analyze the IOAM data. 835 Another example is a packet length attack, in which an attacker 836 pushes headers associated with IOAM Option-Types into data packets, 837 causing these packets to be increased beyond the MTU size, resulting 838 in fragmentation or in packet drops. Such DoS attacks can be 839 mitigated by deploying IOAM in confined administrative domains, and 840 by defining performance limits on IOAM encapsulation and IOAM 841 exporting. By limiting the rate and/or percentage of packets that 842 are subject to IOAM encpasulation and the rate of exported traffic, 843 it is possible to cofine the impact of such attacks. 845 Since IOAM options may include timestamps, if network devices use 846 synchronization protocols then any attack on the time protocol 847 [RFC7384] can compromise the integrity of the timestamp-related data 848 fields. Synchronization attacks can be mitigated by combining a 849 secured time distribution scheme, e.g., [RFC8915], and by using 850 redundant clock sources [RFC5905] and/or redundant network paths for 851 the time distribution protocol [RFC8039]. 853 At the management plane, attacks may be implemented by misconfiguring 854 or by maliciously configuring IOAM-enabled nodes in a way that 855 enables other attacks. Thus, IOAM configuration should be secured in 856 a way that authenticates authorized users and verifies the integrity 857 of configuration procedures. 859 Notably, IOAM is expected to be deployed in specific network domains, 860 thus confining the potential attack vectors to within the network 861 domain. Indeed, in order to limit the scope of threats to within the 862 current network domain the network operator is expected to enforce 863 policies that prevent IOAM traffic from leaking outside of the IOAM 864 domain, and prevent IOAM data from outside the domain to be processed 865 and used within the domain. Note that the Immediate Export mode 866 (reference to be added in a future revision) can mitigate the 867 potential threat of IOAM data leaking through data packets. 869 11. Acknowledgements 871 The authors would like to thank Tal Mizrahi, Eric Vyncke, Nalini 872 Elkins, Srihari Raghavan, Ranganathan T S, Barak Gafni, Karthik Babu 873 Harichandra Babu, Akshaya Nadahalli, LJ Wobker, Erik Nordmark, 874 Vengada Prasad Govindan, Andrew Yourtchenko, Aviv Kfir, Tianran Zhou, 875 Zhenbin (Robin), Joe Clarke, Al Morton, Tom Herbet, Haoyu song, and 876 Mickey Spiegel for the comments and advice on IOAM. 878 12. References 880 12.1. Normative References 882 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 883 Requirement Levels", BCP 14, RFC 2119, 884 DOI 10.17487/RFC2119, March 1997, 885 . 887 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 888 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 889 May 2017, . 891 12.2. Informative References 893 [I-D.ali-spring-ioam-srv6] 894 Ali, Z., Gandhi, R., Filsfils, C., Brockners, F., Nainar, 895 N., Pignataro, C., Li, C., Chen, M., and G. Dawra, 896 "Segment Routing Header encapsulation for In-situ OAM 897 Data", draft-ali-spring-ioam-srv6-04 (work in progress), 898 July 2021. 900 [I-D.brockners-ippm-ioam-geneve] 901 Brockners, F., Bhandari, S., Govindan, V. P., Pignataro, 902 C., Nainar, N. K., Gredler, H., Leddy, J., Youell, S., 903 Mizrahi, T., Lapukhov, P., Gafni, B., Kfir, A., and M. 904 Spiegel, "Geneve encapsulation for In-situ OAM Data", 905 draft-brockners-ippm-ioam-geneve-05 (work in progress), 906 November 2020. 908 [I-D.brockners-ippm-ioam-vxlan-gpe] 909 Brockners, F., Bhandari, S., Govindan, V. P., Pignataro, 910 C., Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Kfir, 911 A., Gafni, B., Lapukhov, P., and M. Spiegel, "VXLAN-GPE 912 Encapsulation for In-situ OAM Data", draft-brockners-ippm- 913 ioam-vxlan-gpe-03 (work in progress), November 2019. 915 [I-D.gandhi-spring-ioam-sr-mpls] 916 Gandhi, R., Ali, Z., Filsfils, C., Brockners, F., Wen, B., 917 and V. Kozak, "Segment Routing with MPLS Data Plane 918 Encapsulation for In-situ OAM Data", draft-gandhi-spring- 919 ioam-sr-mpls-02 (work in progress), August 2019. 921 [I-D.ietf-ippm-ioam-data] 922 Brockners, F., Bhandari, S., and T. Mizrahi, "Data Fields 923 for In-situ OAM", draft-ietf-ippm-ioam-data-15 (work in 924 progress), October 2021. 926 [I-D.ietf-ippm-ioam-direct-export] 927 Song, H., Gafni, B., Zhou, T., Li, Z., Brockners, F., 928 Bhandari, S., Sivakolundu, R., and T. Mizrahi, "In-situ 929 OAM Direct Exporting", draft-ietf-ippm-ioam-direct- 930 export-07 (work in progress), October 2021. 932 [I-D.ietf-ippm-ioam-flags] 933 Mizrahi, T., Brockners, F., Bhandari, S., Sivakolundu, R., 934 Pignataro, C., Kfir, A., Gafni, B., Spiegel, M., and J. 935 Lemon, "In-situ OAM Loopback and Active Flags", draft- 936 ietf-ippm-ioam-flags-07 (work in progress), October 2021. 938 [I-D.ietf-ippm-ioam-ipv6-options] 939 Bhandari, S. and F. Brockners, "In-situ OAM IPv6 Options", 940 draft-ietf-ippm-ioam-ipv6-options-06 (work in progress), 941 July 2021. 943 [I-D.ietf-nvo3-geneve] 944 Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic 945 Network Virtualization Encapsulation", draft-ietf- 946 nvo3-geneve-16 (work in progress), March 2020. 948 [I-D.ietf-nvo3-vxlan-gpe] 949 (Editor), F. M., (editor), L. K., and U. E. (editor), 950 "Generic Protocol Extension for VXLAN (VXLAN-GPE)", draft- 951 ietf-nvo3-vxlan-gpe-12 (work in progress), September 2021. 953 [I-D.ietf-sfc-ioam-nsh] 954 Brockners, F. and S. Bhandari, "Network Service Header 955 (NSH) Encapsulation for In-situ OAM (IOAM) Data", draft- 956 ietf-sfc-ioam-nsh-06 (work in progress), July 2021. 958 [I-D.ietf-sfc-proof-of-transit] 959 Brockners, F., Bhandari, S., Mizrahi, T., Dara, S., and S. 960 Youell, "Proof of Transit", draft-ietf-sfc-proof-of- 961 transit-08 (work in progress), November 2020. 963 [I-D.ioametal-ippm-6man-ioam-ipv6-deployment] 964 Bhandari, S., Brockners, F., Mizrahi, T., Kfir, A., Gafni, 965 B., Spiegel, M., Krishnan, S., and M. Smith, "Deployment 966 Considerations for In-situ OAM with IPv6 Options", draft- 967 ioametal-ippm-6man-ioam-ipv6-deployment-03 (work in 968 progress), March 2020. 970 [I-D.mizrahi-ippm-ioam-profile] 971 Mizrahi, T., Brockners, F., Bhandari, S., Sivakolundu, R., 972 Pignataro, C., Kfir, A., Gafni, B., Spiegel, M., Zhou, T., 973 and J. Lemon, "In Situ OAM Profiles", draft-mizrahi-ippm- 974 ioam-profile-05 (work in progress), August 2021. 976 [I-D.spiegel-ippm-ioam-rawexport] 977 Spiegel, M., Brockners, F., Bhandari, S., and R. 978 Sivakolundu, "In-situ OAM raw data export with IPFIX", 979 draft-spiegel-ippm-ioam-rawexport-05 (work in progress), 980 July 2021. 982 [I-D.weis-ippm-ioam-eth] 983 Weis, B., Brockners, F., Hill, C., Bhandari, S., Govindan, 984 V. P., Pignataro, C., Gredler, H., Leddy, J., Youell, S., 985 Mizrahi, T., Kfir, A., Gafni, B., Lapukhov, P., and M. 986 Spiegel, "EtherType Protocol Identification of In-situ OAM 987 Data", draft-weis-ippm-ioam-eth-04 (work in progress), May 988 2020. 990 [I-D.zhou-ippm-ioam-yang] 991 Zhou, T., Guichard, J., Brockners, F., and S. Raghavan, "A 992 YANG Data Model for In-Situ OAM", draft-zhou-ippm-ioam- 993 yang-08 (work in progress), July 2020. 995 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 996 "Network Time Protocol Version 4: Protocol and Algorithms 997 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 998 . 1000 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1001 Weingarten, "An Overview of Operations, Administration, 1002 and Maintenance (OAM) Tools", RFC 7276, 1003 DOI 10.17487/RFC7276, June 2014, 1004 . 1006 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1007 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1008 October 2014, . 1010 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 1011 Chaining (SFC) Architecture", RFC 7665, 1012 DOI 10.17487/RFC7665, October 2015, 1013 . 1015 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1016 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1017 May 2016, . 1019 [RFC8039] Shpiner, A., Tse, R., Schelp, C., and T. Mizrahi, 1020 "Multipath Time Synchronization", RFC 8039, 1021 DOI 10.17487/RFC8039, December 2016, 1022 . 1024 [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., 1025 "Network Service Header (NSH)", RFC 8300, 1026 DOI 10.17487/RFC8300, January 2018, 1027 . 1029 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 1030 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 1031 . 1033 [RFC8915] Franke, D., Sibold, D., Teichel, K., Dansarie, M., and R. 1034 Sundblad, "Network Time Security for the Network Time 1035 Protocol", RFC 8915, DOI 10.17487/RFC8915, September 2020, 1036 . 1038 Authors' Addresses 1040 Frank Brockners (editor) 1041 Cisco Systems, Inc. 1042 Hansaallee 249, 3rd Floor 1043 DUESSELDORF, NORDRHEIN-WESTFALEN 40549 1044 Germany 1046 Email: fbrockne@cisco.com 1048 Shwetha Bhandari (editor) 1049 Thoughtspot 1050 3rd Floor, Indiqube Orion, 24th Main Rd, Garden Layout, HSR Layout 1051 Bangalore, KARNATAKA 560 102 1052 India 1054 Email: shwetha.bhandari@thoughtspot.com 1056 Daniel Bernier 1057 Bell Canada 1058 Canada 1060 Email: daniel.bernier@bell.ca 1062 Tal Mizrahi (editor) 1063 Huawei 1064 8-2 Matam 1065 Haifa 3190501 1066 Israel 1068 Email: tal.mizrahi.phd@gmail.com