idnits 2.17.1 draft-ietf-ippm-ioam-data-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 04, 2019) is 1756 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 658 -- Looks like a reference, but probably isn't: '1' on line 520 -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE1588v2' -- Possible downref: Non-RFC (?) normative reference: ref. 'POSIX' == Outdated reference: A later version (-09) exists of draft-ietf-ntp-packet-timestamps-06 == Outdated reference: A later version (-16) exists of draft-ietf-nvo3-geneve-13 == Outdated reference: A later version (-13) exists of draft-ietf-nvo3-vxlan-gpe-07 == Outdated reference: A later version (-07) exists of draft-spiegel-ippm-ioam-rawexport-01 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ippm F. Brockners 3 Internet-Draft S. Bhandari 4 Intended status: Standards Track C. Pignataro 5 Expires: January 5, 2020 Cisco 6 H. Gredler 7 RtBrick Inc. 8 J. Leddy 10 S. Youell 11 JPMC 12 T. Mizrahi 13 Huawei Network.IO Innovation Lab 14 D. Mozes 16 P. Lapukhov 17 Facebook 18 R. Chang 19 Barefoot Networks 20 D. Bernier 21 Bell Canada 22 J. Lemon 23 Broadcom 24 July 04, 2019 26 Data Fields for In-situ OAM 27 draft-ietf-ippm-ioam-data-06 29 Abstract 31 In-situ Operations, Administration, and Maintenance (IOAM) records 32 operational and telemetry information in the packet while the packet 33 traverses a path between two points in the network. This document 34 discusses the data fields and associated data types for in-situ OAM. 35 In-situ OAM data fields can be embedded into a variety of transports 36 such as NSH, Segment Routing, Geneve, native IPv6 (via extension 37 header), or IPv4. In-situ OAM can be used to complement OAM 38 mechanisms based on e.g. ICMP or other types of probe packets. 40 Status of This Memo 42 This Internet-Draft is submitted in full conformance with the 43 provisions of BCP 78 and BCP 79. 45 Internet-Drafts are working documents of the Internet Engineering 46 Task Force (IETF). Note that other groups may also distribute 47 working documents as Internet-Drafts. The list of current Internet- 48 Drafts is at https://datatracker.ietf.org/drafts/current/. 50 Internet-Drafts are draft documents valid for a maximum of six months 51 and may be updated, replaced, or obsoleted by other documents at any 52 time. It is inappropriate to use Internet-Drafts as reference 53 material or to cite them other than as "work in progress." 55 This Internet-Draft will expire on January 5, 2020. 57 Copyright Notice 59 Copyright (c) 2019 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (https://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the Simplified BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 75 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 76 3. Scope, Applicability, and Assumptions . . . . . . . . . . . . 4 77 4. IOAM Data Types and Formats . . . . . . . . . . . . . . . . . 5 78 4.1. IOAM Namespaces . . . . . . . . . . . . . . . . . . . . . 7 79 4.2. IOAM Tracing Options . . . . . . . . . . . . . . . . . . 9 80 4.2.1. Pre-allocated and Incremental Trace Options . . . . . 11 81 4.2.2. IOAM node data fields and associated formats . . . . 15 82 4.2.3. Examples of IOAM node data . . . . . . . . . . . . . 21 83 4.3. IOAM Proof of Transit Option . . . . . . . . . . . . . . 22 84 4.3.1. IOAM Proof of Transit Type 0 . . . . . . . . . . . . 24 85 4.4. IOAM Edge-to-Edge Option . . . . . . . . . . . . . . . . 25 86 5. Timestamp Formats . . . . . . . . . . . . . . . . . . . . . . 27 87 5.1. PTP Truncated Timestamp Format . . . . . . . . . . . . . 27 88 5.2. NTP 64-bit Timestamp Format . . . . . . . . . . . . . . . 28 89 5.3. POSIX-based Timestamp Format . . . . . . . . . . . . . . 29 90 6. IOAM Data Export . . . . . . . . . . . . . . . . . . . . . . 31 91 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31 92 7.1. Creation of a new In-Situ OAM Protocol Parameters 93 Registry (IOAM) Protocol Parameters IANA registry . . . . 31 94 7.2. IOAM Type Registry . . . . . . . . . . . . . . . . . . . 32 95 7.3. IOAM Trace Type Registry . . . . . . . . . . . . . . . . 32 96 7.4. IOAM Trace Flags Registry . . . . . . . . . . . . . . . . 33 97 7.5. IOAM POT Type Registry . . . . . . . . . . . . . . . . . 33 98 7.6. IOAM POT Flags Registry . . . . . . . . . . . . . . . . . 33 99 7.7. IOAM E2E Type Registry . . . . . . . . . . . . . . . . . 33 100 7.8. IOAM Namespace-ID Registry . . . . . . . . . . . . . . . 34 101 8. Security Considerations . . . . . . . . . . . . . . . . . . . 34 102 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35 103 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 36 104 10.1. Normative References . . . . . . . . . . . . . . . . . . 36 105 10.2. Informative References . . . . . . . . . . . . . . . . . 36 106 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 38 108 1. Introduction 110 This document defines data fields for "in-situ" Operations, 111 Administration, and Maintenance (IOAM). In-situ OAM records OAM 112 information within the packet while the packet traverses a particular 113 network domain. The term "in-situ" refers to the fact that the OAM 114 data is added to the data packets rather than is being sent within 115 packets specifically dedicated to OAM. IOAM is to complement 116 mechanisms such as Ping or Traceroute, or more recent active probing 117 mechanisms as described in [I-D.lapukhov-dataplane-probe]. In terms 118 of "active" or "passive" OAM, "in-situ" OAM can be considered a 119 hybrid OAM type. While no extra packets are sent, IOAM adds 120 information to the packets therefore cannot be considered passive. 121 In terms of the classification given in [RFC7799] IOAM could be 122 portrayed as Hybrid Type 1. "In-situ" mechanisms do not require 123 extra packets to be sent and hence don't change the packet traffic 124 mix within the network. IOAM mechanisms can be leveraged where 125 mechanisms using e.g. ICMP do not apply or do not offer the desired 126 results, such as proving that a certain traffic flow takes a pre- 127 defined path, SLA verification for the live data traffic, detailed 128 statistics on traffic distribution paths in networks that distribute 129 traffic across multiple paths, or scenarios in which probe traffic is 130 potentially handled differently from regular data traffic by the 131 network devices. 133 2. Conventions 135 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 136 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 137 document are to be interpreted as described in [RFC2119]. 139 Abbreviations used in this document: 141 E2E Edge to Edge 143 Geneve: Generic Network Virtualization Encapsulation 144 [I-D.ietf-nvo3-geneve] 146 IOAM: In-situ Operations, Administration, and Maintenance 148 MTU: Maximum Transmit Unit 150 NSH: Network Service Header [RFC8300] 152 OAM: Operations, Administration, and Maintenance 154 POT: Proof of Transit 156 SFC: Service Function Chain 158 SID: Segment Identifier 160 SR: Segment Routing 162 VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol 163 Extension [I-D.ietf-nvo3-vxlan-gpe] 165 3. Scope, Applicability, and Assumptions 167 IOAM deployment assumes a set of constraints, requirements, and 168 guiding principles which are described in this section. 170 Scope: This document defines the data fields and associated data 171 types for in-situ OAM. The in-situ OAM data field can be transported 172 by a variety of transport protocols, including NSH, Segment Routing, 173 Geneve, IPv6, or IPv4. Specification details for these different 174 transport protocols are outside the scope of this document. 176 Deployment domain (or scope) of in-situ OAM deployment: IOAM is a 177 network domain focused feature, with "network domain" being a set of 178 network devices or entities within a single administration. For 179 example, a network domain can include an enterprise campus using 180 physical connections between devices or an overlay network using 181 virtual connections / tunnels for connectivity between said devices. 182 A network domain is defined by its perimeter or edge. Designers of 183 carrier protocols for IOAM must specify mechanisms to ensure that 184 IOAM data stays within an IOAM domain. In addition, the operator of 185 such a domain is expected to put provisions in place to ensure that 186 IOAM data does not leak beyond the edge of an IOAM domain, e.g. using 187 for example packet filtering methods. The operator should consider 188 potential operational impact of IOAM to mechanisms such as ECMP 189 processing (e.g. load-balancing schemes based on packet length could 190 be impacted by the increased packet size due to IOAM), path MTU (i.e. 191 ensure that the MTU of all links within a domain is sufficiently 192 large to support the increased packet size due to IOAM) and ICMP 193 message handling (i.e. in case of a native IPv6 transport, IOAM 194 support for ICMPv6 Echo Request/Reply could desired which would 195 translate into ICMPv6 extensions to enable IOAM data fields to be 196 copied from an Echo Request message to an Echo Reply message). 198 IOAM control points: IOAM data fields are added to or removed from 199 the live user traffic by the devices which form the edge of a domain. 200 Devices within an IOAM domain can update and/or add IOAM data-fields. 201 Domain edge devices can be hosts or network devices. 203 Traffic-sets that IOAM is applied to: IOAM can be deployed on all or 204 only on subsets of the live user traffic. It SHOULD be possible to 205 enable IOAM on a selected set of traffic (e.g., per interface, based 206 on an access control list or flow specification defining a specific 207 set of traffic, etc.) The selected set of traffic can also be all 208 traffic. 210 Encapsulation independence: Data formats for IOAM SHOULD be defined 211 in a transport-independent manner. IOAM applies to a variety of 212 encapsulating protocols. A definition of how IOAM data fields are 213 carried by different transport protocols is outside the scope of this 214 document. 216 Layering: If several encapsulation protocols (e.g., in case of 217 tunneling) are stacked on top of each other, IOAM data-records could 218 be present at every layer. The behavior follows the ships-in-the- 219 night model, i.e. IOAM data in one layer is independent from IOAM 220 data in another layer. Layering allows operators to instrument the 221 protocol layer they want to measure. The different layers could, but 222 do not have to share the same IOAM encapsulation and decapsulation. 224 IOAM implementation: The IOAM data-field definitions take the 225 specifics of devices with hardware data-plane and software data-plane 226 into account. 228 4. IOAM Data Types and Formats 230 This section defines IOAM data types and data fields and associated 231 data types required for IOAM. 233 To accommodate the different uses of IOAM, IOAM data fields fall into 234 different categories, as specified below. In IOAM these categories 235 are referred to as IOAM-Types. A common registry is maintained for 236 IOAM-Types, see Section 7.2 for details. Corresponding to these 237 IOAM-Types, different IOAM data fields are defined. IOAM data fields 238 can be encapsulated into a variety of protocols, such as NSH, Geneve, 239 IPv6, etc. The definition of how IOAM data fields are encapsulated 240 into other protocols is outside the scope of this document. 242 This document defines four IOAM-Types, as specified in this section: 244 o Pre-allocated Trace Option 246 o Incremental Trace Option 248 o Proof of Transit (POT) Option 250 o Edge-to-Edge (E2E) Option 252 IOAM is expected to be deployed in a specific domain rather than on 253 the overall Internet. The part of the network which employs IOAM is 254 referred to as the "IOAM-domain". IOAM data is added to a packet 255 upon entering the IOAM-domain and is removed from the packet when 256 exiting the domain. Within the IOAM-domain, the IOAM data may be 257 updated by network nodes that the packet traverses. The device which 258 adds an IOAM data container to the packet to capture IOAM data is 259 called the "IOAM encapsulating node", whereas the device which 260 removes the IOAM data container is referred to as the "IOAM 261 decapsulating node". Nodes within the domain which are aware of IOAM 262 data and read and/or write or process the IOAM data are called "IOAM 263 transit nodes". IOAM nodes which add or remove the IOAM data fields 264 can also update the IOAM data fields at the same time. Or in other 265 words, IOAM encapsulating or decapsulating nodes can also serve as 266 IOAM transit nodes at the same time. Note that not every node in an 267 IOAM domain needs to be an IOAM transit node. For example, a 268 deployment might require that packets traverse a set of firewalls. 269 In that case, only the set of firewall nodes would be IOAM transit 270 nodes rather than all nodes. 272 An IOAM encapsulating node incorporates one or more IOAM-Types (from 273 the list of four IOAM-Types above) into packets that IOAM is enabled 274 for. If IOAM is enabled for a selected subset of the traffic, the 275 encapsulating node is responsible for applying the IOAM functionality 276 to the selected subset. 278 An IOAM transit node updates one or more of the IOAM data fields. If 279 both the pre-allocated and the incremental trace options are present 280 in the packet, each IOAM transit node will update at most one of 281 these options. A transit node cannot add new IOAM options to a 282 packet, and cannot change an IOAM Edge-to-Edge Option. 284 An IOAM decapsulating node removes all the IOAM-Types from packets. 286 The role of a node (i.e. encapsulating, transit, decapsulating) is 287 defined within an IOAM namespace (see below). A node can have 288 different roles in different IOAM namespaces. 290 4.1. IOAM Namespaces 292 A subset or all of the IOAM option types and associated IOAM data 293 fields can be associated to an IOAM namespace. Namespaces add 294 further context to IOAM option types and associated IOAM data fields. 295 Any IOAM namespace MUST interpret the IOAM option types and 296 associated IOAM data fields per the definition in this document. 297 Namespaces group nodes to support different deployment approaches of 298 IOAM (see a few example use-cases below) as well as resolve issues 299 which can occur due to IOAM data fields not being globally unique 300 (e.g. IOAM node identifiers do not have to be globally unique). 301 IOAM data fields are defined within an IOAM namespace. 303 An IOAM namespace is identified by a 16-bit namespace identifier 304 (Namespace-ID). Namespace identifiers MUST be present and populated 305 in all IOAM option headers. The Namespace-ID value is divided into 306 two sub-ranges: 308 o An operator-assigned range from 0x0001 to 0x7FFF 310 o An IANA-assigned range from 0x8000 to 0xFFFF 312 The IANA-assigned range is intended to allow future extensions to 313 have new and interoperable IOAM functionality, while the operator- 314 assigned range is intended to be domain specific, and managed by the 315 network operator. The Namespace-ID value of 0x0000 is default and 316 known to all the nodes implementing IOAM. 318 Namespace identifiers allow devices which are IOAM capable to 319 determine: 321 o whether IOAM option header(s) need to be processed by a device: If 322 the Namespace-ID contained in a packet does not match any 323 Namespace-ID the node is configured to operate on, then the node 324 MUST NOT change the contents of the IOAM data fields. 326 o which IOAM option headers need to be processed/updated in case 327 there are multiple IOAM option headers present in the packet. 328 Multiple option headers can be present in a packet in case of 329 overlapping IOAM domains or in case of a layered IOAM deployment. 331 o whether IOAM option header(s) should be removed from the packet, 332 e.g. at a domain edge or domain boundary. 334 IOAM namespaces support several different uses: 336 o Namespaces can be used by an operator to distinguish different 337 operational domains. Devices at domain edges can filter on 338 Namespace-IDs to provide for proper IOAM domain isolation. 340 o Namespaces provide additional context for IOAM data fields and 341 thus ensure that IOAM data is unique and can be interpreted 342 properly by management stations or network controllers. While, 343 for example, the IOAM node identifier (Node-ID) does not need to 344 be unique in a deployment (e.g. an operator may wish to use 345 different Node-IDs for different IOAM layers, even within the same 346 device; or Node-IDs might not be unique for other organizational 347 reasons, such as after a merger of two formerly separated 348 organizations), the combination of Node-ID and Namespace-ID will 349 always be unique. Similarly, namespaces can be used to define how 350 certain IOAM data fields are interpreted: IOAM offers three 351 different timestamp format options. The Namespace-ID can be used 352 to determine the timestamp format. IOAM data fields (e.g. buffer 353 occupancy) which do not have a unit associated are to be 354 interpreted within the context of a namespace. 356 o Namespaces can be used to identify different sets of devices 357 (e.g., different types of devices) in a deployment: If an operator 358 desires to insert different IOAM data based on the device, the 359 devices could be grouped into multiple namespaces. This could be 360 due to the fact that the IOAM feature set differs between 361 different sets of devices, or it could be for reasons of optimized 362 space usage in the packet header. This could also stem from 363 hardware or operational limitations on the size of the trace data 364 that can be added and processed, preventing collection of a full 365 trace for a flow. 367 * Assigning different Namespace-IDs to different sets of nodes or 368 network partitions and using the Namespace-ID as a selector at 369 the IOAM encapsulating node, a full trace for a flow could be 370 collected and constructed via partial traces in different 371 packets of the same flow. Example: An operator could choose to 372 group the devices of a domain into two namespaces, in a way 373 that at average, only every second hop would be recorded by any 374 device. To retrieve a full view of the deployment, the 375 captured IOAM data fields of the two namespaces need to be 376 correlated. 378 * Assigning different Namespace-IDs to different sets of nodes or 379 network partitions and using a separate IOAM header for each 380 Namespace-ID, a full trace for a flow could be collected and 381 constructed via partial traces from each IOAM header in each of 382 the packets in the flow. Example: An operator could choose to 383 group the devices of a domain into two namespaces, in a way 384 that each namespace is represented by one of two IOAM headers 385 in the packet. Each node would record data only for the IOAM 386 namespace that it belongs to, ignoring the other IOAM header 387 with a namespace to which it doesn't belong. To retrieve a 388 full view of the deployment, the captured IOAM data fields of 389 the two namespaces need to be correlated. 391 4.2. IOAM Tracing Options 393 "IOAM tracing data" is expected to be collected at every node that a 394 packet traverses to ensure visibility into the entire path a packet 395 takes within an IOAM domain, i.e., in a typical deployment all nodes 396 in an in-situ OAM-domain would participate in IOAM and thus be IOAM 397 transit nodes, IOAM encapsulating or IOAM decapsulating nodes. If 398 not all nodes within a domain are IOAM capable, IOAM tracing 399 information (i.e., node data) will only be collected on those nodes 400 which are IOAM capable. Nodes which are not IOAM capable will 401 forward the packet without any changes to the IOAM data fields. The 402 maximum number of hops and the minimum path MTU of the IOAM domain is 403 assumed to be known. 405 To optimize hardware and software implementations tracing is defined 406 as two separate options. Any deployment MAY choose to configure and 407 support one or both of the following options. An implementation of 408 the transport protocol that carries these in-situ OAM data MAY choose 409 to support only one of the options. In the event that both options 410 are utilized at the same time, the Incremental Trace Option MUST be 411 placed before the Pre-allocated Trace Option. Given that the 412 operator knows which equipment is deployed in a particular IOAM, the 413 operator will decide by means of configuration which type(s) of trace 414 options will be enabled for a particular domain. 416 Pre-allocated Trace Option: This trace option is defined as a 417 container of node data fields with pre-allocated space for each 418 node to populate its information. This option is useful for 419 software implementations where it is efficient to allocate the 420 space once and index into the array to populate the data during 421 transit. The IOAM encapsulating node allocates the option header 422 and sets the fields in the option header. The in situ OAM 423 encapsulating node allocates an array which is used to store 424 operational data retrieved from every node while the packet 425 traverses the domain. IOAM transit nodes update the content of 426 the array, and possibly update the checksums of outer headers. A 427 pointer which is part of the IOAM trace data points to the next 428 empty slot in the array. An IOAM transit node that updates the 429 content of the pre-allocated option also updates the value of the 430 pointer, which specifies where the next IOAM transit node fills in 431 its data. 433 Incremental Trace Option: This trace option is defined as a 434 container of node data fields where each node allocates and pushes 435 its node data immediately following the option header. This type 436 of trace recording is useful for some of the hardware 437 implementations as this eliminates the need for the transit 438 network elements to read the full array in the option and allows 439 for arbitrarily long packets as the MTU allows. The in-situ OAM 440 encapsulating node allocates the option header. The in-situ OAM 441 encapsulating node based on operational state and configuration 442 sets the fields in the header that control what node data fields 443 should be collected, and how large the node data list can grow. 444 The in-situ OAM transit nodes push their node data to the node 445 data list, decrease the remaining length available to subsequent 446 nodes, and adjust the lengths and possibly checksums in outer 447 headers. 449 Every node data entry is to hold information for a particular IOAM 450 transit node that is traversed by a packet. The in-situ OAM 451 decapsulating node removes the IOAM data and processes and/or exports 452 the metadata. IOAM data uses its own name-space for information such 453 as node identifier or interface identifier. This allows for a 454 domain-specific definition and interpretation. For example: In one 455 case an interface-id could point to a physical interface (e.g., to 456 understand which physical interface of an aggregated link is used 457 when receiving or transmitting a packet) whereas in another case it 458 could refer to a logical interface (e.g., in case of tunnels). 460 The following IOAM data is defined for IOAM tracing: 462 o Identification of the IOAM node. An IOAM node identifier can 463 match to a device identifier or a particular control point or 464 subsystem within a device. 466 o Identification of the interface that a packet was received on, 467 i.e. ingress interface. 469 o Identification of the interface that a packet was sent out on, 470 i.e. egress interface. 472 o Time of day when the packet was processed by the node. Different 473 definitions of processing time are feasible and expected, though 474 it is important that all devices of an in-situ OAM domain follow 475 the same definition. 477 o Generic data: Format-free information where syntax and semantic of 478 the information is defined by the operator in a specific 479 deployment. For a specific deployment, all IOAM nodes should 480 interpret the generic data the same way. Examples for generic 481 IOAM data include geo-location information (location of the node 482 at the time the packet was processed), buffer queue fill level or 483 cache fill level at the time the packet was processed, or even a 484 battery charge level. 486 o A mechanism to detect whether IOAM trace data was added at every 487 hop or whether certain hops in the domain weren't in-situ OAM 488 transit nodes. 490 The "node data list" array in the packet is populated iteratively as 491 the packet traverses the network, starting with the last entry of the 492 array, i.e., "node data list [n]" is the first entry to be populated, 493 "node data list [n-1]" is the second one, etc. 495 4.2.1. Pre-allocated and Incremental Trace Options 497 The in-situ OAM pre-allocated trace option and the in-situ OAM 498 incremental trace option have similar formats. Except where noted 499 below, the internal formats and fields of the two trace options are 500 identical. 502 Pre-allocated and incremental trace option headers: 504 0 1 2 3 505 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 507 | Namespace-ID |NodeLen | Flags | RemainingLen| 508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 | IOAM-Trace-Type | Reserved | 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 512 The trace option data MUST be 4-octet aligned: 514 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 515 | | | 516 | node data list [0] | | 517 | | | 518 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ D 519 | | a 520 | node data list [1] | t 521 | | a 522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 523 ~ ... ~ S 524 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ p 525 | | a 526 | node data list [n-1] | c 527 | | e 528 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 529 | | | 530 | node data list [n] | | 531 | | | 532 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 534 Namespace-ID: 16-bit identifier of an IOAM namespace. The 535 Namespace-ID value of 0x0000 is defined as the default value and 536 MUST be known to all the nodes implementing IOAM. For any other 537 Namespace-ID value that does not match any Namespace-ID the node 538 is configured to operate on, the node MUST NOT change the contents 539 of the IOAM data fields. 541 NodeLen: 5-bit unsigned integer. This field specifies the length of 542 data added by each node in multiples of 4-octets, excluding the 543 length of the "Opaque State Snapshot" field. 545 If IOAM-Trace-Type bit 7 is not set, then NodeLen specifies the 546 actual length added by each node. If IOAM-Trace-Type bit 7 is 547 set, then the actual length added by a node would be (NodeLen + 548 Opaque Data Length). 550 For example, if 3 IOAM-Trace-Type bits are set and none of them 551 are wide, then NodeLen would be 3. If 3 IOAM-Trace-Type bits are 552 set and 2 of them are wide, then NodeLen would be 5. 554 An IOAM encapsulating node must set NodeLen. 556 A node receiving an IOAM Pre-allocated or Incremental Trace Option 557 may rely on the NodeLen value, or it may ignore the NodeLen value 558 and calculate the node length from the IOAM-Trace-Type bits. 560 Flags 4-bit field. Flags are allocated by IANA, as specified in 561 Section 7.4. The current document allocates a single flag as 562 follows: 564 Bit 0 "Overflow" (O-bit) (most significant bit). This bit is set 565 by the network element if there are not enough octets left to 566 record node data, no field is added and the overflow "O-bit" 567 must be set to "1" in the header. This is useful for transit 568 nodes to ignore further processing of the option. 570 RemainingLen: 7-bit unsigned integer. This field specifies the data 571 space in multiples of 4-octets remaining for recording the node 572 data, before the node data list is considered to have overflowed. 573 When RemainingLen reaches 0, nodes are no longer allowed to add 574 node data. Given that the sender knows the minimum path MTU, the 575 sender MAY set the initial value of RemainingLen according to the 576 number of node data bytes allowed before exceeding the MTU. 577 Subsequent nodes can carry out a simple comparison between 578 RemainingLen and NodeLen, along with the length of the "Opaque 579 State Snapshot" if applicable, to determine whether or not data 580 can be added by this node. When node data is added, the node MUST 581 decrease RemainingLen by the amount of data added. In the pre- 582 allocated trace option, this is used as an offset in data space to 583 record the node data element. 585 IOAM-Trace-Type: A 24-bit identifier which specifies which data 586 types are used in this node data list. 588 The IOAM-Trace-Type value is a bit field. The following bit 589 fields are defined in this document, with details on each field 590 described in the Section 4.2.2. The order of packing the data 591 fields in each node data element follows the bit order of the 592 IOAM-Trace-Type field, as follows: 594 Bit 0 (Most significant bit) When set indicates presence of 595 Hop_Lim and node_id in the node data. 597 Bit 1 When set indicates presence of ingress_if_id and 598 egress_if_id (short format) in the node data. 600 Bit 2 When set indicates presence of timestamp seconds in the 601 node data. 603 Bit 3 When set indicates presence of timestamp subseconds in 604 the node data. 606 Bit 4 When set indicates presence of transit delay in the node 607 data. 609 Bit 5 When set indicates presence of namespace specific data 610 (short format) in the node data. 612 Bit 6 When set indicates presence of queue depth in the node 613 data. 615 Bit 7 When set indicates presence of variable length Opaque 616 State Snapshot field. 618 Bit 8 When set indicates presence of Hop_Lim and node_id in 619 wide format in the node data. 621 Bit 9 When set indicates presence of ingress_if_id and 622 egress_if_id in wide format in the node data. 624 Bit 10 When set indicates presence of namespace specific data in 625 wide format in the node data. 627 Bit 11 When set indicates presence of buffer occupancy in the 628 node data. 630 Bit 12-22 Undefined. An IOAM encapsulating node MUST set the 631 value of each of these bits to 0. If an IOAM transit 632 node receives a packet with one or more of these bits set 633 to 1, it must either: 635 1. Add corresponding node data filled with the reserved 636 value 0xFFFFFFFF, after the node data fields for the 637 IOAM-Trace-Type bits defined above, such that the 638 total node data added by this node in units of 639 4-octets is equal to NodeLen, or 641 2. Not add any node data fields to the packet, even for 642 the IOAM-Trace-Type bits defined above. 644 Bit 23 When set indicates presence of the Checksum Complement 645 node data. 647 Section 4.2.2 describes the IOAM data types and their formats. 648 Within an in-situ OAM domain possible combinations of these bits 649 making the IOAM-Trace-Type can be restricted by configuration 650 knobs. 652 Reserved: 8-bits. Must be zero. 654 Node data List [n]: Variable-length field. The type of which is 655 determined by the IOAM-Trace-Type bit representing the n-th node 656 data in the node data list. The node data list is encoded 657 starting from the last node data of the path. The first element 658 of the node data list (node data list [0]) contains the last node 659 of the path while the last node data of the node data list (node 660 data list[n]) contains the first node data of the path traced. 661 Populating the node data list in this way ensures that the order 662 of node data list is the same for incremental and pre-allocated 663 trace options. In the pre-allocated trace option, the index 664 contained in RemainingLen identifies the offset for current active 665 node data to be populated. 667 4.2.2. IOAM node data fields and associated formats 669 All the data fields MUST be 4-octet aligned. If a node which is 670 supposed to update an IOAM data field is not capable of populating 671 the value of a field set in the IOAM-Trace-Type, the field value MUST 672 be set to 0xFFFFFFFF for 4-octet fields or 0xFFFFFFFFFFFFFFFF for 673 8-octet fields, indicating that the value is not populated, except 674 when explicitly specified in the field description below. 676 Data field and associated data type for each of the data field is 677 shown below: 679 Hop_Lim and node_id: 4-octet field defined as follows: 681 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 682 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 683 | Hop_Lim | node_id | 684 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 686 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 687 value in the packet at the node that records this data. Hop 688 Limit information is used to identify the location of the node 689 in the communication path. This is copied from the lower 690 layer, e.g., TTL value in IPv4 header or hop limit field from 691 IPv6 header of the packet when the packet is ready for 692 transmission. The semantics of the Hop_Lim field depend on the 693 lower layer protocol that IOAM is encapsulated over, and 694 therefore its specific semantics are outside the scope of this 695 memo. 697 node_id: 3-octet unsigned integer. Node identifier field to 698 uniquely identify a node within in-situ OAM domain. The 699 procedure to allocate, manage and map the node_ids is beyond 700 the scope of this document. 702 ingress_if_id and egress_if_id: 4-octet field defined as follows: 704 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 705 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 706 | ingress_if_id | egress_if_id | 707 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 709 ingress_if_id: 2-octet unsigned integer. Interface identifier to 710 record the ingress interface the packet was received on. 712 egress_if_id: 2-octet unsigned integer. Interface identifier to 713 record the egress interface the packet is forwarded out of. 715 Note that due to the fact that IOAM uses its own namespaces for 716 IOAM data fields, data fields like interface identifiers can be 717 used in a flexible way to represent system resources that are 718 associated with ingressing or egressing packets, i.e. an IOAM 719 interface ID could represent a physical interface, a virtual or 720 logical interface, or even a queue. 722 timestamp seconds: 4-octet unsigned integer. Absolute timestamp in 723 seconds that specifies the time at which the packet was received 724 by the node. This field has three possible formats; based on 725 either PTP [IEEE1588v2], NTP [RFC5905], or POSIX [POSIX]. The 726 three timestamp formats are specified in Section 5. In all three 727 cases, the Timestamp Seconds field contains the 32 most 728 significant bits of the timestamp format that is specified in 729 Section 5. If a node is not capable of populating this field, it 730 assigns the value 0xFFFFFFFF. Note that this is a legitimate 731 value that is valid for 1 second in approximately 136 years; the 732 analyzer should correlate several packets or compare the timestamp 733 value to its own time-of-day in order to detect the error 734 indication. 736 timestamp subseconds: 4-octet unsigned integer. Absolute timestamp 737 in subseconds that specifies the time at which the packet was 738 received by the node. This field has three possible formats; 739 based on either PTP [IEEE1588v2], NTP [RFC5905], or POSIX [POSIX]. 740 The three timestamp formats are specified in Section 5. In all 741 three cases, the Timestamp Subseconds field contains the 32 least 742 significant bits of the timestamp format that is specified in 743 Section 5. If a node is not capable of populating this field, it 744 assigns the value 0xFFFFFFFF. Note that this is a legitimate 745 value in the NTP format, valid for approximately 233 picoseconds 746 in every second. If the NTP format is used the analyzer should 747 correlate several packets in order to detect the error indication. 749 transit delay: 4-octet unsigned integer in the range 0 to 2^31-1. 750 It is the time in nanoseconds the packet spent in the transit 751 node. This can serve as an indication of the queuing delay at the 752 node. If the transit delay exceeds 2^31-1 nanoseconds then the 753 top bit 'O' is set to indicate overflow and value set to 754 0x80000000. When this field is part of the data field but a node 755 populating the field is not able to fill it, the field position in 756 the field must be filled with value 0xFFFFFFFF to mean not 757 populated. 759 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 760 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 761 |O| transit delay | 762 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 764 namespace specific data: 4-octet field which can be used by the node 765 to add namespace specific data. This represents a "free-format" 766 4-octet bit field with its semantics defined in the context of a 767 specific namespace. 769 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 770 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 771 | namespace specific data | 772 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 774 queue depth: 4-octet unsigned integer field. This field indicates 775 the current length of the egress interface queue of the interface 776 from where the packet is forwarded out. The queue depth is 777 expressed as the current number of memory buffers used by the 778 queue (a packet may consume one or more memory buffers, depending 779 on its size). 781 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 782 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 783 | queue depth | 784 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 786 Opaque State Snapshot: Variable length field. It allows the network 787 element to store an arbitrary state in the node data field, 788 without a pre-defined schema. The schema is to be defined within 789 the context of a namespace. The schema needs to be made known to 790 the analyzer by some out-of-band mechanism. The specification of 791 this mechanism is beyond the scope of this document. A 24-bit 792 "Schema Id" field, interpreted within the context of a namespace, 793 indicates which particular schema is used, and should be 794 configured on the network element by the operator. 796 0 1 2 3 797 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 798 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 799 | Length | Schema ID | 800 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 801 | | 802 | | 803 | Opaque data | 804 ~ ~ 805 . . 806 . . 807 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 809 Length: 1-octet unsigned integer. It is the length in multiples 810 of 4-octets of the Opaque data field that follows Schema Id. 812 Schema ID: 3-octet unsigned integer identifying the schema of 813 Opaque data. 815 Opaque data: Variable length field. This field is interpreted as 816 specified by the schema identified by the Schema ID. 818 When this field is part of the data field but a node populating 819 the field has no opaque state data to report, the Length must be 820 set to 0 and the Schema ID must be set to 0xFFFFFF to mean no 821 schema. 823 Hop_Lim and node_id wide: 8-octet field defined as follows: 825 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 826 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 827 | Hop_Lim | node_id ~ 828 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 829 ~ node_id (contd) | 830 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 832 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 833 value in the packet at the node that records this data. Hop 834 Limit information is used to identify the location of the node 835 in the communication path. This is copied from the lower layer 836 for e.g. TTL value in IPv4 header or hop limit field from IPv6 837 header of the packet. The semantics of the Hop_Lim field 838 depend on the lower layer protocol that IOAM is encapsulated 839 over, and therefore its specific semantics are outside the 840 scope of this memo. 842 node_id: 7-octet unsigned integer. Node identifier field to 843 uniquely identify a node within in-situ OAM domain. The 844 procedure to allocate, manage and map the node_ids is beyond 845 the scope of this document. 847 ingress_if_id and egress_if_id wide: 8-octet field defined as 848 follows: 850 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 851 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 852 | ingress_if_id | 853 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 854 | egress_if_id | 855 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 857 ingress_if_id: 4-octet unsigned integer. Interface identifier to 858 record the ingress interface the packet was received on. 860 egress_if_id: 4-octet unsigned integer. Interface identifier to 861 record the egress interface the packet is forwarded out of. 863 namespace specific data wide: 8-octet field which can be used by the 864 node to add namespace specific data. This represents a "free- 865 format" 8-octet bit field with its semantics defined in the 866 context of a specific namespace. 868 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 869 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 870 | namespace specific data ~ 871 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 872 ~ namespace specific data (contd) | 873 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 875 buffer occupancy: 4-octet unsigned integer field. This field 876 indicates the current status of the occupancy of the common buffer 877 pool used by a set of queues. The units of this field depend on 878 the equipment type and deployment and has to be interpreted within 879 the context of a namespace and/or node-id if used. 881 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 882 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 883 | buffer occupancy | 884 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 886 Checksum Complement: 4-octet node data which contains a two-octet 887 Checksum Complement field, and a 2-octet reserved field. The 888 Checksum Complement is useful when IOAM is transported over 889 encapsulations that make use of a UDP transport, such as VXLAN-GPE 890 or Geneve. Without the Checksum Complement, nodes adding IOAM 891 node data must update the UDP Checksum field. When the Checksum 892 Complement is present, an IOAM encapsulating node or IOAM transit 893 node adding node data MUST carry out one of the following two 894 alternatives in order to maintain the correctness of the UDP 895 Checksum value: 897 1. Recompute the UDP Checksum field. 899 2. Use the Checksum Complement to make a checksum-neutral update 900 in the UDP payload; the Checksum Complement is assigned a 901 value that complements the rest of the node data fields that 902 were added by the current node, causing the existing UDP 903 Checksum field to remain correct. 905 IOAM decapsulating nodes MUST recompute the UDP Checksum field, 906 since they do not know whether previous hops modified the UDP 907 Checksum field or the Checksum Complement field. 909 Checksum Complement fields are used in a similar manner in 910 [RFC7820] and [RFC7821]. 912 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 913 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 914 | Checksum Complement | Reserved | 915 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 917 4.2.3. Examples of IOAM node data 919 An entry in the "node data list" array can have different formats, 920 following the needs of the deployment. Some deployments might only 921 be interested in recording the node identifiers, whereas others might 922 be interested in recording node identifier and timestamp. The 923 section defines different types that an entry in "node data list" can 924 take. 926 0xD40000: IOAM-Trace-Type is 0xD40000 then the format of node data 927 is: 929 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 930 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 931 | Hop_Lim | node_id | 932 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 933 | ingress_if_id | egress_if_id | 934 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 935 | timestamp subseconds | 936 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 937 | namespace specific data | 938 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 940 0xC00000: IOAM-Trace-Type is 0xC00000 then the format is: 942 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 943 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 944 | Hop_Lim | node_id | 945 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 946 | ingress_if_id | egress_if_id | 947 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 949 0x900000: IOAM-Trace-Type is 0x900000 then the format is: 951 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 952 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 953 | Hop_Lim | node_id | 954 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 955 | timestamp subseconds | 956 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 958 0x840000: IOAM-Trace-Type is 0x840000 then the format is: 960 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 961 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 962 | Hop_Lim | node_id | 963 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 964 | namespace specific data | 965 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 967 0x940000: IOAM-Trace-Type is 0x940000 then the format is: 969 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 970 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 971 | Hop_Lim | node_id | 972 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 973 | timestamp subseconds | 974 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 975 | namespace specific data | 976 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 978 0x318000: IOAM-Trace-Type is 0x318000 then the format is: 980 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 981 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 982 | timestamp seconds | 983 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 984 | timestamp subseconds | 985 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 986 | Length | Schema Id | 987 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 988 | | 989 | | 990 | Opaque data | 991 ~ ~ 992 . . 993 . . 994 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 995 | Hop_Lim | node_id | 996 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 997 | node_id(contd) | 998 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1000 4.3. IOAM Proof of Transit Option 1002 IOAM Proof of Transit data is to support the path or service function 1003 chain [RFC7665] verification use cases. Proof-of-transit uses 1004 methods like nested hashing or nested encryption of the IOAM data or 1005 mechanisms such as Shamir's Secret Sharing Schema (SSSS). While 1006 details on how the IOAM data for the proof of transit option is 1007 processed at IOAM encapsulating, decapsulating and transit nodes are 1008 outside the scope of the document, all of these approaches share the 1009 need to uniquely identify a packet as well as iteratively operate on 1010 a set of information that is handed from node to node. 1011 Correspondingly, two pieces of information are added as IOAM data to 1012 the packet: 1014 o Random: Unique identifier for the packet (e.g., 64-bits allow for 1015 the unique identification of 2^64 packets). 1017 o Cumulative: Information which is handed from node to node and 1018 updated by every node according to a verification algorithm. 1020 IOAM proof of transit option header: 1022 0 1 2 3 1023 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1024 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1025 | Namespace-ID |IOAM POT Type | IOAM POT flags| 1026 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1028 IOAM proof of transit option data MUST be 4-octet aligned.: 1030 0 1 2 3 1031 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1032 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1033 | POT Option data field determined by IOAM-POT-Type | 1034 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1036 Namespace-ID: 16-bit identifier of an IOAM namespace. The 1037 Namespace-ID value of 0x0000 is defined as the default value and 1038 MUST be known to all the nodes implementing IOAM. For any other 1039 Namespace-ID value that does not match any Namespace-ID the node 1040 is configured to operate on, the node MUST NOT change the contents 1041 of the IOAM data fields. 1043 IOAM POT Type: 8-bit identifier of a particular POT variant that 1044 specifies the POT data that is included. This document defines 1045 POT Type 0: 1047 0: POT data is a 16 Octet field as described below. 1049 IOAM POT flags: 8-bit. Following flags are defined: 1051 Bit 0 "Profile-to-use" (P-bit) (most significant bit). For IOAM 1052 POT types that use a maximum of two profiles to drive 1053 computation, indicates which POT-profile is used. The two 1054 profiles are numbered 0, 1. 1056 Bit 1-7 Reserved: Must be set to zero upon transmission and 1057 ignored upon receipt. 1059 POT Option data: Variable-length field. The type of which is 1060 determined by the IOAM-POT-Type. 1062 4.3.1. IOAM Proof of Transit Type 0 1064 IOAM proof of transit option of IOAM POT Type 0: 1066 0 1 2 3 1067 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1068 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1069 | Namespace-ID |IOAM POT Type=0|P|R R R R R R R| 1070 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 1071 | Random | | 1072 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ P 1073 | Random(contd) | O 1074 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ T 1075 | Cumulative | | 1076 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 1077 | Cumulative (contd) | | 1078 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 1080 Namespace-ID: 16-bit identifier of an IOAM namespace. The 1081 Namespace-ID value of 0x0000 is defined as the default value and 1082 MUST be known to all the nodes implementing IOAM. For any other 1083 Namespace-ID value that does not match any Namespace-ID the node 1084 is configured to operate on, the node MUST NOT change the contents 1085 of the IOAM data fields. 1087 IOAM POT Type: 8-bit identifier of a particular POT variant that 1088 specifies the POT data that is included. This section defines the 1089 POT data when the IOAM POT Type is set to the value 0. 1091 P bit: 1-bit. "Profile-to-use" (P-bit) (most significant bit). 1092 Indicates which POT-profile is used to generate the Cumulative. 1093 Any node participating in POT will have a maximum of 2 profiles 1094 configured that drive the computation of cumulative. The two 1095 profiles are numbered 0, 1. This bit conveys whether profile 0 or 1096 profile 1 is used to compute the Cumulative. 1098 R (7 bits): 7-bit IOAM POT flags for future use. MUST be set to 1099 zero upon transmission and ignored upon receipt. 1101 Random: 64-bit Per packet Random number. 1103 Cumulative: 64-bit Cumulative that is updated at specific nodes by 1104 processing per packet Random number field and configured 1105 parameters. 1107 Note: Larger or smaller sizes of "Random" and "Cumulative" data are 1108 feasible and could be required for certain deployments (e.g. in case 1109 of space constraints in the transport protocol used). Future 1110 versions of this document will address different sizes of data for 1111 "proof of transit". 1113 4.4. IOAM Edge-to-Edge Option 1115 The IOAM edge-to-edge option is to carry data that is added by the 1116 IOAM encapsulating node and interpreted by IOAM decapsulating node. 1117 The IOAM transit nodes MAY process the data without modifying it. 1119 IOAM edge-to-edge option header: 1121 0 1 2 3 1122 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1123 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1124 | Namespace-ID | IOAM-E2E-Type | 1125 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1127 IOAM edge-to-edge option data MUST be 4-octet aligned: 1129 0 1 2 3 1130 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1131 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1132 | E2E Option data field determined by IOAM-E2E-Type | 1133 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1135 Namespace-ID: 16-bit identifier of an IOAM namespace. The 1136 Namespace-ID value of 0x0000 is defined as the default value and 1137 MUST be known to all the nodes implementing IOAM. For any other 1138 Namespace-ID value that does not match any Namespace-ID the node 1139 is configured to operate on, then the node MUST NOT change the 1140 contents of the IOAM data fields. 1142 IOAM-E2E-Type: A 16-bit identifier which specifies which data types 1143 are used in the E2E option data. The IOAM-E2E-Type value is a bit 1144 field. The order of packing the E2E option data field elements 1145 follows the bit order of the IOAM-E2E-Type field, as follows: 1147 Bit 0 (Most significant bit) When set indicates presence of a 1148 64-bit sequence number added to a specific "packet group" 1149 which is used to detect packet loss, packet reordering, 1150 or packet duplication within the group. The "packet 1151 group" is deployment dependent and defined at the IOAM 1152 encapsulating node e.g. by n-tuple based classification 1153 of packets. 1155 Bit 1 When set indicates presence of a 32-bit sequence number 1156 added to a specific "packet group" which is used to 1157 detect packet loss, packet reordering, or packet 1158 duplication within that group. The "packet group" is 1159 deployment dependent and defined at the IOAM 1160 encapsulating node e.g. by n-tuple based classification 1161 of packets. 1163 Bit 2 When set indicates presence of timestamp seconds for the 1164 transmission of the frame. This 4-octet field has three 1165 possible formats; based on either PTP [IEEE1588v2], NTP 1166 [RFC5905], or POSIX [POSIX]. The three timestamp formats 1167 are specified in Section 5. In all three cases, the 1168 Timestamp Seconds field contains the 32 most significant 1169 bits of the timestamp format that is specified in 1170 Section 5. If a node is not capable of populating this 1171 field, it assigns the value 0xFFFFFFFF. Note that this 1172 is a legitimate value that is valid for 1 second in 1173 approximately 136 years; the analyzer should correlate 1174 several packets or compare the timestamp value to its own 1175 time-of-day in order to detect the error indication. 1177 Bit 3 When set indicates presence of timestamp subseconds for 1178 the transmission of the frame. This 4-octet field has 1179 three possible formats; based on either PTP [IEEE1588v2], 1180 NTP [RFC5905], or POSIX [POSIX]. The three timestamp 1181 formats are specified in Section 5. In all three cases, 1182 the Timestamp Subseconds field contains the 32 least 1183 significant bits of the timestamp format that is 1184 specified in Section 5. If a node is not capable of 1185 populating this field, it assigns the value 0xFFFFFFFF. 1186 Note that this is a legitimate value in the NTP format, 1187 valid for approximately 233 picoseconds in every second. 1188 If the NTP format is used the analyzer should correlate 1189 several packets in order to detect the error indication. 1191 Bit 4-15 Undefined. An IOAM encapsulating node Must set the value 1192 of these bits to zero upon transmission and ignore upon 1193 receipt. 1195 E2E Option data: Variable-length field. The type of which is 1196 determined by the IOAM-E2E-Type. 1198 5. Timestamp Formats 1200 The IOAM data fields include a timestamp field which is represented 1201 in one of three possible timestamp formats. It is assumed that the 1202 management plane is responsible for determining which timestamp 1203 format is used. 1205 5.1. PTP Truncated Timestamp Format 1207 The Precision Time Protocol (PTP) [IEEE1588v2] uses an 80-bit 1208 timestamp format. The truncated timestamp format is a 64-bit field, 1209 which is the 64 least significant bits of the 80-bit PTP timestamp. 1210 The PTP truncated format is specified in Section 4.3 of 1211 [I-D.ietf-ntp-packet-timestamps], and the details are presented below 1212 for the sake of completeness. 1214 0 1 2 3 1215 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1216 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1217 | Seconds | 1218 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1219 | Nanoseconds | 1220 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1222 Figure 1: PTP [IEEE1588v2] Truncated Timestamp Format 1224 Timestamp field format: 1226 Seconds: specifies the integer portion of the number of seconds 1227 since the epoch. 1229 + Size: 32 bits. 1231 + Units: seconds. 1233 Nanoseconds: specifies the fractional portion of the number of 1234 seconds since the epoch. 1236 + Size: 32 bits. 1238 + Units: nanoseconds. The value of this field is in the range 0 1239 to (10^9)-1. 1241 Epoch: 1243 The PTP [IEEE1588v2] epoch is 1 January 1970 00:00:00 TAI, which 1244 is 31 December 1969 23:59:51.999918 UTC. 1246 Resolution: 1248 The resolution is 1 nanosecond. 1250 Wraparound: 1252 This time format wraps around every 2^32 seconds, which is roughly 1253 136 years. The next wraparound will occur in the year 2106. 1255 Synchronization Aspects: 1257 It is assumed that nodes that run this protocol are synchronized 1258 among themselves. Nodes may be synchronized to a global reference 1259 time. Note that if PTP [IEEE1588v2] is used for synchronization, 1260 the timestamp may be derived from the PTP-synchronized clock, 1261 allowing the timestamp to be measured with respect to the clock of 1262 an PTP Grandmaster clock. 1264 The PTP truncated timestamp format is not affected by leap 1265 seconds. 1267 5.2. NTP 64-bit Timestamp Format 1269 The Network Time Protocol (NTP) [RFC5905] timestamp format is 64 bits 1270 long. This format is specified in Section 4.2.1 of 1271 [I-D.ietf-ntp-packet-timestamps], and the details are presented below 1272 for the sake of completeness. 1274 0 1 2 3 1275 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1276 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1277 | Seconds | 1278 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1279 | Fraction | 1280 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1282 Figure 2: NTP [RFC5905] 64-bit Timestamp Format 1284 Timestamp field format: 1286 Seconds: specifies the integer portion of the number of seconds 1287 since the epoch. 1289 + Size: 32 bits. 1291 + Units: seconds. 1293 Fraction: specifies the fractional portion of the number of 1294 seconds since the epoch. 1296 + Size: 32 bits. 1298 + Units: the unit is 2^(-32) seconds, which is roughly equal to 1299 233 picoseconds. 1301 Epoch: 1303 The epoch is 1 January 1900 at 00:00 UTC. 1305 Resolution: 1307 The resolution is 2^(-32) seconds. 1309 Wraparound: 1311 This time format wraps around every 2^32 seconds, which is roughly 1312 136 years. The next wraparound will occur in the year 2036. 1314 Synchronization Aspects: 1316 Nodes that use this timestamp format will typically be 1317 synchronized to UTC using NTP [RFC5905]. Thus, the timestamp may 1318 be derived from the NTP-synchronized clock, allowing the timestamp 1319 to be measured with respect to the clock of an NTP server. 1321 The NTP timestamp format is affected by leap seconds; it 1322 represents the number of seconds since the epoch minus the number 1323 of leap seconds that have occurred since the epoch. The value of 1324 a timestamp during or slightly after a leap second may be 1325 temporarily inaccurate. 1327 5.3. POSIX-based Timestamp Format 1329 This timestamp format is based on the POSIX time format [POSIX]. The 1330 detailed specification of the timestamp format used in this document 1331 is presented below. 1333 0 1 2 3 1334 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1336 | Seconds | 1337 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1338 | Microseconds | 1339 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1341 Figure 3: POSIX-based Timestamp Format 1343 Timestamp field format: 1345 Seconds: specifies the integer portion of the number of seconds 1346 since the epoch. 1348 + Size: 32 bits. 1350 + Units: seconds. 1352 Microseconds: specifies the fractional portion of the number of 1353 seconds since the epoch. 1355 + Size: 32 bits. 1357 + Units: the unit is microseconds. The value of this field is in 1358 the range 0 to (10^6)-1. 1360 Epoch: 1362 The epoch is 1 January 1970 00:00:00 TAI, which is 31 December 1363 1969 23:59:51.999918 UTC. 1365 Resolution: 1367 The resolution is 1 microsecond. 1369 Wraparound: 1371 This time format wraps around every 2^32 seconds, which is roughly 1372 136 years. The next wraparound will occur in the year 2106. 1374 Synchronization Aspects: 1376 It is assumed that nodes that use this timestamp format run Linux 1377 operating system, and hence use the POSIX time. In some cases 1378 nodes may be synchronized to UTC using a synchronization mechanism 1379 that is outside the scope of this document, such as NTP [RFC5905]. 1380 Thus, the timestamp may be derived from the NTP-synchronized 1381 clock, allowing the timestamp to be measured with respect to the 1382 clock of an NTP server. 1384 The POSIX-based timestamp format is affected by leap seconds; it 1385 represents the number of seconds since the epoch minus the number 1386 of leap seconds that have occurred since the epoch. The value of 1387 a timestamp during or slightly after a leap second may be 1388 temporarily inaccurate. 1390 6. IOAM Data Export 1392 IOAM nodes collect information for packets traversing a domain that 1393 supports IOAM. IOAM decapsulating nodes as well as IOAM transit 1394 nodes can choose to retrieve IOAM information from the packet, 1395 process the information further and export the information using 1396 e.g., IPFIX. The mechanisms and associated data formats for 1397 exporting IOAM data is outside the scope of this document. 1399 Raw data export of IOAM data using IPFIX is discussed in 1400 [I-D.spiegel-ippm-ioam-rawexport]. 1402 7. IANA Considerations 1404 This document requests the following IANA Actions. 1406 7.1. Creation of a new In-Situ OAM Protocol Parameters Registry (IOAM) 1407 Protocol Parameters IANA registry 1409 IANA is requested to create a new protocol registry for "In-Situ OAM 1410 (IOAM) Protocol Parameters". This is the common registry that will 1411 include registrations for all IOAM namespaces. Each Registry, whose 1412 names are listed below: 1414 IOAM Type 1416 IOAM Trace Type 1418 IOAM Trace flags 1420 IOAM POT Type 1422 IOAM POT flags 1424 IOAM E2E Type 1426 IOAM Namespace-ID 1428 will contain the current set of possibilities defined in this 1429 document. New registries in this name space are created via RFC 1430 Required process as per [RFC8126]. 1432 The subsequent sub-sections detail the registries herein contained. 1434 7.2. IOAM Type Registry 1436 This registry defines 128 code points for the IOAM-Type field for 1437 identifying IOAM options as explained in Section 4. The following 1438 code points are defined in this draft: 1440 0 IOAM Pre-allocated Trace Option Type 1442 1 IOAM Incremental Trace Option Type 1444 2 IOAM POT Option Type 1446 3 IOAM E2E Option Type 1448 4 - 127 are available for assignment via RFC Required process as per 1449 [RFC8126]. 1451 7.3. IOAM Trace Type Registry 1453 This registry defines code point for each bit in the 24-bit IOAM- 1454 Trace-Type field for Pre-allocated trace option and Incremental trace 1455 option defined in Section 4.2. The meaning of Bits 0 - 11 for trace 1456 type are defined in this document in Paragraph 5 of Section 4.2.1: 1458 Bit 0 hop_Lim and node_id in short format 1460 Bit 1 ingress_if_id and egress_if_id in short format 1462 Bit 2 timestamp seconds 1464 Bit 3 timestamp subseconds 1466 Bit 4 transit delay 1468 Bit 5 namespace specific data in short format 1470 Bit 6 queue depth 1472 Bit 7 variable length Opaque State Snapshot 1474 Bit 8 hop_Lim and node_id in wide format 1475 Bit 9 ingress_if_id and egress_if_id in wide format 1477 Bit 10 namespace specific data in wide format 1479 Bit 11 buffer occupancy 1481 Bit 23 checksum complement 1483 The meaning for Bits 12 - 22 are available for assignment via RFC 1484 Required process as per [RFC8126]. 1486 7.4. IOAM Trace Flags Registry 1488 This registry defines code points for each bit in the 4 bit flags for 1489 the Pre-allocated trace option and for the Incremental trace option 1490 defined in Section 4.2. The meaning of Bit 0 (the most significant 1491 bit) for trace flags is defined in this document in Paragraph 3 of 1492 Section 4.2.1: 1494 Bit 0 "Overflow" (O-bit) 1496 7.5. IOAM POT Type Registry 1498 This registry defines 256 code points to define IOAM POT Type for 1499 IOAM proof of transit option Section 4.3. The code point value 0 is 1500 defined in this document: 1502 0: 16 Octet POT data 1504 1 - 255 are available for assignment via RFC Required process as per 1505 [RFC8126]. 1507 7.6. IOAM POT Flags Registry 1509 This registry defines code points for each bit in the 8 bit flags for 1510 IOAM POT option defined in Section 4.3. The meaning of Bit 0 for 1511 IOAM POT flags is defined in this document in Section 4.3: 1513 Bit 0 "Profile-to-use" (P-bit) 1515 The meaning for Bits 1 - 7 are available for assignment via RFC 1516 Required process as per [RFC8126]. 1518 7.7. IOAM E2E Type Registry 1520 This registry defines code points for each bit in the 16 bit IOAM- 1521 E2E-Type field for IOAM E2E option Section 4.4. The meaning of Bit 0 1522 - 3 are defined in this document: 1524 Bit 0 64-bit sequence number 1526 Bit 1 32-bit sequence number 1528 Bit 2 timestamp seconds 1530 Bit 3 timestamp subseconds 1532 The meaning of Bits 4 - 15 are available for assignment via RFC 1533 Required process as per [RFC8126]. 1535 7.8. IOAM Namespace-ID Registry 1537 IANA is requested to set up an "IOAM Namespace-ID Registry", 1538 containing 16-bit values. The meaning of Bit 0 is defined in this 1539 document. IANA is requested to reserve the values 0x0001 to 0x7FFF 1540 for private use (managed by operators), as specified in Section 4.1 1541 of the current document. Registry entries for the values 0x8000 to 1542 0xFFFF are to be assigned via the "Expert Review" policy defined in 1543 [RFC8126]. 1545 0: default namespace (known to all IOAM nodes) 1547 0x0001 - 0x7FFF: reserved for private use 1549 0x8000 - 0xFFFF: unassigned 1551 8. Security Considerations 1553 As discussed in [RFC7276], a successful attack on an OAM protocol in 1554 general, and specifically on IOAM, can prevent the detection of 1555 failures or anomalies, or create a false illusion of nonexistent 1556 ones. 1558 The Proof of Transit option (Section Section 4.3) is used for 1559 verifying the path of data packets. The security considerations of 1560 POT are further discussed in [I-D.brockners-proof-of-transit]. 1562 The data elements of IOAM can be used for network reconnaissance, 1563 allowing attackers to collect information about network paths, 1564 performance, queue states, buffer occupancy and other information. 1565 Note that in case IOAM is used in "immediate export" mode (reference 1566 to be added in a future revision), the IOAM related trace information 1567 would not be available in the customer data packets, but would 1568 trigger export of packet related IOAM information at every node. 1569 IOAM data export and securing IOAM data export is outside the scope 1570 of this document. 1572 IOAM can be used as a means for implementing Denial of Service (DoS) 1573 attacks, or for amplifying them. For example, a malicious attacker 1574 can add an IOAM header to packets in order to consume the resources 1575 of network devices that take part in IOAM or collectors that analyze 1576 the IOAM data. Another example is a packet length attack, in which 1577 an attacker pushes IOAM headers into data packets, causing these 1578 packets to be increased beyond the MTU size, resulting in 1579 fragmentation or in packet drops. 1581 Since IOAM options may include timestamps, if network devices use 1582 synchronization protocols then any attack on the time protocol 1583 [RFC7384] can compromise the integrity of the timestamp-related data 1584 fields. 1586 At the management plane, attacks may be implemented by misconfiguring 1587 or by maliciously configuring IOAM-enabled nodes in a way that 1588 enables other attacks. Thus, IOAM configuration should be secured in 1589 a way that authenticates authorized users and verifies the integrity 1590 of configuration procedures. 1592 Notably, IOAM is expected to be deployed in specific network domains, 1593 thus confining the potential attack vectors to within the network 1594 domain. Indeed, in order to limit the scope of threats to within the 1595 current network domain the network operator is expected to enforce 1596 policies that prevent IOAM traffic from leaking outside of the IOAM 1597 domain, and prevent IOAM data from outside the domain to be processed 1598 and used within the domain. Note that the Immediate Export mode 1599 (reference to be added in a future revision) can mitigate the 1600 potential threat of IOAM data leaking through data packets. 1602 9. Acknowledgements 1604 The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari 1605 Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya 1606 Nadahalli, LJ Wobker, Erik Nordmark, Vengada Prasad Govindan, Andrew 1607 Yourtchenko, Aviv Kfir, Tianran Zhou, Haoyu song and Robin 1608 for the comments and advice. 1610 This document leverages and builds on top of several concepts 1611 described in [I-D.kitamura-ipv6-record-route]. The authors would 1612 like to acknowledge the work done by the author Hiroshi Kitamura and 1613 people involved in writing it. 1615 The authors would like to gracefully acknowledge useful review and 1616 insightful comments received from Joe Clarke, Al Morton, and Mickey 1617 Spiegel. 1619 The authors would like to acknowledge the contribution of "Immediate 1620 export" of IOAM trace by Barak Gafni. 1622 10. References 1624 10.1. Normative References 1626 [IEEE1588v2] 1627 Institute of Electrical and Electronics Engineers, "IEEE 1628 Std 1588-2008 - IEEE Standard for a Precision Clock 1629 Synchronization Protocol for Networked Measurement and 1630 Control Systems", IEEE Std 1588-2008, 2008, 1631 . 1634 [POSIX] Institute of Electrical and Electronics Engineers, "IEEE 1635 Std 1003.1-2008 (Revision of IEEE Std 1003.1-2004) - IEEE 1636 Standard for Information Technology - Portable Operating 1637 System Interface (POSIX(R))", IEEE Std 1003.1-2008, 2008, 1638 . 1641 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1642 Requirement Levels", BCP 14, RFC 2119, 1643 DOI 10.17487/RFC2119, March 1997, 1644 . 1646 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 1647 "Network Time Protocol Version 4: Protocol and Algorithms 1648 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 1649 . 1651 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 1652 Writing an IANA Considerations Section in RFCs", BCP 26, 1653 RFC 8126, DOI 10.17487/RFC8126, June 2017, 1654 . 1656 10.2. Informative References 1658 [I-D.brockners-proof-of-transit] 1659 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 1660 Leddy, J., Youell, S., Mozes, D., and T. Mizrahi, "Proof 1661 of Transit", draft-brockners-proof-of-transit-05 (work in 1662 progress), May 2018. 1664 [I-D.ietf-ntp-packet-timestamps] 1665 Mizrahi, T., Fabini, J., and A. Morton, "Guidelines for 1666 Defining Packet Timestamps", draft-ietf-ntp-packet- 1667 timestamps-06 (work in progress), February 2019. 1669 [I-D.ietf-nvo3-geneve] 1670 Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic 1671 Network Virtualization Encapsulation", draft-ietf- 1672 nvo3-geneve-13 (work in progress), March 2019. 1674 [I-D.ietf-nvo3-vxlan-gpe] 1675 Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol 1676 Extension for VXLAN", draft-ietf-nvo3-vxlan-gpe-07 (work 1677 in progress), April 2019. 1679 [I-D.kitamura-ipv6-record-route] 1680 Kitamura, H., "Record Route for IPv6 (PR6) Hop-by-Hop 1681 Option Extension", draft-kitamura-ipv6-record-route-00 1682 (work in progress), November 2000. 1684 [I-D.lapukhov-dataplane-probe] 1685 Lapukhov, P. and r. remy@barefootnetworks.com, "Data-plane 1686 probe for in-band telemetry collection", draft-lapukhov- 1687 dataplane-probe-01 (work in progress), June 2016. 1689 [I-D.spiegel-ippm-ioam-rawexport] 1690 Spiegel, M., Brockners, F., Bhandari, S., and R. 1691 Sivakolundu, "In-situ OAM raw data export with IPFIX", 1692 draft-spiegel-ippm-ioam-rawexport-01 (work in progress), 1693 October 2018. 1695 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1696 Weingarten, "An Overview of Operations, Administration, 1697 and Maintenance (OAM) Tools", RFC 7276, 1698 DOI 10.17487/RFC7276, June 2014, 1699 . 1701 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1702 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1703 October 2014, . 1705 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 1706 Chaining (SFC) Architecture", RFC 7665, 1707 DOI 10.17487/RFC7665, October 2015, 1708 . 1710 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1711 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1712 May 2016, . 1714 [RFC7820] Mizrahi, T., "UDP Checksum Complement in the One-Way 1715 Active Measurement Protocol (OWAMP) and Two-Way Active 1716 Measurement Protocol (TWAMP)", RFC 7820, 1717 DOI 10.17487/RFC7820, March 2016, 1718 . 1720 [RFC7821] Mizrahi, T., "UDP Checksum Complement in the Network Time 1721 Protocol (NTP)", RFC 7821, DOI 10.17487/RFC7821, March 1722 2016, . 1724 [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., 1725 "Network Service Header (NSH)", RFC 8300, 1726 DOI 10.17487/RFC8300, January 2018, 1727 . 1729 Authors' Addresses 1731 Frank Brockners 1732 Cisco Systems, Inc. 1733 Hansaallee 249, 3rd Floor 1734 DUESSELDORF, NORDRHEIN-WESTFALEN 40549 1735 Germany 1737 Email: fbrockne@cisco.com 1739 Shwetha Bhandari 1740 Cisco Systems, Inc. 1741 Cessna Business Park, Sarjapura Marathalli Outer Ring Road 1742 Bangalore, KARNATAKA 560 087 1743 India 1745 Email: shwethab@cisco.com 1747 Carlos Pignataro 1748 Cisco Systems, Inc. 1749 7200-11 Kit Creek Road 1750 Research Triangle Park, NC 27709 1751 United States 1753 Email: cpignata@cisco.com 1754 Hannes Gredler 1755 RtBrick Inc. 1757 Email: hannes@rtbrick.com 1759 John Leddy 1760 United States 1762 Email: john@leddy.net 1764 Stephen Youell 1765 JP Morgan Chase 1766 25 Bank Street 1767 London E14 5JP 1768 United Kingdom 1770 Email: stephen.youell@jpmorgan.com 1772 Tal Mizrahi 1773 Huawei Network.IO Innovation Lab 1774 Israel 1776 Email: tal.mizrahi.phd@gmail.com 1778 David Mozes 1780 Email: mosesster@gmail.com 1782 Petr Lapukhov 1783 Facebook 1784 1 Hacker Way 1785 Menlo Park, CA 94025 1786 US 1788 Email: petr@fb.com 1790 Remy Chang 1791 Barefoot Networks 1792 4750 Patrick Henry Drive 1793 Santa Clara, CA 95054 1794 US 1795 Daniel Bernier 1796 Bell Canada 1797 Canada 1799 Email: daniel.bernier@bell.ca 1801 John Lemon 1802 Broadcom 1803 270 Innovation Drive 1804 San Jose, CA 95134 1805 US 1807 Email: john.lemon@broadcom.com