idnits 2.17.1 draft-ietf-ippm-ioam-data-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 4, 2017) is 2418 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 615 -- Looks like a reference, but probably isn't: '1' on line 517 -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE1588v2' == Outdated reference: A later version (-16) exists of draft-ietf-nvo3-geneve-04 == Outdated reference: A later version (-13) exists of draft-ietf-nvo3-vxlan-gpe-04 == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-19 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ippm F. Brockners 3 Internet-Draft S. Bhandari 4 Intended status: Standards Track C. Pignataro 5 Expires: March 8, 2018 Cisco 6 H. Gredler 7 RtBrick Inc. 8 J. Leddy 9 Comcast 10 S. Youell 11 JPMC 12 T. Mizrahi 13 Marvell 14 D. Mozes 15 Mellanox Technologies Ltd. 16 P. Lapukhov 17 Facebook 18 R. Chang 19 Barefoot Networks 20 D. Bernier 21 Bell Canada 22 September 4, 2017 24 Data Fields for In-situ OAM 25 draft-ietf-ippm-ioam-data-00 27 Abstract 29 In-situ Operations, Administration, and Maintenance (IOAM) records 30 operational and telemetry information in the packet while the packet 31 traverses a path between two points in the network. This document 32 discusses the data fields and associated data types for in-situ OAM. 33 In-situ OAM data fields can be embedded into a variety of transports 34 such as NSH, Segment Routing, Geneve, native IPv6 (via extension 35 header), or IPv4. In-situ OAM can be used to complement OAM 36 mechanisms based on e.g. ICMP or other types of probe packets. 38 Status of This Memo 40 This Internet-Draft is submitted in full conformance with the 41 provisions of BCP 78 and BCP 79. 43 Internet-Drafts are working documents of the Internet Engineering 44 Task Force (IETF). Note that other groups may also distribute 45 working documents as Internet-Drafts. The list of current Internet- 46 Drafts is at http://datatracker.ietf.org/drafts/current/. 48 Internet-Drafts are draft documents valid for a maximum of six months 49 and may be updated, replaced, or obsoleted by other documents at any 50 time. It is inappropriate to use Internet-Drafts as reference 51 material or to cite them other than as "work in progress." 53 This Internet-Draft will expire on March 8, 2018. 55 Copyright Notice 57 Copyright (c) 2017 IETF Trust and the persons identified as the 58 document authors. All rights reserved. 60 This document is subject to BCP 78 and the IETF Trust's Legal 61 Provisions Relating to IETF Documents 62 (http://trustee.ietf.org/license-info) in effect on the date of 63 publication of this document. Please review these documents 64 carefully, as they describe your rights and restrictions with respect 65 to this document. Code Components extracted from this document must 66 include Simplified BSD License text as described in Section 4.e of 67 the Trust Legal Provisions and are provided without warranty as 68 described in the Simplified BSD License. 70 Table of Contents 72 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 73 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 74 3. Scope, Applicability, and Assumptions . . . . . . . . . . . . 4 75 4. IOAM Data Types and Formats . . . . . . . . . . . . . . . . . 5 76 4.1. IOAM Tracing Options . . . . . . . . . . . . . . . . . . 6 77 4.1.1. Pre-allocated Trace Option . . . . . . . . . . . . . 8 78 4.1.2. Incremental Trace Option . . . . . . . . . . . . . . 11 79 4.1.3. IOAM node data fields and associated formats . . . . 14 80 4.1.4. Examples of IOAM node data . . . . . . . . . . . . . 19 81 4.2. IOAM Proof of Transit Option . . . . . . . . . . . . . . 21 82 4.3. IOAM Edge-to-Edge Option . . . . . . . . . . . . . . . . 23 83 5. IOAM Data Export . . . . . . . . . . . . . . . . . . . . . . 23 84 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 85 6.1. Creation of a New In-Situ OAM 86 (IOAM) Protocol Parameters IANA registry . . . . . . . . 24 87 6.2. IOAM Trace Type Registry . . . . . . . . . . . . . . . . 24 88 6.3. IOAM Trace Flags Registry . . . . . . . . . . . . . . . . 24 89 6.4. IOAM POT Type Registry . . . . . . . . . . . . . . . . . 25 90 6.5. IOAM E2E Type Registry . . . . . . . . . . . . . . . . . 25 91 7. Manageability Considerations . . . . . . . . . . . . . . . . 25 92 8. Security Considerations . . . . . . . . . . . . . . . . . . . 25 93 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 94 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 95 10.1. Normative References . . . . . . . . . . . . . . . . . . 25 96 10.2. Informative References . . . . . . . . . . . . . . . . . 26 97 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 99 1. Introduction 101 This document defines data fields for "in-situ" Operations, 102 Administration, and Maintenance (IOAM). In-situ OAM records OAM 103 information within the packet while the packet traverses a particular 104 network domain. The term "in-situ" refers to the fact that the OAM 105 data is added to the data packets rather than is being sent within 106 packets specifically dedicated to OAM. A discussion of the 107 motivation and requirements for in-situ OAM can be found in 108 [I-D.brockners-inband-oam-requirements]. IOAM is to complement 109 mechanisms such as Ping or Traceroute, or more recent active probing 110 mechanisms as described in [I-D.lapukhov-dataplane-probe]. In terms 111 of "active" or "passive" OAM, "in-situ" OAM can be considered a 112 hybrid OAM type. While no extra packets are sent, IOAM adds 113 information to the packets therefore cannot be considered passive. 114 In terms of the classification given in [RFC7799] IOAM could be 115 portrayed as Hybrid Type 1. "In-situ" mechanisms do not require 116 extra packets to be sent and hence don't change the packet traffic 117 mix within the network. IOAM mechanisms can be leveraged where 118 mechanisms using e.g. ICMP do not apply or do not offer the desired 119 results, such as proving that a certain traffic flow takes a pre- 120 defined path, SLA verification for the live data traffic, detailed 121 statistics on traffic distribution paths in networks that distribute 122 traffic across multiple paths, or scenarios in which probe traffic is 123 potentially handled differently from regular data traffic by the 124 network devices. 126 2. Conventions 128 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 129 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 130 document are to be interpreted as described in [RFC2119]. 132 Abbreviations used in this document: 134 E2E Edge to Edge 136 Geneve: Generic Network Virtualization Encapsulation 137 [I-D.ietf-nvo3-geneve] 139 IOAM: In-situ Operations, Administration, and Maintenance 141 MTU: Maximum Transmit Unit 143 NSH: Network Service Header [I-D.ietf-sfc-nsh] 144 OAM: Operations, Administration, and Maintenance 146 POT: Proof of Transit 148 SFC: Service Function Chain 150 SID: Segment Identifier 152 SR: Segment Routing 154 VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol 155 Extension [I-D.ietf-nvo3-vxlan-gpe] 157 3. Scope, Applicability, and Assumptions 159 IOAM deployment assumes a set of constraints, requirements, and 160 guiding principles which are described in this section. 162 Scope: This document defines the data fields and associated data 163 types for in-situ OAM. The in-situ OAM data field can be transported 164 by a variety of transport protocols, including NSH, Segment Routing, 165 Geneve, IPv6, or IPv4. Specification details for these different 166 transport protocols are outside the scope of this document. 168 Deployment domain (or scope) of in-situ OAM deployment: IOAM is a 169 network domain focused feature, with "network domain" being a set of 170 network devices or entities within a single administration. For 171 example, a network domain can include an enterprise campus using 172 physical connections between devices or an overlay network using 173 virtual connections / tunnels for connectivity between said devices. 174 A network domain is defined by its perimeter or edge. Designers of 175 carrier protocols for IOAM must specify mechanisms to ensure that 176 IOAM data stays within an IOAM domain. In addition, the operator of 177 such a domain is expected to put provisions in place to ensure that 178 IOAM data does not leak beyond the edge of an IOAM domain, e.g. using 179 for example packet filtering methods. The operator should consider 180 potential operational impact of IOAM to mechanisms such as ECMP 181 processing (e.g. load-balancing schemes based on packet length could 182 be impacted by the increased packet size due to IOAM), path MTU (i.e. 183 ensure that the MTU of all links within a domain is sufficiently 184 large to support the increased packet size due to IOAM) and ICMP 185 message handling (i.e. in case of a native IPv6 transport, IOAM 186 support for ICMPv6 Echo Request/Reply could desired which would 187 translate into ICMPv6 extensions to enable IOAM data fields to be 188 copied from an Echo Request message to an Echo Reply message). 190 IOAM control points: IOAM data fields are added to or removed from 191 the live user traffic by the devices which form the edge of a domain. 193 Devices within an IOAM domain can update and/or add IOAM data-fields. 194 Domain edge devices can be hosts or network devices. 196 Traffic-sets that IOAM is applied to: IOAM can be deployed on all or 197 only on subsets of the live user traffic. It SHOULD be possible to 198 enable IOAM on a selected set of traffic (e.g., per interface, based 199 on an access control list or flow specification defining a specific 200 set of traffic, etc.) The selected set of traffic can also be all 201 traffic. 203 Encapsulation independence: Data formats for IOAM SHOULD be defined 204 in a transport-independent manner. IOAM applies to a variety of 205 encapsulating protocols. A definition of how IOAM data fields are 206 carried by different transport protocols is outside the scope of this 207 document. 209 Layering: If several encapsulation protocols (e.g., in case of 210 tunneling) are stacked on top of each other, IOAM data-records could 211 be present at every layer. The behavior follows the ships-in-the- 212 night model. 214 Combination with active OAM mechanisms: IOAM should be usable for 215 active network probing, enabling for example a customized version of 216 traceroute. Decapsulating IOAM nodes may have an ability to send the 217 IOAM information retrieved from the packet back to the source address 218 of the packet or to the encapsulating node. 220 IOAM implementation: The IOAM data-field definitions take the 221 specifics of devices with hardware data-plane and software data-plane 222 into account. 224 4. IOAM Data Types and Formats 226 This section defines IOAM data types and data fields and associated 227 data types required for IOAM. The different uses of IOAM require the 228 definition of different types of data. The IOAM data fields for the 229 data being carried corresponds to the three main categories of IOAM 230 data defined in [I-D.brockners-inband-oam-requirements], which are: 231 edge-to-edge, per node, and for selected nodes only. 233 Transport options for IOAM data are outside the scope of this memo, 234 and are discussed in [I-D.brockners-inband-oam-transport]. IOAM data 235 fields are fixed length data fields. A bit field determines the set 236 of OAM data fields embedded in a packet. Depending on the type of 237 the encapsulation, a counter field indicates how many data fields are 238 included in a particular packet. 240 IOAM is expected to be deployed in a specific domain rather than on 241 the overall Internet. The part of the network which employs IOAM is 242 referred to as the "IOAM-domain". IOAM data is added to a packet 243 upon entering the IOAM-domain and is removed from the packet when 244 exiting the domain. Within the IOAM-domain, the IOAM data may be 245 updated by network nodes that the packet traverses. The device which 246 adds an IOAM data container to the packet to capture IOAM data is 247 called the "IOAM encapsulating node", whereas the device which 248 removes the IOAM data container is referred to as the "IOAM 249 decapsulating node". Nodes within the domain which are aware of IOAM 250 data and read and/or write or process the IOAM data are called "IOAM 251 transit nodes". IOAM nodes which add or remove the IOAM data 252 container can also update the IOAM data fields at the same time. Or 253 in other words, IOAM encapsulation or decapsulating nodes can also 254 serve as IOAM transit nodes at the same time. Note that not every 255 node in an IOAM domain needs to be an IOAM transit node. For 256 example, a Segment Routing deployment might require the segment 257 routing path to be verified. In that case, only the SR nodes would 258 also be IOAM transit nodes rather than all nodes. 260 4.1. IOAM Tracing Options 262 "IOAM tracing data" is expected to be collected at every node that a 263 packet traverses to ensure visibility into the entire path a packet 264 takes within an IOAM domain, i.e., in a typical deployment all nodes 265 in an in-situ OAM-domain would participate in IOAM and thus be IOAM 266 transit nodes, IOAM encapsulating or IOAM decapsulating nodes. If 267 not all nodes within a domain are IOAM capable, IOAM tracing 268 information will only be collected on those nodes which are IOAM 269 capable. Nodes which are not IOAM capable will forward the packet 270 without any changes to the IOAM data fields. The maximum number of 271 hops and the minimum path MTU of the IOAM domain is assumed to be 272 known. 274 To optimize hardware and software implementations tracing is defined 275 as two separate options. Any deployment MAY choose to configure and 276 support one or both of the following options. An implementation of 277 the transport protocol that carries these in-situ OAM data MAY choose 278 to support only one of the options. In the event that both options 279 are utilized at the same time, the Incremental Trace Option MUST be 280 placed before the Pre-allocated Trace Option. Given that the 281 operator knows which equipment is deployed in a particular IOAM, the 282 operator will decide by means of configuration which type(s) of trace 283 options will be enabled for a particular domain. 285 Pre-allocated Trace Option: This trace option is defined as a 286 container of node data fields with pre-allocated space for each 287 node to populate its information. This option is useful for 288 software implementations where it is efficient to allocate the 289 space once and index into the array to populate the data during 290 transit. The IOAM encapsulating node allocates the option header 291 and sets the fields in the option header. The in situ OAM 292 encapsulating node allocates an array which is used to store 293 operational data retrieved from every node while the packet 294 traverses the domain. IOAM transit nodes update the content of 295 the array. A pointer which is part of the IOAM trace data points 296 to the next empty slot in the array, which is where the next IOAM 297 transit node fills in its data. 299 Incremental Trace Option: This trace option is defined as a 300 container of node data fields where each node allocates and pushes 301 its node data immediately following the option header. The 302 maximum length of the node data list is written into the option 303 header. This type of trace recording is useful for some of the 304 hardware implementations as this eliminates the need for the 305 transit network elements to read the full array in the option and 306 allows for arbitrarily long packets as the MTU allows. The in- 307 situ OAM encapsulating node allocates the option header. The in- 308 situ OAM encapsulating node based on operational state and 309 configuration sets the fields in the header to control how large 310 the node data list can grow. IOAM transit nodes push their node 311 data to the node data list and increment the number of node data 312 fields in the header. 314 Every node data entry is to hold information for a particular IOAM 315 transit node that is traversed by a packet. The in-situ OAM 316 decapsulating node removes the IOAM data and processes and/or exports 317 the metadata. IOAM data uses its own name-space for information such 318 as node identifier or interface identifier. This allows for a 319 domain-specific definition and interpretation. For example: In one 320 case an interface-id could point to a physical interface (e.g., to 321 understand which physical interface of an aggregated link is used 322 when receiving or transmitting a packet) whereas in another case it 323 could refer to a logical interface (e.g., in case of tunnels). 325 The following IOAM data is defined for IOAM tracing: 327 o Identification of the IOAM node. An IOAM node identifier can 328 match to a device identifier or a particular control point or 329 subsystem within a device. 331 o Identification of the interface that a packet was received on, 332 i.e. ingress interface. 334 o Identification of the interface that a packet was sent out on, 335 i.e. egress interface. 337 o Time of day when the packet was processed by the node. Different 338 definitions of processing time are feasible and expected, though 339 it is important that all devices of an in-situ OAM domain follow 340 the same definition. 342 o Generic data: Format-free information where syntax and semantic of 343 the information is defined by the operator in a specific 344 deployment. For a specific deployment, all IOAM nodes should 345 interpret the generic data the same way. Examples for generic 346 IOAM data include geo-location information (location of the node 347 at the time the packet was processed), buffer queue fill level or 348 cache fill level at the time the packet was processed, or even a 349 battery charge level. 351 o A mechanism to detect whether IOAM trace data was added at every 352 hop or whether certain hops in the domain weren't in-situ OAM 353 transit nodes. 355 The "node data list" array in the packet is populated iteratively as 356 the packet traverses the network, starting with the last entry of the 357 array, i.e., "node data list [n]" is the first entry to be populated, 358 "node data list [n-1]" is the second one, etc. 360 4.1.1. Pre-allocated Trace Option 361 In-situ OAM pre-allocated trace option: 363 Pre-allocated trace option header: 365 0 1 2 3 366 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 367 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 368 | IOAM-Trace-Type |NodeLen| Flags | Octets-left | 369 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 371 Pre-allocated Trace Option Data MUST be 4-octet aligned: 372 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 373 | | | 374 | node data list [0] | | 375 | | | 376 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ D 377 | | a 378 | node data list [1] | t 379 | | a 380 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 381 ~ ... ~ S 382 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ p 383 | | a 384 | node data list [n-1] | c 385 | | e 386 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 387 | | | 388 | node data list [n] | | 389 | | | 390 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 392 IOAM-Trace-Type: A 16-bit identifier which specifies which data 393 types are used in this node data list. 395 The IOAM-Trace-Type value is a bit field. The following bit 396 fields are defined in this document, with details on each field 397 described in the Section 4.1.3. The order of packing the data 398 fields in each node data element follows the bit order of the 399 IOAM-Trace-Type field, as follows: 401 Bit 0 (Most significant bit) When set indicates presence of 402 Hop_Lim and node_id in the node data. 404 Bit 1 When set indicates presence of ingress_if_id and 405 egress_if_id (short format) in the node data. 407 Bit 2 When set indicates presence of timestamp seconds in the 408 node data 410 Bit 3 When set indicates presence of timestamp nanoseconds in 411 the node data. 413 Bit 4 When set indicates presence of transit delay in the node 414 data. 416 Bit 5 When set indicates presence of app_data (short format) in 417 the node data. 419 Bit 6 When set indicates presence of queue depth in the node 420 data. 422 Bit 7 When set indicates presence of variable length Opaque 423 State Snapshot field. 425 Bit 8 When set indicates presence of Hop_Lim and node_id in 426 wide format in the node data. 428 Bit 9 When set indicates presence of ingress_if_id and 429 egress_if_id in wide format in the node data. 431 Bit 10 When set indicates presence of app_data wide in the node 432 data. 434 Bit 11 When set indicates presence of the Checksum Complement 435 node data. 437 Bit 12-15 Undefined in this draft. 439 Section 4.1.3 describes the IOAM data types and their formats. 440 Within an in-situ OAM domain possible combinations of these bits 441 making the IOAM-Trace-Type can be restricted by configuration 442 knobs. 444 Node Data Length: 4-bit unsigned integer. This field specifies the 445 length of data added by each node in multiples of 4-octets. For 446 example, if 3 IOAM-Trace-Type bits are set and none of them is 447 wide, then the Node Data Length would be 3. If 3 IOAM-Trace-Type 448 bits are set and 2 of them are wide, then the Node Data Length 449 would be 5. 451 Flags 5-bit field. Following flags are defined: 453 Bit 0 "Overflow" (O-bit) (most significant bit). This bit is set 454 by the network element if there is not enough number of octets 455 left to record node data, no field is added and the overflow 456 "O-bit" must be set to "1" in the header. This is useful for 457 transit nodes to ignore further processing of the option. 459 Bit 1 "Loopback" (L-bit). Loopback mode is used to send a copy 460 of a packet back towards the source. Loopback mode assumes 461 that a return path from transit nodes and destination nodes 462 towards the source exists. The encapsulating node decides 463 (e.g. using a filter) which packets loopback mode is enabled 464 for by setting the loopback bit. The encapsulating node also 465 needs to ensure that sufficient space is available in the IOAM 466 header for loopback operation. The loopback bit when set 467 indicates to the transit nodes processing this option to create 468 a copy of the packet received and send this copy of the packet 469 back to the source of the packet while it continues to forward 470 the original packet towards the destination. The source 471 address of the original packet is used as destination address 472 in the copied packet. The address of the node performing the 473 copy operation is used as the source address. The L-bit MUST 474 be cleared in the copy of the packet a nodes sends it back 475 towards the source. On its way back towards the source, the 476 packet is processed like a regular packet with IOAM 477 information. Once the return packet reaches the IOAM domain 478 boundary IOAM decapsulation occurs as with any other packet 479 containing IOAM information. 481 Bit 2-4 Reserved: Must be zero. 483 Octets-left: 7-bit unsigned integer. It is the data space in 484 multiples of 4-octets remaining for recording the node data. This 485 is used as an offset in data space to record the node data 486 element. 488 Node data List [n]: Variable-length field. The type of which is 489 determined by the IOAM-Trace-Type representing the n-th node data 490 in the node data list. The node data list is encoded starting 491 from the last node data of the path. The first element of the 492 node data list (node data list [0]) contains the last node of the 493 path while the last node data of the node data list (node data 494 list[n]) contains the first node data of the path traced. The 495 index contained in "Octets-left" identifies the offset for current 496 active node data to be populated. 498 4.1.2. Incremental Trace Option 499 In-situ OAM incremental trace option: 501 In-situ OAM incremental trace option Header: 503 0 1 2 3 504 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 505 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 506 | IOAM-Trace-Type |NodeLen| Flags | Max Length | 507 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 IOAM Incremental Trace Option Data MUST be 4-octet aligned: 511 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 512 | | 513 | node data list [0] | 514 | | 515 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 516 | | 517 | node data list [1] | 518 | | 519 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 520 ~ ... ~ 521 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 522 | | 523 | node data list [n-1] | 524 | | 525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 526 | | 527 | node data list [n] | 528 | | 529 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 531 IOAM-trace-type: A 16-bit identifier which specifies which data 532 types are used in this node data list. 534 The IOAM-Trace-Type value is a bit field. The following bit 535 fields are defined in this document, with details on each field 536 described in the Section 4.1.3. The order of packing the data 537 fields in each node data element follows the bit order of the 538 IOAM-Trace-Type field, as follows: 540 Bit 0 (Most significant bit) When set indicates presence of 541 Hop_Lim and node_id in the node data. 543 Bit 1 When set indicates presence of ingress_if_id and 544 egress_if_id (short format) in the node data. 546 Bit 2 When set indicates presence of timestamp seconds in the 547 node data. 549 Bit 3 When set indicates presence of timestamp nanoseconds in 550 the node data. 552 Bit 4 When set indicates presence of transit delay in the node 553 data. 555 Bit 5 When set indicates presence of app_data in the node data. 557 Bit 6 When set indicates presence of queue depth in the node 558 data. 560 Bit 7 When set indicates presence of variable length Opaque 561 State Snapshot field. 563 Bit 8 When set indicates presence of Hop_Lim and node_id wide 564 in the node data. 566 Bit 9 When set indicates presence of ingress_if_id and 567 egress_if_id in wide format in the node data. 569 Bit 10 When set indicates presence of app_data wide in the node 570 data. 572 Bit 11 When set indicates presence of the Checksum Complement 573 node data. 575 Bit 12-15 Undefined in this draft. 577 Section 4.1.3 describes the IOAM data types and their formats. 579 Node Data Length: 4-bit unsigned integer. This field specifies the 580 length of data added by each node in multiples of 4-octets. For 581 example, if 3 IOAM-Trace-Type bits are set and none of them is 582 wide, then the Node Data Length would be 3. If 3 IOAM-Trace-Type 583 bits are set and 2 of them are wide, then the Node Data Length 584 would be 5. 586 Flags 5-bit field. Following flags are defined: 588 Bit 0 "Overflow" (O-bit) (least significant bit). This bit is 589 set by the network element if there is not enough number of 590 octets left to record node data, no field is added and the 591 overflow "O-bit" must be set to "1" in the header. This is 592 useful for transit nodes to ignore further processing of the 593 option. 595 Bit 1 "Loopback" (L-bit). This bit when set indicates to the 596 transit nodes processing this option to send a copy of the 597 packet back to the source of the packet while it continues to 598 forward the original packet towards the destination. The L-bit 599 MUST be cleared in the copy of the packet before sending it. 601 Bit 2-4 Reserved. Must be zero. 603 Maximum Length: 7-bit unsigned integer. This field specifies the 604 maximum length of the node data list in multiples of 4-octets. 605 Given that the sender knows the minimum path MTU, the sender can 606 set the maximum length according to the number of node data bytes 607 allowed before exceeding the MTU. Thus, a simple comparison 608 between "Opt data Len" and "Max Length" allows to decide whether 609 or not data could be added. 611 Node data List [n]: Variable-length field. The type of which is 612 determined by the OAM Type representing the n-th node data in the 613 node data list. The node data list is encoded starting from the 614 last node data of the path. The first element of the node data 615 list (node data list [0]) contains the last node of the path while 616 the last node data of the node data list (node data list[n]) 617 contains the first node data of the path traced. 619 4.1.3. IOAM node data fields and associated formats 621 All the data fields MUST be 4-octet aligned. The IOAM encapsulating 622 node MUST initialize data fields that it adds to the packet to zero. 623 If a node which is supposed to update an IOAM data field is not 624 capable of populating the value of a field set in the IOAM-Trace- 625 Type, the field value MUST be left unaltered except when explicitly 626 specified in the field description below. In the description of data 627 below if zero is valid value then a non-zero value to mean not 628 populated is specified. 630 Data field and associated data type for each of the data field is 631 shown below: 633 Hop_Lim and node_id: 4-octet field defined as follows: 635 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 | Hop_Lim | node_id | 638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 640 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 641 value in the packet at the node that records this data. Hop 642 Limit information is used to identify the location of the node 643 in the communication path. This is copied from the lower 644 layer, e.g., TTL value in IPv4 header or hop limit field from 645 IPv6 header of the packet when the packet is ready for 646 transmission. The semantics of the Hop_Lim field depend on the 647 lower layer protocol that IOAM is encapsulated over, and 648 therefore its specific semantics are outside the scope of this 649 memo. 651 node_id: 3-octet unsigned integer. Node identifier field to 652 uniquely identify a node within in-situ OAM domain. The 653 procedure to allocate, manage and map the node_ids is beyond 654 the scope of this document. 656 ingress_if_id and egress_if_id: 4-octet field defined as follows: 657 When this field is part of the data field but a node populating 658 the field is not able to fill it, the position in the field must 659 be filled with value 0xFFFFFFFF to mean not populated. 661 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 662 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 663 | ingress_if_id | egress_if_id | 664 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 666 ingress_if_id: 2-octet unsigned integer. Interface identifier to 667 record the ingress interface the packet was received on. 669 egress_if_id: 2-octet unsigned integer. Interface identifier to 670 record the egress interface the packet is forwarded out of. 672 timestamp seconds: 4-octet unsigned integer. Absolute timestamp in 673 seconds that specifies the time at which the packet was received 674 by the node. The structure of this field is identical to the most 675 significant 32 bits of the 64 least significant bits of the 676 [IEEE1588v2] timestamp. This truncated field consists of a 32-bit 677 seconds field. As defined in [IEEE1588v2], the timestamp 678 specifies the number of seconds elapsed since 1 January 1970 679 00:00:00 according to the International Atomic Time (TAI). 681 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 682 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 683 | timestamp seconds | 684 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 686 timestamp nanoseconds: 4-octet unsigned integer in the range 0 to 687 10^9-1. This timestamp specifies the fractional part of the wall 688 clock time at which the packet was received by the node in units 689 of nanoseconds. This field is identical to the 32 least 690 significant bits of the [IEEE1588v2] timestamp. This fields 691 allows for delay computation between any two nodes in the network 692 when the nodes are time synchronized. When this field is part of 693 the data field but a node populating the field is not able to fill 694 it, the field position in the field must be filled with value 695 0xFFFFFFFF to mean not populated. 697 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 698 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 699 | timestamp nanoseconds | 700 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 702 transit delay: 4-octet unsigned integer in the range 0 to 2^30-1. 703 It is the time in nanoseconds the packet spent in the transit 704 node. This can serve as an indication of the queuing delay at the 705 node. If the transit delay exceeds 2^30-1 nanoseconds then the 706 top bit 'O' is set to indicate overflow. When this field is part 707 of the data field but a node populating the field is not able to 708 fill it, the field position in the field must be filled with value 709 0xFFFFFFFF to mean not populated. 711 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 712 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 713 |O| transit delay | 714 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 716 app_data: 4-octet placeholder which can be used by the node to add 717 application specific data. App_data represents a "free-format" 718 4-octet bit field with its semantics defined by a specific 719 deployment. 721 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 722 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 723 | app_data | 724 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 726 queue depth: 4-octet unsigned integer field. This field indicates 727 the current length of the egress interface queue of the interface 728 from where the packet is forwarded out. The queue depth is 729 expressed as the current number of memory buffers used by the 730 queue (a packet may consume one or more memory buffers, depending 731 on its size). When this field is part of the data field but a 732 node populating the field is not able to fill it, the field 733 position in the field must be filled with value 0xFFFFFFFF to mean 734 not populated. 736 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 737 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 738 | queue depth | 739 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 741 Opaque State Snapshot: Variable length field. It allows the network 742 element to store an arbitrary state in the node data field , 743 without a pre-defined schema. The schema needs to be made known 744 to the analyzer by some out-of-band mechanism. The specification 745 of this mechanism is beyond the scope of this document. The 746 24-bit "Schema Id" field in the field indicates which particular 747 schema is used, and should be configured on the network element by 748 the operator. 750 0 1 2 3 751 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 752 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 753 | Length | Schema ID | 754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 755 | | 756 | | 757 | Opaque data | 758 ~ ~ 759 . . 760 . . 761 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 763 Length: 1-octet unsigned integer. It is the length in octets of 764 the Opaque data field that follows Schema Id. It MUST always 765 be a multiple of 4. 767 Schema ID: 3-octet unsigned integer identifying the schema of 768 Opaque data. 770 Opaque data: Variable length field. This field is interpreted as 771 specified by the schema identified by the Schema ID. 773 Hop_Lim and node_id wide: 8-octet field defined as follows: 775 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 776 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 777 | Hop_Lim | node_id ~ 778 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 779 ~ node_id (contd) | 780 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 782 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 783 value in the packet at the node that records this data. Hop 784 Limit information is used to identify the location of the node 785 in the communication path. This is copied from the lower layer 786 for e.g. TTL value in IPv4 header or hop limit field from IPv6 787 header of the packet. The semantics of the Hop_Lim field 788 depend on the lower layer protocol that IOAM is encapsulated 789 over, and therefore its specific semantics are outside the 790 scope of this memo. 792 node_id: 7-octet unsigned integer. Node identifier field to 793 uniquely identify a node within in-situ OAM domain. The 794 procedure to allocate, manage and map the node_ids is beyond 795 the scope of this document. 797 ingress_if_id and egress_if_id wide: 8-octet field defined as 798 follows: When this field is part of the data field but a node 799 populating the field is not able to fill it, the field position in 800 the field must be filled with value 0xFFFFFFFFFFFFFFFF to mean not 801 populated. 803 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 804 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 805 | ingress_if_id | 806 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 807 | egress_if_id | 808 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 810 ingress_if_id: 4-octet unsigned integer. Interface identifier to 811 record the ingress interface the packet was received on. 813 egress_if_id: 4-octet unsigned integer. Interface identifier to 814 record the egress interface the packet is forwarded out of. 816 app_data wide: 8-octet placeholder which can be used by the node to 817 add application specific data. App data represents a "free- 818 format" 8-octed bit field with its semantics defined by a specific 819 deployment. 821 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 822 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 823 | app data ~ 824 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 825 ~ app data (contd) | 826 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 828 Checksum Complement: 4-octet node data which contains a two-octet 829 Checksum Complement field, and a 2-octet reserved field. The 830 Checksum Complement can be used when IOAM is transported over 831 encapsulations that make use of a UDP transport, such as VXLAN-GPE 832 or Geneve. In this case, incorporating the IOAM node data 833 requires the UDP Checksum field to be updated. Rather than to 834 recompute the Chekcsum field, a node can use the Checksum 835 Complement to make a checksum-neutral update in the UDP payload; 836 the Checksum Complement is assigned a value that complements the 837 rest of the node data fields that were added by the current node, 838 causing the existing UDP Checksum field to remain correct. 839 Checksum Complement fields are used in a similar manner in 840 [RFC7820] and [RFC7821]. 842 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 843 8 9 0 1 2 3 4 5 6 7 8 9 0 1 844 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 845 | Checksum Complement | Reserved | 846 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 848 4.1.4. Examples of IOAM node data 850 An entry in the "node data list" array can have different formats, 851 following the needs of the deployment. Some deployments might only 852 be interested in recording the node identifiers, whereas others might 853 be interested in recording node identifier and timestamp. The 854 section defines different types that an entry in "node data list" can 855 take. 857 0x002B: IOAM-Trace-Type is 0x2B then the format of node data is: 859 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 860 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 861 | Hop_Lim | node_id | 862 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 863 | ingress_if_id | egress_if_id | 864 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 865 | timestamp nanoseconds | 866 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 867 | app_data | 868 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 870 0x0003: IOAM-Trace-Type is 0x0003 then the format is: 872 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 873 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 874 | Hop_Lim | node_id | 875 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 876 | ingress_if_id | egress_if_id | 877 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 879 0x0009: IOAM-Trace-Type is 0x0009 then the format is: 881 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 882 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 883 | Hop_Lim | node_id | 884 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 885 | timestamp nanoseconds | 886 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 888 0x0021: IOAM-Trace-Type is 0x0021 then the format is: 890 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 891 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 892 | Hop_Lim | node_id | 893 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 894 | app_data | 895 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 897 0x0029: IOAM-Trace-Type is 0x0029 then the format is: 899 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 901 | Hop_Lim | node_id | 902 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 903 | timestamp nanoseconds | 904 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 905 | app_data | 906 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 908 0x018C: IOAM-Trace-Type is 0x104D then the format is: 910 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 911 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 912 | timestamp seconds | 913 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 914 | timestamp nanoseconds | 915 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 916 | Length | Schema Id | 917 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 918 | | 919 | | 920 | Opaque data | 921 ~ ~ 922 . . 923 . . 924 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 925 | Hop_Lim | node_id | 926 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 927 | node_id(contd) | 928 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 930 4.2. IOAM Proof of Transit Option 932 IOAM Proof of Transit data is to support the path or service function 933 chain [RFC7665] verification use cases. Proof-of-transit uses 934 methods like nested hashing or nested encryption of the IOAM data or 935 mechanisms such as Shamir's Secret Sharing Schema (SSSS). While 936 details on how the IOAM data for the proof of transit option is 937 processed at IOAM encapsulating, decapsulating and transit nodes are 938 outside the scope of the document, all of these approaches share the 939 need to uniquely identify a packet as well as iteratively operate on 940 a set of information that is handed from node to node. 941 Correspondingly, two pieces of information are added as IOAM data to 942 the packet: 944 o Random: Unique identifier for the packet (e.g., 64-bits allow for 945 the unique identification of 2^64 packets). 947 o Cumulative: Information which is handed from node to node and 948 updated by every node according to a verification algorithm. 950 IOAM proof of transit option: 952 IOAM proof of transit option header: 954 0 1 2 3 4 5 6 7 955 +-+-+-+-+-+-+-+-+ 956 |IOAM POT Type|P| 957 +-+-+-+-+-+-+-+-+ 959 IOAM proof of transit option data MUST be 4-octet aligned: 961 0 1 2 3 962 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 963 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 964 | Random | | 965 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ P 966 | Random(contd) | O 967 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ T 968 | Cumulative | | 969 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 970 | Cumulative (contd) | | 971 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 973 IOAM POT Type: 7-bit identifier of a particular POT variant that 974 dictates the POT data that is included. This document defines POT 975 Type 0: 977 0: POT data is a 16 Octet field as described below. 979 Profile to use (P): 1-bit. Indicates which POT-profile is used to 980 generate the Cumulative. Any node participating in POT will have 981 a maximum of 2 profiles configured that drive the computation of 982 cumulative. The two profiles are numbered 0, 1. This bit conveys 983 whether profile 0 or profile 1 is used to compute the Cumulative. 985 Random: 64-bit Per packet Random number. 987 Cumulative: 64-bit Cumulative that is updated at specific nodes by 988 processing per packet Random number field and configured 989 parameters. 991 Note: Larger or smaller sizes of "Random" and "Cumulative" data are 992 feasible and could be required for certain deployments (e.g. in case 993 of space constraints in the transport protocol used). Future 994 versions of this document will address different sizes of data for 995 "proof of transit". 997 4.3. IOAM Edge-to-Edge Option 999 The IOAM edge-to-edge option is to carry data that is added by the 1000 IOAM encapsulating node and interpreted by IOAM decapsulating node. 1001 The IOAM transit nodes MAY process the data without modifying it. 1003 Currently only sequence numbers use the IOAM edge-to-edge option. In 1004 order to detect packet loss, packet reordering, or packet duplication 1005 in an in-situ OAM-domain, sequence numbers can be added to packets of 1006 a particular tube (see [I-D.hildebrand-spud-prototype]). Each tube 1007 leverages a dedicated namespace for its sequence numbers. 1009 IOAM edge-to-edge option: 1011 IOAM edge-to-edge option header: 1013 0 1 2 3 4 5 6 7 1014 +-+-+-+-+-+-+-+-+ 1015 | IOAM-E2E-Type | 1016 +-+-+-+-+-+-+-+-+ 1018 IOAM edge-to-edge option data MUST be 4-octet aligned: 1020 0 1 2 3 1021 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1022 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1023 | E2E Option data field determined by IOAM-E2E-Type | 1024 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1026 IOAM-E2E-Type: 8-bit identifier of a particular in situ OAM E2E 1027 variant. 1029 0: E2E option data is a 64-bit sequence number added to a 1030 specific tube which is used to identify packet loss and 1031 reordering for that tube. 1033 5. IOAM Data Export 1035 IOAM nodes collect information for packets traversing a domain that 1036 supports IOAM. IOAM decapsulating nodes as well as IOAM transit 1037 nodes can choose to retrieve IOAM information from the packet, 1038 process the information further and export the information using 1039 e.g., IPFIX. 1041 The discussion of IOAM data processing and export is left for a 1042 future version of this document. 1044 6. IANA Considerations 1046 This document requests the following IANA Actions. 1048 6.1. Creation of a New In-Situ OAM (IOAM) Protocol Parameters IANA 1049 registry 1051 IANA is requested to create a new protocol registry for "In-Situ OAM 1052 (IOAM) Protocol Parameters". This is the common registry that will 1053 include registrations for all IOAM namespaces. Each Registry, whose 1054 names are listed below: 1056 IOAM Trace Type 1058 IOAM Trace flags 1060 IOAM POT Type 1062 IOAM E2E Type 1064 will contain the current set of possibilities defined in this 1065 document. New registries in this name space are created via RFC 1066 Required process as per [RFC8126]. 1068 The subsequent sub-sections detail the registries herein contained. 1070 6.2. IOAM Trace Type Registry 1072 This registry defines code point for each bit in the 16-bit IOAM- 1073 Trace-Type field for Pre-allocated trace option and Incremental trace 1074 option defined in Section 4.1. The meaning of Bit 0 - 11 for trace 1075 type are defined in this document in Paragraph 1 of (Section 4.1.1). 1076 The meaning for Bit 12 - 15 are available for assignment via RFC 1077 Required process as per [RFC8126]. 1079 6.3. IOAM Trace Flags Registry 1081 This registry defines code point for each bit in the 5 bit flags for 1082 Pre-allocated trace option and Incremental trace option defined in 1083 Section 4.1. The meaning of Bit 0 - 1 for trace flags are defined in 1084 this document in Paragraph 5 of Section 4.1.1. The meaning for Bit 2 1085 - 4 are available for assignment via RFC Required process as per 1086 [RFC8126]. 1088 6.4. IOAM POT Type Registry 1090 This registry defines 128 code points to define IOAM POT Type for 1091 IOAM proof of transit option Section 4.2. The code point value 0 is 1092 defined in this document, 1 - 127 are available for assignment via 1093 RFC Required process as per [RFC8126]. 1095 6.5. IOAM E2E Type Registry 1097 This registry defines 256 code points to define IOAM-E2E-Type for 1098 IOAM E2E option Section 4.3. The code point value 0 is defined in 1099 this document, 1 - 255 are available for assignments via RFC Required 1100 process as per [RFC8126]. 1102 7. Manageability Considerations 1104 Manageability considerations will be addressed in a later version of 1105 this document.. 1107 8. Security Considerations 1109 Security considerations will be addressed in a later version of this 1110 document. For a discussion of security requirements of in-situ OAM, 1111 please refer to [I-D.brockners-inband-oam-requirements]. 1113 9. Acknowledgements 1115 The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari 1116 Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya 1117 Nadahalli, LJ Wobker, Erik Nordmark, Vengada Prasad Govindan, and 1118 Andrew Yourtchenko for the comments and advice. 1120 This document leverages and builds on top of several concepts 1121 described in [I-D.kitamura-ipv6-record-route]. The authors would 1122 like to acknowledge the work done by the author Hiroshi Kitamura and 1123 people involved in writing it. 1125 The authors would like to gracefully acknowledge useful review and 1126 insightful comments received from Joe Clarke, Al Morton, and Mickey 1127 Spiegel. 1129 10. References 1131 10.1. Normative References 1133 [IEEE1588v2] 1134 Institute of Electrical and Electronics Engineers, 1135 "1588-2008 - IEEE Standard for a Precision Clock 1136 Synchronization Protocol for Networked Measurement and 1137 Control Systems", IEEE Std 1588-2008, 2008, 1138 . 1141 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1142 Requirement Levels", BCP 14, RFC 2119, 1143 DOI 10.17487/RFC2119, March 1997, . 1146 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 1147 Writing an IANA Considerations Section in RFCs", BCP 26, 1148 RFC 8126, DOI 10.17487/RFC8126, June 2017, 1149 . 1151 10.2. Informative References 1153 [I-D.brockners-inband-oam-requirements] 1154 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 1155 Gredler, H., Leddy, J., Youell, S., Mozes, D., Mizrahi, 1156 T., <>, P., and r. remy@barefootnetworks.com, 1157 "Requirements for In-situ OAM", draft-brockners-inband- 1158 oam-requirements-03 (work in progress), March 2017. 1160 [I-D.brockners-inband-oam-transport] 1161 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 1162 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, 1163 D., Lapukhov, P., and R. Chang, "Encapsulations for In- 1164 situ OAM Data", draft-brockners-inband-oam-transport-05 1165 (work in progress), July 2017. 1167 [I-D.hildebrand-spud-prototype] 1168 Hildebrand, J. and B. Trammell, "Substrate Protocol for 1169 User Datagrams (SPUD) Prototype", draft-hildebrand-spud- 1170 prototype-03 (work in progress), March 2015. 1172 [I-D.ietf-nvo3-geneve] 1173 Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic 1174 Network Virtualization Encapsulation", draft-ietf- 1175 nvo3-geneve-04 (work in progress), March 2017. 1177 [I-D.ietf-nvo3-vxlan-gpe] 1178 Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol 1179 Extension for VXLAN", draft-ietf-nvo3-vxlan-gpe-04 (work 1180 in progress), April 2017. 1182 [I-D.ietf-sfc-nsh] 1183 Quinn, P., Elzur, U., and C. Pignataro, "Network Service 1184 Header (NSH)", draft-ietf-sfc-nsh-19 (work in progress), 1185 August 2017. 1187 [I-D.kitamura-ipv6-record-route] 1188 Kitamura, H., "Record Route for IPv6 (PR6) Hop-by-Hop 1189 Option Extension", draft-kitamura-ipv6-record-route-00 1190 (work in progress), November 2000. 1192 [I-D.lapukhov-dataplane-probe] 1193 Lapukhov, P. and r. remy@barefootnetworks.com, "Data-plane 1194 probe for in-band telemetry collection", draft-lapukhov- 1195 dataplane-probe-01 (work in progress), June 2016. 1197 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 1198 Chaining (SFC) Architecture", RFC 7665, 1199 DOI 10.17487/RFC7665, October 2015, . 1202 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1203 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1204 May 2016, . 1206 [RFC7820] Mizrahi, T., "UDP Checksum Complement in the One-Way 1207 Active Measurement Protocol (OWAMP) and Two-Way Active 1208 Measurement Protocol (TWAMP)", RFC 7820, 1209 DOI 10.17487/RFC7820, March 2016, . 1212 [RFC7821] Mizrahi, T., "UDP Checksum Complement in the Network Time 1213 Protocol (NTP)", RFC 7821, DOI 10.17487/RFC7821, March 1214 2016, . 1216 Authors' Addresses 1218 Frank Brockners 1219 Cisco Systems, Inc. 1220 Hansaallee 249, 3rd Floor 1221 DUESSELDORF, NORDRHEIN-WESTFALEN 40549 1222 Germany 1224 Email: fbrockne@cisco.com 1225 Shwetha Bhandari 1226 Cisco Systems, Inc. 1227 Cessna Business Park, Sarjapura Marathalli Outer Ring Road 1228 Bangalore, KARNATAKA 560 087 1229 India 1231 Email: shwethab@cisco.com 1233 Carlos Pignataro 1234 Cisco Systems, Inc. 1235 7200-11 Kit Creek Road 1236 Research Triangle Park, NC 27709 1237 United States 1239 Email: cpignata@cisco.com 1241 Hannes Gredler 1242 RtBrick Inc. 1244 Email: hannes@rtbrick.com 1246 John Leddy 1247 Comcast 1249 Email: John_Leddy@cable.comcast.com 1251 Stephen Youell 1252 JP Morgan Chase 1253 25 Bank Street 1254 London E14 5JP 1255 United Kingdom 1257 Email: stephen.youell@jpmorgan.com 1259 Tal Mizrahi 1260 Marvell 1261 6 Hamada St. 1262 Yokneam 2066721 1263 Israel 1265 Email: talmi@marvell.com 1266 David Mozes 1267 Mellanox Technologies Ltd. 1269 Email: davidm@mellanox.com 1271 Petr Lapukhov 1272 Facebook 1273 1 Hacker Way 1274 Menlo Park, CA 94025 1275 US 1277 Email: petr@fb.com 1279 Remy Chang 1280 Barefoot Networks 1281 2185 Park Boulevard 1282 Palo Alto, CA 94306 1283 US 1285 Daniel 1286 Bell Canada 1288 Email: daniel.bernier@bell.ca