idnits 2.17.1 draft-brockners-inband-oam-data-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 29, 2017) is 2524 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '0' on line 572 -- Looks like a reference, but probably isn't: '1' on line 486 == Outdated reference: A later version (-05) exists of draft-brockners-inband-oam-transport-03 == Outdated reference: A later version (-16) exists of draft-ietf-nvo3-geneve-04 == Outdated reference: A later version (-13) exists of draft-ietf-nvo3-vxlan-gpe-04 == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-12 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ippm F. Brockners 3 Internet-Draft S. Bhandari 4 Intended status: Experimental C. Pignataro 5 Expires: November 30, 2017 Cisco 6 H. Gredler 7 RtBrick Inc. 8 J. Leddy 9 Comcast 10 S. Youell 11 JPMC 12 T. Mizrahi 13 Marvell 14 D. Mozes 15 Mellanox Technologies Ltd. 16 P. Lapukhov 17 Facebook 18 R. Chang 19 Barefoot Networks 20 D. Bernier 21 Bell Canada 22 May 29, 2017 24 Data Fields for In-situ OAM 25 draft-brockners-inband-oam-data-05 27 Abstract 29 In-situ Operations, Administration, and Maintenance (IOAM) records 30 operational and telemetry information in the packet while the packet 31 traverses a path between two points in the network. This document 32 discusses the data fields and associated data types for in-situ OAM. 33 In-situ OAM data fields can be embedded into a variety of transports 34 such as NSH, Segment Routing, Geneve, native IPv6 (via extension 35 header), or IPv4. In-situ OAM can be used to complement OAM 36 mechanisms based on e.g. ICMP or other types of probe packets. 38 Status of This Memo 40 This Internet-Draft is submitted in full conformance with the 41 provisions of BCP 78 and BCP 79. 43 Internet-Drafts are working documents of the Internet Engineering 44 Task Force (IETF). Note that other groups may also distribute 45 working documents as Internet-Drafts. The list of current Internet- 46 Drafts is at http://datatracker.ietf.org/drafts/current/. 48 Internet-Drafts are draft documents valid for a maximum of six months 49 and may be updated, replaced, or obsoleted by other documents at any 50 time. It is inappropriate to use Internet-Drafts as reference 51 material or to cite them other than as "work in progress." 53 This Internet-Draft will expire on November 30, 2017. 55 Copyright Notice 57 Copyright (c) 2017 IETF Trust and the persons identified as the 58 document authors. All rights reserved. 60 This document is subject to BCP 78 and the IETF Trust's Legal 61 Provisions Relating to IETF Documents 62 (http://trustee.ietf.org/license-info) in effect on the date of 63 publication of this document. Please review these documents 64 carefully, as they describe your rights and restrictions with respect 65 to this document. Code Components extracted from this document must 66 include Simplified BSD License text as described in Section 4.e of 67 the Trust Legal Provisions and are provided without warranty as 68 described in the Simplified BSD License. 70 Table of Contents 72 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 73 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 74 3. Scope, Applicability, and Assumptions . . . . . . . . . . . . 4 75 4. In-situ OAM Data Types and Formats . . . . . . . . . . . . . 5 76 4.1. In-situ OAM Tracing Options . . . . . . . . . . . . . . . 6 77 4.1.1. Pre-allocated Trace Option . . . . . . . . . . . . . 8 78 4.1.2. Incremental Trace Option . . . . . . . . . . . . . . 11 79 4.1.3. In-situ OAM node data fields and associated formats . 13 80 4.1.4. Examples of In-situ OAM node data . . . . . . . . . . 17 81 4.2. In-situ OAM Proof of Transit Option . . . . . . . . . . . 19 82 4.3. In-situ OAM Edge-to-Edge Option . . . . . . . . . . . . . 21 83 5. In-situ OAM Data Export . . . . . . . . . . . . . . . . . . . 21 84 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 85 7. Manageability Considerations . . . . . . . . . . . . . . . . 22 86 8. Security Considerations . . . . . . . . . . . . . . . . . . . 22 87 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 88 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 89 10.1. Normative References . . . . . . . . . . . . . . . . . . 22 90 10.2. Informative References . . . . . . . . . . . . . . . . . 23 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 93 1. Introduction 95 This document defines data fields for "in-situ" Operations, 96 Administration, and Maintenance (OAM). In-situ OAM records OAM 97 information within the packet while the packet traverses a particular 98 network domain. The term "in-situ" refers to the fact that the OAM 99 data is added to the data packets rather than is being sent within 100 packets specifically dedicated to OAM. A discussion of the 101 motivation and requirements for in-situ OAM can be found in 102 [I-D.brockners-inband-oam-requirements]. In-situ OAM is to 103 complement mechanisms such as Ping or Traceroute, or more recent 104 active probing mechanisms as described in 105 [I-D.lapukhov-dataplane-probe]. In terms of "active" or "passive" 106 OAM, "in-situ" OAM can be considered a hybrid OAM type. While no 107 extra packets are sent, in-situ OAM adds information to the packets 108 therefore cannot be considered passive. In terms of the 109 classification given in [RFC7799] in-situ OAM could be portrayed as 110 "hybrid OAM, type 1". "In-situ" mechanisms do not require extra 111 packets to be sent and hence don't change the packet traffic mix 112 within the network. In-situ OAM mechanisms can be leveraged where 113 mechanisms using e.g. ICMP do not apply or do not offer the desired 114 results, such as proving that a certain traffic flow takes a pre- 115 defined path, SLA verification for the live data traffic, detailed 116 statistics on traffic distribution paths in networks that distribute 117 traffic across multiple paths, or scenarios in which probe traffic is 118 potentially handled differently from regular data traffic by the 119 network devices. 121 2. Conventions 123 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 124 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 125 document are to be interpreted as described in [RFC2119]. 127 Abbreviations used in this document: 129 Geneve: Generic Network Virtualization Encapsulation 130 [I-D.ietf-nvo3-geneve] 132 IOAM: In-situ Operations, Administration, and Maintenance 134 MTU: Maximum Transmit Unit 136 NSH: Network Service Header [I-D.ietf-sfc-nsh] 138 OAM: Operations, Administration, and Maintenance 140 SFC: Service Function Chain 141 SID: Segment Identifier 143 SR: Segment Routing 145 VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol 146 Extension [I-D.ietf-nvo3-vxlan-gpe] 148 3. Scope, Applicability, and Assumptions 150 In-situ OAM deployment assumes a set of constraints, requirements, 151 and guiding principles which are described in this section. 153 Scope: This document defines the data fields and associated data 154 types for in-situ OAM. The in-situ OAM data field can be transported 155 by a variety of transport protocols, including NSH, Segment Routing, 156 Geneve, IPv6, or IPv4. Encapsulation details for these different 157 transport protocols are outside the scope of this document. 159 Deployment domain (or scope) of in-situ OAM deployment: IOAM is a 160 network domain focused feature, with "network domain" being a set of 161 network devices or entities within a single administration. For 162 example, a network domain can include an enterprise campus using 163 physical connections between devices or an overlay network using 164 virtual connections / tunnels for connectivity between said devices. 165 A network domain is defined by its perimeter or edge. Designers of 166 carrier protocols for IOAM must specify mechanisms to ensure that in- 167 situ OAM data stays within an IOAM domain. In addition, the operator 168 of such a domain is expected to put provisions in place to ensure 169 that IOAM data does not leak beyond the edge of an IOAM domain, e.g. 170 using for example packet filtering methods. The operator should 171 consider potential operational impact of IOAM to mechanisms such as 172 ECMP processing (e.g. load-balancing schemes based on packet length 173 could be impacted by the increased packet size due to IOAM), path MTU 174 (i.e. ensure that the MTU of all links within a domain is 175 sufficiently large to support the increased packet size due to IOAM) 176 and ICMP message handling (i.e. in case of a native IPv6 transport, 177 IOAM support for ICMPv6 Echo Request/Reply could desired which would 178 translate into ICMPv6 extensions to enable IOAM data fields to be 179 copied from an Echo Request message to an Echo Reply message). 181 In-situ OAM control points: IOAM data fields are added to or removed 182 from the live user traffic by the devices which form the edge of a 183 domain. Devices within an IOAM domain can update and/or add IOAM 184 data-fields. Domain edge devices can be hosts or network devices. 186 Traffic-sets that in-situ OAM is applied to: IOAM can be deployed on 187 all or only on subsets of the live user traffic. It SHOULD be 188 possible to enable in-situ OAM on a selected set of traffic (e.g., 189 per interface, based on an access control list or flow specification 190 defining a specific set of traffic, etc.) The selected set of 191 traffic can also be all traffic. 193 Encapsulation independence: Data formats for in-situ OAM SHOULD be 194 defined in a transport-independent manner. In-situ OAM applies to a 195 variety of encapsulating protocols. A definition of how IOAM data 196 fields are carried by different transport protocols is outside the 197 scope of this document. 199 Layering: If several encapsulation protocols (e.g., in case of 200 tunneling) are stacked on top of each other, in-situ OAM data-records 201 could be present at every layer. The behavior follows the ships-in- 202 the-night model. 204 Combination with active OAM mechanisms: In-situ OAM should be usable 205 for active network probing, enabling for example a customized version 206 of traceroute. Decapsulating in-situ OAM nodes may have an ability 207 to send the in-situ OAM information retrieved from the packet back to 208 the source address of the packet or to the encapsulating node. 210 In-situ OAM implementation: The IOAM data-field definitions take the 211 specifics of devices with hardware data-plane and software data-plane 212 into account. 214 4. In-situ OAM Data Types and Formats 216 This section defines in-situ OAM data types and data fields and 217 associated data types required for in-situ OAM. The different uses 218 of in-situ OAM require the definition of different types of data. 219 The in-situ OAM data fields for the data being carried corresponds to 220 the three main categories of in-situ OAM data defined in 221 [I-D.brockners-inband-oam-requirements], which are: edge-to-edge, per 222 node, and for selected nodes only. 224 Transport options for in-situ OAM data are outside the scope of this 225 memo, and are discussed in [I-D.brockners-inband-oam-transport]. In- 226 situ OAM data fields are fixed length data fields. A bit field 227 determines the set of OAM data fields embedded in a packet. 228 Depending on the type of the encapsulation, a counter field indicates 229 how many data fields are included in a particular packet. 231 In-situ OAM is expected to be deployed in a specific domain rather 232 than on the overall Internet. The part of the network which employs 233 in- situ OAM is referred to as the "in-situ OAM-domain". In-situ OAM 234 data is added to a packet upon entering the in-situ OAM-domain and is 235 removed from the packet when exiting the domain. Within the in-situ 236 OAM-domain, the in-situ OAM data may be updated by network nodes that 237 the packet traverses. The device which adds an in-situ OAM data 238 container to the packet to capture in-situ OAM data is called the 239 "in-situ OAM encapsulating node", whereas the device which removes 240 the in-situ OAM data container is referred to as the "in-situ OAM 241 decapsulating node". Nodes within the domain which are aware of in- 242 situ OAM data and read and/or write or process the in-situ OAM data 243 are called "in-situ OAM transit nodes". Note that not every node in 244 an in-situ OAM domain needs to be an in-situ OAM transit node. For 245 example, a Segment Routing deployment might require the segment 246 routing path to be verified. In that case, only the SR nodes would 247 also be in-situ OAM transit nodes rather than all nodes. 249 4.1. In-situ OAM Tracing Options 251 "In-situ OAM tracing data" is expected to be collected at every node 252 that a packet traverses, i.e., in a typical deployment all nodes in 253 an in-situ OAM-domain would participate in in-situ OAM and thus be 254 in-situ OAM transit nodes, in-situ OAM encapsulating or in-situ OAM 255 decapsulating nodes. The maximum number of hops and the minimum path 256 MTU of the in-situ OAM domain is assumed to be known. 258 To optimize hardware and software implementations tracing is defined 259 as two separate options. Any deployment MAY choose to configure and 260 support one or both of the following options. An implementation of 261 the transport protocol that carries these in-situ OAM data MAY choose 262 to support only one of the options. In the event that both options 263 are utilized at the same time, the Incremental Trace Option MUST be 264 placed before the Pre-allocated Trace Option. 266 Pre-allocated Trace Option: This trace option is defined as a 267 container of node data fields with pre-allocated space for each 268 node to populate its information. This option is useful for 269 software implementations where it is efficient to allocate the 270 space once and index into the array to populate the data during 271 transit. The in-situ OAM encapsulating node allocates the option 272 header and sets the fields in the option header. The in situ OAM 273 encapsulating node allocates an array which is used to store 274 operational data retrieved from every node while the packet 275 traverses the domain. In-situ OAM transit nodes update the 276 content of the array. A pointer which is part of the in-situ OAM 277 trace data points to the next empty slot in the array, which is 278 where the next in-situ OAM transit node fills in its data. 280 Incremental Trace Option: This trace option is defined as a 281 container of node data fields where each node allocates and pushes 282 its node data immediately following the option header. The number 283 of node data fields recorded and maximum number of node data that 284 can be recorded are written into the option header. This type of 285 trace recording is useful for some of the hardware implementations 286 as this eliminates the need for the transit network elements to 287 read the full array in the option and allows for arbitrarily long 288 packets as the MTU allows. The in-situ OAM encapsulating node 289 allocates the option header. The in-situ OAM encapsulating node 290 based on operational state and configuration sets the fields in 291 the header to control how large the node data list can grow. In- 292 situ OAM transit nodes push their node data to the node data list 293 and increment the number of node data fields in the header. 295 Every node data entry is to hold information for a particular in-situ 296 OAM transit node that is traversed by a packet. The in-situ OAM 297 decapsulating node removes the in-situ OAM data and processes and/or 298 exports the metadata. In-situ OAM data uses its own name-space for 299 information such as node identifier or interface identifier. This 300 allows for a domain-specific definition and interpretation. For 301 example: In one case an interface-id could point to a physical 302 interface (e.g., to understand which physical interface of an 303 aggregated link is used when receiving or transmitting a packet) 304 whereas in another case it could refer to a logical interface (e.g., 305 in case of tunnels). 307 The following in-situ OAM data is defined for in-situ OAM tracing: 309 o Identification of the in-situ OAM node. An in-situ OAM node 310 identifier can match to a device identifier or a particular 311 control point or subsystem within a device. 313 o Identification of the interface that a packet was received on. 315 o Identification of the interface that a packet was sent out on. 317 o Time of day when the packet was processed by the node. Different 318 definitions of processing time are feasible and expected, though 319 it is important that all devices of an in-situ OAM domain follow 320 the same definition. 322 o Generic data: Format-free information where syntax and semantic of 323 the information is defined by the operator in a specific 324 deployment. For a specific deployment, all in-situ OAM nodes 325 should interpret the generic data the same way. Examples for 326 generic in-situ OAM data include geo-location information 327 (location of the node at the time the packet was processed), 328 buffer queue fill level or cache fill level at the time the packet 329 was processed, or even a battery charge level. 331 o A mechanism to detect whether in-situ OAM trace data was added at 332 every hop or whether certain hops in the domain weren't in-situ 333 OAM transit nodes. 335 The "node data list" array in the packet is populated iteratively as 336 the packet traverses the network, starting with the last entry of the 337 array, i.e., "node data list [n]" is the first entry to be populated, 338 "node data list [n-1]" is the second one, etc. 340 4.1.1. Pre-allocated Trace Option 342 In-situ OAM Pre-allocated Trace Option: 344 Pre-allocated Trace Option header: 346 0 1 2 3 347 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 349 | IOAM-Trace-Type | Octets-left | Flags | 350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 352 Pre-allocated Trace Option Data MUST be 4-byte aligned: 353 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 354 | | | 355 | node data list [0] | | 356 | | | 357 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ D 358 | | a 359 | node data list [1] | t 360 | | a 361 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 362 ~ ... ~ S 363 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ p 364 | | a 365 | node data list [n-1] | c 366 | | e 367 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 368 | | | 369 | node data list [n] | | 370 | | | 371 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 373 IOAM-Trace-Type: A 16-bit identifier which specifies which data 374 types are used in this node data list. 376 The IOAM-Trace-Type value is a bit field. The following bit 377 fields are defined in this document, with details on each field 378 described in the Section 4.1.3. The order of packing the data 379 fields in each node data element follows the bit order of the 380 IOAM-Trace-Type field, as follows: 382 Bit 0 (Least significant bit) When set indicates presence of 383 Hop_Lim and node_id in the node data. 385 Bit 1 When set indicates presence of ingress_if_id and 386 egress_if_id (short format) in the node data. 388 Bit 2 When set indicates presence of timestamp seconds in the 389 node data 391 Bit 3 When set indicates presence of timestamp nanoseconds in 392 the node data. 394 Bit 4 When set indicates presence of transit delay in the node 395 data. 397 Bit 5 When set indicates presence of app_data (short format) in 398 the node data. 400 Bit 6 When set indicates presence of queue depth in the node 401 data. 403 Bit 7 When set indicates presence of variable length Opaque 404 State Snapshot field. 406 Bit 8 When set indicates presence of Hop_Lim and node_id in 407 wide format in the node data. 409 Bit 9 When set indicates presence of ingress_if_id and 410 egress_if_id in wide format in the node data. 412 Bit 10 When set indicates presence of app_data wide in the node 413 data. 415 Bit 11-15 Undefined in this draft. 417 Section 4.1.4 describes the in-situ OAM data types and their 418 formats. Within an in-situ OAM domain possible combinations of 419 these bits making the IOAM-Trace-Type can be restricted by 420 configuration knobs. 422 Octets-left: 8-bit unsigned integer. It is the data space in octets 423 remaining for recording the node data. This is used as an offset 424 in octets in data space to record node data element. 426 Flags 8-bit field. The following flags are defined: 428 Bit 0 "Overflow" (O-bit) (most significant bit). This bit is set 429 by the network element if there is not enough number of bytes 430 left to record node data, no field is added and the overflow 431 "O-bit" must be set to "1" in the header. This is useful for 432 transit nodes to ignore further processing of the option. 434 Bit 1 "Loopback" (L-bit). Loopback mode is used to send a copy 435 of a packet back towards the source. Loopback mode assumes 436 that a return path from transit nodes and destination nodes 437 towards the source exists. The encapsulating node decides 438 (e.g. using a filter) which packets loopback mode is enabled 439 for by setting the loopback bit. The encapsulating node also 440 needs to ensure that sufficient space is available in the IOAM 441 header for loopback operation. The loopback bit when set 442 indicates to the transit nodes processing this option to create 443 a copy of the packet received and send this copy of the packet 444 back to the source of the packet while it continues to forward 445 the original packet towards the destination. The source 446 address of the original packet is used as destination address 447 in the copied packet. The address of the node performing the 448 copy operation is used as the source address. The L-bit MUST 449 be cleared in the copy of the packet a nodes sends it back 450 towards the source. On its way back towards the source, the 451 packet is processed like a regular packet with IOAM 452 information. Once the return packet reaches the IOAM domain 453 boundary IOAM decapsulation occurs as with any other packet 454 containing IOAM information. 456 Node data List [n]: Variable-length field. The type of which is 457 determined by the IOAM-Trace-Type representing the n-th node data 458 in the node data list. The node data list is encoded starting 459 from the last node data of the path. The first element of the 460 node data list (node data list [0]) contains the last node of the 461 path while the last node data of the node data list (node data 462 list[n]) contains the first node data of the path traced. The 463 index contained in "Octets-left" identifies the offset for current 464 active node data to be populated. 466 4.1.2. Incremental Trace Option 468 In-situ OAM Incremental Trace Option: ' 470 In-situ OAM Incremental Trace Option Header: 472 0 1 2 3 473 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 474 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 475 | IOAM-Trace-Type | Maximum Length| Flags | 476 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 In-situ OAM Incremental Trace Option Data MUST be 4-byte aligned: 480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 481 | | 482 | node data list [0] | 483 | | 484 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 485 | | 486 | node data list [1] | 487 | | 488 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 489 ~ ... ~ 490 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 491 | | 492 | node data list [n-1] | 493 | | 494 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 495 | | 496 | node data list [n] | 497 | | 498 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 500 IOAM-trace-type: A 16-bit identifier which specifies which data 501 types are used in this node data list. 503 The IOAM-Trace-Type value is a bit field. The following bit 504 fields are defined in this document, with details on each field 505 described in the Section 4.1.3. The order of packing the data 506 fields in each node data element follows the bit order of the 507 IOAM-Trace-Type field, as follows: 509 Bit 0 When set indicates presence of Hop_Lim and node_id in the 510 node data. 512 Bit 1 When set indicates presence of ingress_if_id and 513 egress_if_id in the node data. 515 Bit 2 When set indicates presence of timestamp seconds in the 516 node data. 518 Bit 3 When set indicates presence of timestamp nanoseconds in 519 the node data. 521 Bit 4 When set indicates presence of transit delay in the node 522 data. 524 Bit 5 When set indicates presence of app_data in the node data. 526 Bit 6 When set indicates presence of queue depth in the node 527 data. 529 Bit 7 When set indicates presence of variable length Opaque 530 State Snapshot field. 532 Bit 8 When set indicates presence of Hop_Lim and node_id wide 533 in the node data. 535 Bit 9 When set indicates presence of ingress_if_id and 536 egress_if_id wide in the node data. 538 Bit 10 When set indicates presence of app_data wide in the node 539 data. 541 Bit 11-15 Undefined in this draft. 543 Section 4.1.4 describes the in-situ OAM data types and their 544 formats. 546 Maximum Length: 8-bit unsigned integer. This field specifies the 547 maximum length of the node data list in octets. Given that the 548 sender knows the minimum path MTU, the sender can set the maximum 549 of node data bytes allowed before exceeding the MTU. Thus, a 550 simple comparison between "Opt data Len" and "Max Length" allows 551 to decide whether or not data could be added. 553 Flags 8-bit field. Following flags are defined: 555 Bit 0 "Overflow" (O-bit) (least significant bit). This bit is 556 set by the network element if there is not enough number of 557 bytes left to record node data, no field is added and the 558 overflow "O-bit" must be set to "1" in the header. This is 559 useful for transit nodes to ignore further processing of the 560 option. 562 Bit 1 "Loopback" (L-bit). This bit when set indicates to the 563 transit nodes processing this option to send a copy of the 564 packet back to the source of the packet while it continues to 565 forward the original packet towards the destination. The L-bit 566 MUST be cleared in the copy of the packet before sending it. 568 Node data List [n]: Variable-length field. The type of which is 569 determined by the OAM Type representing the n-th node data in the 570 node data list. The node data list is encoded starting from the 571 last node data of the path. The first element of the node data 572 list (node data list [0]) contains the last node of the path while 573 the last node data of the node data list (node data list[n]) 574 contains the first node data of the path traced. 576 4.1.3. In-situ OAM node data fields and associated formats 578 All the data fields MUST be 4-byte aligned. The IOAM encapsulating 579 node MUST initialize data fields that it adds to the packet to zero. 580 If a node which is supposed to update an IOAM data field is not 581 capable of populating the value of a field set in the IOAM-Trace- 582 Type, the field value MUST be left unaltered except when explicitly 583 specified in the field description below. In the description of data 584 below if zero is valid value then a non-zero value to mean not 585 populated is specified. 587 Data field and associated data type for each of the data field is 588 shown below: 590 Hop_Lim and node_id: 4-octet field defined as follows: 592 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 593 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 594 | Hop_Lim | node_id | 595 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 597 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 598 value in the packet at the node that records this data. Hop 599 Limit information is used to identify the location of the node 600 in the communication path. This is copied from the lower 601 layer, e.g., TTL value in IPv4 header or hop limit field from 602 IPv6 header of the packet when the packet is ready for 603 transmission. 605 node_id: 3-octet unsigned integer. Node identifier field to 606 uniquely identify a node within in-situ OAM domain. The 607 procedure to allocate, manage and map the node_ids is beyond 608 the scope of this document. 610 ingress_if_id and egress_if_id: 4-octet field defined as follows: 611 When this field is part of the data field but a node populating 612 the field is not able to fill it, the position in the field must 613 be filled with value 0xFFFFFFFF to mean not populated. 615 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 616 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 617 | ingress_if_id | egress_if_id | 618 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 620 ingress_if_id: 2-octet unsigned integer. Interface identifier to 621 record the ingress interface the packet was received on. 623 egress_if_id: 2-octet unsigned integer. Interface identifier to 624 record the egress interface the packet is forwarded out of. 626 timestamp seconds: 4-octet unsigned integer. Absolute timestamp in 627 seconds that specifies the time at which the packet was received 628 by the node. The structure of this field is identical to the most 629 significant 32 bits of the 64 least significant bits of the 630 [IEEE1588v2] timestamp. This truncated field consists of a 32-bit 631 seconds field. As defined in [IEEE1588v2], the timestamp 632 specifies the number of seconds elapsed since 1 January 1970 633 00:00:00 according to the International Atomic Time (TAI). 635 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 | timestamp seconds | 638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 640 timestamp nanoseconds: 4-octet unsigned integer in the range 0 to 641 10^9-1. This timestamp specifies the fractional part of the wall 642 clock time at which the packet was received by the node in units 643 of nanoseconds. This field is identical to the 32 least 644 significant bits of the [IEEE1588v2] timestamp. This fields 645 allows for delay computation between any two nodes in the network 646 when the nodes are time synchronized. When this field is part of 647 the data field but a node populating the field is not able to fill 648 it, the field position in the field must be filled with value 649 0xFFFFFFFF to mean not populated. 651 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 652 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 653 | timestamp nanoseconds | 654 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 655 transit delay: 4-octet unsigned integer in the range 0 to 2^30-1. 656 It is the time in nanoseconds the packet spent in the transit 657 node. This can serve as an indication of the queuing delay at the 658 node. If the transit delay exceeds 2^30-1 nanoseconds then the 659 top bit 'O' is set to indicate overflow. When this field is part 660 of the data field but a node populating the field is not able to 661 fill it, the field position in the field must be filled with value 662 0xFFFFFFFF to mean not populated. 664 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 666 |O| transit delay | 667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 669 app_data: 4-octet placeholder which can be used by the node to add 670 application specific data 672 queue depth: 4-octet unsigned integer field. This field indicates 673 the current length of the egress interface queue of the interface 674 from where the packet is forwarded out. The queue depth is 675 expressed as the current number of memory buffers used by the 676 queue (a packet may consume one or more memory buffers, depending 677 on its size). When this field is part of the data field but a 678 node populating the field is not able to fill it, the field 679 position in the field must be filled with value 0xFFFFFFFF to mean 680 not populated. 682 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 683 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 684 | queue depth | 685 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 687 Opaque State Snapshot: Variable length field. It allows the network 688 element to store an arbitrary state in the node data field , 689 without a pre-defined schema. The schema needs to be made known 690 to the analyzer by some out-of-band mechanism. The specification 691 of this mechanism is beyond the scope of this document. The 692 24-bit "Schema Id" field in the field indicates which particular 693 schema is used, and should be configured on the network element by 694 the operator. 696 0 1 2 3 697 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 698 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 699 | Length | Schema ID | 700 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 701 | | 702 | | 703 | Opaque data | 704 ~ ~ 705 . . 706 . . 707 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 709 Length: 1-octet unsigned integer. It is the length in octets of 710 the Opaque data field that follows Schema Id. It MUST always 711 be a multiple of 4. 713 Schema ID: 3-octet unsigned integer identifying the schema of 714 Opaque data. 716 Opaque data: Variable length field. This field is interpreted as 717 specified by the schema identified by the Schema ID. 719 Hop_Lim and node_id wide: 8-octet field defined as follows: 721 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 722 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 723 | Hop_Lim | node_id ~ 724 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 725 ~ node_id (contd) | 726 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 728 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 729 value in the packet at the node that records this data. Hop 730 Limit information is used to identify the location of the node 731 in the communication path. This is copied from the lower layer 732 for e.g. TTL value in IPv4 header or hop limit field from IPv6 733 header of the packet. 735 node_id: 7-octet unsigned integer. Node identifier field to 736 uniquely identify a node within in-situ OAM domain. The 737 procedure to allocate, manage and map the node_ids is beyond 738 the scope of this document. 740 ingress_if_id and egress_if_id wide: 8-octet field defined as 741 follows: When this field is part of the data field but a node 742 populating the field is not able to fill it, the field position in 743 the field must be filled with value 0xFFFFFFFF to mean not 744 populated. 746 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 747 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 748 | ingress_if_id | 749 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 750 | egress_if_id | 751 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 753 ingress_if_id: 4-octet unsigned integer. Interface identifier to 754 record the ingress interface the packet was received on. 756 egress_if_id: 4-octet unsigned integer. Interface identifier to 757 record the egress interface the packet is forwarded out of. 759 app_data wide: 8-octet placeholder which can be used by the node to 760 add application specific data. 762 4.1.4. Examples of In-situ OAM node data 764 An entry in the "node data list" array can have different formats, 765 following the needs of the deployment. Some deployments might only 766 be interested in recording the node identifiers, whereas others might 767 be interested in recording node identifier and timestamp. The 768 section defines different types that an entry in "node data list" can 769 take. 771 0x002B: IOAM-Trace-Type is 0x2B then the format of node data is: 773 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 774 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 775 | Hop_Lim | node_id | 776 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 777 | ingress_if_id | egress_if_id | 778 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 779 | timestamp nanoseconds | 780 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 781 | app_data | 782 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 784 0x0003: IOAM-Trace-Type is 0x0003 then the format is: 786 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 787 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 788 | Hop_Lim | node_id | 789 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 790 | ingress_if_id | egress_if_id | 791 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 793 0x0009: IOAM-Trace-Type is 0x0009 then the format is: 795 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 796 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 797 | Hop_Lim | node_id | 798 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 799 | timestamp nanoseconds | 800 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 802 0x0021: IOAM-Trace-Type is 0x0021 then the format is: 804 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 805 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 806 | Hop_Lim | node_id | 807 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 808 | app_data | 809 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 811 0x0029: IOAM-Trace-Type is 0x0029 then the format is: 813 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 814 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 815 | Hop_Lim | node_id | 816 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 817 | timestamp nanoseconds | 818 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 819 | app_data | 820 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 822 0x018C: IOAM-Trace-Type is 0x104D then the format is: 824 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 825 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 826 | timestamp seconds | 827 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 828 | timestamp nanoseconds | 829 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 830 | Length | Schema Id | 831 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 832 | | 833 | | 834 | Opaque data | 835 ~ ~ 836 . . 837 . . 838 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 839 | Hop_Lim | node_id | 840 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 841 | node_id(contd) | 842 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 844 4.2. In-situ OAM Proof of Transit Option 846 In-situ OAM Proof of Transit data is to support the path or service 847 function chain [RFC7665] verification use cases. Proof-of-transit 848 uses methods like nested hashing or nested encryption of the in-situ 849 OAM data or mechanisms such as Shamir's Secret Sharing Schema (SSSS). 850 While details on how the in-situ OAM data for the proof of transit 851 option is processed at in-situ OAM encapsulating, decapsulating and 852 transit nodes are outside the scope of the document, all of these 853 approaches share the need to uniquely identify a packet as well as 854 iteratively operate on a set of information that is handed from node 855 to node. Correspondingly, two pieces of information are added as in- 856 situ OAM data to the packet: 858 o Random: Unique identifier for the packet (e.g., 64-bits allow for 859 the unique identification of 2^64 packets). 861 o Cumulative: Information which is handed from node to node and 862 updated by every node according to a verification algorithm. 864 In-situ OAM Proof of Transit Option: 866 In-situ OAM Proof of Transit Option Header: 868 0 1 2 3 4 5 6 7 869 +-+-+-+-+-+-+-+-+ 870 |IOAM POT Type|P| 871 +-+-+-+-+-+-+-+-+ 873 In-situ OAM Proof of Transit Option Data MUST be 4-byte aligned: 875 0 1 2 3 876 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 877 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 878 | Random | | 879 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ P 880 | Random(contd) | O 881 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ T 882 | Cumulative | | 883 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 884 | Cumulative (contd) | | 885 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 887 IOAM POT Type: 7-bit identifier of a particular POT variant that 888 dictates the POT data that is included. This document defines POT 889 Type 0: 891 0: POT data is a 16 Octet field as described below. 893 Profile to use (P): 1-bit. Indicates which POT-profile is used to 894 generate the Cumulative. Any node participating in POT will have 895 a maximum of 2 profiles configured that drive the computation of 896 cumulative. The two profiles are numbered 0, 1. This bit conveys 897 whether profile 0 or profile 1 is used to compute the Cumulative. 899 Random: 64-bit Per packet Random number. 901 Cumulative: 64-bit Cumulative that is updated at specific nodes by 902 processing per packet Random number field and configured 903 parameters. 905 Note: Larger or smaller sizes of "Random" and "Cumulative" data are 906 feasible and could be required for certain deployments (e.g. in case 907 of space constraints in the transport protocol used). Future 908 versions of this document will address different sizes of data for 909 "proof of transit". 911 4.3. In-situ OAM Edge-to-Edge Option 913 The in-situ OAM Edge-to-Edge Option is to carry data that is added by 914 the in-situ OAM encapsulating node and interpreted by in-situ OAM 915 decapsulating node. The in-situ OAM transit nodes MAY process the 916 data without modifying it. 918 Currently only sequence numbers use the in-situ OAM Edge-to-Edge 919 option. In order to detect packet loss, packet reordering, or packet 920 duplication in an in-situ OAM-domain, sequence numbers can be added 921 to packets of a particular tube (see 922 [I-D.hildebrand-spud-prototype]). Each tube leverages a dedicated 923 namespace for its sequence numbers. 925 In-situ OAM Edge-to-Edge Option: 927 In-situ OAM Edge-to-Edge Option Header: 929 0 1 2 3 4 5 6 7 930 +-+-+-+-+-+-+-+-+ 931 | IOAM-E2E-Type | 932 +-+-+-+-+-+-+-+-+ 934 In-situ OAM Edge-to-Edge Option Data MUST be 4-byte aligned: 936 0 1 2 3 937 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 938 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 939 | E2E Option data field determined by IOAM-E2E-Type | 940 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 942 IOAM-E2E-Type: 8-bit identifier of a particular in situ OAM E2E 943 variant. 945 0: E2E option data is a 64-bit sequence number added to a 946 specific tube which is used to identify packet loss and 947 reordering for that tube. 949 5. In-situ OAM Data Export 951 In-situ OAM nodes collect information for packets traversing a domain 952 that supports in-situ OAM. The device at the domain edge (which 953 could also be an end-host) which receives a packet with in-situ OAM 954 information chooses how to process the in-situ OAM data collected 955 within the packet. This decapsulating node can simply discard the 956 information collected, can process the information further, or export 957 the information using e.g., IPFIX. 959 The discussion of in-situ OAM data processing and export is left for 960 a future version of this document. 962 6. IANA Considerations 964 IANA considerations will be added in a future version of this 965 document. 967 7. Manageability Considerations 969 Manageability considerations will be addressed in a later version of 970 this document.. 972 8. Security Considerations 974 Security considerations will be addressed in a later version of this 975 document. For a discussion of security requirements of in-situ OAM, 976 please refer to [I-D.brockners-inband-oam-requirements]. 978 9. Acknowledgements 980 The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari 981 Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya 982 Nadahalli, LJ Wobker, Erik Nordmark, Vengada Prasad Govindan, and 983 Andrew Yourtchenko for the comments and advice. This document 984 leverages and builds on top of several concepts described in 985 [I-D.kitamura-ipv6-record-route]. The authors would like to 986 acknowledge the work done by the author Hiroshi Kitamura and people 987 involved in writing it. 989 10. References 991 10.1. Normative References 993 [I-D.brockners-inband-oam-requirements] 994 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 995 Gredler, H., Leddy, J., Youell, S., Mozes, D., Mizrahi, 996 T., <>, P., and r. remy@barefootnetworks.com, 997 "Requirements for In-situ OAM", draft-brockners-inband- 998 oam-requirements-03 (work in progress), March 2017. 1000 [IEEE1588v2] 1001 Institute of Electrical and Electronics Engineers, 1002 "1588-2008 - IEEE Standard for a Precision Clock 1003 Synchronization Protocol for Networked Measurement and 1004 Control Systems", IEEE Std 1588-2008, 2008, 1005 . 1008 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1009 Requirement Levels", BCP 14, RFC 2119, 1010 DOI 10.17487/RFC2119, March 1997, 1011 . 1013 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1014 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1015 May 2016, . 1017 10.2. Informative References 1019 [I-D.brockners-inband-oam-transport] 1020 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 1021 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, 1022 D., Lapukhov, P., and R. <>, "Encapsulations for In-situ 1023 OAM Data", draft-brockners-inband-oam-transport-03 (work 1024 in progress), March 2017. 1026 [I-D.hildebrand-spud-prototype] 1027 Hildebrand, J. and B. Trammell, "Substrate Protocol for 1028 User Datagrams (SPUD) Prototype", draft-hildebrand-spud- 1029 prototype-03 (work in progress), March 2015. 1031 [I-D.ietf-nvo3-geneve] 1032 Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic 1033 Network Virtualization Encapsulation", draft-ietf- 1034 nvo3-geneve-04 (work in progress), March 2017. 1036 [I-D.ietf-nvo3-vxlan-gpe] 1037 Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol 1038 Extension for VXLAN", draft-ietf-nvo3-vxlan-gpe-04 (work 1039 in progress), April 2017. 1041 [I-D.ietf-sfc-nsh] 1042 Quinn, P. and U. Elzur, "Network Service Header", draft- 1043 ietf-sfc-nsh-12 (work in progress), February 2017. 1045 [I-D.kitamura-ipv6-record-route] 1046 Kitamura, H., "Record Route for IPv6 (PR6) Hop-by-Hop 1047 Option Extension", draft-kitamura-ipv6-record-route-00 1048 (work in progress), November 2000. 1050 [I-D.lapukhov-dataplane-probe] 1051 Lapukhov, P. and r. remy@barefootnetworks.com, "Data-plane 1052 probe for in-band telemetry collection", draft-lapukhov- 1053 dataplane-probe-01 (work in progress), June 2016. 1055 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 1056 Chaining (SFC) Architecture", RFC 7665, 1057 DOI 10.17487/RFC7665, October 2015, 1058 . 1060 Authors' Addresses 1062 Frank Brockners 1063 Cisco Systems, Inc. 1064 Hansaallee 249, 3rd Floor 1065 DUESSELDORF, NORDRHEIN-WESTFALEN 40549 1066 Germany 1068 Email: fbrockne@cisco.com 1070 Shwetha Bhandari 1071 Cisco Systems, Inc. 1072 Cessna Business Park, Sarjapura Marathalli Outer Ring Road 1073 Bangalore, KARNATAKA 560 087 1074 India 1076 Email: shwethab@cisco.com 1078 Carlos Pignataro 1079 Cisco Systems, Inc. 1080 7200-11 Kit Creek Road 1081 Research Triangle Park, NC 27709 1082 United States 1084 Email: cpignata@cisco.com 1086 Hannes Gredler 1087 RtBrick Inc. 1089 Email: hannes@rtbrick.com 1091 John Leddy 1092 Comcast 1094 Email: John_Leddy@cable.comcast.com 1095 Stephen Youell 1096 JP Morgan Chase 1097 25 Bank Street 1098 London E14 5JP 1099 United Kingdom 1101 Email: stephen.youell@jpmorgan.com 1103 Tal Mizrahi 1104 Marvell 1105 6 Hamada St. 1106 Yokneam 2066721 1107 Israel 1109 Email: talmi@marvell.com 1111 David Mozes 1112 Mellanox Technologies Ltd. 1114 Email: davidm@mellanox.com 1116 Petr Lapukhov 1117 Facebook 1118 1 Hacker Way 1119 Menlo Park, CA 94025 1120 US 1122 Email: petr@fb.com 1124 Remy Chang 1125 Barefoot Networks 1126 2185 Park Boulevard 1127 Palo Alto, CA 94306 1128 US 1130 Daniel 1131 Bell Canada 1133 Email: daniel.bernier@bell.ca