idnits 2.17.1 draft-brockners-inband-oam-data-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 13, 2017) is 2598 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '0' on line 502 -- Looks like a reference, but probably isn't: '1' on line 416 == Outdated reference: A later version (-03) exists of draft-brockners-inband-oam-requirements-02 == Outdated reference: A later version (-05) exists of draft-brockners-inband-oam-transport-02 == Outdated reference: A later version (-16) exists of draft-ietf-nvo3-geneve-03 == Outdated reference: A later version (-13) exists of draft-ietf-nvo3-vxlan-gpe-03 == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-12 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ippm F. Brockners 3 Internet-Draft S. Bhandari 4 Intended status: Experimental C. Pignataro 5 Expires: September 14, 2017 Cisco 6 H. Gredler 7 RtBrick Inc. 8 J. Leddy 9 Comcast 10 S. Youell 11 JMPC 12 T. Mizrahi 13 Marvell 14 D. Mozes 15 Mellanox Technologies Ltd. 16 P. Lapukhov 17 Facebook 18 R. Chang 19 Barefoot Networks 20 March 13, 2017 22 Data Fields for In-situ OAM 23 draft-brockners-inband-oam-data-03 25 Abstract 27 In-situ Operations, Administration, and Maintenance (IOAM) records 28 operational and telemetry information in the packet while the packet 29 traverses a path between two points in the network. This document 30 discusses the data fields and associated data types for in-situ OAM. 31 In-situ OAM data fields can be embedded into a variety of transports 32 such as NSH, Segment Routing, VXLAN-GPE, Geneve, native IPv6 (via 33 extension header), or IPv4. In-situ OAM can be used to complement 34 current out-of-band OAM mechanisms based on ICMP or other types of 35 probe packets. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at http://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on September 14, 2017. 54 Copyright Notice 56 Copyright (c) 2017 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (http://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 72 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 73 3. In-situ OAM Data Types and Formats . . . . . . . . . . . . . 4 74 3.1. In-situ OAM Tracing Options . . . . . . . . . . . . . . . 4 75 3.1.1. Pre-allocated Trace Option . . . . . . . . . . . . . 6 76 3.1.2. Incremental Trace Option . . . . . . . . . . . . . . 9 77 3.1.3. In-situ OAM node data fields and associated formats . 12 78 3.1.4. Examples of In-situ OAM node data . . . . . . . . . . 16 79 3.2. In-situ OAM Proof of Transit Option . . . . . . . . . . . 18 80 3.3. In-situ OAM Edge-to-Edge Option . . . . . . . . . . . . . 20 81 4. In-situ OAM Data Export . . . . . . . . . . . . . . . . . . . 20 82 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 83 6. Manageability Considerations . . . . . . . . . . . . . . . . 21 84 7. Security Considerations . . . . . . . . . . . . . . . . . . . 21 85 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 86 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 87 9.1. Normative References . . . . . . . . . . . . . . . . . . 21 88 9.2. Informative References . . . . . . . . . . . . . . . . . 22 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 91 1. Introduction 93 This document defines data fields for "in-situ" Operations, 94 Administration, and Maintenance (OAM). In-situ OAM records OAM 95 information within the packet while the packet traverses a particular 96 network domain. The term "in-situ" refers to the fact that the OAM 97 data is added to the data packets rather than is being sent within 98 packets specifically dedicated to OAM. A discussion of the 99 motivation and requirements for in-situ OAM can be found in 100 [I-D.brockners-inband-oam-requirements]. In-situ OAM is to 101 complement "out-of-band" or "active" mechanisms such as Ping or 102 Traceroute, or more recent active probing mechanisms as described in 103 [I-D.lapukhov-dataplane-probe]. In-situ OAM mechanisms can be 104 leveraged where current out-of-band mechanisms do not apply or do not 105 offer the desired results, such as proving that a certain traffic 106 flow takes a pre-defined path, SLA verification for the live data 107 traffic, detailed statistics on traffic distribution paths in 108 networks that distribute traffic across multiple paths, or scenarios 109 in which probe traffic is potentially handled differently from 110 regular data traffic by the network devices. 112 This document defines the data fields and associated data types for 113 in-situ OAM The in-situ OAM data field can be transported by a 114 variety of transport protocols, including NSH, Segment Routing, 115 VXLAN-GPE, Geneve, IPv6, or IPv4. Encapsulation details for these 116 different transport protocols are outside the scope of this document. 118 2. Conventions 120 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 121 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 122 document are to be interpreted as described in [RFC2119]. 124 Abbreviations used in this document: 126 Geneve: Generic Network Virtualization Encapsulation 127 [I-D.ietf-nvo3-geneve] 129 IOAM: In-situ Operations, Administration, and Maintenance 131 MTU: Maximum Transmit Unit 133 NSH: Network Service Header [I-D.ietf-sfc-nsh] 135 OAM: Operations, Administration, and Maintenance 137 SFC: Service Function Chain 139 SID: Segment Identifier 141 SR: Segment Routing 143 VXLAN-GPE: Virtual eXtensible Local Area Network, Generic Protocol 144 Extension [I-D.ietf-nvo3-vxlan-gpe] 146 3. In-situ OAM Data Types and Formats 148 This section defines in-situ OAM data types and data fields and 149 associated data types required for in-situ OAM. The different uses 150 of in-situ OAM require the definition of different types of data. 151 The in-situ OAM data fields for the data being carried corresponds to 152 the three main categories of in-situ OAM data defined in 153 [I-D.brockners-inband-oam-requirements], which are: edge-to-edge, per 154 node, and for selected nodes only. 156 Transport options for in-situ OAM data are outside the scope of this 157 memo, and are discussed in [I-D.brockners-inband-oam-transport]. In- 158 situ OAM data fields are fixed length data fields. A bit field 159 determines the set of OAM data fields embedded in a packet. 160 Depending on the type of the encapsulation, a counter field indicates 161 how many data fields are included in a particular packet. 163 In-situ OAM is expected to be deployed in a specific domain rather 164 than on the overall Internet. The part of the network which employs 165 in- situ OAM is referred to as the "in-situ OAM-domain". In-situ OAM 166 data is added to a packet upon entering the in-situ OAM-domain and is 167 removed from the packet when exiting the domain. Within the in-situ 168 OAM-domain, the in-situ OAM data may be updated by network nodes that 169 the packet traverses. The device which adds an in-situ OAM data 170 container to the packet to capture in-situ OAM data is called the 171 "in-situ OAM encapsulating node", whereas the device which removes 172 the in-situ OAM data container is referred to as the "in-situ OAM 173 decapsulating node". Nodes within the domain which are aware of in- 174 situ OAM data and read and/or write or process the in-situ OAM data 175 are called "in-situ OAM transit nodes". Note that not every node in 176 an in-situ OAM domain needs to be an in-situ OAM transit node. For 177 example, a Segment Routing deployment might require the segment 178 routing path to be verified. In that case, only the SR nodes would 179 also be in-situ OAM transit nodes rather than all nodes. 181 3.1. In-situ OAM Tracing Options 183 "In-situ OAM tracing data" is expected to be collected at every node 184 that a packet traverses, i.e., in a typical deployment all nodes in 185 an in-situ OAM-domain would participate in in-situ OAM and thus be 186 in-situ OAM transit nodes, in-situ OAM encapsulating or in-situ OAM 187 decapsulating nodes. The maximum number of hops and the minimum path 188 MTU of the in-situ OAM domain is assumed to be known. 190 To optimize hardware and software implementations tracing is defined 191 as two separate options. Any deployment MAY choose to configure and 192 support one or both of the following options. An implementation of 193 the transport protocol that carries these in-situ OAM data MAY choose 194 to support only one of the options. In the event that both options 195 are utilized at the same time, the Incremental Trace Option MUST be 196 placed before the Pre-allocated Trace Option. 198 Pre-allocated Trace Option: This trace option is defined as a 199 container of node data fields with pre-allocated space for each 200 node to populate its information. This option is useful for 201 software implementations where it is efficient to allocate the 202 space once and index into the array to populate the data during 203 transit. The in-situ OAM encapsulating node allocates the option 204 header and sets the fields in the option header. The in situ OAM 205 encapsulating node allocates an array which is used to store 206 operational data retrieved from every node while the packet 207 traverses the domain. In-situ OAM transit nodes update the 208 content of the array. A pointer which is part of the in-situ OAM 209 trace data points to the next empty slot in the array, which is 210 where the next in-situ OAM transit node fills in its data. 212 Incremental Trace Option: This trace option is defined as a 213 container of node data fields where each node allocates and pushes 214 its node data immediately following the option header. The number 215 of node data fields recorded and maximum number of node data that 216 can be recorded are written into the option header. This type of 217 trace recording is useful for some of the hardware implementations 218 as this eliminates the need for the transit network elements to 219 read the full array in the option and allows for arbitrarily long 220 packets as the MTU allows. The in-situ OAM encapsulating node 221 allocates the option header. The in-situ OAM encapsulating node 222 based on operational state and configuration sets the fields in 223 the header to control how large the node data list can grow. In- 224 situ OAM transit nodes push their node data to the node data list 225 and increment the number of node data fields in the header. 227 Every node data entry is to hold information for a particular in-situ 228 OAM transit node that is traversed by a packet. The in-situ OAM 229 decapsulating node removes the in-situ OAM data and processes and/or 230 exports the metadata. In-situ OAM data uses its own name-space for 231 information such as node identifier or interface identifier. This 232 allows for a domain-specific definition and interpretation. For 233 example: In one case an interface-id could point to a physical 234 interface (e.g., to understand which physical interface of an 235 aggregated link is used when receiving or transmitting a packet) 236 whereas in another case it could refer to a logical interface (e.g., 237 in case of tunnels). 239 The following in-situ OAM data is defined for in-situ OAM tracing: 241 o Identification of the in-situ OAM node. An in-situ OAM node 242 identifier can match to a device identifier or a particular 243 control point or subsystem within a device. 245 o Identification of the interface that a packet was received on. 247 o Identification of the interface that a packet was sent out on. 249 o Time of day when the packet was processed by the node. Different 250 definitions of processing time are feasible and expected, though 251 it is important that all devices of an in-situ OAM domain follow 252 the same definition. 254 o Generic data: Format-free information where syntax and semantic of 255 the information is defined by the operator in a specific 256 deployment. For a specific deployment, all in-situ OAM nodes 257 should interpret the generic data the same way. Examples for 258 generic in-situ OAM data include geo-location information 259 (location of the node at the time the packet was processed), 260 buffer queue fill level or cache fill level at the time the packet 261 was processed, or even a battery charge level. 263 o A mechanism to detect whether in-situ OAM trace data was added at 264 every hop or whether certain hops in the domain weren't in-situ 265 OAM transit nodes. 267 The "node data list" array in the packet is populated iteratively as 268 the packet traverses the network, starting with the last entry of the 269 array, i.e., "node data list [n]" is the first entry to be populated, 270 "node data list [n-1]" is the second one, etc. 272 3.1.1. Pre-allocated Trace Option 273 In-situ OAM Pre-allocated Trace Option: 275 Pre-allocated Trace Option header: 277 0 1 2 3 278 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 279 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 280 | IOAM-Trace-Type | Octets-left | Flags | 281 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 283 Pre-allocated Trace Option Data MUST be 4-byte aligned: 284 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 285 | | | 286 | node data list [0] | | 287 | | | 288 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ D 289 | | a 290 | node data list [1] | t 291 | | a 292 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 293 ~ ... ~ S 294 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ p 295 | | a 296 | node data list [n-1] | c 297 | | e 298 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 299 | | | 300 | node data list [n] | | 301 | | | 302 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 304 IOAM-Trace-Type: A 16-bit identifier which specifies which data 305 types are used in this node data list. 307 The IOAM-Trace-Type value is a bit field. The following bit 308 fields are defined in this document, with details on each field 309 described in the Section 3.1.3. The order of packing the data 310 fields in each node data element follows the bit order of the 311 IOAM-Trace-Type field, as follows: 313 Bit 0 (Least significant bit) When set indicates presence of 314 Hop_Lim and node_id in the node data. 316 Bit 1 When set indicates presence of ingress_if_id and 317 egress_if_id (short format) in the node data. 319 Bit 2 When set indicates presence of timestamp seconds in the 320 node data 322 Bit 3 When set indicates presence of timestamp nanoseconds in 323 the node data. 325 Bit 4 When set indicates presence of transit delay in the node 326 data. 328 Bit 5 When set indicates presence of app_data (short format) in 329 the node data. 331 Bit 6 When set indicates presence of queue depth in the node 332 data. 334 Bit 7 When set indicates presence of variable length Opaque 335 State Snapshot field. 337 Bit 8 When set indicates presence of Hop_Lim and node_id in 338 wide format in the node data. 340 Bit 9 When set indicates presence of ingress_if_id and 341 egress_if_id in wide format in the node data. 343 Bit 10 When set indicates presence of app_data wide in the node 344 data. 346 Bit 11-15 Undefined in this draft. 348 Section 3.1.4 describes the in-situ OAM data types and their 349 formats. Within an in-situ OAM domain possible combinations of 350 these bits making the IOAM-Trace-Type can be restricted by 351 configuration knobs. 353 Octets-left: 8-bit unsigned integer. It is the data space in octets 354 remaining for recording the node data. This is used as an offset 355 in octets in data space to record node data element. 357 Flags 8-bit field. The following flags are defined: 359 Bit 0 "Overflow" (O-bit) (most significant bit). This bit is set 360 by the network element if there is not enough number of bytes 361 left to record node data, no field is added and the overflow 362 "O-bit" must be set to "1" in the header. This is useful for 363 transit nodes to ignore further processing of the option. 365 Bit 1 "Loopback" (L-bit). Loopback mode is used to send a copy 366 of a packet back towards the source. Loopback mode assumes 367 that a return path from transit nodes and destination nodes 368 towards the source exists. The encapsulating node decides 369 (e.g. using a filter) which packets loopback mode is enabled 370 for by setting the loopback bit. The encapsulating node also 371 needs to ensure that sufficient space is available in the IOAM 372 header for loopback operation. The loopback bit when set 373 indicates to the transit nodes processing this option to create 374 a copy of the packet received and send this copy of the packet 375 back to the source of the packet while it continues to forward 376 the original packet towards the destination. The source 377 address of the original packet is used as destination address 378 in the copied packet. The address of the node performing the 379 copy operation is used as the source address. The L-bit MUST 380 be cleared in the copy of the packet a nodes sends it back 381 towards the source. On its way back towards the source, the 382 packet is processed like a regular packet with IOAM 383 information. Once the return packet reaches the IOAM domain 384 boundary IOAM decapsulation occurs as with any other packet 385 containing IOAM information. 387 Node data List [n]: Variable-length field. The type of which is 388 determined by the IOAM-Trace-Type representing the n-th node data 389 in the node data list. The node data list is encoded starting 390 from the last node data of the path. The first element of the 391 node data list (node data list [0]) contains the last node of the 392 path while the last node data of the node data list (node data 393 list[n]) contains the first node data of the path traced. The 394 index contained in "Octets-left" identifies the offset for current 395 active node data to be populated. 397 3.1.2. Incremental Trace Option 398 In-situ OAM Incremental Trace Option: ' 400 In-situ OAM Incremental Trace Option Header: 402 0 1 2 3 403 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 404 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 405 | IOAM-Trace-Type | Maximum Length| Flags | 406 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 408 In-situ OAM Incremental Trace Option Data MUST be 4-byte aligned: 410 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 411 | | 412 | node data list [0] | 413 | | 414 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 415 | | 416 | node data list [1] | 417 | | 418 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 419 ~ ... ~ 420 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 421 | | 422 | node data list [n-1] | 423 | | 424 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 425 | | 426 | node data list [n] | 427 | | 428 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 430 IOAM-trace-type: A 16-bit identifier which specifies which data 431 types are used in this node data list. 433 The IOAM-Trace-Type value is a bit field. The following bit 434 fields are defined in this document, with details on each field 435 described in the Section 3.1.3. The order of packing the data 436 fields in each node data element follows the bit order of the 437 IOAM-Trace-Type field, as follows: 439 Bit 0 When set indicates presence of Hop_Lim and node_id in the 440 node data. 442 Bit 1 When set indicates presence of ingress_if_id and 443 egress_if_id in the node data. 445 Bit 2 When set indicates presence of timestamp seconds in the 446 node data. 448 Bit 3 When set indicates presence of timestamp nanoseconds in 449 the node data. 451 Bit 4 When set indicates presence of transit delay in the node 452 data. 454 Bit 5 When set indicates presence of app_data in the node data. 456 Bit 6 When set indicates presence of queue depth in the node 457 data. 459 Bit 7 When set indicates presence of variable length Opaque 460 State Snapshot field. 462 Bit 8 When set indicates presence of Hop_Lim and node_id wide 463 in the node data. 465 Bit 9 When set indicates presence of ingress_if_id and 466 egress_if_id wide in the node data. 468 Bit 10 When set indicates presence of app_data wide in the node 469 data. 471 Bit 11-15 Undefined in this draft. 473 Section 3.1.4 describes the in-situ OAM data types and their 474 formats. 476 Maximum Length: 8-bit unsigned integer. This field specifies the 477 maximum length of the node data list in octets. Given that the 478 sender knows the minimum path MTU, the sender can set the maximum 479 of node data bytes allowed before exceeding the MTU. Thus, a 480 simple comparison between "Opt data Len" and "Max Length" allows 481 to decide whether or not data could be added. 483 Flags 8-bit field. Following flags are defined: 485 Bit 0 "Overflow" (O-bit) (least significant bit). This bit is 486 set by the network element if there is not enough number of 487 bytes left to record node data, no field is added and the 488 overflow "O-bit" must be set to "1" in the header. This is 489 useful for transit nodes to ignore further processing of the 490 option. 492 Bit 1 "Loopback" (L-bit). This bit when set indicates to the 493 transit nodes processing this option to send a copy of the 494 packet back to the source of the packet while it continues to 495 forward the original packet towards the destination. The L-bit 496 MUST be cleared in the copy of the packet before sending it. 498 Node data List [n]: Variable-length field. The type of which is 499 determined by the OAM Type representing the n-th node data in the 500 node data list. The node data list is encoded starting from the 501 last node data of the path. The first element of the node data 502 list (node data list [0]) contains the last node of the path while 503 the last node data of the node data list (node data list[n]) 504 contains the first node data of the path traced. 506 3.1.3. In-situ OAM node data fields and associated formats 508 All the data fields MUST be 4-byte aligned. The IOAM encapsulating 509 node MUST initialize data fields that it adds to the packet to zero. 510 If a node which is supposed to update an IOAM data field is not 511 capable of populating the value of a field set in the IOAM-Trace- 512 Type, the field value MUST be left unaltered except when explicitly 513 specified in the field description below. In the description of data 514 below if zero is valid value then a non-zero value to mean not 515 populated is specified. 517 Data field and associated data type for each of the data field is 518 shown below: 520 Hop_Lim and node_id: 4-octet field defined as follows: 522 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 523 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 524 | Hop_Lim | node_id | 525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 527 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 528 value in the packet at the node that records this data. Hop 529 Limit information is used to identify the location of the node 530 in the communication path. This is copied from the lower 531 layer, e.g., TTL value in IPv4 header or hop limit field from 532 IPv6 header of the packet when the packet is ready for 533 transmission. 535 node_id: 3-octet unsigned integer. Node identifier field to 536 uniquely identify a node within in-situ OAM domain. The 537 procedure to allocate, manage and map the node_ids is beyond 538 the scope of this document. 540 ingress_if_id and egress_if_id: 4-octet field defined as follows: 541 When this field is part of the data field but a node populating 542 the field is not able to fill it, the position in the field must 543 be filled with value 0xFFFF to mean not populated. 545 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 546 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 547 | ingress_if_id | egress_if_id | 548 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 550 ingress_if_id: 2-octet unsigned integer. Interface identifier to 551 record the ingress interface the packet was received on. 553 egress_if_id: 2-octet unsigned integer. Interface identifier to 554 record the egress interface the packet is forwarded out of. 556 timestamp seconds: 4-octet unsigned integer. Absolute timestamp in 557 seconds that specifies the time at which the packet was received 558 by the node. The structure of this field is identical to the most 559 significant 32 bits of the 64 least significant bits of the 560 [IEEE1588v2] timestamp. This truncated field consists of a 32-bit 561 seconds field. As defined in [IEEE1588v2], the timestamp 562 specifies the number of seconds elapsed since 1 January 1970 563 00:00:00 according to the International Atomic Time (TAI). 565 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 566 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 567 | timestamp seconds | 568 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 570 timestamp nanoseconds: 4-octet unsigned integer in the range 0 to 571 10^9-1. This timestamp specifies the fractional part of the wall 572 clock time at which the packet was received by the node in units 573 of nanoseconds. This field is identical to the 32 least 574 significant bits of the [IEEE1588v2] timestamp. This fields 575 allows for delay computation between any two nodes in the network 576 when the nodes are time synchronized. When this field is part of 577 the data field but a node populating the field is not able to fill 578 it, the field position in the field must be filled with value 579 0xFFFF to mean not populated. 581 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 582 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 583 | timestamp nanoseconds | 584 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 586 transit delay: 4-octet unsigned integer in the range 0 to 2^30-1. 587 It is the time in nanoseconds the packet spent in the transit 588 node. This can serve as an indication of the queuing delay at the 589 node. If the transit delay exceeds 2^30-1 nanoseconds then the 590 top bit 'O' is set to indicate overflow. When this field is part 591 of the data field but a node populating the field is not able to 592 fill it, the field position in the field must be filled with value 593 0xFFFF to mean not populated. 595 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 597 |O| transit delay | 598 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 600 app_data: 4-octet placeholder which can be used by the node to add 601 application specific data 603 queue depth: 4-octet unsigned integer field. This field indicates 604 the current length of the egress interface queue of the interface 605 from where the packet is forwarded out. The queue depth is 606 expressed as the current number of memory buffers used by the 607 queue (a packet may consume one or more memory buffers, depending 608 on its size). When this field is part of the data field but a 609 node populating the field is not able to fill it, the field 610 position in the field must be filled with value 0xFFFF to mean not 611 populated. 613 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 614 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 615 | queue depth | 616 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 618 Opaque State Snapshot: Variable length field. It allows the network 619 element to store an arbitrary state in the node data field , 620 without a pre-defined schema. The schema needs to be made known 621 to the analyzer by some out-of-band means. The 24-bit "Schema Id" 622 field in the field indicates which particular schema is used, and 623 should be configured on the network element by the operator. 625 0 1 2 3 626 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 627 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 628 | Length | Schema ID | 629 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 630 | | 631 | | 632 | Opaque data | 633 ~ ~ 634 . . 635 . . 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 638 Length: 1-octet unsigned integer. It is the length in octets of 639 the Opaque data field that follows Schema Id. It MUST always 640 be a multiple of 4. 642 Schema ID: 3-octet unsigned integer identifying the schema of 643 Opaque data. 645 Opaque data: Variable length field. This field is interpreted as 646 specified by the schema identified by the Schema ID. 648 Hop_Lim and node_id wide: 8-octet field defined as follows: 650 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 651 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 652 | Hop_Lim | node_id ~ 653 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 654 ~ node_id (contd) | 655 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 657 Hop_Lim: 1-octet unsigned integer. It is set to the Hop Limit 658 value in the packet at the node that records this data. Hop 659 Limit information is used to identify the location of the node 660 in the communication path. This is copied from the lower layer 661 for e.g. TTL value in IPv4 header or hop limit field from IPv6 662 header of the packet. 664 node_id: 7-octet unsigned integer. Node identifier field to 665 uniquely identify a node within in-situ OAM domain. The 666 procedure to allocate, manage and map the node_ids is beyond 667 the scope of this document. 669 ingress_if_id and egress_if_id wide: 8-octet field defined as 670 follows: When this field is part of the data field but a node 671 populating the field is not able to fill it, the field position in 672 the field must be filled with value 0xFFFFFFFF to mean not 673 populated. 675 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 676 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 677 | ingress_if_id | 678 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 679 | egress_if_id | 680 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 682 ingress_if_id: 4-octet unsigned integer. Interface identifier to 683 record the ingress interface the packet was received on. 685 egress_if_id: 4-octet unsigned integer. Interface identifier to 686 record the egress interface the packet is forwarded out of. 688 app_data wide: 8-octet placeholder which can be used by the node to 689 add application specific data. 691 3.1.4. Examples of In-situ OAM node data 693 An entry in the "node data list" array can have different formats, 694 following the needs of the deployment. Some deployments might only 695 be interested in recording the node identifiers, whereas others might 696 be interested in recording node identifier and timestamp. The 697 section defines different types that an entry in "node data list" can 698 take. 700 0x002B: IOAM-Trace-Type is 0x2B then the format of node data is: 702 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 703 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 704 | Hop_Lim | node_id | 705 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 706 | ingress_if_id | egress_if_id | 707 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 708 | timestamp nanoseconds | 709 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 710 | app_data | 711 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 713 0x0003: IOAM-Trace-Type is 0x0003 then the format is: 715 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 716 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 717 | Hop_Lim | node_id | 718 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 719 | ingress_if_id | egress_if_id | 720 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 722 0x0009: IOAM-Trace-Type is 0x0009 then the format is: 724 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 725 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 726 | Hop_Lim | node_id | 727 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 728 | timestamp nanoseconds | 729 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 731 0x0021: IOAM-Trace-Type is 0x0021 then the format is: 733 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 734 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 735 | Hop_Lim | node_id | 736 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 737 | app_data | 738 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 740 0x0029: IOAM-Trace-Type is 0x0029 then the format is: 742 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 743 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 744 | Hop_Lim | node_id | 745 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 746 | timestamp nanoseconds | 747 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 748 | app_data | 749 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 751 0x018C: IOAM-Trace-Type is 0x104D then the format is: 753 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 755 | timestamp seconds | 756 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 757 | timestamp nanoseconds | 758 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 759 | Length | Schema Id | 760 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 761 | | 762 | | 763 | Opaque data | 764 ~ ~ 765 . . 766 . . 767 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 768 | Hop_Lim | node_id | 769 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 770 | node_id(contd) | 771 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 773 3.2. In-situ OAM Proof of Transit Option 775 In-situ OAM Proof of Transit data is to support the path or service 776 function chain [RFC7665] verification use cases. Proof-of-transit 777 uses methods like nested hashing or nested encryption of the in-situ 778 OAM data or mechanisms such as Shamir's Secret Sharing Schema (SSSS). 779 While details on how the in-situ OAM data for the proof of transit 780 option is processed at in-situ OAM encapsulating, decapsulating and 781 transit nodes are outside the scope of the document, all of these 782 approaches share the need to uniquely identify a packet as well as 783 iteratively operate on a set of information that is handed from node 784 to node. Correspondingly, two pieces of information are added as in- 785 situ OAM data to the packet: 787 o Random: Unique identifier for the packet (e.g., 64-bits allow for 788 the unique identification of 2^64 packets). 790 o Cumulative: Information which is handed from node to node and 791 updated by every node according to a verification algorithm. 793 In-situ OAM Proof of Transit Option: 795 In-situ OAM Proof of Transit Option Header: 797 0 1 2 3 4 5 6 7 798 +-+-+-+-+-+-+-+-+ 799 |IOAM POT Type|P| 800 +-+-+-+-+-+-+-+-+ 802 In-situ OAM Proof of Transit Option Data MUST be 4-byte aligned: 804 0 1 2 3 805 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 806 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 807 | Random | | 808 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ P 809 | Random(contd) | O 810 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ T 811 | Cumulative | | 812 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 813 | Cumulative (contd) | | 814 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 816 IOAM POT Type: 7-bit identifier of a particular POT variant that 817 dictates the POT data that is included. This document defines POT 818 Type 0: 820 0: POT data is a 16 Octet field as described below. 822 Profile to use (P): 1-bit. Indicates which POT-profile is used to 823 generate the Cumulative. Any node participating in POT will have 824 a maximum of 2 profiles configured that drive the computation of 825 cumulative. The two profiles are numbered 0, 1. This bit conveys 826 whether profile 0 or profile 1 is used to compute the Cumulative. 828 Random: 64-bit Per packet Random number. 830 Cumulative: 64-bit Cumulative that is updated at specific nodes by 831 processing per packet Random number field and configured 832 parameters. 834 Note: Larger or smaller sizes of "Random" and "Cumulative" data are 835 feasible and could be required for certain deployments (e.g. in case 836 of space constraints in the transport protocol used). Future 837 versions of this document will address different sizes of data for 838 "proof of transit". 840 3.3. In-situ OAM Edge-to-Edge Option 842 The in-situ OAM Edge-to-Edge Option is to carry data that is added by 843 the in-situ OAM encapsulating node and interpreted by in-situ OAM 844 decapsulating node. The in-situ OAM transit nodes MAY process the 845 data without modifying it. 847 Currently only sequence numbers use the in-situ OAM Edge-to-Edge 848 option. In order to detect packet loss, packet reordering, or packet 849 duplication in an in-situ OAM-domain, sequence numbers can be added 850 to packets of a particular tube (see 851 [I-D.hildebrand-spud-prototype]). Each tube leverages a dedicated 852 namespace for its sequence numbers. 854 In-situ OAM Edge-to-Edge Option: 856 In-situ OAM Edge-to-Edge Option Header: 858 0 1 2 3 4 5 6 7 859 +-+-+-+-+-+-+-+-+ 860 | IOAM-E2E-Type | 861 +-+-+-+-+-+-+-+-+ 863 In-situ OAM Edge-to-Edge Option Data MUST be 4-byte aligned: 865 0 1 2 3 866 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 867 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 868 | E2E Option data field determined by IOAM-E2E-Type | 869 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 871 IOAM-E2E-Type: 8-bit identifier of a particular in situ OAM E2E 872 variant. 874 0: E2E option data is a 64-bit sequence number added to a 875 specific tube which is used to identify packet loss and 876 reordering for that tube. 878 4. In-situ OAM Data Export 880 In-situ OAM nodes collect information for packets traversing a domain 881 that supports in-situ OAM. The device at the domain edge (which 882 could also be an end-host) which receives a packet with in-situ OAM 883 information chooses how to process the in-situ OAM data collected 884 within the packet. This decapsulating node can simply discard the 885 information collected, can process the information further, or export 886 the information using e.g., IPFIX. 888 The discussion of in-situ OAM data processing and export is left for 889 a future version of this document. 891 5. IANA Considerations 893 IANA considerations will be added in a future version of this 894 document. 896 6. Manageability Considerations 898 Manageability considerations will be addressed in a later version of 899 this document.. 901 7. Security Considerations 903 Security considerations will be addressed in a later version of this 904 document. For a discussion of security requirements of in-situ OAM, 905 please refer to [I-D.brockners-inband-oam-requirements]. 907 8. Acknowledgements 909 The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari 910 Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya 911 Nadahalli, LJ Wobker, Erik Nordmark, Vengada Prasad Govindan, and 912 Andrew Yourtchenko for the comments and advice. This document 913 leverages and builds on top of several concepts described in 914 [I-D.kitamura-ipv6-record-route]. The authors would like to 915 acknowledge the work done by the author Hiroshi Kitamura and people 916 involved in writing it. 918 9. References 920 9.1. Normative References 922 [I-D.brockners-inband-oam-requirements] 923 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 924 Gredler, H., Leddy, J., Youell, S., Mozes, D., Mizrahi, 925 T., <>, P., and r. remy@barefootnetworks.com, 926 "Requirements for In-situ OAM", draft-brockners-inband- 927 oam-requirements-02 (work in progress), October 2016. 929 [IEEE1588v2] 930 Institute of Electrical and Electronics Engineers, 931 "1588-2008 - IEEE Standard for a Precision Clock 932 Synchronization Protocol for Networked Measurement and 933 Control Systems", IEEE Std 1588-2008, 2008, 934 . 937 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 938 Requirement Levels", BCP 14, RFC 2119, 939 DOI 10.17487/RFC2119, March 1997, 940 . 942 9.2. Informative References 944 [I-D.brockners-inband-oam-transport] 945 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 946 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 947 P., and R. <>, "Encapsulations for In-situ OAM Data", 948 draft-brockners-inband-oam-transport-02 (work in 949 progress), October 2016. 951 [I-D.hildebrand-spud-prototype] 952 Hildebrand, J. and B. Trammell, "Substrate Protocol for 953 User Datagrams (SPUD) Prototype", draft-hildebrand-spud- 954 prototype-03 (work in progress), March 2015. 956 [I-D.ietf-nvo3-geneve] 957 Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic 958 Network Virtualization Encapsulation", draft-ietf- 959 nvo3-geneve-03 (work in progress), September 2016. 961 [I-D.ietf-nvo3-vxlan-gpe] 962 Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol 963 Extension for VXLAN", draft-ietf-nvo3-vxlan-gpe-03 (work 964 in progress), October 2016. 966 [I-D.ietf-sfc-nsh] 967 Quinn, P. and U. Elzur, "Network Service Header", draft- 968 ietf-sfc-nsh-12 (work in progress), February 2017. 970 [I-D.kitamura-ipv6-record-route] 971 Kitamura, H., "Record Route for IPv6 (PR6) Hop-by-Hop 972 Option Extension", draft-kitamura-ipv6-record-route-00 973 (work in progress), November 2000. 975 [I-D.lapukhov-dataplane-probe] 976 Lapukhov, P. and r. remy@barefootnetworks.com, "Data-plane 977 probe for in-band telemetry collection", draft-lapukhov- 978 dataplane-probe-01 (work in progress), June 2016. 980 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 981 Chaining (SFC) Architecture", RFC 7665, 982 DOI 10.17487/RFC7665, October 2015, 983 . 985 Authors' Addresses 987 Frank Brockners 988 Cisco Systems, Inc. 989 Hansaallee 249, 3rd Floor 990 DUESSELDORF, NORDRHEIN-WESTFALEN 40549 991 Germany 993 Email: fbrockne@cisco.com 995 Shwetha Bhandari 996 Cisco Systems, Inc. 997 Cessna Business Park, Sarjapura Marathalli Outer Ring Road 998 Bangalore, KARNATAKA 560 087 999 India 1001 Email: shwethab@cisco.com 1003 Carlos Pignataro 1004 Cisco Systems, Inc. 1005 7200-11 Kit Creek Road 1006 Research Triangle Park, NC 27709 1007 United States 1009 Email: cpignata@cisco.com 1011 Hannes Gredler 1012 RtBrick Inc. 1014 Email: hannes@rtbrick.com 1016 John Leddy 1017 Comcast 1019 Email: John_Leddy@cable.comcast.com 1021 Stephen Youell 1022 JP Morgan Chase 1023 25 Bank Street 1024 London E14 5JP 1025 United Kingdom 1027 Email: stephen.youell@jpmorgan.com 1028 Tal Mizrahi 1029 Marvell 1030 6 Hamada St. 1031 Yokneam 2066721 1032 Israel 1034 Email: talmi@marvell.com 1036 David Mozes 1037 Mellanox Technologies Ltd. 1039 Email: davidm@mellanox.com 1041 Petr Lapukhov 1042 Facebook 1043 1 Hacker Way 1044 Menlo Park, CA 94025 1045 US 1047 Email: petr@fb.com 1049 Remy Chang 1050 Barefoot Networks 1051 2185 Park Boulevard 1052 Palo Alto, CA 94306 1053 US