idnits 2.17.1 draft-kumar-ippm-ifa-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 2 longer pages, the longest (page 13) being 78 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 12, 2021) is 1073 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Missing Reference: 'RFC791' is mentioned on line 1475, but not defined == Unused Reference: 'RFC0791' is defined on line 1427, but no explicit reference was found in the text == Unused Reference: 'I-D.kumar-ifa' is defined on line 1439, but no explicit reference was found in the text Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Kumar 3 Internet-Draft S. Anubolu 4 Intended status: Experimental J. Lemon 5 Expires: November 16, 2021 R. Manur 6 Broadcom Inc. 7 H. Holbrook 8 Arista Networks 9 A. Ghanwani 10 Dell EMC 11 D. Cai 12 H. Ou 13 AliBaba Inc. 14 Y. Li 15 Huawei 16 X. Wang 17 Fujian Ruijie Networks co., ltd. 18 May 12, 2021 20 Inband Flow Analyzer 21 draft-kumar-ippm-ifa-03 23 Abstract 25 Inband Flow Analyzer (IFA) records flow specific information from an 26 end station and/or switches across a network. This document 27 discusses the method to collect data on a per hop basis across a 28 network and perform localized or end to end analytics operations on 29 the data. This document also describes a transport-agnostic header 30 definition that may be used for tunneled and non-tunneled flows 31 alike. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at https://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on November 16, 2021. 50 Copyright Notice 52 Copyright (c) 2021 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (https://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 68 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 69 1.2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 3 70 1.3. Applicability . . . . . . . . . . . . . . . . . . . . . . 4 71 1.4. Motivation . . . . . . . . . . . . . . . . . . . . . . . 4 72 2. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4 73 2.1. Encapsulation Requirements . . . . . . . . . . . . . . . 4 74 2.2. Operational Requirements . . . . . . . . . . . . . . . . 5 75 2.3. Cost and Performance Requirements . . . . . . . . . . . . 6 76 3. IFA Operations . . . . . . . . . . . . . . . . . . . . . . . 6 77 3.1. IFA Zones . . . . . . . . . . . . . . . . . . . . . . . . 8 78 3.2. IFA Function Nodes . . . . . . . . . . . . . . . . . . . 8 79 3.2.1. Initiating Function Node . . . . . . . . . . . . . . 9 80 3.2.2. Transit Function Node . . . . . . . . . . . . . . . . 9 81 3.2.3. Terminating Function Node . . . . . . . . . . . . . . 9 82 3.2.4. Metadata Fragmentation Function . . . . . . . . . . . 9 83 3.3. IFA Cloning, Truncation, and Drop . . . . . . . . . . . . 10 84 3.4. IFA Header . . . . . . . . . . . . . . . . . . . . . . . 10 85 3.4.1. IFA Metadata Header . . . . . . . . . . . . . . . . . 13 86 3.4.2. IFA Checksum Header . . . . . . . . . . . . . . . . . 13 87 3.4.3. IFA Metadata Fragmentation (MF) Header . . . . . . . 14 88 3.5. IFA Metadata . . . . . . . . . . . . . . . . . . . . . . 15 89 3.5.1. Global Name Space (GNS) Identifier . . . . . . . . . 15 90 3.5.2. Local Name Space (LNS) Identifier . . . . . . . . . . 16 91 3.5.3. Device ID . . . . . . . . . . . . . . . . . . . . . . 16 92 3.6. IFA Network Overhead . . . . . . . . . . . . . . . . . . 16 93 3.7. IFA Analytics . . . . . . . . . . . . . . . . . . . . . . 17 94 3.8. IFA Packet Format . . . . . . . . . . . . . . . . . . . . 17 95 3.8.1. IFA Packet Format with TS Flag Set . . . . . . . . . 18 96 3.8.2. VxLAN Packet . . . . . . . . . . . . . . . . . . . . 20 97 3.8.3. GRE Packet . . . . . . . . . . . . . . . . . . . . . 22 98 3.8.4. Geneve Packet . . . . . . . . . . . . . . . . . . . . 23 99 3.8.5. IPinIP Packet . . . . . . . . . . . . . . . . . . . . 25 100 3.8.6. IPv6 Extension Headers with IFA . . . . . . . . . . . 26 101 3.8.7. IP AH/ESP/WESP Packet . . . . . . . . . . . . . . . . 28 102 3.9. IFA Load Balancing . . . . . . . . . . . . . . . . . . . 30 103 4. Interoperability Considerations . . . . . . . . . . . . . . . 30 104 5. Security Considerations . . . . . . . . . . . . . . . . . . . 31 105 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 31 106 6.1. Normative References . . . . . . . . . . . . . . . . . . 31 107 6.2. Informative References . . . . . . . . . . . . . . . . . 31 108 Appendix A. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 109 A.1. Probe Marker . . . . . . . . . . . . . . . . . . . . . . 32 110 A.2. DSCP . . . . . . . . . . . . . . . . . . . . . . . . . . 32 111 A.3. IP Options . . . . . . . . . . . . . . . . . . . . . . . 32 112 A.4. IPv4 Identification or Reserved Flag . . . . . . . . . . 33 113 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 33 115 1. Introduction 117 This document describes Inband Flow Analyzer (IFA) which is a 118 mechanism to mark packets in a flow to enable the collection of 119 metadata regarding the analyzed flow. IFA defines an IFA header to 120 mark the flow and direct the collection of analyzed metadata per 121 marked packet per hop across a network. The ability to mark a packet 122 using an IFA OAM header can also be leveraged to create synthetic 123 flows meant for network data collection. This document describes a 124 mechanism that may be used to monitor live traffic and/or create 125 synthetic flows. This document also describes IFA zones, IFA 126 reports, and IFA metadata. IFA does not require changes to protocol 127 headers in order to collect metadata or analyze flows. IFA puts 128 minimal requirements on switching silicon. 130 1.1. Terminology 132 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 133 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 134 document are to be interpreted as described in RFC 2119 [RFC2119]. 136 IFA: Inband Flow Analyzer 138 MTU: Maximum Transmit Unit 140 1.2. Scope 142 This document describes IFA deployment, the type of traffic that is 143 supported, header definitions, analytics, and data path functions. 145 IFA deployment involves defining an IFA zone and understanding the 146 requirements in terms of traffic overhead and points of data 147 collection. Given that IFA provides the ability to perform local 148 analytics on the collected data, this document describes the scope of 149 the analytics function as well. The scope of IFA is from an end 150 station and/or ToR, through any/all nodes in the network, and 151 terminating in a network switch and/or an end station. 153 IFA can create a synthetic stream of traffic and use it to collect 154 metadata along the path. This sampled stream is later discarded. 155 IFA can also insert metadata on a per packet basis in live traffic. 156 Inband insertion of metadata can be done within the payload or via 157 tail stamping. 159 This draft defines an identification mechanism using a dedicated 160 protocol type in the IP header for identifying IFA. 162 1.3. Applicability 164 IFA is capable of providing traffic analysis in an encapsulation- 165 agnostic manner. Simple TCP and UDP flows, as well as tunneled 166 flows, can be monitored. IFA can be enabled on an end station, or it 167 can be enabled just on network switches. Enabling IFA on an end 168 station provides better scalability and visibility by monitoring 169 intra end station or inter end station traffic. IFA performs best 170 when there is hardware assistance for deriving the flow metadata in 171 the data path. This document describes data path functions for IFA. 173 1.4. Motivation 175 The main motivation for IFA is to collect analyzed metadata from 176 packets within a flow for a given application. The definition of the 177 IFA header ensures that it works for any IP packet, and with minimal 178 impact on hardware performance. 180 2. Requirements 182 IFA requirements are defined with operational efficiency, performance 183 of the network, and cost of hardware in mind. 185 2.1. Encapsulation Requirements 187 IFA packets MUST be clearly marked and identifiable so that a 188 networking element in the flow path can insert metadata or perform 189 other IFA operations. 191 IFA packets need to be easily identified for performance reasons. 192 IFA packet identification MUST be the same for all the IP packet 193 types. This means that expensive hardware modifications are not 194 needed for supporting new protocol types. 196 Since IFA packet processing is a data path function, the IFA header 197 MUST keep the processing overhead minimal. Simple parsing in the 198 switch hardware with localized read/write fields in IFA header will 199 optimize the switch performance and cost. 201 A single IFA encapsulation MUST support IPv4 and IPv6 protocol types 202 for tunneled and non-tunneled packets, preserving the fields used for 203 load balancing hash computation. 205 IFA MAY support a checksum for the entire IFA metadata stack instead 206 of a checksum per metadata element. 208 2.2. Operational Requirements 210 IFA MUST preserve the flow path across the network. 212 IFA MUST incur minimal traffic overhead. 214 IFA MUST provide an option to clone and truncate a packet to avoid 215 disrupting the PMTU discovery of a network. 217 Cloning SHOULD be supported. Sampling of cloned traffic MUST be at a 218 sampled ratio to keep the network overhead to a minimum. 220 IFA MUST provide the ability to insert metadata on cloned traffic. 222 IFA MUST provide the ability to insert metadata on live traffic. 224 IFA MAY provide the ability to specify checksum validation on the IFA 225 header and metadata. 227 IFA MUST provide the ability to define a zone using hop count. 229 IFA MUST provide the ability for a networking element to perform 230 metadata insertion in the payload. 232 IFA MAY provide the ability for networking element to insert metadata 233 as tail stamping. 235 IFA MUST be able to support an IFA zone name space, also referred to 236 as a global name space. 238 IFA MUST be able to support a per hop name space, also referred to as 239 a local name space. 241 IFA MAY be able to support fragmentation of metadata. Fragmentation 242 is needed to support a large number of hops in the network path. 244 2.3. Cost and Performance Requirements 246 The IFA header and metadata MUST be treated as foreign data present 247 in the application data. IFA SHOULD be able to insert or strip the 248 IFA header and metadata without modifying the layer 4 headers. This 249 will help keep the cost of hardware down with no degradation in 250 performance. 252 IFA MUST support the ability to clone and/or truncate, live traffic 253 for IFA metadata insertion. This is needed for PMTU protocols to 254 work within the IFA zone. 256 The IFA header MUST provide the ability to differentiate between a 257 cloned packet and an original packet. This is needed for hardware to 258 be able to identify and filter the cloned traffic at the edge of an 259 IFA zone. 261 IFA encapsulation MUST provide mechanism to avoid impacting the parse 262 depth of hardware for packet processing. 264 IFA MUST NOT require pre-allocation for reserving the space in a 265 packet. The overhead of managing reserved space in a packet can 266 result in performance degradation. 268 3. IFA Operations 270 IFA performs flow analysis, and possible actions on the flow data, 271 inband. Once a flow is enabled for analysis, a node with the role of 272 "Initiator" makes a copy of the flow or samples the live traffic 273 flow, or tags a live traffic flow for analysis and data collection. 274 Copying of a flow is done by sampling or cloning the flow. These new 275 packets are representative packets of the original flow and possess 276 the exact same characteristics as the original flow. This means that 277 IFA packets traverse the same path in the network and same queues in 278 the networking element as the original packet would. Figure 1 shows 279 the IFA based Telemetry Framework. The terminating node is 280 responsible for terminating the IFA flow by summarizing the metadata 281 of the entire path and sending it to a Collector. 283 +----------+ 284 | | 285 |Collector |--------------+ 286 | | | 287 +----------+ | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 +--------------+ +------------+ +----+-----------+ 297 |Initiator Node| |Transit Node| |Terminating Node| 298 | +------+ | | +------+ | | +------+ | 299 | | IFA | |------->| | IFA | |------->| | IFA | | 300 | +------+ |IFA flow| +------+ |IFA flow| +------+ | 301 +--------------+ +------------+ +----------------+ 303 Figure 1: IFA Zone Framework without fragmentation 304 +----------+ 305 | | 306 |Collector |<-------------+ 307 +----->| | | 308 | +----------+ | 309 | | 310 | | 311 | | 312 | | 313 | | 314 | | 315 | | 316 | | 317 +--------------+ | +------------+ +----+----------+ 318 |Initiator Node| | |Transit Node| |Fragmenter Node| 319 | +------+ | | | +------+ | | +------+ | 320 | | IFA | |------->| | IFA | |------->| | IFA | | 321 | +------+ |IFA flow| +------+ |IFA flow| +------+ | 322 +--------------+ | +------------+ +---------------+ 323 | |IFA 324 | |flow 325 | v 326 | +---------------+ +------+-----+ 327 | |Terminator Node| |Transit Node| 328 +--| +------+ | | +------+ | 329 | | IFA | |<-------| | IFA | | 330 | +------+ |IFA flow| +------+ | 331 +---------------+ +------------+ 332 Figure 2 IFA Zone Framework with metadata fragmentation 334 3.1. IFA Zones 336 An IFA zone is the domain of interest where IFA monitoring is 337 enabled. An IFA zone MUST have designated IFA function nodes. An 338 IFA zone can be controlled by setting an appropriate TTL value in the 339 L3 header. Initiating and Terminating function nodes are always at 340 the edge of the IFA zone. Internal nodes in the IFA zone are always 341 Transit function nodes. 343 3.2. IFA Function Nodes 345 There are three types of IFA functional nodes with respect to a 346 specific or set of flows. Each node MAY perform metadata 347 fragmentation function as well. 349 3.2.1. Initiating Function Node 351 An end station, a switch, or any other middleware can perform the IFA 352 initiating function. It is advantageous to keep this role closest to 353 the application to maximize flow visibility. An IFA initiating 354 function node performs the following functions for a flow: 356 - Samples the flow traffic of interest based on a configuration. 357 - Converts the traffic into an IFA flow by adding an IFA header to 358 each sample. 359 - Updates the packet with initiating function node metadata. 360 - MAY mandate a specific template ID metadata by all networking 361 elements. 362 - MAY mandate tail stamping of metadata by all networking elements. 364 3.2.2. Transit Function Node 366 An IFA transit node is responsible for inserting transit node 367 metadata in the IFA packets in the specified flow. 369 3.2.3. Terminating Function Node 371 An IFA terminating node is responsible for the following for a flow: 372 - Inserts terminating node metadata in an IFA packet. 373 - Performs a local analytics function on one or more segments of 374 metadata, e.g., threshold breach for residence time, congestion 375 notifications, and so on. 376 - Filters an IFA flow in case of cloned traffic. 377 - Sends a copy or report of the packet to collector. 378 - Removes the IFA headers and forwards the packet in case of live 379 traffic. 381 3.2.4. Metadata Fragmentation Function 383 There are cases where the size of metadata may grow too big for link 384 MTU or path MTU, or where it imposes excessive overhead for the 385 terminating function node to remove it. This is specially true in 386 networks with a large number of hops between initiator function node 387 and terminating function node. This is also true where the size of 388 per hop metadata itself is large. For such cases, IFA defines a 389 metadata fragmentation function. Metadata fragmentation function 390 allows, removal of metadata from the packet and send a copy/report of 391 the packet to collector. Correlation of metadata fragments and 392 recreation of metadata stack for the entire flow path is done by the 393 collector. 395 There is no dedicated node performing the metadata fragmentation 396 function. As an IFA packet traverses the hops in an IFA zone, any 397 node MAY detect the need to fragment the packet's metadata stack and 398 perform metadata fragmentation. 400 Metadata fragmentation is done if the IFA header in the packet has 401 "MDF" bit set and the current length of the metadata would exceed the 402 maximum length after the addition of metadata by the current node. A 403 node MAY create a copy of the packet or create an IFA report, remove 404 the existing metadata stack from the packet, insert its own metadata, 405 and finally forward the packet. A node MAY also update the IFA MDF 406 (Meta Data Fragment) header fragment identifier, current length, IP 407 length, and IP header checksum. 409 The maximum length in an IFA header, if set to "0", MAY trigger the 410 metadata fragmentation special function. This mechanism can be used 411 to generate IFA reports at each hop and never insert metadata in the 412 packet. If maximum length is set to "0", a node MAY ONLY create an 413 IFA report or copy of the packet including its own metadata. A node 414 MUST NOT update the IFA MD header current length, IP length, or 415 insert metadata in the IFA packet. The node MUST increment the IFA 416 MDF header fragment identifier field. 418 3.3. IFA Cloning, Truncation, and Drop 420 IFA allows cloning of live traffic. It is expected that cloned 421 traffic will have the same network path characterization as the 422 original traffic i.e. follow the same network path, use the same 423 queues etc. 425 Cloned traffic can be truncated to accommodate the PMTU of the IFA 426 zone. 428 Cloned traffic MUST be dropped by the terminating function node of 429 the IFA zone. 431 3.4. IFA Header 433 The IFA header is described below. An experimental IP protocol 434 number is used in the IP header to identify an IFA packet. The IP 435 header protocol type field is copied into the IFA header NextHdr 436 field for hardware to correctly interpret the layer 4 header. 438 0 1 2 3 439 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 440 IPv4 Header: 441 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 442 |Version| IHL |Type of Service| Total Length | 443 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 444 | Identification |Flags| Fragment Offset | 445 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 446 | Time to Live | Protocol = IFA| Header Checksum | 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 | Source IPv4 Address | 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 | Destination IPv4 Address | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 452 or 453 IPv6 Header: 454 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 455 |Version| Traffic Class | Flow Label | 456 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 457 | Payload Length |Next Hdr = IFA | Hop Limit | 458 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 459 | | 460 + + 461 | | 462 + Source IPv6 Address + 463 | | 464 + + 465 | | 466 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 467 | | 468 + + 469 | | 470 + Destination IPv6 Address + 471 | | 472 + + 473 | | 474 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 476 IFA Header: 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 479 | | | | | | |F|S| |A| | | 480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 482 Figure 2: IFA 2 Header Format 484 (1) Version (4 bits) - Specifies the version of IFA header. 486 (2) GNS (4 bits) - Global Name Space. Specifies the IFA zone scoped 487 name space for IFA metadata. 489 (3) Protocol Type (8 bits) - IP Header protocol type. This is copied 490 from the IP header. 492 (4) Flags (8 bits) 493 0: R - Reserved. MUST be initialized to 0 on transmission and 494 ignored on receipt. 496 1: R - Reserved. MUST be initialized to 0 on transmission and 497 ignored on receipt. 499 2: R - Reserved. MUST be initialized to 0 on transmission and 500 ignored on receipt. 502 3: MF - Metadata Fragment. Indicates the presence of the 503 optional metadata fragment header. This header is inserted and 504 initialized by the initiator node. If the MF bit is set, nodes 505 in the path MAY perform fragmentation of metadata stack if the 506 current length exceeds the maximum length. 508 4: TS - Tail Stamp. Indicates the IFA zone is requiring tail 509 stamping of metadata. 511 5: I - Inband. Indicates this is live traffic. Strip and 512 forward MUST be performed by the terminator node if this bit is 513 set. 515 6: TA - Turn Around. Indicates that the IFA packet needs to be 516 turned around at the terminating node of the IFA zone and sent 517 back to source IP address. This bit MAY be used for probe 518 packets where probes are collection bidirectional information in 519 the network. This is same as echo request and echo reply. A 520 packet MAY be generated with TA bit set and collects metadata in 521 one direction and after it is turned around by the terminating 522 function node, collects metadata in the reverse direction. 524 7: C - Checksum - Indicates the presence of the optional 525 checksum header. The checksum MUST be computed and updated for 526 the IFA header and metadata at each node that modified the 527 header and/or metadata. A node MAY perform checksum validation 528 before updating the checksum. 530 (5) Max Length (8 bits) - Specifies the maximum allowed length of the 531 metadata stack in multiples of 4 octets. This field is initialized 532 by the initiator node. Each node in the path MUST compare the 533 current length with the max length, and if the current length equals 534 or exceeds the max length, the transit nodes MUST stop inserting 535 metadata. 537 3.4.1. IFA Metadata Header 539 The IFA metadata header is always present. 541 0 1 2 3 542 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 544 | Request Vector| Action Vector | Hop Limit | Current Length| 545 | |L|C|R|R|R|R|R|R| | | 546 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 548 Figure 3: IFA Metadata Header Format 550 Request Vector (8 bits) - This vector specifies the presence of 551 fields as specified by GNS. Fields are always 4-octet aligned. This 552 field can be made extensible by defining a new GNS for an IFA zone. 554 Action Vector (8 bits) - This vector specifies node-local or end-to- 555 end action on the IFA packets. 557 0: L - Loss. Loss bit to measure packet loss. 559 1: C - Color. Color bit to mark the packet. 561 2: R - Reserved. MUST be initialized to 0 on transmission and 562 ignored on receipt. 564 3: R - Reserved. MUST be initialized to 0 on transmission and 565 ignored on receipt. 567 4: R - Reserved. MUST be initialized to 0 on transmission and 568 ignored on receipt. 570 5: R - Reserved. MUST be initialized to 0 on transmission and 571 ignored on receipt. 573 6: R - Reserved. MUST be initialized to 0 on transmission and 574 ignored on receipt. 576 7: R - Reserved. MUST be initialized to 0 on transmission and 577 ignored on receipt. 579 Hop Limit (8 bits) - Specifies the maximum allowed hops in an IFA 580 zone. This field is initialized by the initiator node. The hop 581 limit MUST be decremented at each hop. If the incoming hop limit is 582 0, current nodes MUST NOT insert metadata. A value of 0xFF means 583 that the Hop limit check MUST be ignored. 585 Current Length (8 bits) - Specifies the current length of the 586 metadata in multiples of 4 octets. 588 3.4.2. IFA Checksum Header 590 The IFA checksum header is optional. Presence of the checksum header 591 is indicated by the C bit in the flags field of the IFA header. 593 0 1 2 3 594 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 595 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 596 | Checksum | Reserved | 597 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 599 Figure 4: IFA Checksum Header Format 601 Checksum (16 bits) - The checksum covers the IFA header and metadata 602 stack. Initiator function node MAY compute the full checksum 603 including IFA header and metadata. Other nodes MAY compute delta 604 checksum for the inserted/deleted metadata. 606 Reserved (16 bits) - Reserved. MUST be initialized to 0 on 607 transmission and ignored on receipt. 609 3.4.3. IFA Metadata Fragmentation (MF) Header 611 The IFA metadata fragmentation (MF) header is optional. Presence of 612 the fragmentation header is indicated by the MF bit in the flags 613 field of the IFA header. 615 0 1 2 3 616 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 617 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 618 | Packet ID | MF ID |L| 619 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 621 Figure 5: IFA MF Header Format 623 Packet ID (26 bits) - Packet identification value generated by the 624 initiator node. This value is node scoped. 626 Metadata Fragment ID (5 bits) - The initiator MUST initialize this 627 value to 0. A node performing metadata fragmentation function MUST 628 increment the value by 1. 630 L (1 bit) - This bit is set by the node creating the last metadata 631 fragment. This will ALWAYS be the terminating function node. If 632 incoming hop limit is "0", terminating function node will still 633 generate copy/report of the packet and MUST set L bit. Collector 634 MUST implement mechanism to recover from lost packets/reports with L 635 bit set. 637 The MF header is a fixed overhead of 4 octets per packet. A network 638 operator MUST identify the need for using IFA metadata fragmentation. 639 The following network conditions can be considered: 641 - If an IFA packet may exceed the link or path MTU of the flow path 643 - If there are large number of hops in a flow path and MAY trigger 644 link or path MTU breach 646 - If the length of metadata creates excessive overhead for 647 terminating function node to delete the metadata. 649 - If each hop needs to generate its own IFA report (postcard 650 mechanism) 652 With 26 bits of packet id, a maximum datagram lifetime (MDL) of 3 653 seconds, and an average Internet mix (IMIX) packet size of 512 bytes, 654 we get 183.25 Gbps of IFA traffic bit rate per node before the packet 655 identifier wraps around. The collector can use [device id, packet 656 id, MF id, L] to rebuild the fragmented packet. 658 5 bits of MF id will support 32 metadata fragments. 660 3.5. IFA Metadata 662 The IFA metadata is the information inserted by each hop after the 663 IFA header. The IFA metadata can be inserted at the following 664 offsets: 666 - Payload Stamping: Immediately after the layer 4 header. This is the 667 default setting. 668 - Tail Stamping: After the end of the packet. This is controlled by 669 the TS bit in the flags field of the IFA header. 671 0 1 2 3 672 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 673 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 674 | LNS | Device ID | 675 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 676 | | 677 | | 678 ~ LNS/GNS defined metadata (contd) ~ 679 . . 680 . . 681 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 683 Figure 6: IFA Metadata Format 685 The IFA metadata header contains a set fields as defined by the name 686 space identifier. Two types of name space identifiers are proposed. 688 3.5.1. Global Name Space (GNS) Identifier 690 A Global Name Space (GNS) is specified in the IFA header by the 691 initiator node. The scope of a GNS is an IFA zone. All networking 692 elements in an IFA zone MUST insert metadata as per the GNS ID 693 specified in the IFA header. This is defined as the "Uniform Mode" 694 of deployment. 696 A GNS value of 0xF indicates that metadata in an IFA zone is defined 697 by the LNS of each hop. 699 The advantage of using the uniform mode is having a simple and 700 uniform metadata stack. This means less load on a collector for 701 parsing. 703 The disadvantage is that metadata fields are supported based on the 704 least capable networking element in the IFA zone. 706 3.5.2. Local Name Space (LNS) Identifier 708 A Local Name Space (LNS) is specified in the metadata header. A GNS 709 value of 0xF in the IFA header indicates the presence of an LNS. 710 This is defined as the "Non-uniform Mode" of deployment. 712 A switch pipeline MUST parse the GNS field in the IFA header. The 713 parsing result will dictate the name space ID that the hop needs to 714 comply with. 716 The advantage of using the non-uniform mode is having a flexible 717 metadata stack. This allows each hop to include the most relevant 718 data for that hop. 720 The disadvantage is more complex parsing by a collector. 722 3.5.3. Device ID 724 A 28-bit unique identifier for the device inserting the metadata. If 725 a GNS other than 0xF is present, then the device ID can be expanded 726 to a 32 bit value. This is to support including an IPv4 loopback 727 address as a Device ID. 729 3.6. IFA Network Overhead 731 A common problem associated with inserting metadata on a per packet 732 per flow basis is the amount of traffic overhead on the network. IFA 733 2 is defined to minimize the overhead on the network. 735 IFA Base Header : 4 octets 736 IFA Metadata Header : 4 octets 737 IFA Checksum Header : 4 octets 738 IFA Fragmentation Header : 4 octets 740 Minimum Overhead: 741 IFA header : 4 octets 743 IFA Metadata Header : 4 octets 745 Total Min Overhead : 8 octets per packet 747 3.7. IFA Analytics 749 There are two kinds of actions considered in this proposal. 751 (1) Action Bit MAP in IFA Header - This is encoded in the IFA 752 header. Each node in the path MAY use the action bitmap to 753 insert or not insert the metadata based on exceeding a locally- 754 specified threshold. Not inserting the metadata is indicated by 755 setting the field value to -1 (all 1s). 757 (2) Terminating Node Actions - A terminating node may decide to 758 perform threshold or other actions on the set of metadata in the 759 packet. This information is not encoded in the IFA header. 761 3.8. IFA Packet Format 763 The IFA header is treated as a layer 3 extension header. IFA header 764 and metadata stack length is reflected in IP total length field. 765 IPv6 extension headers are ordered. The IFA header MUST be the last 766 extension header in the IPv6 extension header chain. Similarly in 767 case of IPv4 AH/ESP/WESP extension headers, IFA header MUST be the 768 last extension header. 770 0 1 2 3 771 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 772 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 773 ~ ~ 774 | IP Header | 775 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 776 ~ ~ 777 | IFA Header | 778 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 779 ~ ~ 780 | Layer 4 Header | 781 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 782 ~ ~ 783 | IFA Metadata Header | 784 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 785 ~ ~ 786 | IFA Metadata Stack | 787 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 788 ~ ~ 789 | Payload | 790 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 791 Figure 8 IFA Packet Format 793 3.8.1. IFA Packet Format with TS Flag Set 795 In case the Tail Stamp flag is set in the IFA header, the IFA 796 metadata header and metadata stack are inserted at the end of the 797 packet just before the FCS. Each node inserts metadata at the bottom 798 of IFA metadata stack. 800 One of the key advantages of using TS is to support legacy devices 801 and/or appliances that need to look at the layer 5 data. The IP 802 length and IP header checksum are updated at each hop inserting 803 metadata. This is the same as without the TS flag. 805 0 1 2 3 806 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 807 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 808 ~ ~ 809 | IP Header | 810 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 811 ~ ~ 812 | IFA Header | 813 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 814 ~ ~ 815 | Layer 4 | 816 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 817 ~ ~ 818 | Payload | 819 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 820 ~ ~ 821 | IFA Metadata Stack | 822 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 823 ~ ~ 824 | IFA Metadata Header | 825 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 826 | FCS | 827 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 829 Figure 7: IFA Packet Format with TS 3.8.1 TCP/UDP Packet 831 0 1 2 3 832 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 833 IPv4 Header: 834 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 835 |Version| IHL |Type of Service| Total Length | 836 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 837 | Identification |Flags| Fragment Offset | 838 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 839 | Time to Live | Protocol = IFA| Header Checksum | 840 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 841 | Source IPv4 Address | 842 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 843 | Destination IPv4 Address | 844 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 845 or 846 IPv6 Header: 847 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 848 |Version| Traffic Class | Flow Label | 849 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 850 | Payload Length |Next Hdr = IFA | Hop Limit | 851 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 852 | | 853 + + 854 | | 855 + Source IPv6 Address + 856 | | 857 + + 858 | | 859 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 860 | | 861 + + 862 | | 863 + Destination IPv6 Address + 864 | | 865 + + 866 | | 867 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 869 IFA Header: 870 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 871 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 872 | | | | | | |F|S| |A| | | 873 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 875 TCP Header: 876 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 877 | Source Port | Destination Port | 878 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 880 | Sequence Number | 881 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 882 | Acknowledgment Number | 883 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 884 | Data | |U|A|P|R|S|F| | 885 | Offset| Reserved |R|C|S|S|Y|I| Window | 886 | | |G|K|H|T|N|N| | 887 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 888 | Checksum | Urgent Pointer | 889 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 890 | Options | Padding | 891 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 893 IFA Metadata Header: 894 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 895 ~ ~ 896 | IFA Metadata Header | 897 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 899 IFA Metadata Stack: 900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 901 ~ ~ 902 | IFA Metadata Stack | 903 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 905 TCP Payload: 906 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 907 ~ ~ 908 | data | 909 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 911 Figure 8: TCP/UDP IFA Packet Format 913 3.8.2. VxLAN Packet 915 0 1 2 3 916 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 917 IPv4 Header: 918 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 919 |Version| IHL |Type of Service| Total Length | 920 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 921 | Identification |Flags| Fragment Offset | 922 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 923 | Time to Live | Protocol = IFA| Header Checksum | 924 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 925 | Source IPv4 Address | 926 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 927 | Destination IPv4 Address | 928 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 929 or 930 IPv6 Header: 931 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 932 |Version| Traffic Class | Flow Label | 933 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 934 | Payload Length |Next Hdr = IFA | Hop Limit | 935 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 936 | | 937 + + 938 | | 939 + Source IPv4 Address + 940 | | 941 + + 942 | | 943 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 944 | | 945 + + 946 | | 947 + Destination IPv6 Address + 948 | | 949 + + 950 | | 951 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 953 IFA Header: 954 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 955 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 956 | | | | | | |F|S| |A| | | 957 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 959 Outer UDP Header: 960 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 961 | Source Port | Dest Port = VXLAN Port | 962 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 964 | UDP Length | UDP Checksum | 965 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 967 IFA Metadata Header: 968 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 969 ~ ~ 970 | IFA Metadata Header | 971 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 973 IFA Metadata Stack: 974 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 975 ~ ~ 976 | IFA Metadata Stack | 977 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 979 VXLAN Header: 980 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 981 |R|R|R|R|I|R|R|R| Reserved | 982 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 983 | VXLAN Network Identifier (VNI) | Reserved | 984 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 985 ~ ~ 986 | data | 987 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 989 Figure 9: VxLAN IFA Packet Format 991 3.8.3. GRE Packet 993 0 1 2 3 994 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 995 IPv4 Header: 996 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 997 |Version| IHL |Type of Service| Total Length | 998 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 999 | Identification |Flags| Fragment Offset | 1000 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1001 | Time to Live | Protocol = IFA| Header Checksum | 1002 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1003 | Source IPv4 Address | 1004 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1005 | Destination IPv4 Address | 1006 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1007 or 1008 IPv6 Header: 1009 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1010 |Version| Traffic Class | Flow Label | 1011 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1012 | Payload Length |Next Hdr = IFA | Hop Limit | 1013 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1014 | | 1015 + + 1016 | | 1017 + Source IPv6 Address + 1018 | | 1019 + + 1020 | | 1021 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1022 | | 1023 + + 1024 | | 1025 + Destination IPv6 Address + 1026 | | 1027 + + 1028 | | 1029 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1031 IFA Header: 1032 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1033 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 1034 | | | | | | |F|S| |A| | | 1035 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1037 GRE Header: 1038 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1039 |C| Reserved0 | Ver | Protocol Type | 1040 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1042 | Checksum (optional) | Reserved1 (Optional) | 1043 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1045 IFA Metadata Header: 1046 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1047 ~ ~ 1048 | IFA Metadata Header | 1049 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1051 IFA Metadata Stack: 1052 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1053 ~ ~ 1054 | IFA Metadata Stack | 1055 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1057 GRE Payload: 1058 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1059 ~ ~ 1060 | data | 1061 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1062 Figure 12 GRE IFA Packet Format 1064 3.8.4. Geneve Packet 1066 0 1 2 3 1067 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1068 IPv4 Header: 1069 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1070 |Version| IHL |Type of Service| Total Length | 1071 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1072 | Identification |Flags| Fragment Offset | 1073 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1074 | Time to Live | Protocol = IFA| Header Checksum | 1075 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1076 | Source IPv4 Address | 1077 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1078 | Destination IPv4 Address | 1079 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1080 or 1081 IPv6 Header: 1082 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1083 |Version| Traffic Class | Flow Label | 1084 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1085 | Payload Length |Next Hdr = IFA | Hop Limit | 1086 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1087 | | 1088 + + 1089 | | 1090 + Source IPv6 Address + 1091 | | 1092 + + 1093 | | 1094 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1095 | | 1096 + + 1097 | | 1098 + Destination IPv6 Address + 1099 | | 1100 + + 1101 | | 1102 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1104 IFA Header: 1105 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1106 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 1107 | | | | | | |F|S| |A| | | 1108 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1110 Outer UDP Header: 1111 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1112 | Source Port = xxxx | Dest Port = Geneve Port | 1113 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1115 | UDP Length | UDP Checksum | 1116 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1118 IFA Metadata Header: 1119 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1120 ~ ~ 1121 | IFA Metadata Header | 1122 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1124 IFA Metadata Stack: 1125 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1126 ~ ~ 1127 | IFA Metadata Stack | 1128 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1129 Geneve Header: 1130 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1131 |Ver| Opt Len |O|C| Rsvd. | Protocol Type | 1132 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1133 | Virtual Network Identifier (VNI) | Reserved | 1134 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1135 ~ ~ 1136 | Variable Length Options | 1137 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1138 ~ ~ 1139 | data | 1140 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1142 Figure 10: Geneve IFA Packet Format 1144 3.8.5. IPinIP Packet 1146 0 1 2 3 1147 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1148 IPv4 Header: 1149 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1150 |Version| IHL |Type of Service| Total Length | 1151 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1152 | Identification |Flags| Fragment Offset | 1153 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1154 | Time to Live | Protocol = IFA| Header Checksum | 1155 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1156 | Source IPv4 Address | 1157 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1158 | Destination IPv4 Address | 1159 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1160 or 1161 IPv6 Header: 1162 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1163 |Version| Traffic Class | Flow Label | 1164 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1165 | Payload Length |Next Hdr = IFA | Hop Limit | 1166 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1167 | | 1168 + + 1169 | | 1170 + Source IPv6 Address + 1171 | | 1172 + + 1173 | | 1174 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1175 | | 1176 + + 1177 | | 1178 + Destination IPv6 Address + 1179 | | 1180 + + 1181 | | 1182 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1184 IFA Header: 1185 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1186 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 1187 | | | | | | |F|S| |A| | | 1188 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1190 Inner IP Header: 1191 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1192 ~ ~ 1193 | IPv4 or IPv6 Header | 1194 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1196 IFA Metadata Header: 1197 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1198 ~ ~ 1199 | IFA Metadata Header | 1200 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1202 IFA Metadata Stack: 1203 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1204 ~ ~ 1205 | IFA Metadata Stack | 1206 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1208 Inner IP L4 and Payload: 1209 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1210 ~ ~ 1211 | data | 1212 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1214 Figure 11: IPinIP IFA Packet Format 1216 3.8.6. IPv6 Extension Headers with IFA 1218 The IFA header is always the last extension header in the IPv6 1219 extension header chain. The last extension header's next header 1220 field is stored in the IFA next header field and is replaced by the 1221 IFA protocol value. 1223 IPv6 Header: 1224 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1225 |Version| Traffic Class | Flow Label | 1226 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1227 | Payload Length | Next Hdr = 43 | Hop Limit | 1228 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1229 ~ Source IPv6 Address ~ 1230 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1231 ~ Destination IPv6 Address ~ 1232 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1233 | Next Hdr = 60 | Extension Header | 1234 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1235 | Next Hdr = 44 | Extension Header | 1236 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1237 | Next Hdr = IFA| Extension Header | 1238 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1240 IFA Header: 1241 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1242 | Ver=2 | GNS |NextHdr = IP_xx|R|R|R|M|T|I|T|C| Max Length | 1243 | | | | | | |F|S| |A| | | 1244 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1246 Layer 4 Header: 1247 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1248 ~ ~ 1249 | Layer 4 | 1250 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1252 IFA Metadata Header: 1253 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1254 ~ ~ 1255 | IFA Metadata Header | 1256 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1258 IFA Metadata Stack: 1259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1260 ~ ~ 1261 | IFA Metadata Stack | 1262 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1264 UDP/TCP Payload: 1265 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1266 ~ ~ 1267 | data | 1268 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1270 Figure 12: IPv6 Extension Header with IFA Packet Format 1272 3.8.7. IP AH/ESP/WESP Packet 1274 An AH, ESP, or WESP header is treated as a chained header in IPv4. 1275 The IPv4 protocol field is replaced by the AH/ESP/WESP protocol value 1276 and the IPv4 protocol field value is stored in the AH/ESP/WESP next 1277 header field. 1279 The IFA header is ALWAYS placed as the last header in a header chain. 1280 In case of ESP/WESP where layer 4 and payload is encrypted, IFA 1281 metadata stack is placed immediately after IFA header. 1283 IPv4: AH Transport Mode 1284 -------------------------------------------------------------- 1285 | IP hdr | AH |IFA hdr|TCP|IFA MD Hdr| IFA MD stack| Data | 1286 -------------------------------------------------------------- 1287 |<------->| |<----->| |<---------------------->| | 1288 |<------------- mutable field processing ---------->| | 1289 | |<-->| |<->| |<------>| 1290 | |<------------- immutable fields ----------------->| 1291 |<-----------authenticated except for mutable fields-------->| 1293 IPv6: AH Transport Mode 1294 ---------------------------------------------------------------- 1295 | |hop-by-hop, dest, | |IFA| |IFA MD|IFA MD| | 1296 |IP hdr|routing, fragment | AH |hdr|TCP| hdr | stack| Data | 1297 ---------------------------------------------------------------- 1298 |<----------------------->| |<->| |<----------->| | 1299 |<---------- mutable field processing -------------->| | 1300 | |<-->| |<->| |<------->| 1301 | |<--------immutable fields---------->| 1302 |<-------- authenticated except for mutable fields ----------->| 1304 Figure 13: IP AH Transport Mode IFA Packet Format 1306 IPv4: AH Tunnel Mode 1307 ---------------------------------------------------------------- 1308 | | |IFA| Orig IP |IFA MD| IFA MD | | | 1309 |new IP header| AH |hdr| hdr | hdr | stack |TCP| Data | 1310 ---------------------------------------------------------------- 1311 |<----------->| |<->| |<------------->| | | 1312 |<------------- mutable field processing ------->| | | 1313 | |<-->| |<------->| |<----------->| 1314 | |<------------- immutable fields --------------->| 1315 |<--authenticated except for mutable fields in the new IP hdr->| 1317 IPv6: AH Tunnel Mode 1318 ---------------------------------------------------------------- 1319 |new IP header| |IFA| Orig IP |IFA MD| IFA MD | | | 1320 | +extn hdrs | AH |hdr| hdr | hdr | stack |TCP| Data | 1321 ---------------------------------------------------------------- 1322 |<----------->| |<->| |<------------->| | | 1323 |<------------- mutable field processing ------->| | | 1324 | |<-->| |<------->| |<----------->| 1325 | |<------------- immutable fields --------------->| 1326 |<--authenticated except for mutable fields in the new IP hdr->| 1328 Figure 14: IP AH Tunnel Mode IFA Packet Format IPv4: ESP Transport 1329 Mode --------------------------------------------------------------| 1330 IP hdr | ESP |IFA hdr|IFA MD Hdr| IFA MD stack|TCP| Data | ---------- 1331 ----------------------------------------------------|<------->| |<--- 1332 --------------------------->| | |<------------ mutable field 1333 processing -------->| | | |<--->| |<--------->| | |<-------------- 1334 immutable fields ---------------->| |<------------ encrypted except 1335 for mutable fields --------->| 1337 IPv6: ESP Transport Mode 1338 ---------------------------------------------------------------- 1339 | |hop-by-hop, dest, | |IFA|IFA MD|IFA MD| | | 1340 |IP hdr|routing, fragment | ESP|hdr| hdr | stack|TCP| Data | 1341 ---------------------------------------------------------------- 1342 |<----------------------->| |<--------------->| | 1343 |<--------- mutable field processing ----------->| | 1344 | |<-->| |<----------->| 1345 | |<--------immutable fields---------->| 1346 |<---------- encrypted except for mutable fields ------------->| 1348 Figure 15: IP ESP Transport Mode IFA Packet Format 1350 IPv4: ESP Tunnel Mode 1351 ---------------------------------------------------------------- 1352 | | |IFA|IFA MD| IFA MD | Orig IP | | | 1353 |new IP header| AH |hdr| hdr | stack | hdr |TCP| Data | 1354 ---------------------------------------------------------------- 1355 |<----------->| |<----------------->| | 1356 |<----- mutable field processing ----->| | 1357 | |<-->| |<--------------------->| 1358 | |<------------- immutable fields --------------->| 1359 |<-- encrypted except for mutable fields in the new IP hdr --->| 1361 IPv4: ESP Tunnel Mode 1362 ---------------------------------------------------------------- 1363 |new IP header| |IFA|IFA MD| IFA MD | Orig IP | | | 1364 |+extn headers| AH |hdr| hdr | stack | hdr |TCP| Data | 1365 ---------------------------------------------------------------- 1366 |<----------->| |<----------------->| | 1367 |<----- mutable field processing ----->| | 1368 | |<-->| |<--------------------->| 1369 | |<------------- immutable fields --------------->| 1370 |<-- encrypted except for mutable fields in the new IP hdr --->| 1372 Figure 16: IP AH Tunnel Mode IFA Packet Format 1374 3.9. IFA Load Balancing 1376 IFA changes the IP protocol field value to the IFA protocol number. 1377 The IP protocol field value is included in the hash computation. 1378 This will impact load balancing of flows. 1380 The forwarding plane MUST support reading the IP protocol field value 1381 stored in the IFA NextHDR field for hash computation. 1383 The layer 4 header is available at a fixed offset from the IFA header 1384 and is available for hash computation. 1386 Hash computation based on the layer 4 payload will depend on the 1387 length of the IFA metadata stack present. 1389 4. Interoperability Considerations 1391 Version 2 of this protocol specification is not backward compatible 1392 with version 1. 1394 5. Security Considerations 1396 A successful attack on an OAM protocol can prevent the detection of 1397 failures or anomalies, or create a false illusion of nonexistent 1398 ones. 1400 The metadata elements of IFA can be used by attackers to collect 1401 information about the network hops. 1403 Adding IFA headers or adding to IFA metadata can be used to consume 1404 resources within the path being monitored or by a collector. 1406 Adding IFA headers or adding to IFA metadata can be used to force 1407 exceeding the MTU for the path being monitored resulting in 1408 fragmentation and/or packet drops. 1410 IFA is expected to be deployed within controlled network domains, 1411 containing attacks to that controlled domain. Limiting or preventing 1412 monitoring or attacks using IFA requires limiting or preventing 1413 unauthorized access to the domain in which IFA is to be used, and 1414 preventing leaking IFA metadata beyond the controlled domain. 1416 6. References 1418 6.1. Normative References 1420 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1421 Requirement Levels", BCP 14, RFC 2119, 1422 DOI 10.17487/RFC2119, March 1997, 1423 . 1425 6.2. Informative References 1427 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 1428 DOI 10.17487/RFC0791, September 1981, 1429 . 1431 [RFC6864] Touch, J., "Updated Specification of the IPv4 ID Field", 1432 RFC 6864, DOI 10.17487/RFC6864, February 2013, 1433 . 1435 [RFC3514] Bellovin, S., "The Security Flag in the IPv4 Header", 1436 RFC 3514, DOI 10.17487/RFC3514, April 2003, 1437 . 1439 [I-D.kumar-ifa] 1440 Kumar, J., Anubolu, S., He, Z., Manur, R., Cai, D., Ou, 1441 H., Yizhou, L., and S. Suwei, "Inband Flow Analyzer", 1442 draft-kumar-ifa-00 (work in progress), March 2018. 1444 Appendix A. 1446 Appendix A is for informational purposes only. The following options 1447 were considered for the IFA protocol. 1449 A.1. Probe Marker 1451 One of the challenges of using probe signatures in an IFA header is a 1452 false positive. 1454 The IFA version 2 header takes care of large header sizes and 1455 identification based on probe markers. Probe markers can cause false 1456 positives if there is a match on the first 64 bits of the layer 4 1457 payload. 1459 This approach is not a preferred approach, but is supported by this 1460 draft as a version 1.0 header. 1462 A.2. DSCP 1464 [RFC791] EXP/LU Pool 3 can be used for identifying IFA packets. CU 1465 bits can be used for identifying IFA packets. 1467 The problem with using TOS bits is that they are pervasively used in 1468 the network deployment and are responsible for affecting the 1469 forwarding decision. 1471 This approach is not supported or recommended by this draft. 1473 A.3. IP Options 1475 [RFC791] The IP options provide for control functions that are needed 1476 or useful in some situations but unnecessary for the most common 1477 communications. The IP options include provisions for timestamps, 1478 security, and special routing. 1480 There are various problems with this approach. 1481 (1) The IPv4 header size can become arbitrarily large with the 1482 presence of options. 1484 (2) A switch pipeline typically handles IP option packets as 1485 exception path processing and punts them to a host CPU. (3) IP 1486 options make the construction of firewalls cumbersome, and are 1487 typically disallowed or stripped at the perimeter of enterprise 1488 networks by firewalls. 1490 This approach is not supported or recommended by this draft. 1492 A.4. IPv4 Identification or Reserved Flag 1494 [RFC6864] [RFC3514] Another suggestion is to use the IPv4 1495 identification field or reserved flag. This suggestion is also 1496 discarded and not supported for the following reasons: 1498 [RFC6864] prohibits usage of id field for any other purposes. 1500 [RFC3514] prohibits using flags bit 0 for security reasons. 1502 Authors' Addresses 1504 Jai Kumar 1505 Broadcom Inc. 1506 Email: jai.kumar@broadcom.com 1508 Surendra Anubolu 1509 Broadcom Inc. 1510 Email: surendra.anubolu@broadcom.com 1512 John Lemon 1513 Broadcom Inc. 1514 Email: john.lemon@broadcom.com 1516 Rajeev Manur 1517 Broadcom Inc. 1518 Email: rajeev.manur@broadcom.com 1520 Hugh Holbrook 1521 Arista Networks 1522 Email: holbrook@arista.com 1524 Anoop Ghanwani 1525 Dell EMC 1526 Email: anoop.ghanwani@dell.com 1528 Dezhong Cai 1529 AliBaba Inc. 1530 Email: d.cai@alibaba-inc.com 1532 Heidi Ou 1533 AliBaba Inc. 1534 Email: heidi.ou@alibaba-inc.com 1536 Yizhou Li 1537 Huawei Technologies 1538 EMail: liyizhou@huawei.com 1540 Xiaojun Wang 1541 Fujian Ruijie Networks co.,ltd. 1542 EMail: wxj@ruijie.com.cn