idnits 2.17.1 draft-ietf-nvo3-vxlan-gpe-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 10, 2019) is 1836 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 1981 (Obsoleted by RFC 8201) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 6830 (Obsoleted by RFC 9300, RFC 9301) == Outdated reference: A later version (-05) exists of draft-brockners-ippm-ioam-vxlan-gpe-01 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-11 == Outdated reference: A later version (-02) exists of draft-lemon-vxlan-lisp-gpe-gbp-01 Summary: 3 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group F. Maino, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational L. Kreeger, Ed. 5 Expires: October 12, 2019 6 U. Elzur, Ed. 7 Intel 8 April 10, 2019 10 Generic Protocol Extension for VXLAN 11 draft-ietf-nvo3-vxlan-gpe-07 13 Abstract 15 This draft describes extending Virtual eXtensible Local Area Network 16 (VXLAN), via changes to the VXLAN header, with three new 17 capabilities: support for multi-protocol encapsulation, operations, 18 administration and management (OAM) signaling and explicit 19 versioning. 21 Requirements Language 23 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 24 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 25 document are to be interpreted as described in [RFC2119]. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at http://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on October 12, 2019. 44 Copyright Notice 46 Copyright (c) 2019 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 62 2. VXLAN Without Protocol Extension . . . . . . . . . . . . . . 3 63 3. Generic Protocol Extension for VXLAN (VXLAN GPE) . . . . . . 4 64 3.1. VXLAN GPE Header . . . . . . . . . . . . . . . . . . . . 4 65 3.2. Multi Protocol Support . . . . . . . . . . . . . . . . . 5 66 3.3. Replicated BUM Traffic . . . . . . . . . . . . . . . . . 7 67 3.4. OAM Support . . . . . . . . . . . . . . . . . . . . . . . 7 68 3.5. Version Bits . . . . . . . . . . . . . . . . . . . . . . 8 69 4. Outer Encapsulations . . . . . . . . . . . . . . . . . . . . 8 70 4.1. Inner VLAN Tag Handling . . . . . . . . . . . . . . . . . 12 71 4.2. Fragmentation Considerations . . . . . . . . . . . . . . 12 72 5. Backward Compatibility . . . . . . . . . . . . . . . . . . . 12 73 5.1. VXLAN VTEP to VXLAN GPE VTEP . . . . . . . . . . . . . . 12 74 5.2. VXLAN GPE VTEP to VXLAN VTEP . . . . . . . . . . . . . . 12 75 5.3. VXLAN GPE UDP Ports . . . . . . . . . . . . . . . . . . . 13 76 5.4. VXLAN GPE and Encapsulated IP Header Fields . . . . . . . 13 77 6. VXLAN GPE Examples . . . . . . . . . . . . . . . . . . . . . 13 78 7. Security Considerations . . . . . . . . . . . . . . . . . . . 14 79 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 14 80 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 82 10.1. UDP Port . . . . . . . . . . . . . . . . . . . . . . . . 15 83 10.2. VXLAN GPE Next Protocol . . . . . . . . . . . . . . . . 16 84 10.3. VXLAN GPE Flag and Reserved Bits . . . . . . . . . . . . 16 85 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 86 11.1. Normative References . . . . . . . . . . . . . . . . . . 17 87 11.2. Informative References . . . . . . . . . . . . . . . . . 18 88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 90 1. Introduction 92 Virtual eXtensible Local Area Network VXLAN [RFC7348] defines an 93 encapsulation format that encapsulates Ethernet frames in an outer 94 UDP/IP transport. As data centers evolve, the need to carry other 95 protocols encapsulated in an IP packet is required, as well as the 96 need to provide increased visibility and diagnostic capabilities 97 within the overlay. The VXLAN header does not specify the protocol 98 being encapsulated and therefore is currently limited to 99 encapsulating only Ethernet frame payload, nor does it provide the 100 ability to define OAM protocols. In addition, [RFC6335] requires 101 that new transports not use transport layer port numbers to identify 102 tunnel payload, rather it encourages encapsulations to use their own 103 identifiers for this purpose. VXLAN GPE is intended to extend the 104 existing VXLAN protocol to provide protocol typing, OAM, and 105 versioning capabilities. 107 The Version and OAM bits are introduced in Section 3, and the choice 108 of location for these fields is driven by minimizing the impact on 109 existing deployed hardware. 111 In order to facilitate deployments of VXLAN GPE with hardware 112 currently deployed to support VXLAN, changes from legacy VXLAN have 113 been kept to a minimum. Section 5 provides a detailed discussion 114 about how VXLAN GPE addresses the requirement for backward 115 compatibility with VXLAN. 117 2. VXLAN Without Protocol Extension 119 VXLAN provides a method of creating multi-tenant overlay networks by 120 encapsulating packets in IP/UDP along with a header containing a 121 network identifier which is used to isolate tenant traffic in each 122 overlay network from each other. This allows the overlay networks to 123 run over an existing IP network. 125 Through this encapsulation, VXLAN creates stateless tunnels between 126 VXLAN Tunnel End Points (VTEPs) which are responsible for adding/ 127 removing the IP/UDP/VXLAN headers and providing tenant traffic 128 isolation based on the VXLAN Network Identifier (VNI). Tenant 129 systems are unaware that their networking service is being provided 130 by an overlay. 132 When encapsulating packets, a VTEP must know the IP address of the 133 proper remote VTEP at the far end of the tunnel that can deliver the 134 inner packet to the Tenant System corresponding to the inner 135 destination address. In the case of tenant multicast or broadcast, 136 the outer IP address may be an IP multicast group address, or the 137 VTEP may replicate the packet and send it to all known VTEPs. If 138 multicast is used in the underlay network to send encapsulated 139 packets to remote VTEPs, Any Source Multicast is used and each VTEP 140 serving a particular VNI must perform a (*, G) join to the same group 141 IP address. 143 Inner to outer address mapping can be determined in two ways. One is 144 source based learning in the data plane, and the other is 145 distribution via a control plane. 147 Source based learning requires a receiving VTEP to create an inner to 148 outer address mapping by gleaning the information from the received 149 packets by correlating the inner source address to the outer source 150 IP address. When a mapping does not exist, a VTEP forwards the 151 packets to all remote VTEPs participating in the VNI by using IP 152 multicast in the IP underlay network. Each VTEP must be configured 153 with the IP multicast address to use for each VNI. How this occurs 154 is out of scope. 156 The control plane used to distribute inner to outer mappings is also 157 out of scope. It could use a centralized authority or be 158 distributed, or use a hybrid. 160 The VXLAN Network Identifier (VNI) provides scoping for the addresses 161 in the header of the encapsulated PDU. If the encapsulated packet is 162 an Ethernet frame, this means the Ethernet MAC addresses are only 163 unique within a given VNI and may overlap with MAC addresses within a 164 different VNI. If the encapsulated packet is an IP packet, this 165 means the IP addresses are only unique within that VNI. 167 0 1 2 3 168 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 169 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 170 |R|R|R|R|I|R|R|R| Reserved | 171 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 172 | VXLAN Network Identifier (VNI) | Reserved | 173 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 175 Figure 1: VXLAN Header 177 3. Generic Protocol Extension for VXLAN (VXLAN GPE) 179 3.1. VXLAN GPE Header 181 0 1 2 3 182 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 183 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 184 |R|R|Ver|I|P|B|O| Reserved |Next Protocol | 185 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 186 | VXLAN Network Identifier (VNI) | Reserved | 187 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 189 Figure 2: VXLAN GPE Header 191 Flags (8 bits): The first 8 bits of the header are the flag field. 192 The bits designated "R" above are reserved flags. These MUST be 193 set to zero on transmission and ignored on receipt. 195 Version (Ver): Indicates VXLAN GPE protocol version. The initial 196 version is 0. If a receiver does not support the version 197 indicated it MUST drop the packet. 199 Instance Bit (I bit): The I bit MUST be set to indicate a valid VNI. 201 Next Protocol Bit (P bit): The P bit is set to indicate that the 202 Next Protocol field is present. 204 BUM Traffic Bit (B bit): The B bit is set to indicate that this is 205 ingress-replicated BUM Traffic (ie, Broadcast, Unknown unicast, or 206 Multicast). 208 OAM Flag Bit (O bit): The O bit is set to indicate that the packet 209 is an OAM packet. 211 Next Protocol: This 8 bit field indicates the protocol header 212 immediately following the VXLAN GPE header. 214 VNI: This 24 bit field identifies the VXLAN overlay network the 215 inner packet belongs to. Inner packets belonging to different 216 VNIs cannot communicate with each other (unless explicitly allowed 217 by policy). 219 Reserved: Reserved fields MUST be set to zero on transmission and 220 ignored on receipt. 222 3.2. Multi Protocol Support 224 This draft defines the following two changes to the VXLAN header in 225 order to support multi-protocol encapsulation: 227 P Bit: Flag bit 5 is defined as the Next Protocol bit. The P bit 228 MUST be set to 1 to indicate the presence of the 8 bit next 229 protocol field. 231 When UDP dest port=4790, P = 0 the "Next Protocol" field must be 232 set to zero and the payload MUST be ETHERNET(L2) as defined by 233 [RFC7348]. 235 Flag bit 5 was chosen as the P bit because this flag bit is 236 currently reserved in VXLAN. 238 Next Protocol Field: The lower 8 bits of the first word are used to 239 carry a next protocol. This next protocol field contains the 240 protocol of the encapsulated payload packet. A new protocol 241 registry will be requested from IANA, see section 10.2. 243 This draft defines the following Next Protocol values: 245 0x1 : IPv4 247 0x2 : IPv6 249 0x3 : Ethernet 251 0x4 : Network Service Header [RFC8300] 253 0x5 : Multiprotocol Label Switching [RFC3031]. See 254 [I-D.ietf-idr-tunnel-encaps] for more details. 256 0x6: Unassigned. 258 0x7: virtual Broadband Network Gateway (vBNG) 259 [I-D.huang-nvo3-vxlan-gpe-extension-for-vbng]. 261 0x8 to 0x7F: Unassigned. 263 0x80: Group-Based Policy (GBP) [I-D.lemon-vxlan-lisp-gpe-gbp] 265 0x81: In-situ OAM Data (iOAM)[I-D.brockners-ippm-ioam-vxlan-gpe] 267 0x82 to 0xFF: Unassigned. 269 Next protocol values from Ox80 to 0xFF are assigned to protocols 270 encoded as generic "shim" headers. These protocols, when present, 271 MUST be encapsulated before protocols identified by next protocol 272 values from 0x0 to 0x7F. 274 Implementations that are not aware of a given shim header MUST ignore 275 the header and proceed to parse the next protocol. Shim protocols 276 MUST have the first 32 bits defined as: 278 0 1 2 3 279 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 280 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 281 | Type | Length | Reserved | Next Protocol | 282 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 283 | | 284 ~ Protocol Specific Fields ~ 285 | | 286 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 288 Figure 3: Shim Header 290 Where: 292 Type: This field MAY be used to identify different messages of this 293 protocol. 295 Length: The length, in bytes, of this protocol message not including 296 the first 4 bytes. 298 Reserved: The use of this field is reserved to the protocol defined 299 in this message. 301 Next Protocol Field: This next protocol field contains the protocol 302 of the encapsulated payload. The protocol registry will be 303 requested from IANA as per section 10.2. 305 3.3. Replicated BUM Traffic 307 Flag bit 6 is defined as the B bit. When the B bit is set to 1, the 308 packet is marked as an an ingress-replicated BUM Traffic (i.e. 309 Broadcast, Unknown unicast, or Multicast) to help egress VTEP to 310 differentiate between known and unknown unicast. The details of 311 using the B bit are out of scope for this document, but please see 312 [RFC8365] for an example in the EVPN context. As with the P-bit, bit 313 6 is currently a reserved flag in VXLAN. 315 3.4. OAM Support 317 Flag bit 7 is defined as the O bit. When the O bit is set to 1, the 318 packet is an OAM packet and OAM processing MUST occur. Other header 319 fields including Next Protocol MUST adhere to the definitions in 320 Section 3. The OAM protocol details are out of scope for this 321 document. As with the P-bit, bit 7 is currently a reserved flag in 322 VXLAN. 324 3.5. Version Bits 326 VXLAN GPE bits 2 and 3 are defined as version bits. These bits are 327 reserved in VXLAN. The version field is used to ensure backward 328 compatibility going forward with future VXLAN GPE updates. 330 The initial version for VXLAN GPE is 0. 332 4. Outer Encapsulations 334 In addition to the VXLAN GPE header, the packet is further 335 encapsulated in UDP and IP. Data centers based on Ethernet, will 336 then send this IP packet over Ethernet. 338 Outer UDP Header: 340 Destination UDP Port: IANA has assigned the value 4790 for the VXLAN 341 GPE UDP port. This well-known destination port is used when sending 342 VXLAN GPE encapsulated packets. 344 Source UDP Port: The source UDP port is used as entropy for devices 345 forwarding encapsulated packets across the underlay (ECMP for IP 346 routers, or load splitting for link aggregation by bridges). Tenant 347 traffic flows should all use the same source UDP port to lower the 348 chances of packet reordering by the underlay for a given flow. It is 349 recommended for VTEPs to generate this port number using a hash of 350 the inner packet headers. Implementations MAY use the entire 16 bit 351 source UDP port for entropy. 353 UDP Checksum: Source VTEPs MAY either calculate a valid checksum, or 354 if this is not possible, set the checksum to zero. When calculating 355 a checksum, it MUST be calculated across the entire packet (outer IP 356 header, UDP header, VXLAN GPE header and payload packet). All 357 receiving VTEPs must accept a checksum value of zero. If the 358 receiving VTEP is capable of validating the checksum, it MAY validate 359 a non-zero checksum and MUST discard the packet if the checksum is 360 determined to be invalid. 362 Outer IP Header: 364 This is the header used by the underlay network to deliver packets 365 between VTEPs. The destination IP address can be a unicast or a 366 multicast IP address. The source IP address must be the source VTEP 367 IP address which can be used to return tenant packets to the tenant 368 system source address within the inner packet header. 370 When the outer IP header is IPv4, VTEPs MUST set the DF bit. 372 Outer Ethernet Header: 374 Most data centers networks are built on Ethernet. Assuming the outer 375 IP packet is being sent across Ethernet, there will be an Ethernet 376 header used to deliver the IP packet to the next hop, which could be 377 the destination VTEP or be a router used to forward the IP packet 378 towards the destination VTEP. If VLANs are in use within the data 379 center, then this Ethernet header would also contain a VLAN tag. 381 The following figures show the entire stack of protocol headers that 382 would be seen on an Ethernet link carrying encapsulated packets from 383 a VTEP across the underlay network for both IPv4 and IPv6 based 384 underlay networks. 386 0 1 2 3 387 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 389 Outer Ethernet Header: 390 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 391 | Outer Destination MAC Address | 392 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 393 | Outer Destination MAC Address | Outer Source MAC Address | 394 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 395 | Outer Source MAC Address | 396 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 397 | Opt Ethertype = C-Tag 802.1Q | Outer VLAN Tag | 398 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 399 | Ethertype = 0x0800 | 400 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 402 Outer IPv4 Header: 403 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 404 |Version| IHL |Type of Service| Total Length | 405 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 406 | Identification |Flags| Fragment Offset | 407 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 408 | Time to Live |Protocl=17(UDP)| Header Checksum | 409 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 410 | Outer Source IPv4 Address | 411 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 412 | Outer Destination IPv4 Address | 413 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 415 Outer UDP Header: 416 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 417 | Source Port | Dest Port = 4790 | 418 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 419 | UDP Length | UDP Checksum | 420 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 422 VXLAN GPE Header: 423 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 424 |R|R|Ver|I|P|R|O| Reserved |Next Protocol | 425 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 426 | VXLAN Network Identifier (VNI) | Reserved | 427 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 429 Payload: 430 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 431 | Depends on VXLAN GPE Next Protocol field above. | 432 | Note that if the payload is Ethernet, then the original | 433 | Ethernet Frame's FCS is not included. | 434 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 436 Frame Check Sequence: 437 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 438 | New FCS (Frame Check Sequence) for Outer Ethernet Frame | 439 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 441 Figure 4: Outer Headers for VXLAN GPE over IPv4 443 0 1 2 3 444 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 446 Outer Ethernet Header: 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 | Outer Destination MAC Address | 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 | Outer Destination MAC Address | Outer Source MAC Address | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 452 | Outer Source MAC Address | 453 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 454 | Opt Ethertype = C-Tag 802.1Q | Outer VLAN Tag | 455 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 456 | Ethertype = 0x86DD | 457 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 459 Outer IPv6 Header: 460 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 461 |Version| Traffic Class | Flow Label | 462 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 463 | Payload Length | NxtHdr=17(UDP)| Hop Limit | 464 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 465 | | 466 + + 467 | | 468 + Outer Source IPv6 Address + 469 | | 470 + + 471 | | 472 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 473 | | 474 + + 475 | | 476 + Outer Destination IPv6 Address + 477 | | 478 + + 479 | | 480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 482 Outer UDP Header: 483 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 484 | Source Port | Dest Port = 4790 | 485 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 486 | UDP Length | UDP Checksum | 487 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 489 VXLAN GPE Header: 490 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 491 |R|R|Ver|I|P|R|O| Reserved |Next Protocol | 492 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 493 | VXLAN Network Identifier (VNI) | Reserved | 494 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 496 Payload: 497 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 498 | Depends on VXLAN GPE Next Protocol field above. | 499 | Note that if the payload is Ethernet, then the original | 500 | Ethernet Frame's FCS is not included. | 501 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 503 Frame Check Sequence: 504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 505 | New FCS (Frame Check Sequence) for Outer Ethernet Frame | 506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 508 Figure 5: Outer Headers for VXLAN GPE over IPv6 510 4.1. Inner VLAN Tag Handling 512 If the inner packet (as indicated by the VXLAN GPE Next Protocol 513 field) is an Ethernet frame, it is recommended that it does not 514 contain a VLAN tag. In the most common scenarios, the tenant VLAN 515 tag is translated into a VXLAN Network Identifier. In these 516 scenarios, VTEPs should never send an inner Ethernet frame with a 517 VLAN tag, and a VTEP performing decapsulation should discard any 518 inner frames received with a VLAN tag. However, if the VTEPs are 519 specifically configured to support it for a specific VXLAN Network 520 Identifier, a VTEP may support transparent transport of the inner 521 VLAN tag between all tenant systems on that VNI. The VTEP never 522 looks at the value of the inner VLAN tag, but simply passes it across 523 the underlay. 525 4.2. Fragmentation Considerations 527 VTEPs MUST never fragment an encapsulated VXLAN GPE packet, and when 528 the outer IP header is IPv4, VTEPs MUST set the DF bit in the outer 529 IPv4 header. It is recommended that the underlay network be 530 configured to carry an MTU at least large enough to accommodate the 531 added encapsulation headers. It is recommended that VTEPs perform 532 Path MTU discovery [RFC1191] [RFC1981] to determine if the underlay 533 network can carry the encapsulated payload packet. 535 5. Backward Compatibility 537 5.1. VXLAN VTEP to VXLAN GPE VTEP 539 A VXLAN VTEP conforms to VXLAN frame format and uses UDP destination 540 port 4789 when sending traffic to VXLAN GPE VTEP. As per VXLAN, 541 reserved bits 5 and 7, VXLAN GPE P and O-bits respectively must be 542 set to zero. The remaining reserved bits must be zero, including the 543 VXLAN GPE version field, bits 2 and 3. The encapsulated payload MUST 544 be Ethernet. 546 5.2. VXLAN GPE VTEP to VXLAN VTEP 548 A VXLAN GPE VTEP MUST NOT encapsulate non-Ethernet frames to a VXLAN 549 VTEP. When encapsulating Ethernet frames to a VXLAN VTEP, the VXLAN 550 GPE VTEP MUST conform to VXLAN frame format and hence will set the P 551 bit to 0, the Next Protocol to 0 and use UDP destination port 4789. 552 A VXLAN GPE VTEP MUST also set O = 0 and Ver = 0 when encapsulating 553 Ethernet frames to VXLAN VTEP. The receiving VXLAN VTEP will treat 554 this packet as a VXLAN packet. 556 A method for determining the capabilities of a VXLAN VTEP (GPE or 557 non-GPE) is out of the scope of this draft. 559 5.3. VXLAN GPE UDP Ports 561 VXLAN GPE uses a IANA assigned UDP destination port, 4790, when 562 sending traffic to VXLAN GPE VTEPs. 564 5.4. VXLAN GPE and Encapsulated IP Header Fields 566 When encapsulating and decapsulating IPv4 and IPv6 packets, certain 567 fields, such as IPv4 Time to Live (TTL) from the inner IP header need 568 to be considered. VXLAN GPE IP encapsulation and decapsulation 569 utilizes the techniques described in [RFC6830], section 5.3. 571 6. VXLAN GPE Examples 573 This section provides three examples of protocols encapsulated using 574 the Generic Protocol Extension for VXLAN described in this document. 576 0 1 2 3 577 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 578 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 579 |R|R|0|0|I|1|R|0| Reserved | NP = IPv4 | 580 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 581 | VXLAN Network Identifier (VNI) | Reserved | 582 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 583 | Original IPv4 Packet | 584 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 586 Figure 6: IPv4 and VXLAN GPE 588 0 1 2 3 589 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 590 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 591 |R|R|0|0|I|1|R|0| Reserved | NP = IPv6 | 592 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 593 | VXLAN Network Identifier (VNI) | Reserved | 594 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 595 | Original IPv6 Packet | 596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 598 Figure 7: IPv6 and VXLAN GPE 600 0 1 2 3 601 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 602 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 603 |R|R|0|0|I|1|R|0| Reserved |NP = Ethernet | 604 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 605 | VXLAN Network Identifier (VNI) | Reserved | 606 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 607 | Original Ethernet Frame | 608 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 610 Figure 8: Ethernet and VXLAN GPE 612 7. Security Considerations 614 VXLAN's security is focused on issues around L2 encapsulation into 615 L3. With VXLAN GPE, issues such as spoofing, flooding, and traffic 616 redirection are dependent on the particular protocol payload 617 encapsulated. 619 8. Contributors 620 Paul Quinn 621 Cisco Systems 622 paulq@cisco.com 624 Rajeev Manur 625 Broadcom 626 rmanur@broadcom.com 628 Michael Smith 629 Cisco Systems 630 michsmit@cisco.com 632 Darrel Lewis 633 Cisco Systems 634 darlewis@cisco.com 636 Puneet Agarwal 637 Innovium, Inc 638 puneet@acm.org 640 Lucy Yong 641 Huawei USA 642 lucy.yong@huawei.com 644 Xiaohu Xu 645 Huawei Technologies 646 xuxiaohu@huawei.com 648 Pankaj Garg 649 Microsoft 650 pankajg@microsoft.com 652 David Melman 653 Marvell 654 davidme@marvell.com 656 9. Acknowledgments 658 A special thank you goes to Dino Farinacci for his guidance and 659 detailed review. 661 10. IANA Considerations 663 10.1. UDP Port 665 UDP 4790 port has been assigned by IANA for VXLAN GPE. 667 10.2. VXLAN GPE Next Protocol 669 IANA is requested to set up a registry of "Next Protocol". These are 670 8-bit values. Next Protocol values in the table below are defined in 671 this draft. New values are assigned via Standards Action [RFC5226]. 673 +---------------+-------------+---------------+ 674 | Next Protocol | Description | Reference | 675 +---------------+-------------+---------------+ 676 | 0x0 | Reserved | This Document | 677 | 0x1 | IPv4 | This Document | 678 | 0x2 | IPv6 | This Document | 679 | 0x3 | Ethernet | This Document | 680 | 0x4 | NSH | This Document | 681 | 0x5 | MPLS | This Document | 682 | 0x6 | Unassigned | | 683 | 0x7 | vBNG | This Document | 684 | 0x8..0x7F | Unassigned | | 685 | 0x80 | GBP | This Document | 686 | 0x81 | iOAM | This Document | 687 | 0x82..0xFF | Unassigned | | 688 +---------------+-------------+---------------+ 690 10.3. VXLAN GPE Flag and Reserved Bits 692 There are ten flag bits at the beginning of the VXLAN GPE header, 693 followed by 16 reserved bits and an 8-bit reserved field at the end 694 of the header. New bits are assigned via Standards Action [RFC5226]. 696 Bits 0-1 - Reserve6 698 Bits 2-3 - Version 700 Bit 4 - Instance ID (I bit) 702 Bit 5 - Next Protocol (P bit) 704 Bit 6 - Reserved 706 Bit 7 - OAM (O bit) 708 Bit 8-23 - Reserved 710 Bits 24-31 in the 2nd Word -- Reserved 712 Reserved bits/fields MUST be set to 0 by the sender and ignored by 713 the receiver. 715 11. References 717 11.1. Normative References 719 [RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191, 720 DOI 10.17487/RFC1191, November 1990, . 723 [RFC1981] McCann, J., Deering, S., and J. Mogul, "Path MTU Discovery 724 for IP version 6", RFC 1981, DOI 10.17487/RFC1981, August 725 1996, . 727 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 728 Requirement Levels", BCP 14, RFC 2119, 729 DOI 10.17487/RFC2119, March 1997, . 732 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 733 Label Switching Architecture", RFC 3031, 734 DOI 10.17487/RFC3031, January 2001, . 737 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 738 IANA Considerations Section in RFCs", RFC 5226, 739 DOI 10.17487/RFC5226, May 2008, . 742 [RFC6335] Cotton, M., Eggert, L., Touch, J., Westerlund, M., and S. 743 Cheshire, "Internet Assigned Numbers Authority (IANA) 744 Procedures for the Management of the Service Name and 745 Transport Protocol Port Number Registry", BCP 165, 746 RFC 6335, DOI 10.17487/RFC6335, August 2011, 747 . 749 [RFC6830] Farinacci, D., Fuller, V., Meyer, D., and D. Lewis, "The 750 Locator/ID Separation Protocol (LISP)", RFC 6830, 751 DOI 10.17487/RFC6830, January 2013, . 754 [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, 755 L., Sridhar, T., Bursell, M., and C. Wright, "Virtual 756 eXtensible Local Area Network (VXLAN): A Framework for 757 Overlaying Virtualized Layer 2 Networks over Layer 3 758 Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014, 759 . 761 [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., 762 "Network Service Header (NSH)", RFC 8300, 763 DOI 10.17487/RFC8300, January 2018, . 766 [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., 767 Uttaro, J., and W. Henderickx, "A Network Virtualization 768 Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, 769 DOI 10.17487/RFC8365, March 2018, . 772 11.2. Informative References 774 [I-D.brockners-ippm-ioam-vxlan-gpe] 775 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 776 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Kfir, A., 777 Gafni, B., Lapukhov, P., and M. Spiegel, "VXLAN-GPE 778 Encapsulation for In-situ OAM Data", draft-brockners-ippm- 779 ioam-vxlan-gpe-01 (work in progress), June 2018. 781 [I-D.huang-nvo3-vxlan-gpe-extension-for-vbng] 782 Huang, L., Hu, S., Wang, Z., and T. Ao, "VXLAN GPE 783 Extension for Packets Exchange Between Control and User 784 Plane of vBNG", draft-huang-nvo3-vxlan-gpe-extension-for- 785 vbng-01 (work in progress), October 2017. 787 [I-D.ietf-idr-tunnel-encaps] 788 Rosen, E., Patel, K., and G. Velde, "The BGP Tunnel 789 Encapsulation Attribute", draft-ietf-idr-tunnel-encaps-11 790 (work in progress), February 2019. 792 [I-D.lemon-vxlan-lisp-gpe-gbp] 793 Lemon, J., Maino, F., Smith, M., and A. Isaac, "Group 794 Policy Encoding with VXLAN-GPE and LISP-GPE", draft-lemon- 795 vxlan-lisp-gpe-gbp-01 (work in progress), March 2019. 797 Authors' Addresses 799 Fabio Maino (Editor) 800 Cisco Systems 802 Email: fmaino@cisco.com 804 Larry Kreeger (editor) 806 Email: lkreeger@gmail.com 807 Uri Elzur (editor) 808 Intel 810 Email: uri.elzur@intel.com