idnits 2.17.1 draft-quinn-vxlan-gpe-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (February 25, 2015) is 3347 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Unused Reference: 'RFC0768' is defined on line 614, but no explicit reference was found in the text == Unused Reference: 'RFC0791' is defined on line 617, but no explicit reference was found in the text == Unused Reference: 'RFC2119' is defined on line 620, but no explicit reference was found in the text == Unused Reference: 'RFC1700' is defined on line 634, but no explicit reference was found in the text ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) -- Obsolete informational reference (is this intentional?): RFC 1700 (Obsoleted by RFC 3232) -- Obsolete informational reference (is this intentional?): RFC 1981 (Obsoleted by RFC 8201) -- Obsolete informational reference (is this intentional?): RFC 6830 (Obsoleted by RFC 9300, RFC 9301) Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Quinn 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Experimental R. Manur 5 Expires: August 29, 2015 Broadcom 6 L. Kreeger 7 D. Lewis 8 F. Maino 9 M. Smith 10 Cisco Systems, Inc. 11 P. Agarwal 13 L. Yong 14 Huawei USA 15 X. Xu 16 Huawei Technologies 17 U. Elzur 18 Intel 19 P. Garg 20 Microsoft 21 D. Melman 22 Marvell 23 February 25, 2015 25 Generic Protocol Extension for VXLAN 26 draft-quinn-vxlan-gpe-04.txt 28 Abstract 30 This draft describes extending Virtual eXtensible Local Area Network 31 (VXLAN), via changes to the VXLAN header, with three new 32 capabilities: support for multi-protocol encapsulation, operations, 33 administration and management (OAM) signaling and explicit 34 versioning. 36 Status of this Memo 38 This Internet-Draft is submitted in full conformance with the 39 provisions of BCP 78 and BCP 79. 41 Internet-Drafts are working documents of the Internet Engineering 42 Task Force (IETF). Note that other groups may also distribute 43 working documents as Internet-Drafts. The list of current Internet- 44 Drafts is at http://datatracker.ietf.org/drafts/current/. 46 Internet-Drafts are draft documents valid for a maximum of six months 47 and may be updated, replaced, or obsoleted by other documents at any 48 time. It is inappropriate to use Internet-Drafts as reference 49 material or to cite them other than as "work in progress." 51 This Internet-Draft will expire on August 29, 2015. 53 Copyright Notice 55 Copyright (c) 2015 IETF Trust and the persons identified as the 56 document authors. All rights reserved. 58 This document is subject to BCP 78 and the IETF Trust's Legal 59 Provisions Relating to IETF Documents 60 (http://trustee.ietf.org/license-info) in effect on the date of 61 publication of this document. Please review these documents 62 carefully, as they describe your rights and restrictions with respect 63 to this document. Code Components extracted from this document must 64 include Simplified BSD License text as described in Section 4.e of 65 the Trust Legal Provisions and are provided without warranty as 66 described in the Simplified BSD License. 68 Table of Contents 70 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 71 2. VXLAN Without Protocol Extension . . . . . . . . . . . . . . . 5 72 3. Generic Protocol Extension for VXLAN (VXLAN GPE) . . . . . . . 7 73 3.1. VXLAN GPE Header . . . . . . . . . . . . . . . . . . . . . 7 74 3.2. Multi Protocol Support . . . . . . . . . . . . . . . . . . 8 75 3.3. OAM Support . . . . . . . . . . . . . . . . . . . . . . . 8 76 3.4. Version Bits . . . . . . . . . . . . . . . . . . . . . . . 8 77 4. Outer Encapsulations . . . . . . . . . . . . . . . . . . . . . 9 78 4.1. Inner VLAN Tag Handling . . . . . . . . . . . . . . . . . 13 79 4.2. Fragmentation Considerations . . . . . . . . . . . . . . . 13 80 5. Backward Compatibility . . . . . . . . . . . . . . . . . . . . 14 81 5.1. VXLAN VTEP to VXLAN GPE VTEP . . . . . . . . . . . . . . . 14 82 5.2. VXLAN GPE VTEP to VXLAN VTEP . . . . . . . . . . . . . . . 14 83 5.3. VXLAN GPE UDP Ports . . . . . . . . . . . . . . . . . . . 14 84 5.4. VXLAN GPE and Encapsulated IP Header Fields . . . . . . . 14 85 6. VXLAN GPE Examples . . . . . . . . . . . . . . . . . . . . . . 15 86 7. Security Considerations . . . . . . . . . . . . . . . . . . . 17 87 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 18 88 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 89 9.1. UDP Port . . . . . . . . . . . . . . . . . . . . . . . . . 19 90 9.2. VXLAN GPE Next Protocol . . . . . . . . . . . . . . . . . 19 91 9.3. VXLAN GPE Flag and Reserved Bits . . . . . . . . . . . . . 19 92 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 93 10.1. Normative References . . . . . . . . . . . . . . . . . . . 20 94 10.2. Informative References . . . . . . . . . . . . . . . . . . 20 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 97 1. Introduction 99 Virtual eXtensible Local Area Network VXLAN [RFC7348] defines an 100 encapsulation format that encapsulates Ethernet frames in an outer 101 UDP/IP transport. As data centers evolve, the need to carry other 102 protocols encapsulated in an IP packet is required, as well as the 103 need to provide increased visibility and diagnostic capabilities 104 within the overlay. The VXLAN header does not specify the protocol 105 being encapsulated and therefore is currently limited to 106 encapsulating only Ethernet frame payload, nor does it provide the 107 ability to define OAM protocols. In addition, [RFC6335] requires 108 that new transports not use transport layer port numbers to identify 109 tunnel payload, rather it encourages encapsulations to use their own 110 identifiers for this purpose. VXLAN GPE is intended to extend the 111 existing VXLAN protocol to provide protocol typing, OAM, and 112 versioning capabilities. 114 The Version and OAM bits are introduced in Section 3, and the choice 115 of location for these fields is driven by minimizing the impact on 116 existing deployed hardware. 118 In order to facilitate deployments of VXLAN GPE with hardware 119 currently deployed to support VXLAN, changes from legacy VXLAN have 120 been kept to a minimum. Section 5 provides a detailed discussion 121 about how VXLAN GPE addresses the requirement for backward 122 compatibility with VXLAN. 124 2. VXLAN Without Protocol Extension 126 VXLAN provides a method of creating multi-tenant overlay networks by 127 encapsulating packets in IP/UDP along with a header containing a 128 network identifier which is used to isolate tenant traffic in each 129 overlay network from each other. This allows the overlay networks to 130 run over an existing IP network. 132 Through this encapsulation, VXLAN creates stateless tunnels between 133 VXLAN Tunnel End Points (VTEPs) which are responsible for adding/ 134 removing the IP/UDP/VXLAN headers and providing tenant traffic 135 isolation based on the VXLAN Network Identifier (VNI). Tenant 136 systems are unaware that their networking service is being provided 137 by an overlay. 139 When encapsulating packets, a VTEP must know the IP address of the 140 proper remote VTEP at the far end of the tunnel that can deliver the 141 inner packet to the Tenant System corresponding to the inner 142 destination address. In the case of tenant multicast or broadcast, 143 the outer IP address may be an IP multicast group address, or the 144 VTEP may replicate the packet and send it to all known VTEPs. If 145 multicast is used in the underlay network to send encapsulated 146 packets to remote VTEPs, Any Source Multicast is used and each VTEP 147 serving a particular VNI must perform a (*, G) join to the same group 148 IP address. 150 Inner to outer address mapping can be determined in two ways. One is 151 source based learning in the data plane, and the other is 152 distribution via a control plane. 154 Source based learning requires a receiving VTEP to create an inner to 155 outer address mapping by gleaning the information from the received 156 packets by correlating the inner source address to the outer source 157 IP address. When a mapping does not exist, a VTEP forwards the 158 packets to all remote VTEPs participating in the VNI by using IP 159 multicast in the IP underlay network. Each VTEP must be configured 160 with the IP multicast address to use for each VNI. How this occurs 161 is out of scope. 163 The control plane used to distribute inner to outer mappings is also 164 out of scope. It could use a centralized authority or be 165 distributed, or use a hybrid. 167 The VXLAN Network Identifier (VNI) provides scoping for the addresses 168 in the header of the encapsulated PDU. If the encapsulated packet is 169 an Ethernet frame, this means the Ethernet MAC addresses are only 170 unique within a given VNI and may overlap with MAC addresses within a 171 different VNI. If the encapsulated packet is an IP packet, this 172 means the IP addresses are only unique within that VNI. 174 0 1 2 3 175 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 176 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 177 |R|R|R|R|I|R|R|R| Reserved | 178 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 179 | VXLAN Network Identifier (VNI) | Reserved | 180 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 182 Figure 1: VXLAN Header 184 3. Generic Protocol Extension for VXLAN (VXLAN GPE) 186 3.1. VXLAN GPE Header 188 0 1 2 3 189 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 190 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 191 |R|R|Ver|I|P|R|O| Reserved |Next Protocol | 192 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 193 | VXLAN Network Identifier (VNI) | Reserved | 194 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 196 Figure 2: VXLAN GPE Header 198 Flags (8 bits): The first 8 bits of the header are the flag field. 199 The bits designated "R" above are reserved flags. These MUST be 200 set to zero on transmission and ignored on receipt. 202 Version (Ver): Indicates VXLAN GPE protocol version. The initial 203 version is 0. If a receiver does not support the version 204 indicated it MUST drop the packet. 206 Instance Bit (I bit): The I bit MUST be set to indicate a valid VNI. 208 Next Protocol Bit (P bit): The P bit is set to indicate that the 209 Next Protocol field is present. 211 OAM Flag Bit (O bit): The O bit is set to indicate that the packet 212 is an OAM packet. 214 Next Protocol: This 8 bit field indicates the protocol header 215 immediately following the VXLAN GPE header. 217 VNI: This 24 bit field identifies the VXLAN overlay network the 218 inner packet belongs to. Inner packets belonging to different 219 VNIs cannot communicate with each other (unless explicitly allowed 220 by policy). 222 Reserved: Reserved fields MUST be set to zero on transmission and 223 ignored on receipt. 225 3.2. Multi Protocol Support 227 This draft defines the following two changes to the VXLAN header in 228 order to support multi-protocol encapsulation: 230 P Bit: Flag bit 5 is defined as the Next Protocol bit. The P bit 231 MUST be set to 1 to indicate the presence of the 8 bit next 232 protocol field. When P=1, the destination UDP port MUST be 4790. 234 P = 0 indicates that the payload MUST conform to VXLAN as defined 235 in [RFC7348], including destination UDP port. 237 Flag bit 5 was chosen as the P bit because this flag bit is 238 currently reserved in VXLAN. 240 Next Protocol Field: The lower 8 bits of the first word are used to 241 carry a next protocol. This next protocol field contains the 242 protocol of the encapsulated payload packet. A new protocol 243 registry will be requested from IANA, see section 9.2. 245 This draft defines the following Next Protocol values: 247 0x1 : IPv4 248 0x2 : IPv6 249 0x3 : Ethernet 250 0x4 : Network Service Header [NSH] 252 3.3. OAM Support 254 Flag bit 7 is defined as the O bit. When the O bit is set to 1, the 255 packet is an OAM packet and OAM processing MUST occur. Other header 256 fields including Next Protocol MUST adhere to the definitions in 257 section 3. The OAM protocol details are out of scope for this 258 document. As with the P-bit, bit 7 is currently a reserved flag in 259 VXLAN. 261 3.4. Version Bits 263 VXLAN GPE bits 2 and 3 are defined as version bits. These bits are 264 reserved in VXLAN. The version field is used to ensure backward 265 compatibility going forward with future VXLAN GPE updates. 267 The initial version for VXLAN GPE is 0. 269 4. Outer Encapsulations 271 In addition to the VXLAN GPE header, the packet is further 272 encapsulated in UDP and IP. Data centers based on Ethernet, will 273 then send this IP packet over Ethernet. 275 Outer UDP Header: 277 Destination UDP Port: IANA has assigned the value 4790 for the VXLAN 278 GPE UDP port. This well-known destination port is used when sending 279 VXLAN GPE encapsulated packets. 281 Source UDP Port: The source UDP port is used as entropy for devices 282 forwarding encapsulated packets across the underlay (ECMP for IP 283 routers, or load splitting for link aggregation by bridges). Tenant 284 traffic flows should all use the same source UDP port to lower the 285 chances of packet reordering by the underlay for a given flow. It is 286 recommended for VTEPs to generate this port number using a hash of 287 the inner packet headers. 289 UDP Checksum: Source VTEPs MAY either calculate a valid checksum, or 290 if this is not possible, set the checksum to zero. When calculating 291 a checksum, it MUST be calculated across the entire packet (outer IP 292 header, UDP header, VXLAN GPE header and payload packet). All 293 receiving VTEPs must accept a checksum value of zero. If the 294 receiving VTEP is capable of validating the checksum, it MAY validate 295 a non-zero checksum and MUST discard the packet if the checksum is 296 determined to be invalid. 298 Outer IP Header: 300 This is the header used by the underlay network to deliver packets 301 between VTEPs. The destination IP address can be a unicast or a 302 multicast IP address. The source IP address must be the source VTEP 303 IP address which can be used to return tenant packets to the tenant 304 system source address within the inner packet header. 306 When the outer IP header is IPv4, VTEPs MUST set the DF bit. 308 Outer Ethernet Header: 310 Most data centers networks are built on Ethernet. Assuming the outer 311 IP packet is being sent across Ethernet, there will be an Ethernet 312 header used to deliver the IP packet to the next hop, which could be 313 the destination VTEP or be a router used to forward the IP packet 314 towards the destination VTEP. If VLANs are in use within the data 315 center, then this Ethernet header would also contain a VLAN tag. 317 The following figures show the entire stack of protocol headers that 318 would be seen on an Ethernet link carrying encapsulated packets from 319 a VTEP across the underlay network for both IPv4 and IPv6 based 320 underlay networks. 322 0 1 2 3 323 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 325 Outer Ethernet Header: 326 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 327 | Outer Destination MAC Address | 328 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 329 | Outer Destination MAC Address | Outer Source MAC Address | 330 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 331 | Outer Source MAC Address | 332 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 333 | Opt Ethertype = C-Tag 802.1Q | Outer VLAN Tag | 334 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 335 | Ethertype = 0x0800 | 336 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 338 Outer IPv4 Header: 339 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 340 |Version| IHL |Type of Service| Total Length | 341 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 342 | Identification |Flags| Fragment Offset | 343 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 344 | Time to Live |Protocl=17(UDP)| Header Checksum | 345 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 346 | Outer Source IPv4 Address | 347 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 348 | Outer Destination IPv4 Address | 349 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 351 Outer UDP Header: 352 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 353 | Source Port | Dest Port = 4790 | 354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 355 | UDP Length | UDP Checksum | 356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 358 VXLAN GPE Header: 359 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 360 |R|R|Ver|I|P|R|O| Reserved |Next Protocol | 361 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 362 | VXLAN Network Identifier (VNI) | Reserved | 363 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 365 Payload: 366 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 367 | Depends on VXLAN GPE Next Protocol field above. | 368 | Note that if the payload is Ethernet, then the original | 369 | Ethernet Frame's FCS is not included. | 370 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 372 Frame Check Sequence: 373 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 374 | New FCS (Frame Check Sequence) for Outer Ethernet Frame | 375 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 377 Figure 3: Outer Headers for VXLAN GPE over IPv4 379 0 1 2 3 380 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 382 Outer Ethernet Header: 383 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 384 | Outer Destination MAC Address | 385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 386 | Outer Destination MAC Address | Outer Source MAC Address | 387 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 388 | Outer Source MAC Address | 389 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 390 | Opt Ethertype = C-Tag 802.1Q | Outer VLAN Tag | 391 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 392 | Ethertype = 0x86DD | 393 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 395 Outer IPv6 Header: 396 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 397 |Version| Traffic Class | Flow Label | 398 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 399 | Payload Length | NxtHdr=17(UDP)| Hop Limit | 400 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 401 | | 402 + + 403 | | 404 + Outer Source IPv6 Address + 405 | | 406 + + 407 | | 408 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 409 | | 410 + + 411 | | 412 + Outer Destination IPv6 Address + 413 | | 414 + + 415 | | 416 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 418 Outer UDP Header: 419 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 420 | Source Port | Dest Port = 4790 | 421 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 422 | UDP Length | UDP Checksum | 423 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 425 VXLAN GPE Header: 426 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 427 |R|R|Ver|I|P|R|O| Reserved |Next Protocol | 428 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 429 | VXLAN Network Identifier (VNI) | Reserved | 430 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 432 Payload: 433 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 434 | Depends on VXLAN GPE Next Protocol field above. | 435 | Note that if the payload is Ethernet, then the original | 436 | Ethernet Frame's FCS is not included. | 437 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 439 Frame Check Sequence: 440 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 441 | New FCS (Frame Check Sequence) for Outer Ethernet Frame | 442 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 444 Figure X: Outer Headers for VXLAN GPE over IPv6 446 Figure 4: Outer Headers for VXLAN GPE over IPv6 448 4.1. Inner VLAN Tag Handling 450 If the inner packet (as indicated by the VXLAN GPE Next Protocol 451 field) is an Ethernet frame, it is recommended that it does not 452 contain a VLAN tag. In the most common scenarios, the tenant VLAN 453 tag is translated into a VXLAN Network Identifier. In these 454 scenarios, VTEPs should never send an inner Ethernet frame with a 455 VLAN tag, and a VTEP performing decapsulation should discard any 456 inner frames received with a VLAN tag. However, if the VTEPs are 457 specifically configured to support it for a specific VXLAN Network 458 Identifier, a VTEP may support transparent transport of the inner 459 VLAN tag between all tenant systems on that VNI. The VTEP never 460 looks at the value of the inner VLAN tag, but simply passes it across 461 the underlay. 463 4.2. Fragmentation Considerations 465 VTEPs MUST never fragment an encapsulated VXLAN GPE packet, and when 466 the outer IP header is IPv4, VTEPs MUST set the DF bit in the outer 467 IPv4 header. It is recommended that the underlay network be 468 configured to carry an MTU at least large enough to accommodate the 469 added encapsulation headers. It is recommended that VTEPs perform 470 Path MTU discovery [RFC1191] [RFC1981] to determine if the underlay 471 network can carry the encapsulated payload packet. 473 5. Backward Compatibility 475 5.1. VXLAN VTEP to VXLAN GPE VTEP 477 A VXLAN VTEP conforms to VXLAN frame format and uses UDP destination 478 port 4789 when sending traffic to VXLAN GPE VTEP. As per VXLAN, 479 reserved bits 5 and 7, VXLAN GPE P and O-bits respectively must be 480 set to zero. The remaining reserved bits must be zero, including the 481 VXLAN GPE version field, bits 2 and 3. The encapsulated payload MUST 482 be Ethernet. 484 5.2. VXLAN GPE VTEP to VXLAN VTEP 486 A VXLAN GPE VTEP MUST NOT encapsulate non-Ethernet frames to a VXLAN 487 VTEP. When encapsulating Ethernet frames to a VXLAN VTEP, the VXLAN 488 GPE VTEP MUST conform to VXLAN frame format and hence will set the P 489 bit to 0, the Next Protocol to 0 and use UDP destination port 4789. 490 A VXLAN GPE VTEP MUST also set O = 0 and Ver = 0 when encapsulating 491 Ethernet frames to VXLAN VTEP. The receiving VXLAN VTEP will treat 492 this packet as a VXLAN packet. 494 A method for determining the capabilities of a VXLAN VTEP (GPE or 495 non-GPE) is out of the scope of this draft. 497 5.3. VXLAN GPE UDP Ports 499 VXLAN GPE uses a IANA assigned UDP destination port, 4790, when 500 sending traffic to VXLAN GPE VTEPs. 502 5.4. VXLAN GPE and Encapsulated IP Header Fields 504 When encapsulating and decapsulating IPv4 and IPv6 packets, certain 505 fields, such as IPv4 Time to Live (TTL) from the inner IP header need 506 to be considered. VXLAN GPE IP encapsulation and decapsulation 507 utilizes the techniques described in [RFC6830], section 5.3. 509 6. VXLAN GPE Examples 511 This section provides three examples of protocols encapsulated using 512 the Generic Protocol Extension for VXLAN described in this document. 514 0 1 2 3 515 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 516 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 517 |R|R|0|0|I|1|R|0| Reserved | NP = IPv4 | 518 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 519 | VXLAN Network Identifier (VNI) | Reserved | 520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 521 | Original IPv4 Packet | 522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 524 Figure 5: IPv4 and VXLAN GPE 526 0 1 2 3 527 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 528 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 529 |R|R|0|0|I|1|R|0| Reserved | NP = IPv6 | 530 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 531 | VXLAN Network Identifier (VNI) | Reserved | 532 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 533 | Original IPv6 Packet | 534 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 536 Figure 6: IPv6 and VXLAN GPE 538 0 1 2 3 539 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 540 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 541 |R|R|0|0|I|1|R|0| Reserved |NP = Ethernet | 542 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 543 | VXLAN Network Identifier (VNI) | Reserved | 544 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 545 | Original Ethernet Frame | 546 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 548 Figure 7: Ethernet and VXLAN GPE 550 7. Security Considerations 552 VXLAN's security is focused on issues around L2 encapsulation into 553 L3. With VXLAN GPE, issues such as spoofing, flooding, and traffic 554 redirection are dependent on the particular protocol payload 555 encapsulated. 557 8. Acknowledgments 559 A special thank you goes to Dino Farinacci for his guidance and 560 detailed review. 562 9. IANA Considerations 564 9.1. UDP Port 566 UDP 4790 port has been assigned by IANA for VXLAN GPE. 568 9.2. VXLAN GPE Next Protocol 570 IANA is requested to set up a registry of "Next Protocol". These are 571 8-bit values. Next Protocol values 0, 1, 2, 3 and 4 are defined in 572 this draft. New values are assigned via Standards Action [RFC5226]. 574 +---------------+-------------+---------------+ 575 | Next Protocol | Description | Reference | 576 +---------------+-------------+---------------+ 577 | 0 | Reserved | This document | 578 | | | | 579 | 1 | IPv4 | This document | 580 | | | | 581 | 2 | IPv6 | This document | 582 | | | | 583 | 3 | Ethernet | This document | 584 | | | | 585 | 4 | NSH | This document | 586 | | | | 587 | 5..253 | Unassigned | | 588 +---------------+-------------+---------------+ 590 Table 1 592 9.3. VXLAN GPE Flag and Reserved Bits 594 There are ten flag bits at the beginning of the VXLAN GPE header, 595 followed by 16 reserved bits and an 8-bit reserved field at the end 596 of the header. New bits are assigned via Standards Action [RFC5226]. 598 Bits 0-1 - Reserved 599 Bits 2-3 - Version 600 Bit 4 - Instance ID (I bit) 601 Bit 5 - Next Protocol (P bit) 602 Bit 6 - Reserved 603 Bit 7 - OAM (O bit) 604 Bits 8-23 - Reserved 605 Bits 24-31 in the 2nd Word -- Reserved 607 Reserved bits/fields MUST be set to 0 by the sender and ignored by 608 the receiver. 610 10. References 612 10.1. Normative References 614 [RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 615 August 1980. 617 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 618 September 1981. 620 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 621 Requirement Levels", BCP 14, RFC 2119, March 1997. 623 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 624 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 625 May 2008. 627 10.2. Informative References 629 [NSH] Quinn, P. and et al. , "Network Service Header", 2014. 631 [RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191, 632 November 1990. 634 [RFC1700] Reynolds, J. and J. Postel, "Assigned Numbers", RFC 1700, 635 October 1994. 637 [RFC1981] McCann, J., Deering, S., and J. Mogul, "Path MTU Discovery 638 for IP version 6", RFC 1981, August 1996. 640 [RFC6335] Cotton, M., Eggert, L., Touch, J., Westerlund, M., and S. 641 Cheshire, "Internet Assigned Numbers Authority (IANA) 642 Procedures for the Management of the Service Name and 643 Transport Protocol Port Number Registry", BCP 165, 644 RFC 6335, August 2011. 646 [RFC6830] Farinacci, D., Fuller, V., Meyer, D., and D. Lewis, "The 647 Locator/ID Separation Protocol (LISP)", RFC 6830, 648 January 2013. 650 [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, 651 L., Sridhar, T., Bursell, M., and C. Wright, "Virtual 652 eXtensible Local Area Network (VXLAN): A Framework for 653 Overlaying Virtualized Layer 2 Networks over Layer 3 654 Networks", RFC 7348, August 2014. 656 Authors' Addresses 658 Paul Quinn 659 Cisco Systems, Inc. 661 Email: paulq@cisco.com 663 Rajeev Manur 664 Broadcom 666 Email: rmanur@broadcom.com 668 Larry Kreeger 669 Cisco Systems, Inc. 671 Email: kreeger@cisco.com 673 Darrel Lewis 674 Cisco Systems, Inc. 676 Email: darlewis@cisco.com 678 Fabio Maino 679 Cisco Systems, Inc. 681 Email: fmaino@cisco.com 683 Michael Smith 684 Cisco Systems, Inc. 686 Email: michsmit@cisco.com 688 Puneet Agarwal 690 Email: puneet@acm.org 692 Lucy Yong 693 Huawei USA 695 Email: lucy.yong@huawei.com 696 Xiaohu Xu 697 Huawei Technologies 699 Email: xuxiaohu@huawei.com 701 Uri Elzur 702 Intel 704 Email: uri.elzur@intel.com 706 Pankaj Garg 707 Microsoft 709 Email: Garg.Pankaj@microsoft.com 711 David Melman 712 Marvell 714 Email: davidme@marvell.com