idnits 2.17.1 draft-sd-l2vpn-evpn-overlay-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 9 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 433 has weird spacing: '... be set to VN...' -- The document date (October 21, 2013) is 3837 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'GRE' is mentioned on line 261, but not defined == Missing Reference: 'RFC4023' is mentioned on line 481, but not defined == Missing Reference: 'NOV3-Framework' is mentioned on line 547, but not defined == Missing Reference: 'RFC6514' is mentioned on line 813, but not defined == Unused Reference: 'NOV3-FRWK' is defined on line 1039, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 4272 ** Obsolete normative reference: RFC 5512 (Obsoleted by RFC 9012) == Outdated reference: A later version (-07) exists of draft-ietf-l2vpn-evpn-req-01 == Outdated reference: A later version (-08) exists of draft-sridharan-virtualization-nvgre-01 == Outdated reference: A later version (-09) exists of draft-mahalingam-dutt-dcops-vxlan-02 == Outdated reference: A later version (-11) exists of draft-ietf-l2vpn-evpn-02 == Outdated reference: A later version (-04) exists of draft-ietf-nvo3-overlay-problem-statement-01 == Outdated reference: A later version (-09) exists of draft-ietf-nvo3-framework-01 Summary: 3 errors (**), 0 flaws (~~), 13 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 L2VPN Workgroup A. Sajassi (Editor) 3 INTERNET-DRAFT Cisco 4 Intended Status: Standards Track 5 J. Drake (Editor) 6 Y. Rekhter Juniper 7 R. Shekhar 8 B. Schliesser Nabil Bitar 9 Juniper Verizon 11 S. Salam Aldrin Isaac 12 K. Patel Bloomberg 13 D. Rao 14 S. Thoria James Uttaro 15 Cisco AT&T 17 L. Yong W. Henderickx 18 Huawei Alcatel-Lucent 20 Expires: April 21, 2014 October 21, 2013 22 A Network Virtualization Overlay Solution using EVPN 23 draft-sd-l2vpn-evpn-overlay-02 25 Abstract 27 This document describes how EVPN can be used as an NVO solution and 28 explores the various tunnel encapsulation options over IP and their 29 impact on the EVPN control-plane and procedures. In particular, the 30 following encapsulation options are analyzed: MPLS over GRE, VXLAN, 31 and NVGRE. 33 Status of this Memo 35 This Internet-Draft is submitted to IETF in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF), its areas, and its working groups. Note that 40 other groups may also distribute working documents as 41 Internet-Drafts. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 47 The list of current Internet-Drafts can be accessed at 48 http://www.ietf.org/1id-abstracts.html 50 The list of Internet-Draft Shadow Directories can be accessed at 51 http://www.ietf.org/shadow.html 53 Copyright and License Notice 55 Copyright (c) 2012 IETF Trust and the persons identified as the 56 document authors. All rights reserved. 58 This document is subject to BCP 78 and the IETF Trust's Legal 59 Provisions Relating to IETF Documents 60 (http://trustee.ietf.org/license-info) in effect on the date of 61 publication of this document. Please review these documents 62 carefully, as they describe your rights and restrictions with respect 63 to this document. Code Components extracted from this document must 64 include Simplified BSD License text as described in Section 4.e of 65 the Trust Legal Provisions and are provided without warranty as 66 described in the Simplified BSD License. 68 Table of Contents 70 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 71 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 72 2 EVPN Features . . . . . . . . . . . . . . . . . . . . . . . . . 5 73 3 Encapsulation Options for EVPN Overlays . . . . . . . . . . . . 6 74 3.1 VXLAN/NVGRE Encapsulation . . . . . . . . . . . . . . . . . 6 75 3.1.1 Virtual Identifiers Scope . . . . . . . . . . . . . . . 7 76 3.1.1.1 Data Center Interconnect with Gateway . . . . . . . 7 77 3.1.1.2 Data Center Interconnect without Gateway . . . . . . 8 78 3.1.2 Virtual Identifiers to EVI Mapping . . . . . . . . . . . 8 79 3.1.2.1 Auto Derivation of RT & RD . . . . . . . . . . . . . 9 80 3.1.3 Constructing EVPN BGP Routes . . . . . . . . . . . . . 10 81 3.1.3.1 Constructing E-VPN MAC Address Advertisement 82 Route . . . . . . . . . . . . . . . . . . . . . . . 11 83 3.2 MPLS over GRE . . . . . . . . . . . . . . . . . . . . . . . 11 84 4 EVPN with Multiple Data Plane Encapsulations . . . . . . . . . 12 85 5 NVE Residing in Hypervisor . . . . . . . . . . . . . . . . . . 12 86 5.1 Impact on EVPN BGP Routes & Attributes for VXLAN/NVGRE 87 Encapsulation . . . . . . . . . . . . . . . . . . . . . . . 12 88 5.2 Impact on EVPN Procedures for VXLAN/NVGRE Encapsulation . . 13 89 6 NVE Residing in ToR Switch . . . . . . . . . . . . . . . . . . 14 90 6.1 EVPN Multi-Homing Features . . . . . . . . . . . . . . . . 14 91 6.1.1 Multi-homed Ethernet Segment Auto-Discovery . . . . . . 14 92 6.1.2 Fast Convergence and Mass Withdraw . . . . . . . . . . . 14 93 6.1.3 Split-Horizon . . . . . . . . . . . . . . . . . . . . . 15 94 6.1.4 Aliasing and Backup-Path . . . . . . . . . . . . . . . . 15 95 6.1.5 DF Election . . . . . . . . . . . . . . . . . . . . . . 16 96 6.2 Impact on EVPN BGP Routes & Attributes . . . . . . . . . . . 16 97 6.3 Impact on EVPN Procedures . . . . . . . . . . . . . . . . . 16 98 6.3.1 Split Horizon . . . . . . . . . . . . . . . . . . . . . 17 99 6.3.2 Aliasing and Backup-Path . . . . . . . . . . . . . . . . 18 100 7 Support for Multicast . . . . . . . . . . . . . . . . . . . . . 18 101 8 Support for NVEs with data plane MAC learning . . . . . . . . . 19 102 8.1 Advertising NVE capabilities . . . . . . . . . . . . . . . . 20 103 8.2 Advertising flood lists for ingress replication . . . . . . 20 104 9 Inter-AS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 105 10 Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 22 106 11 Security Considerations . . . . . . . . . . . . . . . . . . . 22 107 12 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 108 13 References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 109 11.1 Normative References . . . . . . . . . . . . . . . . . . . 22 110 11.2 Informative References . . . . . . . . . . . . . . . . . . 23 111 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 113 1 Introduction 115 In the context of this document, a Network Virtualization Overlay 116 (NVO) is a solution to address the requirements of a multi-tenant 117 data center, especially one with virtualized hosts, e.g., Virtual 118 Machines (VMs). The key requirements of such a solution, as described 119 in [Problem-Statement], are: 121 - Isolation of network traffic per tenant 123 - Support for a large number of tenants (tens or hundreds of 124 thousands) 126 - Extending L2 connectivity among different VMs belonging to a given 127 tenant segment (subnet) across different PODs within a data center or 128 between different data centers 130 - Allowing a given VM to move between different physical points of 131 attachment within a given L2 segment 133 The underlay network for NVO solutions is assumed to provide IP 134 connectivity between NVO endpoints (NVEs). 136 This document describes how EVPN can be used as an NVO solution and 137 explores applicability of EVPN functions and procedures. In 138 particular, it describes the various tunnel encapsulation options for 139 EVPN over IP, and their impact on the EVPN control-plane and 140 procedures for two main scenarios: 142 a) when the NVE resides in the hypervisor, and 143 b) when the NVE resides in a ToR device 145 Note that the use of EVPN as an NVO solution does not necessarily 146 mandate that the BGP control-plane be running on the NVE. For such 147 scenarios, it is still possible to leverage the EVPN solution by 148 using XMPP, or alternative mechanisms, to extend the control-plane to 149 the NVE as discussed in [L3VPN-ENDSYSTEMS]. 151 The possible encapsulation options for EVPN overlays that are 152 analyzed in this document are: 154 - VXLAN and NVGRE 155 - MPLS over GRE 157 Before getting into the description of the different encapsulation 158 options for EVPN over IP, it is important to highlight the EVPN 159 solution's main features, how those features are currently supported, 160 and any impact that the encapsulation has on those features. 162 1.1 Terminology 164 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 165 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 166 document are to be interpreted as described in RFC 2119 [KEYWORDS]. 168 NVE: Network Virtualization Endpoint 170 Virtual Identifier: refers to a VXLAN VNI or NVGRE VSID 172 2 EVPN Features 174 EVPN was originally designed to support the requirements detailed in 175 [EVPN-REQ] and therefore has the following attributes which directly 176 address control plane scaling and ease of deployment issues. 178 1) Control plane traffic is distributed with BGP and Broadcast and 179 Multicast traffic is sent using a shared multicast tree or with 180 ingress replication. 182 2) Control plane learning is used for MAC (and IP) addresses instead 183 of data plane learning. The latter requires the flooding of unknown 184 unicast and ARP frames; whereas, the former does not require any 185 flooding. 187 3) Route Reflector is used to reduce a full mesh of BGP sessions 188 among PE devices to a single BGP session between a PE and the RR. 189 Furthermore, RR hierarchy can be leveraged to scale the number BGP 190 routes on the RR. 192 4) Auto-discovery via BGP is used to discover PE devices 193 participating in a given VPN, PE devices participating in a given 194 redundancy group, tunnel encapsulation types, multicast tunnel type, 195 multicast members, etc. 197 5) All-Active multi-homing is used. This allows a given customer 198 device (CE) to have multiple links to multiple PEs, and traffic 199 to/from that CE fully utilizes all of these links. This set of links 200 is termed an Ethernet Segment (ES). 202 6) Mass withdraw is used. When a link between a CE and a PE fails, 203 the PEs in all EVPNs associated with that failed link are notified 204 via the withdrawal of a single EVPN route regardless of how many MAC 205 addresses are located at the CE. 207 7) Route filtering and constrained route distribution are used to 208 ensure that the control plane traffic for a given EVPN is only 209 distributed to the PEs in that EVPN. 211 8) The internal identifier of a broadcast domain, the Ethernet Tag, 212 is a 32 bit number, which is mapped into whatever broadcast domain 213 identifier, e.g., VLAN ID, is understood by the attaching CE device. 214 This means that when 802.1q interfaces are used, there are up to 4096 215 distinct VLAN IDs for each attaching CE device in a given EVPN. 217 9) VM Mobility mechanisms ensure that all PEs in a given EVPN know 218 the ES with which a given VM, as identified by its MAC and IP 219 addresses, is currently associated. 221 10) Route Targets are used to allow the operator (or customer) to 222 define a spectrum of logical network topologies including mesh, hub & 223 spoke, and extranets (e.g., a VPN whose sites are owned by different 224 enterprises), without the need for proprietary software or the aid of 225 other virtual or physical devices. 227 11) Because the design goal for NVO is millions of instances per 228 common physical infrastructure, the scaling properties of the control 229 plane for NVO are extremely important. EVPN and the extensions 230 described herein, are designed with this level of scalability in 231 mind. 233 3 Encapsulation Options for EVPN Overlays 235 3.1 VXLAN/NVGRE Encapsulation 237 Both VXLAN and NVGRE are examples of technologies that provide a data 238 plane encapsulation which is used to transport a packet over the 239 common physical infrastructure between NVEs, VXLAN Tunnel End Point 240 (VTEPs) in VXLAN and Network Virtualization Endpoint (NVEs) in NVGRE. 241 Both of these technologies include the identifier of the specific NVO 242 instance, Virtual Network Identifier (VNI) in VXLAN and Virtual 243 Subnet Identifier (VSID), NVGRE, in each packet. 245 Note that a Provider Edge (PE) is equivalent to a VTEP/NVE. 247 [VXLAN] encapsulation is based on UDP, with an 8-byte header 248 following the UDP header. VXLAN provides a 24-bit VNI, which 249 typically provides a one-to-one mapping to the tenant VLAN ID, as 250 described in [VXLAN]. In this scenario, the VTEP does not include an 251 inner VLAN tag on frame encapsulation, and discards decapsulated 252 frames with an inner VLAN tag. This mode of operation in [VXLAN] maps 253 to VLAN Based Service in [EVPN], where a tenant VLAN ID gets mapped 254 to an EVPN instance (EVI). 256 [VXLAN] also provides an option of including an inner VLAN tag in the 257 encapsulated frame, if explicitly configured at the VTEP. This mode 258 of operation maps to VLAN Bundle Service in [EVPN], where the VLANs 259 of a given tenant get mapped to an EVI. 261 [NVGRE] encapsulation is based on [GRE] and it mandates the inclusion 262 of the optional GRE Key field which carries the VSID. There is a one- 263 to-one mapping between the VSID and the tenant VLAN ID, as described 264 in [NVGRE] and the inclusion of an inner VLAN tag is prohibited. This 265 mode of operation in [NVGRE] maps to VLAN Based Service in [EVPN]. In 266 other words, [NVGRE] prohibits the application of VLAN Bundle Service 267 in [EVPN] and it only requires VLAN Based Service in [EVPN]. 269 As described in the next section there is no change to the encoding 270 of EVPN routes to support VXLAN or NVGRE encapsulation except for the 271 use of BGP Encapsulation extended community. However, there is 272 potential impact to the EVPN procedures depending on where the NVE is 273 located (i.e., in hypervisor or TOR) and whether multi-homing 274 capabilities are required. 276 3.1.1 Virtual Identifiers Scope 278 Although VNI or VSID are defined as 24-bit globally unique values, 279 there are scenarios in which it is desirable to use a locally 280 significant value for VNI or VSID, especially in the context of data 281 center interconnect: 283 3.1.1.1 Data Center Interconnect with Gateway 285 In the case where NVEs in different data centers need to be 286 interconnected, and a Gateway is employed at the edge of the data 287 center network, the NVEs should treat the VNI or VSID as a globally 288 unique identifier within a data center. This is because the Gateway 289 will provide the functionality of translating the VNI or VSID when 290 crossing network boundaries, which may align with operator span of 291 control boundaries. As an example, consider the network of Figure 1 292 below. Assume there are three network operators: one for each of the 293 DC1, DC2 and WAN networks. The Gateways at the edge of the data 294 centers are responsible for translating the VNIs / VSIDs between the 295 values used in each of the data center networks and the values used 296 in the WAN. 298 +--------------+ 299 | | 300 +---------+ | WAN | +---------+ 301 +----+ | +---+ +----+ +----+ +---+ | +----+ 302 |NVE1|--| | | |WAN | |WAN | | | |--|NVE3| 303 +----+ |IP |GW |--|Edge| |Edge|--|GW | IP | +----+ 304 +----+ |Fabric +---+ +----+ +----+ +---+ Fabric | +----+ 305 |NVE2|--| | | | | |--|NVE4| 306 +----+ +---------+ +--------------+ +---------+ +----+ 308 |<------ DC 1 ------> <------ DC2 ------>| 310 Figure 1: Data Center Interconnect with Gateway 312 3.1.1.2 Data Center Interconnect without Gateway 314 In the case where NVEs in different data centers need to be 315 interconnected, and Gateways are not employed at the edge of the data 316 center network, it is useful to treat the VNIs or VSIDs as locally 317 significant identifiers (e.g., as an MPLS label). More specifically, 318 the VNI or VSID value that is used by the transmitting NVE is 319 allocated by the NVE that is receiving the traffic (in other words, 320 this is a "downstream assigned" MPLS label). This allows the VNI or 321 VSID space to be decoupled between different data center networks 322 without the need for a dedicated Gateway at the edge of the data 323 centers. 325 +--------------+ 326 | | 327 +---------+ | WAN | +---------+ 328 +----+ | | +----+ +----+ | | +----+ 329 |NVE1|--| | |WAN | |WAN | | |--|NVE3| 330 +----+ |IP Fabric|---|Edge| |Edge|--|IP Fabric| +----+ 331 +----+ | | +----+ +----+ | | +----+ 332 |NVE2|--| | | | | |--|NVE4| 333 +----+ +---------+ +--------------+ +---------+ +----+ 335 |<------ DC 1 -----> <---- DC2 ------>| 337 Figure 2: Data Center Interconnect without Gateway 339 3.1.2 Virtual Identifiers to EVI Mapping 341 When the EVPN control plane is used in conjunction with VXLAN or 342 NVGRE, two options for mapping the VXLAN VNI or NVGRE VSID to an EVPN 343 Instance (EVI) are possible: 345 1. Option 1: Single Virtual Identifier per EVI 347 In this option, every VNI or VSID is mapped to a unique EVI. As such, 348 a BGP RD and RT is needed per VNI / VSID on every VTEP. The advantage 349 of this model is that it allows the BGP RT constraint mechanisms to 350 be used in order to limit the propagation and import of routes to 351 only the VTEPs that are interested in a given VNI (or VSID). The 352 disadvantage of this model may be the provisioning overhead if RD and 353 RT are not derived automatically from VNI (for VSID). 355 In this option, the MAC-VRF table is identified by the RT in the 356 control plane (because Ethernet Tag field in the MAC route is set to 357 zero) and by the VNI (or VSID) in the data-plane. 359 2. Option 2: Multiple Virtual Identifiers per EVI 361 In this option, multiple VNIs or VSIDs are mapped to a unique EVI. 362 For example, if a tenant has multiple segments/subnets each 363 represented by a VNI (or VSID), then all the VNIs (or VSIDs) for that 364 tenant are mapped to a single EVI - e.g., the EVI in this case 365 represents the tenant and not a subnet . The advantage of this model 366 is that it doesn't require the provisioning of RD/RT per VNI or VSID 367 which is a moot point if auto-derivation is used. The disadvantage of 368 this model is that routes would be imported by VTEPs that may not be 369 interested in a given VNI (or VSID). 371 In this option the MAC-VRF table is identified by the VNI (or VSID) 372 in both the control plane and the data-plane. 374 3.1.2.1 Auto Derivation of RT & RD 376 When the option of a single VNI (or VSID) per EVI is used, it is 377 important to auto-derive RD and RT for EVPN BGP routes in order to 378 simplify configuration for data center operations. RD can be auto- 379 derive as described in [EVPN] and RT can be auto-derived as described 380 next. 382 Since a gateway PE as depicted in figure-1 participates in both the 383 DCN and WAN BGP sessions, it is important that when RT values are 384 auto-derived for VNIs (or VSIDs), there is no conflict in RT spaces 385 between DCN and WAN networks assuming that both are operating within 386 the same AS. Also, there can be scenarios where both VXLAN and NVGRE 387 encapsulations may be needed within the same DCN and their 388 corresponding VNIs and VSIDs are administered independently which 389 means VNI and VSID spaces can overlap. In order to ensure that no 390 such conflict in RT spaces arises, RT values for DCNs are auto- 391 derived as follow: 393 - 2 bytes of global admin field of the RT is set to the AS number. 395 - Three least significant bytes of the local admin field of the RT is 396 set to the VNI or VSID, I-SID, or VID. The most significant bit of 397 the local admin field of the RT is set as follow: 398 0: auto-derived 399 1: manually-derived 401 - The remaining 7 bits of the most significant byte of the local 402 admin field of the RT identifies the space in which the other 3 bytes 403 are defined. The following spaces are defined: 404 0 : EVI 405 1 : VXLAN 406 2 : NVGRE 407 3 : I-SID 408 4 : VID 410 3.1.3 Constructing EVPN BGP Routes 412 In EVPN, an MPLS label distributed by the egress PE via the EVPN 413 control plane and placed in the MPLS header of a given packet by the 414 ingress PE. This label is used upon receipt of that packet by the 415 egress PE to disposition that packet. This is very similar to the 416 use of the VNI or VSID by the egress VTEP or NVE, respectively, with 417 the difference being that an MPLS label has local significance and is 418 distributed by the EVPN control plane, while a VNI or VSID typically 419 has global significance. 421 As discussed in Section 3.1.1 above, there are scenarios in which it 422 is desirable to use a locally significant value for VNI or VSID and 423 in such such scenarios, MPLS label is advertised in EVPN BGP routes 424 and it is used in VXLAN or NVGRE encapsulation as a 20-bit value for 425 VNI or VSID. 427 This memo specifies that when EVPN is used with a VXLAN or NVGRE data 428 plane and when a globally significant VNI or VSID is desirable, then 429 for VNI-based mode (single VNI per EVI), the Ethernet Tag field of 430 EVPN BGP routes (which is a 4-octet field) MUST be set to zero just 431 like the VLAN-based mode in baseline EVPN. If VNI-aware bundle mode 432 (multiple VNIs per EVI) is desired, then the Ethernet Tag field of 433 EVPN BGP routes MUST be set to VNI (or VSID) accordingly just like 434 VLAN-aware bundle mode in baseline EVPN. In both cases, the MPLS 435 label field of the EVPN BGP routes MUST be set to zero. 437 This memo also specifies that when EVPN is used with a VXLAN or NVGRE 438 data plane and when a locally significant VNI or VSID is desirable, 439 then MPLS field of EVPN BGP routes (which is a 3-octet field) MUST be 440 used and Ethernet Tag field MUST be set to zero. In such scenarios, 441 only VNI-based mode (single VNI per EVI) is supported. 443 In order to indicate that a VXLAN or NVGRE data plane encapsulation 444 rather than MPLS label stack encapsulation is to be used, the BGP 445 Encapsulation extended community defined in [RFC5512] is included 446 with EVPN MAC route, Inclusive Multicast route, or per EVI Ethernet 447 AD route advertised by an egress PE. Two new values, one for VXLAN 448 and one for NVGRE, will be defined to extend the list of 449 encapsulation types defined in [RFC5512]: 451 + 3 - VXLAN Encapsulation 452 + 4 - NVGRE Encapsulation 454 If BGP Encapsulation extended community is not present, then the 455 default encapsulation MPLS encapsulation (or statically configured 456 encapsulation) is used. 458 3.1.3.1 Constructing E-VPN MAC Address Advertisement Route 460 In EVPN, unicast MAC addresses are advertised via MAC Advertisement 461 route. The Ethernet Tag field in this route is set zero for the VNI- 462 based mode and set to VNI (or VSID) for VNI-aware bundle mode. The 463 MPLS label field is set to zero. The encapsulation is set via the BGP 464 Encapsulation extended community as described in section 3.1.3. 466 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 467 be set to the IPv4 or IPv6 address of the NVE. The remaining fields 468 in the route are set as per EVPN. 470 3.2 MPLS over GRE 472 The EVPN data-plane is modeled as an EVPN MPLS client layer sitting 473 over an MPLS PSN tunnel. Some of the EVPN functions (split-horizon, 474 aliasing and repair-path) are tied to the MPLS client layer. If MPLS 475 over GRE encapsulation is used, then the EVPN MPLS client layer can 476 be carried over an IP PSN tunnel transparently. Therefore, there is 477 no impact to the EVPN procedures and associated data-plane 478 operation. 480 The existing standards for MPLS over GRE encapsulation as defined by 481 [RFC4023] can be used for this purpose; however, when it is used in 482 conjunction with EVPN the key field MUST be present, and SHOULD be 483 used to provide a 32-bit entropy field. The Checksum and Sequence 484 Number fields are not needed and their corresponding C and S bits 485 MUST be set to zero. 487 4 EVPN with Multiple Data Plane Encapsulations 489 The use of the BGP Encapsulation extended community allows each PE in 490 a given EVPN to know whether the other PEs in that EVPN support MPLS 491 label stack, VXLAN, and/or NVGRE data plane encapsulations. I.e., PEs 492 in a given EVPN may support multiple data plane encapsulations. 494 If BGP Encapsulation extended community is not present, then the 495 default MPLS encapsulation (or statically configured encapsulation) 496 is used. However, if this attribute is present, then an ingress PE 497 can send a frame to an egress PE only if the set of encapsulations 498 advertised by the egress PE in the subject MAC Advertisement or Per 499 EVI Ethernet AD route, forms a non-empty intersection with the set of 500 encapsulations supported by the ingress PE, and it is at the 501 discretion of the ingress PE which encapsulation to choose from this 502 intersection. 504 An ingress node that uses shared multicast trees for sending 505 broadcast or multicast frames MUST maintain distinct trees for each 506 different encapsulation type. 508 It is the responsibility of the operator of a given EVPN to ensure 509 that all of the PEs in that EVPN support at least one common 510 encapsulation. If this condition is violated, it could result in 511 service disruption or failure. The use of the BGP Encapsulation 512 extended community provides a method to detect when this condition is 513 violated but the actions to be taken are at the discretion of the 514 operator and are outside the scope of this document. 516 5 NVE Residing in Hypervisor 518 When a PE and its CEs are co-located in the same physical device, 519 e.g., when the PE resides in a server and the CEs are its VMs, the 520 links between them are virtual and they typically share fate; i.e., 521 the subject CEs are typically not multi-homed or if they are multi- 522 homed, the multi-homing is a purely local matter to the server 523 hosting the VM, and need not be "visible" to any other PEs, and thus 524 does not require any specific protocol mechanisms. The most common 525 case of this is when the NVE resides in the hypervisor. 527 In the sub-sections that follow, we will discuss the impact on EVPN 528 procedures for the case when the NVE resides on the hypervisor and 529 the VXLAN or NVGRE encapsulation is used. 531 5.1 Impact on EVPN BGP Routes & Attributes for VXLAN/NVGRE Encapsulation 533 As discussed above, both [NVGRE] and [VXLAN] do not require the 534 tenant VLAN tag to be sent in BGP routes. Therefore, the 4-octet 535 Ethernet Tag field in the EVPN BGP routes can be used to represent 536 the globally significant value for VXLAN VNI or NVGRE VSID and MPLS 537 field can be used to represent the locally significant value for VNI 538 or VSID. 540 When the VXLAN VNI or NVGRE VSID is assumed to be a global value, one 541 might question the need for the Route Distinguisher (RD) in the EVPN 542 routes. In the scenario where all data centers are under a single 543 administrative domain, and there is a single global VNI/VSID space, 544 the RD MAY be set to zero in the EVPN routes. However, in the 545 scenario where different groups of data centers are under different 546 administrative domains, and these data centers are connected via one 547 or more backbone core providers as described in [NOV3-Framework], the 548 RD must be a unique value per EVI or per NVE as described in [EVPN]. 549 In other words, whenever there is more than one administrative domain 550 for global VNI or VSID, then a non-zero RD MUST be used, or whenever 551 the VNI or VSID value have local significance, then a non-zero RD 552 MUST be used. It is recommend to use a non-zero RD at all time. 554 When the NVEs reside on the hypervisor, the EVPN BGP routes and 555 attributes associated with multi-homing are no longer required. This 556 reduces the required routes and attributes to the following subset of 557 five out of the set of eight : 559 - MAC Advertisement Route 560 - Inclusive Multicast Ethernet Tag Route 561 - MAC Mobility Extended Community 562 - Default Gateway Extended Community 564 As mentioned in section 3.1.1, BGP Encapsulation extended community 565 as defined in [RFC5512] SHOULD be used along with MAC Advertisement 566 Route or Ethernet AD Route to indicate the supported encapsulations. 568 5.2 Impact on EVPN Procedures for VXLAN/NVGRE Encapsulation 570 When the NVEs reside on the hypervisors, the EVPN procedures 571 associated with multi-homing are no longer required. This limits the 572 procedures on the NVE to the following subset of the EVPN procedures: 574 1. Local learning of MAC addresses received from the VMs per section 575 10.1 of [EVPN]. 577 2. Advertising locally learned MAC addresses in BGP using the MAC 578 Advertisement routes. 580 3. Performing remote learning using BGP per Section 10.2 of [EVPN]. 582 4. Discovering other NVEs and constructing the multicast tunnels 583 using the Inclusive Multicast Ethernet Tag routes. 585 5. Handling MAC address mobility events per the procedures of Section 586 16 in [EVPN]. 588 6 NVE Residing in ToR Switch 590 In this section, we discuss the scenario where the NVEs reside in the 591 Top of Rack (ToR) switches AND the servers (where VMs are residing) 592 are multi-homed to these ToR switches. The multi-homing may operate 593 in All-Active or Active/Standby redundancy mode. If the servers are 594 single-homed to the ToR switches, then the scenario becomes similar 595 to that where the NVE resides in the hypervisor, as discussed in 596 Section 5, as far as the required EVPN functionality. 598 [EVPN] defines a set of BGP routes, attributes and procedures to 599 support multi-homing. We first describe these functions and 600 procedures, then discuss which of these are impacted by the 601 encapsulation (such as VXLAN or NVGRE) and what modifications are 602 required. 604 6.1 EVPN Multi-Homing Features 606 In this section, we will recap the multi-homing features of EVPN to 607 highlight the encapsulation dependencies. The section only describes 608 the features and functions at a high-level. For more details, the 609 reader is to refer to [EVPN]. 611 6.1.1 Multi-homed Ethernet Segment Auto-Discovery 613 EVPN NVEs (or PEs) connected to the same Ethernet Segment (e.g. the 614 same server via LAG) can automatically discover each other with 615 minimal to no configuration through the exchange of BGP routes. 617 6.1.2 Fast Convergence and Mass Withdraw 619 EVPN defines a mechanism to efficiently and quickly signal, to remote 620 NVEs, the need to update their forwarding tables upon the occurrence 621 of a failure in connectivity to an Ethernet segment (e.g., a link or 622 a port failure). This is done by having each NVE advertise an 623 Ethernet A-D Route per Ethernet segment for each locally attached 624 segment. Upon a failure in connectivity to the attached segment, the 625 NVE withdraws the corresponding Ethernet A-D route. This triggers all 626 NVEs that receive the withdrawal to update their next-hop adjacencies 627 for all MAC addresses associated with the Ethernet segment in 628 question. If no other NVE had advertised an Ethernet A-D route for 629 the same segment, then the NVE that received the withdrawal simply 630 invalidates the MAC entries for that segment. Otherwise, the NVE 631 updates the next-hop adjacencies to point to the backup NVE(s). 633 6.1.3 Split-Horizon 635 Consider a station that is multi-homed to two or more NVEs on an 636 Ethernet segment ES1, with all-active redundancy. If the station 637 sends a multicast, broadcast or unknown unicast packet to a 638 particular NVE, say NE1, then NE1 will forward that packet to all or 639 subset of the other NVEs in the EVPN instance. In this case the NVEs, 640 other than NE1, that the station is multi-homed to MUST drop the 641 packet and not forward back to the station. This is referred to as 642 "split horizon" filtering. 644 6.1.4 Aliasing and Backup-Path 646 In the case where a station is multi-homed to multiple NVEs, it is 647 possible that only a single NVE learns a set of the MAC addresses 648 associated with traffic transmitted by the station. This leads to a 649 situation where remote NVEs receive MAC advertisement routes, for 650 these addresses, from a single NVE even though multiple NVEs are 651 connected to the multi-homed station. As a result, the remote NVEs 652 are not able to effectively load-balance traffic among the NVEs 653 connected to the multi-homed Ethernet segment. This could be the 654 case, for e.g. when the NVEs perform data-path learning on the 655 access, and the load-balancing function on the station hashes traffic 656 from a given source MAC address to a single NVE. Another scenario 657 where this occurs is when the NVEs rely on control plane learning on 658 the access (e.g. using ARP), since ARP traffic will be hashed to a 659 single link in the LAG. 661 To alleviate this issue, EVPN introduces the concept of Aliasing. 662 This refers to the ability of an NVE to signal that it has 663 reachability to a given locally attached Ethernet segment, even when 664 it has learnt no MAC addresses from that segment. The Ethernet A-D 665 route per EVI is used to that end. Remote NVEs which receive MAC 666 advertisement routes with non-zero ESI SHOULD consider the MAC 667 address as reachable via all NVEs that advertise reachability to the 668 relevant Segment using Ethernet A-D routes with the same ESI and with 669 the Active-Standby flag reset. 671 Backup-Path is a closely related function, albeit it applies to the 672 case where the redundancy mode is Active/Standby. In this case, the 673 NVE signals that it has reachability to a given locally attached 674 Ethernet Segment using the Ethernet A-D route as well. Remote NVEs 675 which receive the MAC advertisement routes, with non-zero ESI, SHOULD 676 consider the MAC address as reachable via the advertising NVE. 678 Furthermore, the remote NVEs SHOULD install a Backup-Path, for said 679 MAC, to the NVE which had advertised reachability to the relevant 680 Segment using an Ethernet A-D route with the same ESI and with the 681 Active-Standby flag set. 683 6.1.5 DF Election 685 Consider a station that is a host or a VM that is multi-homed 686 directly to more than one NVE in an EVPN on a given Ethernet segment. 687 One or more Ethernet Tags may be configured on the Ethernet segment. 688 In this scenario only one of the PEs, referred to as the Designated 689 Forwarder (DF), is responsible for certain actions: 691 - Sending multicast and broadcast traffic, on a given Ethernet Tag on 692 a particular Ethernet segment, to the station. 694 - Flooding unknown unicast traffic (i.e. traffic for which an NVE 695 does not know the destination MAC address), on a given Ethernet Tag 696 on a particular Ethernet segment to the station, if the environment 697 requires flooding of unknown unicast traffic. 699 This is required in order to prevent duplicate delivery of multi- 700 destination frames to a multi-homed host or VM, in case of all-active 701 redundancy. 703 6.2 Impact on EVPN BGP Routes & Attributes 705 Since multi-homing is supported in this scenario, then the entire set 706 of BGP routes and attributes defined in [EVPN] are used. As discussed 707 in Section 3.1, the VSID or VNI is encoded in the Ethernet Tag field 708 of the routes if globally significant or in the MPLS label field if 709 locally significant. 711 As mentioned in section 3.1.1, BGP Encapsulation extended community 712 as defined in [RFC5512] SHOULD be used along with MAC Advertisement 713 Route or Ethernet AD Route to indicate the supported encapsulations. 715 6.3 Impact on EVPN Procedures 717 Two cases need to be examined here, depending on whether the NVEs are 718 operating in Active/Standby or in All-Active redundancy. 720 First, let's consider the case of Active/Standby redundancy, where 721 the hosts are multi-homed to a set of NVEs, however, only a single 722 NVE is active at a given point of time for a given VNI or VSID. In 723 this case, the Split-Horizon and Aliasing functions are not required 724 but other functions such as multi-homed Ethernet segment auto- 725 discovery, fast convergence and mass withdraw, repair path, and DF 726 election are required. In this case, the impact of the use of the 727 VXLAN/NVGRE encapsulation on the EVPN procedures is when the Backup- 728 Path function is supported, as discussed next: 730 In EVPN, the NVEs connected to a multi-homed site using 731 Active/Standby redundancy optionally advertise a VPN label, in the 732 Ethernet A-D Route per EVI, used to send traffic to the backup NVE in 733 the case where the primary NVE fails. In the case where VXLAN or 734 NVGRE encapsulation is used, some alternative means that does not 735 rely on MPLS labels is required to support Backup-Path. This is 736 discussed in Section 4.3.2 below. If the Backup-Path function is not 737 used, then the VXLAN/NVGRE encapsulation would have no impact on the 738 EVPN procedures. 740 Second, let's consider the case of All-Active redundancy. In this 741 case, out of the EVPN multi-homing features listed in section 4.1, 742 the use of the VXLAN or NVGRE encapsulation impacts the Split-Horizon 743 and Aliasing features, since those two rely on the MPLS client layer. 744 Given that this MPLS client layer is absent with these types of 745 encapsulations, alternative procedures and mechanisms are needed to 746 provide the required functions. Those are discussed in detail next. 748 6.3.1 Split Horizon 750 In EVPN, an MPLS label is used for split-horizon filtering to support 751 active/active multi-homing where an ingress ToR switch adds a label 752 corresponding to the site of origin (aka ESI MPLS Label) when 753 encapsulating the packet. The egress ToR switch checks the ESI MPLS 754 label when attempting to forward a multi-destination frame out an 755 interface, and if the label corresponds to the same site identifier 756 (ESI) associated with that interface, the packet gets dropped. This 757 prevents the occurrence of forwarding loops. 759 Since the VXLAN or NVGRE encapsulation does not include this ESI MPLS 760 label, other means of performing the split-horizon filtering function 761 MUST be devised. The following approach is recommended for split- 762 horizon filtering when VXLAN or NVGRE encapsulation is used. 764 Every NVE track the IP address(es) associated with the other NVE(s) 765 with which it has shared multi-homed Ethernet Segments. When the NVE 766 receives a multi-destination frame from the overlay network, it 767 examines the source IP address in the tunnel header (which 768 corresponds to the ingress NVE) and filters out the frame on all 769 local interfaces connected to Ethernet Segments that are shared with 770 the ingress NVE. With this approach, it is required that the ingress 771 NVE performs replication locally to all directly attached Ethernet 772 Segments (regardless of the DF Election state) for all flooded 773 traffic ingress from the access interfaces (i.e. from the hosts). 774 This approach is referred to as "Local Bias", and has the advantage 775 that only a single IP address needs to be used per NVE for split- 776 horizon filtering, as opposed to requiring an IP address per Ethernet 777 Segment per NVE. 779 In order to prevent unhealthy interactions between the split horizon 780 procedures defined in [EVPN] and the local bias procedures described 781 in this memo, a mix of MPLS over GRE encapsulations on the one hand 782 and VXLAN/NVGRE encapsulations on the other on a given Ethernet 783 Segment is prohibited. The use of the BGP Encapsulation extended 784 community provides a method to detect when this condition is violated 785 but the actions to be taken are at the discretion of the operator and 786 are outside the scope of this document. 788 6.3.2 Aliasing and Backup-Path 790 The Aliasing and the Backup-Path procedures for VXLAN/NVGRE 791 encapsulation is very similar to the ones for MPLS. In case of MPLS, 792 two different Ethernet AD routes are used for this purpose. The one 793 used for Aliasing has a VPN scope and carries a VPN label but the one 794 used for Backup-Path has Ethernet segment scope and doesn't carry any 795 VPN specific info (e.g., Ethernet Tag and MPLS label are set to 796 zero). The same two routes are used when VXLAN or NVGRE encapsulation 797 is used with the difference that when Ethernet AD route is used for 798 Aliasing with VPN scope, the Ethernet Tag field is set to VNI or VSID 799 to indicate VPN scope (and MPLS field may be set to a VPN label if 800 needed). 802 7 Support for Multicast 804 The E-VPN Inclusive Multicast BGP route is used to discover the 805 multicast tunnels among the endpoints associated with a given VXLAN 806 VNI or NVGRE VSID. The Ethernet Tag field of this route is used to 807 encode the VNI for VLXAN or VSID for NVGRE. The Originating router's 808 IP address field is set to the NVE's IP address. This route is tagged 809 with the PMSI Tunnel attribute, which is used to encode the type of 810 multicast tunnel to be used as well as the multicast tunnel 811 identifier. The tunnel encapsulation is encoded by adding the BGP 812 Encapsulation extended community as per section 3.1.1. The following 813 tunnel types as defined in [RFC6514] can be used in the PMSI tunnel 814 attribute for VXLAN/NVGRE: 816 + 3 - PIM-SSM Tree 817 + 4 - PIM-SM Tree 818 + 5 - BIDIR-PIM Tree 819 + 6 - Ingress Replication 821 Except for Ingress Replication, this multicast tunnel is used by the 822 PE originating the route for sending multicast traffic to other PEs, 823 and is used by PEs that receive this route for receiving the traffic 824 originated by CEs connected to the PE that originated the route. 826 In the scenario where the multicast tunnel is a tree, both the 827 Inclusive as well as the Aggregate Inclusive variants may be used. In 828 the former case, a multicast tree is dedicated to a VNI or VSID. 829 Whereas, in the latter, a multicast tree is shared among multiple 830 VNIs or VSIDs. This is done by having the NVEs advertise multiple 831 Inclusive Multicast routes with different VNI or VSID encoded in the 832 Ethernet Tag field, but with the same tunnel identifier encoded in 833 the PMSI Tunnel attribute. 835 8 Support for NVEs with data plane MAC learning 837 In an overlay network it possible to have a mix of NVEs, such that 838 only a subset of the NVEs are capable of participating in control 839 plane MAC learning via EVPN. The other subset of NVEs would perform 840 conversational MAC learning in data plane. It must be possible for 841 NVEs with this mixed capability to still be part of the same overlay 842 network. 844 If the administrative policy of an EVPN NVE requires for flooding of 845 unknown unicast, then the following procedures are not needed; 846 however, if the administrative policy of the EVPN NVE requires no 847 flooding of unknown unicast, then for such a mixed overlay network to 848 operate correctly, the following requirements MUST be met: 850 - When an NVE capable of doing control-plane MAC learning via EVPN 851 wants to send an unknown unicast frame, it MUST send it to a subset 852 of NVEs in the VNI that only have data plane MAC learning capability. 853 This can be achieved by creating a flood list for each VNI to carry 854 unknown unicast traffic, which only the subset of NVEs with data 855 plane MAC learning are part of. Section 8.2 describes the procedure 856 to accomplish this. 858 - Broadcast traffic MUST be sent to all NVEs in the VNI regardless of 859 the MAC learning capability. A separate flood list for each VNI to 860 carry broadcast traffic can be created for this, and all NVEs in the 861 VNI would be part of this flood list. Section 8.2 describes the 862 procedure to accomplish this. 864 - When an NVE capable of only data plane MAC learning wants to send 865 an unknown unicast frame, it MUST send it to all NVEs in the VNI. 866 This can be achieved by flooding the unknown unicast frame in the 867 broadcast flood list (as described earlier). 869 8.1 Advertising NVE capabilities 871 BGP Encapsulation extended community is used to signal NVE 872 capabilities. NVE capabilities are used to build different types of 873 flood lists in the broadcast domain for optimal forwarding in the 874 case of NVEs with mixed capabilities, as described in section 8. The 875 reserved field of the BGP Encapsulation extended community is 876 repurposed to indicate NVE capabilities as following: 878 0 1 2 3 879 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 880 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 881 | 0x03 | 0x0c | Reserved |U| 882 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 883 | Reserved | Tunnel Type | 884 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 886 U bit indicates that the NVE must be included in unknown unicast 887 flood list 889 The Reserved fields must be set to zero and ignored on receipt. 891 8.2 Advertising flood lists for ingress replication 893 Flooding of unknown unicast, broadcast and multicast can either be 894 achieved by using multicast trees in the underlay or using ingress 895 replication. If IP multicast is used for flooding, separate flood 896 lists, as described in section 8, can be created by using separate IP 897 multicast groups for different flood lists. If ingress replication is 898 used for flooding, then the EVPN capable NVEs must maintain separate 899 flood lists depending on advertised NVE capability. Either way, there 900 is a need to signal which NVEs are part of which flood lists. This 901 section describes enhancements to BGP signaling required to achieve 902 this. 904 The EVPN Inclusive Multicast Route along with NVE capabilities as 905 described in section 8.1 can be used to build different flood lists. 906 The Inclusive Multicast Route is encoded as follows: The Ethernet Tag 907 field is set to the VNI for VXLAN and VSID for NVGRE. The 908 Originator's IP address field is set to the NVE's IP address. The 909 Next Hop field of the MP_REACH_NLRI attribute of the route is set to 910 NVE's IP address. The Inclusive Multicast route is tagged with the 911 PMSI tunnel attribute. The BGP Encapsulation extended community is 912 included with U, B or K bit set as described in section 8.1 to enable 913 an NVE to be part of a specific flood list depending on its 914 capabilities. 916 9 Inter-AS 918 For inter-AS operation, two scenarios must be considered: 920 - Scenario 1: The tunnel endpoint IP addresses are public 921 - Scenario 2: The tunnel endpoint IP addresses are private 923 In the first scenario, inter-AS operation is straight-forward and 924 follows existing BGP inter-AS procedures. However, in the first 925 scenario where the tunnel endpoint IP addresses are public, there may 926 be security concern regarding the distribution of these addresses 927 among different ASes. This security concern is one of the main 928 reasons for having the so called inter-AS "option-B" in MPLS VPN 929 solutions such as EVPN. 931 The second scenario is more challenging, because the absence of the 932 MPLS client layer from the VXLAN encapsulation creates a situation 933 where the ASBR has no fully qualified indication within the tunnel 934 header as to where the tunnel endpoint resides. To elaborate on this, 935 recall that with MPLS, the client layer labels (i.e. the VPN labels) 936 are downstream assigned. As such, this label implicitly has a 937 connotation of the tunnel endpoint, and it is sufficient for the ASBR 938 to look up the client layer label in order to identify the label 939 translation required as well as the tunnel endpoint to which a given 940 packet is being destined. With the VXLAN encapsulation, the VNI is 941 globally assigned and hence is shared among all endpoints. The 942 destination IP address is the only field which identifies the tunnel 943 endpoint in the tunnel header, and this address is privately managed 944 by every data center network. Since the tunnel address is allocated 945 out of a private address pool, then we either need to do a lookup 946 based on VTEP IP address in context of a VRF (e.g., use IP-VPN) or 947 terminate the VXLAN tunnel and do a lookup based on the tenant's MAC 948 address to identify the egress tunnel on the ASBR. This effectively 949 mandates that the ASBR to either run another overlay solution such as 950 IP-VPN over MPLS/IP core network or to be aware of the MAC addresses 951 of all VMs in its local AS, at the very least. 953 If VNIs/VSIDs have local significance, then the inter-AS operation 954 can be simplified to that of MPLS and thus MPLS inter-AS option B and 955 C can be leveraged in here. That's why the use of local significance 956 VNIs/VSIDs (e.g., MPLS labels) are recommended for inter-AS operation 957 of DC networks without gateways. 959 10 Acknowledgement 961 The authors would like to thank David Smith, John Mullooly, Thomas 962 Nadeau for their valuable comments and feedback. 964 11 Security Considerations 966 This document uses IP-based tunnel technologies to support data 967 plane transport. Consequently, the security considerations of those 968 tunnel technologies apply. This document defines support for [VXLAN] 969 and [NVGRE]. The security considerations from those documents as well 970 as [RFC4301] apply to the data plane aspects of this document. 972 As with [RFC5512], any modification of the information that is used 973 to form encapsulation headers, to choose a tunnel type, or to choose 974 a particular tunnel for a particular payload type may lead to user 975 data packets getting misrouted, misdelivered, and/or dropped. 977 More broadly, the security considerations for the transport of IP 978 reachability information using BGP are discussed in [RFC4271] and 979 [RFC4272], and are equally applicable for the extensions described 980 in this document. 982 If the integrity of the BGP session is not itself protected, then an 983 imposter could mount a denial-of-service attack by establishing 984 numerous BGP sessions and forcing an IPsec SA to be created for each 985 one. However, as such an imposter could wreak havoc on the entire 986 routing system, this particular sort of attack is probably not of 987 any special importance. 989 It should be noted that a BGP session may itself be transported over 990 an IPsec tunnel. Such IPsec tunnels can provide additional security 991 to a BGP session. The management of such IPsec tunnels is outside 992 the scope of this document. 994 12 IANA Considerations 996 13 References 998 11.1 Normative References 1000 [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate 1001 Requirement Levels", BCP 14, RFC 2119, March 1997. 1003 [RFC4271] Y. Rekhter, Ed., T. Li, Ed., S. Hares, Ed., "A Border 1004 Gateway Protocol 4 (BGP-4)", January 2006. 1006 [RFC4272] S. Murphy, "BGP Security Vulnerabilities Analysis.", 1007 January 2006. 1009 [RFC4301] S. Kent, K. Seo., "Security Architecture for the 1010 Internet Protocol.", December 2005. 1012 [RFC5512] Mohapatra, P. and E. Rosen, "The BGP Encapsulation 1013 Subsequent Address Family Identifier (SAFI) and the BGP 1014 Tunnel Encapsulation Attribute", RFC 5512, April 2009. 1016 11.2 Informative References 1018 [EVPN-REQ] Sajassi et al., "Requirements for Ethernet VPN (EVPN)", 1019 draft-ietf-l2vpn-evpn-req-01.txt, work in progress, October 21, 2012. 1021 [NVGRE] Sridhavan, M., et al., "NVGRE: Network Virtualization using 1022 Generic Routing Encapsulation", draft-sridharan-virtualization-nvgre- 1023 01.txt, July 8, 2012. 1025 [VXLAN] Dutt, D., et al, "VXLAN: A Framework for Overlaying 1026 Virtualized Layer 2 Networks over Layer 3 Networks", draft- 1027 mahalingam-dutt-dcops-vxlan-02.txt, August 22, 2012. 1029 [EVPN] Sajassi et al., "BGP MPLS Based Ethernet VPN", draft-ietf- 1030 l2vpn-evpn-02.txt, work in progress, February, 2012. 1032 [Problem-Statement] Narten et al., "Problem Statement: Overlays for 1033 Network Virtualization", draft-ietf-nvo3-overlay-problem-statement- 1034 01, September 2012. 1036 [L3VPN-ENDSYSTEMS] Marques et al., "BGP-signaled End-system IP/VPNs", 1037 draft-ietf-l3vpn-end-system, work in progress, October 2012. 1039 [NOV3-FRWK] Lasserre et al., "Framework for DC Network 1040 Virtualization", draft-ietf-nvo3-framework-01.txt, work in progress, 1041 October 2012. 1043 Authors' Addresses 1045 Ali Sajassi 1046 Cisco 1047 Email: sajassi@cisco.com 1049 John Drake 1050 Juniper Networks 1051 Email: jdrake@juniper.net 1052 Nabil Bitar 1053 Verizon Communications 1054 Email : nabil.n.bitar@verizon.com 1056 Aldrin Isaac 1057 Bloomberg 1058 Email: aisaac71@bloomberg.net 1060 James Uttaro 1061 AT&T 1062 Email: uttaro@att.com 1064 Wim Henderickx 1065 Alcatel-Lucent 1066 e-mail: wim.henderickx@alcatel-lucent.com 1068 Ravi Shekhar 1069 Juniper Networks 1070 Email: rshekhar@juniper.net 1072 Samer Salam 1073 Cisco 1074 Email: ssalam@cisco.com 1076 Keyur Patel 1077 Cisco 1078 Email: Keyupate@cisco.com 1080 Dhananjaya Rao 1081 Cisco 1082 Email: dhrao@cisco.com 1084 Samir Thoria 1085 Cisco 1086 Email: sthoria@cisco.com