idnits 2.17.1 draft-sd-l2vpn-evpn-overlay-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 5 instances of too long lines in the document, the longest one being 2 characters in excess of 72. ** There are 12 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 18, 2014) is 3593 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2119' is mentioned on line 169, but not defined == Missing Reference: 'GRE' is mentioned on line 304, but not defined == Missing Reference: 'RFC4023' is mentioned on line 540, but not defined == Missing Reference: 'NOV3-Framework' is mentioned on line 606, but not defined == Missing Reference: 'RFC6514' is mentioned on line 865, but not defined == Unused Reference: 'KEYWORDS' is defined on line 971, but no explicit reference was found in the text == Unused Reference: 'NOV3-FRWK' is defined on line 1010, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 4272 ** Obsolete normative reference: RFC 5512 (Obsoleted by RFC 9012) == Outdated reference: A later version (-07) exists of draft-ietf-l2vpn-evpn-req-01 == Outdated reference: A later version (-08) exists of draft-sridharan-virtualization-nvgre-01 == Outdated reference: A later version (-09) exists of draft-mahalingam-dutt-dcops-vxlan-02 == Outdated reference: A later version (-11) exists of draft-ietf-l2vpn-evpn-02 == Outdated reference: A later version (-04) exists of draft-ietf-nvo3-overlay-problem-statement-01 == Outdated reference: A later version (-09) exists of draft-ietf-nvo3-framework-01 Summary: 4 errors (**), 0 flaws (~~), 14 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 L2VPN Workgroup A. Sajassi (Editor) 3 INTERNET-DRAFT Cisco 4 Intended Status: Standards Track 5 J. Drake (Editor) 6 Y. Rekhter Juniper 7 R. Shekhar 8 B. Schliesser Nabil Bitar 9 Juniper Verizon 11 S. Salam Aldrin Isaac 12 K. Patel Bloomberg 13 D. Rao 14 S. Thoria James Uttaro 15 Cisco AT&T 17 L. Yong W. Henderickx 18 Huawei Alcatel-Lucent 20 D. Cai 21 S. Sinha 22 Cisco 24 Expires: December 18, 2014 June 18, 2014 26 A Network Virtualization Overlay Solution using EVPN 27 draft-sd-l2vpn-evpn-overlay-03 29 Abstract 31 This document describes how EVPN can be used as an NVO solution and 32 explores the various tunnel encapsulation options over IP and their 33 impact on the EVPN control-plane and procedures. In particular, the 34 following encapsulation options are analyzed: MPLS over GRE, VXLAN, 35 and NVGRE. 37 Status of this Memo 39 This Internet-Draft is submitted to IETF in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF), its areas, and its working groups. Note that 44 other groups may also distribute working documents as 45 Internet-Drafts. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 The list of current Internet-Drafts can be accessed at 53 http://www.ietf.org/1id-abstracts.html 55 The list of Internet-Draft Shadow Directories can be accessed at 56 http://www.ietf.org/shadow.html 58 Copyright and License Notice 60 Copyright (c) 2012 IETF Trust and the persons identified as the 61 document authors. All rights reserved. 63 This document is subject to BCP 78 and the IETF Trust's Legal 64 Provisions Relating to IETF Documents 65 (http://trustee.ietf.org/license-info) in effect on the date of 66 publication of this document. Please review these documents 67 carefully, as they describe your rights and restrictions with respect 68 to this document. Code Components extracted from this document must 69 include Simplified BSD License text as described in Section 4.e of 70 the Trust Legal Provisions and are provided without warranty as 71 described in the Simplified BSD License. 73 Table of Contents 75 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 76 2 Specification of Requirements . . . . . . . . . . . . . . . . . 5 77 3 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 78 4 EVPN Features . . . . . . . . . . . . . . . . . . . . . . . . . 6 79 5 Encapsulation Options for EVPN Overlays . . . . . . . . . . . . 7 80 5.1 VXLAN/NVGRE Encapsulation . . . . . . . . . . . . . . . . . 7 81 5.1.1 Virtual Identifiers Scope . . . . . . . . . . . . . . . 8 82 5.1.1.1 Data Center Interconnect with Gateway . . . . . . . 8 83 5.1.1.2 Data Center Interconnect without Gateway . . . . . . 9 84 5.1.2 Virtual Identifiers to EVI Mapping . . . . . . . . . . . 9 85 5.1.2.1 Auto Derivation of RT . . . . . . . . . . . . . . . 10 86 5.1.3 Constructing EVPN BGP Routes . . . . . . . . . . . . . 11 87 5.2 MPLS over GRE . . . . . . . . . . . . . . . . . . . . . . . 12 88 6 EVPN with Multiple Data Plane Encapsulations . . . . . . . . . 13 89 7 NVE Residing in Hypervisor . . . . . . . . . . . . . . . . . . 13 90 7.1 Impact on EVPN BGP Routes & Attributes for VXLAN/NVGRE 91 Encapsulation . . . . . . . . . . . . . . . . . . . . . . . 14 92 7.2 Impact on EVPN Procedures for VXLAN/NVGRE Encapsulation . . 14 94 8 NVE Residing in ToR Switch . . . . . . . . . . . . . . . . . . 15 95 8.1 EVPN Multi-Homing Features . . . . . . . . . . . . . . . . 15 96 8.1.1 Multi-homed Ethernet Segment Auto-Discovery . . . . . . 16 97 8.1.2 Fast Convergence and Mass Withdraw . . . . . . . . . . . 16 98 8.1.3 Split-Horizon . . . . . . . . . . . . . . . . . . . . . 16 99 8.1.4 Aliasing and Backup-Path . . . . . . . . . . . . . . . . 16 100 8.1.5 DF Election . . . . . . . . . . . . . . . . . . . . . . 17 101 8.2 Impact on EVPN BGP Routes & Attributes . . . . . . . . . . . 17 102 8.3 Impact on EVPN Procedures . . . . . . . . . . . . . . . . . 17 103 8.3.1 Split Horizon . . . . . . . . . . . . . . . . . . . . . 18 104 8.3.2 Aliasing and Backup-Path . . . . . . . . . . . . . . . . 19 105 9 Support for Multicast . . . . . . . . . . . . . . . . . . . . . 19 106 10 Inter-AS . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 107 11 Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 21 108 12 Security Considerations . . . . . . . . . . . . . . . . . . . 21 109 13 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 110 14 References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 111 14.1 Normative References . . . . . . . . . . . . . . . . . . . 22 112 14.2 Informative References . . . . . . . . . . . . . . . . . . 22 113 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 115 1 Introduction 117 In the context of this document, a Network Virtualization Overlay 118 (NVO) is a solution to address the requirements of a multi-tenant 119 data center, especially one with virtualized hosts, e.g., Virtual 120 Machines (VMs). The key requirements of such a solution, as described 121 in [Problem-Statement], are: 123 - Isolation of network traffic per tenant 125 - Support for a large number of tenants (tens or hundreds of 126 thousands) 128 - Extending L2 connectivity among different VMs belonging to a given 129 tenant segment (subnet) across different PODs within a data center or 130 between different data centers 132 - Allowing a given VM to move between different physical points of 133 attachment within a given L2 segment 135 The underlay network for NVO solutions is assumed to provide IP 136 connectivity between NVO endpoints (NVEs). 138 This document describes how Ethernet VPN (EVPN) can be used as an NVO 139 solution and explores applicability of EVPN functions and procedures. 140 In particular, it describes the various tunnel encapsulation options 141 for EVPN over IP, and their impact on the EVPN control-plane and 142 procedures for two main scenarios: 144 a) when the NVE resides in the hypervisor, and 145 b) when the NVE resides in a ToR device 147 Note that the use of EVPN as an NVO solution does not necessarily 148 mandate that the BGP control-plane be running on the NVE. For such 149 scenarios, it is still possible to leverage the EVPN solution by 150 using XMPP, or alternative mechanisms, to extend the control-plane to 151 the NVE as discussed in [L3VPN-ENDSYSTEMS]. 153 The possible encapsulation options for EVPN overlays that are 154 analyzed in this document are: 156 - VXLAN and NVGRE 157 - MPLS over GRE 159 Before getting into the description of the different encapsulation 160 options for EVPN over IP, it is important to highlight the EVPN 161 solution's main features, how those features are currently supported, 162 and any impact that the encapsulation has on those features. 164 2 Specification of Requirements 166 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 167 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 168 document are to be interpreted as described in [RFC2119]. 170 3 Terminology 172 NVO: Network Virtualization Overlay 174 NVE: Network Virtualization Endpoint 176 VNI: Virtual Network Identifier (for VxLAN) 178 VSID: VIrtual Subnet Identifier (for NVGRE) 180 EVPN: Ethernet VPN 182 EVI: An EVPN instance spanning across the PEs participating in that 183 EVPN 185 MAC-VRF: A Virtual Routing and Forwarding table for MAC addresses on 186 a PE for an EVI 188 Ethernet Segment Identifier (ESI): If a CE is multi-homed to two or 189 more PEs, the set of Ethernet links that attaches the CE to the PEs 190 is an 'Ethernet segment'. Ethernet segments MUST have a unique non- 191 zero identifier, the 'Ethernet Segment Identifier'. 193 Ethernet Tag: An Ethernet Tag identifies a particular broadcast 194 domain, e.g., a VLAN. An EVPN instance consists of one or more 195 broadcast domains. Ethernet tag(s) are assigned to the broadcast 196 domains of a given EVPN instance by the provider of that EVPN, and 197 each PE in that EVPN instance performs a mapping between broadcast 198 domain identifier(s) understood by each of its attached CEs and the 199 corresponding Ethernet tag. 201 Single-Active Multihoming: When a device or a network is multihomed 202 to a group of two or more PEs and when only a single PE in such a 203 redundancy group can forward traffic to/from the multihomed device or 204 network for a given VLAN, such multihoming is referred to as "Single- 205 Active" 207 All-Active Multihoming: When a device is multihomed to a group of two 208 or more PEs and when all PEs in such redundancy group can forward 209 traffic to/from the multihomed device or network for a given VLAN, 210 such multihoming is referred to as "All-Active". 212 4 EVPN Features 214 EVPN was originally designed to support the requirements detailed in 215 [EVPN-REQ] and therefore has the following attributes which directly 216 address control plane scaling and ease of deployment issues. 218 1) Control plane traffic is distributed with BGP and Broadcast and 219 Multicast traffic is sent using a shared multicast tree or with 220 ingress replication. 222 2) Control plane learning is used for MAC (and IP) addresses instead 223 of data plane learning. The latter requires the flooding of unknown 224 unicast and ARP frames; whereas, the former does not require any 225 flooding. 227 3) Route Reflector is used to reduce a full mesh of BGP sessions 228 among PE devices to a single BGP session between a PE and the RR. 229 Furthermore, RR hierarchy can be leveraged to scale the number BGP 230 routes on the RR. 232 4) Auto-discovery via BGP is used to discover PE devices 233 participating in a given VPN, PE devices participating in a given 234 redundancy group, tunnel encapsulation types, multicast tunnel type, 235 multicast members, etc. 237 5) All-Active multihoming is used. This allows a given customer 238 device (CE) to have multiple links to multiple PEs, and traffic 239 to/from that CE fully utilizes all of these links. This set of links 240 is termed an Ethernet Segment (ES). 242 6) When a link between a CE and a PE fails, the PEs for that EVI are 243 notified of the failure via the withdrawal of a single EVPN route. 244 This allows those PEs to remove the withdrawing PE as a next hop for 245 every MAC address associated with the failed link. This is termed 246 'mass withdrawal'. 248 7) BGP route filtering and constrained route distribution are 249 leveraged to ensure that the control plane traffic for a given EVI is 250 only distributed to the PEs in that EVI. 252 8) When a 802.1Q interface is used between a CE and a PE, each of the 253 VLAN ID (VID) on that interface can be mapped onto a bridge domain 254 (for upto 4094 such bridge domains). All these bridge domains can 255 also be mapped onto a single EVI (in case of VLAN-aware bundle 256 service). 258 9) VM Mobility mechanisms ensure that all PEs in a given EVI know 259 the ES with which a given VM, as identified by its MAC and IP 260 addresses, is currently associated. 262 10) Route Targets are used to allow the operator (or customer) to 263 define a spectrum of logical network topologies including mesh, hub & 264 spoke, and extranets (e.g., a VPN whose sites are owned by different 265 enterprises), without the need for proprietary software or the aid of 266 other virtual or physical devices. 268 11) Because the design goal for NVO is millions of instances per 269 common physical infrastructure, the scaling properties of the control 270 plane for NVO are extremely important. EVPN and the extensions 271 described herein, are designed with this level of scalability in 272 mind. 274 5 Encapsulation Options for EVPN Overlays 276 5.1 VXLAN/NVGRE Encapsulation 278 Both VXLAN and NVGRE are examples of technologies that provide a data 279 plane encapsulation which is used to transport a packet over the 280 common physical IP infrastructure between NVEs, VXLAN Tunnel End 281 Point (VTEPs) in VXLAN and Network Virtualization Endpoint (NVEs) in 282 NVGRE. Both of these technologies include the identifier of the 283 specific NVO instance, Virtual Network Identifier (VNI) in VXLAN and 284 Virtual Subnet Identifier (VSID), NVGRE, in each packet. 286 Note that a Provider Edge (PE) is equivalent to a VTEP/NVE. 288 [VXLAN] encapsulation is based on UDP, with an 8-byte header 289 following the UDP header. VXLAN provides a 24-bit VNI, which 290 typically provides a one-to-one mapping to the tenant VLAN ID, as 291 described in [VXLAN]. In this scenario, the VTEP does not include an 292 inner VLAN tag on frame encapsulation, and discards decapsulated 293 frames with an inner VLAN tag. This mode of operation in [VXLAN] maps 294 to VLAN Based Service in [EVPN], where a tenant VLAN ID gets mapped 295 to an EVPN instance (EVI). 297 [VXLAN] also provides an option of including an inner VLAN tag in the 298 encapsulated frame, if explicitly configured at the VTEP. This mode 299 of operation can either map to VLAN Based Service or VLAN Bundle 300 Service in [EVPN] because inner VLAN tag is not used for lookup by 301 the disposition PE when performing VXLAN decapsulation as described 302 in section 6 of [VXLAN]. 304 [NVGRE] encapsulation is based on [GRE] and it mandates the inclusion 305 of the optional GRE Key field which carries the VSID. There is a one- 306 to-one mapping between the VSID and the tenant VLAN ID, as described 307 in [NVGRE] and the inclusion of an inner VLAN tag is prohibited. This 308 mode of operation in [NVGRE] maps to VLAN Based Service in [EVPN]. 310 As described in the next section there is no change to the encoding 311 of EVPN routes to support VXLAN or NVGRE encapsulation except for the 312 use of BGP Encapsulation extended community. However, there is 313 potential impact to the EVPN procedures depending on where the NVE is 314 located (i.e., in hypervisor or TOR) and whether multi-homing 315 capabilities are required. 317 5.1.1 Virtual Identifiers Scope 319 Although VNI or VSID are defined as 24-bit globally unique values, 320 there are scenarios in which it is desirable to use a locally 321 significant value for VNI or VSID, especially in the context of data 322 center interconnect: 324 5.1.1.1 Data Center Interconnect with Gateway 326 In the case where NVEs in different data centers need to be 327 interconnected, and the NVEs need to use VNIs or VSIDs as a globally 328 unique identifiers within a data center, then a Gateway needs to be 329 employed at the edge of the data center network. This is because the 330 Gateway will provide the functionality of translating the VNI or VSID 331 when crossing network boundaries, which may align with operator span 332 of control boundaries. As an example, consider the network of Figure 333 1 below. Assume there are three network operators: one for each of 334 the DC1, DC2 and WAN networks. The Gateways at the edge of the data 335 centers are responsible for translating the VNIs / VSIDs between the 336 values used in each of the data center networks and the values used 337 in the WAN. 339 +--------------+ 340 | | 341 +---------+ | WAN | +---------+ 342 +----+ | +---+ +----+ +----+ +---+ | +----+ 343 |NVE1|--| | | |WAN | |WAN | | | |--|NVE3| 344 +----+ |IP |GW |--|Edge| |Edge|--|GW | IP | +----+ 345 +----+ |Fabric +---+ +----+ +----+ +---+ Fabric | +----+ 346 |NVE2|--| | | | | |--|NVE4| 347 +----+ +---------+ +--------------+ +---------+ +----+ 349 |<------ DC 1 ------> <------ DC2 ------>| 351 Figure 1: Data Center Interconnect with Gateway 353 5.1.1.2 Data Center Interconnect without Gateway 355 In the case where NVEs in different data centers need to be 356 interconnected, and the NVEs need to use locally assigned VNIs or 357 VSIDs (e.g., as MPLS labels), then there may be no need to employ 358 Gateways at the edge of the data center network. More specifically, 359 the VNI or VSID value that is used by the transmitting NVE is 360 allocated by the NVE that is receiving the traffic (in other words, 361 this is a "downstream assigned" MPLS label). This allows the VNI or 362 VSID space to be decoupled between different data center networks 363 without the need for a dedicated Gateway at the edge of the data 364 centers. 366 +--------------+ 367 | | 368 +---------+ | WAN | +---------+ 369 +----+ | | +----+ +----+ | | +----+ 370 |NVE1|--| | |WAN | |WAN | | |--|NVE3| 371 +----+ |IP Fabric|---|Edge| |Edge|--|IP Fabric| +----+ 372 +----+ | | +----+ +----+ | | +----+ 373 |NVE2|--| | | | | |--|NVE4| 374 +----+ +---------+ +--------------+ +---------+ +----+ 376 |<------ DC 1 -----> <---- DC2 ------>| 378 Figure 2: Data Center Interconnect without Gateway 380 5.1.2 Virtual Identifiers to EVI Mapping 382 When the EVPN control plane is used in conjunction with VXLAN or 383 NVGRE, two options for mapping the VXLAN VNI or NVGRE VSID to an EVI 384 are possible: 386 1. Option 1: Single Subnet per EVI 388 In this option, a single subnet represented by a VNI or VSID is 389 mapped to a unique EVI. As such, a BGP RD and RT is needed per VNI / 390 VSID on every VTEP. The advantage of this model is that it allows the 391 BGP RT constraint mechanisms to be used in order to limit the 392 propagation and import of routes to only the VTEPs that are 393 interested in a given VNI or VSID. The disadvantage of this model may 394 be the provisioning overhead if RD and RT are not derived 395 automatically from VNI or VSID. 397 In this option, the MAC-VRF table is identified by the RT in the 398 control plane and by the VNI or VSID for the data-plane. In this 399 option, the specific the MAC-VRF table corresponds to only a single 400 bridge domain (e.g., a single subnet). 402 2. Option 2: Multiple Subnets per EVI 404 In this option, multiple subnets each represented by a unique VNI or 405 VSID are mapped to a unique EVI. For example, if a tenant has 406 multiple segments/subnets each represented by a VNI or VSID, then all 407 the VNIs (or VSIDs) for that tenant are mapped to a single EVI - 408 e.g., the EVI in this case represents the tenant and not a subnet . 409 The advantage of this model is that it doesn't require the 410 provisioning of RD/RT per VNI or VSID. However, this is a moot point 411 if option 1 with if auto-derivation is used. The disadvantage of this 412 model is that routes would be imported by VTEPs that may not be 413 interested in a given VNI or VSID. 415 In this option the MAC-VRF table is identified by the RT in the 416 control plane and a specific bridge domain for that MAC-VRF is 417 identified by the in the control plane. In this 418 option, the VNI/VSID in the data-plane is sufficient to identify a 419 specific bridge domain - e.g., no need to do a lookup based on 420 VNI/VSID field and Ethernet Tag ID fields to identify a bridge 421 domain. 423 5.1.2.1 Auto Derivation of RT 425 When the option of a single VNI or VSID per EVI is used, it is 426 important to auto-derive RT for EVPN BGP routes in order to simplify 427 configuration for data center operations. RD can be derived easily as 428 described in [EVPN] and RT can be auto-derived as described next. 430 Since a gateway PE as depicted in figure-1 participates in both the 431 DCN and WAN BGP sessions, it is important that when RT values are 432 auto-derived for VNIs (or VSIDs), there is no conflict in RT spaces 433 between DCN and WAN networks assuming that both are operating within 434 the same AS. Also, there can be scenarios where both VXLAN and NVGRE 435 encapsulations may be needed within the same DCN and their 436 corresponding VNIs and VSIDs are administered independently which 437 means VNI and VSID spaces can overlap. In order to ensure that no 438 such conflict in RT spaces arises, RT values for DCNs are auto- 439 derived as follow: 441 0 1 2 3 4 442 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 0 443 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++ 444 | AS # |A| TYPE| D-ID | Service Instance ID| 445 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++ 447 - 2 bytes of global admin field of the RT is set to the AS number. 449 - Three least significant bytes of the local admin field of the RT is 450 set to the VNI or VSID, I-SID, or VID. The most significant bit of 451 the local admin field of the RT is set as follow: 452 0: auto-derived 453 1: manually-derived 455 - The next 3 bits of the most significant byte of the local admin 456 field of the RT identifies the space in which the other 3 bytes are 457 defined. The following spaces are defined: 458 0 : VID 459 1 : VXLAN 460 2 : NVGRE 461 3 : I-SID 462 4 : EVI 463 5 : dual-VID 465 - The remaining 4 bits of the most significant byte of the local 466 admin field of the RT identifies the domain-id. The default value of 467 domain-id is zero indicating that only a single numbering space exist 468 for a given technology. However, if there are more than one number 469 space exist for a given technology (e.g., overlapping VXLAN spaces), 470 then each of the number spaces need to be identify by their 471 corresponding domain-id starting from 1. 473 5.1.3 Constructing EVPN BGP Routes 475 In EVPN, an MPLS label is distributed by the egress PE via the EVPN 476 control plane and is placed in the MPLS header of a given packet by 477 the ingress PE. This label is used upon receipt of that packet by the 478 egress PE for disposition of that packet. This is very similar to the 479 use of the VNI or VSID by the egress VTEP or NVE, respectively, with 480 the difference being that an MPLS label has local significance while 481 a VNI or VSID typically has global significance. Accordingly, and 482 specifically to support the option of locally assigned VNIs, the MPLS 483 label field in the MAC Advertisement, Ethernet AD per EVI, and 484 Inclusive Multicast Ethernet Tag routes is used to carry the VNI or 485 VSID. For the balance of this memo, the MPLS label field will be 486 referred to as the VNI/VSID field. The VNI/VSID field is used for 487 both locally and globally assigned VNIs or VSIDs. 489 For the VNI based mode (a single VNI per EVI), the Ethernet Tag field 490 in the MAC Advertisement, Ethernet AD per EVI, and Inclusive 491 Multicast route MUST be set to zero just as in the VLAN Based service 492 in [EVPN]. For the VNI bundle mode (multiple VNIs per EVI with a 493 single bridge domain), the Ethernet Tag field in the MAC 494 Advertisement, Ethernet AD per EVI, and Inclusive Multicast Ethernet 495 Tag routes MUST be set to zero just as in the VLAN Bundle service in 496 [EVPN]. 498 For the VNI-aware bundle mode (multiple VNIs per EVI each with its 499 own bridge domain), the Ethernet Tag field in the MAC Advertisement, 500 Ethernet AD per EVI, and Inclusive Multicast route MUST identify a 501 bridge domain within an EVI and the set of Ethernet Tags for that EVI 502 needs to be configured consistently on all PEs within that EVI. The 503 value advertised in the Ethernet Tag field MAY be a VNI as long as it 504 matches the existing semantics of the Ethernet Tag, i.e., it 505 identifies a bridge domain within an EVI and the set of VNIs are 506 configured consistently on each PE in that EVI. 508 In order to indicate that which type of data plane encapsulation 509 (i.e., VXLAN, NVGRE, MPLS, or MPLS in GRE) is to be used, the BGP 510 Encapsulation extended community defined in [RFC5512] is included 511 with all EVPN routes (i.e. MAC Advertisement, Ethernet AD per EVI, 512 Ethernet AD per ESI, Inclusive Multicast Ethernet Tag, and Ethernet 513 Segment) advertised by an egress PE. Four new values will be defined 514 to extend the list of encapsulation types defined in [RFC5512]: 516 + TBD (IANA assigned) - VXLAN Encapsulation 517 + TBD (IANA assigned) - NVGRE Encapsulation 518 + TBD (IANA assigned) - MPLS Encapsulation 519 + TBD (IANA assigned) - MPLS in GRE Encapsulation 521 If the BGP Encapsulation extended community is not present, then the 522 default MPLS encapsulation or a statically configured encapsulation 523 is assumed. 525 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 526 be set to the IPv4 or IPv6 address of the NVE. The remaining fields 527 in each route are set as per [EVPN]. 529 5.2 MPLS over GRE 531 The EVPN data-plane is modeled as an EVPN MPLS client layer sitting 532 over an MPLS PSN tunnel. Some of the EVPN functions (split-horizon, 533 aliasing and repair-path) are tied to the MPLS client layer. If MPLS 534 over GRE encapsulation is used, then the EVPN MPLS client layer can 535 be carried over an IP PSN tunnel transparently. Therefore, there is 536 no impact to the EVPN procedures and associated data-plane 537 operation. 539 The existing standards for MPLS over GRE encapsulation as defined by 540 [RFC4023] can be used for this purpose; however, when it is used in 541 conjunction with EVPN the key field SHOULD be present, and SHOULD be 542 used to provide a 32-bit entropy field. The Checksum and Sequence 543 Number fields are not needed and their corresponding C and S bits 544 MUST be set to zero. 546 6 EVPN with Multiple Data Plane Encapsulations 548 The use of the BGP Encapsulation extended community allows each PE in 549 a given EVI to know each of the encapsulations supported by each of 550 the other PEs in that EVI. I.e., each of the PEs in a given EVI may 551 support multiple data plane encapsulations. An ingress PE can send a 552 frame to an egress PE only if the set of encapsulations advertised by 553 the egress PE in the subject MAC Advertisement or Per EVI Ethernet AD 554 route, forms a non-empty intersection with the set of encapsulations 555 supported by the ingress PE, and it is at the discretion of the 556 ingress PE which encapsulation to choose from this intersection. 557 (As noted in section 5.1.3, if the BGP Encapsulation extended 558 community is not present, then the default MPLS encapsulation or a 559 statically configured encapsulation is assumed.) 561 If BGP Encapsulation extended community is not present, then the 562 default MPLS encapsulation (or statically configured encapsulation) 563 is used. However, if this attribute is present, then an ingress PE 564 can send a frame to an egress PE only if the set of encapsulations 565 advertised by the egress PE in the subject MAC Advertisement or Per 566 EVI Ethernet AD route, forms a non-empty intersection with the set of 567 encapsulations supported by the ingress PE, and it is at the 568 discretion of the ingress PE which encapsulation to choose from this 569 intersection. 571 An ingress node that uses shared multicast trees for sending 572 broadcast or multicast frames MUST maintain distinct trees for each 573 different encapsulation type. 575 It is the responsibility of the operator of a given EVI to ensure 576 that all of the PEs in that EVI support at least one common 577 encapsulation. If this condition is violated, it could result in 578 service disruption or failure. The use of the BGP Encapsulation 579 extended community provides a method to detect when this condition is 580 violated but the actions to be taken are at the discretion of the 581 operator and are outside the scope of this document. 583 7 NVE Residing in Hypervisor 584 When a PE and its CEs are co-located in the same physical device, 585 e.g., when the PE resides in a server and the CEs are its VMs, the 586 links between them are virtual and they typically share fate; i.e., 587 the subject CEs are typically not multi-homed or if they are multi- 588 homed, the multi-homing is a purely local matter to the server 589 hosting the VM, and need not be "visible" to any other PEs, and thus 590 does not require any specific protocol mechanisms. The most common 591 case of this is when the NVE resides in the hypervisor. 593 In the sub-sections that follow, we will discuss the impact on EVPN 594 procedures for the case when the NVE resides on the hypervisor and 595 the VXLAN or NVGRE encapsulation is used. 597 7.1 Impact on EVPN BGP Routes & Attributes for VXLAN/NVGRE Encapsulation 599 When the VXLAN VNI or NVGRE VSID is assumed to be a global value, one 600 might question the need for the Route Distinguisher (RD) in the EVPN 601 routes. In the scenario where all data centers are under a single 602 administrative domain, and there is a single global VNI/VSID space, 603 the RD MAY be set to zero in the EVPN routes. However, in the 604 scenario where different groups of data centers are under different 605 administrative domains, and these data centers are connected via one 606 or more backbone core providers as described in [NOV3-Framework], the 607 RD must be a unique value per EVI or per NVE as described in [EVPN]. 608 In other words, whenever there is more than one administrative domain 609 for global VNI or VSID, then a non-zero RD MUST be used, or whenever 610 the VNI or VSID value have local significance, then a non-zero RD 611 MUST be used. It is recommend to use a non-zero RD at all time. 613 When the NVEs reside on the hypervisor, the EVPN BGP routes and 614 attributes associated with multi-homing are no longer required. This 615 reduces the required routes and attributes to the following subset of 616 four out of the set of eight : 618 - MAC Advertisement Route 619 - Inclusive Multicast Ethernet Tag Route 620 - MAC Mobility Extended Community 621 - Default Gateway Extended Community 623 However, as noted in section 8.6 of [EVPN] in order to enable a 624 single-homed ingress PE to take advantage of fast convergence, 625 aliasing, and backup-path when interacting with multi-homed egress 626 PEs attached to a given Ethernet segment, a single-homed ingress PE 627 SHOULD be able to receive and process Ethernet AD per ES and Ethernet 628 AD per EVI routes." 630 7.2 Impact on EVPN Procedures for VXLAN/NVGRE Encapsulation 631 When the NVEs reside on the hypervisors, the EVPN procedures 632 associated with multi-homing are no longer required. This limits the 633 procedures on the NVE to the following subset of the EVPN procedures: 635 1. Local learning of MAC addresses received from the VMs per section 636 10.1 of [EVPN]. 638 2. Advertising locally learned MAC addresses in BGP using the MAC 639 Advertisement routes. 641 3. Performing remote learning using BGP per Section 10.2 of [EVPN]. 643 4. Discovering other NVEs and constructing the multicast tunnels 644 using the Inclusive Multicast Ethernet Tag routes. 646 5. Handling MAC address mobility events per the procedures of Section 647 16 in [EVPN]. 649 However, as noted in section 8.6 of [EVPN] in order to enable a 650 single-homed ingress PE to take advantage of fast convergence, 651 aliasing, and back-up path when interacting with multi-homed egress 652 PEs attached to a given Ethernet segment, a single-homed ingress PE 653 SHOULD implement the ingress node processing of Ethernet AD per ES 654 and Ethernet AD per EVI routes as defined in sections 8.2 Fast 655 Convergence and 8.4 Aliasing and Backup-Path of [EVPN]. 657 8 NVE Residing in ToR Switch 659 In this section, we discuss the scenario where the NVEs reside in the 660 Top of Rack (ToR) switches AND the servers (where VMs are residing) 661 are multi-homed to these ToR switches. The multi-homing may operate 662 in All-Active or Single-Active redundancy mode. If the servers are 663 single-homed to the ToR switches, then the scenario becomes similar 664 to that where the NVE resides in the hypervisor, as discussed in 665 Section 5, as far as the required EVPN functionality. 667 [EVPN] defines a set of BGP routes, attributes and procedures to 668 support multi-homing. We first describe these functions and 669 procedures, then discuss which of these are impacted by the 670 encapsulation (such as VXLAN or NVGRE) and what modifications are 671 required. 673 8.1 EVPN Multi-Homing Features 675 In this section, we will recap the multi-homing features of EVPN to 676 highlight the encapsulation dependencies. The section only describes 677 the features and functions at a high-level. For more details, the 678 reader is to refer to [EVPN]. 680 8.1.1 Multi-homed Ethernet Segment Auto-Discovery 682 EVPN NVEs (or PEs) connected to the same Ethernet Segment (e.g. the 683 same server via LAG) can automatically discover each other with 684 minimal to no configuration through the exchange of BGP routes. 686 8.1.2 Fast Convergence and Mass Withdraw 688 EVPN defines a mechanism to efficiently and quickly signal, to remote 689 NVEs, the need to update their forwarding tables upon the occurrence 690 of a failure in connectivity to an Ethernet segment (e.g., a link or 691 a port failure). This is done by having each NVE advertise an 692 Ethernet A-D Route per Ethernet segment for each locally attached 693 segment. Upon a failure in connectivity to the attached segment, the 694 NVE withdraws the corresponding Ethernet A-D route. This triggers all 695 NVEs that receive the withdrawal to update their next-hop adjacencies 696 for all MAC addresses associated with the Ethernet segment in 697 question. If no other NVE had advertised an Ethernet A-D route for 698 the same segment, then the NVE that received the withdrawal simply 699 invalidates the MAC entries for that segment. Otherwise, the NVE 700 updates the next-hop adjacencies to point to the backup NVE(s). 702 8.1.3 Split-Horizon 704 If a CE that is multi-homed to two or more NVEs on an Ethernet 705 segment ES1 operating in all-active redundancy mode sends a 706 multicast, broadcast or unknown unicast packet to a one of these 707 NVEs, then that NVE will forward that packet to all of the other PEs 708 in that EVI including the other NVEs attached to ES1 and those NVEs 709 MUST drop the packet and not forward back to the originating CE. 710 This is termed 'split horizon filtering'. 712 8.1.4 Aliasing and Backup-Path 714 In the case where a station is multi-homed to multiple NVEs, it is 715 possible that only a single NVE learns a set of the MAC addresses 716 associated with traffic transmitted by the station. This leads to a 717 situation where remote NVEs receive MAC advertisement routes, for 718 these addresses, from a single NVE even though multiple NVEs are 719 connected to the multi-homed station. As a result, the remote NVEs 720 are not able to effectively load-balance traffic among the NVEs 721 connected to the multi-homed Ethernet segment. This could be the 722 case, for e.g. when the NVEs perform data-path learning on the 723 access, and the load-balancing function on the station hashes traffic 724 from a given source MAC address to a single NVE. Another scenario 725 where this occurs is when the NVEs rely on control plane learning on 726 the access (e.g. using ARP), since ARP traffic will be hashed to a 727 single link in the LAG. 729 To alleviate this issue, EVPN introduces the concept of Aliasing. 730 This refers to the ability of an NVE to signal that it has 731 reachability to a given locally attached Ethernet segment, even when 732 it has learnt no MAC addresses from that segment. The Ethernet A-D 733 route per EVI is used to that end. Remote NVEs which receive MAC 734 advertisement routes with non-zero ESI SHOULD consider the MAC 735 address as reachable via all NVEs that advertise reachability to the 736 relevant Segment using Ethernet A-D routes with the same ESI and with 737 the Single-Active flag reset. 739 Backup-Path is a closely related function, albeit it applies to the 740 case where the redundancy mode is Single-Active. In this case, the 741 NVE signals that it has reachability to a given locally attached 742 Ethernet Segment using the Ethernet A-D route as well. Remote NVEs 743 which receive the MAC advertisement routes, with non-zero ESI, SHOULD 744 consider the MAC address as reachable via the advertising NVE. 745 Furthermore, the remote NVEs SHOULD install a Backup-Path, for said 746 MAC, to the NVE which had advertised reachability to the relevant 747 Segment using an Ethernet A-D route with the same ESI and with the 748 Single-Active flag set. 750 8.1.5 DF Election 752 If a CE is multi-homed to two or more NVEs on an Ethernet segment 753 operating in all-active redundancy mode, then for a given EVI only 754 one of these NVEs, termed the Designated Forwarder (DF) is 755 responsible for sending it broadcast, multicast, and, if configured 756 for that EVI, unknown unicast frames. 758 This is required in order to prevent duplicate delivery of multi- 759 destination frames to a multi-homed host or VM, in case of all-active 760 redundancy. 762 8.2 Impact on EVPN BGP Routes & Attributes 764 Since multi-homing is supported in this scenario, then the entire set 765 of BGP routes and attributes defined in [EVPN] are used. As discussed 766 in Section 3.1.3, the VSID or VNI is carried in the VNI/VSID field in 767 the MAC Advertisement, Ethernet AD per EVI, and Inclusive Multicast 768 Ethernet Tag routes. 770 8.3 Impact on EVPN Procedures 772 Two cases need to be examined here, depending on whether the NVEs are 773 operating in Active/Standby or in All-Active redundancy. 775 First, let's consider the case of Active/Standby redundancy, where 776 the hosts are multi-homed to a set of NVEs, however, only a single 777 NVE is active at a given point of time for a given VNI or VSID. In 778 this case, the Split-Horizon and Aliasing functions are not required 779 but other functions such as multi-homed Ethernet segment auto- 780 discovery, fast convergence and mass withdraw, repair path, and DF 781 election are required. In this case, the impact of the use of the 782 VXLAN/NVGRE encapsulation on the EVPN procedures is when the Backup- 783 Path function is supported, as discussed next: 785 In EVPN, the NVEs connected to a multi-homed site using 786 Active/Standby redundancy optionally advertise a VPN label, in the 787 Ethernet A-D Route per EVI, used to send traffic to the backup NVE in 788 the case where the primary NVE fails. In the case where VXLAN or 789 NVGRE encapsulation is used, some alternative means that does not 790 rely on MPLS labels is required to support Backup-Path. This is 791 discussed in Section 4.3.2 below. If the Backup-Path function is not 792 used, then the VXLAN/NVGRE encapsulation would have no impact on the 793 EVPN procedures. 795 Second, let's consider the case of All-Active redundancy. In this 796 case, out of the EVPN multi-homing features listed in section 4.1, 797 the use of the VXLAN or NVGRE encapsulation impacts the Split-Horizon 798 and Aliasing features, since those two rely on the MPLS client layer. 799 Given that this MPLS client layer is absent with these types of 800 encapsulations, alternative procedures and mechanisms are needed to 801 provide the required functions. Those are discussed in detail next. 803 8.3.1 Split Horizon 805 In EVPN, an MPLS label is used for split-horizon filtering to support 806 active/active multi-homing where an ingress ToR switch adds a label 807 corresponding to the site of origin (aka ESI MPLS Label) when 808 encapsulating the packet. The egress ToR switch checks the ESI MPLS 809 label when attempting to forward a multi-destination frame out an 810 interface, and if the label corresponds to the same site identifier 811 (ESI) associated with that interface, the packet gets dropped. This 812 prevents the occurrence of forwarding loops. 814 Since the VXLAN or NVGRE encapsulation does not include this ESI MPLS 815 label, other means of performing the split-horizon filtering function 816 MUST be devised. The following approach is recommended for split- 817 horizon filtering when VXLAN or NVGRE encapsulation is used. 819 Every NVE track the IP address(es) associated with the other NVE(s) 820 with which it has shared multi-homed Ethernet Segments. When the NVE 821 receives a multi-destination frame from the overlay network, it 822 examines the source IP address in the tunnel header (which 823 corresponds to the ingress NVE) and filters out the frame on all 824 local interfaces connected to Ethernet Segments that are shared with 825 the ingress NVE. With this approach, it is required that the ingress 826 NVE performs replication locally to all directly attached Ethernet 827 Segments (regardless of the DF Election state) for all flooded 828 traffic ingress from the access interfaces (i.e. from the hosts). 829 This approach is referred to as "Local Bias", and has the advantage 830 that only a single IP address needs to be used per NVE for split- 831 horizon filtering, as opposed to requiring an IP address per Ethernet 832 Segment per NVE. 834 In order to prevent unhealthy interactions between the split horizon 835 procedures defined in [EVPN] and the local bias procedures described 836 in this document, a mix of MPLS over GRE encapsulations on the one 837 hand and VXLAN/NVGRE encapsulations on the other on a given Ethernet 838 Segment is prohibited. 840 8.3.2 Aliasing and Backup-Path 842 The Aliasing and the Backup-Path procedures for VXLAN/NVGRE 843 encapsulation is very similar to the ones for MPLS. In case of MPLS, 844 two different Ethernet AD routes are used for this purpose. The one 845 used for Aliasing has a VPN scope and carries a VPN label but the one 846 used for Backup-Path has Ethernet segment scope and doesn't carry any 847 VPN specific info (e.g., Ethernet Tag and MPLS label are set to 848 zero). The same two routes are used when VXLAN or NVGRE encapsulation 849 is used with the difference that when Ethernet AD route is used for 850 Aliasing with VPN scope, the Ethernet Tag field is set to VNI or VSID 851 to indicate VPN scope (and MPLS field may be set to a VPN label if 852 needed). 854 9 Support for Multicast 856 The E-VPN Inclusive Multicast BGP route is used to discover the 857 multicast tunnels among the endpoints associated with a given VXLAN 858 VNI or NVGRE VSID. The Ethernet Tag field of this route is used to 859 encode the VNI for VLXAN or VSID for NVGRE. The Originating router's 860 IP address field is set to the NVE's IP address. This route is tagged 861 with the PMSI Tunnel attribute, which is used to encode the type of 862 multicast tunnel to be used as well as the multicast tunnel 863 identifier. The tunnel encapsulation is encoded by adding the BGP 864 Encapsulation extended community as per section 3.1.1. The following 865 tunnel types as defined in [RFC6514] can be used in the PMSI tunnel 866 attribute for VXLAN/NVGRE: 868 + 3 - PIM-SSM Tree 869 + 4 - PIM-SM Tree 870 + 5 - BIDIR-PIM Tree 871 + 6 - Ingress Replication 873 Except for Ingress Replication, this multicast tunnel is used by the 874 PE originating the route for sending multicast traffic to other PEs, 875 and is used by PEs that receive this route for receiving the traffic 876 originated by CEs connected to the PE that originated the route. 878 In the scenario where the multicast tunnel is a tree, both the 879 Inclusive as well as the Aggregate Inclusive variants may be used. In 880 the former case, a multicast tree is dedicated to a VNI or VSID. 881 Whereas, in the latter, a multicast tree is shared among multiple 882 VNIs or VSIDs. This is done by having the NVEs advertise multiple 883 Inclusive Multicast routes with different VNI or VSID encoded in the 884 Ethernet Tag field, but with the same tunnel identifier encoded in 885 the PMSI Tunnel attribute. 887 10 Inter-AS 889 For inter-AS operation, two scenarios must be considered: 891 - Scenario 1: The tunnel endpoint IP addresses are public 892 - Scenario 2: The tunnel endpoint IP addresses are private 894 In the first scenario, inter-AS operation is straight-forward and 895 follows existing BGP inter-AS procedures. However, in the first 896 scenario where the tunnel endpoint IP addresses are public, there may 897 be security concern regarding the distribution of these addresses 898 among different ASes. This security concern is one of the main 899 reasons for having the so called inter-AS "option-B" in MPLS VPN 900 solutions such as EVPN. 902 The second scenario is more challenging, because the absence of the 903 MPLS client layer from the VXLAN encapsulation creates a situation 904 where the ASBR has no fully qualified indication within the tunnel 905 header as to where the tunnel endpoint resides. To elaborate on this, 906 recall that with MPLS, the client layer labels (i.e. the VPN labels) 907 are downstream assigned. As such, this label implicitly has a 908 connotation of the tunnel endpoint, and it is sufficient for the ASBR 909 to look up the client layer label in order to identify the label 910 translation required as well as the tunnel endpoint to which a given 911 packet is being destined. With the VXLAN encapsulation, the VNI is 912 globally assigned and hence is shared among all endpoints. The 913 destination IP address is the only field which identifies the tunnel 914 endpoint in the tunnel header, and this address is privately managed 915 by every data center network. Since the tunnel address is allocated 916 out of a private address pool, then we either need to do a lookup 917 based on VTEP IP address in context of a VRF (e.g., use IP-VPN) or 918 terminate the VXLAN tunnel and do a lookup based on the tenant's MAC 919 address to identify the egress tunnel on the ASBR. This effectively 920 mandates that the ASBR to either run another overlay solution such as 921 IP-VPN over MPLS/IP core network or to be aware of the MAC addresses 922 of all VMs in its local AS, at the very least. 924 If VNIs/VSIDs have local significance, then the inter-AS operation 925 can be simplified to that of MPLS and thus MPLS inter-AS option B and 926 C can be leveraged in here. That's why the use of local significance 927 VNIs/VSIDs (e.g., MPLS labels) are recommended for inter-AS operation 928 of DC networks without gateways. 930 11 Acknowledgement 932 The authors would like to thank David Smith, John Mullooly, Thomas 933 Nadeau for their valuable comments and feedback. 935 12 Security Considerations 937 This document uses IP-based tunnel technologies to support data 938 plane transport. Consequently, the security considerations of those 939 tunnel technologies apply. This document defines support for [VXLAN] 940 and [NVGRE]. The security considerations from those documents as well 941 as [RFC4301] apply to the data plane aspects of this document. 943 As with [RFC5512], any modification of the information that is used 944 to form encapsulation headers, to choose a tunnel type, or to choose 945 a particular tunnel for a particular payload type may lead to user 946 data packets getting misrouted, misdelivered, and/or dropped. 948 More broadly, the security considerations for the transport of IP 949 reachability information using BGP are discussed in [RFC4271] and 950 [RFC4272], and are equally applicable for the extensions described 951 in this document. 953 If the integrity of the BGP session is not itself protected, then an 954 imposter could mount a denial-of-service attack by establishing 955 numerous BGP sessions and forcing an IPsec SA to be created for each 956 one. However, as such an imposter could wreak havoc on the entire 957 routing system, this particular sort of attack is probably not of 958 any special importance. 960 It should be noted that a BGP session may itself be transported over 961 an IPsec tunnel. Such IPsec tunnels can provide additional security 962 to a BGP session. The management of such IPsec tunnels is outside 963 the scope of this document. 965 13 IANA Considerations 967 14 References 969 14.1 Normative References 971 [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate 972 Requirement Levels", BCP 14, RFC 2119, March 1997. 974 [RFC4271] Y. Rekhter, Ed., T. Li, Ed., S. Hares, Ed., "A Border 975 Gateway Protocol 4 (BGP-4)", January 2006. 977 [RFC4272] S. Murphy, "BGP Security Vulnerabilities Analysis.", 978 January 2006. 980 [RFC4301] S. Kent, K. Seo., "Security Architecture for the 981 Internet Protocol.", December 2005. 983 [RFC5512] Mohapatra, P. and E. Rosen, "The BGP Encapsulation 984 Subsequent Address Family Identifier (SAFI) and the BGP 985 Tunnel Encapsulation Attribute", RFC 5512, April 2009. 987 14.2 Informative References 989 [EVPN-REQ] Sajassi et al., "Requirements for Ethernet VPN (EVPN)", 990 draft-ietf-l2vpn-evpn-req-01.txt, work in progress, October 21, 2012. 992 [NVGRE] Sridhavan, M., et al., "NVGRE: Network Virtualization using 993 Generic Routing Encapsulation", draft-sridharan-virtualization-nvgre- 994 01.txt, July 8, 2012. 996 [VXLAN] Dutt, D., et al, "VXLAN: A Framework for Overlaying 997 Virtualized Layer 2 Networks over Layer 3 Networks", draft- 998 mahalingam-dutt-dcops-vxlan-02.txt, August 22, 2012. 1000 [EVPN] Sajassi et al., "BGP MPLS Based Ethernet VPN", draft-ietf- 1001 l2vpn-evpn-02.txt, work in progress, February, 2012. 1003 [Problem-Statement] Narten et al., "Problem Statement: Overlays for 1004 Network Virtualization", draft-ietf-nvo3-overlay-problem-statement- 1005 01, September 2012. 1007 [L3VPN-ENDSYSTEMS] Marques et al., "BGP-signaled End-system IP/VPNs", 1008 draft-ietf-l3vpn-end-system, work in progress, October 2012. 1010 [NOV3-FRWK] Lasserre et al., "Framework for DC Network 1011 Virtualization", draft-ietf-nvo3-framework-01.txt, work in progress, 1012 October 2012. 1014 Authors' Addresses 1016 Ali Sajassi 1017 Cisco 1018 Email: sajassi@cisco.com 1020 John Drake 1021 Juniper Networks 1022 Email: jdrake@juniper.net 1024 Nabil Bitar 1025 Verizon Communications 1026 Email : nabil.n.bitar@verizon.com 1028 Aldrin Isaac 1029 Bloomberg 1030 Email: aisaac71@bloomberg.net 1032 James Uttaro 1033 AT&T 1034 Email: uttaro@att.com 1036 Wim Henderickx 1037 Alcatel-Lucent 1038 e-mail: wim.henderickx@alcatel-lucent.com 1040 Ravi Shekhar 1041 Juniper Networks 1042 Email: rshekhar@juniper.net 1044 Samer Salam 1045 Cisco 1046 Email: ssalam@cisco.com 1048 Keyur Patel 1049 Cisco 1050 Email: Keyupate@cisco.com 1052 Dhananjaya Rao 1053 Cisco 1054 Email: dhrao@cisco.com 1056 Samir Thoria 1057 Cisco 1058 Email: sthoria@cisco.com