idnits 2.17.1 draft-ietf-bess-evpn-overlay-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 5 instances of too long lines in the document, the longest one being 2 characters in excess of 72. ** There are 13 instances of lines with control characters in the document. ** The abstract seems to contain references ([RFC7432]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 24, 2015) is 3349 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2119' is mentioned on line 172, but not defined == Missing Reference: 'GRE' is mentioned on line 307, but not defined == Missing Reference: 'RFC4023' is mentioned on line 558, but not defined == Missing Reference: 'NOV3-Framework' is mentioned on line 613, but not defined == Missing Reference: 'RFC6514' is mentioned on line 858, but not defined == Unused Reference: 'KEYWORDS' is defined on line 974, but no explicit reference was found in the text == Unused Reference: 'NOV3-FRWK' is defined on line 1013, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 4272 ** Obsolete normative reference: RFC 5512 (Obsoleted by RFC 9012) == Outdated reference: A later version (-08) exists of draft-sridharan-virtualization-nvgre-07 == Outdated reference: A later version (-04) exists of draft-ietf-nvo3-overlay-problem-statement-01 == Outdated reference: A later version (-09) exists of draft-ietf-nvo3-framework-01 Summary: 5 errors (**), 0 flaws (~~), 11 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 L2VPN Workgroup A. Sajassi (Editor) 3 INTERNET-DRAFT Cisco 4 Intended Status: Standards Track 5 J. Drake (Editor) 6 Y. Rekhter Juniper 7 R. Shekhar 8 B. Schliesser Nabil Bitar 9 Juniper Verizon 11 S. Salam Aldrin Isaac 12 K. Patel Bloomberg 13 D. Rao 14 S. Thoria James Uttaro 15 Cisco AT&T 17 L. Yong W. Henderickx 18 Huawei Alcatel-Lucent 20 D. Cai 21 S. Sinha 22 Cisco 24 Wen Lin 25 Nischal Sheth 26 Juniper 28 Expires: August 24, 2015 February 24, 2015 30 A Network Virtualization Overlay Solution using EVPN 31 draft-ietf-bess-evpn-overlay-01 33 Abstract 35 This document describes how Ethernet VPN (EVPN) [RFC7432] can be used 36 as an Network Virtualization Overlay (NVO) solution and explores the 37 various tunnel encapsulation options over IP and their impact on the 38 EVPN control-plane and procedures. In particular, the following 39 encapsulation options are analyzed: VXLAN, NVGRE, and MPLS over GRE. 41 Status of this Memo 43 This Internet-Draft is submitted to IETF in full conformance with the 44 provisions of BCP 78 and BCP 79. 46 Internet-Drafts are working documents of the Internet Engineering 47 Task Force (IETF), its areas, and its working groups. Note that 48 other groups may also distribute working documents as 49 Internet-Drafts. 51 Internet-Drafts are draft documents valid for a maximum of six months 52 and may be updated, replaced, or obsoleted by other documents at any 53 time. It is inappropriate to use Internet-Drafts as reference 54 material or to cite them other than as "work in progress." 56 The list of current Internet-Drafts can be accessed at 57 http://www.ietf.org/1id-abstracts.html 59 The list of Internet-Draft Shadow Directories can be accessed at 60 http://www.ietf.org/shadow.html 62 Copyright and License Notice 64 Copyright (c) 2012 IETF Trust and the persons identified as the 65 document authors. All rights reserved. 67 This document is subject to BCP 78 and the IETF Trust's Legal 68 Provisions Relating to IETF Documents 69 (http://trustee.ietf.org/license-info) in effect on the date of 70 publication of this document. Please review these documents 71 carefully, as they describe your rights and restrictions with respect 72 to this document. Code Components extracted from this document must 73 include Simplified BSD License text as described in Section 4.e of 74 the Trust Legal Provisions and are provided without warranty as 75 described in the Simplified BSD License. 77 Table of Contents 79 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 80 2 Specification of Requirements . . . . . . . . . . . . . . . . . 5 81 3 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 82 4 EVPN Features . . . . . . . . . . . . . . . . . . . . . . . . . 6 83 5 Encapsulation Options for EVPN Overlays . . . . . . . . . . . . 7 84 5.1 VXLAN/NVGRE Encapsulation . . . . . . . . . . . . . . . . . 7 85 5.1.1 Virtual Identifiers Scope . . . . . . . . . . . . . . . 8 86 5.1.1.1 Data Center Interconnect with Gateway . . . . . . . 8 87 5.1.1.2 Data Center Interconnect without Gateway . . . . . . 9 88 5.1.2 Virtual Identifiers to EVI Mapping . . . . . . . . . . . 9 89 5.1.2.1 Auto Derivation of RT . . . . . . . . . . . . . . . 10 90 5.1.3 Constructing EVPN BGP Routes . . . . . . . . . . . . . 11 91 5.2 MPLS over GRE . . . . . . . . . . . . . . . . . . . . . . . 13 92 6 EVPN with Multiple Data Plane Encapsulations . . . . . . . . . 13 93 7 NVE Residing in Hypervisor . . . . . . . . . . . . . . . . . . 14 94 7.1 Impact on EVPN BGP Routes & Attributes for VXLAN/NVGRE 95 Encapsulation . . . . . . . . . . . . . . . . . . . . . . . 14 96 7.2 Impact on EVPN Procedures for VXLAN/NVGRE Encapsulation . . 15 97 8 NVE Residing in ToR Switch . . . . . . . . . . . . . . . . . . 15 98 8.1 EVPN Multi-Homing Features . . . . . . . . . . . . . . . . 16 99 8.1.1 Multi-homed Ethernet Segment Auto-Discovery . . . . . . 16 100 8.1.2 Fast Convergence and Mass Withdraw . . . . . . . . . . . 16 101 8.1.3 Split-Horizon . . . . . . . . . . . . . . . . . . . . . 16 102 8.1.4 Aliasing and Backup-Path . . . . . . . . . . . . . . . . 17 103 8.1.5 DF Election . . . . . . . . . . . . . . . . . . . . . . 17 104 8.2 Impact on EVPN BGP Routes & Attributes . . . . . . . . . . . 18 105 8.3 Impact on EVPN Procedures . . . . . . . . . . . . . . . . . 18 106 8.3.1 Split Horizon . . . . . . . . . . . . . . . . . . . . . 18 107 8.3.2 Aliasing and Backup-Path . . . . . . . . . . . . . . . . 19 108 9 Support for Multicast . . . . . . . . . . . . . . . . . . . . . 19 109 10 Inter-AS . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 110 11 Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 21 111 12 Security Considerations . . . . . . . . . . . . . . . . . . . 21 112 13 IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 113 14 References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 114 14.1 Normative References . . . . . . . . . . . . . . . . . . . 22 115 14.2 Informative References . . . . . . . . . . . . . . . . . . 22 116 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 118 1 Introduction 120 In the context of this document, a Network Virtualization Overlay 121 (NVO) is a solution to address the requirements of a multi-tenant 122 data center, especially one with virtualized hosts, e.g., Virtual 123 Machines (VMs). The key requirements of such a solution, as described 124 in [Problem-Statement], are: 126 - Isolation of network traffic per tenant 128 - Support for a large number of tenants (tens or hundreds of 129 thousands) 131 - Extending L2 connectivity among different VMs belonging to a given 132 tenant segment (subnet) across different PODs within a data center or 133 between different data centers 135 - Allowing a given VM to move between different physical points of 136 attachment within a given L2 segment 138 The underlay network for NVO solutions is assumed to provide IP 139 connectivity between NVO endpoints (NVEs). 141 This document describes how Ethernet VPN (EVPN) can be used as an NVO 142 solution and explores applicability of EVPN functions and procedures. 143 In particular, it describes the various tunnel encapsulation options 144 for EVPN over IP, and their impact on the EVPN control-plane and 145 procedures for two main scenarios: 147 a) when the NVE resides in the hypervisor, and 148 b) when the NVE resides in a Top of Rack (ToR) device 150 Note that the use of EVPN as an NVO solution does not necessarily 151 mandate that the BGP control-plane be running on the NVE. For such 152 scenarios, it is still possible to leverage the EVPN solution by 153 using XMPP, or alternative mechanisms, to extend the control-plane to 154 the NVE as discussed in [L3VPN-ENDSYSTEMS]. 156 The possible encapsulation options for EVPN overlays that are 157 analyzed in this document are: 159 - VXLAN and NVGRE 160 - MPLS over GRE 162 Before getting into the description of the different encapsulation 163 options for EVPN over IP, it is important to highlight the EVPN 164 solution's main features, how those features are currently supported, 165 and any impact that the encapsulation has on those features. 167 2 Specification of Requirements 169 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 170 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 171 document are to be interpreted as described in [RFC2119]. 173 3 Terminology 175 NVO: Network Virtualization Overlay 177 NVE: Network Virtualization Endpoint 179 VNI: Virtual Network Identifier (for VXLAN) 181 VSID: VIrtual Subnet Identifier (for NVGRE) 183 EVPN: Ethernet VPN 185 EVI: An EVPN instance spanning the Provider Edge (PE) devices 186 participating in that EVPN. 188 MAC-VRF: A Virtual Routing and Forwarding table for Media Access 189 Control (MAC) addresses on a PE. 191 Ethernet Segment (ES): When a customer site (device or network) is 192 connected to one or more PEs via a set of Ethernet links, then that 193 set of links is referred to as an 'Ethernet segment'. 195 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 196 identifies an Ethernet segment is called an 'Ethernet Segment 197 Identifier'. 199 Ethernet Tag: An Ethernet tag identifies a particular broadcast 200 domain, e.g., a VLAN. An EVPN instance consists of one or more 201 broadcast domains. 203 PE: Provider Edge device. 205 Single-Active Redundancy Mode: When only a single PE, among all the 206 PEs attached to an Ethernet segment, is allowed to forward traffic 207 to/from that Ethernet segment for a given VLAN, then the Ethernet 208 segment is defined to be operating in Single-Active redundancy mode. 210 All-Active Redundancy Mode: When all PEs attached to an Ethernet 211 segment are allowed to forward known unicast traffic to/from that 212 Ethernet segment for a given VLAN, then the Ethernet segment is 213 defined to be operating in All-Active redundancy mode. 215 4 EVPN Features 217 EVPN was originally designed to support the requirements detailed in 218 [RFC7209] and therefore has the following attributes which directly 219 address control plane scaling and ease of deployment issues. 221 1) Control plane traffic is distributed with BGP and Broadcast and 222 Multicast traffic is sent using a shared multicast tree or with 223 ingress replication. 225 2) Control plane learning is used for MAC (and IP) addresses instead 226 of data plane learning. The latter requires the flooding of unknown 227 unicast and ARP frames; whereas, the former does not require any 228 flooding. 230 3) Route Reflector is used to reduce a full mesh of BGP sessions 231 among PE devices to a single BGP session between a PE and the RR. 232 Furthermore, RR hierarchy can be leveraged to scale the number of BGP 233 routes on the RR. 235 4) Auto-discovery via BGP is used to discover PE devices 236 participating in a given VPN, PE devices participating in a given 237 redundancy group, tunnel encapsulation types, multicast tunnel type, 238 multicast members, etc. 240 5) All-Active multihoming is used. This allows a given customer 241 device (CE) to have multiple links to multiple PEs, and traffic 242 to/from that CE fully utilizes all of these links. This set of links 243 is termed an Ethernet Segment (ES). 245 6) When a link between a CE and a PE fails, the PEs for that EVI are 246 notified of the failure via the withdrawal of a single EVPN route. 247 This allows those PEs to remove the withdrawing PE as a next hop for 248 every MAC address associated with the failed link. This is termed 249 'mass withdrawal'. 251 7) BGP route filtering and constrained route distribution are 252 leveraged to ensure that the control plane traffic for a given EVI is 253 only distributed to the PEs in that EVI. 255 8) When a 802.1Q interface is used between a CE and a PE, each of the 256 VLAN ID (VID) on that interface can be mapped onto a bridge table 257 (for upto 4094 such bridge tables). All these bridge tables may be 258 mapped onto a single MAC-VRF (in case of VLAN-aware bundle service). 260 9) VM Mobility mechanisms ensure that all PEs in a given EVI know 261 the ES with which a given VM, as identified by its MAC and IP 262 addresses, is currently associated. 264 10) Route Targets are used to allow the operator (or customer) to 265 define a spectrum of logical network topologies including mesh, hub & 266 spoke, and extranets (e.g., a VPN whose sites are owned by different 267 enterprises), without the need for proprietary software or the aid of 268 other virtual or physical devices. 270 11) Because the design goal for NVO is millions of instances per 271 common physical infrastructure, the scaling properties of the control 272 plane for NVO are extremely important. EVPN and the extensions 273 described herein, are designed with this level of scalability in 274 mind. 276 5 Encapsulation Options for EVPN Overlays 278 5.1 VXLAN/NVGRE Encapsulation 280 Both VXLAN and NVGRE are examples of technologies that provide a data 281 plane encapsulation which is used to transport a packet over the 282 common physical IP infrastructure between VXLAN Tunnel End Points 283 (VTEPs) in VXLAN network and Network Virtualization Endpoints (NVEs) 284 in NVGRE network. Both of these technologies include the identifier 285 of the specific NVO instance, Virtual Network Identifier (VNI) in 286 VXLAN and Virtual Subnet Identifier (VSID) in NVGRE, in each packet. 288 Note that a Provider Edge (PE) is equivalent to a VTEP/NVE. 290 VXLAN encapsulation is based on UDP, with an 8-byte header following 291 the UDP header. VXLAN provides a 24-bit VNI, which typically provides 292 a one-to-one mapping to the tenant VLAN ID, as described in 293 [RFC7348]. In this scenario, the ingress VTEP does not include an 294 inner VLAN tag on the encapsulated frame, and the egress VTEP 295 discards the frames with an inner VLAN tag. This mode of operation in 296 [RFC7348] maps to VLAN Based Service in [RFC7432], where a tenant 297 VLAN ID gets mapped to an EVPN instance (EVI). 299 VXLAN also provides an option of including an inner VLAN tag in the 300 encapsulated frame, if explicitly configured at the VTEP. This mode 301 of operation can map to VLAN Bundle Service in [RFC7432] because all 302 the tenant's tagged frames map to a single bridge table / MAC-VRF, 303 and the inner VLAN tag is not used for lookup by the disposition PE 304 when performing VXLAN decapsulation as described in section 6 of 305 [RFC7348]. 307 [NVGRE] encapsulation is based on [GRE] and it mandates the inclusion 308 of the optional GRE Key field which carries the VSID. There is a one- 309 to-one mapping between the VSID and the tenant VLAN ID, as described 310 in [NVGRE] and the inclusion of an inner VLAN tag is prohibited. This 311 mode of operation in [NVGRE] maps to VLAN Based Service in 312 [RFC7432]. 314 As described in the next section there is no change to the encoding 315 of EVPN routes to support VXLAN or NVGRE encapsulation except for the 316 use of BGP Encapsulation extended community. However, there is 317 potential impact to the EVPN procedures depending on where the NVE is 318 located (i.e., in hypervisor or TOR) and whether multi-homing 319 capabilities are required. 321 5.1.1 Virtual Identifiers Scope 323 Although VNI or VSID are defined as 24-bit globally unique values, 324 there are scenarios in which it is desirable to use a locally 325 significant value for VNI or VSID, especially in the context of data 326 center interconnect: 328 5.1.1.1 Data Center Interconnect with Gateway 330 In the case where NVEs in different data centers need to be 331 interconnected, and the NVEs need to use VNIs or VSIDs as a globally 332 unique identifiers within a data center, then a Gateway needs to be 333 employed at the edge of the data center network. This is because the 334 Gateway will provide the functionality of translating the VNI or VSID 335 when crossing network boundaries, which may align with operator span 336 of control boundaries. As an example, consider the network of Figure 337 1 below. Assume there are three network operators: one for each of 338 the DC1, DC2 and WAN networks. The Gateways at the edge of the data 339 centers are responsible for translating the VNIs / VSIDs between the 340 values used in each of the data center networks and the values used 341 in the WAN. 343 +--------------+ 344 | | 345 +---------+ | WAN | +---------+ 346 +----+ | +---+ +----+ +----+ +---+ | +----+ 347 |NVE1|--| | | |WAN | |WAN | | | |--|NVE3| 348 +----+ |IP |GW |--|Edge| |Edge|--|GW | IP | +----+ 349 +----+ |Fabric +---+ +----+ +----+ +---+ Fabric | +----+ 350 |NVE2|--| | | | | |--|NVE4| 351 +----+ +---------+ +--------------+ +---------+ +----+ 353 |<------ DC 1 ------> <------ DC2 ------>| 355 Figure 1: Data Center Interconnect with Gateway 357 5.1.1.2 Data Center Interconnect without Gateway 359 In the case where NVEs in different data centers need to be 360 interconnected, and the NVEs need to use locally assigned VNIs or 361 VSIDs (e.g., as MPLS labels), then there may be no need to employ 362 Gateways at the edge of the data center network. More specifically, 363 the VNI or VSID value that is used by the transmitting NVE is 364 allocated by the NVE that is receiving the traffic (in other words, 365 this is a "downstream assigned" MPLS label). This allows the VNI or 366 VSID space to be decoupled between different data center networks 367 without the need for a dedicated Gateway at the edge of the data 368 centers. 370 +--------------+ 371 | | 372 +---------+ | WAN | +---------+ 373 +----+ | | +----+ +----+ | | +----+ 374 |NVE1|--| | |WAN | |WAN | | |--|NVE3| 375 +----+ |IP Fabric|---|Edge| |Edge|--|IP Fabric| +----+ 376 +----+ | | +----+ +----+ | | +----+ 377 |NVE2|--| | | | | |--|NVE4| 378 +----+ +---------+ +--------------+ +---------+ +----+ 380 |<------ DC 1 -----> <---- DC2 ------>| 382 Figure 2: Data Center Interconnect without Gateway 384 5.1.2 Virtual Identifiers to EVI Mapping 386 When the EVPN control plane is used in conjunction with VXLAN or 387 NVGRE, two options for mapping the VXLAN VNI or NVGRE VSID to an EVI 388 are possible: 390 1. Option 1: Single Subnet per EVI 392 In this option, a single subnet represented by a VNI or VSID is 393 mapped to a unique EVI. This corresponds to the VLAN Based service in 394 [RFC7432], where a tenant VLAN ID gets mapped to an EVPN instance 395 (EVI). As such, a BGP RD and RT is needed per VNI / VSID on every 396 VTEP. The advantage of this model is that it allows the BGP RT 397 constraint mechanisms to be used in order to limit the propagation 398 and import of routes to only the VTEPs that are interested in a given 399 VNI or VSID. The disadvantage of this model may be the provisioning 400 overhead if RD and RT are not derived automatically from VNI or 401 VSID. 403 In this option, the MAC-VRF table is identified by the RT in the 404 control plane and by the VNI or VSID in the data-plane. In this 405 option, the specific the MAC-VRF table corresponds to only a single 406 bridge table. 408 2. Option 2: Multiple Subnets per EVI 410 In this option, multiple subnets each represented by a unique VNI or 411 VSID are mapped to a single EVI. For example, if a tenant has 412 multiple segments/subnets each represented by a VNI or VSID, then all 413 the VNIs (or VSIDs) for that tenant are mapped to a single EVI - 414 e.g., the EVI in this case represents the tenant and not a subnet . 415 This corresponds to the VLAN-Aware Bundle service in [RFC7432]. The 416 advantage of this model is that it doesn't require the provisioning 417 of RD/RT per VNI or VSID. However, this is a moot point if option 1 418 with auto-derivation is used. The disadvantage of this model is that 419 routes would be imported by VTEPs that may not be interested in a 420 given VNI or VSID. 422 In this option the MAC-VRF table is identified by the RT in the 423 control plane and a specific bridge table for that MAC-VRF is 424 identified by the in the control plane. In this 425 option, the VNI/VSID in the data-plane is sufficient to identify a 426 specific bridge table - e.g., no need to do a lookup based on 427 VNI/VSID and Ethernet Tag ID fields to identify a bridge table. 429 5.1.2.1 Auto Derivation of RT 431 When the option of a single VNI or VSID per EVI is used, it is 432 important to auto-derive RT for EVPN BGP routes in order to simplify 433 configuration for data center operations. RD can be derived easily as 434 described in [RFC7432] and RT can be auto-derived as described next. 436 Since a gateway PE as depicted in figure-1 participates in both the 437 DCN and WAN BGP sessions, it is important that when RT values are 438 auto-derived for VNIs (or VSIDs), there is no conflict in RT spaces 439 between DCN and WAN networks assuming that both are operating within 440 the same AS. Also, there can be scenarios where both VXLAN and NVGRE 441 encapsulations may be needed within the same DCN and their 442 corresponding VNIs and VSIDs are administered independently which 443 means VNI and VSID spaces can overlap. In order to ensure that no 444 such conflict in RT spaces arises, RT values for DCNs are auto- 445 derived as follow: 447 0 1 2 3 4 448 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 0 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++ 450 | AS # |A| TYPE| D-ID | Service Instance ID| 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++ 453 - 2 bytes of global admin field of the RT is set to the AS number. 455 - Three least significant bytes of the local admin field of the RT is 456 set to the VNI or VSID, I-SID, or VID. The most significant bit of 457 the local admin field of the RT is set as follow: 458 0: auto-derived 459 1: manually-derived 461 - The next 3 bits of the most significant byte of the local admin 462 field of the RT identifies the space in which the other 3 bytes are 463 defined. The following spaces are defined: 464 0 : VID 465 1 : VXLAN 466 2 : NVGRE 467 3 : I-SID 468 4 : EVI 469 5 : dual-VID 471 - The remaining 4 bits of the most significant byte of the local 472 admin field of the RT identifies the domain-id. The default value of 473 domain-id is zero indicating that only a single numbering space exist 474 for a given technology. However, if there are more than one number 475 space exist for a given technology (e.g., overlapping VXLAN spaces), 476 then each of the number spaces need to be identify by their 477 corresponding domain-id starting from 1. 479 5.1.3 Constructing EVPN BGP Routes 480 In EVPN, an MPLS label is distributed by the egress PE via the EVPN 481 control plane and is placed in the MPLS header of a given packet by 482 the ingress PE. This label is used upon receipt of that packet by the 483 egress PE for disposition of that packet. This is very similar to the 484 use of the VNI or VSID by the egress VTEP or NVE, respectively, with 485 the difference being that an MPLS label has local significance while 486 a VNI or VSID typically has global significance. Accordingly, and 487 specifically to support the option of locally assigned VNIs, the MPLS 488 label field in the MAC Advertisement, Ethernet AD per EVI, and 489 Inclusive Multicast Ethernet Tag routes is used to carry the VNI or 490 VSID. For the balance of this memo, the MPLS label field will be 491 referred to as the VNI/VSID field. The VNI/VSID field is used for 492 both local and global VNIs/VSIDs, and for either case the entire 24- 493 bit field is used to encode the VNI/VSID value. 495 For the VLAN based mode (a single VNI per MAC-VRF), the Ethernet Tag 496 field in the MAC/IP Advertisement, Ethernet AD per EVI, and Inclusive 497 Multicast route MUST be set to zero just as in the VLAN Based service 498 in [RFC7432]. 500 For the VNI-aware bundle mode (multiple VNIs per MAC-VRF with each 501 VNI associated with its own bridge table), the Ethernet Tag field in 502 the MAC Advertisement, Ethernet AD per EVI, and Inclusive Multicast 503 route MUST identify a bridge table within a MAC-VRF and the set of 504 Ethernet Tags for that EVI needs to be configured consistently on all 505 PEs within that EVI. For local VNIs, the value advertised in the 506 Ethernet Tag field MUST be set to a VID just as in the VLAN-aware 507 bundle service in [RFC7432]. Such setting must be done consistently 508 on all PE devices participating in that EVI within a given domain. 509 For global VNIs, the value advertised in the Ethernet Tag field 510 SHOULD be set to a VNI as long as it matches the existing semantics 511 of the Ethernet Tag, i.e., it identifies a bridge table within a MAC- 512 VRF and the set of VNIs are configured consistently on each PE in 513 that EVI. It should be noted that if within a single domain, a mix of 514 local and global VNIs are used for the same VLAN-aware bundle 515 service, then the Ethernet Tag field in the EVPN BGP route 516 advertisements SHALL be set to a VID. 518 In order to indicate that which type of data plane encapsulation 519 (i.e., VXLAN, NVGRE, MPLS, or MPLS in GRE) is to be used, the BGP 520 Encapsulation extended community defined in [RFC5512] is included 521 with all EVPN routes (i.e. MAC Advertisement, Ethernet AD per EVI, 522 Ethernet AD per ESI, Inclusive Multicast Ethernet Tag, and Ethernet 523 Segment) advertised by an egress PE. Five new values have been 524 assigned by IANA to extend the list of encapsulation types defined in 525 [RFC5512]: 527 + 8 - VXLAN Encapsulation 528 + 9 - NVGRE Encapsulation 529 + 10 - MPLS Encapsulation 530 + 11 - MPLS in GRE Encapsulation 531 + 12 - VXLAN GPE Encapsulation 533 If the BGP Encapsulation extended community is not present, then the 534 default MPLS encapsulation or a statically configured encapsulation 535 is assumed. 537 The Ethernet Segment and Ethernet AD per ESI routes MAY be advertised 538 with multiple encapsulation types as long as they use the same EVPN 539 multi-homing procedures - e.g., the mix of VXLAN and NVGRE 540 encapsulation types is a valid one but not the mix of VXLAN and MPLS 541 encapsulation types. 543 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 544 be set to the IPv4 or IPv6 address of the NVE. The remaining fields 545 in each route are set as per [RFC7432]. 547 5.2 MPLS over GRE 549 The EVPN data-plane is modeled as an EVPN MPLS client layer sitting 550 over an MPLS PSN tunnel. Some of the EVPN functions (split-horizon, 551 aliasing, and backup-path) are tied to the MPLS client layer. If MPLS 552 over GRE encapsulation is used, then the EVPN MPLS client layer can 553 be carried over an IP PSN tunnel transparently. Therefore, there is 554 no impact to the EVPN procedures and associated data-plane 555 operation. 557 The existing standards for MPLS over GRE encapsulation as defined by 558 [RFC4023] can be used for this purpose; however, when it is used in 559 conjunction with EVPN the key field SHOULD be present, and SHOULD be 560 used to provide a 32-bit entropy field. The Checksum and Sequence 561 Number fields are not needed and their corresponding C and S bits 562 MUST be set to zero. 564 6 EVPN with Multiple Data Plane Encapsulations 566 The use of the BGP Encapsulation extended community allows each PE in 567 a given EVI to know each of the encapsulations supported by each of 568 the other PEs in that EVI. I.e., each of the PEs in a given EVI may 569 support multiple data plane encapsulations. An ingress PE can send a 570 frame to an egress PE only if the set of encapsulations advertised by 571 the egress PE in the subject MAC/IP Advertisement or per EVI Ethernet 572 AD route, forms a non-empty intersection with the set of 573 encapsulations supported by the ingress PE, and it is at the 574 discretion of the ingress PE which encapsulation to choose from this 575 intersection. (As noted in section 5.1.3, if the BGP Encapsulation 576 extended community is not present, then the default MPLS 577 encapsulation or a statically configured encapsulation is assumed.) 579 An ingress node that uses shared multicast trees for sending 580 broadcast or multicast frames MUST maintain distinct trees for each 581 different encapsulation type. 583 It is the responsibility of the operator of a given EVI to ensure 584 that all of the PEs in that EVI support at least one common 585 encapsulation. If this condition is violated, it could result in 586 service disruption or failure. The use of the BGP Encapsulation 587 extended community provides a method to detect when this condition is 588 violated but the actions to be taken are at the discretion of the 589 operator and are outside the scope of this document. 591 7 NVE Residing in Hypervisor 593 When a PE and its CEs are co-located in the same physical device, 594 e.g., when the PE resides in a server and the CEs are its VMs, the 595 links between them are virtual and they typically share fate; i.e., 596 the subject CEs are typically not multi-homed or if they are multi- 597 homed, the multi-homing is a purely local matter to the server 598 hosting the VM, and need not be "visible" to any other PEs, and thus 599 does not require any specific protocol mechanisms. The most common 600 case of this is when the NVE resides in the hypervisor. 602 In the sub-sections that follow, we will discuss the impact on EVPN 603 procedures for the case when the NVE resides on the hypervisor and 604 the VXLAN or NVGRE encapsulation is used. 606 7.1 Impact on EVPN BGP Routes & Attributes for VXLAN/NVGRE Encapsulation 608 In the scenario where all data centers are under a single 609 administrative domain, and there is a single global VNI/VSID space, 610 the RD MAY be set to zero in the EVPN routes. However, in the 611 scenario where different groups of data centers are under different 612 administrative domains, and these data centers are connected via one 613 or more backbone core providers as described in [NOV3-Framework], the 614 RD must be a unique value per EVI or per NVE as described in 615 [RFC7432]. In other words, whenever there is more than one 616 administrative domain for global VNI or VSID, then a non-zero RD MUST 617 be used, or whenever the VNI or VSID value have local significance, 618 then a non-zero RD MUST be used. It is recommend to use a non-zero RD 619 at all time. 621 When the NVEs reside on the hypervisor, the EVPN BGP routes and 622 attributes associated with multi-homing are no longer required. This 623 reduces the required routes and attributes to the following subset of 624 four out of the set of eight : 626 - MAC Advertisement Route 627 - Inclusive Multicast Ethernet Tag Route 628 - MAC Mobility Extended Community 629 - Default Gateway Extended Community 631 However, as noted in section 8.6 of [RFC7432] in order to enable a 632 single-homed ingress PE to take advantage of fast convergence, 633 aliasing, and backup-path when interacting with multi-homed egress 634 PEs attached to a given Ethernet segment, a single-homed ingress PE 635 SHOULD be able to receive and process Ethernet AD per ES and Ethernet 636 AD per EVI routes." 638 7.2 Impact on EVPN Procedures for VXLAN/NVGRE Encapsulation 640 When the NVEs reside on the hypervisors, the EVPN procedures 641 associated with multi-homing are no longer required. This limits the 642 procedures on the NVE to the following subset of the EVPN procedures: 644 1. Local learning of MAC addresses received from the VMs per section 645 10.1 of [RFC7432]. 647 2. Advertising locally learned MAC addresses in BGP using the MAC 648 Advertisement routes. 650 3. Performing remote learning using BGP per Section 10.2 of 651 [RFC7432]. 653 4. Discovering other NVEs and constructing the multicast tunnels 654 using the Inclusive Multicast Ethernet Tag routes. 656 5. Handling MAC address mobility events per the procedures of Section 657 16 in [RFC7432]. 659 However, as noted in section 8.6 of [RFC7432] in order to enable a 660 single-homed ingress PE to take advantage of fast convergence, 661 aliasing, and back-up path when interacting with multi-homed egress 662 PEs attached to a given Ethernet segment, a single-homed ingress PE 663 SHOULD implement the ingress node processing of Ethernet AD per ES 664 and Ethernet AD per EVI routes as defined in sections 8.2 Fast 665 Convergence and 8.4 Aliasing and Backup-Path of [RFC7432]. 667 8 NVE Residing in ToR Switch 668 In this section, we discuss the scenario where the NVEs reside in the 669 Top of Rack (ToR) switches AND the servers (where VMs are residing) 670 are multi-homed to these ToR switches. The multi-homing may operate 671 in All-Active or Single-Active redundancy mode. If the servers are 672 single-homed to the ToR switches, then the scenario becomes similar 673 to that where the NVE resides in the hypervisor, as discussed in 674 Section 5, as far as the required EVPN functionality. 676 [RFC7432] defines a set of BGP routes, attributes and procedures to 677 support multi-homing. We first describe these functions and 678 procedures, then discuss which of these are impacted by the 679 encapsulation (such as VXLAN or NVGRE) and what modifications are 680 required. 682 8.1 EVPN Multi-Homing Features 684 In this section, we will recap the multi-homing features of EVPN to 685 highlight the encapsulation dependencies. The section only describes 686 the features and functions at a high-level. For more details, the 687 reader is to refer to [RFC7432]. 689 8.1.1 Multi-homed Ethernet Segment Auto-Discovery 691 EVPN NVEs (or PEs) connected to the same Ethernet Segment (e.g. the 692 same server via LAG) can automatically discover each other with 693 minimal to no configuration through the exchange of BGP routes. 695 8.1.2 Fast Convergence and Mass Withdraw 697 EVPN defines a mechanism to efficiently and quickly signal, to remote 698 NVEs, the need to update their forwarding tables upon the occurrence 699 of a failure in connectivity to an Ethernet segment (e.g., a link or 700 a port failure). This is done by having each NVE advertise an 701 Ethernet A-D Route per Ethernet segment for each locally attached 702 segment. Upon a failure in connectivity to the attached segment, the 703 NVE withdraws the corresponding Ethernet A-D route. This triggers all 704 NVEs that receive the withdrawal to update their next-hop adjacencies 705 for all MAC addresses associated with the Ethernet segment in 706 question. If no other NVE had advertised an Ethernet A-D route for 707 the same segment, then the NVE that received the withdrawal simply 708 invalidates the MAC entries for that segment. Otherwise, the NVE 709 updates the next-hop adjacencies to point to the backup NVE(s). 711 8.1.3 Split-Horizon 713 If a server is multi-homed to two or more NVEs on an Ethernet segment 714 ES1 operating in all-active redundancy mode sends a multicast, 715 broadcast or unknown unicast packet to a one of these NVEs, then it 716 is important to ensure the packet is not looped back to the server 717 via another NVE connected to this server. The filtering mechanism on 718 the NVE to prevent such loop and packet duplication is called "split 719 horizon filtering'. 721 8.1.4 Aliasing and Backup-Path 723 In the case where a station is multi-homed to multiple NVEs, it is 724 possible that only a single NVE learns a set of the MAC addresses 725 associated with traffic transmitted by the station. This leads to a 726 situation where remote NVEs receive MAC advertisement routes, for 727 these addresses, from a single NVE even though multiple NVEs are 728 connected to the multi-homed station. As a result, the remote NVEs 729 are not able to effectively load-balance traffic among the NVEs 730 connected to the multi-homed Ethernet segment. This could be the 731 case, for e.g. when the NVEs perform data-path learning on the 732 access, and the load-balancing function on the station hashes traffic 733 from a given source MAC address to a single NVE. Another scenario 734 where this occurs is when the NVEs rely on control plane learning on 735 the access (e.g. using ARP), since ARP traffic will be hashed to a 736 single link in the LAG. 738 To alleviate this issue, EVPN introduces the concept of Aliasing. 739 This refers to the ability of an NVE to signal that it has 740 reachability to a given locally attached Ethernet segment, even when 741 it has learnt no MAC addresses from that segment. The Ethernet A-D 742 route per EVI is used to that end. Remote NVEs which receive MAC 743 advertisement routes with non-zero ESI SHOULD consider the MAC 744 address as reachable via all NVEs that advertise reachability to the 745 relevant Segment using Ethernet A-D routes with the same ESI and with 746 the Single-Active flag reset. 748 Backup-Path is a closely related function, albeit it applies to the 749 case where the redundancy mode is Single-Active. In this case, the 750 NVE signals that it has reachability to a given locally attached 751 Ethernet Segment using the Ethernet A-D route as well. Remote NVEs 752 which receive the MAC advertisement routes, with non-zero ESI, SHOULD 753 consider the MAC address as reachable via the advertising NVE. 754 Furthermore, the remote NVEs SHOULD install a Backup-Path, for said 755 MAC, to the NVE which had advertised reachability to the relevant 756 Segment using an Ethernet A-D route with the same ESI and with the 757 Single-Active flag set. 759 8.1.5 DF Election 761 If a CE is multi-homed to two or more NVEs on an Ethernet segment 762 operating in all-active redundancy mode, then for a given EVI only 763 one of these NVEs, termed the Designated Forwarder (DF) is 764 responsible for sending it broadcast, multicast, and, if configured 765 for that EVI, unknown unicast frames. 767 This is required in order to prevent duplicate delivery of multi- 768 destination frames to a multi-homed host or VM, in case of all-active 769 redundancy. 771 8.2 Impact on EVPN BGP Routes & Attributes 773 Since multi-homing is supported in this scenario, then the entire set 774 of BGP routes and attributes defined in [RFC7432] are used. As 775 discussed in Section 3.1.3, the VSID or VNI is carried in the 776 VNI/VSID field in the MAC Advertisement, Ethernet AD per EVI, and 777 Inclusive Multicast Ethernet Tag routes. 779 8.3 Impact on EVPN Procedures 781 Two cases need to be examined here, depending on whether the NVEs are 782 operating in Active/Standby or in All-Active redundancy. 784 First, lets consider the case of Active/Standby redundancy, where the 785 hosts are multi-homed to a set of NVEs, however, only a single NVE is 786 active at a given point of time for a given VNI or VSID. In this 787 case, the split-horizon and the aliasing functions are not required 788 but other functions such as multi-homed Ethernet segment auto- 789 discovery, fast convergence and mass withdraw, backup path, and DF 790 election are required. 792 Second, let's consider the case of All-Active redundancy. In this 793 case, out of the EVPN multi-homing features listed in section 8.1, 794 the use of the VXLAN or NVGRE encapsulation impacts the split-horizon 795 and aliasing features, since those two rely on the MPLS client layer. 796 Given that this MPLS client layer is absent with these types of 797 encapsulations, alternative procedures and mechanisms are needed to 798 provide the required functions. Those are discussed in detail next. 800 8.3.1 Split Horizon 802 In EVPN, an MPLS label is used for split-horizon filtering to support 803 active/active multi-homing where an ingress NVE adds a label 804 corresponding to the site of origin (aka ESI Label) when 805 encapsulating the packet. The egress NVE checks the ESI label when 806 attempting to forward a multi-destination frame out an interface, and 807 if the label corresponds to the same site identifier (ESI) associated 808 with that interface, the packet gets dropped. This prevents the 809 occurrence of forwarding loops. 811 Since the VXLAN or NVGRE encapsulation does not include this ESI 812 label, other means of performing the split-horizon filtering function 813 MUST be devised. The following approach is recommended for split- 814 horizon filtering when VXLAN or NVGRE encapsulation is used. 816 Every NVE track the IP address(es) associated with the other NVE(s) 817 with which it has shared multi-homed Ethernet Segments. When the NVE 818 receives a multi-destination frame from the overlay network, it 819 examines the source IP address in the tunnel header (which 820 corresponds to the ingress NVE) and filters out the frame on all 821 local interfaces connected to Ethernet Segments that are shared with 822 the ingress NVE. With this approach, it is required that the ingress 823 NVE performs replication locally to all directly attached Ethernet 824 Segments (regardless of the DF Election state) for all flooded 825 traffic ingress from the access interfaces (i.e. from the hosts). 826 This approach is referred to as "Local Bias", and has the advantage 827 that only a single IP address needs to be used per NVE for split- 828 horizon filtering, as opposed to requiring an IP address per Ethernet 829 Segment per NVE. 831 In order to prevent unhealthy interactions between the split horizon 832 procedures defined in [RFC7432] and the local bias procedures 833 described in this document, a mix of MPLS over GRE encapsulations on 834 the one hand and VXLAN/NVGRE encapsulations on the other on a given 835 Ethernet Segment is prohibited. 837 8.3.2 Aliasing and Backup-Path 839 The Aliasing and the Backup-Path procedures for VXLAN/NVGRE 840 encapsulation is very similar to the ones for MPLS. In case of MPLS, 841 two different Ethernet AD routes are used for this purpose. The one 842 used for Aliasing has a VPN scope and carries a VPN label but the one 843 used for Backup-Path has Ethernet segment scope and doesn't carry any 844 VPN specific info (e.g., Ethernet Tag and MPLS label are set to 845 zero). 847 9 Support for Multicast 849 The E-VPN Inclusive Multicast BGP route is used to discover the 850 multicast tunnels among the endpoints associated with a given VXLAN 851 VNI or NVGRE VSID. The Ethernet Tag field of this route is used to 852 encode the VNI for VLXAN or VSID for NVGRE. The Originating router's 853 IP address field is set to the NVE's IP address. This route is tagged 854 with the PMSI Tunnel attribute, which is used to encode the type of 855 multicast tunnel to be used as well as the multicast tunnel 856 identifier. The tunnel encapsulation is encoded by adding the BGP 857 Encapsulation extended community as per section 3.1.1. The following 858 tunnel types as defined in [RFC6514] can be used in the PMSI tunnel 859 attribute for VXLAN/NVGRE: 861 + 3 - PIM-SSM Tree 862 + 4 - PIM-SM Tree 863 + 5 - BIDIR-PIM Tree 864 + 6 - Ingress Replication 866 Except for Ingress Replication, this multicast tunnel is used by the 867 PE originating the route for sending multicast traffic to other PEs, 868 and is used by PEs that receive this route for receiving the traffic 869 originated by CEs connected to the PE that originated the route. 871 In the scenario where the multicast tunnel is a tree, both the 872 Inclusive as well as the Aggregate Inclusive variants may be used. In 873 the former case, a multicast tree is dedicated to a VNI or VSID. 874 Whereas, in the latter, a multicast tree is shared among multiple 875 VNIs or VSIDs. This is done by having the NVEs advertise multiple 876 Inclusive Multicast routes with different VNI or VSID encoded in the 877 Ethernet Tag field, but with the same tunnel identifier encoded in 878 the PMSI Tunnel attribute. 880 10 Inter-AS 882 For inter-AS operation, two scenarios must be considered: 884 - Scenario 1: The tunnel endpoint IP addresses are public 885 - Scenario 2: The tunnel endpoint IP addresses are private 887 In the first scenario, inter-AS operation is straight-forward and 888 follows existing BGP inter-AS procedures. However, in the first 889 scenario where the tunnel endpoint IP addresses are public, there may 890 be security concern regarding the distribution of these addresses 891 among different ASes. This security concern is one of the main 892 reasons for having the so called inter-AS "option-B" in MPLS VPN 893 solutions such as EVPN. 895 The second scenario is more challenging, because the absence of the 896 MPLS client layer from the VXLAN encapsulation creates a situation 897 where the ASBR has no fully qualified indication within the tunnel 898 header as to where the tunnel endpoint resides. To elaborate on this, 899 recall that with MPLS, the client layer labels (i.e. the VPN labels) 900 are downstream assigned. As such, this label implicitly has a 901 connotation of the tunnel endpoint, and it is sufficient for the ASBR 902 to look up the client layer label in order to identify the label 903 translation required as well as the tunnel endpoint to which a given 904 packet is being destined. With the VXLAN encapsulation, the VNI is 905 globally assigned and hence is shared among all endpoints. The 906 destination IP address is the only field which identifies the tunnel 907 endpoint in the tunnel header, and this address is privately managed 908 by every data center network. Since the tunnel address is allocated 909 out of a private address pool, then we either need to do a lookup 910 based on VTEP IP address in context of a VRF (e.g., use IP-VPN) or 911 terminate the VXLAN tunnel and do a lookup based on the tenant's MAC 912 address to identify the egress tunnel on the ASBR. This effectively 913 mandates that the ASBR to either run another overlay solution such as 914 IP-VPN over MPLS/IP core network or to be aware of the MAC addresses 915 of all VMs in its local AS, at the very least. 917 If VNIs/VSIDs have local significance, then the inter-AS operation 918 can be simplified to that of MPLS and thus MPLS inter-AS option B and 919 C can be leveraged in here. That's why the use of local significance 920 VNIs/VSIDs (e.g., MPLS labels) are recommended for inter-AS operation 921 of DC networks without gateways. 923 11 Acknowledgement 925 The authors would like to thank David Smith, John Mullooly, Thomas 926 Nadeau for their valuable comments and feedback. 928 12 Security Considerations 930 This document uses IP-based tunnel technologies to support data 931 plane transport. Consequently, the security considerations of those 932 tunnel technologies apply. This document defines support for VXLAN 933 and NVGRE encapsulations. The security considerations from those 934 documents as well as [RFC4301] apply to the data plane aspects of 935 this document. 937 As with [RFC5512], any modification of the information that is used 938 to form encapsulation headers, to choose a tunnel type, or to choose 939 a particular tunnel for a particular payload type may lead to user 940 data packets getting misrouted, misdelivered, and/or dropped. 942 More broadly, the security considerations for the transport of IP 943 reachability information using BGP are discussed in [RFC4271] and 944 [RFC4272], and are equally applicable for the extensions described 945 in this document. 947 If the integrity of the BGP session is not itself protected, then an 948 imposter could mount a denial-of-service attack by establishing 949 numerous BGP sessions and forcing an IPsec SA to be created for each 950 one. However, as such an imposter could wreak havoc on the entire 951 routing system, this particular sort of attack is probably not of 952 any special importance. 954 It should be noted that a BGP session may itself be transported over 955 an IPsec tunnel. Such IPsec tunnels can provide additional security 956 to a BGP session. The management of such IPsec tunnels is outside 957 the scope of this document. 959 13 IANA Considerations 961 IANA has allocated the following BGP Tunnel Encapsulation Attribute 962 Tunnel Types: 964 8 VXLAN Encapsulation 965 9 NVGRE Encapsulation 966 10 MPLS Encapsulation 967 11 MPLS in GRE Encapsulation 968 12 VXLAN GPE Encapsulation 970 14 References 972 14.1 Normative References 974 [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate 975 Requirement Levels", BCP 14, RFC 2119, March 1997. 977 [RFC4271] Y. Rekhter, Ed., T. Li, Ed., S. Hares, Ed., "A Border 978 Gateway Protocol 4 (BGP-4)", January 2006. 980 [RFC4272] S. Murphy, "BGP Security Vulnerabilities Analysis.", 981 January 2006. 983 [RFC4301] S. Kent, K. Seo., "Security Architecture for the 984 Internet Protocol.", December 2005. 986 [RFC5512] Mohapatra, P. and E. Rosen, "The BGP Encapsulation 987 Subsequent Address Family Identifier (SAFI) and the BGP 988 Tunnel Encapsulation Attribute", RFC 5512, April 2009. 990 [RFC7432] Sajassi et al., "BGP MPLS Based Ethernet VPN", RFC 7432, 991 February 2014 993 14.2 Informative References 995 [RFC7209] Sajassi et al., "Requirements for Ethernet VPN (EVPN)", RFC 996 7209, May 2014 998 [RFC7348] Mahalingam, M., et al, "VXLAN: A Framework for Overlaying 999 Virtualized Layer 2 Networks over Layer 3 Networks", RFC 7348, August 1000 2014 1002 [NVGRE] Garg, P., et al., "NVGRE: Network Virtualization using 1003 Generic Routing Encapsulation", draft-sridharan-virtualization-nvgre- 1004 07.txt, November 11, 2014 1006 [Problem-Statement] Narten et al., "Problem Statement: Overlays for 1007 Network Virtualization", draft-ietf-nvo3-overlay-problem-statement- 1008 01, September 2012. 1010 [L3VPN-ENDSYSTEMS] Marques et al., "BGP-signaled End-system IP/VPNs", 1011 draft-ietf-l3vpn-end-system, work in progress, October 2012. 1013 [NOV3-FRWK] Lasserre et al., "Framework for DC Network 1014 Virtualization", draft-ietf-nvo3-framework-01.txt, work in progress, 1015 October 2012. 1017 Authors' Addresses 1019 Ali Sajassi 1020 Cisco 1021 Email: sajassi@cisco.com 1023 John Drake 1024 Juniper Networks 1025 Email: jdrake@juniper.net 1027 Nabil Bitar 1028 Verizon Communications 1029 Email : nabil.n.bitar@verizon.com 1031 Aldrin Isaac 1032 Bloomberg 1033 Email: aisaac71@bloomberg.net 1035 James Uttaro 1036 AT&T 1037 Email: uttaro@att.com 1038 Wim Henderickx 1039 Alcatel-Lucent 1040 e-mail: wim.henderickx@alcatel-lucent.com 1042 Ravi Shekhar 1043 Juniper Networks 1044 Email: rshekhar@juniper.net 1046 Samer Salam 1047 Cisco 1048 Email: ssalam@cisco.com 1050 Keyur Patel 1051 Cisco 1052 Email: Keyupate@cisco.com 1054 Dhananjaya Rao 1055 Cisco 1056 Email: dhrao@cisco.com 1058 Samir Thoria 1059 Cisco 1060 Email: sthoria@cisco.com