idnits 2.17.1 draft-boutros-l2vpn-vxlan-evpn-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 14 instances of too long lines in the document, the longest one being 2 characters in excess of 72. ** There are 3 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 25 has weird spacing: '...provide intra...' -- The document date (July 2, 2014) is 3576 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'EVPN-OVERLY' is mentioned on line 121, but not defined == Missing Reference: 'RFC2119' is mentioned on line 129, but not defined == Missing Reference: 'LACP' is mentioned on line 259, but not defined == Missing Reference: 'PBB-EVPN' is mentioned on line 311, but not defined == Unused Reference: 'KEYWORDS' is defined on line 563, but no explicit reference was found in the text == Unused Reference: 'TRILL' is defined on line 571, but no explicit reference was found in the text == Unused Reference: 'NVGRE' is defined on line 578, but no explicit reference was found in the text == Outdated reference: A later version (-11) exists of draft-ietf-l2vpn-evpn-00 == Outdated reference: A later version (-02) exists of draft-ietf-l2vpn-trill-evpn-00 == Outdated reference: A later version (-09) exists of draft-mahalingam-dutt-dcops-vxlan-02 == Outdated reference: A later version (-08) exists of draft-sridharan-virtualization-nvgre-01 Summary: 2 errors (**), 0 flaws (~~), 13 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT Sami Boutros 3 Intended Status: Informational Ali Sajassi 4 Samer Salam 5 Dennis Cai 6 Samir Thoria 7 Cisco Systems 9 Tapraj Singh 10 John Drake 11 Juniper Networks 13 Jeff Tantsura 14 Ericsson 16 Expires: January 3, 2015 July 2, 2014 18 VXLAN DCI Using EVPN 19 draft-boutros-l2vpn-vxlan-evpn-04.txt 21 Abstract 23 This document describes how Ethernet VPN (E-VPN) technology can be 24 used to interconnect VXLAN or NVGRE networks over an MPLS/IP network. 25 This is to provide intra-subnet connectivity at Layer 2 and control- 26 plane separation among the interconnected VXLAN or NVGRE networks. 27 The scope of the learning of host MAC addresses in VXLAN or NVGRE 28 network is limited to data plane learning in this document. 30 Status of this Memo 32 This Internet-Draft is submitted to IETF in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF), its areas, and its working groups. Note that 37 other groups may also distribute working documents as 38 Internet-Drafts. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 The list of current Internet-Drafts can be accessed at 46 http://www.ietf.org/1id-abstracts.html 47 The list of Internet-Draft Shadow Directories can be accessed at 48 http://www.ietf.org/shadow.html 50 Copyright and License Notice 52 Copyright (c) 2014 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 68 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 4 69 2. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 2.1. Control Plane Separation among VXLAN/NVGRE Networks . . . . 4 71 2.2 All-Active Multi-homing . . . . . . . . . . . . . . . . . . 5 72 2.3 Layer 2 Extension of VNIs/VSIDs over the MPLS/IP Network . . 5 73 2.4 Support for Integrated Routing and Bridging (IRB) . . . . . 5 74 3. Solution Overview . . . . . . . . . . . . . . . . . . . . . . . 5 75 3.1. Redundancy and All-Active Multi-homing . . . . . . . . . . 6 76 4. EVPN Routes . . . . . . . . . . . . . . . . . . . . . . . . . 7 77 4.1. BGP MAC Advertisement Route . . . . . . . . . . . . . . . 7 78 4.2. Ethernet Auto-Discovery Route . . . . . . . . . . . . . . 8 79 4.3. Per VPN Route Targets . . . . . . . . . . . . . . . . . . 8 80 4.4 Inclusive Multicast Route . . . . . . . . . . . . . . . . . 8 81 4.5. Unicast Forwarding . . . . . . . . . . . . . . . . . . . . 8 82 4.6. Handling Multicast . . . . . . . . . . . . . . . . . . . . 9 83 4.6.2. Multicast Stitching with Per-VNI Load Balancing . . . . 9 84 5. NVGRE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 85 6. Use Cases Overview . . . . . . . . . . . . . . . . . . . . . . 10 86 6.1. Homogeneous Network DCI interconnect Use cases . . . . . . 10 87 6.1.1. VNI Base Mode EVPN Service Use Case . . . . . . . . . . 10 88 6.1.2. VNI Bundle Service Use Case Scenario . . . . . . . . . 11 89 6.1.3. VNI Translation Use Case . . . . . . . . . . . . . . 12 90 6.2. Heterogeneous Network DCI Use Cases Scenarios . . . . . . . 12 91 6.2.1. VXLAN VLAN Interworking Over EVPN Use Case Scenario . . 12 93 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 13 94 8. Security Considerations . . . . . . . . . . . . . . . . . . . 13 95 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 96 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 97 10.1 Normative References . . . . . . . . . . . . . . . . . . . 13 98 10.2 Informative References . . . . . . . . . . . . . . . . . . 13 99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14 101 1 Introduction 103 [EVPN] introduces a solution for multipoint L2VPN services, with 104 advanced multi-homing capabilities, using BGP control plane over the 105 core MPLS/IP network. [VXLAN] defines a tunneling scheme to overlay 106 Layer 2 networks on top of Layer 3 networks. [VXLAN] allows for 107 optimal forwarding of Ethernet frames with support for multipathing 108 of unicast and multicast traffic. VXLAN uses UDP/IP encapsulation for 109 tunneling. 111 In this document, we discuss how Ethernet VPN (EVPN) technology can 112 be used to interconnect VXLAN or NVGRE networks over an MPLS/IP 113 network. This is achieved by terminating the VxLAN tunnel at the 114 hand-off points, performing data plane MAC learning of customer 115 traffic and providing intra-subnet connectivity for the customers at 116 Layer 2 across the MPLS/IP core. The solution maintains control-plane 117 separation among the interconnected VXLAN or NVGRE networks. The 118 scope of the learning of host MAC addresses in VXLAN or NVGRE network 119 is limited to data plane learning in this document. The distribution 120 of MAC addresses in control plane using BGP in VXLAN or NVGRE network 121 is outside of the scope of this document and it is covered in [EVPN- 122 OVERLY]. 124 1.1 Terminology 126 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 127 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 128 document are to be interpreted as described in RFC 2119 [RFC2119]. 130 LDP: Label Distribution Protocol. MAC: Media Access Control MPLS: 131 Multi Protocol Label Switching. OAM: Operations, Administration and 132 Maintenance. PE: Provide Edge Node. PW: PseudoWire. TLV: Type, 133 Length, and Value. VPLS: Virtual Private LAN Services. VXLAN: Virtual 134 eXtensible Local Area Network. VTEP: VXLAN Tunnel End Point VNI: 135 VXLAN Network Identifier (or VXLAN Segment ID) ToR: Top of Rack 136 switch. 138 2. Requirements 140 2.1. Control Plane Separation among VXLAN/NVGRE Networks 142 It is required to maintain control-plane separation for the underlay 143 networks (e.g., among the various VXLAN/NVGRE networks) being 144 interconnected over the MPLS/IP network. This ensures the following 145 characteristics: 147 - scalability of the IGP control plane in large deployments and fault 148 domain localization, where link or node failures in one site do not 149 trigger re-convergence in remote sites. 151 - scalability of multicast trees as the number of interconnected 152 networks scales. 154 2.2 All-Active Multi-homing 156 It is important to allow for all-active multi-homing of the 157 VXLAN/NVGRE network to MPLS/IP network where traffic from a VTEP can 158 arrive at any of the PEs and can be forwarded accordingly over the 159 MPLS/IP network. Furthermore, traffic destined to a VTEP can be 160 received over the MPLS/IP network at any of the PEs connected to the 161 VXLAN/NVGRE network and be forwarded accordingly. The solution MUST 162 support all-active multi-homing to an VXLAN/NVGRE network. 164 2.3 Layer 2 Extension of VNIs/VSIDs over the MPLS/IP Network 166 It is required to extend the VXLAN VNIs or NVGRE VSIDs over the 167 MPLS/IP network to provide intra-subnet connectivity between the 168 hosts (e.g. VMs) at Layer 2. 170 2.4 Support for Integrated Routing and Bridging (IRB) 172 The data center WAN edge node is required to support integrated 173 routing and bridging in order to accommodate both inter-subnet 174 routing and intra-subnet bridging for a given VNI/VSID. For example, 175 inter-subnet switching is required when a remote host connected to an 176 enterprise IP-VPN site wants to access an application resided on a 177 VM. 179 3. Solution Overview 181 Every VXLAN/NVGRE network, which is connected to the MPLS/IP core, 182 runs an independent instance of the IGP control-plane. Each PE 183 participates in the IGP control plane instance of its VXLAN/NVGRE 184 network. 186 Each PE node terminates the VXLAN or NVGRE data-plane encapsulation 187 where each VNI or VSID is mapped to a bridge-domain. The PE performs 188 data plane MAC learning on the traffic received from the VXLAN/NVGRE 189 network. 191 Each PE node implements EVPN or PBB-EVPN to distribute in BGP either 192 the client MAC addresses learnt over the VXLAN tunnel in case of 193 EVPN, or the PEs' B-MAC addresses in case of PBB-EVPN. In the PBB- 194 EVPN case, client MAC addresses will continue to be learnt in data 195 plane. 197 Each PE node would encapsulate the Ethernet frames with MPLS when 198 sending the packets over the MPLS core and with the VXLAN or NVGRE 199 tunnel header when sending the packets over the VXLAN or NVGRE 200 Network. 202 +--------------+ 203 | | 204 +---------+ +----+ MPLS +----+ +---------+ 205 +-----+ | |---|PE1 | |PE3 |--| | +-----+ 206 |VTEP1|--| | +----+ +----+ | |--|VTEP3| 207 +-----+ | VXLAN | +----+ +----+ | VXLAN | +-----+ 208 +-----+ | |---|PE2 | |PE4 |--| | +-----+ 209 |VTEP2|--| | +----+Backbone+----+ | |--|VTEP4| 210 +-----+ +---------+ +--------------+ +---------+ +-----+ 212 |<--- Underlay IGP ---->|<-Overlay BGP->|<--- Underlay IGP --->| CP 214 |<----- VXLAN --------->||<------ VXLAN ------->| DP 215 |<----MPLS----->| 217 Legend: CP = Control Plane View DP = Data Plane View 219 Figure 1: Interconnecting VXLAN Networks with VXLAN-EVPN 221 3.1. Redundancy and All-Active Multi-homing 223 When a VXLAN network is multi-homed to two or more PEs, and provided 224 that these PEs have the same IGP distance to a given NVE, the 225 solution MUST support load-balancing of traffic between the NVE and 226 the MPLS network, among all the multi-homed PEs. This maximizes the 227 use of the bisectional bandwidth of the VXLAN network. One of the 228 main capabilities of EVPN/PBB-EVPN is the support for all-active 229 multi-homing, where the known unicast traffic to/from a multi-homed 230 site can be forwarded by any of the PEs attached to that site. This 231 ensures optimal usage of multiple paths and load balancing. EVPN/PBB- 232 EVPN, through its DF election and split-horizon filtering mechanisms, 233 ensures that no packet duplication or forwarding loops result in such 234 scenarios. In this solution, the VXLAN network is treated as a 235 multi-homed site for the purpose of EVPN operation. 237 Since the context of this solution is VXLAN networks with data-plane 238 learning paradigm, it is important for the multi-homing mechanism to 239 ensure stability of the MAC forwarding tables at the NVEs, while 240 supporting all-active forwarding at the PEs. For example, in Figure 1 241 above, if each PE uses a distinct IP address for its VTEP tunnel, 242 then for a given VNI, when an NVE learns a host's MAC address against 243 the originating VTEP source address, its MAC forwarding table will 244 keep flip-flopping among the VTEP addresses of the local PEs. This is 245 because a flow associated with the same host MAC address can arrive 246 at any of the PE devices. In order to ensure that there is no 247 flip/flopping of MAC-to-VTEP address associations, an IP Anycast 248 address MUST be used as the VTEP address on all PEs multi-homed to a 249 given VXLAN network. The use of IP Anycast address has two 250 advantages: 252 a) It prevents any flip/flopping in the forwarding tables for the 253 MAC-to-VTEP associations 255 b) It enables load-balancing via ECMP for DCI traffic among the 256 multi-homed PEs 258 In the baseline [EVPN] draft, the all-active multi-homing is 259 described for a multi-homed device (MHD) using [LACP] and the single- 260 active multi-homing is described for a multi-homed network (MHN) 261 using [802.1Q]. In this draft, the all-active multi-homing is 262 described for a VXLAN MHN. This implies some changes to the filtering 263 which will be described in details in the multicast section (Section 264 4.6.2). 266 The filtering used for BUM traffic of all-active multi-homing in 267 [EVPN] is asymmetric; where the BUM traffic from the MPLS/IP network 268 towards the multi-homed site is filtered on non-DF PE(s) and it 269 passes thorough the DF PE. There is no filtering of BUM traffic 270 originating from the multi-homed site because of the use of Ethernet 271 Link Aggregation: the MHD hashes the BUM traffic to only a single 272 link. However, in this solution because BUM traffic can arrive at 273 both PEs in both core-to-site and site-to-core directions, the 274 filtering needs to be symmetric just like the filtering of BUM 275 traffic for single-active multi-homing (on a per service 276 instance/VLAN basis). 278 4. EVPN Routes 280 This solution leverages the same BGP Routes and Attributes defined in 281 [EVPN], adapted as follows: 283 4.1. BGP MAC Advertisement Route 285 This route and its associated modes are used to distribute the 286 customer MAC addresses learnt in data plane over the VXLAN tunnel in 287 case of EVPN. Or can be used to distribute the provider Backbone MAC 288 addresses in case of PBB-EVPN. 290 In case of EVPN, the Ethernet Tag ID of this route is set to zero for 291 VNI-based mode, where there is one-to-one mapping between a VNI and 292 an EVI. In such case, there is no need to carry the VNI in the MAC 293 advertisement route because BD ID can be derived from the RT 294 associated with this route. However, for VNI-aware bundle mode, where 295 there is multiple VNIs can be mapped to the same EVI, the Ethernet 296 Tag ID MUST be set to the VNI. At the receiving PE, the BD ID is 297 derived from the combination of RT + VNI - e.g., the RT identifies 298 the associated EVI on that PE and the VNI identifies the 299 corresponding BD ID within that EVI. 301 The Ethernet Tag field can be set to a normalized value that maps to 302 the VNI, in VNI aware bundling services, this would make the VNI 303 value of local significance in multiple Data centers. Data plane need 304 to map to this normalized VNI value and have it on the IP VxLAN 305 packets exchanged between the DCIs. 307 4.2. Ethernet Auto-Discovery Route 309 When EVPN is used, the application of this route is as specified in 310 [EVPN]. However, when PBB-EVPN is used, there is no need for this 311 route per [PBB-EVPN]. 313 4.3. Per VPN Route Targets 315 VXLAN-EVPN uses the same set of route targets defined in [EVPN]. 317 4.4 Inclusive Multicast Route 319 The EVPN Inclusive Multicast route is used for auto-discovery of PE 320 devices participating in the same tenant virtual network identified 321 by a VNI over the MPLS network. It also enables the stitching of the 322 IP multicast trees, which are local to each VXLAN site, with the 323 Label Switched Multicast (LSM) trees of the MPLS network. 325 The Inclusive Multicast Route is encoded as follow: 327 - Ethernet Tag ID is set to zero for VNI-based mode and to VNI for 328 VNI-aware bundle mode. 330 - Originating Router's IP Address is set to one of the PE's IP 331 addresses. 333 All other fields are set as defined in [EVPN]. 335 Please see section 4.6 "Handling Multicast" 337 4.5. Unicast Forwarding 339 Host MAC addresses will be learnt in data plane from the VXLAN 340 network and associated with the corresponding VTEP identified by the 341 source IP address. Host MAC addresses will be learnt in control plane 342 if EVPN is implemented over the MPLS/IP core, or in the data-plane if 343 PBB-EVPN is implemented over the MPLS core. When Host MAC addressed 344 are learned in data plane over MPLS/IP core [in case of PBB-EVPN], 345 they are associated with their corresponding BMAC addresses. 347 L2 Unicast traffic destined to the VXLAN network will be encapsulated 348 with the IP/UDP header and the corresponding customer bridge VNI. 350 L2 Unicast traffic destined to the MPLS/IP network will be 351 encapsulated with the MPLS label. 353 4.6. Handling Multicast 355 Each VXLAN network independently builds its P2MP or MP2MP shared 356 multicast trees. A P2MP or MP2MP tree is built for one or more VNIs 357 local to the VXLAN network. 359 In the MPLS/IP network, multiple options are available for the 360 delivery of multicast traffic: - Ingress replication - LSM 361 with Inclusive trees - LSM with Aggregate Inclusive trees - 362 LSM with Selective trees - LSM with Aggregate Selective trees 364 When LSM is used, the trees are P2MP. 366 The PE nodes are responsible for stitching the IP multicast trees, on 367 the access side, to the ingress replication tunnels or LSM trees in 368 the MPLS/IP core. The stitching must ensure that the following 369 characteristics are maintained at all times: 371 1. Avoiding Packet Duplication: In the case where the VXLAN network 372 is multi-homed to multiple PE nodes, if all of the PE nodes forward 373 the same multicast frame, then packet duplication would arise. This 374 applies to both multicast traffic from site to core as well as from 375 core to site. 377 2. Avoiding Forwarding Loops: In the case of VXLAN network multi- 378 homing, the solution must ensure that a multicast frame forwarded by 379 a given PE to the MPLS core is not forwarded back by another PE (in 380 the same VXLAN network) to the VXLAN network of origin. The same 381 applies for traffic in the core to site direction. 383 The following approach of per-VNI load balancing can guarantee proper 384 stitching that meets the above requirements. 386 4.6.2. Multicast Stitching with Per-VNI Load Balancing 387 To setup multicast trees in the VXLAN network for DC applications, 388 PIM Bidir can be of special interest because it reduces the amount of 389 multicast state in the network significantly. Furthermore, it 390 alleviates any special processing for RPF check since PIM Bidir 391 doesn't require any RPF check. The RP for PIM Bidir can be any of the 392 spine nodes. Multiple trees can be built (e.g., one tree rooted per 393 spine node) for efficient load-balancing within the network. All PEs 394 participating in the multi-homing of the VXLAN network join all the 395 trees. Therefore, for a given tree, all PEs receive BUM traffic. DF 396 election procedures of [EVPN] are used to ensure that only traffic 397 to/from a single PE is forwarded, thus avoiding packet duplications 398 and forwarding loops. For load-balancing of BUM traffic, when a PE or 399 an NVE wants to send BUM traffic over the VXLAN network, it selects 400 one of the trees based on its VNI and forwards all the traffic for 401 that VNI on that tree. PIM SM will be described in future revision 402 of this draft. 404 Multicast traffic from VXLAN/NVGRE is first subjected to filtering 405 based on DF election procedures of [EVPN] using the VNI as the 406 Ethernet Tag. This is similar to filtering in [EVPN] in principal; 407 however, instead of VLAN ID, VNI is used for filtering, and instead 408 of being 802.1Q frame, it is a VXLAN encapsulated packet. On the DF 409 PE, where the multicast traffic is allowed to be forwarded, the VNI 410 is used to select a bridge domain,. After the packet is de- 411 capsulated, an L2 lookup is performed based on host MAC DA. It should 412 be noted that the MAC learning is performed in data-plane for the 413 traffic received from the VXLAN/NVGRE network and the host MAC SA is 414 learnt against the source VTEP address. 416 The PE nodes, connected to a multi-homed VXLAN network, perform BGP 417 DF election to decide which PE node is responsible for forwarding 418 multicast traffic associated with a given VNI. A PE would forward 419 multicast traffic for a given VNI only when it is the DF for this 420 VNI. This forwarding rule applies in both the site-to-core as well as 421 core-to-site directions. 423 5. NVGRE 425 Just like VXLAN, all the above specification would apply for NVGRE, 426 replacing the VNI with Virtual Subnet Identifier (VSID) and the VTEP 427 with NVGRE Endpoint. 429 6. Use Cases Overview 430 6.1. Homogeneous Network DCI interconnect Use cases This covers DCI 431 interconnect of two or more VXLAN based Data center over MPLS enabled 432 EVPN core. 434 6.1.1. VNI Base Mode EVPN Service Use Case This use case handles the 435 EVPN service where there is one to one mapping between a VNI and an 436 EVI. Ethernet TAG ID of EVPN BGP NLRI should be set to Zero. BD ID 437 can be derived from the RT associated with the EVI/VNI. 439 +---+ +---+ 440 | H1| +---+ +-------+ +--+ +---------+ +---+ +-------+ +---+ | H3| 441 | M1|--+ +-+ +-+PE1+-+ +-+PE3+--+ +--+ +--| M3| 442 +---+ | | | | +--+ |MPLS Core| +---+ | | | | +---+ 443 +---+ |NVE| | VXLAN | | (EVPN) | | VXLAN | |NVE| +---+ 444 | H2| | 1 | | | +--+ | | +---+ | | | 2 | | H4| 445 | M2|--+ +-+ +-+PE2+-+ +-+PE4+--+ +--+ +--| M4| 446 +---+ +---+ +-------+ +--+ +---------+ +---+ +-------+ +---+ +---+ 447 +--------++------+--------++------+--------++------+--------++--------+ 448 |Original||VXLAN |Original||MPLS |Original||VXLAN |Original||Original| 449 |Ethernet||Header|Ethernet||Header|Ethernet||Header|Ethernet||Ethernet| 450 |Frame || |Frame || |Frame || |Frame ||Frame | 451 +--------++------+--------++------+--------++------+--------++--------+ 452 |<----Data Center Site1-->|<------EVPN Core>|<----Data Center Site2-->| 454 Figure 2 VNI Base Service Packet Flow. 456 VNI base Service(One VNI mapped to one EVI). 458 Hosts H1, H2, H3 and H4 are hosts and there associated MAC addresses 459 are M1, M2, M3 and M4. PE1, PE2, PE3 and PE4 are the VXLAN-EVPN 460 gateways. NVE1 and NVE2 are the originators of the VXLAN based 461 network. 463 When host H1 in Data Center Site1 communicates with H3 in Data Center 464 Site2, H1 forms a layer2 packet with source IP address as IP1 and 465 Source MAC M1, Destination IP as IP3 and Destination MAC as 466 M3(assuming that ARP resolution already happened). VNE1 learns Source 467 MAC and lookup in bridge domain for the Destination MAC. Based on the 468 MAC lookup, the frame needs to be sent to VXLAN network. VXLAN 469 encapsulation is added to the original Ethernet frame and frame is 470 sent over the VXLAN tunnel. Frames arrives at PE1. PE1(i.e. VXLAN 471 gateway), identifies that frame is a VXLAN frame. The VXLAN header is 472 de-capsulated and Destination MAC lookup is done in the bridge domain 473 table of the EVI. Lookup of destination MAC results in the EVPN 474 unicast NH. This NH will be used for identifying the labels (tunnel 475 label and service label) to be added over the EVPN core. Similar 476 processing is done on the other side of DCI. 478 6.1.2. VNI Bundle Service Use Case Scenario 480 In the case of VNI-aware bundle service mode, there are multiple VNIs 481 are mapped to one EVI. The Ethernet TAG ID must be set to the VNI ID 482 in the EVPN BGP NLRIs. MPLS label allocation in this use case 483 scenario can be done either per EVI or per EVI, VNI ID basis. If MPLS 484 label allocation is done per EVI basis, then in data path there is a 485 need to push a VLAN TAG for identifying bridge-domain at egress PE so 486 that Destination MAC address lookup can be done on the bridge domain. 488 6.1.3. VNI Translation Use Case 489 +---+ +---+ 490 | H1| +---+ +-------+ +---+ +----------+ +---+ +-------+ +---+ | H3| 491 | M1|-+ +-+ +-+PE1+-+ +-+PE3+-+ +-+ +-| M3| 492 +---+ | | | | +---+ |MPLS Core | +---+ | | | | +---+ 493 +---+ |NVE| | VXLAN | | (EVPN) | | VXLAN | |NVE| +---+ 494 | H2| | 1 | | | +---+ | | +---+ | | | 2 | | H4| 495 | M2|-+ +-+ +-+PE2+-+ +-+PE4+-+ +-+ +-| M4| 496 +---+ +---+ +-------+ +---+ +----------+ +---+ +-------+ +---+ +---+ 497 |<----VNI ID A--->|<-------EVI-A------->|<----VNI_ID_B--->| 499 Figure 3 VNI Translation Use Case Scenarios. 501 There are two or more Data Center sites. These Data Center sites 502 might use different VNI ID for same service. For example, Service A 503 usage "VNI_ID_A" at data center site1 and "VNI_ID_B" for same service 504 in data center site 2. VNI ID A is terminated at ingress EVPN PE and 505 VNI ID B is encapsulated at the egress EVPN PE. 507 6.2. Heterogeneous Network DCI Use Cases Scenarios 509 Data Center sites are upgraded slowly; so heterogeneous network DCI 510 solution is required from the perspective of migration approach from 511 traditional data center to VXLAN based data center. For Example Data 512 Center Site1 is upgrade to VXLAN but Data Center Site 2 and 3 are 513 still layer2/VLAN based data centers. For these use cases, it is 514 required to provide VXLAN VLAN interworking over EVPN core. 516 6.2.1. VXLAN VLAN Interworking Over EVPN Use Case Scenario 518 The new data center site is VXLAN based data center site. But the 519 older data center sites are still based on the VLAN. 521 +---+ +---+ 522 | H1| +---+ +------+ +---+ +---------+ +---+ +-------+ +---+ | H3| 523 | M1|-+ +-+ +-+PE1+-+ +-+PE3+-+ +-+ +-| M3| 524 +---+ | | | | +---+ |MPLS Core| +---+ | | | | +---+ 525 +---+ |NVE| |VXLAN | | (EVPN) | | L2 | |NVE| +---+ 526 | H2| | 1 | | | +---+ | | +---+ |Network| | 2 | | H4| 527 | M2|-+ +-+ +-+PE2+-+ +-+PE4+-+ +-+ +-| M4| 528 +---+ +---+ +------+ +---+ +---------+ +---+ +-------+ +---+ +---+ 529 |<--Data Center Site1->|<---EVPN Core--->|<--Data Center Site2-->| 530 +-----+ +------+-----+ +------+------+-----+ +------+-----+ +-----+ 531 |L2 | |VXLAN |L2 | |MPLS |VLAN |L2 | |VLAN |L2 | |L2 | 532 |Frame| |Header|Frame| |Header|Header|Frame| |Header|Frame| |Frame| 533 +-----+ +------+-----+ +------+------+-----+ +------+-----+ +-----+ 535 Figure 5 VXLAN VLAN interworking over EVPN Use Case 537 If a service that are represented by VXLAN on one site of data center 538 and via VLAN at different data center sites, then it is a recommended 539 to model the service as a VNI base EVPN service. The BGP NLRIs will 540 always advertise VLAN ID TAG as '0' in BGP routes. The advantage with 541 this approach is that there is no requirement to do the VNI 542 normalization at EVPN core. VNI ID A is terminated at ingress EVPN PE 543 and "VLAN ID B" is encapsulated at the egress EVPN PE. 545 7. Acknowledgements 547 The authors would like to acknowledge Wen Lin contributions to this 548 document. 550 8. Security Considerations 552 There are no additional security aspects that need to be discussed 553 here. 555 9. IANA Considerations 557 TBD 559 10. References 561 10.1 Normative References 563 [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate 564 Requirement Levels", BCP 14, RFC 2119, March 1997. 566 10.2 Informative References 568 [EVPN] Sajassi et al., "BGP MPLS Based Ethernet VPN", draft-ietf- 569 l2vpn-evpn-00.txt, work in progress, February, 2012. 571 [TRILL] Sajassi et al., TRILL-EVPN draft-ietf-l2vpn-trill-evpn-00, 572 work in progress, June 2012. 574 [VXLAN] Mahalingam, Dutt et al., A Framework for Overlaying 575 Virtualized Layer 2 Networks over Layer 3 Networks draft-mahalingam- 576 dutt-dcops-vxlan-02.txt, work in progress, August, 2012. 578 [NVGRE] Sridharan et al., Network Virtualization using Generic 579 Routing Encapsulation draft-sridharan-virtualization-nvgre-01.txt, 580 work in progress, July, 2012. 582 Authors' Addresses 584 Sami Boutros 585 Cisco Systems 587 EMail: sboutros@cisco.com 589 Ali Sajassi 590 Cisco Systems 592 EMail: sajassi@cisco.com 594 Samer Salam 595 Cisco Systems 596 EMail: ssalam@cisco.com 598 Dennis Cai 599 Cisco Systems 600 EMail: dcai@cisco.com 602 Tapraj Singh 603 Juniper Networks 604 Email: tsingh@juniper.net 606 John Drake 607 Juniper Networks 608 Email: jdrake@juniper.net 610 Samir Thoria 611 Cisco 612 EMail: sthoria@cisco.com 614 Jeff Tantsura 615 Ericsson 616 Email: jeff.tantsura@ericsson.com