idnits 2.17.1 draft-ietf-l2vpn-evpn-11.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 18, 2014) is 3471 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 5226 (Obsoleted by RFC 8126) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Sajassi, Ed. 3 INTERNET-DRAFT Cisco 4 Category: Standards Track 5 R. Aggarwal 6 J. Drake Arktan 7 Juniper Networks 8 N. Bitar 9 W. Henderickx Verizon 10 Alcatel-Lucent 11 Aldrin Isaac 12 Bloomberg 14 J. Uttaro 15 AT&T 17 Expires: April 18, 2015 October 18, 2014 19 BGP MPLS Based Ethernet VPN 20 draft-ietf-l2vpn-evpn-11 22 Status of this Memo 24 This Internet-Draft is submitted to IETF in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as 30 Internet-Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/1id-abstracts.html 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html 43 Copyright and License Notice 45 Copyright (c) 2014 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (http://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Abstract 60 This document describes procedures for BGP MPLS based Ethernet VPNs 61 (EVPN). The procedures described here meet the requirements specified 62 in RFC7209 - Requirements for Ethernet VPN. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 67 2. Specification of requirements . . . . . . . . . . . . . . . . . 5 68 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 69 4. BGP MPLS Based EVPN Overview . . . . . . . . . . . . . . . . . 6 70 5. Ethernet Segment . . . . . . . . . . . . . . . . . . . . . . . 7 71 6. Ethernet Tag ID . . . . . . . . . . . . . . . . . . . . . . . . 10 72 6.1 VLAN Based Service Interface . . . . . . . . . . . . . . . . 11 73 6.2 VLAN Bundle Service Interface . . . . . . . . . . . . . . . 11 74 6.2.1 Port Based Service Interface . . . . . . . . . . . . . . 11 75 6.3 VLAN Aware Bundle Service Interface . . . . . . . . . . . . 11 76 6.3.1 Port Based VLAN Aware Service Interface . . . . . . . . 12 77 7. BGP EVPN Routes . . . . . . . . . . . . . . . . . . . . . . . . 12 78 7.1. Ethernet Auto-Discovery Route . . . . . . . . . . . . . . . 13 79 7.2. MAC/IP Advertisement Route . . . . . . . . . . . . . . . . 13 80 7.3. Inclusive Multicast Ethernet Tag Route . . . . . . . . . . 14 81 7.4 Ethernet Segment Route . . . . . . . . . . . . . . . . . . . 15 82 7.5 ESI Label Extended Community . . . . . . . . . . . . . . . . 15 83 7.6 ES-Import Route Target . . . . . . . . . . . . . . . . . . . 16 84 7.7 MAC Mobility Extended Community . . . . . . . . . . . . . . 16 85 7.8 Default Gateway Extended Community . . . . . . . . . . . . . 17 86 7.9 Route Distinguisher Assignment per EVI . . . . . . . . . . . 17 87 7.10 Route Targets . . . . . . . . . . . . . . . . . . . . . . . 17 88 7.10.1 Auto-Derivation from the Ethernet Tag ID . . . . . . . 17 89 8. Multi-homing Functions . . . . . . . . . . . . . . . . . . . . 18 90 8.1 Multi-homed Ethernet Segment Auto-Discovery . . . . . . . . 18 91 8.1.1 Constructing the Ethernet Segment Route . . . . . . . . 18 92 8.2 Fast Convergence . . . . . . . . . . . . . . . . . . . . . . 18 93 8.2.1 Constructing Ethernet A-D per Ethernet Segment Route . . 19 94 8.2.1.1. Ethernet A-D Route Targets . . . . . . . . . . . . 20 95 8.3 Split Horizon . . . . . . . . . . . . . . . . . . . . . . . 20 96 8.3.1 ESI Label Assignment . . . . . . . . . . . . . . . . . . 21 97 8.3.1.1 Ingress Replication . . . . . . . . . . . . . . . . 21 98 8.3.1.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . . 22 99 8.4 Aliasing and Backup-Path . . . . . . . . . . . . . . . . . . 23 100 8.4.1 Constructing Ethernet A-D per EVPN Instance Route . . . 24 101 8.5 Designated Forwarder Election . . . . . . . . . . . . . . . 25 102 8.6. Interoperability with Single-homing PEs . . . . . . . . . . 27 103 9. Determining Reachability to Unicast MAC Addresses . . . . . . . 27 104 9.1. Local Learning . . . . . . . . . . . . . . . . . . . . . . 28 105 9.2. Remote learning . . . . . . . . . . . . . . . . . . . . . . 28 106 9.2.1. Constructing MAC/IP Address Advertisement . . . . . . . 28 107 9.2.2 Route Resolution . . . . . . . . . . . . . . . . . . . . 30 108 10. ARP and ND . . . . . . . . . . . . . . . . . . . . . . . . . . 31 109 10.1 Default Gateway . . . . . . . . . . . . . . . . . . . . . . 32 110 11. Handling of Multi-Destination Traffic . . . . . . . . . . . . 33 111 11.1. Constructing Inclusive Multicast Ethernet Tag Route . . . 34 112 11.2. P-Tunnel Identification . . . . . . . . . . . . . . . . . 34 113 12. Processing of Unknown Unicast Packets . . . . . . . . . . . . 35 114 12.1. Ingress Replication . . . . . . . . . . . . . . . . . . . 36 115 12.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . . . . . . 36 116 13. Forwarding Unicast Packets . . . . . . . . . . . . . . . . . . 37 117 13.1. Forwarding packets received from a CE . . . . . . . . . . 37 118 13.2. Forwarding packets received from a remote PE . . . . . . . 38 119 13.2.1. Unknown Unicast Forwarding . . . . . . . . . . . . . . 38 120 13.2.2. Known Unicast Forwarding . . . . . . . . . . . . . . . 38 121 14. Load Balancing of Unicast Frames . . . . . . . . . . . . . . . 38 122 14.1. Load balancing of traffic from a PE to remote CEs . . . . 39 123 14.1.1 Single-Active Redundancy Mode . . . . . . . . . . . . . 39 124 14.1.2 All-Active Redundancy Mode . . . . . . . . . . . . . . 39 125 14.2. Load balancing of traffic between a PE and a local CE . . 41 126 14.2.1. Data plane learning . . . . . . . . . . . . . . . . . 41 127 14.2.2. Control plane learning . . . . . . . . . . . . . . . . 41 128 15. MAC Mobility . . . . . . . . . . . . . . . . . . . . . . . . . 42 129 15.1. MAC Duplication Issue . . . . . . . . . . . . . . . . . . 43 130 15.2. Sticky MAC addresses . . . . . . . . . . . . . . . . . . . 44 131 16. Multicast & Broadcast . . . . . . . . . . . . . . . . . . . . 44 132 16.1. Ingress Replication . . . . . . . . . . . . . . . . . . . 44 133 16.2. P2MP LSPs . . . . . . . . . . . . . . . . . . . . . . . . 44 134 16.2.1. Inclusive Trees . . . . . . . . . . . . . . . . . . . 45 135 17. Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 45 136 17.1. Transit Link and Node Failures between PEs . . . . . . . . 45 137 17.2. PE Failures . . . . . . . . . . . . . . . . . . . . . . . 46 138 17.3. PE to CE Network Failures . . . . . . . . . . . . . . . . 46 139 18. Frame Ordering . . . . . . . . . . . . . . . . . . . . . . . . 46 140 19. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 47 141 20. Security Considerations . . . . . . . . . . . . . . . . . . . 47 142 21. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 49 143 22. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 49 144 23. References . . . . . . . . . . . . . . . . . . . . . . . . . . 50 145 23.1 Normative References . . . . . . . . . . . . . . . . . . . 50 146 23.2 Informative References . . . . . . . . . . . . . . . . . . 50 147 24. Author's Address . . . . . . . . . . . . . . . . . . . . . . . 51 149 1. Introduction 151 This document describes procedures for BGP MPLS based Ethernet VPNs 152 (EVPN). The procedures described here meet the requirements specified 153 in [RFC7209]. Please refer to [RFC7209] for the detailed 154 requirements and motivation. EVPN requires extensions to existing 155 IP/MPLS protocols as described in this document. In addition to these 156 extensions EVPN uses several building blocks from existing MPLS 157 technologies. 159 2. Specification of requirements 161 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 162 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 163 document are to be interpreted as described in [RFC2119]. 165 3. Terminology 167 Broadcast Domain: In a bridged network, it corresponds to a Virtual 168 LAN (VLAN); where a VLAN is typically represented by a single VLAN ID 169 (VID), but can be represented by several VIDs where Shared VLAN 170 Learning (SVL) is used per [802.1Q]. 172 Bridge Domain: An instantiation of a broadcast domain on a bridge 173 node 175 CE: Customer Edge device e.g., host or router or switch 177 EVI: An EVPN instance spanning across the PEs participating in that 178 EVPN 180 MAC-VRF: A Virtual Routing and Forwarding table for MAC addresses on 181 a PE for an EVI 183 Ethernet Segment (ES): If a multi-homed device or network is 184 connected to two or more PEs via a set of Ethernet links, then that 185 set of links is referred to as an 'Ethernet segment'. 187 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 188 identifies an Ethernet Segment is called an 'Ethernet Segment 189 Identifier'. 191 Ethernet Tag: An Ethernet Tag identifies a particular broadcast 192 domain, e.g., a VLAN. An EVPN instance consists of one or more 193 broadcast domains. Ethernet tag(s) are assigned to the broadcast 194 domains of a given EVPN instance by the provider of that EVPN, and 195 each PE in that EVPN instance performs a mapping between broadcast 196 domain identifier(s) understood by each of its attached CEs and the 197 corresponding Ethernet tag. 199 LACP: Link Aggregation Control Protocol 201 MP2MP: Multipoint to Multipoint 203 P2MP: Point to Multipoint 205 P2P: Point to Point 207 Single-Active Redundancy Mode: When only a single PE, among all the 208 PEs attached to an Ethernet segment, is allowed to forward traffic 209 to/from that Ethernet Segment, then the Ethernet segment is defined 210 to be operating in Single-Active redundancy mode. 212 All-Active Redundancy Mode: When all PEs attached to an Ethernet 213 segment are allowed to forward traffic to/from that Ethernet Segment, 214 then the Ethernet segment is defined to be operating in All-Active 215 redundancy mode. 217 4. BGP MPLS Based EVPN Overview 219 This section provides an overview of EVPN. An EVPN instance comprises 220 CEs that are connected to PEs that form the edge of the MPLS 221 infrastructure. A CE may be a host, a router or a switch. The PEs 222 provide virtual Layer 2 bridged connectivity between the CEs. There 223 may be multiple EVPN instances in the provider's network. 225 The PEs may be connected by an MPLS LSP infrastructure which provides 226 the benefits of MPLS technology such as fast-reroute, resiliency, 227 etc. The PEs may also be connected by an IP infrastructure in which 228 case IP/GRE tunneling or other IP tunneling can be used between the 229 PEs. The detailed procedures in this version of this document are 230 specified only for MPLS LSPs as the tunneling technology. However 231 these procedures are designed to be extensible to IP tunneling as the 232 Packet Switched Network (PSN) tunneling technology. 234 In an EVPN, MAC learning between PEs occurs not in the data plane (as 235 happens with traditional bridging in VPLS [RFC4761] or [RFC4762]) but 236 in the control plane. Control plane learning offers greater control 237 over the MAC learning process, such as restricting who learns what, 238 and the ability to apply policies. Furthermore, the control plane 239 chosen for advertising MAC reachability information is multi-protocol 240 (MP) BGP (similar to IP VPNs (RFC 4364)). This provides flexibility 241 and the ability to preserve the "virtualization" or isolation of 242 groups of interacting agents (hosts, servers, virtual machines) from 243 each other. In EVPN, PEs advertise the MAC addresses learned from the 244 CEs that are connected to them, along with an MPLS label, to other 245 PEs in the control plane using MP-BGP. Control plane learning enables 246 load balancing of traffic to and from CEs that are multi-homed to 247 multiple PEs. This is in addition to load balancing across the MPLS 248 core via multiple LSPs between the same pair of PEs. In other words 249 it allows CEs to connect to multiple active points of attachment. It 250 also improves convergence times in the event of certain network 251 failures. 253 However, learning between PEs and CEs is done by the method best 254 suited to the CE: data plane learning, IEEE 802.1x, LLDP, 802.1aq, 255 ARP, management plane or other protocols. 257 It is a local decision as to whether the Layer 2 forwarding table on 258 a PE is populated with all the MAC destination addresses known to the 259 control plane, or whether the PE implements a cache based scheme. For 260 instance the MAC forwarding table may be populated only with the MAC 261 destinations of the active flows transiting a specific PE. 263 The policy attributes of EVPN are very similar to those of IP-VPN. A 264 EVPN instance requires a Route Distinguisher (RD) which is unique per 265 PE and one or more globally unique Route-Targets (RTs). A CE attaches 266 to a MAC-VRF on a PE, on an Ethernet interface which may be 267 configured for one or more Ethernet Tags, e.g., VLAN IDs. Some 268 deployment scenarios guarantee uniqueness of VLAN IDs across EVPN 269 instances: all points of attachment for a given EVPN instance use the 270 same VLAN ID, and no other EVPN instance uses this VLAN ID. This 271 document refers to this case as a "Unique VLAN EVPN" and describes 272 simplified procedures to optimize for it. 274 5. Ethernet Segment 276 If a CE is multi-homed to two or more PEs, the set of Ethernet links 277 constitutes an "Ethernet Segment". An Ethernet segment may appear to 278 the CE as a Link Aggregation Group (LAG). Ethernet segments have an 279 identifier, called the "Ethernet Segment Identifier" (ESI) which is 280 encoded as a ten octets integer in line format with the most 281 significant octet sent first. The following two ESI values are 282 reserved: 284 - ESI 0 denotes a single-homed CE. 286 - ESI {0xFF} (repeated 10 times) is known as MAX-ESI and is 287 reserved. 289 In general, an Ethernet segment SHOULD have a non-reserved ESI that 290 is unique network wide (i.e., across all EVPN instances on all the 291 PEs). If the CE(s) constituting an Ethernet Segment is (are) managed 292 by the network operator, then ESI uniqueness should be guaranteed; 293 however, if the CE(s) is (are) not managed, then the operator MUST 294 configure a network-wide unique ESI for that Ethernet Segment. This 295 is required to enable auto-discovery of Ethernet Segments and DF 296 election. 298 In a network with managed and not-managed CEs, the ESI has the 299 following format: 301 +---+---+---+---+---+---+---+---+---+---+ 302 | T | ESI Value | 303 +---+---+---+---+---+---+---+---+---+---+ 305 Where: 307 T (ESI Type) is a 1-octet field (most significant octet) that 308 specifies the format of the remaining nine octets (ESI Value). The 309 following 6 ESI types can be used: 311 - Type 0 (T=0x00) - This type indicates an arbitrary nine-octet ESI 312 value, which is managed and configured by the operator. 314 - Type 1 (T=0x01) - When IEEE 802.1AX LACP is used between the PEs 315 and CEs, this ESI type indicates an auto-generated ESI value 316 determined from LACP by concatenating the following parameters: 318 + CE LACP six octets System MAC address. The CE LACP System MAC 319 address MUST be encoded in the high order six octets of the ESI 320 Value field. 322 + CE LACP two octets Port Key. The CE LACP port key MUST be 323 encoded in the two octets next to the System MAC address. 325 + The remaining octet will be set to 0x00. 327 As far as the CE is concerned, it would treat the multiple PEs 328 that it is connected to as the same switch. This allows the CE 329 to aggregate links that are attached to different PEs in the 330 same bundle. 332 This mechanism could be used only if it produces ESIs that 333 satisfy the uniqueness requirement specified above. 335 - Type 2 (T=0x02) - This type is used in the case of indirectly 336 connected hosts via a bridged LAN between the CEs and the PEs. The 337 ESI Value is auto-generated and determined based on the Layer 2 338 bridge protocol as follows: If MST is used in the bridged LAN then 339 the value of the ESI is derived by listening to BPDUs on the Ethernet 340 segment. To achieve this the PE is not required to run MST. However 341 the PE must learn the Root Bridge MAC address and Bridge Priority of 342 the root of the Internal Spanning Tree (IST) by listening to the 343 BPDUs. The ESI Value is constructed as follows: 345 + Root Bridge six octets MAC address. The Root Bridge MAC 346 address MUST be encoded in the high order six octets of the 347 ESI Value field. 349 + Root Bridge two octets Priority. The CE Root Bridge Priority 350 MUST be encoded in the two octets next to the Root Bridge 351 MAC address. 353 + The remaining octet will be set to 0x00. 355 This mechanism could be used only if it produces ESIs that 356 satisfy the uniqueness requirement specified above. 358 - Type 3 (T=0x03) - This type indicates a MAC-based ESI Value that 359 can be auto-generated or configured by the operator. The ESI Value is 360 constructed as follows: 362 + System MAC address (six octets). The PE MAC address MUST 363 be encoded in the high order six octets of the ESI Value field. 365 + Local Discriminator value (three octets). The Local 366 Discriminator MUST be encoded in the low order three octets 367 of the ESI Value. 369 This mechanism could be used only if it produces ESIs that 370 satisfy the uniqueness requirement specified above. 372 - Type 4 (T=0x04) - This type indicates a router-ID ESI Value that 373 can be auto-generated or configured by the operator. The ESI Value is 374 constructed as follows: 376 + Router ID (four octets). The system router ID MUST be encoded 377 in the high order four octets of the ESI Value field. 379 + Local Discriminator value (four octets). The Local 380 Discriminator MUST be encoded in the four octets next to the 381 IP address. 383 + The low order octet of the ESI Value will be set to 0x00. 385 This mechanism could be used only if it produces ESIs that 386 satisfy the uniqueness requirement specified above. 388 - Type 5 (T=0x05) - This type indicates an AS-based ESI Value that 389 can be auto-generated or configured by the operator. The ESI Value is 390 constructed as follows: 392 + AS number (four octets). This is an AS number owned by the 393 system and MUST be encoded in the high order four octets of the 394 ESI Value field. If a two-octet AS number is used, the high 395 order extra two octets will be 0x0000. 397 + Local Discriminator value (four octets). The Local 398 Discriminator MUST be encoded in the four octets next to 399 the AS number. 401 + The low order octet of the ESI Value will be set to 0x00. 403 This mechanism could be used only if it produces ESIs that 404 satisfy the uniqueness requirement specified above. 406 6. Ethernet Tag ID 408 An Ethernet Tag ID is a 32-bit field containing either a 12-bit or a 409 24-bit identifier that identifies a particular broadcast domain 410 (e.g., a VLAN) in an EVPN Instance. The 12-bit identifier is called 411 VLAN ID (VID). An EVPN Instance consists of one or more broadcast 412 domains (one or more VLANs). VLANs are assigned to a given EVPN 413 Instance by the provider of the EVPN service. A given VLAN can itself 414 be represented by multiple VLAN IDs (VIDs). In such cases, the PEs 415 participating in that VLAN for a given EVPN instance are responsible 416 for performing VLAN ID translation to/from locally attached CE 417 devices. 419 If a VLAN is represented by a single VID across all PE devices 420 participating in that VLAN for that EVPN instance, then there is no 421 need for VID translation at the PEs. Furthermore, some deployment 422 scenarios guarantee uniqueness of VIDs across all EVPN instances; 423 all points of attachment for a given EVPN instance use the same VID 424 and no other EVPN instances use that VID. This allows the RT(s) for 425 each EVPN instance to be derived automatically from the corresponding 426 VID, as described in section 7.10.1. 428 The following subsections discuss the relationship between broadcast 429 domains (e.g., VLANs), Ethernet Tag IDs (e.g., VIDs), and MAC-VRFs as 430 well as the setting of the Ethernet Tag ID, in the various EVPN BGP 431 routes (defined in section 8), for the different types of service 432 interfaces described in [RFC7209]. 434 The following value of Ethernet Tag ID is reserved: 436 - Ethernet Tag ID {0xFFFFFFFF} is known as MAX-ET 438 6.1 VLAN Based Service Interface 440 With this service interface, an EVPN instance consists of only a 441 single broadcast domain (e.g., a single VLAN). Therefore, there is a 442 one to one mapping between a VID on this interface and a MAC-VRF. 443 Since a MAC-VRF corresponds to a single VLAN, it consists of a single 444 bridge domain corresponding to that VLAN. If the VLAN is represented 445 by multiple VIDs (e.g., a different VID per Ethernet Segment per PE), 446 then each PE needs to perform VID translation for frames destined to 447 its Ethernet Segment(s). In such scenarios, the Ethernet frames 448 transported over MPLS/IP network SHOULD remain tagged with the 449 originating VID and a VID translation MUST be supported in the data 450 path and MUST be performed on the disposition PE. The Ethernet Tag ID 451 in all EVPN routes MUST be set to 0. 453 6.2 VLAN Bundle Service Interface 455 With this service interface, an EVPN instance corresponds to several 456 broadcast domains (e.g., several VLANs); however, only a single 457 bridge domain is maintained per MAC-VRF which means multiple VLANs 458 share the same bridge domain. This implies MAC addresses MUST be 459 unique across different VLANs for this service to work. In other 460 words, there is a many-to-one mapping between VLANs and a MAC-VRF, 461 and the MAC-VRF consists of a single bridge domain. Furthermore, a 462 single VLAN must be represented by a single VID - e.g., no VID 463 translation is allowed for this service interface type. The MPLS 464 encapsulated frames MUST remain tagged with the originating VID. Tag 465 translation is NOT permitted. The Ethernet Tag ID in all EVPN routes 466 MUST be set to 0. 468 6.2.1 Port Based Service Interface 470 This service interface is a special case of the VLAN Bundle service 471 interface, where all of the VLANs on the port are part of the same 472 service and map to the same bundle. The procedures are identical to 473 those described in section 6.2. 475 6.3 VLAN Aware Bundle Service Interface 477 With this service interface, an EVPN instance consists of several 478 broadcast domains (e.g., several VLANs) with each VLAN having its own 479 bridge domain - i.e., multiple bridge domains (one per VLAN) is 480 maintained by a single MAC-VRF corresponding to the EVPN instance. In 481 the case where a single VLAN is represented by different VIDs on 482 different CEs and thus VID translation is required, a normalized 483 Ethernet Tag ID (VID) MUST be carried in the MPLS encapsulated frames 484 and a Ethernet Tag ID translation function MUST be supported in the 485 data path. This translation MUST be performed in data path on both 486 the imposition as well as the disposition PEs (translating to 487 normalized Ethernet Tag ID on imposition PE and translating to local 488 Ethernet Tag ID on disposition PE). The Ethernet Tag ID in all EVPN 489 routes MUST be set to the normalized value assigned by the EVPN 490 provider. 492 6.3.1 Port Based VLAN Aware Service Interface 494 This service interface is a special case of the VLAN Aware Bundle 495 service interface, where all of the VLANs on the port are part of the 496 same service and are mapped to a single bundle but without any VID 497 translation. The procedures are subset of those described in section 498 6.3. 500 7. BGP EVPN Routes 502 This document defines a new BGP Network Layer Reachability 503 Information (NLRI), called the EVPN NLRI. 505 Following is the format of the EVPN NLRI: 507 +-----------------------------------+ 508 | Route Type (1 octet) | 509 +-----------------------------------+ 510 | Length (1 octet) | 511 +-----------------------------------+ 512 | Route Type specific (variable) | 513 +-----------------------------------+ 515 The Route Type field defines encoding of the rest of the EVPN NLRI 516 (Route Type specific EVPN NLRI). 518 The Length field indicates the length in octets of the Route Type 519 specific field of EVPN NLRI. 521 This document defines the following Route Types: 523 + 1 - Ethernet Auto-Discovery (A-D) route 524 + 2 - MAC/IP advertisement route 525 + 3 - Inclusive Multicast Ethernet Tag Route 526 + 4 - Ethernet Segment Route 528 The detailed encoding and procedures for these route types are 529 described in subsequent sections. 531 The EVPN NLRI is carried in BGP [RFC4271] using BGP Multiprotocol 532 Extensions [RFC4760] with an Address Family Identifier (AFI) of 25 533 (L2VPN) and a Subsequent Address Family Identifier (SAFI) of 70 534 (EVPN). The NLRI field in the MP_REACH_NLRI/MP_UNREACH_NLRI attribute 535 contains the EVPN NLRI (encoded as specified above). 537 In order for two BGP speakers to exchange labeled EVPN NLRI, they 538 must use BGP Capabilities Advertisement to ensure that they both are 539 capable of properly processing such NLRI. This is done as specified 540 in [RFC4760], by using capability code 1 (multiprotocol BGP) with an 541 AFI of 25 (L2VPN) and a SAFI of 70 (EVPN). 543 7.1. Ethernet Auto-Discovery Route 545 A Ethernet A-D route type specific EVPN NLRI consists of the 546 following: 548 +---------------------------------------+ 549 | Route Distinguisher (RD) (8 octets) | 550 +---------------------------------------+ 551 |Ethernet Segment Identifier (10 octets)| 552 +---------------------------------------+ 553 | Ethernet Tag ID (4 octets) | 554 +---------------------------------------+ 555 | MPLS Label (3 octets) | 556 +---------------------------------------+ 558 For the purpose of BGP route key processing, only the Ethernet 559 Segment Identifier and the Ethernet Tag ID are considered to be part 560 of the prefix in the NLRI. The MPLS Label field is to be treated as a 561 route attribute as opposed to being part of the route. 563 For procedures and usage of this route please see section 8.2 "Fast 564 Convergence" and section 8.4 "Aliasing". 566 7.2. MAC/IP Advertisement Route 568 A MAC/IP advertisement route type specific EVPN NLRI consists of the 569 following: 571 +---------------------------------------+ 572 | RD (8 octets) | 573 +---------------------------------------+ 574 |Ethernet Segment Identifier (10 octets)| 575 +---------------------------------------+ 576 | Ethernet Tag ID (4 octets) | 577 +---------------------------------------+ 578 | MAC Address Length (1 octet) | 579 +---------------------------------------+ 580 | MAC Address (6 octets) | 581 +---------------------------------------+ 582 | IP Address Length (1 octet) | 583 +---------------------------------------+ 584 | IP Address (0 or 4 or 16 octets) | 585 +---------------------------------------+ 586 | MPLS Label1 (3 octets) | 587 +---------------------------------------+ 588 | MPLS Label2 (0 or 3 octets) | 589 +---------------------------------------+ 591 For the purpose of BGP route key processing, only the Ethernet Tag 592 ID, MAC Address Length, MAC Address, IP Address Length, and IP 593 Address Address fields are considered to be part of the prefix in the 594 NLRI. The Ethernet Segment Identifier and MPLS Label1 and MPLS Label2 595 fields are to be treated as route attributes as opposed to being part 596 of the "route". The IP address length is in bits. 598 For procedures and usage of this route please see section 9 599 "Determining Reachability to Unicast MAC Addresses" and section 14 600 "Load Balancing of Unicast Packets". 602 7.3. Inclusive Multicast Ethernet Tag Route 604 An Inclusive Multicast Ethernet Tag route type specific EVPN NLRI 605 consists of the following: 607 +---------------------------------------+ 608 | RD (8 octets) | 609 +---------------------------------------+ 610 | Ethernet Tag ID (4 octets) | 611 +---------------------------------------+ 612 | IP Address Length (1 octet) | 613 +---------------------------------------+ 614 | Originating Router's IP Addr | 615 | (4 or 16 octets) | 616 +---------------------------------------+ 618 For procedures and usage of this route please see section 11 619 "Handling of Multi-Destination Traffic", section 13 "Processing of 620 Unknown Unicast Traffic" and section 16 "Multicast". The IP address 621 length is in bits. For the purpose of BGP route key processing, only 622 the Ethernet Tag ID, IP Address Length, and Originating Router's IP 623 Address fields are considered to be part of the prefix in the NLRI. 625 7.4 Ethernet Segment Route 627 An Ethernet Segment route type specific EVPN NLRI consists of the 628 following: 630 +---------------------------------------+ 631 | RD (8 octets) | 632 +---------------------------------------+ 633 |Ethernet Segment Identifier (10 octets)| 634 +---------------------------------------+ 635 | IP Address Length (1 octet) | 636 +---------------------------------------+ 637 | Originating Router's IP Addr | 638 | (4 or 16 octets) | 639 +---------------------------------------+ 641 For procedures and usage of this route please see section 8.5 642 "Designated Forwarder Election". The IP address length is in bits. 643 For the purpose of BGP route key processing, only the Ethernet 644 Segment ID, IP Address Length, and Originating Router's IP Address 645 fields are considered to be part of the prefix in the NLRI. 647 7.5 ESI Label Extended Community 649 This extended community is a new transitive extended community with 650 the Type field is 0x06, and the Sub-Type of 0x01. It may be 651 advertised along with Ethernet Auto-Discovery routes and it enables 652 split-horizon procedures for multi-homed sites as described in 653 section 8.3 "Split Horizon". ESI Label represents an ES by the 654 advertising PE and it is used in split-horizon filtering by other PEs 655 that are connected to the same multi-homed Ethernet Segment. 657 Each ESI Label Extended Community is encoded as a 8-octet value as 658 follows: 660 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 661 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 662 | Type=0x06 | Sub-Type=0x01 | Flags(1 Octet)| Reserved=0 | 663 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 664 | Reserved = 0 | ESI Label | 665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 667 The low order bit of the flags octet is defined as the "Single- 668 Active" bit. A value of 0 means that the multi-homed site is 669 operating in All-Active redundancy mode and a value of 1 means that 670 the multi-homed site is operating in Single-Active redundancy mode. 672 7.6 ES-Import Route Target 674 This is a new transitive Route Target extended community carried with 675 the Ethernet Segment route. When used, it enables all the PEs 676 connected to the same multi-homed site to import the Ethernet Segment 677 routes. The value is derived automatically from the ESI by encoding 678 the high order 6-octet portion of the 9-octet ESI Value in the ES- 679 Import Route Target. The high order 6-octet of the ESI incorporates 680 MAC address of ESI (for type 1, 2, and 3) which when encoded in this 681 RT and used in the RT constrain feature, it enables proper route- 682 target filtering. The format of this extended community is as 683 follows: 685 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 686 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 687 | Type=0x06 | Sub-Type=0x02 | ES-Import | 688 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 689 | ES-Import Cont'd | 690 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 692 This document expands the definition of the Route Target extended 693 community to allow the value of high order octet (Type field) to be 694 0x06 (in addition to the values specified in rfc4360). The value of 695 low order octet (Sub-Type field) of 0x02 indicates that this extended 696 community is of type "Route Target". The new value for Type field of 697 0x06 indicates that the structure of this RT is a six-octet value 698 (e.g., a MAC address). A BGP speaker that implements RT-Constrain 699 [RFC4684] MUST apply the RT Constraint procedures to the ES-import RT 700 as well. 702 For procedures and usage of this attribute, please see section 8.1 703 "Multi-homed Ethernet Segment Auto-Discovery". 705 7.7 MAC Mobility Extended Community 707 This extended community is a new transitive extended community with 708 the Type field of 0x06 and the Sub-Type of 0x00. It may be advertised 709 along with MAC Advertisement routes. The procedures for using this 710 Extended Community are described in section 15 "MAC Mobility". 712 The MAC Mobility Extended Community is encoded as an 8-octet value as 713 follows: 715 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 716 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 717 | Type=0x06 | Sub-Type=0x00 |Flags(1 octet)| Reserved=0 | 718 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 719 | Sequence Number | 720 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 722 The low order bit of the flags octet is defined as the 723 "Sticky/static" flag and may be set to 1. A value of 1 means that the 724 MAC address is static and cannot move. The sequence number is used to 725 ensure that PEs retain the correct MAC advertisement route when 726 multiple updates occur for the same MAC address. 728 7.8 Default Gateway Extended Community 730 The Default Gateway community is an Extended Community of an Opaque 731 Type (see 3.3 of rfc4360). It is a transitive community, which means 732 that the first octet is 0x03. The value of the second octet (Sub- 733 Type) is 0x0d (Default Gateway) as assigned by IANA. The Value field 734 of this community is reserved (set to 0 by the senders, ignored by 735 the receivers). 737 7.9 Route Distinguisher Assignment per EVI 739 Route Distinguisher (RD) MUST be set to the RD of the EVI that is 740 advertising the NLRI. An RD MUST be assigned for a given EVI on a PE. 741 This RD MUST be unique across all EVIs on a PE. It is RECOMMENDED to 742 use the Type 1 RD [RFC4364]. The value field comprises an IP address 743 of the PE (typically, the loopback address) followed by a number 744 unique to the PE. This number may be generated by the PE. Or in the 745 Unique VLAN EVPN case, the low order 12 bits may be the 12 bit VLAN 746 ID, with the remaining high order 4 bits set to 0. 748 7.10 Route Targets 750 The EVPN route MAY carry one or more Route Target (RT) attributes. 751 RTs may be configured (as in IP VPNs), or may be derived 752 automatically. 754 If a PE uses RT-Constrain, the PE advertises all such RTs using RT 755 Constraints per [RFC4684]. The use of RT Constrains allows each 756 Ethernet A-D route to reach only those PEs that are configured to 757 import at least one RT from the set of RTs carried in the EVPN route. 759 7.10.1 Auto-Derivation from the Ethernet Tag ID 761 For the "Unique VLAN EVPN" scenario, it is highly desirable to auto- 762 derive the RT from the Ethernet Tag ID (VLAN ID) for that EVPN 763 instance. The following is the procedure for performing such auto- 764 derivation. 766 + The Global Administrator field of the RT MUST be set 767 to the Autonomous System (AS) number that the PE is 768 associated with. 770 + The 12-bit VLAN ID MUST be encoded in the lowest 12 bits of 771 the Local Administrator field. 773 8. Multi-homing Functions 775 This section discusses the functions, procedures and associated BGP 776 routes used to support multi-homing in EVPN. This covers both multi- 777 homed device (MHD) as well as multi-homed network (MHN) scenarios. 779 8.1 Multi-homed Ethernet Segment Auto-Discovery 781 PEs connected to the same Ethernet segment can automatically discover 782 each other with minimal to no configuration through the exchange of 783 the Ethernet Segment route. 785 8.1.1 Constructing the Ethernet Segment Route 787 The Route-Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The value 788 field comprises an IP address of the PE (typically, the loopback 789 address) followed by 0's. 791 The Ethernet Segment Identifier (ESI) MUST be set to the ten octet 792 value described in section 5. 794 The BGP advertisement that advertises the Ethernet Segment route MUST 795 also carry an ES-Import route target, as defined in section 7.6. 797 The Ethernet Segment Route filtering MUST be done such that the 798 Ethernet Segment Route is imported only by the PEs that are multi- 799 homed to the same Ethernet Segment. To that end, each PE that is 800 connected to a particular Ethernet segment constructs an import 801 filtering rule to import a route that carries the ES-Import extended 802 community, constructed from the ESI. 804 8.2 Fast Convergence 806 In EVPN, MAC address reachability is learnt via the BGP control-plane 807 over the MPLS network. As such, in the absence of any fast protection 808 mechanism, the network convergence time is a function of the number 809 of MAC Advertisement routes that must be withdrawn by the PE 810 encountering a failure. For highly scaled environments, this scheme 811 yields slow convergence. 813 To alleviate this, EVPN defines a mechanism to efficiently and 814 quickly signal, to remote PE nodes, the need to update their 815 forwarding tables upon the occurrence of a failure in connectivity to 816 an Ethernet segment. This is done by having each PE advertise a set 817 of one or more Ethernet A-D per Ethernet segment (per ES) routes for 818 each locally attached Ethernet segment (refer to section 8.2.1 below 819 for details on how these routes are constructed). The reason that a 820 PE may need to advertise more than one Ethernet A-D per ES route for 821 a given ES is that the ES may be in a multiplicity of EVIs and the 822 RTs for all of these EVIs may not fit into a single route. 823 Advertising a set of Ethernet A-D per ES routes for the ES allows 824 each route to contain a subset of the complete set of RTs. 826 Upon a failure in connectivity to the attached segment, the PE 827 withdraws the corresponding set of Ethernet A-D Per ES routes. This 828 triggers all PEs that receive the withdrawal to update their next-hop 829 adjacencies for all MAC addresses associated with the Ethernet 830 segment in question. If no other PE had advertised an Ethernet A-D 831 route for the same segment, then the PE that received the withdrawal 832 simply invalidates the MAC entries for that segment. Otherwise, the 833 PE updates the next-hop adjacencies to point to the backup PE(s). 835 8.2.1 Constructing Ethernet A-D per Ethernet Segment Route 837 This section describes the procedures used to construct the Ethernet 838 A-D per ES route, which is used for fast convergence (as discussed 839 above) and for advertising the ESI label used for split-horizon 840 filtering (as discussed in section 8.3). Support of this route is 841 REQUIRED. 843 The Route-Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The value 844 field comprises an IP address of the PE (typically, the loopback 845 address) followed by a number unique to the PE. 847 The Ethernet Segment Identifier MUST be a ten octet entity as 848 described in section "Ethernet Segment". The Ethernet A-D route is 849 not needed when the Segment Identifier is set to 0 (e.g., single- 850 homed scenarios). 852 The Ethernet Tag ID MUST be set to MAX-ET. 854 The MPLS label in the NLRI MUST be set to 0. 856 The "ESI Label Extended Community" MUST be included in the route. If 857 All-Active redundancy mode is desired, then the "Single-Active" bit 858 in the flags of the ESI Label Extended Community MUST be set to 0 and 859 the MPLS label in that extended community MUST be set to a valid MPLS 860 label value. The MPLS label in this Extended Community is referred to 861 as the ESI label and MUST have the same value in each Ethernet A-D 862 per ES route advertised for the ES. This label MUST be a downstream 863 assigned MPLS label if the advertising PE is using ingress 864 replication for receiving multicast, broadcast or unknown unicast 865 traffic from other PEs. If the advertising PE is using P2MP MPLS LSPs 866 for sending multicast, broadcast or unknown unicast traffic, then 867 this label MUST be an upstream assigned MPLS label. The usage of this 868 label is described in section 8.3. 870 If Single-Active redundancy mode is desired, then the "Single-Active" 871 bit in the flags of the ESI Label Extended Community MUST be set to 1 872 and the ESI label SHOULD be set to a valid MPLS label value. 874 8.2.1.1. Ethernet A-D Route Targets 876 Each Ethernet A-D per ES route MUST carry one or more Route Target 877 (RT) attributes. The set of Ethernet A-D routes per ES MUST carry the 878 entire set of RTs for all the EVPN instances to which the Ethernet 879 Segment belongs. 881 8.3 Split Horizon 883 Consider a CE that is multi-homed to two or more PEs on an Ethernet 884 segment ES1 operating in All-Active redundancy mode. If the CE sends 885 a broadcast, unknown unicast, or multicast (BUM) packet to one of the 886 non-Designated Forwarder (non-DF) PEs, say PE1, then PE1 will forward 887 that packet to all or subset of the other PEs in that EVPN instance 888 including the Designated Forwarder (DF) PE for that Ethernet segment. 889 In this case the DF PE that the CE is multi-homed to MUST drop the 890 packet and not forward back to the CE. This filtering is referred to 891 as "split horizon" filtering in this document. 893 When a set of PEs operating in Single-Active redundancy mode, the use 894 of this split-horizon filtering mechanism is highly recommended 895 because it prevents transient loop at the time of failure or recovery 896 impacting the Ethernet Segment - e.g., when two PEs thinks that both 897 are DFs for that segment before DF election procedure settles down. 899 In order to achieve this split horizon function, every BUM packet 900 originating from a non-DF PE is encapsulated with an MPLS label that 901 identifies the Ethernet segment of origin (i.e. the segment from 902 which the frame entered the EVPN network). This label is referred to 903 as the ESI label, and MUST be distributed by all PEs when operating 904 in All-Active redundancy mode using a set of Ethernet A-D per ES 905 routes per section 8.2.1 above. The ESI label SHOULD be distributed 906 by all PEs when operating in Single-Active redundancy mode using a 907 set of Ethernet A-D per ES route. This route is imported by the PEs 908 connected to the Ethernet Segment and also by the PEs that have at 909 least one EVPN instance in common with the Ethernet Segment in the 910 route. As described in section 8.1.1, the route MUST carry an ESI 911 Label Extended Community with a valid ESI label. The disposition PE 912 rely on the value of the ESI label to determine whether or not a BUM 913 frame is allowed to egress a specific Ethernet segment. 915 8.3.1 ESI Label Assignment 917 The following subsections describe the assignment procedures for the 918 ESI label, which differ depending on the type of tunnels being used 919 to deliver multi-destination packets in the EVPN network. 921 8.3.1.1 Ingress Replication 923 Each PE attached to a given ES that is operating in All-Active or 924 Single-Active redundancy mode and that uses ingress replication to 925 receive BUM traffic advertises a downstream assigned ESI label in the 926 set of Ethernet A-D per ES routes for that ES. This label MUST be 927 programmed in the platform label space by the advertising PE and the 928 forwarding entry for this label must result in NOT forwarding packets 929 received with this label onto the Ethernet segment for which the 930 label was distributed. 932 The rules for the inclusion of the ESI label in a BUM packet by the 933 ingress PE operating in All-Active redundancy mode are as follows: 935 A non-DF ingress PE MUST include the ESI label distributed by the DF 936 egress PE in the copy of a BUM packet sent to it. 938 An ingress PE (DF or non-DF) SHOULD include the ESI label distributed 939 by each non-DF egress PE in the copy of a BUM packet sent to it. 941 The rules for the inclusion of the ESI label in a BUM packet by the 942 ingress PE operating in Single-Active redundancy mode are as follows: 944 An ingress DF PE SHOULD include the ESI label distributed by the 945 egress PE in the copy of a BUM packet sent to it. 947 In both All-Active and Single-Active redundancy mode, an ingress PE 948 MUST NOT include an ESI label in the copy of a BUM packet sent to an 949 egress PE that is not attached to the ES through which the BUM packet 950 entered the EVI. 952 As an example, consider PE1 and PE2 that are multi-homed to CE1 on 953 ES1 and operating in All-Active multi-homing mode. Further consider 954 that PE1 is using P2P or MP2P LSPs to send packets to PE2. Consider 955 that PE1 is the non-DF for VLAN1 and PE2 is the DF for VLAN1, and PE1 956 receives a BUM packet from CE1 on VLAN1 on ES1. In this scenario, PE2 957 distributes an Inclusive Multicast Ethernet Tag route for VLAN1 958 corresponding to an EVPN instance. So, when PE1 sends a BUM packet, 959 that it receives from CE1, it MUST first push onto the MPLS label 960 stack the ESI label that PE2 has distributed for ES1. It MUST then 961 push on the MPLS label distributed by PE2 in the Inclusive Multicast 962 Ethernet Tag route for VLAN1. The resulting packet is further 963 encapsulated in the P2P or MP2P LSP label stack required to transmit 964 the packet to PE2. When PE2 receives this packet, it determines the 965 set of ESIs to replicate the packet to from the top MPLS label, after 966 any P2P or MP2P LSP labels have been removed. If the next label is 967 the ESI label assigned by PE2 for ES1, then PE2 MUST NOT forward the 968 packet onto ES1. If the next label is an ESI label which has not been 969 assigned by PE2, then PE2 MUST drop the packet. It should be noted 970 that in this scenario, if PE2 receives a BUM packet for VLAN1 from 971 CE1, then it SHOULD encapsulate the packet with an ESI label received 972 from PE1 when sending it to PE1 in order to avoid any transient loop 973 during a failure scenario impacting ES1 (e.g., port or link failure). 975 8.3.1.2. P2MP MPLS LSPs 977 The non-DF PEs attached to a given ES that is operating in All-Active 978 redundancy mode and that use P2MP LSPs to send BUM traffic advertise 979 an upstream assigned ESI label in the set of Ethernet A-D per ES 980 routes for that ES. This label is upstream assigned by the PE that 981 advertises the route. This label MUST be programmed by the other PEs, 982 that are connected to the ESI advertised in the route, in the context 983 label space for the advertising PE. Further the forwarding entry for 984 this label must result in NOT forwarding packets received with this 985 label onto the Ethernet segment that the label was distributed for. 986 This label MUST also be programmed by the other PEs, that import the 987 route but are not connected to the ESI advertised in the route, in 988 the context label space for the advertising PE. Further the 989 forwarding entry for this label must be a POP with no other 990 associated action. 992 The DF PE attached to a given ES that is operating in Single-Active 993 redundancy mode and that use P2MP LSPs to send BUM traffic should 994 advertise an upstream assigned ESI label in the set of Ethernet A-D 995 per ES routes for that ES just as above paragraph. 997 As an example, consider PE1 and PE2 that are multi-homed to CE1 on 998 ES1 and operating in All-Active multi-homing mode. Also consider PE3 999 belongs to one of the EVPN instances of ES1. Further, assume that 1000 PE1 which is the non-DF, using P2MP MPLS LSPs to send BUM packets. 1001 When PE1 sends a BUM packet, that it receives from CE1, it MUST first 1002 push onto the MPLS label stack the ESI label that it has assigned for 1003 the ESI that the packet was received on. The resulting packet is 1004 further encapsulated in the P2MP MPLS label stack necessary to 1005 transmit the packet to the other PEs. Penultimate hop popping MUST be 1006 disabled on the P2MP LSPs used in the MPLS transport infrastructure 1007 for EVPN. When PE2 receives this packet, it de-capsulates the top 1008 MPLS label and forwards the packet using the context label space 1009 determined by the top label. If the next label is the ESI label 1010 assigned by PE1 to ES1, then PE2 MUST NOT forward the packet onto 1011 ES1. When PE3 receives this packet, it de-capsulates the top MPLS 1012 label and forwards the packet using the context label space 1013 determined by the top label. If the next label is the ESI label 1014 assigned by PE1 to ES1 and PE3 is not connected to ES1, then PE3 MUST 1015 pop the label and flood the packet over all local ESIs in that EVPN 1016 instance. It should be noted that when PE2 sends a BUM frame over a 1017 P2MP LSP, it should encapsulate the frame with an ESI label even 1018 though it is the DF for that VLAN in order to avoid any transient 1019 loop during a failure scenario impacting ES1 (e.g., port or link 1020 failure). 1022 8.4 Aliasing and Backup-Path 1024 In the case where a CE is multi-homed to multiple PE nodes, using a 1025 LAG with All-Active redundancy, it is possible that only a single PE 1026 learns a set of the MAC addresses associated with traffic transmitted 1027 by the CE. This leads to a situation where remote PE nodes receive 1028 MAC advertisement routes, for these addresses, from a single PE even 1029 though multiple PEs are connected to the multi-homed segment. As a 1030 result, the remote PEs are not able to effectively load-balance 1031 traffic among the PE nodes connected to the multi-homed Ethernet 1032 segment. This could be the case, for e.g. when the PEs perform data- 1033 plane learning on the access, and the load-balancing function on the 1034 CE hashes traffic from a given source MAC address to a single PE. 1035 Another scenario where this occurs is when the PEs rely on control 1036 plane learning on the access (e.g. using ARP), since ARP traffic will 1037 be hashed to a single link in the LAG. 1039 To address this issue, EVPN introduces the concept of 'Aliasing' 1040 which is the ability of a PE to signal that it has reachability to an 1041 EVPN instance on a given ES even when it has learnt no MAC addresses 1042 from that EVI/ES. The Ethernet A-D per EVI route is used for this 1043 purpose. A remote PE that receives a MAC advertisement route with 1044 non-reserved ESI SHOULD consider the advertised MAC address to be 1045 reachable via all PEs that have advertised reachability to that MAC 1046 address' EVI/ES via the combination of an Ethernet A-D per EVI route 1047 for that EVI/ES (and Ethernet Tag if applicable) AND Ethernet A-D per 1048 ES routes for that ES with the 'Single-Active' bit in the flags of 1049 the ESI Label Extended Community set to 0. 1051 Note that the Ethernet A-D per EVI route may be received by a remote 1052 PE before it receives the set of Ethernet A-D per ES routes. 1053 Therefore, in order to handle corner cases and race conditions, the 1054 Ethernet A-D per EVI route MUST NOT be used for traffic forwarding by 1055 a remote PE until it also receives the associated set of Ethernet A-D 1056 per ES routes. 1058 Backup-path is a closely related function, but it is used in Single- 1059 Active redundancy mode. In this case a PE also advertises that it 1060 has reachability to a give EVI/ES using same combination of Ethernet 1061 A-D per EVI route and Ethernet A-D per ES route as above, but with 1062 the 'Single-Active' bit in the flags of the ESI Label Extended 1063 Community set to 1. A remote PE that receives a MAC advertisement 1064 route with non-reserved ESI SHOULD consider the advertised MAC 1065 address to be reachable via any PE that has advertised this 1066 combination of Ethernet A-D routes and it SHOULD install a backup- 1067 path for that MAC address. 1069 8.4.1 Constructing Ethernet A-D per EVPN Instance Route 1071 This section describes the procedures used to construct the Ethernet 1072 A-D per EVPN Instance (EVI) route, which is used for aliasing (as 1073 discussed above). Support of this route is OPTIONAL. 1075 Route-Distinguisher (RD) MUST be set to the RD of the EVI that is 1076 advertising the NLRI per section 7.9. 1078 The Ethernet Segment Identifier MUST be a ten octet entity as 1079 described in section "Ethernet Segment Identifier". The Ethernet A-D 1080 route is not needed when the Segment Identifier is set to 0. 1082 The Ethernet Tag ID is the identifier of an Ethernet Tag on the 1083 Ethernet segment. This value may be a 12 bit VLAN ID, in which case 1084 the low order 12 bits are set to the VLAN ID and the high order 20 1085 bits are set to 0. Or it may be another Ethernet Tag used by the 1086 EVPN. It MAY be set to the default Ethernet Tag on the Ethernet 1087 segment or to the value 0. 1089 Note that the above allows the Ethernet A-D route to be advertised 1090 with one of the following granularities: 1092 + One Ethernet A-D route for a given tuple 1093 per EVI. This is applicable when the PE uses MPLS-based 1094 disposition. 1096 + One Ethernet A-D route per (where the Ethernet 1097 Tag ID is set to 0). This is applicable when the PE uses 1098 MAC-based disposition, or when the PE uses MPLS-based 1099 disposition when no VLAN translation is required. 1101 The usage of the MPLS label is described in the section on "Load 1102 Balancing of Unicast Packets". 1104 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1105 be set to the IPv4 or IPv6 address of the advertising PE. 1107 The Ethernet A-D route MUST carry one or more Route Target (RT) 1108 attributes per section 7.10. 1110 8.5 Designated Forwarder Election 1112 Consider a CE that is a host or a router that is multi-homed directly 1113 to more than one PE in an EVPN instance on a given Ethernet segment. 1114 One or more Ethernet Tags may be configured on the Ethernet segment. 1115 In this scenario only one of the PEs, referred to as the Designated 1116 Forwarder (DF), is responsible for certain actions: 1118 - Sending multicast and broadcast traffic, on a given Ethernet 1119 Tag on a particular Ethernet segment, to the CE. 1121 - Flooding unknown unicast traffic (i.e. traffic for 1122 which a PE does not know the destination MAC address), 1123 on a given Ethernet Tag on a particular Ethernet segment 1124 to the CE, if the environment requires flooding of 1125 unknown unicast traffic. 1127 Note that this behavior, which allows selecting a DF at the 1128 granularity of for multicast, broadcast and unknown 1129 unicast traffic, is the default behavior in this specification. 1131 Note that a CE always sends packets belonging to a specific flow 1132 using a single link towards a PE. For instance, if the CE is a host 1133 then, as mentioned earlier, the host treats the multiple links that 1134 it uses to reach the PEs as a Link Aggregation Group (LAG). The CE 1135 employs a local hashing function to map traffic flows onto links in 1136 the LAG. 1138 If a bridged network is multi-homed to more than one PE in an EVPN 1139 network via switches, then the support of All-Active redundancy mode 1140 requires the bridged network to be connected to two or more PEs using 1141 a LAG. 1143 If a bridged network does not connect to the PEs using LAG, then only 1144 one of the links between the switched bridged network and the PEs 1145 must be the active link for a given EVPN instance. In this case, the 1146 set of Ethernet A-D per ES routes advertised by each PE MUST have the 1147 'Single-Active' bit in the flags of the ESI Label Extended Community 1148 set to 1. 1150 The default procedure for DF election at the granularity of is referred to as "service carving". With service carving, it is 1152 possible to elect multiple DFs per Ethernet Segment (one per EVI) in 1153 order to perform load-balancing of multi-destination traffic destined 1154 to a given Segment. The load-balancing procedures carve up the EVI 1155 space among the PE nodes evenly, in such a way that every PE is the 1156 DF for a disjoint set of EVIs. The procedure for service carving is 1157 as follows: 1159 1. When a PE discovers the ESI of the attached Ethernet Segment, it 1160 advertises an Ethernet Segment route with the associated ES-Import 1161 extended community attribute. 1163 2. The PE then starts a timer (default value = 3 seconds) to allow 1164 the reception of Ethernet Segment routes from other PE nodes 1165 connected to the same Ethernet Segment. This timer value should be 1166 same across all PEs connected to the same Ethernet Segment. 1168 3. When the timer expires, each PE builds an ordered list of the IP 1169 addresses of all the PE nodes connected to the Ethernet Segment 1170 (including itself), in increasing numeric value. Each IP address in 1171 this list is extracted from the "Originator Router's IP address" 1172 field of the advertised Ethernet Segment route. Every PE is then 1173 given an ordinal indicating its position in the ordered list, 1174 starting with 0 as the ordinal for the PE with the numerically lowest 1175 IP address. The ordinals are used to determine which PE node will be 1176 the DF for a given EVPN instance on the Ethernet Segment using the 1177 following rule: 1179 Assuming a redundancy group of N PE nodes, the PE with ordinal i is 1180 the DF for an EVPN instance with an associated Ethernet Tag value V 1181 when (V mod N) = i. In the case where multiple Ethernet Tags are 1182 associated with a single EVPN instance, then the numerically lowest 1183 Ethernet Tag value in that EVPN instance on that ES MUST be used in 1184 the modulo function. 1186 It should be noted that using "Originator Router's IP address" field 1187 in the Ethernet Segment route to get the PE IP address needed for the 1188 ordered list, allows for a CE to be multi-homed across different ASes 1189 if such need ever arises. 1191 4. The PE that is elected as a DF for a given EVPN instance will 1192 unblock traffic for the Ethernet Tags associated with that EVPN 1193 instance. Note that the DF PE unblocks multi-destination traffic in 1194 the egress direction towards the Segment. All non-DF PEs continue to 1195 drop multi-destination traffic (for the associated EVPN instances) in 1196 the egress direction towards the Segment. 1198 In the case of link or port failure, the affected PE withdraws its 1199 Ethernet Segment route. This will re-trigger the service carving 1200 procedures on all the PEs in the RG. For PE node failure, or upon PE 1201 commissioning or decommissioning, the PEs re-trigger the service 1202 carving. In case of a Single-Active multi-homing, when a service 1203 moves from one PE in the RG to another PE as a result of re-carving, 1204 the PE, which ends up being the elected DF for the service, SHOULD 1205 trigger a MAC address flush notification towards the associated 1206 Ethernet Segment. This can be done, for e.g. using IEEE 802.1ak MVRP 1207 'new' declaration. 1209 8.6. Interoperability with Single-homing PEs 1211 Let's refer to PEs that only support single-homed CE devices as 1212 single-homing PEs. For single-homing PEs, all the above multi-homing 1213 procedures can be omitted; however, to allow for single-homing PEs to 1214 fully inter-operate with multi-homing PEs, some of the multi-homing 1215 procedures described above SHOULD be supported even by single-homing 1216 PEs: 1218 - procedures related to processing Ethernet A-D route for the purpose 1219 of Fast Convergence (8.2 Fast Convergence), to let single-homing PEs 1220 benefit from fast convergence 1222 - procedures related to processing Ethernet A-D route for the purpose 1223 of Aliasing (8.4 Aliasing and Backup-path), to let single-homing PEs 1224 benefit from load balancing 1226 - procedures related to processing Ethernet A-D route for the purpose 1227 of Backup-path (8.4 Aliasing and Backup-path), to let single-homing 1228 PEs to benefit from the corresponding convergence improvement 1230 9. Determining Reachability to Unicast MAC Addresses 1232 PEs forward packets that they receive based on the destination MAC 1233 address. This implies that PEs must be able to learn how to reach a 1234 given destination unicast MAC address. 1236 There are two components to MAC address learning, "local learning" 1237 and "remote learning": 1239 9.1. Local Learning 1241 A particular PE must be able to learn the MAC addresses from the CEs 1242 that are connected to it. This is referred to as local learning. 1244 The PEs in a particular EVPN instance MUST support local data plane 1245 learning using standard IEEE Ethernet learning procedures. A PE must 1246 be capable of learning MAC addresses in the data plane when it 1247 receives packets such as the following from the CE network: 1249 - DHCP requests 1251 - ARP request for its own MAC. 1253 - ARP request for a peer. 1255 Alternatively PEs MAY learn the MAC addresses of the CEs in the 1256 control plane or via management plane integration between the PEs and 1257 the CEs. 1259 There are applications where a MAC address that is reachable via a 1260 given PE on a locally attached Segment (e.g. with ESI X) may move 1261 such that it becomes reachable via another PE on another Segment 1262 (e.g. with ESI Y). This is referred to as a "MAC Mobility". 1263 Procedures to support this are described in section "MAC Mobility". 1265 9.2. Remote learning 1267 A particular PE must be able to determine how to send traffic to MAC 1268 addresses that belong to or are behind CEs connected to other PEs 1269 i.e. to remote CEs or hosts behind remote CEs. We call such MAC 1270 addresses "remote" MAC addresses. 1272 This document requires a PE to learn remote MAC addresses in the 1273 control plane. In order to achieve this, each PE advertises the MAC 1274 addresses it learns from its locally attached CEs in the control 1275 plane, to all the other PEs in that EVPN instance, using MP-BGP and 1276 specifically the MAC Advertisement route. 1278 9.2.1. Constructing MAC/IP Address Advertisement 1280 BGP is extended to advertise these MAC addresses using the MAC/IP 1281 Advertisement route type in the EVPN NLRI. 1283 The RD MUST be the RD of the EVI that is advertising the NLRI. The 1284 procedures for setting the RD for a given EVI are described in 1285 section 7.9. 1287 The Ethernet Segment Identifier is set to the ten octet ESI described 1288 in section "Ethernet Segment". 1290 The Ethernet Tag ID may be zero or may represent a valid Ethernet Tag 1291 ID. This field may be non-zero when there are multiple bridge 1292 domains in the MAC-VRF (i.e., the PE needs to perform qualified 1293 learning for the VLANs in that MAC-VRF). 1295 When the the Ethernet Tag ID in the NLRI is set to a non-zero value, 1296 for a particular bridge domain, then this Ethernet Tag ID may either 1297 be the CE's Ethernet tag value (e.g., CE VLAN ID) or the EVPN 1298 provider's Ethernet tag value (e.g., provider VLAN ID). The latter 1299 would be the case if the CE Ethernet tags (e.g., CE VLAN ID) for a 1300 particular bridge domain are different on different CEs. 1302 The MAC address length field is in bits and it is set to 48. The MAC 1303 address length values other than 48 bits, are outside the scope of 1304 this document. The encoding of a MAC address MUST be the 6-octet MAC 1305 address specified by [802.1D-ORIG] [802.1D-REV]. 1307 The IP Address Field is optional. By default, the IP Address Length 1308 field is set to 0 and the IP address field is omitted from the route. 1309 When a valid IP address needs to be advertised, it is then encoded in 1310 this route. When an IP address is present, the IP Address Length 1311 field is in bits and it is set to 32 or 128 bits. Other IP Address 1312 Length values are outside the scope of this document. The encoding of 1313 an IP address MUST be either 4 octets for IPv4 or 16 octets for IPv6. 1314 The length field of EVPN NLRI (which is in octets and is described in 1315 section 7) is sufficient to determine whether an IP address is 1316 encoded in this route and if so, whether the encoded IP address is 1317 IPV4 or IPv6. 1319 The MPLS label1 field is encoded as 3 octets, where the high-order 20 1320 bits contain the label value. The MPLS label1 MUST be downstream 1321 assigned and it is associated with the MAC address being advertised 1322 by the advertising PE. The advertising PE uses this label when it 1323 receives an MPLS-encapsulated packet to perform forwarding based on 1324 the destination MAC address toward the CE. The forwarding procedures 1325 are specified in sections 13 and 14. 1327 A PE may advertise the same single EVPN label for all MAC addresses 1328 in a given EVI. This label assignment is referred to as a per EVI 1329 label assignment. Alternatively, a PE may advertise a unique EVPN 1330 label per combination. This label assignment is 1331 referred to as a per label assignment. As a third 1332 option, a PE may advertise a unique EVPN label per MAC address. This 1333 label assignment is referred to as a per MAC label assignment. All of 1334 these label assignment methods have their tradeoffs. The choice of a 1335 particular label assignment methodology is purely local to the PE 1336 that originates the route. 1338 Per EVI label assignment requires the least number of EVPN labels, 1339 but requires a MAC lookup in addition to an MPLS lookup on an egress 1340 PE for forwarding. On the other hand, a unique label per or a unique label per MAC allows an egress PE to 1342 forward a packet that it receives from another PE, to the connected 1343 CE, after looking up only the MPLS labels without having to perform a 1344 MAC lookup. This includes the capability to perform appropriate VLAN 1345 ID translation on egress to the CE. 1347 The MPLS label2 field is an optional field and if it is present, then 1348 it is encoded as 3 octets, where the high-order 20 bits contain the 1349 label value. 1351 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1352 be set to the IPv4 or IPv6 address of the advertising PE. 1354 The BGP advertisement for the MAC advertisement route MUST also carry 1355 one or more Route Target (RT) attributes. RTs may be configured (as 1356 in IP VPNs), or may be derived automatically from the Ethernet Tag 1357 ID, in the Unique VLAN case, as described in section 7.10.1. 1359 It is to be noted that this document does not require PEs to create 1360 forwarding state for remote MACs when they are learnt in the control 1361 plane. When this forwarding state is actually created is a local 1362 implementation matter. 1364 9.2.2 Route Resolution 1366 If the Ethernet Segment Identifier field in a received MAC 1367 Advertisement route is set to the reserved ESI value of 0 or MAX-ESI, 1368 then if the receiving PE decides to install forwarding state for the 1369 associated MAC address, it MUST be based on the MAC Advertisement 1370 route alone. 1372 If the Ethernet Segment Identifier field in a received MAC 1373 Advertisement route is set to a non-reserved ESI, and the receiving 1374 PE is locally attached to the same ESI, then the PE does not alter 1375 its forwarding state based on the received route. This ensures that 1376 local routes are preferred to remote routes. 1378 If the Ethernet Segment Identifier field in a received MAC 1379 Advertisement route is set to a non-reserved ESI, then if the 1380 receiving PE decides to install forwarding state for the associated 1381 MAC address, it MUST be when both the MAC Advertisement route AND the 1382 associated set of Ethernet A-D per ES routes have been received. The 1383 dependency of MAC routes installation on Ethernet A-D per ES routes, 1384 is to ensure that MAC routes don't get accidentally installed during 1385 mass withdraw period. 1387 To illustrate this with an example, consider two PEs (PE1 and PE2) 1388 connected to a multi-homed Ethernet Segment ES1. All-Active 1389 redundancy mode is assumed. A given MAC address M1 is learnt by PE1 1390 but not PE2. On PE3, the following states may arise: 1392 T1- When the MAC Advertisement Route from PE1 and the set of Ethernet 1393 A-D per ES routes and Ethernet A-D per EVI routes from PE1 and PE2 1394 are received, PE3 can forward traffic destined to M1 to both PE1 and 1395 PE2. 1397 T2- If after T1, PE1 withdraws its set of Ethernet A-D per ES routes, 1398 then PE3 forwards traffic destined to M1 to PE2 only. 1400 T2'- If after T1, PE2 withdraws its set of Ethernet A-D per ES 1401 routes, then PE3 forwards traffic destined to M1 to PE1 only. 1403 T2''- If after T1, PE1 withdraws its MAC Advertisement route, then 1404 PE3 treats traffic to M1 as unknown unicast. 1406 T3- PE2 also advertises a MAC route for M1 and then PE1 withdraws its 1407 MAC route for M1. PE3 continues forwarding traffic destined to M1 to 1408 both PE1 and PE2. In other words, despite M1 withdrawal by PE1, PE3 1409 forwards the traffic destined to M1 to both PE1 and PE2. This is 1410 because a flow from the CE, resulting in M1 traffic getting hashed to 1411 PE1, can get terminated resulting in M1 to aged out in PE1; however, 1412 M1 can be reachable by both PE1 and PE2. 1414 10. ARP and ND 1416 The IP address field in the MAC advertisement route may optionally 1417 carry one of the IP addresses associated with the MAC address. This 1418 provides an option which can be used to minimize the flooding of ARP 1419 or Neighbor Discovery (ND) messages over the MPLS network and to 1420 remote CEs. This option also minimizes ARP (or ND) message processing 1421 on end-stations/hosts connected to the EVPN network. A PE may learn 1422 the IP address associated with a MAC address in the control or 1423 management plane between the CE and the PE. Or, it may learn this 1424 binding by snooping certain messages to or from a CE. When a PE 1425 learns the IP address associated with a MAC address, of a locally 1426 connected CE, it may advertise this address to other PEs by including 1427 it in the MAC Advertisement route. The IP Address may be an IPv4 1428 address encoded using four octets, or an IPv6 address encoded using 1429 sixteen octets. For ARP and ND purposes, the IP Address length field 1430 MUST be set to 32 for an IPv4 address or to 128 for an IPv6 address. 1432 If there are multiple IP addresses associated with a MAC address, 1433 then multiple MAC advertisement routes MUST be generated, one for 1434 each IP address. For instance, this may be the case when there are 1435 both an IPv4 and an IPv6 address associated with the same MAC address 1436 for dual-IP stack scenarios. When the IP address is dissociated with 1437 the MAC address, then the MAC advertisement route with that 1438 particular IP address MUST be withdrawn. 1440 Note that a MAC-only route can be advertised along with but 1441 independent from MAC/IP route for scenarios where the MAC learning 1442 over access network/node is done in data-plane and independent from 1443 ARP snooping that generates MAC/IP route. In such scenarios when the 1444 ARP entry times out and causes the MAC/IP to be withdrawn, then the 1445 MAC information will not be lost. In scenarios where host MAC/IP is 1446 learned via management or control plane, then the sender PE may only 1447 generates and advertises MAC/IP route. If the receiving PE receives 1448 both the MAC-only route and the MAC/IP route, then when it receives a 1449 withdraw message for the MAC/IP route, it MUST delete the 1450 corresponding entry from the ARP table but not the MAC entry from the 1451 MAC-VRF table unless it receives a withdraw message for MAC-only 1452 route. 1454 When a PE receives an ARP request for an IP address from a CE, and if 1455 the PE has the MAC address binding for that IP address, the PE SHOULD 1456 perform ARP proxy by responding to the ARP request. 1458 10.1 Default Gateway 1460 When a PE needs to perform inter-subnet forwarding where each subnet 1461 is represented by a different broadcast domain (e.g., different VLAN) 1462 the inter-subnet forwarding is performed at layer 3 and the PE that 1463 performs such function is called the default gateway for the EVPN 1464 instance. In this case when the PE receives an ARP Request for the IP 1465 address configured as the default gateway address, the PE originates 1466 an ARP Reply. 1468 Each PE that acts as a default gateway for a given EVPN instance MAY 1469 advertise in the EVPN control plane its default gateway MAC address 1470 using the MAC/IP advertisement route, and indicates that such route 1471 is associated with the default gateway. This is accomplished by 1472 requiring the route to carry the Default Gateway extended community 1473 defined in [Section 7.8 Default Gateway Extended Community]. The ESI 1474 field is set to zero when advertising the MAC route with the Default 1475 Gateway extended community. 1477 The IP address field of the MAC/IP advertisement route is set to the 1478 default GW IP address for that subnet (e.g., EVPN instance). For a 1479 given subnet (e.g., VLAN or EVPN instance), the default GW IP address 1480 is the same across all the participant PEs. The inclusion of this IP 1481 address enables the receiving PE to check its configured default GW 1482 IP address against the one received in the MAC/IP advertisement route 1483 for that subnet (or EVPN instance) and if there is a discrepancy, 1484 then the PE SHOULD notify the operator and log an error message. 1486 Unless it is known a priori (by means outside of this document) that 1487 all PEs of a given EVPN instance act as a default gateway for that 1488 EVPN instance, the MPLS label MUST be set to a valid downstream 1489 assigned label. 1491 Furthermore, even if all PEs of a given EVPN instance do act as a 1492 default gateway for that EVPN instance, but only some, but not all, 1493 of these PEs have sufficient (routing) information to provide inter- 1494 subnet routing for all the inter-subnet traffic originated within the 1495 subnet associated with the EVPN instance, then when such PE 1496 advertises in the EVPN control plane its default gateway MAC address 1497 using the MAC advertisement route, and indicates that such route is 1498 associated with the default gateway, the route MUST carry a valid 1499 downstream assigned label. 1501 If all PEs of a given EVPN instance act as a default gateway for that 1502 EVPN instance, and the same default gateway MAC address is used 1503 across all gateway devices, then no such advertisement is needed. 1504 However, if each default gateway uses a different MAC address, then 1505 each default gateway needs to be aware of other gateways' MAC 1506 addresses and thus the need for such advertisement. This is called 1507 MAC address aliasing since a single default GW can be represented by 1508 multiple MAC addresses. 1510 Each PE that receives this route and imports it as per procedures 1511 specified in this document follows the procedures in this section 1512 when replying to ARP Requests that it receives. 1514 Each PE that acts as a default gateway for a given EVPN instance that 1515 receives this route and imports it as per procedures specified in 1516 this document MUST create MAC forwarding state that enables it to 1517 apply IP forwarding to the packets destined to the MAC address 1518 carried in the route. 1520 11. Handling of Multi-Destination Traffic 1522 Procedures are required for a given PE to send broadcast or multicast 1523 traffic, received from a CE encapsulated in a given Ethernet Tag 1524 (VLAN) in an EVPN instance, to all the other PEs that span that 1525 Ethernet Tag (VLAN) in that EVPN instance. In certain scenarios, 1526 described in section "Processing of Unknown Unicast Packets", a given 1527 PE may also need to flood unknown unicast traffic to other PEs. 1529 The PEs in a particular EVPN instance may use ingress replication, 1530 P2MP LSPs or MP2MP LSPs to send unknown unicast, broadcast or 1531 multicast traffic to other PEs. 1533 Each PE MUST advertise an "Inclusive Multicast Ethernet Tag Route" to 1534 enable the above. The following subsection provides the procedures to 1535 construct the Inclusive Multicast Ethernet Tag route. Subsequent 1536 subsections describe in further detail its usage. 1538 11.1. Constructing Inclusive Multicast Ethernet Tag Route 1540 The RD MUST be the RD of the EVI that is advertising the NLRI. The 1541 procedures for setting the RD for a given EVPN instance on a PE are 1542 described in section 7.9. 1544 The Ethernet Tag ID is the identifier of the Ethernet Tag. It may be 1545 set to 0 or to a valid Ethernet Tag value. 1547 The Originating Router's IP address MUST be set to an IP address of 1548 the PE that should be common for all the EVIs on the PE (e.,g., this 1549 address may be PE's loopback address). The IP Address Length field is 1550 in bits. 1552 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1553 be set to the same IP address as the one carried in the Originating 1554 Router's IP Address field. 1556 The BGP advertisement for the Inclusive Multicast Ethernet Tag route 1557 MUST also carry one or more Route Target (RT) attributes. The 1558 assignment of RTs described in the section 7.10 MUST be followed. 1560 11.2. P-Tunnel Identification 1562 In order to identify the P-Tunnel used for sending broadcast, unknown 1563 unicast or multicast traffic, the Inclusive Multicast Ethernet Tag 1564 route MUST carry a "PMSI Tunnel Attribute" as specified in [BGP 1565 MVPN]. 1567 Depending on the technology used for the P-tunnel for the EVPN 1568 instance on the PE, the PMSI Tunnel attribute of the Inclusive 1569 Multicast Ethernet Tag route is constructed as follows. 1571 + If the PE that originates the advertisement uses a 1572 P-Multicast tree for the P-tunnel for EVPN, the PMSI 1573 Tunnel attribute MUST contain the identity of the tree 1574 (note that the PE could create the identity of the 1575 tree prior to the actual instantiation of the tree). 1577 + A PE that uses a P-Multicast tree for the P-tunnel MAY 1578 aggregate two or more EVPN instances (EVIs) present 1579 on the PE onto the same tree. In this case, in addition 1580 to carrying the identity of the tree, the PMSI Tunnel 1581 attribute MUST carry an MPLS upstream assigned label which 1582 the PE has bound uniquely to the EVI associated with this 1583 update (as determined by its RTs). 1585 If the PE has already advertised Inclusive Multicast 1586 Ethernet Tag routes for two or more EVIs that it now 1587 desires to aggregate, then the PE MUST re-advertise 1588 those routes. The re-advertised routes MUST be the same 1589 as the original ones, except for the PMSI Tunnel attribute 1590 and the label carried in that attribute. 1592 + If the PE that originates the advertisement uses ingress 1593 replication for the P-tunnel for EVPN, the route MUST 1594 include the PMSI Tunnel attribute with the Tunnel Type set to 1595 Ingress Replication and Tunnel Identifier set to a routable 1596 address of the PE. The PMSI Tunnel attribute MUST carry a 1597 downstream assigned MPLS label. This label is used to 1598 demultiplex the broadcast, multicast or unknown unicast EVPN 1599 traffic received over a MP2P tunnel by the PE. 1601 + The Leaf Information Required flag of the PMSI Tunnel 1602 attribute MUST be set to zero, and MUST be ignored on receipt. 1604 12. Processing of Unknown Unicast Packets 1606 The procedures in this document do not require the PEs to flood 1607 unknown unicast traffic to other PEs. If PEs learn CE MAC addresses 1608 via a control plane protocol, the PEs can then distribute MAC 1609 addresses via BGP, and all unicast MAC addresses will be learnt prior 1610 to traffic to those destinations. 1612 However, if a destination MAC address of a received packet is not 1613 known by the PE, the PE may have to flood the packet. When flooding, 1614 one must take into account "split horizon forwarding" as follows: The 1615 principles behind the following procedures are borrowed from the 1616 split horizon forwarding rules in VPLS solutions [RFC4761] and 1617 [RFC4762]. When a PE capable of flooding (say PEx) receives an 1618 unknown destination MAC address, it floods the frame. If the frame 1619 arrived from an attached CE, PEx must send a copy of that frame on 1620 every Ethernet Segment (belonging to that EVI) for which it is the 1621 DF, other than the Ethernet Segment on which it received the frame. 1622 In addition, the PE must flood the frame to all other PEs 1623 participating in that EVPN instance. If, on the other hand, the frame 1624 arrived from another PE (say PEy), PEx must send a copy of the packet 1625 on each Ethernet Segment (belonging to that EVI) for which it is the 1626 DF. PEx MUST NOT send the frame to other PEs, since PEy would have 1627 already done so. Split horizon forwarding rules apply to unknown MAC 1628 addresses. 1630 Whether or not to flood packets to unknown destination MAC addresses 1631 should be an administrative choice, depending on how learning happens 1632 between CEs and PEs. 1634 The PEs in a particular EVPN instance may use ingress replication 1635 using RSVP-TE P2P LSPs or LDP MP2P LSPs for sending unknown unicast 1636 traffic to other PEs. Or they may use RSVP-TE P2MP or LDP P2MP for 1637 sending such traffic to other PEs. 1639 12.1. Ingress Replication 1641 If ingress replication is in use, the P-Tunnel attribute, carried in 1642 the Inclusive Multicast Ethernet Tag routes for the EVPN instance, 1643 specifies the downstream label that the other PEs can use to send 1644 unknown unicast, multicast or broadcast traffic for that EVPN 1645 instance to this particular PE. 1647 The PE that receives a packet with this particular MPLS label MUST 1648 treat the packet as a broadcast, multicast or unknown unicast packet. 1649 Further if the MAC address is a unicast MAC address, the PE MUST 1650 treat the packet as an unknown unicast packet. 1652 12.2. P2MP MPLS LSPs 1654 The procedures for using P2MP LSPs are very similar to VPLS 1655 procedures [RFC7117]. The P-Tunnel attribute used by a PE for sending 1656 unknown unicast, broadcast or multicast traffic for a particular EVPN 1657 instance is advertised in the Inclusive Ethernet Tag Multicast route 1658 as described in section "Handling of Multi-Destination Traffic". 1660 The P-Tunnel attribute specifies the P2MP LSP identifier. This is the 1661 equivalent of an Inclusive tree in [RFC7117]. Note that multiple 1662 Ethernet Tags, which may be in different EVPN instances, may use the 1663 same P2MP LSP, using upstream labels [RFC7117]. This is the 1664 equivalent of an Aggregate Inclusive tree in [RFC7117]. When P2MP 1665 LSPs are used for flooding unknown unicast traffic, packet re- 1666 ordering is possible. 1668 The PE that receives a packet on the P2MP LSP specified in the PMSI 1669 Tunnel Attribute MUST treat the packet as a broadcast, multicast or 1670 unknown unicast packet. Further if the MAC address is a unicast MAC 1671 address, the PE MUST treat the packet as an unknown unicast packet. 1673 13. Forwarding Unicast Packets 1675 This section describes procedures for forwarding unicast packets by 1676 PEs, where such packets are received from either directly connected 1677 CEs, or from some other PEs. 1679 13.1. Forwarding packets received from a CE 1681 When a PE receives a packet from a CE, on a given Ethernet Tag ID, it 1682 must first look up the source MAC address of the packet. In certain 1683 environments that enable MAC security, the source MAC address MAY be 1684 used to validate the host identity and determine that traffic from 1685 the host can be allowed into the network. Source MAC lookup MAY also 1686 be used for local MAC address learning. 1688 If the PE decides to forward the packet, the destination MAC address 1689 of the packet must be looked up. If the PE has received MAC address 1690 advertisements for this destination MAC address from one or more 1691 other PEs or learned it from locally connected CEs, it is considered 1692 as a known MAC address. Otherwise, the MAC address is considered as 1693 an unknown MAC address. 1695 For known MAC addresses the PE forwards this packet to one of the 1696 remote PEs or to a locally attached CE. When forwarding to a remote 1697 PE, the packet is encapsulated in the EVPN MPLS label advertised by 1698 the remote PE, for that MAC address, and in the MPLS LSP label stack 1699 to reach the remote PE. 1701 If the MAC address is unknown and if the administrative policy on the 1702 PE requires flooding of unknown unicast traffic then: 1704 - The PE MUST flood the packet to other PEs. The PE MUST first 1705 encapsulate the packet in the ESI MPLS label as described in section 1706 8.3. If ingress replication is used, the packet MUST be replicated to 1707 each remote PE with the VPN label being an MPLS label determined as 1708 follows: This is the MPLS label advertised by the remote PE in a PMSI 1709 Tunnel Attribute in the Inclusive Multicast Ethernet Tag route for an 1710 combination. The Ethernet Tag in the 1711 route may be the same as the Ethernet Tag associated with the 1712 interface on which the ingress PE receives the packet. If P2MP LSPs 1713 are being used the packet MUST be sent on the P2MP LSP that the PE is 1714 the root of for the Ethernet Tag in the EVPN instance. If the same 1715 P2MP LSP is used for all Ethernet Tags, then all the PEs in the EVPN 1716 instance MUST be the leaves of the P2MP LSP. If a distinct P2MP LSP 1717 is used for a given Ethernet Tag in the EVPN instance, then only the 1718 PEs in the Ethernet Tag MUST be the leaves of the P2MP LSP. The 1719 packet MUST be encapsulated in the P2MP LSP label stack. 1721 If the MAC address is unknown then, if the administrative policy on 1722 the PE does not allow flooding of unknown unicast traffic: 1724 - The PE MUST drop the packet. 1726 13.2. Forwarding packets received from a remote PE 1728 This section described the procedures for forwarding known and 1729 unknown unicast packets received from a remote PE. 1731 13.2.1. Unknown Unicast Forwarding 1733 When a PE receives an MPLS packet from a remote PE then, after 1734 processing the MPLS label stack, if the top MPLS label ends up being 1735 a P2MP LSP label associated with an EVPN instance or in case of 1736 ingress replication the downstream label advertised in the P-Tunnel 1737 attribute, and after performing the split horizon procedures 1738 described in section 8.3: 1740 - If the PE is the designated forwarder of BUM traffic on a 1741 particular set of ESIs for the Ethernet Tag, the default behavior is 1742 for the PE to flood the packet on these ESIs. In other words, the 1743 default behavior is for the PE to assume that for BUM traffic, it is 1744 not required to perform a destination MAC address lookup. As an 1745 option, the PE may perform a destination MAC lookup to flood the 1746 packet to only a subset of the CE interfaces in the Ethernet Tag. For 1747 instance the PE may decide to not flood an BUM packet on certain 1748 Ethernet segments even if it is the DF on the Ethernet segment, based 1749 on administrative policy. 1751 - If the PE is not the designated forwarder on any of the ESIs for 1752 the Ethernet Tag, the default behavior is for it to drop the packet. 1754 13.2.2. Known Unicast Forwarding 1756 If the top MPLS label ends up being an EVPN label that was advertised 1757 in the unicast MAC advertisements, then the PE either forwards the 1758 packet based on CE next-hop forwarding information associated with 1759 the label or does a destination MAC address lookup to forward the 1760 packet to a CE. 1762 14. Load Balancing of Unicast Frames 1763 This section specifies the load balancing procedures for sending 1764 known unicast frames to a multi-homed CE. 1766 14.1. Load balancing of traffic from a PE to remote CEs 1768 Whenever a remote PE imports a MAC advertisement for a given in an EVI, it MUST examine all imported Ethernet A-D 1770 routes for that ESI in order to determine the load-balancing 1771 characteristics of the Ethernet segment. 1773 14.1.1 Single-Active Redundancy Mode 1775 For a given ES, if the remote PE has imported the set of Ethernet A-D 1776 per ES routes from at least one PE, where the "Single-Active" flag in 1777 the ESI Label Extended Community is set, then the remote PE MUST 1778 deduce that the ES is operating in Single-Active redundancy mode. As 1779 such, the MAC address will be reachable only via the PE announcing 1780 the associated MAC Advertisement route - this is referred to as the 1781 primary PE. The other PEs advertising the set of Ethernet A-D per ES 1782 routes for the same ES provide backup paths for that ES, in case the 1783 primary PE encounters a failure, and are referred to as backup PEs. 1784 It should be noted that the primary PE for a given is the 1785 DF for that . 1787 If the primary PE encounters a failure, it MAY withdraw its set of 1788 Ethernet A-D per ES routes for the affected ES prior to withdrawing 1789 it set of MAC Advertisement routes. 1791 If there is only one backup PE for a given ES, the remote PE MAY use 1792 the primary PE's withdrawal of its set of Ethernet A-D per ES routes 1793 as a trigger to update its forwarding entries, for the associated MAC 1794 addresses, to point towards the backup PE. As the backup PE starts 1795 learning the MAC addresses over its attached ES, it will start 1796 sending MAC Advertisement routes while the failed PE withdraws its 1797 routes. This mechanism minimizes the flooding of traffic during fail- 1798 over events. 1800 If there is more than one backup PE for a given ES, the remote PE 1801 MUST use the primary PE's withdrawal of its set of Ethernet A-D per 1802 ES routes as a trigger to start flooding traffic for the associated 1803 MAC addresses (as long as flooding of unknown unicast is 1804 administratively allowed), as it is not possible to select a single 1805 backup PE. 1807 14.1.2 All-Active Redundancy Mode 1809 For a given ES, if the remote PE has imported the set of Ethernet A-D 1810 per ES routes from one or more PEs and none of them have the "Single- 1811 Active" flag in the ESI Label Extended Community set, then the remote 1812 PE MUST deduce that the ES is operating in All-Active redundancy 1813 mode. A remote PE that receives a MAC advertisement route with non- 1814 reserved ESI SHOULD consider the advertised MAC address to be 1815 reachable via all PEs that have advertised reachability to that MAC 1816 address' EVI/ES via the combination of an Ethernet A-D per EVI route 1817 for that EVI/ES (and Ethernet Tag if applicable) AND an Ethernet A-D 1818 per ES route for that ES. The remote PE MUST use received MAC 1819 Advertisement routes and Ethernet A-D per EVI/per ES routes to 1820 construct the set of next-hops for the advertised MAC address. 1822 Each next-hop comprises an MPLS label stack that is to be used by the 1823 egress PE to forward the packet. This label stack is determined as 1824 follows: 1826 -If the next-hop is constructed as a result of a MAC route then this 1827 label stack MUST be used. However, if the MAC route doesn't exist for 1828 that PE, then the next-hop and MPLS label stack is constructed as a 1829 result of the Ethernet A-D routes. Note that the following 1830 description applies to determining the label stack for a particular 1831 next-hop to reach a given PE, from which the remote PE has received 1832 and imported Ethernet A-D routes that have the matching ESI and 1833 Ethernet Tag as the one present in the MAC advertisement. The 1834 Ethernet A-D routes mentioned in the following description refer to 1835 the ones imported from this given PE. 1837 -If a set of Ethernet A-D per ES routes for that ES AND an Ethernet 1838 A-D route per EVI exist, only then the label from that latter route 1839 must be used. 1841 The following example explains the above. 1843 Consider a CE (CE1) that is dual-homed to two PEs (PE1 and PE2) on a 1844 LAG interface (ES1), and is sending packets with source MAC address 1845 MAC1 on VLAN1 (mapped to EVI1). A remote PE, say PE3, is able to 1846 learn that MAC1 is reachable via PE1 and PE2. Both PE1 and PE2 may 1847 advertise MAC1 in BGP if they receive packets with MAC1 from CE1. If 1848 this is not the case, and if MAC1 is advertised only by PE1, PE3 1849 still considers MAC1 as reachable via both PE1 and PE2 as both PE1 1850 and PE2 advertise a set of Ethernet A-D per ES routes for ES1 as well 1851 as an Ethernet A-D per EVI route for . 1853 The MPLS label stack to send the packets to PE1 is the MPLS LSP stack 1854 to get to PE1 (at top of the stack) followed by the EVPN label 1855 advertised by PE1 for CE1's MAC . 1857 The MPLS label stack to send packets to PE2 is the MPLS LSP stack to 1858 get to PE2 (at top of the stack) followed by the MPLS label in the 1859 Ethernet A-D route advertised by PE2 for , if PE2 has not 1860 advertised MAC1 in BGP. 1862 We will refer to these label stacks as MPLS next-hops. 1864 The remote PE (PE3) can now load balance the traffic it receives from 1865 its CEs, destined for CE1, between PE1 and PE2. PE3 may use N-Tuple 1866 flow information to hash traffic into one of the MPLS next-hops for 1867 load balancing of IP traffic. Alternatively PE3 may rely on the 1868 source MAC addresses for load balancing. 1870 Note that once PE3 decides to send a particular packet to PE1 or PE2 1871 it can pick one out of multiple possible paths to reach the 1872 particular remote PE using regular MPLS procedures. For instance, if 1873 the tunneling technology is based on RSVP-TE LSPs, and PE3 decides to 1874 send a particular packet to PE1, then PE3 can choose from multiple 1875 RSVP-TE LSPs that have PE1 as their destination. 1877 When PE1 or PE2 receive the packet destined for CE1 from PE3, if the 1878 packet is a known unicast, it is forwarded to CE1. If it is a BUM 1879 packet then only one of PE1 or PE2 must forward the packet to the CE. 1880 Which of PE1 or PE2 forward this packet to the CE is determined based 1881 on which of the two is the DF. 1883 14.2. Load balancing of traffic between a PE and a local CE 1885 A CE may be configured with more than one interface connected to 1886 different PEs or the same PE for load balancing, using a technology 1887 such as LAG. The PE(s) and the CE can load balance traffic onto these 1888 interfaces using one of the following mechanisms. 1890 14.2.1. Data plane learning 1892 Consider that the PEs perform data plane learning for local MAC 1893 addresses learned from local CEs. This enables the PE(s) to learn a 1894 particular MAC address and associate it with one or more interfaces, 1895 if the technology between the PE and the CE supports multi-pathing. 1896 The PEs can now load balance traffic destined to that MAC address on 1897 the multiple interfaces. 1899 Whether the CE can load balance traffic that it generates on the 1900 multiple interfaces is dependent on the CE implementation. 1902 14.2.2. Control plane learning 1904 The CE can be a host that advertises the same MAC address using a 1905 control protocol on all interfaces. This enables the PE(s) to learn 1906 the host's MAC address and associate it with all interfaces. The PEs 1907 can now load balance traffic destined to the host on all these 1908 interfaces. The host can also load balance the traffic it generates 1909 onto these interfaces and the PE that receives the traffic employs 1910 EVPN forwarding procedures to forward the traffic. 1912 15. MAC Mobility 1914 It is possible for a given host or end-station (as defined by its MAC 1915 address) to move from one Ethernet segment to another; this is 1916 referred to as 'MAC Mobility' or 'MAC move' and it is different from 1917 the multi-homing situation in which a given MAC address is reachable 1918 via multiple PEs for the same Ethernet segment. In a MAC move, there 1919 would be two sets of MAC Advertisement routes, one set with the new 1920 Ethernet segment and one set with the previous Ethernet segment, and 1921 the MAC address would appear to be reachable via each of these 1922 segments. 1924 In order to allow all of the PEs in the EVPN instance to correctly 1925 determine the current location of the MAC address, all advertisements 1926 of it being reachable via the previous Ethernet segment MUST be 1927 withdrawn by the PEs, for the previous Ethernet segment, that had 1928 advertised it. 1930 If local learning is performed using the data plane, these PEs will 1931 not be able to detect that the MAC address has moved to another 1932 Ethernet segment and the receipt of MAC Advertisement routes, with 1933 the MAC Mobility extended community attribute, from other PEs serves 1934 as the trigger for these PEs to withdraw their advertisements. If 1935 local learning is performed using the control or management planes, 1936 these interactions serve as the trigger for these PEs to withdraw 1937 their advertisements. 1939 In a situation where there are multiple moves of a given MAC, 1940 possibly between the same two Ethernet segments, there may be 1941 multiple withdrawals and re-advertisements. In order to ensure that 1942 all PEs in the EVPN instance receive all of these correctly through 1943 the intervening BGP infrastructure, it is necessary to introduce a 1944 sequence number into the MAC Mobility extended community attribute. 1946 An implementation MUST handle the scenarios where the sequence number 1947 wraps around to process mobility event correctly. 1949 Every MAC mobility event for a given MAC address will contain a 1950 sequence number that is set using the following rules: 1952 - A PE advertising a MAC address for the first time advertises it 1953 with no MAC Mobility extended community attribute. 1955 - A PE detecting a locally attached MAC address for which it had 1956 previously received a MAC Advertisement route with a different 1957 Ethernet segment identifier advertises the MAC address in a MAC 1958 Advertisement route tagged with a MAC Mobility extended community 1959 attribute with a sequence number one greater than the sequence number 1960 in the MAC mobility attribute of the received MAC Advertisement 1961 route. In the case of the first mobility event for a given MAC 1962 address, where the received MAC Advertisement route does not carry a 1963 MAC Mobility attribute, the value of the sequence number in the 1964 received route is assumed to be 0 for purpose of this processing. 1966 - A PE detecting a locally attached MAC address for which it had 1967 previously received a MAC Advertisement route with the same non-zero 1968 Ethernet segment identifier advertises it with: 1969 i. no MAC Mobility extended community attribute, if the received 1970 route did not carry said attribute. 1972 ii. a MAC Mobility extended community attribute with the sequence 1973 number equal to the highest of the sequence number(s) in the 1974 received MAC Advertisement route(s), if the received route(s) is 1975 (are) tagged with a MAC Mobility extended community attribute. 1977 - A PE detecting a locally attached MAC address for which it had 1978 previously received a MAC Advertisement route with the same zero 1979 Ethernet segment identifier (single-homed scenarios) advertises it 1980 with MAC mobility extended community attribute with the sequence 1981 number set properly. In case of single-homed scenarios, there is no 1982 need for ESI comparison. The reason ESI comparison is done for multi- 1983 homing, is to prevent false detection of MAC move among the PEs 1984 attached to the same multi-homed site. 1986 A PE receiving a MAC Advertisement route for a MAC address with a 1987 different Ethernet segment identifier and a higher sequence number 1988 than that which it had previously advertised, withdraws its MAC 1989 Advertisement route. If two (or more) PEs advertise the same MAC 1990 address with same sequence number but different Ethernet segment 1991 identifiers, a PE that receives these routes selects the route 1992 advertised by the PE with lowest IP address as the best route. If the 1993 PE is the originator of the MAC route and it receives the same MAC 1994 address with the same sequence number that it generated, it will 1995 compare its own IP address with the IP address of the remote PE and 1996 will select the lowest IP. If its own route is not the best one, it 1997 will withdraw the route. 1999 15.1. MAC Duplication Issue 2001 A situation may arise where the same MAC address is learned by 2002 different PEs in the same VLAN because of two (or more hosts) being 2003 mis-configured with the same (duplicate) MAC address. In such 2004 situation, the traffic originating from these hosts would trigger 2005 continuous MAC moves among the PEs attached to these hosts. It is 2006 important to recognize such situation and avoid incrementing the 2007 sequence number (in the MAC Mobility attribute) to infinity. In order 2008 to remedy such situation, a PE that detects a MAC mobility event by 2009 way of local learning starts an M-second timer (default value of M = 2010 180) and if it detects N MAC moves before the timer expires (default 2011 value for N = 5), it concludes that a duplicate MAC situation has 2012 occurred. The PE MUST alert the operator and stop sending and 2013 processing any BGP MAC Advertisement routes for that MAC address till 2014 a corrective action is taken by the operator. The values of M and N 2015 MUST be configurable to allow for flexibility in operator control. 2016 Note that the other PEs in the E-VPN instance will forward the 2017 traffic for the duplicate MAC address to one of the PEs advertising 2018 the duplicate MAC address. 2020 15.2. Sticky MAC addresses 2022 There are scenarios in which it is desired to configure some MAC 2023 addresses as static so that they are not subjected to MAC move. In 2024 such scenarios, these MAC addresses are advertised with MAC Mobility 2025 Extended Community where static flag is set to 1 and sequence number 2026 is set to zero. If a PE receives such advertisements and later learns 2027 the same MAC address(es) via local learning, then the PE MUST alert 2028 the operator. 2030 16. Multicast & Broadcast 2032 The PEs in a particular EVPN instance may use ingress replication or 2033 P2MP LSPs to send multicast traffic to other PEs. 2035 16.1. Ingress Replication 2037 The PEs may use ingress replication for flooding BUM traffic as 2038 described in section "Handling of Multi-Destination Traffic". A given 2039 broadcast packet must be sent to all the remote PEs. However a given 2040 multicast packet for a multicast flow may be sent to only a subset of 2041 the PEs. Specifically a given multicast flow may be sent to only 2042 those PEs that have receivers that are interested in the multicast 2043 flow. Determining which of the PEs have receivers for a given 2044 multicast flow is done using explicit tracking described below. 2046 16.2. P2MP LSPs 2047 A PE may use an "Inclusive" tree for sending an BUM packet. This 2048 terminology is borrowed from [RFC7117]. 2050 A variety of transport technologies may be used in the SP network. 2051 For inclusive P-Multicast trees, these transport technologies include 2052 point-to-multipoint LSPs created by RSVP-TE or mLDP. 2054 16.2.1. Inclusive Trees 2056 An Inclusive Tree allows the use of a single multicast distribution 2057 tree, referred to as an Inclusive P-Multicast tree, in the SP network 2058 to carry all the multicast traffic from a specified set of EVPN 2059 instances on a given PE. A particular P-Multicast tree can be set up 2060 to carry the traffic originated by sites belonging to a single EVPN 2061 instance, or to carry the traffic originated by sites belonging to 2062 several EVPN instances. The ability to carry the traffic of more than 2063 one EVPN instance on the same tree is termed 'Aggregation' and the 2064 tree is called an Aggregate Inclusive P-Multicast tree or Aggregate 2065 Inclusive tree for short. The Aggregate Inclusive tree needs to 2066 include every PE that is a member of any of the EVPN instances that 2067 are using the tree. This implies that a PE may receive BUM traffic 2068 even if it doesn't have any receivers that are interested in 2069 receiving that traffic. 2071 An Inclusive or Aggregate Inclusive tree as defined in this document 2072 is a P2MP tree. A P2MP tree is used to carry traffic only for EVPN 2073 CEs that are connected to the PE that is the root of the tree. 2075 The procedures for signaling an Inclusive tree are the same as those 2076 in [RFC7117] with the VPLS-AD route replaced with the Inclusive 2077 Multicast Ethernet Tag route. The P-Tunnel attribute [RFC7117] for an 2078 Inclusive tree is advertised with the Inclusive Multicast Ethernet 2079 Tag route as described in section "Handling of Multi-Destination 2080 Traffic". Note that for an Aggregate Inclusive tree, a PE can 2081 "aggregate" multiple EVPN instances on the same P2MP LSP using 2082 upstream labels. The procedures for aggregation are the same as those 2083 described in [RFC7117], with VPLS A-D routes replaced by EVPN 2084 Inclusive Multicast Ethernet Tag routes. 2086 17. Convergence 2088 This section describes failure recovery from different types of 2089 network failures. 2091 17.1. Transit Link and Node Failures between PEs 2093 The use of existing MPLS Fast-Reroute mechanisms can provide failure 2094 recovery in the order of 50ms, in the event of transit link and node 2095 failures in the infrastructure that connects the PEs. 2097 17.2. PE Failures 2099 Consider a host CE1 that is dual homed to PE1 and PE2. If PE1 fails, 2100 a remote PE, PE3, can discover this based on the failure of the BGP 2101 session. This failure detection can be in the sub-second range if 2102 BFD is used to detect BGP session failure. PE3 can update its 2103 forwarding state to start sending all traffic for CE1 to only PE2. 2105 17.3. PE to CE Network Failures 2107 If the connectivity between the multi-homed CE and one of the PEs 2108 that it is attached to, fails, the PE MUST withdraw the set of 2109 Ethernet A-D per ES routes that had been previously advertised for 2110 that ES. When the MAC entry on the PE ages out, the PE MUST withdraw 2111 the MAC address from BGP. Note that to aid convergence, the Ethernet 2112 A-D per EVI routes MAY be withdrawn before the MAC routes. This 2113 enables the remote PEs to remove the MPLS next-hop to this particular 2114 PE from the set of MPLS next-hops that can be used to forward traffic 2115 to the CE. 2117 When a Ethernet Tag is decommissioned on an Ethernet segment, then 2118 the PE MUST withdraw the Ethernet A-D per EVI route(s) announced for 2119 the that are impacted by the decommissioning. In 2120 addition, the PE MUST also withdraw the MAC advertisement routes that 2121 are impacted by the decommissioning. 2123 The Ethernet A-D per ES routes should be used by an implementation to 2124 optimize the withdrawal of MAC advertisement routes. When a PE 2125 receives a withdrawal of a particular Ethernet A-D route from a PE it 2126 SHOULD consider all the MAC advertisement routes, that are learned 2127 from the same ESI as in the Ethernet A-D route, from the advertising 2128 PE, as having been withdrawn. This optimizes the network convergence 2129 times in the event of PE to CE failures. 2131 18. Frame Ordering 2133 In a MAC address, if the value of the 1st nibble (bits 8 thorough 5) 2134 of the most significant octet of the destination MAC address (which 2135 follows the last MPLS label) happens to be 0x4 or 0x6, then the 2136 Ethernet frame can be misinterpreted as an IPv4 or IPv6 packet by 2137 intermediate P nodes performing ECMP based on deep packet inspection, 2138 thus resulting in load balancing packets belonging to the same flow 2139 on different ECMP paths and subjecting them to different delays. 2140 Therefore, packets belonging to the same flow can arrive at the 2141 destination out of order. This out of order delivery can happen 2142 during steady state in absence of any failures resulting in 2143 significant impact to the network operation. 2145 In order to avoid any such mis-ordering, the following rules are 2146 applied: 2148 - If a network uses deep packet inspection for its ECMP, then the 2149 "Preferred PW MPLS Control Word" per [RFC4385] SHOULD be used with 2150 the value of 0 (e.g., a 4-octet field with value of zero) when 2151 sending EVPN encapsulated packets over a MP2P LSP. 2153 - If a network uses Entropy label [RFC6790], then the control word 2154 SHOULD NOT be used when sending EVPN encapsulated packet over a MP2P 2155 LSP. 2157 - When sending EVPN encapsulated packets over a P2MP LSP or P2P LSP, 2158 then the control world SHOULD NOT be used. 2160 19. Acknowledgements 2162 Special thanks to Yakov Rekhter for reviewing this draft several 2163 times and providing valuable comments and for his very engaging 2164 discussions on several topics of this draft that helped shape this 2165 document. We would also like to thank Pedro Marques, Kaushik Ghosh, 2166 Nischal Sheth, Robert Raszuk, Amit Shukla, and Nadeem Mohammed for 2167 discussions that helped shape this document. We would also like to 2168 thank Han Nguyen for his comments and support of this work. We would 2169 also like to thank Steve Kensil and Reshad Rahman for their reviews. 2170 We would like to thank Jorge Rabadan for his contribution to section 2171 5 of this draft. We like to thank Thomas Morin for his review of this 2172 draft and his contribution of section 8.6. Many thanks to Jakob Heitz 2173 for his help to improve several sections of this draft. 2175 We would also like to thank Clarence Filsfils, Dennis Cai, Quaizar 2176 Vohra, Kireeti Kompella, Apurva Mehta for their contributions to this 2177 document. 2179 Last but not least, special thanks to Giles Heron (our WG chair) for 2180 his detailed review of this document in preparation for WG LC and 2181 making many valuable suggestions. 2183 20. Security Considerations 2185 Security considerations discussed in [RFC4761] and [RFC4762] apply to 2186 this document for MAC learning in data-plane over an Attachment 2187 Circuit (AC) and for flooding of unknown unicast and ARP messages 2188 over the MPLS/IP core. Security considerations discussed in [RFC4364] 2189 apply to this document for MAC learning in control-plane over the 2190 MPLS/IP core. This section describes additional considerations. 2192 As mentioned in [RFC4761], there are two aspects to achieving data 2193 privacy and protecting against denial-of-service attacks in a VPN: 2194 securing the control plane and protecting the forwarding path. 2195 Compromise of the control plane could result in a PE sending customer 2196 data belonging to some EVPN to another EVPN, or black-holing EVPN 2197 customer data, or even sending it to an eavesdropper; none of which 2198 are acceptable from a data privacy point of view. In addition, 2199 compromise of the control plane could result in black-holing EVPN 2200 customer data and could provide opportunities for unauthorized EVPN 2201 data usage (e.g., exploiting traffic replication within a multicast 2202 tree to amplify a denial-of-service attack based on sending large 2203 amounts of traffic). 2205 The mechanisms in this document use BGP for the control plane. Hence, 2206 techniques such as in [RFC5925] help authenticate BGP messages, 2207 making it harder to spoof updates (which can be used to divert EVPN 2208 traffic to the wrong EVPN instance) or withdrawals (denial-of-service 2209 attacks). In the multi-AS methods (b) and (c), this also means 2210 protecting the inter-AS BGP sessions, between the ASBRs, the PEs, or 2211 the Route Reflectors. 2213 Further discussion of security considerations for BGP may be found in 2214 the BGP specification itself [RFC4271] and in the security analysis 2215 for BGP [RFC4272]. The original discussion of the use of the TCP MD5 2216 signature option to protect BGP sessions is found in [RFC5925], while 2217 [RFC6952] includes an analysis of BGP keying and authentication 2218 issues. 2220 Note that [RFC5925] will not help in keeping MPLS labels private -- 2221 knowing the labels, one can eavesdrop on EVPN traffic. Such 2222 eavesdropping additionally requires access to the data path within an 2223 SP network. Users of VPN services are expected to take appropriate 2224 precautions (such as encryption) to protect the data exchanged over a 2225 VPN. 2227 One of the requirements for protecting the data plane is that the 2228 MPLS labels be accepted only from valid interfaces. For a PE, valid 2229 interfaces comprise links from other routers in the PE's own AS. For 2230 an ASBR, valid interfaces comprise links from other routers in the 2231 ASBR's own AS, and links from other ASBRs in ASes that have instances 2232 of a given EVPN. It is especially important in the case of multi-AS 2233 EVPN instances that one accept EVPN packets only from valid 2234 interfaces. 2236 It is also important to help limit malicious traffic into a network 2237 for an imposter MAC address. The mechanism described in section 15.1, 2238 shows how duplicate MAC addresses can be detected and continuous 2239 false MAC mobility can be prevented. The mechanism described in 2240 section 15.2 shows how MAC addresses can be pinned to a given 2241 Ethernet Segment, such that if they appear behind any other Ethernet 2242 Segments, the traffic for those MAC addresses can be prevented from 2243 entering the EVPN network from the other Ethernet Segments. 2245 21. Contributors 2247 In addition to the authors listed on the front page, the following 2248 individuals have also helped to shape this document: 2250 Keyur Patel 2251 Samer Salam 2252 Sami Boutros 2253 Cisco 2255 Yakov Rekhter 2256 Ravi Shekhar 2257 Juniper Networks 2259 Florin Balus 2260 Nuage Networks 2262 22. IANA Considerations 2264 This document defines a new NLRI, called "EVPN", to be carried in BGP 2265 using multiprotocol extensions. This NLRI uses the existing AFI of 2266 25 (L2VPN). IANA has assigned it a SAFI value of 70. 2268 IANA has allocated the following EVPN Extended Community sub-types in 2269 [RFC7153] and this document is the only reference for them. 2271 0x00 MAC Mobility [this document] 2272 0x01 ESI Label [this document] 2273 0x02 ES-Import Route Target [this document] 2275 This document is creating a registry called "EVPN Route Types." New 2276 registrations will be made through the "RFC Required" procedure 2277 defined in [RFC5226]. The registry has a maximum value of 255. 2278 Initial registrations are as follows: 2280 0 RESERVED 2281 1 Ethernet Auto-Discovery [this document] 2282 2 MAC/IP Advertisement [this document] 2283 3 Inclusive Multicast Ethernet Tag [this document] 2284 4 Ethernet Segment [this document] 2286 23. References 2288 23.1 Normative References 2290 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2291 Requirement Levels", BCP 14, RFC 2119, March 1997. 2293 [RFC4364] Rosen, Rekhter, et. al., "BGP/MPLS IP VPNs", February 2006 2295 [RFC4761] Kompella, K. and Y. Rekhter, "Virtual Private LAN Service 2296 (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 2297 4761, January 2007. 2299 [RFC4762] Lasserre, M. and V. Kompella, "Virtual Private LAN Service 2300 (VPLS) Using Label Distribution Protocol (LDP) Signaling", 2301 RFC 4762, January 2007. 2303 [RFC4271] Y. Rekhter et. al., "A Border Gateway Protocol 4 (BGP-4)", 2304 RFC 4271, January 2006 2306 [RFC4760] T. Bates et. al., "Multiprotocol Extensions for BGP-4", 2307 RFC 4760, January 2007 2309 [RFC7153] E. Rosen et. al., "IANA Registries for BGP Extended 2310 Communities", RFC 7153, March 2014 2312 23.2 Informative References 2314 [RFC7209] A. Sajassi, R. Aggarwal et. al., "Requirements for 2315 Ethernet VPN", May 2014. 2317 [RFC7117] R. Aggarwal et.al., "Multicast in Virtual Private LAN 2318 Service (VPLS)", February 2014. 2320 [RFC4684] P. Marques et. al., "Constrained Route Distribution for 2321 Border Gateway Protocol/MultiProtocol Label Switching 2322 (BGP/MPLS) Internet Protocol (IP) Virtual Private Networks 2323 (VPNs)", RFC 4684, November 2006. 2325 [RFC6790] K. Kompella et. al, "The Use of Entropy Labels in MPLS 2326 Forwarding", RFC 6790, November 2012. 2328 [RFC4385] S. Bryant et. al, "PWE3 Control Word for Use over an MPLS 2329 PSN", RFC 4385, February 2006 2331 [RFC5925] J. Touch et. al., "The TCP Authentication Option", RFC 2332 5925, June 2010 2334 [RFC5226] T. Narten et. al., "Guidelines for Writing an IANA 2335 Considerations Section in RFCs", RFC 5226, May 2008 2337 [RFC4272] S. Murphy, "BGP Security Vulnerabilities Analysis", RFC 2338 4272, January 2006 2340 [RFC6952] M. Jethanandani et. al., "Analysis of BGP, LDP, PCEP, and 2341 MSDP Issues According to the Keying and Authentication 2342 for Routing Protocols (KARP) Design Guide", RFC 6952, May 2343 2013 2345 [802.1Q] "IEEE Standard for Local and metropolitan area networks - 2346 Media Access Control (MAC) Bridges and Virtual Bridged 2347 Local Area Networks", IEEE Std 802.1Q(tm), 2012 Edition, 2348 October 2012. 2350 24. Author's Address 2352 Ali Sajassi 2353 Cisco 2354 Email: sajassi@cisco.com 2356 Rahul Aggarwal 2357 Email: raggarwa_1@yahoo.com 2359 Nabil Bitar 2360 Verizon Communications 2361 Email : nabil.n.bitar@verizon.com 2363 Aldrin Isaac 2364 Bloomberg 2365 Email: aisaac71@bloomberg.net 2367 James Uttaro 2368 AT&T 2369 Email: uttaro@att.com 2371 John Drake 2372 Juniper Networks 2373 Email: jdrake@juniper.net 2374 Wim Henderickx 2375 Alcatel-Lucent 2376 e-mail: wim.henderickx@alcatel-lucent.com