idnits 2.17.1 draft-ietf-l2vpn-evpn-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 22, 2012) is 4202 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'MPLS-ENCAPS' is mentioned on line 1137, but not defined == Missing Reference: 'MLDP' is mentioned on line 1734, but not defined == Outdated reference: A later version (-07) exists of draft-ietf-l2vpn-evpn-req-01 == Outdated reference: A later version (-16) exists of draft-ietf-l2vpn-vpls-mcast-11 == Outdated reference: A later version (-07) exists of draft-ietf-l2vpn-vpls-multihoming-04 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Sajassi 3 INTERNET-DRAFT Cisco 4 Category: Standards Track 5 R. Aggarwal 6 N. Bitar Arktan 7 Verizon 8 W. Henderickx 9 S. Boutros F. Balus 10 K. Patel Alcatel-Lucent 11 S. Salam 12 Cisco Aldrin Isaac 13 Bloomberg 14 J. Drake 15 R. Shekhar J. Uttaro 16 Juniper Networks AT&T 18 Expires: April 22, 2013 October 22, 2012 20 BGP MPLS Based Ethernet VPN 21 draft-ietf-l2vpn-evpn-02 23 Status of this Memo 25 This Internet-Draft is submitted to IETF in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as 31 Internet-Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/1id-abstracts.html 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html 44 Copyright and License Notice 46 Copyright (c) 2012 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Abstract 61 This document describes procedures for BGP MPLS based Ethernet VPNs 62 (E-VPN). 64 Table of Contents 66 1. Specification of requirements . . . . . . . . . . . . . . . . . 5 67 2. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 5 68 3. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 69 4. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 70 5. BGP MPLS Based E-VPN Overview . . . . . . . . . . . . . . . . . 6 71 6. Ethernet Segment . . . . . . . . . . . . . . . . . . . . . . . 7 72 7. Ethernet Tag . . . . . . . . . . . . . . . . . . . . . . . . . 8 73 7.1 VLAN Based Service Interface . . . . . . . . . . . . . . . . 9 74 7.2 VLAN Bundle Service Interface . . . . . . . . . . . . . . . 9 75 7.2.1 Port Based Service Interface . . . . . . . . . . . . . . 9 76 7.3 VLAN Aware Bundle Service Interface . . . . . . . . . . . . 9 77 7.3.1 Port Based VLAN Aware Service Interface . . . . . . . . 10 78 8. BGP E-VPN NLRI . . . . . . . . . . . . . . . . . . . . . . . . 10 79 8.1. Ethernet Auto-Discovery Route . . . . . . . . . . . . . . . 11 80 8.2. MAC Advertisement Route . . . . . . . . . . . . . . . . . 11 81 8.3. Inclusive Multicast Ethernet Tag Route . . . . . . . . . . 11 82 8.4 Ethernet Segment Route . . . . . . . . . . . . . . . . . . . 12 83 8.5 ESI MPLS Label Extended Community . . . . . . . . . . . . . 12 84 8.6 ES-Import Extended Community . . . . . . . . . . . . . . . . 13 85 8.7 MAC Mobility Extended Community . . . . . . . . . . . . . . 13 86 8.8 Default Gateway Extended Community . . . . . . . . . . . . . 13 87 9. Multi-homing Functions . . . . . . . . . . . . . . . . . . . . 14 88 9.1 Multi-homed Ethernet Segment Auto-Discovery . . . . . . . . 14 89 9.1.1 Constructing the Ethernet Segment Route . . . . . . . . 14 90 9.2 Fast Convergence . . . . . . . . . . . . . . . . . . . . . . 14 91 9.2.1 Constructing the Ethernet A-D Route per Ethernet 92 Segment . . . . . . . . . . . . . . . . . . . . . . . . 15 93 9.2.1.1. Ethernet A-D Route Targets . . . . . . . . . . . . 16 94 9.3 Split Horizon . . . . . . . . . . . . . . . . . . . . . . . 16 95 9.3.1 ESI MPLS Label Assignment . . . . . . . . . . . . . . . 16 96 9.3.1.1 Ingress Replication . . . . . . . . . . . . . . . . 17 97 9.3.1.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . . 17 98 9.3.1.3. MP2MP LSPs . . . . . . . . . . . . . . . . . . . . 18 99 9.4 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . 18 100 9.4.1 Constructing the Ethernet A-D Route per EVI . . . . . . 19 101 9.4.1.1 Ethernet A-D Route Targets . . . . . . . . . . . . . 20 102 9.5 Designated Forwarder Election . . . . . . . . . . . . . . . 20 103 9.5.1 Default DF Election Procedure . . . . . . . . . . . . . 22 104 9.5.2 DF Election with Service Carving . . . . . . . . . . . . 22 105 10. Determining Reachability to Unicast MAC Addresses . . . . . . 23 106 10.1. Local Learning . . . . . . . . . . . . . . . . . . . . . . 23 107 10.2. Remote learning . . . . . . . . . . . . . . . . . . . . . 24 108 10.2.1. Constructing the BGP E-VPN MAC Address Advertisement . 24 109 11. ARP and ND . . . . . . . . . . . . . . . . . . . . . . . . . . 26 110 11.1 Default Gateway . . . . . . . . . . . . . . . . . . . . . . 26 111 12. Handling of Multi-Destination Traffic . . . . . . . . . . . . 27 112 12.1. Construction of the Inclusive Multicast Ethernet Tag 113 Route . . . . . . . . . . . . . . . . . . . . . . . . . . 27 114 12.2. P-Tunnel Identification . . . . . . . . . . . . . . . . . 28 115 13. Processing of Unknown Unicast Packets . . . . . . . . . . . . 29 116 13.1. Ingress Replication . . . . . . . . . . . . . . . . . . . 30 117 13.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . . . . . . 30 118 14. Forwarding Unicast Packets . . . . . . . . . . . . . . . . . . 30 119 14.1. Forwarding packets received from a CE . . . . . . . . . . 30 120 14.2. Forwarding packets received from a remote PE . . . . . . . 31 121 14.2.1. Unknown Unicast Forwarding . . . . . . . . . . . . . . 31 122 14.2.2. Known Unicast Forwarding . . . . . . . . . . . . . . . 32 123 15. Load Balancing of Unicast Frames . . . . . . . . . . . . . . . 32 124 15.1. Load balancing of traffic from an PE to remote CEs . . . . 32 125 15.1.1 Active-Standby Redundancy Mode . . . . . . . . . . . . 32 126 15.1.2 All-Active Redundancy Mode . . . . . . . . . . . . . . 33 127 15.2. Load balancing of traffic between an PE and a local CE . . 35 128 15.2.1. Data plane learning . . . . . . . . . . . . . . . . . 35 129 15.2.2. Control plane learning . . . . . . . . . . . . . . . . 35 130 16. MAC Mobility . . . . . . . . . . . . . . . . . . . . . . . . . 35 131 17. Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . 37 132 17.1. Ingress Replication . . . . . . . . . . . . . . . . . . . 37 133 17.2. P2MP LSPs . . . . . . . . . . . . . . . . . . . . . . . . 37 134 17.3. MP2MP LSPs . . . . . . . . . . . . . . . . . . . . . . . . 37 135 17.3.1. Inclusive Trees . . . . . . . . . . . . . . . . . . . 38 136 17.3.2. Selective Trees . . . . . . . . . . . . . . . . . . . 38 137 17.4. Explicit Tracking . . . . . . . . . . . . . . . . . . . . 39 138 18. Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 39 139 18.1. Transit Link and Node Failures between PEs . . . . . . . . 39 140 18.2. PE Failures . . . . . . . . . . . . . . . . . . . . . . . 39 141 18.2.1. Local Repair . . . . . . . . . . . . . . . . . . . . . 40 142 18.3. PE to CE Network Failures . . . . . . . . . . . . . . . . 40 143 19. LACP State Synchronization . . . . . . . . . . . . . . . . . . 40 144 20. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 41 145 21. Security Considerations . . . . . . . . . . . . . . . . . . . 42 146 22. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 42 147 23. References . . . . . . . . . . . . . . . . . . . . . . . . . . 42 148 23.1 Normative References . . . . . . . . . . . . . . . . . . . 42 149 23.2 Informative References . . . . . . . . . . . . . . . . . . 42 150 24. Author's Address . . . . . . . . . . . . . . . . . . . . . . . 43 152 1. Specification of requirements 154 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 155 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 156 document are to be interpreted as described in [RFC2119]. 158 2. Contributors 160 In addition to the authors listed above, the following individuals 161 also contributed to this document: 163 Quaizar Vohra 164 Kireeti Kompella 165 Apurva Mehta 166 Nadeem Mohammad 167 Juniper Networks 169 Clarence Filsfils 170 Dennis Cai 171 Cisco 173 3. Introduction 175 This document describes procedures for BGP MPLS based Ethernet VPNs 176 (E-VPN). The procedures described here are intended to meet the 177 requirements specified in [EVPN-REQ]. Please refer to [EVPN-REQ] for 178 the detailed requirements and motivation. E-VPN requires extensions 179 to existing IP/MPLS protocols as described in this document. In 180 addition to these extensions E-VPN uses several building blocks from 181 existing MPLS technologies. 183 4. Terminology 185 CE: Customer Edge device e.g., host or router or switch 187 E-VPN Instance (EVI): An E-VPN routing and forwarding instance on a 188 PE. 190 Ethernet segment identifier (ESI): If a CE is multi-homed to two or 191 more PEs, the set of Ethernet links that attaches the CE to the PEs 192 is an 'Ethernet segment'. Ethernet segments MUST have a unique non- 193 zero identifier, the 'Ethernet Segment Identifier'. 195 Ethernet Tag: An Ethernet Tag identifies a particular broadcast 196 domain, e.g., a VLAN. An E-VPN instance consists of one or more 197 broadcast domains. Ethernet tag(s) are assigned to the broadcast 198 domains of a given E-VPN instance by the provider of that E-VPN, and 199 each PE in that E-VPN instance performs a mapping between broadcast 200 domain identifier(s) understood by each of its attached CEs and the 201 corresponding Ethernet tag. 203 Link Aggregation Control Protocol (LACP): 205 Multipoint to Multipoint (MP2MP): 207 Point to Multipoint (P2MP): 209 Point to Point (P2P): 211 5. BGP MPLS Based E-VPN Overview 213 This section provides an overview of E-VPN. 215 An E-VPN comprises CEs that are connected to PEs that form the edge 216 of the MPLS infrastructure. A CE may be a host, a router or a switch. 217 The PEs provide virtual Layer 2 bridged connectivity between the CEs. 218 There may be multiple E-VPNs in the provider's network. 220 The PEs may be connected by an MPLS LSP infrastructure which provides 221 the benefits of MPLS technology such as fast-reroute, resiliency, 222 etc. The PEs may also be connected by an IP infrastructure in which 223 case IP/GRE tunneling or other IP tunneling can be used between the 224 PEs. The detailed procedures in this version of this document are 225 specified only for MPLS LSPs as the tunneling technology. However 226 these procedures are designed to be extensible to IP tunneling as the 227 PSN tunneling technology. 229 In an E-VPN, MAC learning between PEs occurs not in the data plane 230 (as happens with traditional bridging) but in the control plane. 231 Control plane learning offers greater control over the MAC learning 232 process, such as restricting who learns what, and the ability to 233 apply policies. Furthermore, the control plane chosen for 234 advertising MAC reachability information is multi-protocol (MP) BGP 235 (similar to IP VPNs (RFC 4364)). This provides greater scalability 236 and the ability to preserve the "virtualization" or isolation of 237 groups of interacting agents (hosts, servers, virtual machines) from 238 each other. In E-VPN, PEs advertise the MAC addresses learned from 239 the CEs that are connected to them, along with an MPLS label, to 240 other PEs in the control plane using MP-BGP. Control plane learning 241 enables load balancing of traffic to and from CEs that are multi- 242 homed to multiple PEs. This is in addition to load balancing across 243 the MPLS core via multiple LSPs between the same pair of PEs. In 244 other words it allows CEs to connect to multiple active points of 245 attachment. It also improves convergence times in the event of 246 certain network failures. 248 However, learning between PEs and CEs is done by the method best 249 suited to the CE: data plane learning, IEEE 802.1x, LLDP, 802.1aq, 250 ARP, management plane or other protocols. 252 It is a local decision as to whether the Layer 2 forwarding table on 253 an PE is populated with all the MAC destination addresses known to 254 the control plane, or whether the PE implements a cache based scheme. 255 For instance the MAC forwarding table may be populated only with the 256 MAC destinations of the active flows transiting a specific PE. 258 The policy attributes of E-VPN are very similar to those of IP-VPN. 259 An EVI requires a Route-Distinguisher (RD) and one or more Route- 260 Targets (RTs). A CE attaches to an E-VPN instance (EVI) on an PE, on 261 an Ethernet interface which may be configured for one or more 262 Ethernet Tags, e.g., VLANs. Some deployment scenarios guarantee 263 uniqueness of VLANs across E-VPNs: all points of attachment of a 264 given EVI use the same VLAN, and no other EVI uses this VLAN. This 265 document refers to this case as a "Unique VLAN E-VPN" and describes 266 simplified procedures to optimize for it. 268 6. Ethernet Segment 270 If a CE is multi-homed to two or more PEs, the set of Ethernet links 271 constitutes an "Ethernet Segment". An Ethernet segment may appear to 272 the CE as a Link Aggregation Group (LAG). Ethernet segments have an 273 identifier, called the "Ethernet Segment Identifier" (ESI) which is 274 encoded as a ten octets integer. A single-homed CE is considered to 275 be attached to an Ethernet segment with ESI 0. Otherwise, an Ethernet 276 segment MUST have a unique non-zero ESI. The ESI can be assigned 277 using various mechanisms: 278 1. The ESI may be configured. For instance when E-VPNs are used to 279 provide a VPLS service the ESI is fairly analogous to the Multi- 280 homing site ID in [BGP-VPLS-MH]. 282 2. If IEEE 802.1AX LACP is used between the PEs and CEs, then 283 the ESI is determined from LACP by concatenating the following 284 parameters: 286 + CE LACP System Identifier comprised of two octets of System 287 Priority and six octets of System MAC address, where the 288 System Priority is encoded in the most significant two octets. 289 The CE LACP identifier MUST be encoded in the high order eight 290 octets of the ESI. 292 + CE LACP two octets Port Key. The CE LACP port key MUST be 293 encoded in the low order two octets of the ESI. 295 As far as the CE is concerned, it would treat the multiple PEs 296 that it is connected to as the same switch. This allows the CE 297 to aggregate links that are attached to different PEs in the 298 same bundle. 300 3. If LLDP is used between the PEs and CEs that are hosts, then 301 the ESI is determined by LLDP. The ESI will be specified in a 302 following version. 304 4. In the case of indirectly connected hosts via a bridged LAN 305 between the CEs and the PEs, the ESI is determined based on the 306 Layer 2 bridge protocol as follows: If MST is used in the bridged 307 LAN then the value of the ESI is derived by listening to BPDUs on 308 the Ethernet segment. To achieve this the PE is not required to 309 run MST. However the PE must learn the Root Bridge MAC address 310 and Bridge Priority of the root of the Internal Spanning Tree 311 (IST) by listening to the BPDUs. The ESI is constructed as 312 follows: 314 {Bridge Priority (16 bits) , Root Bridge MAC Address (48 bits)} 316 7. Ethernet Tag 318 An Ethernet Tag identifies a particular broadcast domain, e.g. a 319 VLAN, in an EVI. An EVI consists of one or more broadcast domains. 320 Ethernet Tags are assigned to the broadcast domains of a given EVI by 321 the provider of the E-VPN service. Each PE, in a given EVI, performs 322 a mapping between the Ethernet Tag and the corresponding broadcast 323 domain identifier(s) understood by each of its attached CEs (e.g. CE 324 VLAN Identifiers or CE-VIDs). 326 If the broadcast domain identifiers(s) are understood consistently by 327 all of the CEs in an EVI, the broadcast domain identifier(s) MAY be 328 used as the corresponding Ethernet Tag(s). In other words, the 329 Ethernet Tag ID assigned by the provider is numerically equal to the 330 broadcast domain identifier (e.g., CE-VID = Ethernet Tag). 332 Further, some deployment scenarios guarantee uniqueness of broadcast 333 domain identifiers across all EVIs; all points of attachment of a 334 given EVI use the same broadcast domain identifier(s) and no other 335 EVI uses these broadcast domain identifier(s). This allows the RT(s) 336 for each EVI to be derived automatically, as described in section 337 9.4.1.1.1 "Auto-Derivation from the Ethernet Tag ID". 339 The following subsections discuss the relationship between Ethernet 340 Tags, EVIs and broadcast domain identifiers as well as the setting of 341 the Ethernet Tag Identifier, in the various E-VPN BGP routes (defined 342 in section 8), for the different types of service interfaces 343 described in [EVPN-REQ]. 345 7.1 VLAN Based Service Interface 347 With this service interface, there is a one-to-one mapping between 348 the broadcast domain identifier understood by a CE on a port (e.g. 349 CE-VID) and an EVI. Furthermore, there is a single bridge domain per 350 PE for the EVI. Different CEs connected to different PE ports MAY use 351 different broadcast domain identifiers (e.g. CE-VIDs) for the same 352 EVI. If said identifiers are different, the frames SHOULD remain 353 tagged with the originating CE's broadcast domain identifier (e.g. 354 CE-VID). When the CE broadcast domain identifiers are not consistent, 355 a tag translation function MUST be supported in the data path and 356 MUST be performed on the disposition PE. The Ethernet Tag Identifier 357 in all E-VPN routes MUST be set to 0. 359 7.2 VLAN Bundle Service Interface 361 With this service interface, there is a many-to-one mapping between 362 the broadcast domain identifier understood by a CE on a port (e.g. 363 CE-VID) and an EVI. Furthermore, there is a single bridge domain per 364 PE for the EVI. Different CEs connected to different PE ports MUST 365 use the same broadcast domain identifiers (e.g. CE-VIDs) for the same 366 EVI. The MPLS encapsulated frames MUST remain tagged with the 367 originating CE's broadcast domain identifier (e.g. CE-VID). Tag 368 translation is NOT permitted. The Ethernet Tag Identifier in all E- 369 VPN routes MUST be set to 0. 371 7.2.1 Port Based Service Interface 373 This service interface is a special case of the VLAN Bundle service 374 interface, where all of the VLANs on the port are part of the same 375 service and map to the same bundle. The procedures are identical to 376 those described in section 7.2. 378 7.3 VLAN Aware Bundle Service Interface 380 With this service interface, there is a many-to-one mapping between 381 the broadcast domain identifier understood by a CE on a port (e.g. 382 CE-VID) and an EVI. Furthermore, there are multiple bridge domains 383 per PE for the EVI: one broadcast domain per CE broadcast domain 384 identifier. In the case where the CE broadcast domain identifiers are 385 not consistent for different CEs, a normalized Ethernet Tag MUST be 386 carried in the MPLS encapsulated frames and a tag translation 387 function MUST be supported in the data path. This translation MUST be 388 performed on both the imposition as well as the disposition PEs. The 389 Ethernet Tag Identifier in all E-VPN routes MUST be set to the 390 normalized Ethernet Tag assigned by the E-VPN provider. 392 7.3.1 Port Based VLAN Aware Service Interface 394 This service interface is a special case of the VLAN Aware Bundle 395 service interface, where all of the VLANs on the port are part of the 396 same service and map to the same bundle. The procedures are identical 397 to those described in section 7.3. 399 8. BGP E-VPN NLRI 401 This document defines a new BGP NLRI, called the E-VPN NLRI. 403 Following is the format of the E-VPN NLRI: 405 +-----------------------------------+ 406 | Route Type (1 octet) | 407 +-----------------------------------+ 408 | Length (1 octet) | 409 +-----------------------------------+ 410 | Route Type specific (variable) | 411 +-----------------------------------+ 413 The Route Type field defines encoding of the rest of the E-VPN NLRI 414 (Route Type specific E-VPN NLRI). 416 The Length field indicates the length in octets of the Route Type 417 specific field of E-VPN NLRI. 419 This document defines the following Route Types: 421 + 1 - Ethernet Auto-Discovery (A-D) route 422 + 2 - MAC advertisement route 423 + 3 - Inclusive Multicast Route 424 + 4 - Ethernet Segment Route 426 The detailed encoding and procedures for these route types are 427 described in subsequent sections. 429 The E-VPN NLRI is carried in BGP [RFC4271] using BGP Multiprotocol 430 Extensions [RFC4760] with an AFI of TBD and an SAFI of E-VPN (To be 431 assigned by IANA). The NLRI field in the 432 MP_REACH_NLRI/MP_UNREACH_NLRI attribute contains the E-VPN NLRI 433 (encoded as specified above). 435 In order for two BGP speakers to exchange labeled E-VPN NLRI, they 436 must use BGP Capabilities Advertisement to ensure that they both are 437 capable of properly processing such NLRI. This is done as specified 438 in [RFC4760], by using capability code 1 (multiprotocol BGP) with an 439 AFI of TBD and an SAFI of E-VPN. 441 8.1. Ethernet Auto-Discovery Route 443 A Ethernet A-D route type specific E-VPN NLRI consists of the 444 following: 446 +---------------------------------------+ 447 | RD (8 octets) | 448 +---------------------------------------+ 449 |Ethernet Segment Identifier (10 octets)| 450 +---------------------------------------+ 451 | Ethernet Tag ID (4 octets) | 452 +---------------------------------------+ 453 | MPLS Label (3 octets) | 454 +---------------------------------------+ 456 For procedures and usage of this route please see section 9.2 "Fast 457 Convergence" and section 9.4 "Aliasing". 459 8.2. MAC Advertisement Route 461 A MAC advertisement route type specific E-VPN NLRI consists of the 462 following: 464 +---------------------------------------+ 465 | RD (8 octets) | 466 +---------------------------------------+ 467 |Ethernet Segment Identifier (10 octets)| 468 +---------------------------------------+ 469 | Ethernet Tag ID (4 octets) | 470 +---------------------------------------+ 471 | MAC Address Length (1 octet) | 472 +---------------------------------------+ 473 | MAC Address (6 octets) | 474 +---------------------------------------+ 475 | IP Address Length (1 octet) | 476 +---------------------------------------+ 477 | IP Address (4 or 16 octets) | 478 +---------------------------------------+ 479 | MPLS Label (3 octets) | 480 +---------------------------------------+ 482 For procedures and usage of this route please see section 10 483 "Determining Reachability to Unicast MAC Addresses" and section 15 484 "Load Balancing of Unicast Packets". 486 8.3. Inclusive Multicast Ethernet Tag Route 488 An Inclusive Multicast Ethernet Tag route type specific E-VPN NLRI 489 consists of the following: 491 +---------------------------------------+ 492 | RD (8 octets) | 493 +---------------------------------------+ 494 | Ethernet Tag ID (4 octets) | 495 +---------------------------------------+ 496 | IP Address Length (1 octet) | 497 +---------------------------------------+ 498 | Originating Router's IP Addr | 499 | (4 or 16 octets) | 500 +---------------------------------------+ 502 For procedures and usage of this route please see section 12 503 "Handling of Multi-Destination Traffic", section 13 "Processing of 504 Unknown Unicast Traffic" and section 17 "Multicast". 506 8.4 Ethernet Segment Route 508 The Ethernet Segment Route is encoded in the E-VPN NLRI using the 509 Route Type value of 4. The Route Type Specific field of the NLRI is 510 formatted as follows: 512 +---------------------------------------+ 513 | RD (8 octets) | 514 +---------------------------------------+ 515 |Ethernet Segment Identifier (10 octets)| 516 +---------------------------------------+ 518 For procedures and usage of this route please see section 9.5 519 "Designated Forwarder Election". 521 8.5 ESI MPLS Label Extended Community 523 This extended community is a new transitive extended community. It 524 may be advertised along with Ethernet Auto-Discovery routes and it 525 enables split-horizon procedures for multi-homed sites as described 526 in section 9.3 "Split Horizon". 528 Each ESI MPLS Label Extended Community is encoded as a 8-octet value 529 as follows: 531 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 532 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 533 | 0x44 | Sub-Type | Flags (One Octet) |Reserved=0 | 534 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 535 | Reserved = 0| ESI MPLS label | 536 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 538 The low order bit of the flags octet is defined as the "Active- 539 Standby" bit and may be set to 1. A value of 0 means that the multi- 540 homed site is operating in Active-Active mode; whereas, a value of 1 541 means that the multi-homed site is operating in Active-Standby mode. 543 The second low order bit of the flags octet is defined as the "Root- 544 Leaf". A value of 0 means that this label is associated with a Root 545 site; whereas, a value of 1 means that this label is associate with a 546 Leaf site. The other bits must be set to 0. 548 8.6 ES-Import Extended Community 550 This is a new transitive extended community carried with the Ethernet 551 Segment route. When used, it enables all the PEs connected to the 552 same multi-homed site to import the Ethernet Segment routes. The 553 value is derived automatically from the ESI by encoding the 6-byte 554 MAC address portion of the ESI in the ES-Import Extended Community. 555 The format of this extended community is as follows: 557 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 558 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 559 | 0x44 | Sub-Type | ES-Import | 560 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 561 | ES-Import Cont'd | 562 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 564 For procedures and usage of this attribute, please see section 9.1 565 "Redundancy Group Discovery". 567 8.7 MAC Mobility Extended Community 569 This extended community is a new transitive extended community. It 570 may be advertised along with MAC Advertisement routes. The procedures 571 for using this Extended Community are described in section 16 "MAC 572 Mobility". 574 The MAC Mobility Extended Community is encoded as a 8-octet value as 575 follows: 577 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 578 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 579 | 0x44 | Sub-Type | Reserved=0 | 580 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 581 | Sequence Number | 582 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 584 8.8 Default Gateway Extended Community 585 The Default Gateway community is an Extended Community of an Opaque 586 Type (see 3.3 of rfc4360). It is a transitive community, which means 587 that the first octet is 0x03. The value of the second octet (Sub- 588 Type) is 0x030d (Default Gateway) as defined by IANA. The Value field 589 of this community is reserved (set to 0 by the senders, ignored by 590 the receivers). 592 9. Multi-homing Functions 594 This section discusses the functions, procedures and associated BGP 595 routes used to support multi-homing in E-VPN. This covers both multi- 596 homed device (MHD) as well as multi-homed network (MHN) scenarios. 598 9.1 Multi-homed Ethernet Segment Auto-Discovery 600 PEs connected to the same Ethernet segment can automatically discover 601 each other with minimal to no configuration through the exchange of 602 the Ethernet Segment route. 604 9.1.1 Constructing the Ethernet Segment Route 606 The Route-Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The value 607 field comprises an IP address of the MES (typically, the loopback 608 address) followed by 0's. 610 The Ethernet Segment Identifier MUST be set to the ten octet ESI 611 identifier described in section 6. 613 The BGP advertisement that advertises the Ethernet Segment route MUST 614 also carry an ES-Import extended community attribute, as defined in 615 section 8.6. 617 The Ethernet Segment Route filtering MUST be done such that the 618 Ethernet Segment Route is imported only by the PEs that are multi- 619 homed to the same Ethernet Segment. To that end, each PE that is 620 connected to a particular Ethernet segment constructs an import 621 filtering rule to import a route that carries the ES-Import extended 622 community, constructed from the ESI. 624 Note that the new ES-Import extended community is not the same as the 625 Route Target Extended Community. The Ethernet Segment route carries 626 this new ES-Import extended community. The PEs apply filtering on 627 this new extended community. As a result the Ethernet Segment route 628 is imported only by the PEs that are connected to the same Ethernet 629 segment. 631 9.2 Fast Convergence 632 In E-VPN, MAC address reachability is learnt via the BGP control- 633 plane over the MPLS network. As such, in the absence of any fast 634 protection mechanism, the network convergence time is a function of 635 the number of MAC Advertisement routes that must be withdrawn by the 636 PE encountering a failure. For highly scaled environments, this 637 scheme yields slow convergence. 639 To alleviate this, E-VPN defines a mechanism to efficiently and 640 quickly signal, to remote PE nodes, the need to update their 641 forwarding tables upon the occurrence of a failure in connectivity to 642 an Ethernet segment. This is done by having each PE advertise an 643 Ethernet A-D Route per Ethernet segment for each locally attached 644 segment (refer to section 9.2.1 below for details on how this route 645 is constructed). Upon a failure in connectivity to the attached 646 segment, the PE withdraws the corresponding Ethernet A-D route. This 647 triggers all PEs that receive the withdrawal to update their next-hop 648 adjacencies for all MAC addresses associated with the Ethernet 649 segment in question. If no other PE had advertised an Ethernet A-D 650 route for the same segment, then the PE that received the withdrawal 651 simply invalidates the MAC entries for that segment. Otherwise, the 652 PE updates the next-hop adjacencies to point to the backup PE(s). 654 9.2.1 Constructing the Ethernet A-D Route per Ethernet Segment 656 This section describes procedures to construct the Ethernet A-D route 657 when a single such route is advertised by an PE for a given Ethernet 658 Segment. This flavor of the Ethernet A-D route is used for fast 659 convergence (as discussed above) as well as for advertising the ESI 660 MPLS label used for split-horizon filtering (as discussed in section 661 9.2). Support of this route flavor is MANDATORY. 663 Route-Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The value 664 field comprises an IP address of the PE (typically, the loopback 665 address) followed by 0. The reason for such encoding is that the RD 666 cannot be that of a given EVI since the ESI can span across one or 667 more EVIs. 669 The Ethernet Segment Identifier MUST be a ten octet entity as 670 described in section "Ethernet Segment". This document does not 671 specify the use of the Ethernet A-D route when the Segment Identifier 672 is set to 0. 674 The Ethernet Tag ID MUST be set to 0. 676 The MPLS label in the NLRI MUST be set to 0. 678 The "ESI MPLS Label Extended Community" MUST be included in the 679 route. If all-active multi-homing is desired, then the "Active- 680 Standby" bit in the flags of the ESI MPLS Label Extended Community 681 MUST be set to 0 and the MPLS label in that extended community MUST 682 be set to a valid MPLS label value. The MPLS label in this Extended 683 Community is referred to as an "ESI label". This label MUST be a 684 downstream assigned MPLS label if the advertising PE is using ingress 685 replication for receiving multicast, broadcast or unknown unicast 686 traffic from other PEs. If the advertising PE is using P2MP MPLS LSPs 687 for sending multicast, broadcast or unknown unicast traffic, then 688 this label MUST be an upstream assigned MPLS label. The usage of this 689 label is described in section 9.2. 691 If the Ethernet Segment is connected to more than one PE and active- 692 standby multi-homing is desired, then the "Active-Standby" bit in the 693 flags of the ESI MPLS Label Extended Community MUST be set to 1. 695 9.2.1.1. Ethernet A-D Route Targets 697 The Ethernet A-D route MUST carry one or more Route Target (RT) 698 attributes. These RTs MUST be the set of RTs associated with all the 699 EVIs to which the Ethernet Segment, corresponding to the Ethernet A-D 700 route, belongs. 702 9.3 Split Horizon 704 Consider a CE that is multi-homed to two or more PEs on an Ethernet 705 segment ES1. If the CE sends a multicast, broadcast or unknown 706 unicast packet to a particular PE, say PE1, then PE1 will forward 707 that packet to all or subset of the other PEs in the EVI. In this 708 case the PEs, other than PE1, that the CE is multi-homed to MUST drop 709 the packet and not forward back to the CE. This is referred to as 710 "split horizon" filtering in this document. 712 In order to achieve this split horizon function, every multicast, 713 broadcast or unknown unicast packet is encapsulated with an MPLS 714 label that identifies the Ethernet segment of origin (i.e. the 715 segment from which the frame entered the E-VPN network). This label 716 is referred to as the ESI MPLS label, and is distributed using the 717 "Ethernet A-D route per Ethernet Segment" as per the procedures in 718 section 9.1.1 above. This route is imported by the PEs connected to 719 the Ethernet Segment and also by the PEs that have at least one EVI 720 in common with the Ethernet Segment in the route. As described in 721 section 9.1.1, the route MUST carry an ESI MPLS Label Extended 722 Community with a valid ESI MPLS label. The disposition PEs rely on 723 the value of the ESI MPLS label to determine whether or not a flooded 724 frame is allowed to egress a specific Ethernet segment. 726 9.3.1 ESI MPLS Label Assignment 727 The following subsections describe the assignment procedures for the 728 ESI MPLS label, which differ depending on the type of tunnels being 729 used to deliver multi-destination packets in the E-VPN network. 731 9.3.1.1 Ingress Replication 733 An PE that is using ingress replication for sending broadcast, 734 multicast or unknown unicast traffic, distributes to other PEs, that 735 belong to the Ethernet segment, a downstream assigned "ESI MPLS 736 label" in the Ethernet A-D route. This label MUST be programmed in 737 the platform label space by the advertising PE. Further the 738 forwarding entry for this label must result in NOT forwarding packets 739 received with this label onto the Ethernet segment that the label was 740 distributed for. 742 Consider PE1 and PE2 that are multi-homed to CE1 on ES1. Further 743 consider that PE1 is using P2P or MP2P LSPs to send packets to PE2. 744 Consider that PE1 receives a multicast, broadcast or unknown unicast 745 packet from CE1 on VLAN1 on ESI1. In this scenario, PE2 distributes 746 an Inclusive Multicast Ethernet Tag route for VLAN1 in the associated 747 EVI. So, when PE1 sends a multicast, broadcast or unknown unicast 748 packet, that it receives from CE1, it MUST first push onto the MPLS 749 label stack the ESI label that PE2 has distributed for ESI1. It MUST 750 then push on the MPLS label distributed by PE2 in the Inclusive 751 Multicast Ethernet Tag route for VLAN1. The resulting packet is 752 further encapsulated in the P2P or MP2P LSP label stack required to 753 transmit the packet to PE2. When PE2 receives this packet it 754 determines the set of ESIs to replicate the packet to from the top 755 MPLS label, after any P2P or MP2P LSP labels have been removed. If 756 the next label is the ESI label assigned by PE2 for ESI1, then PE2 757 MUST NOT forward the packet onto ESI1. 759 9.3.1.2. P2MP MPLS LSPs 761 An PE that is using P2MP LSPs for sending broadcast, multicast or 762 unknown unicast traffic, distributes to other PEs, that belong to the 763 Ethernet segment or have an E-VPN in common with the Ethernet 764 Segment, an upstream assigned "ESI MPLS label" in the Ethernet A-D 765 route. This label is upstream assigned by the PE that advertises the 766 route. This label MUST be programmed by the other PEs, that are 767 connected to the ESI advertised in the route, in the context label 768 space for the advertising PE. Further the forwarding entry for this 769 label must result in NOT forwarding packets received with this label 770 onto the Ethernet segment that the label was distributed for. This 771 label MUST also be programmed by the other PEs, that import the route 772 but are not connected to the ESI advertised in the route, in the 773 context label space for the advertising PE. Further the forwarding 774 entry for this label must be a POP with no other associated action. 776 Consider PE1 and PE2 that are multi-homed to CE1 on ES1. Also 777 consider PE3 that is in the same EVI as one of the EVIs to which ES1 778 belongs. Further, assume that PE1 is using P2MP MPLS LSPs to send 779 broadcast, multicast or uknown unicast packets. When PE1 sends a 780 multicast, broadcast or unknown unicast packet, that it receives from 781 CE1, it MUST first push onto the MPLS label stack the ESI label that 782 it has assigned for the ESI that the packet was received on. The 783 resulting packet is further encapsulated in the P2MP MPLS label stack 784 necessary to transmit the packet to the other PEs. Penultimate hop 785 popping MUST be disabled on the P2MP LSPs used in the MPLS transport 786 infrastructure for E-VPN. When PE2 receives this packet, it de- 787 capsulates the top MPLS label and forwards the packet using the 788 context label space determined by the top label. If the next label is 789 the ESI label assigned by PE1 to ESI1, then PE2 MUST NOT forward the 790 packet onto ESI1. When PE3 receives this packet, it de-capsulates the 791 top MPLS label and forwards the packet using the context label space 792 determined by the top label. If the next label is the ESI label 793 assigned by PE1 to ESI1 and PE3 is not connected to ESI1, then PE3 794 MUST pop the label and flood the packet over all local ESIs in the 795 EVI. 797 9.3.1.3. MP2MP LSPs 799 The procedures for ESI MPLS Label assignment and usage for MP2MP LSPs 800 will be described in a future version. 802 9.4 Aliasing 804 In the case where a CE is multi-homed to multiple PE nodes, using a 805 LAG with all-active redundancy, it is possible that only a single PE 806 learns a set of the MAC addresses associated with traffic transmitted 807 by the CE. This leads to a situation where remote PE nodes receive 808 MAC advertisement routes, for these addresses, from a single PE even 809 though multiple PEs are connected to the multi-homed segment. As a 810 result, the remote PEs are not able to effectively load-balance 811 traffic among the PE nodes connected to the multi-homed Ethernet 812 segment. This could be the case, for e.g. when the PEs perform data- 813 path learning on the access, and the load-balancing function on the 814 CE hashes traffic from a given source MAC address to a single PE. 815 Another scenario where this occurs is when the PEs rely on control 816 plane learning on the access (e.g. using ARP), since ARP traffic will 817 be hashed to a single link in the LAG. 819 To alleviate this issue, E-VPN introduces the concept of 'Aliasing'. 820 Aliasing refers to the ability of an PE to signal that it has 821 reachability to a given locally attached Ethernet segment, even when 822 it has learnt no MAC addresses from that segment. The Ethernet A-D 823 route per EVI is used to that end. Remote PEs which receive MAC 824 advertisement routes with non-zero ESI SHOULD consider the advertised 825 MAC address as reachable via all PEs which have advertised 826 reachability to the relevant Segment using Ethernet A-D routes with 827 the same ESI (and Ethernet Tag if applicable). 829 This flavor of Ethernet A-D route associated with aliasing can arrive 830 at target PEs asynchronously relative to the flavor of Ethernet A-D 831 route associated with split-horizon and mass-withdraw. Therefore, if 832 Ether A-D route associated with aliasing arrives ahead of the route 833 associated with mass-withdraw, then former must NOT be processed into 834 the FIB till the latter arrives. This will take care of corner cases 835 and race conditions where the Ether A-D route associated with mass- 836 withdraw is withdrawn but a PE still receives the route associated 837 with aliasing routes. 839 9.4.1 Constructing the Ethernet A-D Route per EVI 841 This section describes procedures to construct the Ethernet A-D route 842 when one or more such routes are advertised by an PE for a given EVI. 843 This flavor of the Ethernet A-D route is used for aliasing, and 844 support of this route flavor is OPTIONAL. 846 Route-Distinguisher (RD) MUST be set to the RD of the EVI that is 847 advertising the NLRI. An RD MUST be assigned for a given EVI on an 848 PE. This RD MUST be unique across all EVIs on an PE. It is 849 RECOMMENDED to use the Type 1 RD [RFC4364]. The value field comprises 850 an IP address of the PE (typically, the loopback address) followed by 851 a number unique to the PE. This number may be generated by the PE. 852 Or in the Unique VLAN E-VPN case, the low order 12 bits may be the 12 853 bit VLAN ID, with the remaining high order 4 bits set to 0. 855 The Ethernet Segment Identifier MUST be a ten octet entity as 856 described in section "Ethernet Segment Identifier". This document 857 does not specify the use of the Ethernet A-D route when the Segment 858 Identifier is set to 0. 860 The Ethernet Tag ID is the identifier of an Ethernet Tag on the 861 Ethernet segment. This value may be a 12 bit VLAN ID, in which case 862 the low order 12 bits are set to the VLAN ID and the high order 20 863 bits are set to 0. Or it may be another Ethernet Tag used by the E- 864 VPN. It MAY be set to the default Ethernet Tag on the Ethernet 865 segment or to the value 0. 867 Note that the above allows the Ethernet A-D route to be advertised 868 with one of the following granularities: 870 + One Ethernet A-D route for a given tuple 871 per EVI. This is applicable when the PE uses MPLS-based 872 disposition. 874 + One Ethernet A-D route per (where the Ethernet 875 Tag ID is set to 0). This is applicable when the PE uses 876 MAC-based disposition, or when the PE uses MPLS-based 877 disposition when no VLAN translation is required. 879 The usage of the MPLS label is described in the section on "Load 880 Balancing of Unicast Packets". 882 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 883 be set to the IPv4 or IPv6 address of the advertising PE. 885 9.4.1.1 Ethernet A-D Route Targets 887 The Ethernet A-D route MUST carry one or more Route Target (RT) 888 attributes. RTs may be configured (as in IP VPNs), or may be derived 889 automatically. 891 If an PE uses Route Target Constrain [RT-CONSTRAIN], the PE SHOULD 892 advertise all such RTs using Route Target Constrains. The use of RT 893 Constrains allows each Ethernet A-D route to reach only those PEs 894 that are configured to import at least one RT from the set of RTs 895 carried in the Ethernet A-D route. 897 9.4.1.1.1 Auto-Derivation from the Ethernet Tag ID 899 The following is the procedure for deriving the RT attribute 900 automatically from the Ethernet Tag ID associated with the 901 advertisement: 903 + The Global Administrator field of the RT MUST 904 be set to the Autonomous System (AS) number that the PE 905 belongs to. 907 + The Local Administrator field of the RT contains a 4 908 octets long number that encodes the Ethernet Tag-ID. If the 909 Ethernet Tag-ID is a two octet VLAN ID then it MUST be 910 encoded in the lower two octets of the Local Administrator 911 field and the higher two octets MUST be set to zero. 913 For the "Unique VLAN E-VPN" this results in auto-deriving the RT from 914 the Ethernet Tag, e.g., VLAN ID for that E-VPN. 916 9.5 Designated Forwarder Election 918 Consider a CE that is a host or a router that is multi-homed directly 919 to more than one PE in an E-VPN on a given Ethernet segment. One or 920 more Ethernet Tags may be configured on the Ethernet segment. In this 921 scenario only one of the PEs, referred to as the Designated Forwarder 922 (DF), is responsible for certain actions: 924 - Sending multicast and broadcast traffic, on a given Ethernet 925 Tag on a particular Ethernet segment, to the CE. 927 - Flooding unknown unicast traffic (i.e. traffic for 928 which an PE does not know the destination MAC address), 929 on a given Ethernet Tag on a particular Ethernet segment 930 to the CE, if the environment requires flooding of 931 unknown unicast traffic. 932 Note that this behavior, which allows selecting a DF at the 933 granularity of for multicast, broadcast and unknown 934 unicast traffic, is the default behavior in this specification. 935 Optional mechanisms, which will be specified in the future, will 936 allow selecting a DF at the granularity of . 938 Note that a CE always sends packets belonging to a specific flow 939 using a single link towards an PE. For instance, if the CE is a host 940 then, as mentioned earlier, the host treats the multiple links that 941 it uses to reach the PEs as a Link Aggregation Group (LAG). The CE 942 employs a local hashing function to map traffic flows onto links in 943 the LAG. 945 If a bridged network is multi-homed to more than one PE in an E-VPN 946 via switches, then the support of all-active points of attachments, 947 as described in this specification, requires the bridge network to be 948 connected to two or more PEs using a LAG. In this case the reasons 949 for doing DF election are the same as those described above when a CE 950 is a host or a router. 952 If a bridged network does not connect to the PEs using LAG, then only 953 one of the links between the switched bridged network and the PEs 954 must be the active link for a given Ethernet Tag. In this case, the 955 Ethernet A-D route per Ethernet segment MUST be advertised with the 956 "Active-Standby" flag set to one. Procedures for supporting all- 957 active points of attachments, when a bridge network connects to the 958 PEs using LAG, are for further study. 960 The granularity of the DF election MUST be at least the Ethernet 961 segment via which the CE is multi-homed to the PEs. If the DF 962 election is done at the Ethernet segment granularity then a single PE 963 MUST be elected as the DF on the Ethernet segment. 965 If there are one or more EVIs enabled on the Ethernet segment, then 966 the granularity of the DF election SHOULD be the combination of the 967 Ethernet segment and EVI on that Ethernet segment. In this case a 968 single PE MUST be elected as the DF for a particular EVI on that 969 Ethernet segment. 971 The detailed procedures for DF election are described next. 973 9.5.1 Default DF Election Procedure 975 As a PE discovers the other PEs that are connected to the same 976 Ethernet Segment, using the Ethernet Segment routes, it starts 977 building an ordered list based on the originating PE IP addresses. 978 This list is used to select a DF and a backup DF (BDF) on a per 979 Ethernet Segment basis. By default, the PE with the numerically 980 highest IP address is considered the DF for that Ethernet Segment and 981 the next PE in the list is considered the BDF. 983 If the Ethernet Segment is a multi-homed device, then the elected DF 984 is the only PE that must forward flooded multi-destination packets 985 towards the segment. All other PE nodes must not permit multi- 986 destination packets to egress to the segment. In the case where the 987 DF fails, the BDF takes over its functionality. 989 This procedure enables the election of a single DF per Ethernet 990 Segment, for all EVIs enabled on the segment. It is possible to 991 achieve more granular load-balancing of traffic among the PE nodes by 992 employing Service Carving, as discussed in the next section. 994 9.5.2 DF Election with Service Carving 996 With service carving, it is possible to elect multiple DFs per 997 Ethernet Segment (one per EVI) in order to perform load-balancing of 998 multi-destination traffic destined to a given Segment. The load- 999 balancing procedures carve up the EVI space among the PE nodes 1000 evenly, in such a way that every PE is the DF for a disjoint set of 1001 EVIs. The procedure for service carving is as follows: 1003 1. When a PE discovers the ESI of the attached Ethernet Segment, it 1004 advertises an Ethernet Segment route with the associated ES-Import 1005 extended community attribute. 1007 2. The PE then starts a timer to allow the reception of Ethernet 1008 Segment routes from other PE nodes connected to the same Ethernet 1009 Segment. 1011 3. When the timer expires, each PE builds an ordered list of the IP 1012 addresses of all the PE nodes connected to the Ethernet Segment 1013 (including itself), in increasing numeric value. Every PE is then 1014 given an ordinal indicating its position in the ordered list, 1015 starting with 0 as the ordinal for the PE with the numerically lowest 1016 IP address. The ordinals are used to determine which PE node will be 1017 the DF for a given EVI on the Ethernet Segment using the following 1018 rule: Assuming a redundancy group of N PE nodes, the PE with ordinal 1019 i is the DF for EVI V when (V mod N) = i. 1021 The above procedure results in the entire EVI range being divided up 1022 among the PEs in the RG, regardless of whether a given EVI is 1023 configured/enabled on the associated Ethernet Segment or not. 1025 4. The PE that is elected as a DF for a given EVI will unblock 1026 traffic for that EVI only if the EVI is configured/enabled on the 1027 Segment. Note that the DF PE unblocks multi-destination traffic in 1028 the egress direction towards the Segment. All non-DF PEs continue to 1029 drop multi-destination traffic (for the associated EVIs) in the 1030 egress direction towards the Segment. 1032 In the case of link or port failure, the affected PE withdraws its 1033 Ethernet Segment route. This will re-trigger the service carving 1034 procedures on all the PEs in the RG. For PE node failure, or upon PE 1035 commissioning or decommissioning, the PEs re-trigger the service 1036 carving. When a service moves from one PE in the RG to another PE as 1037 a result of re-carving, the PE, which ends up being the elected DF 1038 for the service, must trigger a MAC address flush notification 1039 towards the associated Ethernet Segment. This can be done, for e.g. 1040 using IEEE 802.1ak MVRP 'new' declaration. 1042 10. Determining Reachability to Unicast MAC Addresses 1044 PEs forward packets that they receive based on the destination MAC 1045 address. This implies that PEs must be able to learn how to reach a 1046 given destination unicast MAC address. 1048 There are two components to MAC address learning, "local learning" 1049 and "remote learning": 1051 10.1. Local Learning 1053 A particular PE must be able to learn the MAC addresses from the CEs 1054 that are connected to it. This is referred to as local learning. 1056 The PEs in a particular E-VPN MUST support local data plane learning 1057 using standard IEEE Ethernet learning procedures. An PE must be 1058 capable of learning MAC addresses in the data plane when it receives 1059 packets such as the following from the CE network: 1061 - DHCP requests 1063 - ARP request for its own MAC. 1065 - ARP request for a peer. 1067 Alternatively PEs MAY learn the MAC addresses of the CEs in the 1068 control plane or via management plane integration between the PEs and 1069 the CEs. 1071 There are applications where a MAC address that is reachable via a 1072 given PE on a locally attached Segment (e.g. with ESI X) may move 1073 such that it becomes reachable via the same PE or another PE on 1074 another Segment (e.g. with ESI Y). This is referred to as a "MAC 1075 Mobility". Procedures to support this are described in section "MAC 1076 Mobility". 1078 10.2. Remote learning 1080 A particular PE must be able to determine how to send traffic to MAC 1081 addresses that belong to or are behind CEs connected to other PEs 1082 i.e. to remote CEs or hosts behind remote CEs. We call such MAC 1083 addresses as "remote" MAC addresses. 1085 This document requires an PE to learn remote MAC addresses in the 1086 control plane. In order to achieve this, each PE advertises the MAC 1087 addresses it learns from its locally attached CEs in the control 1088 plane, to all the other PEs in the EVI, using MP-BGP and specifically 1089 the MAC Advertisement route. 1091 10.2.1. Constructing the BGP E-VPN MAC Address Advertisement 1093 BGP is extended to advertise these MAC addresses using the MAC 1094 Advertisement route type in the E-VPN NLRI. 1096 The RD MUST be the RD of the EVI that is advertising the NLRI. The 1097 procedures for setting the RD for a given EVI are described in 1098 section 9.4.1. 1100 The Ethernet Segment Identifier is set to the ten octet ESI described 1101 in section "Ethernet Segment". 1103 The Ethernet Tag ID may be zero or may represent a valid Ethernet Tag 1104 ID. This field may be non-zero when there are multiple bridge 1105 domains in the EVI (e.g., the PE needs to perform qualified learning 1106 for the VLANs in that EVI). 1108 When the the Ethernet Tag ID in the NLRI is set to a non-zero value, 1109 for a particular bridge domain, then this Ethernet Tag may either be 1110 the Ethernet tag value associated with the CE, e.g., VLAN ID, or it 1111 may be the Ethernet Tag Identifier, e.g., VLAN ID assigned by the E- 1112 VPN provider and mapped to the CE's Ethernet tag. The latter would be 1113 the case if the CE Ethernet tags, e.g., VLAN ID, for a particular 1114 bridge domain are different on different CEs. 1116 The MAC address length field is typically set to 48. However this 1117 specification enables specifying the MAC address as a prefix; in 1118 which case, the MAC address length field is set to the length of the 1119 prefix. This provides the ability to aggregate MAC addresses if the 1120 deployment environment supports that. The encoding of a MAC address 1121 MUST be the 6-octet MAC address specified by [802.1D-ORIG] [802.1D- 1122 REV]. If the MAC address is advertised as a prefix then the trailing 1123 bits of the prefix MUST be set to 0 to ensure that the entire prefix 1124 is encoded as 6 octets. 1126 The IP Address Length field value is set to the number of octets in 1127 the IP Address field. 1129 The IP Address field is optional. By default, the IP Address Length 1130 field is set to 0 and the IP address field is omitted from the route. 1131 When a valid IP address is included, it is encoded as specified in 1132 section 12. 1134 The MPLS label field carries one or more labels (that corresponds to 1135 the stack of labels [MPLS-ENCAPS]). Each label is encoded as 3 1136 octets, where the high-order 20 bits contain the label value, and the 1137 low order bit contains "Bottom of Stack" (as defined in [MPLS- 1138 ENCAPS]). The MPLS label stack MUST be the downstream assigned E-VPN 1139 MPLS label stack that is used by the PE to forward MPLS-encapsulated 1140 Ethernet frames received from remote PEs, where the destination MAC 1141 address in the Ethernet frame is the MAC address advertised in the 1142 above NLRI. The forwarding procedures are specified in section 1143 "Forwarding Unicast Packets" and "Load Balancing of Unicast Packets". 1145 An PE may advertise the same single E-VPN label for all MAC addresses 1146 in a given EVI. This label assignment methodology is referred to as a 1147 per EVI label assignment. Alternatively, an PE may advertise a unique 1148 E-VPN label per combination. This label 1149 assignment methodology is referred to as a per 1150 label assignment. As a third option, an PE may advertise a unique E- 1151 VPN label per MAC address. All of these methodologies have their 1152 tradeoffs. 1154 Per EVI label assignment requires the least number of E-VPN labels, 1155 but requires a MAC lookup in addition to an MPLS lookup on an egress 1156 PE for forwarding. On the other hand, a unique label per or a unique label per MAC allows an egress PE to 1158 forward a packet that it receives from another PE, to the connected 1159 CE, after looking up only the MPLS labels without having to perform a 1160 MAC lookup. This includes the capability to perform appropriate VLAN 1161 ID translation on egress to the CE. 1163 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1164 be set to the IPv4 or IPv6 address of the advertising PE. 1166 The BGP advertisement for the MAC advertisement route MUST also carry 1167 one or more Route Target (RT) attributes. RTs may be configured (as 1168 in IP VPNs), or may be derived automatically from the Ethernet Tag 1169 ID, in the Unique VLAN case, as described in section "Ethernet A-D 1170 Route per E-VPN". 1172 It is to be noted that this document does not require PEs to create 1173 forwarding state for remote MACs when they are learnt in the control 1174 plane. When this forwarding state is actually created is a local 1175 implementation matter. 1177 11. ARP and ND 1179 The IP address field in the MAC advertisement route may optionally 1180 carry one of the IP addresses associated with the MAC address. This 1181 provides an option which can be used to minimize the flooding of ARP 1182 or Neighbor Discovery (ND) messages over the MPLS network and to 1183 remote CEs. This option also minimizes ARP (or ND) message processing 1184 on end-stations/hosts connected to the E-VPN network. An PE may learn 1185 the IP address associated with a MAC address in the control or 1186 management plane between the CE and the PE. Or, it may learn this 1187 binding by snooping certain messages to or from a CE. When an PE 1188 learns the IP address associated with a MAC address, of a locally 1189 connected CE, it may advertise this address to other PEs by including 1190 it in the MAC Advertisement route. The IP Address may be an IPv4 1191 address encoded using four octets, or an IPv6 address encoded using 1192 sixteen octets. The IP Address length field MUST be set to 32 for an 1193 IPv4 address or to 128 for an IPv6 address. 1195 If there are multiple IP addresses associated with a MAC address, 1196 then multiple MAC advertisement routes MUST be generated, one for 1197 each IP address. For instance, this may be the case when there are 1198 both an IPv4 and an IPv6 address associated with the MAC address. 1199 When the IP address is dissociated with the MAC address, then the MAC 1200 advertisement route with that particular IP address MUST be 1201 withdrawn. 1203 When an PE receives an ARP request for an IP address from a CE, and 1204 if the PE has the MAC address binding for that IP address, the PE 1205 SHOULD perform ARP proxy and respond to the ARP request. 1207 11.1 Default Gateway 1208 A PE MAY choose to terminate ARP messages instead of performing ARP 1209 proxy for them. Such scenarios arises when the PE needs to perform 1210 inter-subnet forwarding where each subnet is represented by a 1211 different bridge domain/EVI. In such scenarios the inter-subnet 1212 forwarding is performed at layer 3 and the PE that performs such 1213 function is called the default gateway. 1215 Each PE that acts as a default gateway for a given E-VPN advertises 1216 in the E-VPN control plane its default gateway IP and MAC address 1217 using the MAC advertisement route, and indicates that such route is 1218 associated with the default gateway. This is accomplished by 1219 requiring the route to carry the Default Gateway extended community 1220 defined in [Section 8.8 Default Gateway Extended Community]. 1222 Each PE that receives this route and imports it as per procedures 1223 specified in this document follows the procedures in this section 1224 when replying to ARP Requests that it receives if such Requests are 1225 for the IP address in the received E-VPN route. 1227 Each PE that acts as a default gateway for a given E-VPN that 1228 receives this route and imports it as per procedures specified in 1229 this document MUST create MAC forwarding state that enables it to 1230 apply IP forwarding to the packets destined to the MAC address 1231 carried in the route. 1233 12. Handling of Multi-Destination Traffic 1235 Procedures are required for a given PE to send broadcast or multicast 1236 traffic, received from a CE encapsulated in a given Ethernet Tag in 1237 an EVI, to all the other PEs that span that Ethernet Tag in the EVI. 1238 In certain scenarios, described in section "Processing of Unknown 1239 Unicast Packets", a given PE may also need to flood unknown unicast 1240 traffic to other PEs. 1242 The PEs in a particular E-VPN may use ingress replication, P2MP LSPs 1243 or MP2MP LSPs to send unknown unicast, broadcast or multicast traffic 1244 to other PEs. 1246 Each PE MUST advertise an "Inclusive Multicast Ethernet Tag Route" to 1247 enable the above. The following subsection provides the procedures to 1248 construct the Inclusive Multicast Ethernet Tag route. Subsequent 1249 subsections describe in further detail its usage. 1251 12.1. Construction of the Inclusive Multicast Ethernet Tag Route 1253 The RD MUST be the RD of the EVI that is advertising the NLRI. The 1254 procedures for setting the RD for a given E-VPN are described in 1255 section 9.4.1. 1257 The Ethernet Tag ID is the identifier of the Ethernet Tag. It MAY be 1258 set to 0 or to a valid Ethernet Tag value. 1260 The Originating Router's IP address MUST be set to an IP address of 1261 the PE. This address SHOULD be common for all the EVIs on the PE 1262 (e.,g., this address may be PE's loopback address). 1264 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1265 be set to the same IP address as the one carried in the Originating 1266 Router's IP Address field. 1268 The BGP advertisement for the Inclusive Multicast Ethernet Tag route 1269 MUST also carry one or more Route Target (RT) attributes. The 1270 assignment of RTs described in the section on "Constructing the BGP 1271 E-VPN MAC Address Advertisement" MUST be followed. 1273 12.2. P-Tunnel Identification 1275 In order to identify the P-Tunnel used for sending broadcast, unknown 1276 unicast or multicast traffic, the Inclusive Multicast Ethernet Tag 1277 route MUST carry a "PMSI Tunnel Attribute" as specified in [BGP 1278 MVPN]. 1280 Depending on the technology used for the P-tunnel for the E-VPN on 1281 the PE, the PMSI Tunnel attribute of the Inclusive Multicast Ethernet 1282 Tag route is constructed as follows. 1284 + If the PE that originates the advertisement uses a 1285 P-Multicast tree for the P-tunnel for E-VPN, the PMSI 1286 Tunnel attribute MUST contain the identity of the tree 1287 (note that the PE could create the identity of the 1288 tree prior to the actual instantiation of the tree). 1290 + An PE that uses a P-Multicast tree for the P-tunnel MAY 1291 aggregate two or more Ethernet Tags in the same or different 1292 EVIs present on the PE onto the same tree. In this case, in 1293 addition to carrying the identity of the tree, the PMSI Tunnel 1294 attribute MUST carry an MPLS upstream assigned label which 1295 the PE has bound uniquely to the Ethernet Tag for the EVI 1296 associated with this update (as determined by its RTs). 1298 If the PE has already advertised Inclusive Multicast 1299 Ethernet Tag routes for two or more Ethernet Tags that it 1300 now desires to aggregate, then the PE MUST re-advertise 1301 those routes. The re-advertised routes MUST be the same 1302 as the original ones, except for the PMSI Tunnel attribute 1303 and the label carried in that attribute. 1305 + If the PE that originates the advertisement uses ingress 1306 replication for the P-tunnel for E-VPN, the route MUST 1307 include the PMSI Tunnel attribute with the Tunnel Type set to 1308 Ingress Replication and Tunnel Identifier set to a routable 1309 address of the PE. The PMSI Tunnel attribute MUST carry a 1310 downstream assigned MPLS label. This label is used to 1311 demultiplex the broadcast, multicast or unknown unicast E-VPN 1312 traffic received over a MP2P tunnel by the PE. 1314 + The Leaf Information Required flag of the PMSI Tunnel 1315 attribute MUST be set to zero, and MUST be ignored on receipt. 1317 13. Processing of Unknown Unicast Packets 1319 The procedures in this document do not require the PEs to flood 1320 unknown unicast traffic to other PEs. If PEs learn CE MAC addresses 1321 via a control plane protocol, the PEs can then distribute MAC 1322 addresses via BGP, and all unicast MAC addresses will be learnt prior 1323 to traffic to those destinations. 1325 However, if a destination MAC address of a received packet is not 1326 known by the PE, the PE may have to flood the packet. Flooding must 1327 take into account "split horizon forwarding" as follows: The 1328 principles behind the following procedures are borrowed from the 1329 split horizon forwarding rules in VPLS solutions [RFC4761] and 1330 [RFC4762]. When an PE capable of flooding (say PEx) receives a 1331 broadcast or multicast Ethernet frame, or one with an unknown 1332 destination MAC address, it must flood the frame. If the frame 1333 arrived from an attached CE, PEx must send a copy of the frame to 1334 every other attached CE participating in the EVI, on a different ESI 1335 than the one it received the frame on, as long as the PE is the DF 1336 for the egress ESI. In addition, the PE must flood the frame to all 1337 other PEs participating in the EVI. If, on the other hand, the frame 1338 arrived from another PE (say PEy), PEx must send a copy of the packet 1339 only to attached CEs as long as it is the DF for the egress ESI. PEx 1340 MUST NOT send the frame to other PEs, since PEy would have already 1341 done so. Split horizon forwarding rules apply to broadcast and 1342 multicast packets, as well as packets to an unknown MAC address. 1344 Whether or not to flood packets to unknown destination MAC addresses 1345 should be an administrative choice, depending on how learning happens 1346 between CEs and PEs. 1348 The PEs in a particular E-VPN may use ingress replication using RSVP- 1349 TE P2P LSPs or LDP MP2P LSPs for sending broadcast, multicast and 1350 unknown unicast traffic to other PEs. Or they may use RSVP-TE P2MP or 1351 LDP P2MP or LDP MP2MP LSPs for sending such traffic to other PEs. 1353 13.1. Ingress Replication 1355 If ingress replication is in use, the P-Tunnel attribute, carried in 1356 the Inclusive Multicast Ethernet Tag routes for the EVI, specifies 1357 the downstream label that the other PEs can use to send unknown 1358 unicast, multicast or broadcast traffic for the EVI to this 1359 particular PE. 1361 The PE that receives a packet with this particular MPLS label MUST 1362 treat the packet as a broadcast, multicast or unknown unicast packet. 1363 Further if the MAC address is a unicast MAC address, the PE MUST 1364 treat the packet as an unknown unicast packet. 1366 13.2. P2MP MPLS LSPs 1368 The procedures for using P2MP LSPs are very similar to VPLS 1369 procedures [VPLS-MCAST]. The P-Tunnel attribute used by an PE for 1370 sending unknown unicast, broadcast or multicast traffic for a 1371 particular EVI is advertised in the Inclusive Ethernet Tag Multicast 1372 route as described in section "Handling of Multi-Destination 1373 Traffic". 1375 The P-Tunnel attribute specifies the P2MP LSP identifier. This is the 1376 equivalent of an Inclusive tree in [VPLS-MCAST]. Note that multiple 1377 Ethernet Tags, which may be in different EVIs, may use the same P2MP 1378 LSP, using upstream labels [VPLS-MCAST]. This is the equivalent of an 1379 Aggregate Inclusive tree in [VPLS-MCAST]. When P2MP LSPs are used for 1380 flooding unknown unicast traffic, packet re-ordering is possible. 1382 The PE that receives a packet on the P2MP LSP specified in the PMSI 1383 Tunnel Attribute MUST treat the packet as a broadcast, multicast or 1384 unknown unicast packet. Further if the MAC address is a unicast MAC 1385 address, the PE MUST treat the packet as an unknown unicast packet. 1387 14. Forwarding Unicast Packets 1389 14.1. Forwarding packets received from a CE 1391 When an PE receives a packet from a CE, on a given Ethernet Tag, it 1392 must first look up the source MAC address of the packet. In certain 1393 environments the source MAC address MAY be used to authenticate the 1394 CE and determine that traffic from the host can be allowed into the 1395 network. Source MAC lookup MAY also be used for local MAC address 1396 learning. 1398 If the PE decides to forward the packet, the destination MAC address 1399 of the packet must be looked up. If the PE has received MAC address 1400 advertisements for this destination MAC address from one or more 1401 other PEs or learned it from locally connected CEs, it is considered 1402 as a known MAC address. Otherwise, the MAC address is considered as 1403 an unknown MAC address. 1405 For known MAC addresses the PE forwards this packet to one of the 1406 remote PEs or to a locally attached CE. When forwarding to a remote 1407 PE, the packet is encapsulated in the E-VPN MPLS label advertised by 1408 the remote PE, for that MAC address, and in the MPLS LSP label stack 1409 to reach the remote PE. 1411 If the MAC address is unknown and if the administrative policy on the 1412 PE requires flooding of unknown unicast traffic then: 1414 - The PE MUST flood the packet to other PEs. The 1415 PE MUST first encapsulate the packet in the ESI MPLS 1416 label as described in section 9.3. 1417 If ingress replication is used, the packet MUST be replicated 1418 one or more times to each remote PE with the outermost 1419 label being an MPLS label determined as follows: This 1420 is the MPLS label advertised by the remote PE in a PMSI 1421 Tunnel Attribute in the Inclusive Multicast Ethernet Tag 1422 route for an combination. The Ethernet 1423 Tag in the route must be the same as the Ethernet Tag 1424 associated with the interface on which the ingress PE 1425 receives the packet. If P2MP LSPs are being used the packet 1426 MUST be sent on the P2MP LSP that the PE is the root of for 1427 the Ethernet Tag in the EVI. If the same P2MP LSP is used 1428 for all Ethernet Tags, then all the PEs in the EVI MUST 1429 be the leaves of the P2MP LSP. If a distinct P2MP LSP is 1430 used for a given Ethernet Tag in the EVI, then only the 1431 PEs in the Ethernet Tag MUST be the leaves of the P2MP 1432 LSP. The packet MUST be encapsulated in the P2MP LSP label 1433 stack. 1435 If the MAC address is unknown then, if the administrative policy on 1436 the PE does not allow flooding of unknown unicast traffic: 1438 - The PE MUST drop the packet. 1440 14.2. Forwarding packets received from a remote PE 1441 14.2.1. Unknown Unicast Forwarding 1443 When an PE receives an MPLS packet from a remote PE then, after 1444 processing the MPLS label stack, if the top MPLS label ends up being 1445 a P2MP LSP label associated with an EVI or the downstream label 1446 advertised in the P-Tunnel attribute, and after performing the split 1447 horizon procedures described in section "Split Horizon": 1449 - If the PE is the designated forwarder of unknown unicast, broadcast 1450 or multicast traffic, on a particular set of ESIs for the Ethernet 1451 Tag, the default behavior is for the PE to flood the packet on these 1452 ESIs. In other words, the default behavior is for the PE to assume 1453 that the destination MAC address is unknown unicast, broadcast or 1454 multicast and it is not required to perform a destination MAC address 1455 lookup. As an option, the PE may perform a destination MAC lookup to 1456 flood the packet to only a subset of the CE interfaces in the 1457 Ethernet Tag. For instance the PE may decide to not flood an unknown 1458 unicast packet on certain Ethernet segments even if it is the DF on 1459 the Ethernet segment, based on administrative policy. 1461 - If the PE is not the designated forwarder on any of the ESIs for 1462 the Ethernet Tag, the default behavior is for it to drop the packet. 1464 14.2.2. Known Unicast Forwarding 1466 If the top MPLS label ends up being an E-VPN label that was 1467 advertised in the unicast MAC advertisements, then the PE either 1468 forwards the packet based on CE next-hop forwarding information 1469 associated with the label or does a destination MAC address lookup to 1470 forward the packet to a CE. 1472 15. Load Balancing of Unicast Frames 1474 This section specifies the load balancing procedures for sending 1475 known unicast frames to a multi-homed CE. 1477 15.1. Load balancing of traffic from an PE to remote CEs 1479 Whenever a remote PE imports a MAC advertisement for a given in an EVI, it MUST examine all imported Ethernet A-D 1481 routes for that ESI in order to determine the load-balancing 1482 characteristics of the Ethernet segment. 1484 15.1.1 Active-Standby Redundancy Mode 1486 For a given ESI, if the remote PE has imported an Ethernet A-D route 1487 per Ethernet Segment from at least one PE, where the "Active-Standby" 1488 flag in the ESI MPLS Label Extended Community is set, then the remote 1489 PE MUST deduce that the Ethernet segment is operating in Active- 1490 Standby redundancy mode. As such, the MAC address will be reachable 1491 only via the PE announcing the associated MAC Advertisement route - 1492 this is referred to as the primary PE. The set of other PE nodes 1493 advertising Ethernet A-D routes per Ethernet Segment for the same ESI 1494 serve as backup paths, in case the active PE encounters a failure. 1496 These are referred to as the backup PEs. 1498 If the primary PE encounters a failure, it MAY withdraw its Ethernet 1499 A-D route for the affected segment prior to withdrawing the entire 1500 set of MAC Advertisement routes. In the case where only a single 1501 other backup PE in the network had advertised an Ethernet A-D route 1502 for the same ESI, the remote PE can then use the Ethernet A-D route 1503 withdrawal as a trigger to update its forwarding entries, for the 1504 associated MAC addresses, to point towards the backup PE. As the 1505 backup PE starts learning the MAC addresses over its attached 1506 Ethernet segment, it will start sending MAC Advertisement routes 1507 while the failed PE withdraws its own. This mechanism minimizes the 1508 flooding of traffic during fail-over events. 1510 15.1.2 All-Active Redundancy Mode 1512 If for the given ESI, none of the Ethernet A-D routes per Ethernet 1513 Segment imported by the remote PE have the "Active-Standby" flag set 1514 in the ESI MPLS Label Extended Community, then the remote PE MUST 1515 treat the Ethernet segment as operating in all-active redundancy 1516 mode. The remote PE would then treat the MAC address as reachable via 1517 all of the PE nodes from which it has received both an Ethernet A-D 1518 route per Ethernet Segment as well as an Ethernet A-D route per EVI 1519 for the ESI in question. The remote PE MUST use the MAC advertisement 1520 and eligible Ethernet A-D routes to construct the set of next-hops 1521 that it can use to send the packet to the destination MAC. Each next- 1522 hop comprises an MPLS label stack that is to be used by the egress PE 1523 to forward the packet. This label stack is determined as follows: 1525 -If the next-hop is constructed as a result of a MAC route then this 1526 label stack MUST be used. However, if the MAC route doesn't exist, 1527 then the next-hop and MPLS label stack is constructed as a result of 1528 the Ethernet A-D routes. Note that the following description applies 1529 to determining the label stack for a particular next-hop to reach a 1530 given PE, from which the remote PE has received and imported Ethernet 1531 A-D routes that have the matching ESI and Ethernet Tag as the one 1532 present in the MAC advertisement. The Ethernet A-D routes mentioned 1533 in the following description refer to the ones imported from this 1534 given PE. 1536 -If an Ethernet A-D route per Ethernet Segment for that ESI exists, 1537 together with an Ethernet A-D route per EVI, then the label from that 1538 latter route must be used. 1540 The following example explains the above. 1542 Consider a CE (CE1) that is dual-homed to two PEs (PE1 and PE2) on a 1543 LAG interface (ES1), and is sending packets with MAC address MAC1 on 1544 VLAN1. A remote PE, say PE3, is able to learn that MAC1 is reachable 1545 via PE1 and PE2. Both PE1 and PE2 may advertise MAC1 in BGP if they 1546 receive packets with MAC1 from CE1. If this is not the case, and if 1547 MAC1 is advertised only by PE1, PE3 still considers MAC1 as reachable 1548 via both PE1 and PE2 as both PE1 and PE2 advertise a Ethernet A-D 1549 route per ESI for ESI1 as well as an Ethernet A-D route per EVI for 1550 . 1552 The MPLS label stack to send the packets to PE1 is the MPLS LSP stack 1553 to get to PE1 and the E-VPN label advertised by PE1 for CE1's MAC. 1555 The MPLS label stack to send packets to PE2 is the MPLS LSP stack to 1556 get to PE2 and the MPLS label in the Ethernet A-D route advertised by 1557 PE2 for , if PE2 has not advertised MAC1 in BGP. 1559 We will refer to these label stacks as MPLS next-hops. 1561 The remote PE (PE3) can now load balance the traffic it receives from 1562 its CEs, destined for CE1, between PE1 and PE2. PE3 may use N-Tuple 1563 flow information to hash traffic into one of the MPLS next-hops for 1564 load balancing of IP traffic. Alternatively PE3 may rely on the 1565 source MAC addresses for load balancing. 1567 Note that once PE3 decides to send a particular packet to PE1 or PE2 1568 it can pick one out of multiple possible paths to reach the 1569 particular remote PE using regular MPLS procedures. For instance, if 1570 the tunneling technology is based on RSVP-TE LSPs, and PE3 decides to 1571 send a particular packet to PE1, then PE3 can choose from multiple 1572 RSVP-TE LSPs that have PE1 as their destination. 1574 When PE1 or PE2 receive the packet destined for CE1 from PE3, if the 1575 packet is a unicast MAC packet it is forwarded to CE1. If it is a 1576 multicast or broadcast MAC packet then only one of PE1 or PE2 must 1577 forward the packet to the CE. Which of PE1 or PE2 forward this packet 1578 to the CE is determined based on which of the two is the DF. 1580 If the connectivity between the multi-homed CE and one of the PEs 1581 that it is attached to fails, the PE MUST withdraw the Ethernet Tag 1582 A-D routes, that had been previously advertised, for the Ethernet 1583 Segment to the CE. When the MAC entry on the PE ages out, the PE MUST 1584 withdraw the MAC address from BGP. Note that to aid convergence, the 1585 Ethernet Tag A-D routes MAY be withdrawn before the MAC routes. This 1586 enables the remote PEs to remove the MPLS next-hop to this particular 1587 PE from the set of MPLS next-hops that can be used to forward traffic 1588 to the CE. For further details and procedures on withdrawal of E-VPN 1589 route types in the event of PE to CE failures please section "PE to 1590 CE Network Failures". 1592 15.2. Load balancing of traffic between an PE and a local CE 1594 A CE may be configured with more than one interface connected to 1595 different PEs or the same PE for load balancing, using a technology 1596 such as LAG. The PE(s) and the CE can load balance traffic onto these 1597 interfaces using one of the following mechanisms. 1599 15.2.1. Data plane learning 1601 Consider that the PEs perform data plane learning for local MAC 1602 addresses learned from local CEs. This enables the PE(s) to learn a 1603 particular MAC address and associate it with one or more interfaces, 1604 if the technology between the PE and the CE supports multi-pathing. 1605 The PEs can now load balance traffic destined to that MAC address on 1606 the multiple interfaces. 1608 Whether the CE can load balance traffic that it generates on the 1609 multiple interfaces is dependent on the CE implementation. 1611 15.2.2. Control plane learning 1613 The CE can be a host that advertises the same MAC address using a 1614 control protocol on both interfaces. This enables the PE(s) to learn 1615 the host's MAC address and associate it with one or more interfaces. 1616 The PEs can now load balance traffic destined to the host on the 1617 multiple interfaces. The host can also load balance the traffic it 1618 generates onto these interfaces and the PE that receives the traffic 1619 employs E-VPN forwarding procedures to forward the traffic. 1621 16. MAC Mobility 1623 It is possible for a given host or end-station (as defined by its MAC 1624 address) to move from one Ethernet segment to another; this is 1625 referred to as 'MAC Mobility' or 'MAC move' and it is different from 1626 the multi-homing situation in which a given MAC address is reachable 1627 via multiple PEs for the same Ethernet segment. In a MAC move, there 1628 would be two sets of MAC Advertisement routes, one set with the new 1629 Ethernet segment and one set with the previous Ethernet segment, and 1630 the MAC address would appear to be reachable via each of these 1631 segments. 1633 In order to allow all of the PEs in the E-VPN to correctly determine 1634 the current location of the MAC address, all advertisements of it 1635 being reachable via the previous Ethernet segment MUST be withdrawn 1636 by the PEs, for the previous Ethernet segment, that had advertised 1637 it. 1639 If local learning is performed using the data plane, these PEs will 1640 not be able to detect that the MAC address has moved to another 1641 Ethernet segment and the receipt of MAC Advertisement routes, with 1642 the MAC Mobility extended community attribute, from other PEs serves 1643 as the trigger for these PEs to withdraw their advertisements. If 1644 local learning is performed using the control or management planes, 1645 these interactions serve as the trigger for these PEs to withdraw 1646 their advertisements. 1648 In a situation where there are multiple moves of a given MAC, 1649 possibly between the same two Ethernet segments, there may be 1650 multiple withdrawals and re-advertisements. In order to ensure that 1651 all PEs in the E-VPN receive all of these correctly through the 1652 intervening BGP infrastructure, it is necessary to introduce a 1653 sequence number into the MAC Mobility extended community attribute. 1655 Since the sequence number is an unsigned 32 bit integer, all sequence 1656 number comparisons must be performed modulo 2**32. This unsigned 1657 arithmetic preserves the relationship of sequence numbers as they 1658 cycle from 2**32 - 1 to 0. 1660 Every MAC mobility event for a given MAC address will contain a 1661 sequence number that is set using the following rules: 1663 - A PE advertising a MAC address for the first time advertises it 1664 with no MAC Mobility extended community attribute. 1666 - A PE detecting a locally attached MAC address for which it had 1667 previously received a MAC Advertisement route with a different 1668 Ethernet segment identifier advertises the MAC address in a MAC 1669 Advertisement route tagged with a MAC Mobility extended community 1670 attribute with a sequence number one greater than the sequence number 1671 in the MAC mobility attribute of the received MAC Advertisement 1672 route. In the case of the first mobility event for a given MAC 1673 address, where the received MAC Advertisement route does not carry a 1674 MAC Mobility attribute, the value of the sequence number in the 1675 received route is assumed to be 0 for purpose of this processing. 1677 - A PE detecting a locally attached MAC address for which it had 1678 previously received a MAC Advertisement route with the same Ethernet 1679 segment identifier advertises it with: 1680 i. no MAC Mobility extended community attribute, if the received 1681 route did not carry said attribute. 1683 ii. a MAC Mobility extended community attribute with the sequence 1684 number equal to the sequence number in the received MAC 1685 Advertisement route, if the received route is tagged with a MAC 1686 Mobility extended community attribute. 1688 A PE receiving a MAC Advertisement route for a MAC address with a 1689 different Ethernet segment identifier and a higher sequence number 1690 than that which it had previously advertised, withdraws its MAC 1691 Advertisement route. If two (or more) PEs advertise the same MAC 1692 address with same sequence number but different Ethernet segment 1693 identifiers, a PE that receives these routes selects the route 1694 advertised by the PE with lowest IP address as the best route. 1696 17. Multicast 1698 The PEs in a particular E-VPN may use ingress replication or P2MP 1699 LSPs to send multicast traffic to other PEs. 1701 17.1. Ingress Replication 1703 The PEs may use ingress replication for flooding unknown unicast, 1704 multicast or broadcast traffic as described in section "Handling of 1705 Multi-Destination Traffic". A given unknown unicast or broadcast 1706 packet must be sent to all the remote PEs. However a given multicast 1707 packet for a multicast flow may be sent to only a subset of the PEs. 1708 Specifically a given multicast flow may be sent to only those PEs 1709 that have receivers that are interested in the multicast flow. 1710 Determining which of the PEs have receivers for a given multicast 1711 flow is done using explicit tracking described below. 1713 17.2. P2MP LSPs 1715 An PE may use an "Inclusive" tree for sending an unknown unicast, 1716 broadcast or multicast packet or a "Selective" tree. This terminology 1717 is borrowed from [VPLS-MCAST]. 1719 A variety of transport technologies may be used in the SP network. 1720 For inclusive P-Multicast trees, these transport technologies include 1721 point-to-multipoint LSPs created by RSVP-TE or mLDP. For selective P- 1722 Multicast trees, only unicast PE-PE tunnels (using MPLS or IP/GRE 1723 encapsulation) and P2MP LSPs are supported, and the supported P2MP 1724 LSP signaling protocols are RSVP-TE, and mLDP. 1726 17.3. MP2MP LSPs 1728 The root of the MP2MP LDP LSP advertises the Inclusive Multicast Tag 1729 route with the PMSI Tunnel attribute set to the MP2MP Tunnel 1730 identifier. This advertisement is then sent to all PEs in the E-VPN. 1731 Upon receiving the Inclusive Multicast Tag routes with a PMSI Tunnel 1732 attribute that contains the MP2MP Tunnel identifier, the receiving 1733 PEs initiate the setup of the MP2MP tunnel towards the root using the 1734 procedures in [MLDP]. 1736 17.3.1. Inclusive Trees 1738 An Inclusive Tree allows the use of a single multicast distribution 1739 tree, referred to as an Inclusive P-Multicast tree, in the SP network 1740 to carry all the multicast traffic from a specified set of EVIs on a 1741 given PE. A particular P-Multicast tree can be set up to carry the 1742 traffic originated by sites belonging to a single E-VPN, or to carry 1743 the traffic originated by sites belonging to different E-VPNs. The 1744 ability to carry the traffic of more than one E-VPN on the same tree 1745 is termed 'Aggregation'. The tree needs to include every PE that is a 1746 member of any of the E-VPNs that are using the tree. This implies 1747 that an PE may receive multicast traffic for a multicast stream even 1748 if it doesn't have any receivers that are interested in receiving 1749 traffic for that stream. 1751 An Inclusive P-Multicast tree as defined in this document is a P2MP 1752 tree. A P2MP tree is used to carry traffic only for E-VPN CEs that 1753 are connected to the PE that is the root of the tree. 1755 The procedures for signaling an Inclusive Tree are the same as those 1756 in [VPLS-MCAST] with the VPLS-AD route replaced with the Inclusive 1757 Multicast Ethernet Tag route. The P-Tunnel attribute [VPLS-MCAST] for 1758 an Inclusive tree is advertised in the Inclusive Multicast route as 1759 described in section "Handling of Multi-Destination Traffic". Note 1760 that an PE can "aggregate" multiple inclusive trees for different 1761 EVIs on the same P2MP LSP using upstream labels. The procedures for 1762 aggregation are the same as those described in [VPLS-MCAST], with 1763 VPLS A-D routes replaced by E-VPN Inclusive Multicast routes. 1765 17.3.2. Selective Trees 1767 A Selective P-Multicast tree is used by an PE to send IP multicast 1768 traffic for one or more specific IP multicast streams, originated by 1769 CEs connected to the PE, that belong to the same or different E-VPNs, 1770 to a subset of the PEs that belong to those E-VPNs. Each of the PEs 1771 in the subset should be on the path to a receiver of one or more 1772 multicast streams that are mapped onto the tree. The ability to use 1773 the same tree for multicast streams that belong to different E-VPNs 1774 is termed an PE the ability to create separate SP multicast trees for 1775 specific multicast streams, e.g. high bandwidth multicast streams. 1776 This allows traffic for these multicast streams to reach only those 1777 PE routers that have receivers in these streams. This avoids flooding 1778 other PE routers in the E-VPN. 1780 An SP can use both Inclusive P-Multicast trees and Selective P- 1781 Multicast trees or either of them for a given E-VPN on an PE, based 1782 on local configuration. 1784 The granularity of a selective tree is where S is an 1785 IP multicast source address and G is an IP multicast group address or 1786 G is a multicast MAC address. Wildcard sources and wildcard groups 1787 are supported. Selective trees require explicit tracking as described 1788 below. 1790 A E-VPN PE advertises a selective tree using a E-VPN selective A-D 1791 route. The procedures are the same as those in [VPLS-MCAST] with S- 1792 PMSI A-D routes in [VPLS-MCAST] replaced by E-VPN Selective A-D 1793 routes. The information elements of the E-VPN selective A-D route 1794 are similar to those of the VPLS S-PMSI A-D route with the following 1795 differences. A E-VPN Selective A-D route includes an optional 1796 Ethernet Tag field. Also an E-VPN selective A-D route may encode a 1797 MAC address in the Group field. The encoding details of the E-VPN 1798 selective A-D route will be described in the next revision. 1800 Selective trees can also be aggregated on the same P2MP LSP using 1801 aggregation as described in [VPLS-MCAST]. 1803 17.4. Explicit Tracking 1805 [VPLS-MCAST] describes procedures for explicit tracking that rely on 1806 Leaf A-D routes. The same procedures are used for explicit tracking 1807 in this specification with VPLS Leaf A-D routes replaced with E-VPN 1808 Leaf A-D routes. These procedures allow a root PE to request 1809 multicast membership information for a given (S, G), from leaf PEs. 1810 Leaf PEs rely on IGMP snooping or PIM snooping between the PE and the 1811 CE to determine the multicast membership information. Note that the 1812 procedures in [VPLS-MCAST] do not describe how explicit tracking is 1813 performed if the CEs are enabled with join suppression. The 1814 procedures for this case will be described in a future version. 1816 18. Convergence 1818 This section describes failure recovery from different types of 1819 network failures. 1821 18.1. Transit Link and Node Failures between PEs 1823 The use of existing MPLS Fast-Reroute mechanisms can provide failure 1824 recovery in the order of 50ms, in the event of transit link and node 1825 failures in the infrastructure that connects the PEs. 1827 18.2. PE Failures 1829 Consider a host host1 that is dual homed to PE1 and PE2. If PE1 1830 fails, a remote PE, PE3, can discover this based on the failure of 1831 the BGP session. This failure detection can be in the sub-second 1832 range if BFD is used to detect BGP session failure. PE3 can update 1833 its forwarding state to start sending all traffic for host1 to only 1834 PE2. It is to be noted that this failure recovery is potentially 1835 faster than what would be possible if data plane learning were to be 1836 used. As in that case PE3 would have to rely on re-learning of MAC 1837 addresses via PE2. 1839 18.2.1. Local Repair 1841 It is possible to perform local repair in the case of PE failures. 1842 Details will be specified in the future. 1844 18.3. PE to CE Network Failures 1846 When an Ethernet segment connected to an PE fails or when a Ethernet 1847 Tag is decommissioned on an Ethernet segment, then the PE MUST 1848 withdraw the Ethernet A-D route(s) announced for the that are impacted by the failure or decommissioning. In 1850 addition, the PE MUST also withdraw the MAC advertisement routes that 1851 are impacted by the failure or decommissioning. 1853 The Ethernet A-D routes should be used by an implementation to 1854 optimize the withdrawal of MAC advertisement routes. When an PE 1855 receives a withdrawal of a particular Ethernet A-D route from an PE 1856 it SHOULD consider all the MAC advertisement routes, that are learned 1857 from the same as in the Ethernet A-D route, from 1858 the advertising PE, as having been withdrawn. This optimizes the 1859 network convergence times in the event of PE to CE failures. 1861 19. LACP State Synchronization 1863 This section requires review and discussion amongst the authors and 1864 will be revised in the next version. 1866 To support CE multi-homing with multi-chassis Ethernet bundles, the 1867 PEs connected to a given CE should synchronize [802.1AX] LACP state 1868 amongst each other. This ensures that the PEs can present a single 1869 LACP bundle to the CE. This is required for initial system bring-up 1870 and upon any configuration change. 1872 This includes at least the following LACP specific configuration 1873 parameters: 1875 - System Identifier (MAC Address): uniquely identifies a LACP 1876 speaker. 1877 - System Priority: determines which LACP speaker's port 1878 priorities are used in the Selection logic. 1879 - Aggregator Identifier: uniquely identifies a bundle within 1880 a LACP speaker. 1881 - Aggregator MAC Address: identifies the MAC address of the 1882 bundle. 1883 - Aggregator Key: used to determine which ports can join an 1884 Aggregator. 1885 - Port Number: uniquely identifies an interface within a LACP 1886 speaker. 1887 - Port Key: determines the set of ports that can be bundled. 1888 - Port Priority: determines a port's precedence level to join 1889 a bundle in case the number of eligible ports exceeds the 1890 maximum number of links allowed in a bundle. 1892 Furthermore, the PEs should also synchronize operational (run-time) 1893 data, in order for the LACP Selection logic state-machines to 1894 execute. This operational data includes the following LACP 1895 operational parameters, on a per port basis: 1897 - Partner System Identifier: this is the CE System MAC address. 1898 - Partner System Priority: the CE LACP System Priority 1899 - Partner Port Number: CE's AC port number. 1900 - Partner Port Priority: CE's AC Port Priority. 1901 - Partner Key: CE's key for this AC. 1902 - Partner State: CE's LACP State for the AC. 1903 - Actor State: PE's LACP State for the AC. 1904 - Port State: PE's AC port status. 1906 The above state needs to be communicated between PEs forming a multi- 1907 chassis bundle during LACP initial bringup, upon any configuration 1908 change and upon the occurrence of a failure. 1910 It should be noted that the above configuration and operational state 1911 is localized in scope and is only relevant to PEs which connect to 1912 the same multi-homed CE over a given Ethernet bundle. 1914 Furthermore, the communication of state changes, upon failures, must 1915 occur with minimal latency, in order to minimize the switchover time 1916 and consequent service disruption. The protocol details for 1917 synchronizing the LACP state will be described in the following 1918 version. 1920 20. Acknowledgements 1922 We would like to thank Yakov Rekhter, Pedro Marques, Kaushik Ghosh, 1923 Nischal Sheth, Robert Raszuk, Amit Shukla and Nadeem Mohammed for 1924 discussions that helped shape this document. We would also like to 1925 thank Han Nguyen for his comments and support of this work. We would 1926 also like to thank Steve Kensil for his review. 1928 21. Security Considerations 1930 22. IANA Considerations 1932 23. References 1934 23.1 Normative References 1936 [RFC4364] "BGP/MPLS IP VPNs", Rosen, Rekhter, et. al., February 2006 1938 [RFC4761] Kompella, K. and Y. Rekhter, "Virtual Private LAN Service 1939 (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 1940 4761, January 2007. 1942 [RFC4762] Lasserre, M. and V. Kompella, "Virtual Private LAN Service 1943 (VPLS) Using Label Distribution Protocol (LDP) Signaling", 1944 RFC 4762, January 2007. 1946 [RFC4271] Y. Rekhter et. al., "A Border Gateway Protocol 4 (BGP-4)", 1947 RFC 4271, January 2006 1949 [RFC4760] T. Bates et. al., "Multiprotocol Extensions for BGP-4", RFC 1950 4760, January 2007 1952 23.2 Informative References 1954 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1955 Requirement Levels", BCP 14, RFC 2119, March 1997. 1957 [EVPN-REQ] A. Sajassi, R. Aggarwal et. al., "Requirements for 1958 Ethernet VPN", draft-ietf-l2vpn-evpn-req-01.txt 1960 [VPLS-MCAST] "Multicast in VPLS". R. Aggarwal et.al., draft-ietf- 1961 l2vpn-vpls-mcast-11.txt 1963 [RT-CONSTRAIN] P. Marques et. al., "Constrained Route Distribution 1964 for Border Gateway Protocol/MultiProtocol Label Switching 1965 (BGP/MPLS) Internet Protocol (IP) Virtual Private Networks 1966 (VPNs)", RFC 4684, November 2006 1968 [BGP-VPLS-MH] "BGP based Multi-homing in Virtual Private LAN 1969 Service", K. Kompella et. al., draft-ietf-l2vpn-vpls- 1970 multihoming-04.txt 1972 24. Author's Address 1974 Ali Sajassi 1975 Cisco 1976 Email: sajassi@cisco.com 1978 Rahul Aggarwal 1979 Email: raggarwa_1@yahoo.com 1981 Wim Henderickx 1982 Alcatel-Lucent 1983 e-mail: wim.henderickx@alcatel-lucent.com 1985 Aldrin Isaac 1986 Bloomberg 1987 Email: aisaac71@bloomberg.net 1989 James Uttaro 1990 AT&T 1991 200 S. Laurel Avenue 1992 Middletown, NJ 07748 1993 USA 1994 Email: uttaro@att.com 1996 Nabil Bitar 1997 Verizon Communications 1998 Email : nabil.n.bitar@verizon.com 2000 Ravi Shekhar 2001 Juniper Networks 2002 1194 N. Mathilda Ave. 2003 Sunnyvale, CA 94089 US 2004 Email: rshekhar@juniper.net 2006 Florin Balus 2007 Alcatel-Lucent 2008 e-mail: Florin.Balus@alcatel-lucent.com 2010 Keyur Patel 2011 Cisco 2012 170 West Tasman Drive 2013 San Jose, CA 95134, US 2014 Email: keyupate@cisco.com 2016 Sami Boutros 2017 Cisco 2018 170 West Tasman Drive 2019 San Jose, CA 95134, US 2020 Email: sboutros@cisco.com 2022 Samer Salam 2023 Cisco 2024 Email: ssalam@cisco.com 2026 John Drake 2027 Juniper Networks 2028 Email: jdrake@juniper.net