idnits 2.17.1 draft-ietf-bess-rfc7432bis-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (28 February 2022) is 788 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC4448' is mentioned on line 1075, but not defined == Missing Reference: 'EVI' is mentioned on line 2268, but not defined == Missing Reference: 'BD' is mentioned on line 2268, but not defined == Outdated reference: A later version (-21) exists of draft-ietf-bess-evpn-igmp-mld-proxy-18 == Outdated reference: A later version (-08) exists of draft-ietf-bess-evpn-mh-split-horizon-02 == Outdated reference: A later version (-14) exists of draft-ietf-bess-mvpn-evpn-aggregation-label-06 == Outdated reference: A later version (-14) exists of draft-ietf-bier-evpn-04 Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Working Group A. Sajassi 3 Internet-Draft LA. Burdet, Ed. 4 Intended status: Standards Track Cisco 5 Expires: 1 September 2022 J. Drake 6 Juniper 7 J. Rabadan 8 Nokia 9 28 February 2022 11 BGP MPLS-Based Ethernet VPN 12 draft-ietf-bess-rfc7432bis-03 14 Abstract 16 This document describes procedures for BGP MPLS-based Ethernet VPNs 17 (EVPN). The procedures described here meet the requirements 18 specified in RFC 7209 -- "Requirements for Ethernet VPN (EVPN)". 20 Note to Readers 22 _RFC EDITOR: please remove this section before publication_ 24 The complete and detailed set of all changes between this version and 25 RFC7432 may be found as an Annotated Diff (rfcdiff) here 26 (https://tools.ietf.org/rfcdiff?url1=https://www.rfc- 27 editor.org/rfc/rfc7432.txt&url2=https://www.ietf.org/archive/id/ 28 draft-ietf-bess-rfc7432bis-03.txt). 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 1 September 2022. 47 Copyright Notice 49 Copyright (c) 2022 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Revised BSD License text as 58 described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Revised BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.1. Summary of changes from RFC 7432 . . . . . . . . . . . . 4 65 2. Specification of Requirements . . . . . . . . . . . . . . . . 6 66 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 67 4. BGP MPLS-Based EVPN Overview . . . . . . . . . . . . . . . . 7 68 5. Ethernet Segment . . . . . . . . . . . . . . . . . . . . . . 9 69 6. Ethernet Tag ID . . . . . . . . . . . . . . . . . . . . . . . 12 70 6.1. VLAN-Based Service Interface . . . . . . . . . . . . . . 13 71 6.2. VLAN Bundle Service Interface . . . . . . . . . . . . . . 13 72 6.2.1. Port-Based Service Interface . . . . . . . . . . . . 13 73 6.3. VLAN-Aware Bundle Service Interface . . . . . . . . . . . 13 74 6.3.1. Port-Based VLAN-Aware Service Interface . . . . . . . 14 75 6.4. EVPN PE Model . . . . . . . . . . . . . . . . . . . . . . 14 76 7. BGP EVPN Routes . . . . . . . . . . . . . . . . . . . . . . . 16 77 7.1. Ethernet Auto-discovery Route . . . . . . . . . . . . . . 17 78 7.2. MAC/IP Advertisement Route . . . . . . . . . . . . . . . 17 79 7.3. Inclusive Multicast Ethernet Tag Route . . . . . . . . . 18 80 7.4. Ethernet Segment Route . . . . . . . . . . . . . . . . . 19 81 7.5. ESI Label Extended Community . . . . . . . . . . . . . . 20 82 7.6. ES-Import Route Target . . . . . . . . . . . . . . . . . 21 83 7.7. MAC Mobility Extended Community . . . . . . . . . . . . . 22 84 7.8. Default Gateway Extended Community . . . . . . . . . . . 22 85 7.9. Route Distinguisher Assignment per MAC-VRF . . . . . . . 22 86 7.10. Route Targets . . . . . . . . . . . . . . . . . . . . . . 23 87 7.10.1. Auto-derivation from the Ethernet Tag (VLAN ID) . . 23 88 7.11. EVPN Layer 2 Attributes Extended Community . . . . . . . 23 89 7.11.1. EVPN Layer 2 Attributes Partitioning . . . . . . . . 24 90 7.12. Route Prioritization . . . . . . . . . . . . . . . . . . 26 91 8. Multihoming Functions . . . . . . . . . . . . . . . . . . . . 26 92 8.1. Multihomed Ethernet Segment Auto-discovery . . . . . . . 26 93 8.1.1. Constructing the Ethernet Segment Route . . . . . . . 26 94 8.2. Fast Convergence . . . . . . . . . . . . . . . . . . . . 27 95 8.2.1. Constructing Ethernet A-D per Ethernet Segment 96 Route . . . . . . . . . . . . . . . . . . . . . . . . 28 97 8.2.1.1. Ethernet A-D Route Targets . . . . . . . . . . . 28 98 8.3. Split Horizon . . . . . . . . . . . . . . . . . . . . . . 29 99 8.3.1. ESI Label Assignment . . . . . . . . . . . . . . . . 29 100 8.3.1.1. Ingress Replication . . . . . . . . . . . . . . . 30 101 8.3.1.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . 31 102 8.3.1.3. MP2MP MPLS LSPs . . . . . . . . . . . . . . . . . 32 103 8.4. Aliasing and Backup Path . . . . . . . . . . . . . . . . 32 104 8.4.1. Constructing Ethernet A-D per EVPN Instance Route . . 33 105 8.5. Designated Forwarder Election . . . . . . . . . . . . . . 34 106 8.6. Signaling Primary and Backup DF Elected PEs . . . . . . . 37 107 8.7. Interoperability with Single-Homing PEs . . . . . . . . . 37 108 9. Determining Reachability to Unicast MAC Addresses . . . . . . 37 109 9.1. Local Learning . . . . . . . . . . . . . . . . . . . . . 38 110 9.2. Remote Learning . . . . . . . . . . . . . . . . . . . . . 38 111 9.2.1. Constructing MAC/IP Address Advertisement . . . . . . 38 112 9.2.2. Route Resolution . . . . . . . . . . . . . . . . . . 40 113 10. ARP and ND . . . . . . . . . . . . . . . . . . . . . . . . . 41 114 10.1. Default Gateway . . . . . . . . . . . . . . . . . . . . 42 115 10.1.1. Best Path selection for Default Gateway . . . . . . 44 116 11. Handling of Multi-destination Traffic . . . . . . . . . . . . 44 117 11.1. Constructing Inclusive Multicast Ethernet Tag Route . . 45 118 11.2. P-Tunnel Identification . . . . . . . . . . . . . . . . 45 119 12. Processing of Unknown Unicast Packets . . . . . . . . . . . . 46 120 12.1. Ingress Replication . . . . . . . . . . . . . . . . . . 47 121 12.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . . . . . 47 122 13. Forwarding Unicast Packets . . . . . . . . . . . . . . . . . 48 123 13.1. Forwarding Packets Received from a CE . . . . . . . . . 48 124 13.2. Forwarding Packets Received from a Remote PE . . . . . . 49 125 13.2.1. Unknown Unicast Forwarding . . . . . . . . . . . . . 49 126 13.2.2. Known Unicast Forwarding . . . . . . . . . . . . . . 50 127 14. Load Balancing of Unicast Packets . . . . . . . . . . . . . . 50 128 14.1. Load Balancing of Traffic from a PE to Remote CEs . . . 50 129 14.1.1. Single-Active Redundancy Mode . . . . . . . . . . . 50 130 14.1.2. All-Active Redundancy Mode . . . . . . . . . . . . . 51 131 14.2. Load Balancing of Traffic between a PE and a Local CE . 52 132 14.2.1. Data-Plane Learning . . . . . . . . . . . . . . . . 52 133 14.2.2. Control-Plane Learning . . . . . . . . . . . . . . . 53 134 15. MAC Mobility . . . . . . . . . . . . . . . . . . . . . . . . 53 135 15.1. MAC Duplication Issue . . . . . . . . . . . . . . . . . 55 136 15.2. Sticky MAC Addresses . . . . . . . . . . . . . . . . . . 55 137 15.3. Loop Protection . . . . . . . . . . . . . . . . . . . . 55 138 16. Multicast and Broadcast . . . . . . . . . . . . . . . . . . . 57 139 16.1. Ingress Replication . . . . . . . . . . . . . . . . . . 57 140 16.2. P2MP or MP2MP LSPs . . . . . . . . . . . . . . . . . . . 57 141 16.2.1. Inclusive Trees . . . . . . . . . . . . . . . . . . 57 142 17. Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 58 143 17.1. Transit Link and Node Failures between PEs . . . . . . . 58 144 17.2. PE Failures . . . . . . . . . . . . . . . . . . . . . . 58 145 17.3. PE-to-CE Network Failures . . . . . . . . . . . . . . . 58 146 18. Frame Ordering . . . . . . . . . . . . . . . . . . . . . . . 59 147 18.1. Flow Label . . . . . . . . . . . . . . . . . . . . . . . 60 148 19. Use of Domain-wide Common Block (DCB) Labels . . . . . . . . 60 149 20. Security Considerations . . . . . . . . . . . . . . . . . . . 61 150 21. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 63 151 22. References . . . . . . . . . . . . . . . . . . . . . . . . . 63 152 22.1. Normative References . . . . . . . . . . . . . . . . . . 64 153 22.2. Informative References . . . . . . . . . . . . . . . . . 65 154 Appendix A. Acknowledgments for This Document (2021) . . . . . . 67 155 Appendix B. Contributors for This Document (2021) . . . . . . . 67 156 Appendix C. Acknowledgments from the First Edition (2015) . . . 68 157 C.1. Contributors from the First Edition (2015) . . . . . . . 68 158 C.2. Authors from the First Edition (2015) . . . . . . . . . . 68 159 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 69 161 1. Introduction 163 Virtual Private LAN Service (VPLS), as defined in [RFC4664], 164 [RFC4761], and [RFC4762], is a proven and widely deployed technology. 165 However, the existing solution has a number of limitations when it 166 comes to multihoming and redundancy, multicast optimization, 167 provisioning simplicity, flow-based load balancing, and multipathing; 168 these limitations are important considerations for Data Center (DC) 169 deployments. [RFC7209] describes the motivation for a new solution 170 to address these limitations. It also outlines a set of requirements 171 that the new solution must address. 173 This document describes procedures for a BGP MPLS-based solution 174 called Ethernet VPN (EVPN) to address the requirements specified in 175 [RFC7209]. Please refer to [RFC7209] for the detailed requirements 176 and motivation. EVPN requires extensions to existing IP/MPLS 177 protocols as described in this document. In addition to these 178 extensions, EVPN uses several building blocks from existing MPLS 179 technologies. 181 1.1. Summary of changes from RFC 7432 183 This section describes the significant changes between [RFC4762] and 184 this document. 186 * Updates to Terminology i.a. BD, EVI, Ethernet Tag ID, P-tunnel, 187 DF/BDF/NDF, DCB; 189 * Added Section 6.4 for description and disambiguation of EVPN 190 bridging terminology; 192 * Added ES-Import route target auto-derivation for ESI types 0,4,5; 194 * Precision of 'encoding' language for all references to 'Label' 195 fields; 197 * Added Section 7.11 for usage of EVPN Layer 2 Attributes Extended 198 Community in EVPN Bridging; 200 * Added Section 7.12 proposes relative order-of-magnitude route 201 priority and processing to help achieve fast convergence; 203 * Corrected Section 8.2.1 to include reference to E-TREE exception; 205 * Updated Section 8.5 to include Backup- and Non-Designated 206 Forwarder roles to DF-Election algorithm, description of those 207 roles and signaling updates; 209 * Updated Section 8.5 to specify DF Election behaviour for 210 Originating IP in different family 212 * Added Section 8.3.1.3 for MP2MP MPLS LSPs and updated 213 Section 12.2; 215 * Address conflicts in Best Path algorithm for Default Gateway in 216 Section 10.1.1; 218 * Update to Section 14.1.1 redundancy mode description; 220 * Added Section 15.3 describing a loop detection and protection 221 mechanism; 223 * Added Section 18.1 describing Flow-label usage and signaling (see 224 also new Section 7.11); 226 * Section 19 specifies use of Domain-wide Common Block (DCB) for 227 several cases; 229 * Restructuring, namely Section 8.5 to Section 5, simplify all 230 Ethernet Tag ID references to Section 6 ; and 232 * Corrected Route Target and other extcomm 'attributes' references 233 to 'extended communities'; 235 * Cross-references and editorial changes; [RFC7991] and xml2rfc-v3 236 update (source). 238 2. Specification of Requirements 240 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 241 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 242 document are to be interpreted as described in [RFC2119]. 244 3. Terminology 246 BD: Broadcast Domain. In a bridged network, the broadcast domain 247 corresponds to a Virtual LAN (VLAN), where a VLAN is typically 248 represented by a single VLAN ID (VID) but can be represented by 249 several VIDs where Shared VLAN Learning (SVL) is used per 250 [IEEE.802.1Q_2014]. 252 Bridge Table: An instantiation of a broadcast domain on a MAC-VRF. 254 CE: Customer Edge device, e.g., a host, router, or switch. 256 EVI: An EVPN instance spanning the Provider Edge (PE) devices 257 participating in that EVPN. An EVI may be comprised of one BD 258 (VLAN-based, VLAN Bundle, or Port-based services) or multiple BDs 259 (VLAN-aware Bundle or Port-based VLAN-Aware services). 261 MAC-VRF: A Virtual Routing and Forwarding table for Media Access 262 Control (MAC) addresses on a PE. 264 Ethernet Segment (ES): When a customer site (device or network) is 265 connected to one or more PEs via a set of Ethernet links, then 266 that set of links is referred to as an 'Ethernet segment'. 268 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 269 identifies an Ethernet segment is called an 'Ethernet Segment 270 Identifier'. 272 VID: VLAN Identifier. 274 Ethernet Tag: Used to represent a BD that is configured on a given 275 ES for the purposes of DF election and identification 276 for frames received from the CE. Note that any of the following 277 may be used to represent a BD: VIDs (including Q-in-Q tags), 278 configured IDs, VNIs (Virtual Extensible Local Area Network 279 (VXLAN) Network Identifiers), normalized VIDs, I-SIDs (Service 280 Instance Identifiers), etc., as long as the representation of the 281 BDs is configured consistently across the multihomed PEs attached 282 to that ES. 284 Ethernet Tag ID: Normalized network wide ID that is used to identify 285 a BD within an EVI and carried in EVPN routes. 287 LACP: Link Aggregation Control Protocol. 289 MP2MP: Multipoint to Multipoint. 291 MP2P: Multipoint to Point. 293 P2MP: Point to Multipoint. 295 P2P: Point to Point. 297 P-tunnel: A tunnel through the network of one or more SPs. In this 298 document, P-tunnels are instantiated as bidirectional multicast 299 distribution trees. 301 PE: Provider Edge device. 303 Single-Active Redundancy Mode: When only a single PE, among all the 304 PEs attached to an Ethernet segment, is allowed to forward traffic 305 to/from that Ethernet segment for a given VLAN, then the Ethernet 306 segment is defined to be operating in Single-Active redundancy 307 mode. 309 All-Active Redundancy Mode: When all PEs attached to an Ethernet 310 segment are allowed to forward known unicast traffic to/from that 311 Ethernet segment for a given VLAN, then the Ethernet segment is 312 defined to be operating in All-Active redundancy mode. 314 BUM: Broadcast, unknown unicast, and multicast. 316 DF: Designated Forwarder. 318 Backup-DF (BDF): Backup-Designated Forwarder. 320 Non-DF (NDF): Non-Designated Forwarder. 322 DCB: Domain-wide Common Block (of labels), as in 323 [I-D.ietf-bess-mvpn-evpn-aggregation-label]. 325 AC: Attachment Circuit. 327 4. BGP MPLS-Based EVPN Overview 329 This section provides an overview of EVPN. An EVPN instance 330 comprises Customer Edge devices (CEs) that are connected to Provider 331 Edge devices (PEs) that form the edge of the MPLS infrastructure. A 332 CE may be a host, a router, or a switch. The PEs provide virtual 333 Layer 2 bridged connectivity between the CEs. There may be multiple 334 EVPN instances in the provider's network. 336 The PEs may be connected by an MPLS Label Switched Path (LSP) 337 infrastructure, which provides the benefits of MPLS technology, such 338 as fast reroute, resiliency, etc. The PEs may also be connected by 339 an IP infrastructure, in which case IP/GRE (Generic Routing 340 Encapsulation) tunneling or other IP tunneling can be used between 341 the PEs. The detailed procedures in this document are specified only 342 for MPLS LSPs as the tunneling technology. However, these procedures 343 are designed to be extensible to IP tunneling as the Packet Switched 344 Network (PSN) tunneling technology. 346 In an EVPN, MAC learning between PEs occurs not in the data plane (as 347 happens with traditional bridging in VPLS [RFC4761] [RFC4762]) but in 348 the control plane. Control-plane learning offers greater control 349 over the MAC learning process, such as restricting who learns what, 350 and the ability to apply policies. Furthermore, the control plane 351 chosen for advertising MAC reachability information is multi-protocol 352 (MP) BGP (similar to IP VPNs [RFC4364]). This provides flexibility 353 and the ability to preserve the "virtualization" or isolation of 354 groups of interacting agents (hosts, servers, virtual machines) from 355 each other. In EVPN, PEs advertise the MAC addresses learned from 356 the CEs that are connected to them, along with an MPLS label, to 357 other PEs in the control plane using Multiprotocol BGP (MP-BGP). 358 Control-plane learning enables load balancing of traffic to and from 359 CEs that are multihomed to multiple PEs. This is in addition to load 360 balancing across the MPLS core via multiple LSPs between the same 361 pair of PEs. In other words, it allows CEs to connect to multiple 362 active points of attachment. It also improves convergence times in 363 the event of certain network failures. 365 However, learning between PEs and CEs is done by the method best 366 suited to the CE: data-plane learning, IEEE 802.1x, the Link Layer 367 Discovery Protocol (LLDP), IEEE 802.1aq, Address Resolution Protocol 368 (ARP), management plane, or other protocols. 370 It is a local decision as to whether the Layer 2 forwarding table on 371 a PE is populated with all the MAC destination addresses known to the 372 control plane, or whether the PE implements a cache-based scheme. 373 For instance, the MAC forwarding table may be populated only with the 374 MAC destinations of the active flows transiting a specific PE. 376 The policy attributes of EVPN are very similar to those of IP-VPN. 377 An EVPN instance requires a Route Distinguisher (RD) that is unique 378 per MAC-VRF and one or more globally unique Route Targets (RTs). A 379 CE attaches to a BD on a PE, on an Ethernet interface that may be 380 configured for one or more Ethernet tags. If the Ethernet tags are 381 VLAN IDs, some deployment scenarios guarantee uniqueness of VLAN IDs 382 across EVPN instances: all points of attachment for a given EVPN 383 instance use the same VLAN ID, and no other EVPN instance uses this 384 VLAN ID. This document refers to this case as a "Unique VLAN EVPN" 385 and describes simplified procedures to optimize for it. See for 386 example Section 7.10.1 which describes deriving automatically the 387 RT(s) for each EVPN instance from the corresponding VID. 389 5. Ethernet Segment 391 As indicated in [RFC7209], each Ethernet segment needs a unique 392 identifier in an EVPN. This section defines how such identifiers are 393 assigned and how they are encoded for use in EVPN signaling. Later 394 sections of this document describe the protocol mechanisms that 395 utilize the identifiers. 397 When a customer site is connected to one or more PEs via a set of 398 Ethernet links, then this set of Ethernet links constitutes an 399 "Ethernet segment". For a multihomed site, each Ethernet segment 400 (ES) is identified by a unique non-zero identifier called an Ethernet 401 Segment Identifier (ESI). An ESI is encoded as a 10-octet integer in 402 line format with the most significant octet sent first. The 403 following two ESI values are reserved: 405 - ESI 0 denotes a single-homed site. 407 - ESI {0xFF} (repeated 10 times) is known as MAX-ESI and is reserved. 409 In general, an Ethernet segment SHOULD have a non-reserved ESI that 410 is unique network wide (i.e., across all EVPN instances on all the 411 PEs). If the CE(s) constituting an Ethernet segment is (are) managed 412 by the network operator, then ESI uniqueness should be guaranteed; 413 however, if the CE(s) is (are) not managed, then the operator MUST 414 configure a network-wide unique ESI for that Ethernet segment. This 415 is required to enable auto-discovery of Ethernet segments and 416 Designated Forwarder (DF) election. 418 In a network with managed and non-managed CEs, the ESI has the 419 following format: 421 +---+---+---+---+---+---+---+---+---+---+ 422 | T | ESI Value | 423 +---+---+---+---+---+---+---+---+---+---+ 425 Where: 427 T (ESI Type) is a 1-octet field (most significant octet) that 428 specifies the format of the remaining 9 octets (ESI Value). The 429 following six ESI types can be used: 431 * Type 0 (T=0x00) - This type indicates an arbitrary 9-octet ESI 432 value, which is managed and configured by the operator. 434 * Type 1 (T=0x01) - When IEEE 802.1AX LACP is used between the PEs 435 and CEs, this ESI type indicates an auto-generated ESI value 436 determined from LACP by concatenating the following parameters: 438 - CE LACP System MAC address (6 octets). The CE LACP System MAC 439 address MUST be encoded in the high-order 6 octets of the ESI 440 Value field. 442 - CE LACP Port Key (2 octets). The CE LACP port key MUST be 443 encoded in the 2 octets next to the System MAC address. 445 - The remaining octet will be set to 0x00. 447 As far as the CE is concerned, it would treat the multiple PEs 448 that it is connected to as the same switch. This allows the CE to 449 aggregate links that are attached to different PEs in the same 450 bundle. 451 This mechanism could be used only if it produces ESIs that satisfy 452 the uniqueness requirement specified above. 454 * Type 2 (T=0x02) - This type is used in the case of indirectly 455 connected hosts via a bridged LAN between the CEs and the PEs. 456 The ESI Value is auto-generated and determined based on the Layer 457 2 bridge protocol as follows: If the Multiple Spanning Tree 458 Protocol (MSTP) is used in the bridged LAN, then the value of the 459 ESI is derived by listening to Bridge PDUs (BPDUs) on the Ethernet 460 segment. To achieve this, the PE is not required to run MSTP. 461 However, the PE must learn the Root Bridge MAC address and Bridge 462 Priority of the root of the Internal Spanning Tree (IST) by 463 listening to the BPDUs. The ESI Value is constructed as follows: 465 - Root Bridge MAC address (6 octets). The Root Bridge MAC 466 address MUST be encoded in the high-order 6 octets of the ESI 467 Value field. 469 - Root Bridge Priority (2 octets). The CE Root Bridge Priority 470 MUST be encoded in the 2 octets next to the Root Bridge MAC 471 address. 473 - The remaining octet will be set to 0x00. 475 This mechanism could be used only if it produces ESIs that satisfy 476 the uniqueness requirement specified above. 478 * Type 3 (T=0x03) - This type indicates a MAC-based ESI Value that 479 can be auto-generated or configured by the operator. The ESI 480 Value is constructed as follows: 482 - System MAC address (6 octets). The PE MAC address MUST be 483 encoded in the high-order 6 octets of the ESI Value field. 485 - Local Discriminator value (3 octets). The Local Discriminator 486 value MUST be encoded in the low-order 3 octets of the ESI 487 Value. 489 This mechanism could be used only if it produces ESIs that satisfy 490 the uniqueness requirement specified above. 492 * Type 4 (T=0x04) - This type indicates a router-ID ESI Value that 493 can be auto-generated or configured by the operator. The ESI 494 Value is constructed as follows: 496 - Router ID (4 octets). The system router ID MUST be encoded in 497 the high-order 4 octets of the ESI Value field. 499 - Local Discriminator value (4 octets). The Local Discriminator 500 value MUST be encoded in the 4 octets next to the IP address. 502 - The low-order octet of the ESI Value will be set to 0x00. 504 This mechanism could be used only if it produces ESIs that satisfy 505 the uniqueness requirement specified above. 507 * Type 5 (T=0x05) - This type indicates an Autonomous System 508 (AS)-based ESI Value that can be auto-generated or configured by 509 the operator. The ESI Value is constructed as follows: 511 - AS number (4 octets). This is an AS number owned by the system 512 and MUST be encoded in the high-order 4 octets of the ESI Value 513 field. If a 2-octet AS number is used, the high-order extra 514 octets will be 0x0000. 516 - Local Discriminator value (4 octets). The Local Discriminator 517 value MUST be encoded in the 4 octets next to the AS number. 519 - The low-order octet of the ESI Value will be set to 0x00. 521 This mechanism could be used only if it produces ESIs that satisfy 522 the uniqueness requirement specified above. 524 Note that a CE always sends packets belonging to a specific flow 525 using a single link towards a PE. For instance, if the CE is a host, 526 then, as mentioned earlier, the host treats the multiple links that 527 it uses to reach the PEs as a Link Aggregation Group (LAG). The CE 528 employs a local hashing function to map traffic flows onto links in 529 the LAG. 531 If a bridged network is multihomed to more than one PE in an EVPN 532 network via switches, then the support of All-Active redundancy mode 533 requires the bridged network to be connected to two or more PEs using 534 a LAG. 536 If a bridged network does not connect to the PEs using a LAG, then 537 only one of the links between the bridged network and the PEs must be 538 the active link for a given . In this case, the set of 539 Ethernet A-D per ES routes advertised by each PE MUST have the 540 "Single-Active" bit in the flags of the ESI Label extended community 541 set to 1. 543 6. Ethernet Tag ID 545 An Ethernet Tag ID is a 32-bit field containing either a 12-bit or 546 24-bit identifier that identifies a particular broadcast domain 547 (e.g., a VLAN) in an EVPN instance. The 12-bit identifier is called 548 the VLAN ID (VID). An EVPN instance consists of one or more 549 broadcast domains (one or more VLANs). VLANs are assigned to a given 550 EVPN instance by the provider of the EVPN service. A given VLAN can 551 itself be represented by multiple VIDs. In such cases, the PEs 552 participating in that VLAN for a given EVPN instance are responsible 553 for performing VLAN ID translation to/from locally attached CE 554 devices. 556 The following subsections discuss the relationship between broadcast 557 domains (e.g., VLANs), Ethernet Tag IDs (e.g., VIDs), and MAC-VRFs as 558 well as the setting of the Ethernet Tag ID, in the various EVPN BGP 559 routes (defined in Section 8), for the different types of service 560 interfaces described in [RFC7209]. 562 The following Ethernet Tag ID value is reserved: 564 * Ethernet Tag ID {0xFFFFFFFF} is known as MAX-ET. 566 6.1. VLAN-Based Service Interface 568 With this service interface, an EVPN instance consists of only a 569 single broadcast domain (e.g., a single VLAN). Therefore, there is a 570 one-to-one mapping between a VID on this interface and a MAC-VRF. 571 Since a MAC-VRF corresponds to a single VLAN, it consists of a single 572 bridge table corresponding to that VLAN. If the VLAN is represented 573 by multiple VIDs (e.g., a different VID per Ethernet segment per PE), 574 then each PE needs to perform VID translation for frames destined to 575 its Ethernet segment(s). In such scenarios, the Ethernet frames 576 transported over an MPLS/IP network SHOULD remain tagged with the 577 originating VID, and a VID translation MUST be supported in the data 578 path and MUST be performed on the disposition PE. The Ethernet Tag 579 ID in all EVPN routes MUST be set to 0. 581 6.2. VLAN Bundle Service Interface 583 With this service interface, an EVPN instance corresponds to multiple 584 broadcast domains (e.g., multiple VLANs); however, only a single 585 bridge table is maintained per MAC-VRF, which means multiple VLANs 586 share the same bridge table. This implies that MAC addresses MUST be 587 unique across all VLANs for that EVI in order for this service to 588 work. In other words, there is a many-to-one mapping between VLANs 589 and a MAC-VRF, and the MAC-VRF consists of a single bridge table. 590 Furthermore, a single VLAN must be represented by a single VID -- 591 e.g., no VID translation is allowed for this service interface type. 592 The MPLS-encapsulated frames MUST remain tagged with the originating 593 VID. Tag translation is NOT permitted. The Ethernet Tag ID in all 594 EVPN routes MUST be set to 0. 596 6.2.1. Port-Based Service Interface 598 This service interface is a special case of the VLAN bundle service 599 interface, where all of the VLANs on the port are part of the same 600 service and map to the same bundle. The procedures are identical to 601 those described in Section 6.2. 603 6.3. VLAN-Aware Bundle Service Interface 605 With this service interface, an EVPN instance consists of multiple 606 broadcast domains (e.g., multiple VLANs) with each VLAN having its 607 own bridge table -- i.e., multiple bridge tables (one per VLAN) are 608 maintained by a single MAC-VRF corresponding to the EVPN instance. 610 Broadcast, unknown unicast, or multicast (BUM) traffic is sent only 611 to the CEs in a given broadcast domain; however, the broadcast 612 domains within an EVI either MAY each have their own P-Tunnel or MAY 613 share P-Tunnels -- e.g., all of the broadcast domains in an EVI MAY 614 share a single P-Tunnel. 616 In the case where a single VLAN is represented by a single VID and 617 thus no VID translation is required, an MPLS-encapsulated packet MUST 618 carry that VID. The Ethernet Tag ID in all EVPN routes MUST be set 619 to that VID. The advertising PE MAY advertise the MPLS Label1 in the 620 MAC/IP Advertisement route representing ONLY the EVI or representing 621 both the Ethernet Tag ID and the EVI. This decision is only a local 622 matter by the advertising PE (which is also the disposition PE) and 623 doesn't affect any other PEs. 625 In the case where a single VLAN is represented by different VIDs on 626 different CEs and thus VID translation is required, a normalized 627 Ethernet Tag ID (VID) MUST be carried in the EVPN BGP routes. 628 Furthermore, the advertising PE advertises the MPLS Label1 in the 629 MAC/IP Advertisement route representing both the Ethernet Tag ID and 630 the EVI, so that upon receiving an MPLS-encapsulated packet, it can 631 identify the corresponding bridge table from the MPLS EVPN label and 632 perform Ethernet Tag ID translation ONLY at the disposition PE -- 633 i.e., the Ethernet frames transported over the MPLS/IP network MUST 634 remain tagged with the originating VID, and VID translation is 635 performed on the disposition PE. The Ethernet Tag ID in all EVPN 636 routes MUST be set to the normalized Ethernet Tag ID assigned by the 637 EVPN provider. 639 6.3.1. Port-Based VLAN-Aware Service Interface 641 This service interface is a special case of the VLAN-aware bundle 642 service interface, where all of the VLANs on the port are part of the 643 same service and are mapped to a single bundle but without any VID 644 translation. The procedures are a subset of those described in 645 Section 6.3. 647 6.4. EVPN PE Model 649 Since this document discusses EVPN operation in relationship to MAC- 650 VRF, EVI, Broadcast Domain (BD), and Bridge Table (BT), it is 651 important to understand the relationship between these terms. 652 Therefore, the following PE model is depicted below to illustrate the 653 relationship among them. 655 +--------------------------------------------------+ 656 | | 657 | +------------------+ EVPN PE | 658 | Attachment | +------------------+ | 659 | Circuit(AC1) | | +----------+ | MPLS/NVO tnl 660 ----------------------*Bridge | | +----- 661 | | | |Table(BT1)| | / \ \ 662 | | | | |<------------------> |Eth| 663 | | | | VLAN x | | \ / / 664 | | | +----------+ | +----- 665 | | | ... | | 666 | | | +----------+ | MPLS/NVO tnl 667 | | | |Bridge | | +----- 668 | | | |Table(BT2)| | / \ \ 669 | | | | |<-------------------> |Eth| 670 ----------------------* VLAN y | | \ / / 671 | AC2 | | +----------+ | +----- 672 | | | MAC-VRF1 | | 673 | +-+ RD1/RT1 | | 674 | +------------------+ | 675 | | 676 | | 677 +---------------------------------------------------+ 679 Figure 1: EVPN PE Model 681 A tenant configured for an EVPN service instance (i.e, EVI) on a PE, 682 is instantiated by a single MAC Virtual Routing and Forwarding table 683 (MAC-VRF) on that PE. A MAC-VRF consists of one or more Bridge 684 Tables (BTs) where each BT corresponds to a VLAN (broadcast domain - 685 BD). If a service interface for an EVPN PE is configured in VLAN- 686 Based mode (i.e., section 6.1), then there is only a single BT per 687 MAC-VRF (per EVI) - i.e., there is only one tenant VLAN per EVI. 688 However, if a service interface for an EVPN PE is configured in VLAN- 689 Aware Bundle mode (i.e., section 6.3), then there are several BTs per 690 MAC-VRF (per EVI) - i.e., there are several tenant VLANs per EVI. 691 The relationship among these terms can be summarized as follow: 693 * An EVI consists of one or more BDs and a MAC-VRF consists of one 694 or more BTs, one for each BD. A BD is identified by an Ethernet 695 Tag ID which is typically represented by a single VLAN ID (VID); 696 however, it can be represented by multiple VIDs (i.e., Shared VLAN 697 Learning (SVL) mode in 802.1Q). 699 * In VLAN-based mode, there is one EVI per VLAN and thus one BD/BT 700 per VLAN. Furthermore, there is one BT per MAC-VRF. 702 * In VLAN-bundle service, it can be considered as analogous to SVL 703 mode in 802.1Q i.e., one BD per EVI and one BT per MAC-VRF with 704 multiple VIDs representing that BD. 706 * In VLAN-aware bundle service, there is one EVI with multiple BDs 707 where each BD is represented by a VLAN. Furthermore, there are 708 multiple BTs in a single MAC-VRF. 710 Since a single tenant subnet is typically (and in this document) 711 represented by a VLAN (and thus supported by a single BT), for a 712 given tenant there are as many BTs as there are subnets as shown in 713 the PE model above. 715 MAC-VRF is identified by its corresponding route target and route 716 distinguisher. If operating in EVPN VLAN-Based mode, then a 717 receiving PE that receives an EVPN route with MAC-VRF route target 718 can identify the corresponding BT; however, if operating in EVPN 719 VLAN-Aware Bundle mode, then the receiving PE needs both the MAC-VRF 720 route target and Ethernet Tag ID in order to identify the 721 corresponding BT. 723 7. BGP EVPN Routes 725 This document defines a new BGP Network Layer Reachability 726 Information (NLRI) called the EVPN NLRI. 728 The format of the EVPN NLRI is as follows: 730 +-----------------------------------+ 731 | Route Type (1 octet) | 732 +-----------------------------------+ 733 | Length (1 octet) | 734 +-----------------------------------+ 735 | Route Type specific (variable) | 736 +-----------------------------------+ 738 The Route Type field defines the encoding of the rest of the EVPN 739 NLRI (Route Type specific EVPN NLRI). 741 The Length field indicates the length in octets of the Route Type 742 specific field of the EVPN NLRI. 744 This document defines the following Route Types: 746 + 1 - Ethernet Auto-Discovery (A-D) route 747 + 2 - MAC/IP Advertisement route 748 + 3 - Inclusive Multicast Ethernet Tag route 749 + 4 - Ethernet Segment route 751 The detailed encoding and procedures for these route types are 752 described in subsequent sections. 754 The EVPN NLRI is carried in BGP [RFC4271] using BGP Multiprotocol 755 Extensions [RFC4760] with an Address Family Identifier (AFI) of 25 756 (L2VPN) and a Subsequent Address Family Identifier (SAFI) of 70 757 (EVPN). The NLRI field in the MP_REACH_NLRI/MP_UNREACH_NLRI 758 attribute contains the EVPN NLRI (encoded as specified above). 760 In order for two BGP speakers to exchange labeled EVPN NLRI, they 761 must use BGP Capabilities Advertisements to ensure that they both are 762 capable of properly processing such NLRI. This is done as specified 763 in [RFC4760], by using capability code 1 (multiprotocol BGP) with an 764 AFI of 25 (L2VPN) and a SAFI of 70 (EVPN). 766 7.1. Ethernet Auto-discovery Route 768 An Ethernet A-D route type specific EVPN NLRI consists of the 769 following: 771 +---------------------------------------+ 772 | Route Distinguisher (RD) (8 octets) | 773 +---------------------------------------+ 774 |Ethernet Segment Identifier (10 octets)| 775 +---------------------------------------+ 776 | Ethernet Tag ID (4 octets) | 777 +---------------------------------------+ 778 | MPLS Label (3 octets) | 779 +---------------------------------------+ 781 For the purpose of BGP route key processing, only the Ethernet 782 Segment Identifier and the Ethernet Tag ID are considered to be part 783 of the prefix in the NLRI. The MPLS Label field is to be treated as 784 a route attribute as opposed to being part of the route. 786 The MPLS Label field is encoded as 3 octets, where the high-order 787 20 bits contain the label value. 789 For procedures and usage of this route, please see Sections 8.2 790 ("Fast Convergence") and 8.4 ("Aliasing and Backup Path"). 792 7.2. MAC/IP Advertisement Route 794 A MAC/IP Advertisement route type specific EVPN NLRI consists of the 795 following: 797 +---------------------------------------+ 798 | RD (8 octets) | 799 +---------------------------------------+ 800 |Ethernet Segment Identifier (10 octets)| 801 +---------------------------------------+ 802 | Ethernet Tag ID (4 octets) | 803 +---------------------------------------+ 804 | MAC Address Length (1 octet) | 805 +---------------------------------------+ 806 | MAC Address (6 octets) | 807 +---------------------------------------+ 808 | IP Address Length (1 octet) | 809 +---------------------------------------+ 810 | IP Address (0, 4, or 16 octets) | 811 +---------------------------------------+ 812 | MPLS Label1 (3 octets) | 813 +---------------------------------------+ 814 | MPLS Label2 (0 or 3 octets) | 815 +---------------------------------------+ 817 For the purpose of BGP route key processing, only the Ethernet Tag 818 ID, MAC Address Length, MAC Address, IP Address Length, and IP 819 Address fields are considered to be part of the prefix in the NLRI. 820 The Ethernet Segment Identifier, MPLS Label1, and MPLS Label2 fields 821 are to be treated as route attributes as opposed to being part of the 822 "route". Both the IP and MAC address lengths are in bits. 824 The MPLS Label1 and MPLS Label2 fields are encoded as 3 octets, where 825 the high-order 20 bits contain the label value. 827 For procedures and usage of this route, please see Sections 9 828 ("Determining Reachability to Unicast MAC Addresses") and 14 ("Load 829 Balancing of Unicast Packets"). 831 7.3. Inclusive Multicast Ethernet Tag Route 833 An Inclusive Multicast Ethernet Tag route type specific EVPN NLRI 834 consists of the following: 836 +---------------------------------------+ 837 | RD (8 octets) | 838 +---------------------------------------+ 839 | Ethernet Tag ID (4 octets) | 840 +---------------------------------------+ 841 | IP Address Length (1 octet) | 842 +---------------------------------------+ 843 | Originating Router's IP Address | 844 | (4 or 16 octets) | 845 +---------------------------------------+ 847 For procedures and usage of this route, please see Sections 11 848 ("Handling of Multi-destination Traffic"), 12 ("Processing of Unknown 849 Unicast Packets"), and 16 ("Multicast and Broadcast"). The IP 850 address length is in bits. For the purpose of BGP route key 851 processing, only the Ethernet Tag ID, IP Address Length, and 852 Originating Router's IP Address fields are considered to be part of 853 the prefix in the NLRI. 855 7.4. Ethernet Segment Route 857 An Ethernet Segment route type specific EVPN NLRI consists of the 858 following: 860 +---------------------------------------+ 861 | RD (8 octets) | 862 +---------------------------------------+ 863 |Ethernet Segment Identifier (10 octets)| 864 +---------------------------------------+ 865 | IP Address Length (1 octet) | 866 +---------------------------------------+ 867 | Originating Router's IP Address | 868 | (4 or 16 octets) | 869 +---------------------------------------+ 871 For procedures and usage of this route, please see Section 8.5 872 ("Designated Forwarder Election"). The IP address length is in bits. 873 For the purpose of BGP route key processing, only the Ethernet 874 Segment ID, IP Address Length, and Originating Router's IP Address 875 fields are considered to be part of the prefix in the NLRI. 877 7.5. ESI Label Extended Community 879 This Extended Community is a new transitive Extended Community having 880 a Type field value of 0x06 and the Sub-Type 0x01. It may be 881 advertised along with Ethernet Auto-discovery routes, and it enables 882 split-horizon procedures for multihomed sites as described in 883 Section 8.3 ("Split Horizon"). The ESI Label field represents an ES 884 by the advertising PE, and it is used in split-horizon filtering by 885 other PEs that are connected to the same multihomed Ethernet segment. 887 The ESI Label field is encoded as 3 octets, where the high-order 888 20 bits contain the label value. 890 The ESI label value MAY be zero if no split-horizon filtering 891 procedures are required in any of the VLANs of the Ethernet Segment. 892 This is the case in [RFC8214] or Ethernet Segments using Local Bias 893 procedures in [I-D.ietf-bess-evpn-mh-split-horizon]. 895 Each ESI Label extended community is encoded as an 8-octet value, as 896 follows: 898 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 899 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 900 | Type=0x06 | Sub-Type=0x01 | Flags(1 octet)| Reserved=0 | 901 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 902 | Reserved=0 | ESI Label | 903 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 905 This document creates an IANA registry called "EVPN ESI Multihoming 906 Attributes" (Section 21, where the following bits in Flags octet are 907 initially defined: 909 0 1 2 3 4 5 6 7 910 +-+-+-+-+-+-+-+-+ 911 | MBZ |RED| (MBZ = MUST Be Zero) 912 +-+-+-+-+-+-+-+-+ 914 Name Meaning 915 --------------------------------------------------------------- 916 RED Multihomed site redundancy mode 918 Multihomed site redundancy mode: 920 RED = 00: A value of 00 means that the multihomed site is operating 921 in All-Active redundancy mode. 923 RED = 01: A value of 01 means that the multihomed site is operating 924 in Single-Active redundancy mode. 926 7.6. ES-Import Route Target 928 This is a new transitive Route Target extended community carried with 929 the Ethernet Segment route. When used, it enables all the PEs 930 connected to the same multihomed site to import the Ethernet Segment 931 routes. 933 * The value MAY be derived automatically for ESI Type 0 by encoding 934 the high-order 6-octet portion of the 9-octet ESI Value, which 935 corresponds to part of the arbitrary value configured, in the ES- 936 Import Route Target. 938 * The value is derived automatically for ESI Types 1, 2, and 3, by 939 encoding the high-order 6-octet portion of the 9-octet ESI Value, 940 which corresponds to a MAC address, in the ES-Import Route Target. 942 * The value MAY be derived automatically for ESI Types 4 and 5, by 943 encoding the high-order 6-octet portion of the 9-octet ESI Value, 944 which corresponds to a Router ID or AS number (4-octets) 945 respectively, and 2-octets of Local Discriminator, in the 946 ES-Import Route Target. 948 The format of this Extended Community is as follows: 950 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 951 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 952 | Type=0x06 | Sub-Type=0x02 | ES-Import ~ 953 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 954 ~ ES-Import Cont'd | 955 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 957 This document expands the definition of the Route Target extended 958 community to allow the value of the high-order octet (Type field) to 959 be 0x06 (in addition to the values specified in [RFC4360]). The 960 low-order octet (Sub-Type field) value 0x02 indicates that this 961 Extended Community is of type "Route Target". The new Type field 962 value 0x06 indicates that the structure of this RT is a 6-octet value 963 (e.g., a MAC address). A BGP speaker that implements RT Constraint 964 [RFC4684] MUST apply the RT Constraint procedures to the ES-Import RT 965 as well. 967 For procedures and usage of this extended community, please see 968 Section 8.1 ("Multihomed Ethernet Segment Auto-discovery"). 970 7.7. MAC Mobility Extended Community 972 This Extended Community is a new transitive Extended Community having 973 a Type field value of 0x06 and the Sub-Type 0x00. It may be 974 advertised along with MAC/IP Advertisement routes. The procedures 975 for using this extended community are described in Section 15 ("MAC 976 Mobility"). 978 The MAC Mobility extended community is encoded as an 8-octet value, 979 as follows: 981 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 982 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 983 | Type=0x06 | Sub-Type=0x00 |Flags(1 octet)| Reserved=0 | 984 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 985 | Sequence Number | 986 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 988 The low-order bit of the Flags octet is defined as the 989 "Sticky/static" flag and may be set to 1. A value of 1 means that 990 the MAC address is static and cannot move. The sequence number is 991 used to ensure that PEs retain the correct MAC/IP Advertisement route 992 when multiple updates occur for the same MAC address. 994 7.8. Default Gateway Extended Community 996 The Default Gateway community is an Extended Community of an Opaque 997 Type (see Section 3.3 of [RFC4360]). It is a transitive community, 998 which means that the first octet is 0x03. The value of the second 999 octet (Sub-Type) is 0x0d (Default Gateway) as assigned by IANA. The 1000 Value field of this community is reserved (set to 0 by the senders, 1001 ignored by the receivers). For procedures and usage of this extended 1002 community, please see Section 10.1 ("Default Gateway"). 1004 7.9. Route Distinguisher Assignment per MAC-VRF 1006 The Route Distinguisher (RD) MUST be set to the RD of the MAC-VRF 1007 that is advertising the NLRI. An RD MUST be assigned for a given 1008 MAC-VRF on a PE. This RD MUST be unique across all MAC-VRFs on a PE. 1009 It is RECOMMENDED to use the Type 1 RD [RFC4364]. The value field 1010 comprises an IP address of the PE (typically, the loopback address) 1011 followed by a number unique to the PE. This number may be generated 1012 by the PE. In case of VLAN-based or VLAN Bundle services, this 1013 number may also be generated out of the Ethernet Tag ID for the BD as 1014 long as the value does not exceed a length of 16 bits. Or, in the 1015 Unique VLAN EVPN case, the low-order 12 bits may be the 12-bit VLAN 1016 ID, with the remaining high-order 4 bits set to 0. 1018 7.10. Route Targets 1020 The EVPN route MAY carry one or more Route Target (RT) extended 1021 communities. RTs may be configured (as in IP VPNs) or may be derived 1022 automatically. 1024 If a PE uses RT Constraint, the PE advertises all such RTs using RT 1025 Constraints per [RFC4684]. The use of RT Constraints allows each 1026 EVPN route to reach only those PEs that are configured to import at 1027 least one RT from the set of RTs carried in the EVPN route. 1029 7.10.1. Auto-derivation from the Ethernet Tag (VLAN ID) 1031 For the "Unique VLAN EVPN" scenario (Section 4), it is highly 1032 desirable to auto-derive the RT from the Ethernet Tag (VLAN ID). The 1033 procedure for performing such auto-derivation is as follows: 1035 * The Global Administrator field of the RT MUST be set to the 1036 Autonomous System (AS) number with which the PE is associated. 1038 * The 12-bit VLAN ID MUST be encoded in the lowest 12 bits of the 1039 Local Administrator field, with the remaining bits set to zero. 1041 For VLAN-based and VLAN Bundle services, the RT may also be auto- 1042 derived as per the above rules but replacing the 12-bit VLAN ID with 1043 a 16-bit Ethernet Tag ID configured for the BD. If the Ethernet Tag 1044 ID length is 24 bits, the RT for the MAC-VRF can be auto-derived as 1045 per [RFC8365] section 5.1.2.1. 1047 7.11. EVPN Layer 2 Attributes Extended Community 1049 [RFC8214] defines this extended community ("L2-Attr"), to be included 1050 with per-EVI Ethernet A-D routes and mandatory if multihoming is 1051 enabled. 1053 Usage and applicability of this Extended community to Bridging is 1054 clarified here. 1056 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1057 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1058 | MBZ |RSV|RSV|F|C|P|B| (MBZ = MUST Be Zero) 1059 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1061 The following bits in Control Flags from [RFC8214] are listed here 1062 for completeness only: 1064 Name Meaning 1065 --------------------------------------------------------------- 1066 P If set to 1 in multihoming Single-Active scenarios, 1067 this flag indicates that the advertising PE is the 1068 primary PE. MUST be set to 1 for multihoming 1069 All-Active scenarios by all active PE(s). 1071 B If set to 1 in multihoming Single-Active scenarios, 1072 this flag indicates that the advertising PE is the 1073 backup PE. 1075 C If set to 1, a control word [RFC4448] MUST be present 1076 when sending EVPN packets to this PE. It is 1077 recommended that the control word be included in the 1078 absence of an entropy label [RFC6790]. 1080 The bits in Control Flags are extended by the following defined bits: 1082 Name Meaning 1083 --------------------------------------------------------------- 1084 F If set to 1, a Flow Label MUST be present 1085 when sending EVPN packets to this PE. 1086 If set to 0, a Flow Label MUST NOT be present 1087 when sending EVPN packets to this PE. 1089 For procedures and usage of this extended community, with respect to 1090 Control Word and Flow Label, please see Section 18. ("Frame 1091 Ordering"). 1093 For procedures and usage of this extended community, with respect to 1094 Primary-Backup bits, please see Section 8.5. ("Designated Forwarder 1095 Election"). 1097 7.11.1. EVPN Layer 2 Attributes Partitioning 1099 The information carried in the L2-Attr Extended Community may be ESI- 1100 specific or BD/MAC-VRF-specific. In order to minimize the processing 1101 overhead of configuration-time items such as MTU not expected to 1102 change at runtime based on failures, the Extended Community from 1103 [RFC8214] is partitioned, with a subset of information carried over 1104 each Ethernet A-D per EVI and Inclusive Multicast routes. 1106 The EVPN Layer 2 Attributes Extended Community, when added to 1107 Inclusive Multicast route: 1109 * BD/MAC-VRF attributes MTU, Control Word and Flow Label are 1110 conveyed, and; 1112 * per-ESI attributes P, B MUST be zero. 1114 +-------------------------------------------+ 1115 | Type (0x06) / Sub-type (0x04) (2 octets) | 1116 +-------------------------------------------+ 1117 | Control Flags (2 octets) | 1118 +-------------------------------------------+ 1119 | L2 MTU (2 octets) | 1120 +-------------------------------------------+ 1121 | Reserved (2 octets) | 1122 +-------------------------------------------+ 1124 1 1 1 1 1 1125 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1126 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1127 | MBZ | MBZ |F|C|MBZ| (MBZ = MUST Be Zero) 1128 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1130 The EVPN Layer 2 Attributes Extended Community is included on 1131 Ethernet A-D per EVI route and: 1133 * per-ESI attributes P, B are conveyed, and; 1135 * BD/MAC-VRF attributes MTU, Control Word and Flow Label MUST be 1136 zero. 1138 +-------------------------------------------+ 1139 | Type (0x06) / Sub-type (0x04) (2 octets) | 1140 +-------------------------------------------+ 1141 | Control Flags (2 octets) | 1142 +-------------------------------------------+ 1143 | MBZ (2 octets) | 1144 +-------------------------------------------+ 1145 | Reserved (2 octets) | 1146 +-------------------------------------------+ 1148 1 1 1 1 1 1149 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1150 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1151 | MBZ | MBZ |P|B| (MBZ = MUST Be Zero) 1152 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1154 Note that in both of the above cases, the values conveyed in this 1155 extended community are at the granularity of an individual EVI (or 1156 [EVI, BD] for VLAN-aware bundle) and hence may vary for different 1157 EVIs. 1159 7.12. Route Prioritization 1161 In order to achieve the Fast Convergence referred to in (Section 8.2 1162 ("Fast Convergence")), BGP speakers SHOULD prioritise advertisement, 1163 processing and redistribution of routes based on relative scale of 1164 priority vs. expected or average scale. 1166 1. Ethernet A-D per ES (Mass-Withdraw Route Type 1) and Ethernet 1167 Segment (Route Type 4) are lower scale and highly convergence 1168 affecting, and SHOULD be handled in first order of priority 1170 2. Ethernet A-D per EVI, Inclusive Multicast Ethernet Tag route, and 1171 IP Prefix route defined in [RFC9136] are sent for each Bridge or 1172 AC at medium scale and may be convergence affecting, and SHOULD 1173 be handled in second order of priority 1175 3. MAC advertisement route (zero and nonzero IP portion), Multicast 1176 Join Sync and Multicast Leave Sync routes defined in 1177 [I-D.ietf-bess-evpn-igmp-mld-proxy] are considered 'individual 1178 routes' and very-high scale or of relatively low importance for 1179 fast convergence and SHOULD be handled in last order of priority. 1181 8. Multihoming Functions 1183 This section discusses the functions, procedures, and associated BGP 1184 routes used to support multihoming in EVPN. This covers both 1185 multihomed device (MHD) and multihomed network (MHN) scenarios. 1187 8.1. Multihomed Ethernet Segment Auto-discovery 1189 PEs connected to the same Ethernet segment can automatically discover 1190 each other with minimal to no configuration through the exchange of 1191 the Ethernet Segment route. 1193 8.1.1. Constructing the Ethernet Segment Route 1195 The Route Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The 1196 value field comprises an IP address of the PE (typically, the 1197 loopback address) followed by a number unique to the PE. 1199 The Ethernet Segment Identifier (ESI) MUST be set to the 10-octet 1200 value described in Section 5. 1202 The BGP advertisement that advertises the Ethernet Segment route MUST 1203 also carry an ES-Import Route Target, as defined in Section 7.6. 1205 The Ethernet Segment route filtering MUST be done such that the 1206 Ethernet Segment route is imported only by the PEs that are 1207 multihomed to the same Ethernet segment. To that end, each PE that 1208 is connected to a particular Ethernet segment constructs an import 1209 filtering rule to import a route that carries the ES-Import Route 1210 Target, constructed from the ESI. 1212 8.2. Fast Convergence 1214 In EVPN, MAC address reachability is learned via the BGP control 1215 plane over the MPLS network. As such, in the absence of any fast 1216 protection mechanism, the network convergence time is a function of 1217 the number of MAC/IP Advertisement routes that must be withdrawn by 1218 the PE encountering a failure. For highly scaled environments, this 1219 scheme yields slow convergence. 1221 To alleviate this, EVPN defines a mechanism to efficiently and 1222 quickly signal, to remote PE nodes, the need to update their 1223 forwarding tables upon the occurrence of a failure in connectivity to 1224 an Ethernet segment. This is done by having each PE advertise a set 1225 of one or more Ethernet A-D per ES routes for each locally attached 1226 Ethernet segment (refer to Section 8.2.1 below for details on how 1227 these routes are constructed). A PE may need to advertise more than 1228 one Ethernet A-D per ES route for a given ES because the ES may be in 1229 a multiplicity of EVIs and the RTs for all of these EVIs may not fit 1230 into a single route. Advertising a set of Ethernet A-D per ES routes 1231 for the ES allows each route to contain a subset of the complete set 1232 of RTs. Each Ethernet A-D per ES route is differentiated from the 1233 other routes in the set by a different Route Distinguisher (RD). 1235 Upon a failure in connectivity to the attached segment, the PE 1236 withdraws the corresponding set of Ethernet A-D per ES routes. This 1237 triggers all PEs that receive the withdrawal to update their next-hop 1238 adjacencies for all MAC addresses associated with the Ethernet 1239 segment in question. If no other PE had advertised an Ethernet A-D 1240 per ES route for the same segment, then the PE that received the 1241 withdrawal simply invalidates the MAC entries for that segment. 1242 Otherwise, the PE updates its next-hop adjacencies accordingly. 1244 8.2.1. Constructing Ethernet A-D per Ethernet Segment Route 1246 This section describes the procedures used to construct the Ethernet 1247 A-D per ES route, which is used for fast convergence (as discussed 1248 above) and for advertising the ESI label used for split-horizon 1249 filtering (as discussed in Section 8.3). Support of this route is 1250 REQUIRED. 1252 The Route Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The 1253 value field comprises an IP address of the PE (typically, the 1254 loopback address) followed by a number unique to the PE. 1256 The Ethernet Segment Identifier MUST be a 10-octet entity as 1257 described in Section 5 ("Ethernet Segment"). The Ethernet A-D route 1258 is not needed when the Segment Identifier is set to 0 (e.g., single- 1259 homed scenarios). An exception to this rule is described in 1260 [RFC8317]. 1262 The Ethernet Tag ID MUST be set to MAX-ET. 1264 The MPLS label in the NLRI MUST be set to 0. 1266 The ESI Label extended community MUST be included in the route. If 1267 All-Active redundancy mode is desired, then the "Single-Active" bit 1268 in the flags of the ESI Label extended community MUST be set to 0 and 1269 the MPLS label in that Extended Community MUST be set to a valid MPLS 1270 label value. The MPLS label in this Extended Community is referred 1271 to as the ESI label and MUST have the same value in each Ethernet A-D 1272 per ES route advertised for the ES. This label MUST be a downstream 1273 assigned MPLS label if the advertising PE is using ingress 1274 replication for receiving multicast, broadcast, or unknown unicast 1275 traffic from other PEs. If the advertising PE is using P2MP MPLS 1276 LSPs for sending multicast, broadcast, or unknown unicast traffic, 1277 then this label MUST be an upstream assigned MPLS label, unless DCB 1278 allocated labels are used. The usage of this label is described in 1279 Section 8.3. 1281 If Single-Active redundancy mode is desired, then the "Single-Active" 1282 bit in the flags of the ESI Label extended community MUST be set to 1 1283 and the ESI label SHOULD be set to a valid MPLS label value. 1285 8.2.1.1. Ethernet A-D Route Targets 1287 Each Ethernet A-D per ES route MUST carry one or more Route Target 1288 (RT) extended communities. The set of Ethernet A-D routes per ES 1289 MUST carry the entire set of RTs for all the EVPN instances to which 1290 the Ethernet segment belongs. 1292 8.3. Split Horizon 1294 Consider a CE that is multihomed to two or more PEs on an Ethernet 1295 segment ES1 operating in All-Active redundancy mode. If the CE sends 1296 a broadcast, unknown unicast, or multicast (BUM) packet to one of the 1297 Non-Designated Forwarder (Non-DF) PEs, say PE1, then PE1 will forward 1298 that packet to all or a subset of the other PEs in that EVPN 1299 instance, including the DF PE for that Ethernet segment. In this 1300 case, the DF PE to which the CE is multihomed MUST drop the packet 1301 and not forward back to the CE. This filtering is referred to as 1302 "split-horizon filtering" in this document. 1304 When a set of PEs are operating in Single-Active redundancy mode, the 1305 use of this split-horizon filtering mechanism is highly recommended 1306 because it prevents transient loops at the time of failure or 1307 recovery that would impact the Ethernet segment -- e.g., when two PEs 1308 think that both are DFs for that segment before the DF election 1309 procedure settles down. 1311 In order to achieve this split-horizon function, every BUM packet 1312 originating from a Non-DF PE is encapsulated with an MPLS label that 1313 identifies the Ethernet segment of origin (i.e., the segment from 1314 which the frame entered the EVPN network). This label is referred to 1315 as the ESI label and MUST be distributed by all PEs when operating in 1316 All-Active redundancy mode using a set of Ethernet A-D per ES routes, 1317 per Section 8.2.1 above. The ESI label SHOULD be distributed by all 1318 PEs when operating in Single-Active redundancy mode using a set of 1319 Ethernet A-D per ES routes. These routes are imported by the PEs 1320 connected to the Ethernet segment and also by the PEs that have at 1321 least one EVPN instance in common with the Ethernet segment in the 1322 route. As described in Section 8.1.1, the route MUST carry an ESI 1323 Label extended community with a valid ESI label. The disposition PE 1324 relies on the value of the ESI label to determine whether or not a 1325 BUM frame is allowed to egress a specific Ethernet segment. 1327 8.3.1. ESI Label Assignment 1329 The following subsections describe the assignment procedures for the 1330 ESI label, which differ depending on the type of tunnels being used 1331 to deliver multi-destination packets in the EVPN network. 1333 8.3.1.1. Ingress Replication 1335 Each PE that operates in All-Active or Single-Active redundancy mode 1336 and that uses ingress replication to receive BUM traffic advertises a 1337 downstream assigned ESI label in the set of Ethernet A-D per ES 1338 routes for its attached ES. This label MUST be programmed in the 1339 platform label space by the advertising PE, and the forwarding entry 1340 for this label must result in NOT forwarding packets received with 1341 this label onto the Ethernet segment for which the label was 1342 distributed. 1344 The rules for the inclusion of the ESI label in a BUM packet by the 1345 ingress PE operating in All-Active redundancy mode are as follows: 1347 * A Non-DF ingress PE MUST include the ESI label distributed by the 1348 DF egress PE in the copy of a BUM packet sent to it. 1350 * An ingress PE (DF or Non-DF) SHOULD include the ESI label 1351 distributed by each Non-DF egress PE in the copy of a BUM packet 1352 sent to it. 1354 The rule for the inclusion of the ESI label in a BUM packet by the 1355 ingress PE operating in Single-Active redundancy mode is as follows: 1357 * An ingress DF PE SHOULD include the ESI label distributed by the 1358 egress PE in the copy of a BUM packet sent to it. 1360 In both All-Active and Single-Active redundancy mode, an ingress PE 1361 MUST NOT include an ESI label in the copy of a BUM packet sent to an 1362 egress PE that is not attached to the ES through which the BUM packet 1363 entered the EVI. 1365 As an example, consider PE1 and PE2, which are multihomed to CE1 on 1366 ES1 and operating in All-Active multihoming mode. Further, consider 1367 that PE1 is using P2P or MP2P LSPs to send packets to PE2. Consider 1368 that PE1 is the Non-DF for VLAN1 and PE2 is the DF for VLAN1, and PE1 1369 receives a BUM packet from CE1 on VLAN1 on ES1. In this scenario, 1370 PE2 distributes an Inclusive Multicast Ethernet Tag route for VLAN1 1371 corresponding to an EVPN instance. So, when PE1 sends a BUM packet 1372 that it receives from CE1, it MUST first push onto the MPLS label 1373 stack the ESI label that PE2 has distributed for ES1. It MUST then 1374 push onto the MPLS label stack the MPLS label distributed by PE2 in 1375 the Inclusive Multicast Ethernet Tag route for VLAN1. The resulting 1376 packet is further encapsulated in the P2P or MP2P LSP label stack 1377 required to transmit the packet to PE2. When PE2 receives this 1378 packet, it determines, from the top MPLS label, the set of ESIs to 1379 which it will replicate the packet after any P2P or MP2P LSP labels 1380 have been removed. If the next label is the ESI label assigned by 1381 PE2 for ES1, then PE2 MUST NOT forward the packet onto ES1. If the 1382 next label is an ESI label that has not been assigned by PE2, then 1383 PE2 MUST drop the packet. It should be noted that in this scenario, 1384 if PE2 receives a BUM packet for VLAN1 from CE1, then it SHOULD 1385 encapsulate the packet with an ESI label received from PE1 when 1386 sending it to PE1 in order to avoid any transient loops during a 1387 failure scenario that would impact ES1 (e.g., port or link failure). 1389 8.3.1.2. P2MP MPLS LSPs 1391 The Non-DF PEs that operate in All-Active redundancy mode and that 1392 use P2MP LSPs to send BUM traffic advertise an upstream assigned ESI 1393 label in the set of Ethernet A-D per ES routes for their common 1394 attached ES. This label is upstream assigned by the PE that 1395 advertises the route. This label MUST be programmed by the other PEs 1396 that are connected to the ESI advertised in the route, in the context 1397 label space for the advertising PE. Further, the forwarding entry 1398 for this label must result in NOT forwarding packets received with 1399 this label onto the Ethernet segment for which the label was 1400 distributed. This label MUST also be programmed by the other PEs 1401 that import the route but are not connected to the ESI advertised in 1402 the route, in the context label space for the advertising PE. 1403 Further, the forwarding entry for this label must be a label pop with 1404 no other associated action. 1406 The DF PE that operates in Single-Active redundancy mode and that 1407 uses P2MP LSPs to send BUM traffic should advertise an upstream 1408 assigned ESI label in the set of Ethernet A-D per ES routes for its 1409 attached ES, just as described in the previous paragraph. 1411 As an example, consider PE1 and PE2, which are multihomed to CE1 on 1412 ES1 and operating in All-Active multihoming mode. Also, consider 1413 that PE3 belongs to one of the EVPN instances of ES1. Further, 1414 assume that PE1, which is the Non-DF, is using P2MP MPLS LSPs to send 1415 BUM packets. When PE1 sends a BUM packet that it receives from CE1, 1416 it MUST first push onto the MPLS label stack the ESI label that it 1417 has assigned for the ESI on which the packet was received. The 1418 resulting packet is further encapsulated in the P2MP MPLS label stack 1419 necessary to transmit the packet to the other PEs. Penultimate hop 1420 popping MUST be disabled on the P2MP LSPs used in the MPLS transport 1421 infrastructure for EVPN. When PE2 receives this packet, it 1422 decapsulates the top MPLS label and forwards the packet using the 1423 context label space determined by the top label. If the next label 1424 is the ESI label assigned by PE1 to ES1, then PE2 MUST NOT forward 1425 the packet onto ES1. When PE3 receives this packet, it decapsulates 1426 the top MPLS label and forwards the packet using the context label 1427 space determined by the top label. If the next label is the ESI 1428 label assigned by PE1 to ES1 and PE3 is not connected to ES1, then 1429 PE3 MUST pop the label and flood the packet over all local ESIs in 1430 that EVPN instance. It should be noted that when PE2 sends a BUM 1431 frame over a P2MP LSP, it should encapsulate the frame with an ESI 1432 label even though it is the DF for that VLAN, in order to avoid any 1433 transient loops during a failure scenario that would impact ES1 1434 (e.g., port or link failure). 1436 8.3.1.3. MP2MP MPLS LSPs 1438 The procedures for MP2MP tunnels follow Section 8.3.1.2, with the 1439 exceptions described in this section. 1441 When MP2MP tunnels are used, ESI-labels MUST be allocated from a DCB 1442 and the same label must be used by all the PEs attached to the same 1443 Ethernet Segment. 1445 In that way, any egress PE with local Ethernet Segments can identify 1446 the source ES of the received BUM packets. 1448 8.4. Aliasing and Backup Path 1450 In the case where a CE is multihomed to multiple PE nodes, using a 1451 Link Aggregation Group (LAG) with All-Active redundancy, it is 1452 possible that only a single PE learns a set of the MAC addresses 1453 associated with traffic transmitted by the CE. This leads to a 1454 situation where remote PE nodes receive MAC/IP Advertisement routes 1455 for these addresses from a single PE, even though multiple PEs are 1456 connected to the multihomed segment. As a result, the remote PEs are 1457 not able to effectively load balance traffic among the PE nodes 1458 connected to the multihomed Ethernet segment. This could be the 1459 case, for example, when the PEs perform data-plane learning on the 1460 access, and the load-balancing function on the CE hashes traffic from 1461 a given source MAC address to a single PE. 1463 Another scenario where this occurs is when the PEs rely on control- 1464 plane learning on the access (e.g., using ARP), since ARP traffic 1465 will be hashed to a single link in the LAG. 1467 To address this issue, EVPN introduces the concept of 'aliasing', 1468 which is the ability of a PE to signal that it has reachability to an 1469 EVPN instance on a given ES even when it has learned no MAC addresses 1470 from that EVI/ES. The Ethernet A-D per EVI route is used for this 1471 purpose. A remote PE that receives a MAC/IP Advertisement route with 1472 a non-reserved ESI SHOULD consider the advertised MAC address to be 1473 reachable via all PEs that have advertised reachability to that MAC 1474 address's EVI/ES/Ethernet Tag ID via the combination of an Ethernet 1475 A-D per EVI route for that EVI/ES/Ethernet Tag ID AND Ethernet A-D 1476 per ES routes for that ES with the "Single-Active" bit in the flags 1477 of the ESI Label extended community set to 0. 1479 Note that the Ethernet A-D per EVI route may be received by a remote 1480 PE before it receives the set of Ethernet A-D per ES routes. 1481 Therefore, in order to handle corner cases and race conditions, the 1482 Ethernet A-D per EVI route MUST NOT be used for traffic forwarding by 1483 a remote PE until it also receives the associated set of Ethernet A-D 1484 per ES routes. 1486 The backup path is a closely related function, but it is used in 1487 Single-Active redundancy mode. In this case, a PE also advertises 1488 that it has reachability to a given EVI/ES using the same combination 1489 of Ethernet A-D per EVI route and Ethernet A-D per ES route as 1490 discussed above, but with the "Single-Active" bit in the flags of the 1491 ESI Label extended community set to 1. A remote PE that receives a 1492 MAC/IP Advertisement route with a non-reserved ESI SHOULD consider 1493 the advertised MAC address to be reachable via any PE that has 1494 advertised this combination of Ethernet A-D routes, and it SHOULD 1495 install a backup path for that MAC address. 1497 Please see Section 14.1.1 for a description of the backup paths 1498 operation. 1500 8.4.1. Constructing Ethernet A-D per EVPN Instance Route 1502 This section describes the procedures used to construct the Ethernet 1503 A-D per EVPN instance (EVI) route, which is used for aliasing (as 1504 discussed above). Support of this route is OPTIONAL. 1506 The Route Distinguisher (RD) MUST be set per Section 7.9. 1508 The Ethernet Segment Identifier MUST be a 10-octet entity as 1509 described in Section 5 ("Ethernet Segment"). The Ethernet A-D route 1510 is not needed when the Segment Identifier is set to 0. 1512 The Ethernet Tag ID is set as defined in Section 6. 1514 Note that the above allows the Ethernet A-D per EVI route to be 1515 advertised with one of the following granularities: 1517 * One Ethernet A-D route per tuple per 1518 MAC-VRF. This is applicable when the PE uses MPLS-based 1519 disposition with VID translation or may be applicable when the PE 1520 uses MAC-based disposition with VID translation. 1522 * One Ethernet A-D route for each per MAC-VRF (where the 1523 Ethernet Tag ID is set to 0). This is applicable when the PE uses 1524 MAC-based disposition or MPLS-based disposition without VID 1525 translation. 1527 The usage of the MPLS label is described in Section 14 ("Load 1528 Balancing of Unicast Packets"). 1530 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1531 be set to the IPv4 or IPv6 address of the advertising PE. 1533 The Ethernet A-D per EVI route MUST carry one or more Route Target 1534 (RT) extended communities, per Section 7.10. 1536 8.5. Designated Forwarder Election 1538 Consider a CE that is a host or a router that is multihomed directly 1539 to more than one PE in an EVPN instance on a given Ethernet segment. 1540 In this scenario, only one of the PEs, referred to as the Designated 1541 Forwarder (DF), is responsible for certain actions: 1543 * Sending broadcast and multicast traffic for a given EVI to that 1544 CE. 1546 * If the flooding of unknown unicast traffic (i.e., traffic for 1547 which a PE does not know the destination MAC address, see 1548 Section 12) is allowed, sending unknown unicast traffic for a 1549 given EVI to that CE. 1551 * If the multihoming mode is Single-Active, sending (known) unicast 1552 traffic for a given EVI to that CE. 1554 Note that this behavior, which allows selecting a DF at the 1555 granularity of for is the default behavior in this 1556 specification. 1558 In this same scenario, a second PE referred to as the 1559 Backup-Designated Forwarder (Backup-DF or BDF), is responsible for 1560 assuming the role of DF in the event of DF's failure. Until this 1561 occurs, the Backup-DF PE is a subset of, and behaves like, a Non-DF 1562 PE for all forwarding considerations. 1564 All other PEs, referred to as Non-Designated Forwarder (Non-DF or 1565 NDF) are not responsible for any forwarding nor of assuming any 1566 functionality from the DF in the event of its failure. 1568 The default procedure for DF election at the granularity of 1569 is referred to as "service carving". With service carving, it is 1570 possible to perform load-balancing of traffic destined to a given 1571 segment. The load-balancing procedure carves the set of EVIs on that 1572 ES among the PEs nodes evenly such that every PE is the DF for a 1573 disjoint and distinct set of EVIs for that ES. The procedure for 1574 service carving is as follows according to the DF Election Finite 1575 State Machine as defined in Section 2.1 of [RFC8584]: 1577 1. When a PE discovers the ESI of the attached Ethernet segment, it 1578 advertises an Ethernet Segment route with the associated 1579 ES-Import extended community. 1581 2. The PE then starts a timer (default value = 3 seconds) to allow 1582 the reception of Ethernet Segment routes from other PE nodes 1583 connected to the same Ethernet segment. This timer value should 1584 be the same across all PEs connected to the same Ethernet 1585 segment. 1587 3. When the timer expires, each PE builds an ordered list of the IP 1588 addresses of all the PE nodes connected to the Ethernet segment 1589 (including itself), in increasing numeric value. Each IP address 1590 in this list is extracted from the "IP Address length" and 1591 "Originating Router's IP address" fields of the advertised 1592 Ethernet Segment route. Every PE is then given an ordinal 1593 indicating its position in the ordered list, starting with 0 as 1594 the ordinal for the PE with the lowest IP address length and 1595 numeric value tuple. The ordinals are used to determine which PE 1596 node will be the DF for a given EVPN instance on the Ethernet 1597 segment, using the following rule: 1598 Assuming a redundancy group of N PE nodes, the PE with ordinal i 1599 is the DF for an when (V mod N) = i, where V is the 1600 Ethernet tag for that EVI. For VLAN-Aware Bundle service, then 1601 the numerically lowest Ethernet tag in that EVI MUST be used in 1602 the modulo function. 1603 It should be noted that using the "Originating Router's IP 1604 address" field in the Ethernet Segment route to get the PE IP 1605 address needed for the ordered list allows for a CE to be 1606 multihomed across different ASes if such a need ever arises. 1608 4. For each EVPN instance, a second list of the IP addresses of all 1609 the PE nodes connected to the Ethernet segment is built. The PE 1610 which was determined as DF above is removed from that ordered 1611 candidate list, forming a backup redundancy group of M PE nodes. 1612 Every remaining PE is then given a second ordinal indicating its 1613 position in the secondary ordered list according to the same 1614 criteria as in step 3 above. 1615 The second ordinals are used to determine which PE nodes will be 1616 the BDF for a given EVPN instance on the Ethernet segment, using 1617 the same modulo rule as above, (V mod M) = i. 1619 5. The PE that is elected as a DF for a given will unblock 1620 BUM traffic, or all traffic if in Single-Active mode, for that 1621 EVI on the corresponding ES. Note that the DF PE unblocks BUM 1622 traffic in the egress direction towards the segment. All Non-DF 1623 PEs, including the Backup-DF PE, continue to drop 1624 multi-destination traffic in the egress direction towards that 1625 . 1626 In the case of link or port failure, the affected PE withdraws 1627 its Ethernet Segment route. This will re-trigger the service 1628 carving procedures on all the PEs in the redundancy group: the 1629 expected new-DF will be BDF previously calculated in step 5. For 1630 PE node failure, or upon PE commissioning or decommissioning, the 1631 PEs re-trigger the service carving. In the case of Single-Active 1632 multihoming, when a service moves from one PE in the redundancy 1633 group to another PE as a result of re-carving, the PE, which ends 1634 up being the elected DF for the service, SHOULD trigger a MAC 1635 address flush notification towards the associated Ethernet 1636 segment. This can be done, for example, using the IEEE 802.1ak 1637 Multiple VLAN Registration Protocol (MVRP) 'new' declaration. 1639 It is RECOMMENDED that all future DF Election algorithms specify an 1640 algorithm to select one Designated Forwarder (DF) PE, one Backup-DF 1641 PE and a residual number of Non-DF PE(s). 1643 8.6. Signaling Primary and Backup DF Elected PEs 1645 Once the Primary and Backup DF Elected PEs for a given are 1646 determined, the multi-homed PEs for that ES will each advertise an 1647 Ethernet A-D per EVI route for that EVI and each will include an 1648 L2-Attr extended community with the P and B bits set to reflect the 1649 advertising PE's role for that EVI. 1651 It should be noted if L2-Attr extended community is included for All- 1652 Active mode, then the P bit must be set for all PEs in the redundancy 1653 group. 1655 8.7. Interoperability with Single-Homing PEs 1657 Let's refer to PEs that only support single-homed CE devices as 1658 single-homing PEs. For single-homing PEs, all the above multihoming 1659 procedures can be omitted; however, to allow for single-homing PEs to 1660 fully interoperate with multihoming PEs, some of the multihoming 1661 procedures described above SHOULD be supported even by single- homing 1662 PEs: 1664 * procedures related to processing Ethernet A-D routes for the 1665 purpose of fast convergence (Section 8.2 ("Fast Convergence")), to 1666 let single-homing PEs benefit from fast convergence 1668 * procedures related to processing Ethernet A-D routes for the 1669 purpose of aliasing (Section 8.4 ("Aliasing and Backup Path")), to 1670 let single-homing PEs benefit from load balancing 1672 * procedures related to processing Ethernet A-D routes for the 1673 purpose of a backup path (Section 8.4 ("Aliasing and Backup 1674 Path")), to let single-homing PEs benefit from the corresponding 1675 convergence improvement 1677 9. Determining Reachability to Unicast MAC Addresses 1679 PEs forward packets that they receive based on the destination MAC 1680 address. This implies that PEs must be able to learn how to reach a 1681 given destination unicast MAC address. 1683 There are two components to MAC address learning -- "local learning" 1684 and "remote learning": 1686 9.1. Local Learning 1688 A particular PE must be able to learn the MAC addresses from the CEs 1689 that are connected to it. This is referred to as local learning. 1691 The PEs in a particular EVPN instance MUST support local data-plane 1692 learning using standard IEEE Ethernet learning procedures. A PE must 1693 be capable of learning MAC addresses in the data plane when it 1694 receives packets such as the following from the CE network: 1696 * DHCP requests 1697 * An ARP Request for its own MAC 1698 * An ARP Request for a peer 1700 Alternatively, PEs MAY learn the MAC addresses of the CEs in the 1701 control plane or via management-plane integration between the PEs and 1702 the CEs. 1704 There are applications where a MAC address that is reachable via a 1705 given PE on a locally attached segment (e.g., with ESI X) may move, 1706 such that it becomes reachable via another PE on another segment 1707 (e.g., with ESI Y). This is referred to as "MAC Mobility". 1708 Procedures to support this are described in Section 15 ("MAC 1709 Mobility"). 1711 9.2. Remote Learning 1713 A particular PE must be able to determine how to send traffic to MAC 1714 addresses that belong to or are behind CEs connected to other PEs, 1715 i.e., to remote CEs or hosts behind remote CEs. We call such MAC 1716 addresses "remote" MAC addresses. 1718 This document requires a PE to learn remote MAC addresses in the 1719 control plane. In order to achieve this, each PE advertises the MAC 1720 addresses it learns from its locally attached CEs in the control 1721 plane, to all the other PEs in that EVPN instance, using MP-BGP and, 1722 specifically, the MAC/IP Advertisement route. 1724 9.2.1. Constructing MAC/IP Address Advertisement 1726 BGP is extended to advertise these MAC addresses using the MAC/IP 1727 Advertisement route type in the EVPN NLRI. 1729 The RD MUST be set per Section 7.9. 1731 The Ethernet Segment Identifier is set to the 10-octet ESI described 1732 in Section 5 ("Ethernet Segment"). 1734 The Ethernet Tag ID is set as defined in Section 6. 1736 The MAC Address Length field is in bits, and it is set to 48. MAC 1737 address length values other than 48 bits are outside the scope of 1738 this document. The encoding of a MAC address MUST be the 6-octet MAC 1739 address specified by [IEEE.802.1Q_2014] and [IEEE.802.1D_2004]. 1741 The IP Address field is optional. By default, the IP Address Length 1742 field is set to 0, and the IP Address field is omitted from the 1743 route. When a valid IP address needs to be advertised, it is then 1744 encoded in this route. When an IP address is present, the IP Address 1745 Length field is in bits, and it is set to 32 or 128 bits. Other IP 1746 Address Length values are outside the scope of this document. The 1747 encoding of an IP address MUST be either 4 octets for IPv4 or 1748 16 octets for IPv6. The Length field of the EVPN NLRI (which is in 1749 octets and is described in Section 7) is sufficient to determine 1750 whether an IP address is encoded in this route and, if so, whether 1751 the encoded IP address is IPv4 or IPv6. 1753 The MPLS Label1 field is encoded as 3 octets, where the high-order 1754 20 bits contain the label value. The MPLS Label1 MUST be downstream 1755 assigned, and it is associated with the MAC address being advertised 1756 by the advertising PE. The advertising PE uses this label when it 1757 receives an MPLS-encapsulated packet to perform forwarding based on 1758 the destination MAC address toward the CE. The forwarding procedures 1759 are specified in Sections 13 and 14. 1761 A PE may advertise the same single EVPN label for all MAC addresses 1762 in a given MAC-VRF. This label assignment is referred to as a per 1763 MAC-VRF label assignment. Alternatively, a PE may advertise a unique 1764 EVPN label per combination. This label 1765 assignment is referred to as a per label 1766 assignment. As a third option, a PE may advertise a unique EVPN 1767 label per combination. This label assignment is 1768 referred to as a per label assignment. As a 1769 fourth option, a PE may advertise a unique EVPN label per MAC 1770 address. This label assignment is referred to as a per MAC label 1771 assignment. All of these label assignment methods have their 1772 trade-offs. The choice of a particular label assignment methodology 1773 is purely local to the PE that originates the route. 1775 An assignment per MAC-VRF label requires the least number of EVPN 1776 labels but requires a MAC lookup in addition to an MPLS lookup on an 1777 egress PE for forwarding. On the other hand, a unique label per 1778 or a unique label per MAC allows an egress PE to 1779 forward a packet that it receives from another PE, to the connected 1780 CE, after looking up only the MPLS labels without having to perform a 1781 MAC lookup. This includes the capability to perform appropriate VLAN 1782 ID translation on egress to the CE. 1784 The MPLS Label2 field is an optional field. If it is present, then 1785 it is encoded as 3 octets, where the high-order 20 bits contain the 1786 label value. 1788 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1789 be set to the IPv4 or IPv6 address of the advertising PE. 1791 The BGP advertisement for the MAC/IP Advertisement route MUST also 1792 carry one or more Route Target (RT) extended communities. RTs may be 1793 configured (as in IP VPNs) or may be derived automatically in the 1794 "Unique VLAN EVPN" case from the Ethernet Tag (VLAN ID), as described 1795 in Section 7.10.1. 1797 It is to be noted that this document does not require PEs to create 1798 forwarding state for remote MACs when they are learned in the control 1799 plane. When this forwarding state is actually created is a local 1800 implementation matter. 1802 9.2.2. Route Resolution 1804 If the Ethernet Segment Identifier field in a received MAC/IP 1805 Advertisement route is set to the reserved ESI value of 0 or MAX-ESI, 1806 then if the receiving PE decides to install forwarding state for the 1807 associated MAC address, it MUST be based on the MAC/IP Advertisement 1808 route alone. 1810 If the Ethernet Segment Identifier field in a received MAC/IP 1811 Advertisement route is set to a non-reserved ESI, and the receiving 1812 PE is locally attached to the same ESI, then the PE does not alter 1813 its forwarding state based on the received route. This ensures that 1814 local routes are preferred to remote routes. 1816 If the Ethernet Segment Identifier field in a received MAC/IP 1817 Advertisement route is set to a non-reserved ESI, then if the 1818 receiving PE decides to install forwarding state for the associated 1819 MAC address, it MUST be when both the MAC/IP Advertisement route AND 1820 the associated set of Ethernet A-D per ES routes have been received. 1821 The dependency of MAC route installation on Ethernet A-D per ES 1822 routes is to ensure that MAC routes don't get accidentally installed 1823 during a mass withdraw period. 1825 To illustrate this with an example, consider two PEs (PE1 and PE2) 1826 connected to a multihomed Ethernet segment ES1. All-Active 1827 redundancy mode is assumed. A given MAC address M1 is learned by PE1 1828 but not PE2. On PE3, the following states may arise: 1830 T1 When the MAC/IP Advertisement route from PE1 and the set of 1831 Ethernet A-D per ES routes and Ethernet A-D per EVI routes from 1832 PE1 and PE2 are received, PE3 can forward traffic destined to 1833 M1 to both PE1 and PE2. 1835 T2 If after T1 PE1 withdraws its set of Ethernet A-D per ES 1836 routes, then PE3 forwards traffic destined to M1 to PE2 only. 1838 T2' If after T1 PE2 withdraws its set of Ethernet A-D per ES 1839 routes, then PE3 forwards traffic destined to M1 to PE1 only. 1841 T2'' If after T1 PE1 withdraws its MAC/IP Advertisement route, then 1842 PE3 treats traffic to M1 as unknown unicast. 1844 T3 PE2 also advertises a MAC route for M1, and then PE1 withdraws 1845 its MAC route for M1. PE3 continues forwarding traffic 1846 destined to M1 to both PE1 and PE2. In other words, despite M1 1847 withdrawal by PE1, PE3 forwards the traffic destined to M1 to 1848 both PE1 and PE2. This is because a flow from the CE, 1849 resulting in M1 traffic getting hashed to PE1, can get 1850 terminated, resulting in M1 being aged out in PE1; however, M1 1851 can be reachable by both PE1 and PE2. 1853 10. ARP and ND 1855 The IP Address field in the MAC/IP Advertisement route may optionally 1856 carry one of the IP addresses associated with the MAC address. This 1857 provides an option that can be used to minimize the flooding of ARP 1858 or Neighbor Discovery (ND) messages over the MPLS network and to 1859 remote CEs. This option also minimizes ARP (or ND) message 1860 processing on end-stations/hosts connected to the EVPN network. A PE 1861 may learn the IP address associated with a MAC address in the control 1862 or management plane between the CE and the PE. Or, it may learn this 1863 binding by snooping certain messages to or from a CE. When a PE 1864 learns the IP address associated with a MAC address of a locally 1865 connected CE, it may advertise this address to other PEs by including 1866 it in the MAC/IP Advertisement route. The IP address may be an IPv4 1867 address encoded using 4 octets or an IPv6 address encoded using 1868 16 octets. For ARP and ND purposes, the IP Address Length field MUST 1869 be set to 32 for an IPv4 address or 128 for an IPv6 address. 1871 If there are multiple IP addresses associated with a MAC address, 1872 then multiple MAC/IP Advertisement routes MUST be generated, one for 1873 each IP address. For instance, this may be the case when there are 1874 both an IPv4 and an IPv6 address associated with the same MAC address 1875 for dual-IP-stack scenarios. When the IP address is dissociated with 1876 the MAC address, then the MAC/IP Advertisement route with that 1877 particular IP address MUST be withdrawn. 1879 Note that a MAC-only route can be advertised along with, but 1880 independent from, a MAC/IP route for scenarios where the MAC learning 1881 over an access network/node is done in the data plane and independent 1882 from ARP snooping that generates a MAC/IP route. In such scenarios, 1883 when the ARP entry times out and causes the MAC/IP to be withdrawn, 1884 then the MAC information will not be lost. In scenarios where the 1885 host MAC/IP is learned via the management or control plane, then the 1886 sender PE may only generate and advertise the MAC/IP route. If the 1887 receiving PE receives both the MAC-only route and the MAC/IP route, 1888 then when it receives a withdraw message for the MAC/IP route, it 1889 MUST delete the corresponding entry from the ARP table but not the 1890 MAC entry from the MAC-VRF table, unless it receives a withdraw 1891 message for the MAC-only route. 1893 When a PE receives an ARP Request for an IP address from a CE, and if 1894 the PE has the MAC address binding for that IP address, the PE SHOULD 1895 perform ARP proxy by responding to the ARP Request. 1897 In the same way, when a PE receives a Neighbor Solicitation for an IP 1898 address from a CE, the PE SHOULD perform ND proxy and respond if the 1899 PE has the binding information for the IP. 1901 10.1. Default Gateway 1903 When a PE needs to perform inter-subnet forwarding where each subnet 1904 is represented by a different broadcast domain (e.g., a different 1905 VLAN), the inter-subnet forwarding is performed at Layer 3, and the 1906 PE that performs such a function is called the default gateway for 1907 the EVPN instance. In this case, when the PE receives an ARP Request 1908 for the IP address configured as the default gateway address, the PE 1909 originates an ARP Reply. 1911 Each PE that acts as a default gateway for a given EVPN instance MAY 1912 advertise in the EVPN control plane its default gateway MAC address 1913 using the MAC/IP Advertisement route, and each such PE indicates that 1914 such a route is associated with the default gateway. This is 1915 accomplished by requiring the route to carry the Default Gateway 1916 extended community defined in Section 7.8 ("Default Gateway Extended 1917 Community"). The ESI field is set to zero when advertising the MAC 1918 route with the Default Gateway extended community. 1920 The IP Address field of the MAC/IP Advertisement route is set to the 1921 default gateway IP address for that subnet (e.g., an EVPN instance). 1922 For a given subnet (e.g., a VLAN or EVPN instance), the default 1923 gateway IP address is the same across all the participant PEs. The 1924 inclusion of this IP address enables the receiving PE to check its 1925 configured default gateway IP address against the one received in the 1926 MAC/IP Advertisement route for that subnet (or EVPN instance), and if 1927 there is a discrepancy, then the PE SHOULD notify the operator and 1928 log an error message. 1930 Unless it is known a priori (by means outside of this document) that 1931 all PEs of a given EVPN instance act as a default gateway for that 1932 EVPN instance, the MPLS label MUST be set to a valid downstream 1933 assigned label. 1935 Furthermore, even if all PEs of a given EVPN instance do act as a 1936 default gateway for that EVPN instance, but only some, but not all, 1937 of these PEs have sufficient (routing) information to provide 1938 inter-subnet routing for all the inter-subnet traffic originated 1939 within the subnet associated with the EVPN instance, then when such a 1940 PE advertises in the EVPN control plane its default gateway MAC 1941 address using the MAC/IP Advertisement route and indicates that such 1942 a route is associated with the default gateway, the route MUST carry 1943 a valid downstream assigned label. 1945 If all PEs of a given EVPN instance act as a default gateway for that 1946 EVPN instance, and the same default gateway MAC address is used 1947 across all gateway devices, then no such advertisement is needed. 1948 However, if each default gateway uses a different MAC address, then 1949 each default gateway needs to be aware of other gateways' MAC 1950 addresses and thus the need for such an advertisement. This is 1951 called MAC address aliasing, since a single default gateway can be 1952 represented by multiple MAC addresses. 1954 Each PE that receives this route and imports it as per procedures 1955 specified in this document follows the procedures in this section 1956 when replying to ARP Requests that it receives. 1958 Each PE that acts as a default gateway for a given EVPN instance that 1959 receives this route and imports it as per procedures specified in 1960 this document MUST create MAC forwarding state that enables it to 1961 apply IP forwarding to the packets destined to the MAC address 1962 carried in the route. 1964 10.1.1. Best Path selection for Default Gateway 1966 Default gateway MAC address that is assigned to an IRB interface (for 1967 a subnet) in a PE MUST be unique in context of that subnet. In other 1968 words, the same MAC address cannot be used by a host either 1969 intentionally or accidently. Therefore, in case such conflicts 1970 arises, there needs to be scheme to detect it and resolve it. In 1971 order to properly detect such conflicts, the following BGP best path 1972 selection MUST be applied. 1974 * When comparing two routes, the route which has Default Gateway 1975 extended community is preferred over a route which does not have 1976 the extended comunity. The PE that has advertised the MAC route 1977 without Default Gateway extended community, upon receiving the 1978 route with Default Gateway extended community, SHALL withdraw its 1979 route and raise an alarm. 1981 * When comparing two routes where both routes have the Default 1982 Gateway extended community, normal BGP best path processing is be 1983 applied. 1985 * When comparing local and remote routes with Default Gateway 1986 extended community, the local route is always preferred. 1988 * MAC Mobility extended community SHALL NOT be attached to routes 1989 which also have Default Gateway extended community on the sending 1990 side and SHALL be ignored on the receiving side. 1992 11. Handling of Multi-destination Traffic 1994 Procedures are required for a given PE to flood broadcast or 1995 multicast traffic received from a CE and with a given Ethernet tag to 1996 the other PEs in the associated [EVI, BD] (EVPN instance). In 1997 certain scenarios, as described in Section 12 ("Processing of Unknown 1998 Unicast Packets"), a given PE may also need to flood unknown unicast 1999 traffic to other PEs. 2001 The PEs in a particular EVPN instance may use ingress replication, 2002 P2MP LSPs, or MP2MP LSPs to send unknown unicast, broadcast, or 2003 multicast traffic to other PEs. 2005 Each PE MUST advertise an "Inclusive Multicast Ethernet Tag route" to 2006 enable the above. The following subsection provides the procedures 2007 to construct the Inclusive Multicast Ethernet Tag route. Subsequent 2008 subsections describe its usage in further detail. 2010 11.1. Constructing Inclusive Multicast Ethernet Tag Route 2012 The RD MUST be set per Section 7.9. 2014 The Ethernet Tag ID is set as defined in Section 6. 2016 The Originating Router's IP Address field value MUST be set to an IP 2017 address of the PE that should be common for all the EVIs on the PE 2018 (e.g., this address may be the PE's loopback address). The IP 2019 Address Length field is in bits. 2021 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 2022 be set to the IPv4 or IPv6 address of the advertising PE. 2024 The BGP advertisement for the Inclusive Multicast Ethernet Tag route 2025 MUST also carry one or more Route Target (RT) extended communities. 2026 The assignment of RTs as described in Section 7.10 MUST be followed. 2028 11.2. P-Tunnel Identification 2030 In order to identify the P-tunnel used for sending broadcast, unknown 2031 unicast, or multicast traffic, the Inclusive Multicast Ethernet Tag 2032 route MUST carry a Provider Multicast Service Interface (PMSI) Tunnel 2033 attribute as specified in [RFC6514]. 2035 Depending on the technology used for the P-tunnel for the EVPN 2036 instance on the PE, the PMSI Tunnel attribute of the Inclusive 2037 Multicast Ethernet Tag route is constructed as follows. 2039 * If the PE that originates the advertisement uses a P-multicast 2040 tree for the P-tunnel for EVPN, the PMSI Tunnel attribute MUST 2041 contain the identity of the tree (note that the PE could create 2042 the identity of the tree prior to the actual instantiation of the 2043 tree). 2045 * A PE that uses a P-multicast tree for the P-tunnel MAY aggregate 2046 two or more Broadcast Domains (BDs) present on the PE onto the 2047 same tree. In this case, in addition to carrying the identity of 2048 the tree, the PMSI Tunnel attribute MUST carry an MPLS label, 2049 which the PE has bound uniquely to the BD associated with this 2050 update (as determined by its RTs and Ethernet Tag ID). The 2051 assigned MPLS label is upstream allocated unless the procedures in 2052 section 19 (Use of Domain-wide Common Block (DCB) Labels) are 2053 followed. If the PE has already advertised Inclusive Multicast 2054 Ethernet Tag routes for two or more BDs that it now desires to 2055 aggregate, then the PE MUST re-advertise those routes. The 2056 re-advertised routes MUST be the same as the original ones, except 2057 for the PMSI Tunnel attribute and the label carried in that 2058 attribute. 2060 * If the PE that originates the advertisement uses ingress 2061 replication for the P-tunnel for EVPN, the route MUST include the 2062 PMSI Tunnel attribute with the Tunnel Type set to Ingress 2063 Replication and the Tunnel Identifier set to a routable address of 2064 the PE. The PMSI Tunnel attribute MUST carry a downstream 2065 assigned MPLS label. This label is used to demultiplex the 2066 broadcast, multicast, or unknown unicast EVPN traffic received 2067 over an MP2P tunnel by the PE. 2069 12. Processing of Unknown Unicast Packets 2071 The procedures in this document do not require the PEs to flood 2072 unknown unicast traffic to other PEs. If PEs learn CE MAC addresses 2073 via a control-plane protocol, the PEs can then distribute MAC 2074 addresses via BGP, and all unicast MAC addresses will be learned 2075 prior to traffic to those destinations. 2077 However, if a destination MAC address of a received packet is not 2078 known by the PE, the PE may have to flood the packet. When flooding, 2079 one must take into account "split-horizon forwarding" as follows: The 2080 principles behind the following procedures are borrowed from the 2081 split-horizon forwarding rules in VPLS solutions [RFC4761] [RFC4762]. 2082 When a PE capable of flooding (say PEx) receives an unknown 2083 destination MAC address, it floods the frame. If the frame arrived 2084 from an attached CE, PEx must send a copy of that frame on every 2085 Ethernet segment (belonging to that EVI) for which it is the DF, 2086 other than the Ethernet segment on which it received the frame. In 2087 addition, the PE must flood the frame to all other PEs participating 2088 in that EVPN instance. If, on the other hand, the frame arrived from 2089 another PE (say PEy), PEx must send a copy of the packet on each 2090 Ethernet segment (belonging to that EVI) for which it is the DF. PEx 2091 MUST NOT send the frame to other PEs, since PEy would have already 2092 done so. Split-horizon forwarding rules apply to unknown MAC 2093 addresses. 2095 Whether or not to flood packets to unknown destination MAC addresses 2096 should be an administrative choice, depending on how learning happens 2097 between CEs and PEs. 2099 The PEs in a particular EVPN instance may use ingress replication 2100 using RSVP-TE P2P LSPs or LDP MP2P LSPs for sending unknown unicast 2101 traffic to other PEs. Or, they may use RSVP-TE P2MP or LDP P2MP for 2102 sending such traffic to other PEs. 2104 12.1. Ingress Replication 2106 If ingress replication is in use, the P-tunnel attribute, carried in 2107 the Inclusive Multicast Ethernet Tag routes for the EVPN instance, 2108 specifies the downstream label that the other PEs can use to send 2109 unknown unicast, multicast, or broadcast traffic for that EVPN 2110 instance to this particular PE. 2112 The PE that receives a packet with this particular MPLS label MUST 2113 treat the packet as a broadcast, multicast, or unknown unicast 2114 packet. Further, if the MAC address is a unicast MAC address, the PE 2115 MUST treat the packet as an unknown unicast packet. 2117 12.2. P2MP MPLS LSPs 2119 The procedures for using P2MP or MP2MP LSPs are very similar to the 2120 VPLS procedures described in [RFC7117]. The P-tunnel attribute used 2121 by a PE for sending unknown unicast, broadcast, or multicast traffic 2122 for a particular EVPN instance is advertised in the Inclusive 2123 Multicast Ethernet Tag route as described in Section 11 ("Handling of 2124 Multi-destination Traffic"). 2126 The P-tunnel attribute specifies the P2MP or MP2MP LSP identifier. 2127 This is the equivalent of an Inclusive tree as described in 2128 [RFC7117]. Note that multiple BDs in the same or different EVIs may 2129 use the same P2MP or MP2MP LSP, using upstream labels [RFC7117] or 2130 DCB labels [I-D.ietf-bess-mvpn-evpn-aggregation-label]. This is the 2131 equivalent of an Aggregate Inclusive tree [RFC7117]. When P2MP or 2132 MP2MP LSPs are used for flooding unknown unicast traffic, packet 2133 reordering is possible. 2135 The PE that receives a packet on the P2MP or MP2MP LSP specified in 2136 the PMSI Tunnel attribute MUST treat the packet as a broadcast, 2137 multicast, or unknown unicast packet. Further, if the MAC address is 2138 a unicast MAC address, the PE MUST treat the packet as an unknown 2139 unicast packet. 2141 13. Forwarding Unicast Packets 2143 This section describes procedures for forwarding unicast packets by 2144 PEs, where such packets are received from either directly connected 2145 CEs or some other PEs. 2147 13.1. Forwarding Packets Received from a CE 2149 When a PE receives a packet from a CE with a given Ethernet Tag, it 2150 must first look up the packet's source MAC address. In certain 2151 environments that enable MAC security, the source MAC address MAY be 2152 used to validate the host identity and determine that traffic from 2153 the host can be allowed into the network. Source MAC lookup MAY also 2154 be used for local MAC address learning. 2156 If the PE decides to forward the packet, the destination MAC address 2157 of the packet must be looked up. If the PE has received MAC address 2158 advertisements for this destination MAC address from one or more 2159 other PEs or has learned it from locally connected CEs, the MAC 2160 address is considered a known MAC address. Otherwise, it is 2161 considered an unknown MAC address. 2163 For known MAC addresses, the PE forwards this packet to one of the 2164 remote PEs or to a locally attached CE. When forwarding to a remote 2165 PE, the packet is encapsulated in the EVPN MPLS label advertised by 2166 the remote PE, for that MAC address, and in the MPLS LSP label stack 2167 to reach the remote PE. 2169 If the MAC address is unknown and if the administrative policy on the 2170 PE requires flooding of unknown unicast traffic, then: 2172 * The PE MUST flood the packet to other PEs. The PE MUST first 2173 encapsulate the packet in the ESI MPLS label as described in 2174 Section 8.3. 2175 If ingress replication is used, the packet MUST be replicated to 2176 each remote PE, with the VPN label being the MPLS label advertised 2177 by the remote PE in a PMSI Tunnel attribute in the Inclusive 2178 Multicast Ethernet Tag route for the [EVI, BD] associated with the 2179 received packet's Ethernet tag. 2180 If P2MP LSPs are being used, the packet MUST be sent on the P2MP 2181 LSP of which the PE is the root, for the [EVI, BD] associated with 2182 the received packet's Ethernet tag. If the same P2MP LSP is used 2183 for all the BD's in the EVI, then all the PEs in the EVI MUST be 2184 the leaves of the P2MP LSP. If a different P2MP LSP is used for a 2185 given BD in the EVI, then only the PEs in that BD MUST be the 2186 leaves of the P2MP LSP. The packet MUST be encapsulated in the 2187 P2MP LSP label stack. 2189 If the MAC address is unknown, then, if the administrative policy on 2190 the PE does not allow flooding of unknown unicast traffic: 2192 * the PE MUST drop the packet. 2194 13.2. Forwarding Packets Received from a Remote PE 2196 This section describes the procedures for forwarding known and 2197 unknown unicast packets received from a remote PE. 2199 13.2.1. Unknown Unicast Forwarding 2201 When a PE receives an MPLS packet from a remote PE, then, after 2202 processing the MPLS label stack, if the top MPLS label ends up being 2203 a P2MP LSP label associated with an EVPN instance or -- in the case 2204 of ingress replication -- the downstream label advertised in the 2205 P-tunnel attribute, and after performing the split-horizon procedures 2206 described in Section 8.3: 2208 * If the PE is the designated forwarder of BUM traffic on a 2209 particular set of ESes for the [EVI, BD], the default behavior is 2210 for the PE to flood that traffic to these ESes. In other words, 2211 the default behavior is for the PE to assume that for BUM traffic 2212 it is not required to perform a destination MAC address lookup. 2213 As an option, the PE may perform a destination MAC lookup to flood 2214 the packet to only a subset of these ESes. For instance, the PE 2215 may decide to not flood a BUM packet on certain Ethernet segments 2216 even if it is the DF on the Ethernet segment, based on 2217 administrative policy. 2219 * If the PE is not the designated forwarder for any ES associated 2220 with the [EVI, BD], the default behavior is for it to drop the BUM 2221 traffic. 2223 13.2.2. Known Unicast Forwarding 2225 If the top MPLS label ends up being an EVPN label that was advertised 2226 in the unicast MAC advertisements, then the PE either forwards the 2227 packet based on CE next-hop forwarding information associated with 2228 the label or does a destination MAC address lookup to forward the 2229 packet to a CE. 2231 14. Load Balancing of Unicast Packets 2233 This section specifies the load-balancing procedures for sending 2234 known unicast packets to a multihomed CE. 2236 14.1. Load Balancing of Traffic from a PE to Remote CEs 2238 When a remote PE imports a MAC/IP Advertisement route for a given ES 2239 in a MAC-VRF, it MUST examine all imported Ethernet A-D routes for 2240 that ESI in order to determine the load- balancing characteristics of 2241 the Ethernet segment. 2243 14.1.1. Single-Active Redundancy Mode 2245 For a given ES, if a remote PE has imported the set of Ethernet A-D 2246 per ES routes from at least one PE, where the "Single-Active" flag in 2247 the ESI Label extended community is set, then that remote PE MUST 2248 deduce that the ES is operating in Single-Active redundancy mode. 2250 This means that for a given [EVI, BD], a given MAC address is only 2251 reachable only via the PE announcing the associated MAC/IP 2252 Advertisement route - this PE will also have advertised an Ethernet 2253 A-D per EVI route for that [EVI, BD] with an L2-Attr extended 2254 community in which the P bit is set. I.e., the Primary DF Elected PE 2255 is also responsible for sending known unicast frames to the CE and 2256 receiving unicast and BUM frames from it. Similarly, the Backup DF 2257 Elected PE will have advertised an Ethernet AD per EVI route for 2258 [EVI, BD] with an L2-Attr extended community in which the B bit is 2259 set. 2261 If the Primary DF Elected PE loses connectivity to the CE it SHOULD 2262 withdraw its set of Ethernet A-D per ES routes for the affected ES 2263 prior to withdrawing the affected MAC/IP Advertisement routes. The 2264 Backup DF Elected PE (which is now the Primary DF Elected PE) needs 2265 to advertise an Ethernet A-D per EVI route for [EVI, BD] with an 2266 L2-Attr extended community in which the P bit is set. Furthermore, 2267 the new Backup DF Elected PE needs to advertise an Ethernet A-D per 2268 EVI route for [EVI, BD] with an L2-Attr extended community in which 2269 the B bit is set. 2271 A remote PE SHOULD use the Primary DF Elected PE's withdrawal of its 2272 set of Ethernet A-D per ES routes as a trigger to update its 2273 forwarding entries for the associated MAC addresses to point at the 2274 Backup DF Elected PE. As the Backup DF Elected PE starts learning 2275 the MAC addresses over its attached ES, it will start sending MAC/IP 2276 Advertisement routes while the failed PE withdraws its routes. This 2277 mechanism minimizes the flooding of traffic during fail-over events. 2279 14.1.2. All-Active Redundancy Mode 2281 For a given ES, if the remote PE has imported the set of Ethernet A-D 2282 per ES routes from one or more PEs and none of them have the 2283 "Single-Active" flag in the ESI Label extended community set, then 2284 the remote PE MUST deduce that the ES is operating in All-Active 2285 redundancy mode. A remote PE that receives a MAC/IP Advertisement 2286 route with a non-reserved ESI SHOULD consider the advertised MAC 2287 address to be reachable via all PEs that have advertised reachability 2288 to that MAC address's EVI/ES/Ethernet Tag ID via the combination of 2289 an Ethernet A-D per EVI route for that EVI/ES/Ethernet Tag ID AND an 2290 Ethernet A-D per ES route for that ES. The remote PE MUST use 2291 received MAC/IP Advertisement routes and Ethernet A-D per EVI/per ES 2292 routes to construct the set of next hops for the advertised MAC 2293 address. 2295 Each next hop comprises an MPLS label stack that is to be used to 2296 reach a given egress PE and allow it to forward a packet. The 2297 portion of the MPLS label stack that is to be used by that egress PE 2298 to forward a packet is constructed by the remote PE as follows: 2300 * If a MAC/IP Advertisement route was received from that PE, then 2301 its label stack MUST be used in the next hop. 2303 * Otherwise, the label stack from the Ethernet A-D per EVI route 2304 that matches the MAC address' EVI/ES/Ethernet Tag ID MUST be used 2305 in the next hop. 2307 The following example explains the above. 2309 Consider a CE (CE1) that is dual-homed to two PEs (PE1 and PE2) on a 2310 LAG interface (ES1), and is sending packets with source MAC address 2311 MAC1 on VLAN1 (mapped to EVI1). A remote PE, say PE3, is able to 2312 learn that MAC1 is reachable via PE1 and PE2. Both PE1 and PE2 may 2313 advertise MAC1 if they receive packets with MAC1 from CE1. If this 2314 is not the case, and if MAC1 is advertised only by PE1, PE3 still 2315 considers MAC1 as reachable via both PE1 and PE2, as both PE1 and PE2 2316 advertise a set of Ethernet A-D per ES routes for ES1 as well as an 2317 Ethernet A-D per EVI route for . 2319 The MPLS label stack to send the packets to PE1 is the MPLS LSP stack 2320 to get to PE1 (at the top of the stack) followed by the EVPN label 2321 advertised by PE1 for CE1's MAC. 2323 The MPLS label stack to send packets to PE2 is the MPLS LSP stack to 2324 get to PE2 (at the top of the stack) followed by the MPLS label in 2325 the Ethernet A-D route advertised by PE2 for , if PE2 has 2326 not advertised MAC1 in BGP. 2328 We will refer to these label stacks as MPLS next hops. 2330 The remote PE (PE3) can now load balance the traffic it receives from 2331 its CEs, destined for CE1, between PE1 and PE2. PE3 may use N-tuple 2332 flow information to hash traffic into one of the MPLS next hops for 2333 load balancing of IP traffic. Alternatively, PE3 may rely on the 2334 source MAC addresses for load balancing. 2336 Note that once PE3 decides to send a particular packet to PE1 or PE2, 2337 it can pick one out of multiple possible paths to reach the 2338 particular remote PE using regular MPLS procedures. For instance, if 2339 the tunneling technology is based on RSVP-TE LSPs and PE3 decides to 2340 send a particular packet to PE1, then PE3 can choose from multiple 2341 RSVP-TE LSPs that have PE1 as their destination. 2343 When PE1 or PE2 receives the packet destined for CE1 from PE3, if the 2344 packet is a known unicast, it is forwarded to CE1. 2346 14.2. Load Balancing of Traffic between a PE and a Local CE 2348 A CE may be configured with more than one interface connected to 2349 different PEs or the same PE for load balancing, using a technology 2350 such as a LAG. The PE(s) and the CE can load balance traffic onto 2351 these interfaces using one of the following mechanisms. 2353 14.2.1. Data-Plane Learning 2355 Consider that the PEs perform data-plane learning for local MAC 2356 addresses learned from local CEs. This enables the PE(s) to learn a 2357 particular MAC address and associate it with one or more interfaces, 2358 if the technology between the PE and the CE supports multipathing. 2359 The PEs can now load balance traffic destined to that MAC address on 2360 the multiple interfaces. 2362 Whether the CE can load balance traffic that it generates on the 2363 multiple interfaces is dependent on the CE implementation. 2365 14.2.2. Control-Plane Learning 2367 The CE can be a host that advertises the same MAC address using a 2368 control protocol on all interfaces. This enables the PE(s) to learn 2369 the host's MAC address and associate it with all interfaces. The PEs 2370 can now load balance traffic destined to the host on all these 2371 interfaces. The host can also load balance the traffic it generates 2372 onto these interfaces, and the PE that receives the traffic employs 2373 EVPN forwarding procedures to forward the traffic. 2375 15. MAC Mobility 2377 It is possible for a given host or end-station (as defined by its MAC 2378 address) to move from one Ethernet segment to another; this is 2379 referred to as 'MAC Mobility' or 'MAC move', and it is different from 2380 the multihoming situation in which a given MAC address is reachable 2381 via multiple PEs for the same Ethernet segment. In a MAC move, there 2382 would be two sets of MAC/IP Advertisement routes -- one set with the 2383 new Ethernet segment and one set with the previous Ethernet segment 2384 -- and the MAC address would appear to be reachable via each of these 2385 segments. 2387 In order to allow all of the PEs in the EVPN instance to correctly 2388 determine the current location of the MAC address, all advertisements 2389 of it being reachable via the previous Ethernet segment MUST be 2390 withdrawn by the PEs, for the previous Ethernet segment, that had 2391 advertised it. 2393 If local learning is performed using the data plane, these PEs will 2394 not be able to detect that the MAC address has moved to another 2395 Ethernet segment, and the receipt of MAC/IP Advertisement routes, 2396 with the MAC Mobility extended community, from other PEs serves as 2397 the trigger for these PEs to withdraw their advertisements. If local 2398 learning is performed using the control or management planes, these 2399 interactions serve as the trigger for these PEs to withdraw their 2400 advertisements. 2402 In a situation where there are multiple moves of a given MAC, 2403 possibly between the same two Ethernet segments, there may be 2404 multiple withdrawals and re-advertisements. In order to ensure that 2405 all PEs in the EVPN instance receive all of these correctly through 2406 the intervening BGP infrastructure, introducing a sequence number 2407 into the MAC Mobility extended community is necessary. 2409 In order to process mobility events correctly, an implementation MUST 2410 handle scenarios in which sequence number wraparound occurs. 2412 Every MAC mobility event for a given MAC address will contain a 2413 sequence number that is set using the following rules: 2415 * A PE advertising a MAC address for the first time advertises it 2416 with no MAC Mobility extended community. 2418 * A PE detecting a locally attached MAC address for which it had 2419 previously received a MAC/IP Advertisement route with a different 2420 Ethernet segment identifier advertises the MAC address in a MAC/IP 2421 Advertisement route tagged with a MAC Mobility extended community 2422 with a sequence number one greater than the sequence number in the 2423 MAC Mobility extended community of the received MAC/IP 2424 Advertisement route. In the case of the first mobility event for 2425 a given MAC address, where the received MAC/IP Advertisement route 2426 does not carry a MAC Mobility extended community, the value of the 2427 sequence number in the received route is assumed to be 0 for the 2428 purpose of this processing. 2430 * A PE detecting a locally attached MAC address for which it had 2431 previously received a MAC/IP Advertisement route with the same 2432 non-zero Ethernet segment identifier advertises it with: 2434 1. no MAC Mobility extended community, if the received route did 2435 not carry said extended community. 2437 2. a MAC Mobility extended community with the sequence number 2438 equal to the highest of the sequence number(s) in the received 2439 MAC/IP Advertisement route(s), if the received route(s) is 2440 (are) tagged with a MAC Mobility extended community. 2442 * A PE detecting a locally attached MAC address for which it had 2443 previously received a MAC/IP Advertisement route with the same 2444 zero Ethernet segment identifier (single-homed scenarios) 2445 advertises it with a MAC Mobility extended community with the 2446 sequence number set properly. In the case of single-homed 2447 scenarios, there is no need for ESI comparison. ESI comparison is 2448 done for multihoming in order to prevent false detection of MAC 2449 moves among the PEs attached to the same multihomed site. 2451 A PE receiving a MAC/IP Advertisement route for a MAC address with a 2452 different Ethernet segment identifier and a higher sequence number 2453 than that which it had previously advertised withdraws its MAC/IP 2454 Advertisement route. If two (or more) PEs advertise the same MAC 2455 address with the same sequence number but different Ethernet segment 2456 identifiers, a PE that receives these routes selects the route 2457 advertised by the PE with the lowest IP address as the best route. 2458 If the PE is the originator of the MAC route and it receives the same 2459 MAC address with the same sequence number that it generated, it will 2460 compare its own IP address with the IP address of the remote PE and 2461 will select the lowest IP. If its own route is not the best one, it 2462 will withdraw the route. 2464 15.1. MAC Duplication Issue 2466 A situation may arise where the same MAC address is learned by 2467 different PEs in the same VLAN because of two (or more) hosts being 2468 misconfigured with the same (duplicate) MAC address. In such a 2469 situation, the traffic originating from these hosts would trigger 2470 continuous MAC moves among the PEs attached to these hosts. It is 2471 important to recognize such a situation and avoid incrementing the 2472 sequence number (in the MAC Mobility extended community) to infinity. 2473 In order to remedy such a situation, a PE that detects a MAC mobility 2474 event via local learning starts an M-second timer (with a default 2475 value of M = 180), and if it detects N MAC moves before the timer 2476 expires (with a default value of N = 5), it concludes that a 2477 duplicate-MAC situation has occurred. The PE MUST alert the operator 2478 and stop sending and processing any BGP MAC/IP Advertisement routes 2479 for that MAC address until a corrective action is taken by the 2480 operator. The values of M and N MUST be configurable to allow for 2481 flexibility in operator control. Note that the other PEs in the EVPN 2482 instance will forward the traffic for the duplicate MAC address to 2483 one of the PEs advertising the duplicate MAC address. 2485 15.2. Sticky MAC Addresses 2487 There are scenarios in which it is desired to configure some MAC 2488 addresses as static so that they are not subjected to MAC moves. In 2489 such scenarios, these MAC addresses are advertised with a MAC 2490 Mobility extended community where the static flag is set to 1 and the 2491 sequence number is set to zero. If a PE receives such advertisements 2492 and later learns the same MAC address(es) via local learning, then 2493 the PE MUST alert the operator. 2495 15.3. Loop Protection 2497 The EVPN MAC Duplication procedure in Section 15.1 prevents an 2498 endless EVPN MAC/IP route advertisement exchange for a duplicate MAC 2499 between two (or more) PEs. While this helps the control plane 2500 settle, in case there is backdoor link (loop) between two or more PEs 2501 attached to the same BD, BUM frames being sent by a CE are still 2502 endlessly looped within the BD through the backdoor link and among 2503 the PEs. This may cause unpredictable issues in the CEs connected to 2504 the affected BD. 2506 The EVPN MAC Duplication Mechanism in Section 15.1 MAY be extended 2507 with a Loop-protection action that is applied on the duplicate-MAC 2508 addresses. This additional mechanism resolves loops created by 2509 accidental or intentional backdoor links and SHOULD be enabled in all 2510 the PEs attached to the BD. 2512 After following the procedure in Section 15.1, when a PE detects a 2513 MAC M as duplicate, the PE behaves as follows: 2515 a) Stops advertising M and logs a duplicate event. 2517 b) Initializes a retry-timer, R seconds. 2519 c) Since Loop Protection is enabled, the PE executes a Loop 2520 Protection action, which we refer to as "Black-Holing" M. 2522 When the PE programs M as a Black-Hole MAC in the Bridge Table, M is 2523 no longer associated to the backdoor Attachment Circuit (AC), but to 2524 a Black-Hole destination. 2526 At this point and while M is in Black-Hole state: 2528 a) If a new frame is received (from the EVPN network or the backdoor 2529 AC) with MAC SA = M, the PE identifies M to be Black-Holed and 2530 discards the frame, ending the loop. 2532 b) Optionally, instead of simply discarding the frame with MAC SA = 2533 M, the PE MAY bring down the AC on which the offending frame is 2534 seen last. 2536 c) Optionally, any frame that arrives at the PE with MAC DA = M 2537 SHOULD be discarded too. 2539 When the retry-timer R for M expires, the PE flushes M from the 2540 Bridge Table and the process is restarted. In general, a Black-Hole 2541 MAC M can be flushed from the Bridge Table if any of the following 2542 events occur: 2544 * Retry-timer R for duplicate-MAC M expires (as discussed). R is 2545 initialized when M is detected as duplicate-MAC. Its value is 2546 configurable and SHOULD be at least three times the EVPN MAC 2547 Duplication M-timer window. 2549 * The operator manually flushes a Black-Hole MAC M. This should be 2550 done only if the conditions under which M was identified as 2551 duplicate have been cleared. 2553 * The remote PE withdraws the MAC/IP route for M and there are no 2554 other remote MAC/IP routes for M. 2556 * The remote PE sends a MAC/IP route update for M with the 2557 sticky-bit set (in the MAC Mobility extended community). 2559 16. Multicast and Broadcast 2561 The PEs in a particular EVPN instance may use ingress replication or 2562 P2MP or MP2MP LSPs to send multicast traffic to other PEs. 2564 16.1. Ingress Replication 2566 The PEs may use ingress replication for flooding BUM traffic as 2567 described in Section 11 ("Handling of Multi-destination Traffic"). A 2568 given broadcast packet must be sent to all the remote PEs. However, 2569 a given multicast packet for a multicast flow may be sent to only a 2570 subset of the PEs. Specifically, a given multicast flow may be sent 2571 to only those PEs that have receivers that are interested in the 2572 multicast flow. Determining which of the PEs have receivers for a 2573 given multicast flow is done using the procedures of 2574 [I-D.ietf-bess-evpn-igmp-mld-proxy]. 2576 16.2. P2MP or MP2MP LSPs 2578 A PE may use an "Inclusive" tree for sending a BUM packet. This 2579 terminology is borrowed from [RFC7117]. 2581 A variety of transport technologies may be used in the service 2582 provider (SP) network. For Inclusive P-multicast trees, these 2583 transport technologies include point-to-multipoint LSPs created by 2584 RSVP-TE or Multipoint LDP (mLDP) or BIER. 2586 16.2.1. Inclusive Trees 2588 An Inclusive tree allows the use of a single multicast distribution 2589 tree, referred to as an Inclusive P-multicast tree, in the SP network 2590 to carry all the multicast traffic from a specified set of EVPN 2591 instances on a given PE. A particular P-multicast tree can be set up 2592 to carry the traffic originated by sites belonging to a single EVPN 2593 instance, or to carry the traffic originated by sites belonging to 2594 several EVPN instances. The ability to carry the traffic of more 2595 than one EVPN instance on the same tree is termed 'Aggregation', and 2596 the tree is called an Aggregate Inclusive P-multicast tree or 2597 Aggregate Inclusive tree for short. The Aggregate Inclusive tree 2598 needs to include every PE that is a member of any of the EVPN 2599 instances that are using the tree. This implies that a PE may 2600 receive BUM traffic even if it doesn't have any receivers that are 2601 interested in receiving that traffic. 2603 An Inclusive or Aggregate Inclusive tree as defined in this document 2604 is a P2MP tree. A P2MP or MP2MP tree is used to carry traffic only 2605 for EVPN CEs that are connected to the PE that is the root of the 2606 tree. 2608 The procedures for signaling an Inclusive tree are the same as those 2609 in [RFC7117], with the VPLS A-D route replaced with the Inclusive 2610 Multicast Ethernet Tag route. The P-tunnel attribute [RFC7117] for 2611 an Inclusive tree is advertised with the Inclusive Multicast Ethernet 2612 Tag route as described in Section 11 ("Handling of Multi-destination 2613 Traffic"). Note that for an Aggregate Inclusive tree, a PE can 2614 "aggregate" multiple EVPN instances on the same P2MP LSP using 2615 upstream labels or DCB allocated labels 2616 [I-D.ietf-bess-mvpn-evpn-aggregation-label]. The procedures for 2617 aggregation are the same as those described in [RFC7117], with VPLS 2618 A-D routes replaced by EVPN Inclusive Multicast Ethernet Tag routes. 2620 17. Convergence 2622 This section describes failure recovery from different types of 2623 network failures. 2625 17.1. Transit Link and Node Failures between PEs 2627 The use of existing MPLS fast-reroute mechanisms can provide failure 2628 recovery on the order of 50 ms, in the event of transit link and node 2629 failures in the infrastructure that connects the PEs. 2631 17.2. PE Failures 2633 Consider a host CE1 that is dual-homed to PE1 and PE2. If PE1 fails, 2634 a remote PE, PE3, can discover this based on the failure of the BGP 2635 session. This failure detection can be in the sub-second range if 2636 Bidirectional Forwarding Detection (BFD) is used to detect BGP 2637 session failures. PE3 can update its forwarding state to start 2638 sending all traffic for CE1 to only PE2. 2640 17.3. PE-to-CE Network Failures 2642 If the connectivity between the multihomed CE and one of the PEs to 2643 which it is attached fails, the PE MUST withdraw the set of Ethernet 2644 A-D per ES routes that had been previously advertised for that ES. 2645 This enables the remote PEs to remove the MPLS next hop to this 2646 particular PE from the set of MPLS next hops that can be used to 2647 forward traffic to the CE. When the MAC entry on the PE ages out, 2648 the PE MUST withdraw the MAC address from BGP. 2650 When an EVI is decommissioned on an Ethernet segment the PE MUST 2651 withdraw the Ethernet A-D per EVI route(s) announced for that . In addition, the PE MUST also withdraw the MAC/IP Advertisement 2653 routes that are impacted by the decommissioning. 2655 The Ethernet A-D per ES routes should be used by an implementation to 2656 optimize the withdrawal of MAC/IP Advertisement routes. When a PE 2657 receives a withdrawal of a particular Ethernet A-D route from an 2658 advertising PE, it SHOULD consider all the MAC/IP Advertisement 2659 routes that are learned from the same ESI as in the Ethernet A-D 2660 route from the advertising PE as having been withdrawn. This 2661 optimizes the network convergence times in the event of PE-to-CE 2662 failures. 2664 18. Frame Ordering 2666 In a MAC address, if the value of the first nibble (bits 8 through 5) 2667 of the most significant octet of the destination MAC address (which 2668 follows the last MPLS label) happens to be 0x4 or 0x6, then the 2669 Ethernet frame can be misinterpreted as an IPv4 or IPv6 packet by 2670 intermediate P nodes performing ECMP based on deep packet inspection, 2671 thus resulting in load balancing packets belonging to the same flow 2672 on different ECMP paths and subjecting those packets to different 2673 delays. Therefore, packets belonging to the same flow can arrive at 2674 the destination out of order. This out-of-order delivery can happen 2675 during steady state in the absence of any failures, resulting in 2676 significant impact on network operations. 2678 In order to avoid frame misordering described in Section 18, the 2679 following network-wide rules are applied: 2681 * If a network uses deep packet inspection for its ECMP, then the 2682 "Preferred PW MPLS Control Word" [RFC4385] MUST be used with the 2683 value 0 (e.g., a 4-octet field with a value of zero) when sending 2684 unicast EVPN-encapsulated packets over an MP2P LSP. 2686 * When sending EVPN-encapsulated packets over a P2MP or P2P RSVP-TE 2687 LSP, then the control word SHOULD NOT be used. 2689 * When sending EVPN-encapsulated packets over a P2MP LSP (e.g., 2690 using mLDP signaling), then the control word SHOULD be used. 2692 * If a network uses entropy labels per [RFC6790], then the control 2693 word SHOULD NOT be used when sending EVPN-encapsulated packets 2694 over an MP2P LSP. 2696 18.1. Flow Label 2698 Flow label is used to add entropy to divisible flows, and creates 2699 ECMP load-balancing in the network. The Flow Label MAY be used in 2700 EVPN networks to achieve better load-balancing in the network, when 2701 transit nodes perform deep packet inspection for ECMP hashing. The 2702 following rules apply: 2704 * When F-bit is set to 1, the PE announces the capability of both 2705 sending and receiving flow label for known unicast. If the PE is 2706 capable of supporting Flow Label, then : 2708 - upon receiving the F-bit set (F=1) from a remote PE, it MUST 2709 send known unicast packets to that PE with Flow labels; 2711 - alternately, upon receiving the F-bit unset (F=0) from a 2712 remote PE, it MUST NOT send known unicast packets to that PE 2713 with Flow labels. 2715 * The Flow Label MUST NOT be used for EVPN-encapsulated BUM packets. 2717 * An ingress PE will push the Flow Label at the bottom of the stack 2718 of the EVPN-encapsulated known unicast packets sent to an egress 2719 PE that previously signaled F-bit set to 1. 2721 * If a PE receives a unicast packet with two labels, then it can 2722 differentiate between [VPN label + ESI label] and [VPN label + 2723 Flow label] and there should be no ambiguity between ESI and Flow 2724 labels even if they overlap. The reason for this is that the 2725 downstream assigned VPN label for known unicast is different than 2726 for BUM traffic and ESI label (if present) comes after BUM VPN 2727 label. Therefore, from the VPN label, the receiving PE knows 2728 whether the next label is a ESI label or a Flow label - i.e., if 2729 the VPN label is for known unicast, then the next label MUST be a 2730 flow label and if the VPN label is for BUM traffic, then the next 2731 label MUST be an ESI label because BUM packets are not sent with 2732 Flow labels. 2734 * When sending EVPN-encapsulated packets over a P2MP LSP (either 2735 RSVP-TE or mLDP), flow label SHOULD NOT be used. This is 2736 independant of any F-bit signalling in the L2-Attr Extended 2737 Community which would still apply to unicast. 2739 19. Use of Domain-wide Common Block (DCB) Labels 2741 The use of DCB labels as in 2742 [I-D.ietf-bess-mvpn-evpn-aggregation-label] is RECOMMENDED in the 2743 following cases: 2745 * Aggregate P-multicast trees: A P-multicast tree MAY aggregate the 2746 traffic of two or more BDs on a given ingress PE. When 2747 aggregation is needed, DCB Labels 2748 [I-D.ietf-bess-mvpn-evpn-aggregation-label] MAY be used in the 2749 MPLS label field of the Inclusive Multicast Ethernet Tag routes 2750 PMSI Tunnel Attribute. The use of DCB Labels, instead of upstream 2751 allocated labels, can greatly reduce the number of labels that the 2752 egress PEs need to process when P-multicast tunnel aggregation is 2753 used in a network with a large number of BDs. 2755 * BIER tunnels: As described in [I-D.ietf-bier-evpn], the use of 2756 labels with BIER tunnels in EVPN networks is similar to aggregate 2757 tunnels, since the ingress PE uses upstream allocated labels to 2758 identify the BD. As described in [I-D.ietf-bier-evpn], DCB labels 2759 can be allocated instead of upstream labels in the PMSI Tunnel 2760 Attribute so that the number of labels required on the egress PEs 2761 can be reduced. 2763 * ESI labels: The ESI labels advertised with EVPN A-D per ES routes 2764 MAY be allocated as DCB labels in general, and are RECOMMENDED to 2765 be allocated as DCB labels when used in combination with P2MP/BIER 2766 tunnels. 2768 When MP2MP tunnels are used, ESI-labels MUST be allocated from a DCB 2769 and the same label must be used by all the PEs attached to the same 2770 Ethernet Segment. In that way, any egress PE with local Ethernet 2771 Segments can identify the source ES of the received BUM packets. 2773 20. Security Considerations 2775 Security considerations discussed in [RFC4761] and [RFC4762] apply to 2776 this document for MAC learning in the data plane over an Attachment 2777 Circuit (AC) and for flooding of unknown unicast and ARP messages 2778 over the MPLS/IP core. Security considerations discussed in 2779 [RFC4364] apply to this document for MAC learning in the control 2780 plane over the MPLS/IP core. This section describes additional 2781 considerations. 2783 As mentioned in [RFC4761], there are two aspects to achieving data 2784 privacy and protecting against denial-of-service attacks in a VPN: 2785 securing the control plane and protecting the forwarding path. 2786 Compromise of the control plane could result in a PE sending customer 2787 data belonging to some EVPN to another EVPN, or black-holing EVPN 2788 customer data, or even sending it to an eavesdropper, none of which 2789 are acceptable from a data privacy point of view. In addition, 2790 compromise of the control plane could provide opportunities for 2791 unauthorized EVPN data usage (e.g., exploiting traffic replication 2792 within a multicast tree to amplify a denial-of-service attack based 2793 on sending large amounts of traffic). 2795 The mechanisms in this document use BGP for the control plane. 2796 Hence, techniques such as those discussed in [RFC5925] help 2797 authenticate BGP messages, making it harder to spoof updates (which 2798 can be used to divert EVPN traffic to the wrong EVPN instance) or 2799 withdrawals (denial-of-service attacks). In the multi-AS backbone 2800 options (b) and (c) [RFC4364], this also means protecting the 2801 inter-AS BGP sessions between the Autonomous System Border Routers 2802 (ASBRs), the PEs, or the Route Reflectors. 2804 Further discussion of security considerations for BGP may be found in 2805 the BGP specification itself [RFC4271] and in the security analysis 2806 for BGP [RFC4272]. The original discussion of the use of the TCP MD5 2807 signature option to protect BGP sessions is found in [RFC5925], while 2808 [RFC6952] includes an analysis of BGP keying and authentication 2809 issues. 2811 Note that [RFC5925] will not help in keeping MPLS labels private -- 2812 knowing the labels, one can eavesdrop on EVPN traffic. Such 2813 eavesdropping additionally requires access to the data path within an 2814 SP network. Users of VPN services are expected to take appropriate 2815 precautions (such as encryption) to protect the data exchanged over a 2816 VPN. 2818 One of the requirements for protecting the data plane is that the 2819 MPLS labels be accepted only from valid interfaces. For a PE, valid 2820 interfaces comprise links from other routers in the PE's own AS. For 2821 an ASBR, valid interfaces comprise links from other routers in the 2822 ASBR's own AS, and links from other ASBRs in ASes that have instances 2823 of a given EVPN. It is especially important in the case of multi-AS 2824 EVPN instances that one accept EVPN packets only from valid 2825 interfaces. 2827 It is also important to help limit malicious traffic into a network 2828 for an impostor MAC address. The mechanism described in Section 15.1 2829 shows how duplicate MAC addresses can be detected and continuous 2830 false MAC mobility can be prevented. The mechanism described in 2831 Section 15.2 shows how MAC addresses can be pinned to a given 2832 Ethernet segment, such that if they appear behind any other Ethernet 2833 segments, the traffic for those MAC addresses can be prevented from 2834 entering the EVPN network from the other Ethernet segments. 2836 21. IANA Considerations 2838 This document defines a new NLRI, called "EVPN", to be carried in BGP 2839 using multiprotocol extensions. This NLRI uses the existing AFI of 2840 25 (L2VPN). IANA has assigned BGP EVPNs a SAFI value of 70. 2842 IANA has allocated the following EVPN Extended Community sub-types in 2843 [RFC7153], and this document is the only reference for them, in 2844 addition to [RFC7432]. 2846 0x00 MAC Mobility [RFC7432] 2847 0x01 ESI Label [RFC7432] 2848 0x02 ES-Import Route Target [RFC7432] 2850 This document creates a registry called "EVPN Route Types". New 2851 registrations will be made through the "RFC Required" procedure 2852 defined in [RFC8126]. The registry has a maximum value of 255. 2853 Registrations carried forward from [RFC7432] are as follows: 2855 0 Reserved [RFC7432] 2856 1 Ethernet Auto-discovery [RFC7432] 2857 2 MAC/IP Advertisement [RFC7432] 2858 3 Inclusive Multicast Ethernet Tag [RFC7432] 2859 4 Ethernet Segment [RFC7432] 2861 This document creates a registry called "EVPN ESI Multihoming 2862 Attributes" for the 1-octet Flags field in the ESI Label Extended 2863 Community. New registrations will be made through the "RFC Required" 2864 procedure defined in [RFC8126]. 2866 Initial registrations are as follows: 2868 RED Multihomed site redundancy mode 2869 00 = All-Active 2870 01 = Single-Active 2872 This document requests allocation of bit 3 in the "EVPN Layer 2 2873 Attributes Control Flags" registry with name F: 2875 F Flow Label MUST be present 2877 22. References 2878 22.1. Normative References 2880 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2881 Requirement Levels", BCP 14, RFC 2119, 2882 DOI 10.17487/RFC2119, March 1997, 2883 . 2885 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 2886 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 2887 DOI 10.17487/RFC4271, January 2006, 2888 . 2890 [RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended 2891 Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, 2892 February 2006, . 2894 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 2895 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 2896 2006, . 2898 [RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter, 2899 "Multiprotocol Extensions for BGP-4", RFC 4760, 2900 DOI 10.17487/RFC4760, January 2007, 2901 . 2903 [RFC4761] Kompella, K., Ed. and Y. Rekhter, Ed., "Virtual Private 2904 LAN Service (VPLS) Using BGP for Auto-Discovery and 2905 Signaling", RFC 4761, DOI 10.17487/RFC4761, January 2007, 2906 . 2908 [RFC4762] Lasserre, M., Ed. and V. Kompella, Ed., "Virtual Private 2909 LAN Service (VPLS) Using Label Distribution Protocol (LDP) 2910 Signaling", RFC 4762, DOI 10.17487/RFC4762, January 2007, 2911 . 2913 [RFC7153] Rosen, E. and Y. Rekhter, "IANA Registries for BGP 2914 Extended Communities", RFC 7153, DOI 10.17487/RFC7153, 2915 March 2014, . 2917 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 2918 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 2919 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2920 2015, . 2922 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 2923 Rabadan, "Virtual Private Wire Service Support in Ethernet 2924 VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017, 2925 . 2927 [RFC8584] Rabadan, J., Ed., Mohanty, S., Ed., Sajassi, A., Drake, 2928 J., Nagaraj, K., and S. Sathappan, "Framework for Ethernet 2929 VPN Designated Forwarder Election Extensibility", 2930 RFC 8584, DOI 10.17487/RFC8584, April 2019, 2931 . 2933 22.2. Informative References 2935 [I-D.ietf-bess-evpn-igmp-mld-proxy] 2936 Sajassi, A., Thoria, S., Mishra, M. P., Drake, J., and W. 2937 Lin, "IGMP and MLD Proxy for EVPN", Work in Progress, 2938 Internet-Draft, draft-ietf-bess-evpn-igmp-mld-proxy-18, 16 2939 February 2022, . 2942 [I-D.ietf-bess-evpn-mh-split-horizon] 2943 Rabadan, J., Nagaraj, K., Lin, W., and A. Sajassi, "EVPN 2944 Multi-Homing Extensions for Split Horizon Filtering", Work 2945 in Progress, Internet-Draft, draft-ietf-bess-evpn-mh- 2946 split-horizon-02, 15 October 2021, 2947 . 2950 [I-D.ietf-bess-mvpn-evpn-aggregation-label] 2951 Zhang, Z. J., Rosen, E. C., Lin, W., Li, Z., and I. 2952 Wijnands, "MVPN/EVPN Tunnel Aggregation with Common 2953 Labels", Work in Progress, Internet-Draft, draft-ietf- 2954 bess-mvpn-evpn-aggregation-label-06, 19 April 2021, 2955 . 2958 [I-D.ietf-bier-evpn] 2959 Zhang, Z., Przygienda, T., Sajassi, A., and J. Rabadan, 2960 "EVPN BUM Using BIER", Work in Progress, Internet-Draft, 2961 draft-ietf-bier-evpn-04, 2 December 2020, 2962 . 2965 [IEEE.802.1D_2004] 2966 IEEE, "IEEE Standard for Local and metropolitan area 2967 networks: Media Access Control (MAC) Bridges", IEEE 2968 802.1D-2004, DOI 10.1109/ieeestd.2004.94569, 6 July 2004, 2969 . 2971 [IEEE.802.1Q_2014] 2972 IEEE, "IEEE Standard for Local and metropolitan area 2973 networks--Bridges and Bridged Networks", IEEE 802.1Q-2014, 2974 DOI 10.1109/ieeestd.2014.6991462, 18 December 2014, 2975 . 2978 [RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis", 2979 RFC 4272, DOI 10.17487/RFC4272, January 2006, 2980 . 2982 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 2983 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 2984 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 2985 February 2006, . 2987 [RFC4664] Andersson, L., Ed. and E. Rosen, Ed., "Framework for Layer 2988 2 Virtual Private Networks (L2VPNs)", RFC 4664, 2989 DOI 10.17487/RFC4664, September 2006, 2990 . 2992 [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 2993 R., Patel, K., and J. Guichard, "Constrained Route 2994 Distribution for Border Gateway Protocol/MultiProtocol 2995 Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual 2996 Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, 2997 November 2006, . 2999 [RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP 3000 Authentication Option", RFC 5925, DOI 10.17487/RFC5925, 3001 June 2010, . 3003 [RFC6514] Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP 3004 Encodings and Procedures for Multicast in MPLS/BGP IP 3005 VPNs", RFC 6514, DOI 10.17487/RFC6514, February 2012, 3006 . 3008 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 3009 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 3010 RFC 6790, DOI 10.17487/RFC6790, November 2012, 3011 . 3013 [RFC6952] Jethanandani, M., Patel, K., and L. Zheng, "Analysis of 3014 BGP, LDP, PCEP, and MSDP Issues According to the Keying 3015 and Authentication for Routing Protocols (KARP) Design 3016 Guide", RFC 6952, DOI 10.17487/RFC6952, May 2013, 3017 . 3019 [RFC7117] Aggarwal, R., Ed., Kamite, Y., Fang, L., Rekhter, Y., and 3020 C. Kodeboniya, "Multicast in Virtual Private LAN Service 3021 (VPLS)", RFC 7117, DOI 10.17487/RFC7117, February 2014, 3022 . 3024 [RFC7209] Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N., 3025 Henderickx, W., and A. Isaac, "Requirements for Ethernet 3026 VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014, 3027 . 3029 [RFC7991] Hoffman, P., "The "xml2rfc" Version 3 Vocabulary", 3030 RFC 7991, DOI 10.17487/RFC7991, December 2016, 3031 . 3033 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 3034 Writing an IANA Considerations Section in RFCs", BCP 26, 3035 RFC 8126, DOI 10.17487/RFC8126, June 2017, 3036 . 3038 [RFC8317] Sajassi, A., Ed., Salam, S., Drake, J., Uttaro, J., 3039 Boutros, S., and J. Rabadan, "Ethernet-Tree (E-Tree) 3040 Support in Ethernet VPN (EVPN) and Provider Backbone 3041 Bridging EVPN (PBB-EVPN)", RFC 8317, DOI 10.17487/RFC8317, 3042 January 2018, . 3044 [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., 3045 Uttaro, J., and W. Henderickx, "A Network Virtualization 3046 Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, 3047 DOI 10.17487/RFC8365, March 2018, 3048 . 3050 [RFC9136] Rabadan, J., Ed., Henderickx, W., Drake, J., Lin, W., and 3051 A. Sajassi, "IP Prefix Advertisement in Ethernet VPN 3052 (EVPN)", RFC 9136, DOI 10.17487/RFC9136, October 2021, 3053 . 3055 Appendix A. Acknowledgments for This Document (2021) 3057 Appendix B. Contributors for This Document (2021) 3059 In addition to the authors listed on the front page, the following 3060 co-authors have also contributed to this document: 3062 Appendix C. Acknowledgments from the First Edition (2015) 3064 Special thanks to Yakov Rekhter for reviewing this document several 3065 times and providing valuable comments, and for his very engaging 3066 discussions on several topics of this document that helped shape this 3067 document. We would also like to thank Pedro Marques, Kaushik Ghosh, 3068 Nischal Sheth, Robert Raszuk, Amit Shukla, and Nadeem Mohammed for 3069 discussions that helped shape this document. We would also like to 3070 thank Han Nguyen for his comments and support of this work. We would 3071 also like to thank Steve Kensil and Reshad Rahman for their reviews. 3072 We would like to thank Jorge Rabadan for his contribution to 3073 Section 5 of this document. We would like to thank Thomas Morin for 3074 his review of this document and his contribution of Section 8.7. 3075 Many thanks to Jakob Heitz for his help to improve several sections 3076 of this document. 3078 We would also like to thank Clarence Filsfils, Dennis Cai, Quaizar 3079 Vohra, Kireeti Kompella, and Apurva Mehta for their contributions to 3080 this document. 3082 Last but not least, special thanks to Giles Heron (our WG chair) for 3083 his detailed review of this document in preparation for WG Last Call 3084 and for making many valuable suggestions. 3086 C.1. Contributors from the First Edition (2015) 3088 In addition to the authors listed on the front page, the following 3089 co-authors have also contributed to this document: 3091 Keyur Patel 3092 Samer Salam 3093 Sami Boutros 3094 Cisco 3096 Yakov Rekhter 3097 Ravi Shekhar 3098 Juniper Networks 3100 Florin Balus 3101 Nuage Networks 3103 C.2. Authors from the First Edition (2015) 3105 Original Authors: 3107 Ali Sajassi 3108 Cisco 3109 EMail: sajassi@cisco.com 3111 Rahul Aggarwal 3112 Arktan 3114 EMail: raggarwa_1@yahoo.com 3116 Nabil Bitar 3117 Verizon Communications 3119 EMail : nabil.n.bitar@verizon.com 3121 Aldrin Isaac 3122 Bloomberg 3124 EMail: aisaac71@bloomberg.net 3126 James Uttaro 3127 AT&T 3129 EMail: uttaro@att.com 3131 John Drake 3132 Juniper Networks 3134 EMail: jdrake@juniper.net 3136 Wim Henderickx 3137 Alcatel-Lucent 3139 EMail: wim.henderickx@alcatel-lucent.com 3141 Authors' Addresses 3143 Ali Sajassi 3144 Cisco 3145 Email: sajassi@cisco.com 3147 Luc Andre Burdet (editor) 3148 Cisco 3149 Email: lburdet@cisco.com 3151 John Drake 3152 Juniper 3153 Email: jdrake@juniper.net 3154 Jorge Rabadan 3155 Nokia 3156 Email: jorge.rabadan@nokia.com