idnits 2.17.1 draft-ietf-bess-rfc7432bis-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document obsoletes RFC7432, but the abstract doesn't seem to directly say this. It does mention RFC7432 though, so this could be OK. -- The draft header indicates that this document updates RFC8214, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (7 March 2022) is 781 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC4448' is mentioned on line 1076, but not defined == Missing Reference: 'EVI' is mentioned on line 2271, but not defined == Missing Reference: 'BD' is mentioned on line 2271, but not defined == Outdated reference: A later version (-21) exists of draft-ietf-bess-evpn-igmp-mld-proxy-18 == Outdated reference: A later version (-08) exists of draft-ietf-bess-evpn-mh-split-horizon-02 == Outdated reference: A later version (-14) exists of draft-ietf-bess-mvpn-evpn-aggregation-label-06 == Outdated reference: A later version (-14) exists of draft-ietf-bier-evpn-04 Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Working Group A. Sajassi 3 Internet-Draft LA. Burdet, Ed. 4 Obsoletes: 7432 (if approved) Cisco 5 Updates: 8214 (if approved) J. Drake 6 Intended status: Standards Track Juniper 7 Expires: 8 September 2022 J. Rabadan 8 Nokia 9 7 March 2022 11 BGP MPLS-Based Ethernet VPN 12 draft-ietf-bess-rfc7432bis-04 14 Abstract 16 This document describes procedures for BGP MPLS-based Ethernet VPNs 17 (EVPN). The procedures described here meet the requirements 18 specified in RFC 7209 -- "Requirements for Ethernet VPN (EVPN)". 20 Note to Readers 22 _RFC EDITOR: please remove this section before publication_ 24 The complete and detailed set of all changes between this version and 25 RFC7432 may be found as an Annotated Diff (rfcdiff) here 26 (https://tools.ietf.org/rfcdiff?url1=https://www.rfc- 27 editor.org/rfc/rfc7432.txt&url2=https://www.ietf.org/archive/id/ 28 draft-ietf-bess-rfc7432bis-04.txt). 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 8 September 2022. 47 Copyright Notice 49 Copyright (c) 2022 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Revised BSD License text as 58 described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Revised BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.1. Summary of changes from RFC 7432 . . . . . . . . . . . . 4 65 2. Specification of Requirements . . . . . . . . . . . . . . . . 6 66 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 67 4. BGP MPLS-Based EVPN Overview . . . . . . . . . . . . . . . . 7 68 5. Ethernet Segment . . . . . . . . . . . . . . . . . . . . . . 9 69 6. Ethernet Tag ID . . . . . . . . . . . . . . . . . . . . . . . 12 70 6.1. VLAN-Based Service Interface . . . . . . . . . . . . . . 13 71 6.2. VLAN Bundle Service Interface . . . . . . . . . . . . . . 13 72 6.2.1. Port-Based Service Interface . . . . . . . . . . . . 13 73 6.3. VLAN-Aware Bundle Service Interface . . . . . . . . . . . 13 74 6.3.1. Port-Based VLAN-Aware Service Interface . . . . . . . 14 75 6.4. EVPN PE Model . . . . . . . . . . . . . . . . . . . . . . 14 76 7. BGP EVPN Routes . . . . . . . . . . . . . . . . . . . . . . . 16 77 7.1. Ethernet Auto-discovery Route . . . . . . . . . . . . . . 17 78 7.2. MAC/IP Advertisement Route . . . . . . . . . . . . . . . 17 79 7.3. Inclusive Multicast Ethernet Tag Route . . . . . . . . . 18 80 7.4. Ethernet Segment Route . . . . . . . . . . . . . . . . . 19 81 7.5. ESI Label Extended Community . . . . . . . . . . . . . . 20 82 7.6. ES-Import Route Target . . . . . . . . . . . . . . . . . 21 83 7.7. MAC Mobility Extended Community . . . . . . . . . . . . . 22 84 7.8. Default Gateway Extended Community . . . . . . . . . . . 22 85 7.9. Route Distinguisher Assignment per MAC-VRF . . . . . . . 22 86 7.10. Route Targets . . . . . . . . . . . . . . . . . . . . . . 23 87 7.10.1. Auto-derivation from the Ethernet Tag (VLAN ID) . . 23 88 7.11. EVPN Layer 2 Attributes Extended Community . . . . . . . 23 89 7.11.1. EVPN Layer 2 Attributes Partitioning . . . . . . . . 24 90 7.12. Route Prioritization . . . . . . . . . . . . . . . . . . 26 91 8. Multihoming Functions . . . . . . . . . . . . . . . . . . . . 26 92 8.1. Multihomed Ethernet Segment Auto-discovery . . . . . . . 26 93 8.1.1. Constructing the Ethernet Segment Route . . . . . . . 26 94 8.2. Fast Convergence . . . . . . . . . . . . . . . . . . . . 27 95 8.2.1. Constructing Ethernet A-D per Ethernet Segment 96 Route . . . . . . . . . . . . . . . . . . . . . . . . 28 97 8.2.1.1. Ethernet A-D Route Targets . . . . . . . . . . . 28 98 8.3. Split Horizon . . . . . . . . . . . . . . . . . . . . . . 29 99 8.3.1. ESI Label Assignment . . . . . . . . . . . . . . . . 29 100 8.3.1.1. Ingress Replication . . . . . . . . . . . . . . . 30 101 8.3.1.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . 31 102 8.3.1.3. MP2MP MPLS LSPs . . . . . . . . . . . . . . . . . 32 103 8.4. Aliasing and Backup Path . . . . . . . . . . . . . . . . 32 104 8.4.1. Constructing Ethernet A-D per EVPN Instance Route . . 33 105 8.5. Designated Forwarder Election . . . . . . . . . . . . . . 34 106 8.6. Signaling Primary and Backup DF Elected PEs . . . . . . . 37 107 8.7. Interoperability with Single-Homing PEs . . . . . . . . . 37 108 9. Determining Reachability to Unicast MAC Addresses . . . . . . 37 109 9.1. Local Learning . . . . . . . . . . . . . . . . . . . . . 38 110 9.2. Remote Learning . . . . . . . . . . . . . . . . . . . . . 38 111 9.2.1. Constructing MAC/IP Address Advertisement . . . . . . 38 112 9.2.2. Route Resolution . . . . . . . . . . . . . . . . . . 40 113 10. ARP and ND . . . . . . . . . . . . . . . . . . . . . . . . . 41 114 10.1. Default Gateway . . . . . . . . . . . . . . . . . . . . 42 115 10.1.1. Best Path selection for Default Gateway . . . . . . 44 116 11. Handling of Multi-destination Traffic . . . . . . . . . . . . 44 117 11.1. Constructing Inclusive Multicast Ethernet Tag Route . . 45 118 11.2. P-Tunnel Identification . . . . . . . . . . . . . . . . 45 119 12. Processing of Unknown Unicast Packets . . . . . . . . . . . . 46 120 12.1. Ingress Replication . . . . . . . . . . . . . . . . . . 47 121 12.2. P2MP MPLS LSPs . . . . . . . . . . . . . . . . . . . . . 47 122 13. Forwarding Unicast Packets . . . . . . . . . . . . . . . . . 48 123 13.1. Forwarding Packets Received from a CE . . . . . . . . . 48 124 13.2. Forwarding Packets Received from a Remote PE . . . . . . 49 125 13.2.1. Unknown Unicast Forwarding . . . . . . . . . . . . . 49 126 13.2.2. Known Unicast Forwarding . . . . . . . . . . . . . . 50 127 14. Load Balancing of Unicast Packets . . . . . . . . . . . . . . 50 128 14.1. Load Balancing of Traffic from a PE to Remote CEs . . . 50 129 14.1.1. Single-Active Redundancy Mode . . . . . . . . . . . 50 130 14.1.2. All-Active Redundancy Mode . . . . . . . . . . . . . 51 131 14.2. Load Balancing of Traffic between a PE and a Local CE . 52 132 14.2.1. Data-Plane Learning . . . . . . . . . . . . . . . . 52 133 14.2.2. Control-Plane Learning . . . . . . . . . . . . . . . 53 134 15. MAC Mobility . . . . . . . . . . . . . . . . . . . . . . . . 53 135 15.1. MAC Duplication Issue . . . . . . . . . . . . . . . . . 55 136 15.2. Sticky MAC Addresses . . . . . . . . . . . . . . . . . . 55 137 15.3. Loop Protection . . . . . . . . . . . . . . . . . . . . 55 138 16. Multicast and Broadcast . . . . . . . . . . . . . . . . . . . 57 139 16.1. Ingress Replication . . . . . . . . . . . . . . . . . . 57 140 16.2. P2MP or MP2MP LSPs . . . . . . . . . . . . . . . . . . . 57 141 16.2.1. Inclusive Trees . . . . . . . . . . . . . . . . . . 57 142 17. Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 58 143 17.1. Transit Link and Node Failures between PEs . . . . . . . 58 144 17.2. PE Failures . . . . . . . . . . . . . . . . . . . . . . 58 145 17.3. PE-to-CE Network Failures . . . . . . . . . . . . . . . 58 146 18. Frame Ordering . . . . . . . . . . . . . . . . . . . . . . . 59 147 18.1. Flow Label . . . . . . . . . . . . . . . . . . . . . . . 60 148 19. Use of Domain-wide Common Block (DCB) Labels . . . . . . . . 61 149 20. Security Considerations . . . . . . . . . . . . . . . . . . . 61 150 21. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 63 151 22. References . . . . . . . . . . . . . . . . . . . . . . . . . 63 152 22.1. Normative References . . . . . . . . . . . . . . . . . . 64 153 22.2. Informative References . . . . . . . . . . . . . . . . . 65 154 Appendix A. Acknowledgments for This Document (2021) . . . . . . 67 155 Appendix B. Contributors for This Document (2021) . . . . . . . 67 156 Appendix C. Acknowledgments from the First Edition (2015) . . . 68 157 C.1. Contributors from the First Edition (2015) . . . . . . . 68 158 C.2. Authors from the First Edition (2015) . . . . . . . . . . 68 159 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 69 161 1. Introduction 163 Virtual Private LAN Service (VPLS), as defined in [RFC4664], 164 [RFC4761], and [RFC4762], is a proven and widely deployed technology. 165 However, the existing solution has a number of limitations when it 166 comes to multihoming and redundancy, multicast optimization, 167 provisioning simplicity, flow-based load balancing, and multipathing; 168 these limitations are important considerations for Data Center (DC) 169 deployments. [RFC7209] describes the motivation for a new solution 170 to address these limitations. It also outlines a set of requirements 171 that the new solution must address. 173 This document describes procedures for a BGP MPLS-based solution 174 called Ethernet VPN (EVPN) to address the requirements specified in 175 [RFC7209]. Please refer to [RFC7209] for the detailed requirements 176 and motivation. EVPN requires extensions to existing IP/MPLS 177 protocols as described in this document. In addition to these 178 extensions, EVPN uses several building blocks from existing MPLS 179 technologies. 181 1.1. Summary of changes from RFC 7432 183 This section describes the significant changes between [RFC4762] and 184 this document. 186 * Updates to Terminology i.a. BD, EVI, Ethernet Tag ID, P-tunnel, 187 DF/BDF/NDF, DCB; 189 * Added Section 6.4 for description and disambiguation of EVPN 190 bridging terminology; 192 * Added ES-Import route target auto-derivation for ESI types 0,4,5; 194 * Precision of 'encoding' language for all references to 'Label' 195 fields; 197 * Added Section 7.11 for usage of EVPN Layer 2 Attributes Extended 198 Community in EVPN Bridging; 200 * Added Section 7.12 proposes relative order-of-magnitude route 201 priority and processing to help achieve fast convergence; 203 * Corrected Section 8.2.1 to include reference to E-TREE exception; 205 * Updated Section 8.5 to include Backup- and Non-Designated 206 Forwarder roles to DF-Election algorithm, description of those 207 roles and signaling updates; 209 * Updated Section 8.5 to specify DF Election behaviour for 210 Originating IP in different family 212 * Added Section 8.3.1.3 for MP2MP MPLS LSPs and updated 213 Section 12.2; 215 * Address conflicts in Best Path algorithm for Default Gateway in 216 Section 10.1.1; 218 * Update to Section 14.1.1 redundancy mode description; 220 * Added Section 15.3 describing a loop detection and protection 221 mechanism; 223 * Added Section 18.1 describing Flow-label usage and signaling (see 224 also new Section 7.11); 226 * Section 19 specifies use of Domain-wide Common Block (DCB) for 227 several cases; 229 * Restructuring, namely Section 8.5 to Section 5, simplify all 230 Ethernet Tag ID references to Section 6 ; and 232 * Corrected Route Target and other extcomm 'attributes' references 233 to 'extended communities'; 235 * Cross-references and editorial changes; [RFC7991] and xml2rfc-v3 236 update (source). 238 2. Specification of Requirements 240 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 241 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 242 document are to be interpreted as described in [RFC2119]. 244 3. Terminology 246 BD: Broadcast Domain. In a bridged network, the broadcast domain 247 corresponds to a Virtual LAN (VLAN), where a VLAN is typically 248 represented by a single VLAN ID (VID) but can be represented by 249 several VIDs where Shared VLAN Learning (SVL) is used per 250 [IEEE.802.1Q_2014]. 252 Bridge Table: An instantiation of a broadcast domain on a MAC-VRF. 254 CE: Customer Edge device, e.g., a host, router, or switch. 256 EVI: An EVPN instance spanning the Provider Edge (PE) devices 257 participating in that EVPN. An EVI may be comprised of one BD 258 (VLAN-based, VLAN Bundle, or Port-based services) or multiple BDs 259 (VLAN-aware Bundle or Port-based VLAN-Aware services). 261 MAC-VRF: A Virtual Routing and Forwarding table for Media Access 262 Control (MAC) addresses on a PE. 264 Ethernet Segment (ES): When a customer site (device or network) is 265 connected to one or more PEs via a set of Ethernet links, then 266 that set of links is referred to as an 'Ethernet segment'. 268 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 269 identifies an Ethernet segment is called an 'Ethernet Segment 270 Identifier'. 272 VID: VLAN Identifier. 274 Ethernet Tag: Used to represent a BD that is configured on a given 275 ES for the purposes of DF election and identification 276 for frames received from the CE. Note that any of the following 277 may be used to represent a BD: VIDs (including Q-in-Q tags), 278 configured IDs, VNIs (Virtual Extensible Local Area Network 279 (VXLAN) Network Identifiers), normalized VIDs, I-SIDs (Service 280 Instance Identifiers), etc., as long as the representation of the 281 BDs is configured consistently across the multihomed PEs attached 282 to that ES. 284 Ethernet Tag ID: Normalized network wide ID that is used to identify 285 a BD within an EVI and carried in EVPN routes. 287 LACP: Link Aggregation Control Protocol. 289 MP2MP: Multipoint to Multipoint. 291 MP2P: Multipoint to Point. 293 P2MP: Point to Multipoint. 295 P2P: Point to Point. 297 P-tunnel: A tunnel through the network of one or more SPs. In this 298 document, P-tunnels are instantiated as bidirectional multicast 299 distribution trees. 301 PE: Provider Edge device. 303 Single-Active Redundancy Mode: When only a single PE, among all the 304 PEs attached to an Ethernet segment, is allowed to forward traffic 305 to/from that Ethernet segment for a given VLAN, then the Ethernet 306 segment is defined to be operating in Single-Active redundancy 307 mode. 309 All-Active Redundancy Mode: When all PEs attached to an Ethernet 310 segment are allowed to forward known unicast traffic to/from that 311 Ethernet segment for a given VLAN, then the Ethernet segment is 312 defined to be operating in All-Active redundancy mode. 314 BUM: Broadcast, unknown unicast, and multicast. 316 DF: Designated Forwarder. 318 Backup-DF (BDF): Backup-Designated Forwarder. 320 Non-DF (NDF): Non-Designated Forwarder. 322 DCB: Domain-wide Common Block (of labels), as in 323 [I-D.ietf-bess-mvpn-evpn-aggregation-label]. 325 AC: Attachment Circuit. 327 4. BGP MPLS-Based EVPN Overview 329 This section provides an overview of EVPN. An EVPN instance 330 comprises Customer Edge devices (CEs) that are connected to Provider 331 Edge devices (PEs) that form the edge of the MPLS infrastructure. A 332 CE may be a host, a router, or a switch. The PEs provide virtual 333 Layer 2 bridged connectivity between the CEs. There may be multiple 334 EVPN instances in the provider's network. 336 The PEs may be connected by an MPLS Label Switched Path (LSP) 337 infrastructure, which provides the benefits of MPLS technology, such 338 as fast reroute, resiliency, etc. The PEs may also be connected by 339 an IP infrastructure, in which case IP/GRE (Generic Routing 340 Encapsulation) tunneling or other IP tunneling can be used between 341 the PEs. The detailed procedures in this document are specified only 342 for MPLS LSPs as the tunneling technology. However, these procedures 343 are designed to be extensible to IP tunneling as the Packet Switched 344 Network (PSN) tunneling technology. 346 In an EVPN, MAC learning between PEs occurs not in the data plane (as 347 happens with traditional bridging in VPLS [RFC4761] [RFC4762]) but in 348 the control plane. Control-plane learning offers greater control 349 over the MAC learning process, such as restricting who learns what, 350 and the ability to apply policies. Furthermore, the control plane 351 chosen for advertising MAC reachability information is multi-protocol 352 (MP) BGP (similar to IP VPNs [RFC4364]). This provides flexibility 353 and the ability to preserve the "virtualization" or isolation of 354 groups of interacting agents (hosts, servers, virtual machines) from 355 each other. In EVPN, PEs advertise the MAC addresses learned from 356 the CEs that are connected to them, along with an MPLS label, to 357 other PEs in the control plane using Multiprotocol BGP (MP-BGP). 358 Control-plane learning enables load balancing of traffic to and from 359 CEs that are multihomed to multiple PEs. This is in addition to load 360 balancing across the MPLS core via multiple LSPs between the same 361 pair of PEs. In other words, it allows CEs to connect to multiple 362 active points of attachment. It also improves convergence times in 363 the event of certain network failures. 365 However, learning between PEs and CEs is done by the method best 366 suited to the CE: data-plane learning, IEEE 802.1x, the Link Layer 367 Discovery Protocol (LLDP), IEEE 802.1aq, Address Resolution Protocol 368 (ARP), management plane, or other protocols. 370 It is a local decision as to whether the Layer 2 forwarding table on 371 a PE is populated with all the MAC destination addresses known to the 372 control plane, or whether the PE implements a cache-based scheme. 373 For instance, the MAC forwarding table may be populated only with the 374 MAC destinations of the active flows transiting a specific PE. 376 The policy attributes of EVPN are very similar to those of IP-VPN. 377 An EVPN instance requires a Route Distinguisher (RD) that is unique 378 per MAC-VRF and one or more globally unique Route Targets (RTs). A 379 CE attaches to a BD on a PE, on an Ethernet interface that may be 380 configured for one or more Ethernet tags. If the Ethernet tags are 381 VLAN IDs, some deployment scenarios guarantee uniqueness of VLAN IDs 382 across EVPN instances: all points of attachment for a given EVPN 383 instance use the same VLAN ID, and no other EVPN instance uses this 384 VLAN ID. This document refers to this case as a "Unique VLAN EVPN" 385 and describes simplified procedures to optimize for it. See for 386 example Section 7.10.1 which describes deriving automatically the 387 RT(s) for each EVPN instance from the corresponding VID. 389 5. Ethernet Segment 391 As indicated in [RFC7209], each Ethernet segment needs a unique 392 identifier in an EVPN. This section defines how such identifiers are 393 assigned and how they are encoded for use in EVPN signaling. Later 394 sections of this document describe the protocol mechanisms that 395 utilize the identifiers. 397 When a customer site is connected to one or more PEs via a set of 398 Ethernet links, then this set of Ethernet links constitutes an 399 "Ethernet segment". For a multihomed site, each Ethernet segment 400 (ES) is identified by a unique non-zero identifier called an Ethernet 401 Segment Identifier (ESI). An ESI is encoded as a 10-octet integer in 402 line format with the most significant octet sent first. The 403 following two ESI values are reserved: 405 - ESI 0 denotes a single-homed site. 407 - ESI {0xFF} (repeated 10 times) is known as MAX-ESI and is reserved. 409 In general, an Ethernet segment SHOULD have a non-reserved ESI that 410 is unique network wide (i.e., across all EVPN instances on all the 411 PEs). If the CE(s) constituting an Ethernet segment is (are) managed 412 by the network operator, then ESI uniqueness should be guaranteed; 413 however, if the CE(s) is (are) not managed, then the operator MUST 414 configure a network-wide unique ESI for that Ethernet segment. This 415 is required to enable auto-discovery of Ethernet segments and 416 Designated Forwarder (DF) election. 418 In a network with managed and non-managed CEs, the ESI has the 419 following format: 421 +---+---+---+---+---+---+---+---+---+---+ 422 | T | ESI Value | 423 +---+---+---+---+---+---+---+---+---+---+ 425 Where: 427 T (ESI Type) is a 1-octet field (most significant octet) that 428 specifies the format of the remaining 9 octets (ESI Value). The 429 following six ESI types can be used: 431 * Type 0 (T=0x00) - This type indicates an arbitrary 9-octet ESI 432 value, which is managed and configured by the operator. 434 * Type 1 (T=0x01) - When IEEE 802.1AX LACP is used between the PEs 435 and CEs, this ESI type indicates an auto-generated ESI value 436 determined from LACP by concatenating the following parameters: 438 - CE LACP System MAC address (6 octets). The CE LACP System MAC 439 address MUST be encoded in the high-order 6 octets of the ESI 440 Value field. 442 - CE LACP Port Key (2 octets). The CE LACP port key MUST be 443 encoded in the 2 octets next to the System MAC address. 445 - The remaining octet will be set to 0x00. 447 As far as the CE is concerned, it would treat the multiple PEs 448 that it is connected to as the same switch. This allows the CE to 449 aggregate links that are attached to different PEs in the same 450 bundle. 451 This mechanism could be used only if it produces ESIs that satisfy 452 the uniqueness requirement specified above. 454 * Type 2 (T=0x02) - This type is used in the case of indirectly 455 connected hosts via a bridged LAN between the CEs and the PEs. 456 The ESI Value is auto-generated and determined based on the Layer 457 2 bridge protocol as follows: If the Multiple Spanning Tree 458 Protocol (MSTP) is used in the bridged LAN, then the value of the 459 ESI is derived by listening to Bridge PDUs (BPDUs) on the Ethernet 460 segment. To achieve this, the PE is not required to run MSTP. 461 However, the PE must learn the Root Bridge MAC address and Bridge 462 Priority of the root of the Internal Spanning Tree (IST) by 463 listening to the BPDUs. The ESI Value is constructed as follows: 465 - Root Bridge MAC address (6 octets). The Root Bridge MAC 466 address MUST be encoded in the high-order 6 octets of the ESI 467 Value field. 469 - Root Bridge Priority (2 octets). The CE Root Bridge Priority 470 MUST be encoded in the 2 octets next to the Root Bridge MAC 471 address. 473 - The remaining octet will be set to 0x00. 475 This mechanism could be used only if it produces ESIs that satisfy 476 the uniqueness requirement specified above. 478 * Type 3 (T=0x03) - This type indicates a MAC-based ESI Value that 479 can be auto-generated or configured by the operator. The ESI 480 Value is constructed as follows: 482 - System MAC address (6 octets). The PE MAC address MUST be 483 encoded in the high-order 6 octets of the ESI Value field. 485 - Local Discriminator value (3 octets). The Local Discriminator 486 value MUST be encoded in the low-order 3 octets of the ESI 487 Value. 489 This mechanism could be used only if it produces ESIs that satisfy 490 the uniqueness requirement specified above. 492 * Type 4 (T=0x04) - This type indicates a router-ID ESI Value that 493 can be auto-generated or configured by the operator. The ESI 494 Value is constructed as follows: 496 - Router ID (4 octets). The system router ID MUST be encoded in 497 the high-order 4 octets of the ESI Value field. 499 - Local Discriminator value (4 octets). The Local Discriminator 500 value MUST be encoded in the 4 octets next to the IP address. 502 - The low-order octet of the ESI Value will be set to 0x00. 504 This mechanism could be used only if it produces ESIs that satisfy 505 the uniqueness requirement specified above. 507 * Type 5 (T=0x05) - This type indicates an Autonomous System 508 (AS)-based ESI Value that can be auto-generated or configured by 509 the operator. The ESI Value is constructed as follows: 511 - AS number (4 octets). This is an AS number owned by the system 512 and MUST be encoded in the high-order 4 octets of the ESI Value 513 field. If a 2-octet AS number is used, the high-order extra 514 octets will be 0x0000. 516 - Local Discriminator value (4 octets). The Local Discriminator 517 value MUST be encoded in the 4 octets next to the AS number. 519 - The low-order octet of the ESI Value will be set to 0x00. 521 This mechanism could be used only if it produces ESIs that satisfy 522 the uniqueness requirement specified above. 524 Note that a CE always sends packets belonging to a specific flow 525 using a single link towards a PE. For instance, if the CE is a host, 526 then, as mentioned earlier, the host treats the multiple links that 527 it uses to reach the PEs as a Link Aggregation Group (LAG). The CE 528 employs a local hashing function to map traffic flows onto links in 529 the LAG. 531 If a bridged network is multihomed to more than one PE in an EVPN 532 network via switches, then the support of All-Active redundancy mode 533 requires the bridged network to be connected to two or more PEs using 534 a LAG. 536 If a bridged network does not connect to the PEs using a LAG, then 537 only one of the links between the bridged network and the PEs must be 538 the active link for a given . In this case, the set of 539 Ethernet A-D per ES routes advertised by each PE MUST have the 540 "Single-Active" bit in the flags of the ESI Label extended community 541 set to 1. 543 6. Ethernet Tag ID 545 An Ethernet Tag ID is a 32-bit field containing either a 12-bit or 546 24-bit identifier that identifies a particular broadcast domain 547 (e.g., a VLAN) in an EVPN instance. The 12-bit identifier is called 548 the VLAN ID (VID). An EVPN instance consists of one or more 549 broadcast domains (one or more VLANs). VLANs are assigned to a given 550 EVPN instance by the provider of the EVPN service. A given VLAN can 551 itself be represented by multiple VIDs. In such cases, the PEs 552 participating in that VLAN for a given EVPN instance are responsible 553 for performing VLAN ID translation to/from locally attached CE 554 devices. 556 The following subsections discuss the relationship between broadcast 557 domains (e.g., VLANs), Ethernet Tag IDs (e.g., VIDs), and MAC-VRFs as 558 well as the setting of the Ethernet Tag ID, in the various EVPN BGP 559 routes (defined in Section 8), for the different types of service 560 interfaces described in [RFC7209]. 562 The following Ethernet Tag ID value is reserved: 564 * Ethernet Tag ID {0xFFFFFFFF} is known as MAX-ET. 566 6.1. VLAN-Based Service Interface 568 With this service interface, an EVPN instance consists of only a 569 single broadcast domain (e.g., a single VLAN). Therefore, there is a 570 one-to-one mapping between a VID on this interface and a MAC-VRF. 571 Since a MAC-VRF corresponds to a single VLAN, it consists of a single 572 bridge table corresponding to that VLAN. If the VLAN is represented 573 by multiple VIDs (e.g., a different VID per Ethernet segment per PE), 574 then each PE needs to perform VID translation for frames destined to 575 its Ethernet segment(s). In such scenarios, the Ethernet frames 576 transported over an MPLS/IP network SHOULD remain tagged with the 577 originating VID, and a VID translation MUST be supported in the data 578 path and MUST be performed on the disposition PE. The Ethernet Tag 579 ID in all EVPN routes MUST be set to 0. 581 6.2. VLAN Bundle Service Interface 583 With this service interface, an EVPN instance corresponds to multiple 584 broadcast domains (e.g., multiple VLANs); however, only a single 585 bridge table is maintained per MAC-VRF, which means multiple VLANs 586 share the same bridge table. This implies that MAC addresses MUST be 587 unique across all VLANs for that EVI in order for this service to 588 work. In other words, there is a many-to-one mapping between VLANs 589 and a MAC-VRF, and the MAC-VRF consists of a single bridge table. 590 Furthermore, a single VLAN must be represented by a single VID -- 591 e.g., no VID translation is allowed for this service interface type. 592 The MPLS-encapsulated frames MUST remain tagged with the originating 593 VID. Tag translation is NOT permitted. The Ethernet Tag ID in all 594 EVPN routes MUST be set to 0. 596 6.2.1. Port-Based Service Interface 598 This service interface is a special case of the VLAN bundle service 599 interface, where all of the VLANs on the port are part of the same 600 service and map to the same bundle. The procedures are identical to 601 those described in Section 6.2. 603 6.3. VLAN-Aware Bundle Service Interface 605 With this service interface, an EVPN instance consists of multiple 606 broadcast domains (e.g., multiple VLANs) with each VLAN having its 607 own bridge table -- i.e., multiple bridge tables (one per VLAN) are 608 maintained by a single MAC-VRF corresponding to the EVPN instance. 610 Broadcast, unknown unicast, or multicast (BUM) traffic is sent only 611 to the CEs in a given broadcast domain; however, the broadcast 612 domains within an EVI either MAY each have their own P-Tunnel or MAY 613 share P-Tunnels -- e.g., all of the broadcast domains in an EVI MAY 614 share a single P-Tunnel. 616 In the case where a single VLAN is represented by a single VID and 617 thus no VID translation is required, an MPLS-encapsulated packet MUST 618 carry that VID. The Ethernet Tag ID in all EVPN routes MUST be set 619 to that VID. The advertising PE MAY advertise the MPLS Label in the 620 Ethernet A-D per EVI and MPLS Label1 in the MAC/IP Advertisement 621 routes representing ONLY the EVI or representing both the Ethernet 622 Tag ID and the EVI. This decision is only a local matter by the 623 advertising PE (which is also the disposition PE) and doesn't affect 624 any other PEs. 626 In the case where a single VLAN is represented by different VIDs on 627 different CEs and thus VID translation is required, a normalized 628 Ethernet Tag ID (VID) MUST be carried in the EVPN BGP routes. 629 Furthermore, the advertising PE advertises the MPLS Label1 in the 630 MAC/IP Advertisement route representing both the Ethernet Tag ID and 631 the EVI, so that upon receiving an MPLS-encapsulated packet, it can 632 identify the corresponding bridge table from the MPLS EVPN label and 633 perform Ethernet Tag ID translation ONLY at the disposition PE -- 634 i.e., the Ethernet frames transported over the MPLS/IP network MUST 635 remain tagged with the originating VID, and VID translation is 636 performed on the disposition PE. The Ethernet Tag ID in all EVPN 637 routes MUST be set to the normalized Ethernet Tag ID assigned by the 638 EVPN provider. 640 6.3.1. Port-Based VLAN-Aware Service Interface 642 This service interface is a special case of the VLAN-aware bundle 643 service interface, where all of the VLANs on the port are part of the 644 same service and are mapped to a single bundle but without any VID 645 translation. The procedures are a subset of those described in 646 Section 6.3. 648 6.4. EVPN PE Model 650 Since this document discusses EVPN operation in relationship to MAC- 651 VRF, EVI, Broadcast Domain (BD), and Bridge Table (BT), it is 652 important to understand the relationship between these terms. 653 Therefore, the following PE model is depicted below to illustrate the 654 relationship among them. 656 +--------------------------------------------------+ 657 | | 658 | +------------------+ EVPN PE | 659 | Attachment | +------------------+ | 660 | Circuit(AC1) | | +----------+ | MPLS/NVO tnl 661 ----------------------*Bridge | | +----- 662 | | | |Table(BT1)| | / \ \ 663 | | | | |<------------------> |Eth| 664 | | | | VLAN x | | \ / / 665 | | | +----------+ | +----- 666 | | | ... | | 667 | | | +----------+ | MPLS/NVO tnl 668 | | | |Bridge | | +----- 669 | | | |Table(BT2)| | / \ \ 670 | | | | |<-------------------> |Eth| 671 ----------------------* VLAN y | | \ / / 672 | AC2 | | +----------+ | +----- 673 | | | MAC-VRF1 | | 674 | +-+ RD1/RT1 | | 675 | +------------------+ | 676 | | 677 | | 678 +---------------------------------------------------+ 680 Figure 1: EVPN PE Model 682 A tenant configured for an EVPN service instance (i.e, EVI) on a PE, 683 is instantiated by a single MAC Virtual Routing and Forwarding table 684 (MAC-VRF) on that PE. A MAC-VRF consists of one or more Bridge 685 Tables (BTs) where each BT corresponds to a VLAN (broadcast domain - 686 BD). If a service interface for an EVPN PE is configured in VLAN- 687 Based mode (i.e., Section 6.1), then there is only a single BT per 688 MAC-VRF (per EVI) - i.e., there is only one tenant VLAN per EVI. 689 However, if a service interface for an EVPN PE is configured in VLAN- 690 Aware Bundle mode (i.e., Section 6.3), then there are several BTs per 691 MAC-VRF (per EVI) - i.e., there are several tenant VLANs per EVI. 692 The relationship among these terms can be summarized as follow: 694 * An EVI consists of one or more BDs and a MAC-VRF consists of one 695 or more BTs, one for each BD. A BD is identified by an Ethernet 696 Tag ID which is typically represented by a single VLAN ID (VID); 697 however, it can be represented by multiple VIDs (i.e., Shared VLAN 698 Learning (SVL) mode in 802.1Q). 700 * In VLAN-based mode, there is one EVI per VLAN and thus one BD/BT 701 per VLAN. Furthermore, there is one BT per MAC-VRF. 703 * In VLAN-bundle service, it can be considered as analogous to SVL 704 mode in 802.1Q i.e., one BD per EVI and one BT per MAC-VRF with 705 multiple VIDs representing that BD. 707 * In VLAN-aware bundle service, there is one EVI with multiple BDs 708 where each BD is represented by a VLAN. Furthermore, there are 709 multiple BTs in a single MAC-VRF. 711 Since a single tenant subnet is typically (and in this document) 712 represented by a VLAN (and thus supported by a single BT), for a 713 given tenant there are as many BTs as there are subnets as shown in 714 the PE model above. 716 MAC-VRF is identified by its corresponding route target and route 717 distinguisher. If operating in EVPN VLAN-Based mode, then a 718 receiving PE that receives an EVPN route with MAC-VRF route target 719 can identify the corresponding BT; however, if operating in EVPN 720 VLAN-Aware Bundle mode, then the receiving PE needs both the MAC-VRF 721 route target and Ethernet Tag ID in order to identify the 722 corresponding BT. 724 7. BGP EVPN Routes 726 This document defines a new BGP Network Layer Reachability 727 Information (NLRI) called the EVPN NLRI. 729 The format of the EVPN NLRI is as follows: 731 +-----------------------------------+ 732 | Route Type (1 octet) | 733 +-----------------------------------+ 734 | Length (1 octet) | 735 +-----------------------------------+ 736 | Route Type specific (variable) | 737 +-----------------------------------+ 739 The Route Type field defines the encoding of the rest of the EVPN 740 NLRI (Route Type specific EVPN NLRI). 742 The Length field indicates the length in octets of the Route Type 743 specific field of the EVPN NLRI. 745 This document defines the following Route Types: 747 + 1 - Ethernet Auto-Discovery (A-D) route 748 + 2 - MAC/IP Advertisement route 749 + 3 - Inclusive Multicast Ethernet Tag route 750 + 4 - Ethernet Segment route 752 The detailed encoding and procedures for these route types are 753 described in subsequent sections. 755 The EVPN NLRI is carried in BGP [RFC4271] using BGP Multiprotocol 756 Extensions [RFC4760] with an Address Family Identifier (AFI) of 25 757 (L2VPN) and a Subsequent Address Family Identifier (SAFI) of 70 758 (EVPN). The NLRI field in the MP_REACH_NLRI/MP_UNREACH_NLRI 759 attribute contains the EVPN NLRI (encoded as specified above). 761 In order for two BGP speakers to exchange labeled EVPN NLRI, they 762 must use BGP Capabilities Advertisements to ensure that they both are 763 capable of properly processing such NLRI. This is done as specified 764 in [RFC4760], by using capability code 1 (multiprotocol BGP) with an 765 AFI of 25 (L2VPN) and a SAFI of 70 (EVPN). 767 7.1. Ethernet Auto-discovery Route 769 An Ethernet A-D route type specific EVPN NLRI consists of the 770 following: 772 +---------------------------------------+ 773 | Route Distinguisher (RD) (8 octets) | 774 +---------------------------------------+ 775 |Ethernet Segment Identifier (10 octets)| 776 +---------------------------------------+ 777 | Ethernet Tag ID (4 octets) | 778 +---------------------------------------+ 779 | MPLS Label (3 octets) | 780 +---------------------------------------+ 782 For the purpose of BGP route key processing, only the Ethernet 783 Segment Identifier and the Ethernet Tag ID are considered to be part 784 of the prefix in the NLRI. The MPLS Label field is to be treated as 785 a route attribute as opposed to being part of the route. 787 The MPLS Label field is encoded as 3 octets, where the high-order 788 20 bits contain the label value. 790 For procedures and usage of this route, please see Sections 8.2 791 ("Fast Convergence") and 8.4 ("Aliasing and Backup Path"). 793 7.2. MAC/IP Advertisement Route 795 A MAC/IP Advertisement route type specific EVPN NLRI consists of the 796 following: 798 +---------------------------------------+ 799 | RD (8 octets) | 800 +---------------------------------------+ 801 |Ethernet Segment Identifier (10 octets)| 802 +---------------------------------------+ 803 | Ethernet Tag ID (4 octets) | 804 +---------------------------------------+ 805 | MAC Address Length (1 octet) | 806 +---------------------------------------+ 807 | MAC Address (6 octets) | 808 +---------------------------------------+ 809 | IP Address Length (1 octet) | 810 +---------------------------------------+ 811 | IP Address (0, 4, or 16 octets) | 812 +---------------------------------------+ 813 | MPLS Label1 (3 octets) | 814 +---------------------------------------+ 815 | MPLS Label2 (0 or 3 octets) | 816 +---------------------------------------+ 818 For the purpose of BGP route key processing, only the Ethernet Tag 819 ID, MAC Address Length, MAC Address, IP Address Length, and IP 820 Address fields are considered to be part of the prefix in the NLRI. 821 The Ethernet Segment Identifier, MPLS Label1, and MPLS Label2 fields 822 are to be treated as route attributes as opposed to being part of the 823 "route". Both the IP and MAC address lengths are in bits. 825 The MPLS Label1 and MPLS Label2 fields are encoded as 3 octets, where 826 the high-order 20 bits contain the label value. 828 For procedures and usage of this route, please see Sections 9 829 ("Determining Reachability to Unicast MAC Addresses") and 14 ("Load 830 Balancing of Unicast Packets"). 832 7.3. Inclusive Multicast Ethernet Tag Route 834 An Inclusive Multicast Ethernet Tag route type specific EVPN NLRI 835 consists of the following: 837 +---------------------------------------+ 838 | RD (8 octets) | 839 +---------------------------------------+ 840 | Ethernet Tag ID (4 octets) | 841 +---------------------------------------+ 842 | IP Address Length (1 octet) | 843 +---------------------------------------+ 844 | Originating Router's IP Address | 845 | (4 or 16 octets) | 846 +---------------------------------------+ 848 For procedures and usage of this route, please see Sections 11 849 ("Handling of Multi-destination Traffic"), 12 ("Processing of Unknown 850 Unicast Packets"), and 16 ("Multicast and Broadcast"). The IP 851 address length is in bits. For the purpose of BGP route key 852 processing, only the Ethernet Tag ID, IP Address Length, and 853 Originating Router's IP Address fields are considered to be part of 854 the prefix in the NLRI. 856 7.4. Ethernet Segment Route 858 An Ethernet Segment route type specific EVPN NLRI consists of the 859 following: 861 +---------------------------------------+ 862 | RD (8 octets) | 863 +---------------------------------------+ 864 |Ethernet Segment Identifier (10 octets)| 865 +---------------------------------------+ 866 | IP Address Length (1 octet) | 867 +---------------------------------------+ 868 | Originating Router's IP Address | 869 | (4 or 16 octets) | 870 +---------------------------------------+ 872 For procedures and usage of this route, please see Section 8.5 873 ("Designated Forwarder Election"). The IP address length is in bits. 874 For the purpose of BGP route key processing, only the Ethernet 875 Segment ID, IP Address Length, and Originating Router's IP Address 876 fields are considered to be part of the prefix in the NLRI. 878 7.5. ESI Label Extended Community 880 This Extended Community is a new transitive Extended Community having 881 a Type field value of 0x06 and the Sub-Type 0x01. It may be 882 advertised along with Ethernet Auto-discovery routes, and it enables 883 split-horizon procedures for multihomed sites as described in 884 Section 8.3 ("Split Horizon"). The ESI Label field represents an ES 885 by the advertising PE, and it is used in split-horizon filtering by 886 other PEs that are connected to the same multihomed Ethernet segment. 888 The ESI Label field is encoded as 3 octets, where the high-order 889 20 bits contain the label value. 891 The ESI label value MAY be zero if no split-horizon filtering 892 procedures are required in any of the VLANs of the Ethernet Segment. 893 This is the case in [RFC8214] or Ethernet Segments using Local Bias 894 procedures in [I-D.ietf-bess-evpn-mh-split-horizon]. 896 Each ESI Label extended community is encoded as an 8-octet value, as 897 follows: 899 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 901 | Type=0x06 | Sub-Type=0x01 | Flags(1 octet)| Reserved=0 | 902 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 903 | Reserved=0 | ESI Label | 904 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 906 This document creates an IANA registry called "EVPN ESI Multihoming 907 Attributes" (Section 21 for the Flags octet, where the following 908 field "Multihomed site redundancy mode (RED)" field is defined with 909 initial bit allocations: 911 0 1 2 3 4 5 6 7 912 +-+-+-+-+-+-+-+-+ 913 | MBZ |RED| (MBZ = MUST Be Zero) 914 +-+-+-+-+-+-+-+-+ 916 Name Meaning 917 --------------------------------------------------------------- 918 RED Multihomed site redundancy mode 920 Multihomed site redundancy mode: 922 RED = 00: A value of 00 means that the multihomed site is operating 923 in All-Active redundancy mode. 925 RED = 01: A value of 01 means that the multihomed site is operating 926 in Single-Active redundancy mode. 928 7.6. ES-Import Route Target 930 This is a new transitive Route Target extended community carried with 931 the Ethernet Segment route. When used, it enables all the PEs 932 connected to the same multihomed site to import the Ethernet Segment 933 routes. 935 * The value MAY be derived automatically for ESI Type 0 by encoding 936 the high-order 6-octet portion of the 9-octet ESI Value, which 937 corresponds to part of the arbitrary value configured, in the ES- 938 Import Route Target. 940 * The value is derived automatically for ESI Types 1, 2, and 3, by 941 encoding the high-order 6-octet portion of the 9-octet ESI Value, 942 which corresponds to a MAC address, in the ES-Import Route Target. 944 * The value MAY be derived automatically for ESI Types 4 and 5, by 945 encoding the high-order 6-octet portion of the 9-octet ESI Value, 946 which corresponds to a Router ID or AS number (4-octets) 947 respectively, and 2-octets of Local Discriminator, in the 948 ES-Import Route Target. 950 The format of this Extended Community is as follows: 952 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 953 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 954 | Type=0x06 | Sub-Type=0x02 | ES-Import ~ 955 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 956 ~ ES-Import Cont'd | 957 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 959 This document expands the definition of the Route Target extended 960 community to allow the value of the high-order octet (Type field) to 961 be 0x06 (in addition to the values specified in [RFC4360]). The 962 low-order octet (Sub-Type field) value 0x02 indicates that this 963 Extended Community is of type "Route Target". The new Type field 964 value 0x06 indicates that the structure of this RT is a 6-octet value 965 (e.g., a MAC address). A BGP speaker that implements RT Constraint 966 [RFC4684] MUST apply the RT Constraint procedures to the ES-Import RT 967 as well. 969 For procedures and usage of this extended community, please see 970 Section 8.1 ("Multihomed Ethernet Segment Auto-discovery"). 972 7.7. MAC Mobility Extended Community 974 This Extended Community is a new transitive Extended Community having 975 a Type field value of 0x06 and the Sub-Type 0x00. It may be 976 advertised along with MAC/IP Advertisement routes. The procedures 977 for using this extended community are described in Section 15 ("MAC 978 Mobility"). 980 The MAC Mobility extended community is encoded as an 8-octet value, 981 as follows: 983 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 984 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 985 | Type=0x06 | Sub-Type=0x00 |Flags(1 octet)| Reserved=0 | 986 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 987 | Sequence Number | 988 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 990 The low-order bit of the Flags octet is defined as the 991 "Sticky/static" flag and may be set to 1. A value of 1 means that 992 the MAC address is static and cannot move. The sequence number is 993 used to ensure that PEs retain the correct MAC/IP Advertisement route 994 when multiple updates occur for the same MAC address. 996 7.8. Default Gateway Extended Community 998 The Default Gateway community is an Extended Community of an Opaque 999 Type (see Section 3.3 of [RFC4360]). It is a transitive community, 1000 which means that the first octet is 0x03. The value of the second 1001 octet (Sub-Type) is 0x0d (Default Gateway) as assigned by IANA. The 1002 Value field of this community is reserved (set to 0 by the senders, 1003 ignored by the receivers). For procedures and usage of this extended 1004 community, please see Section 10.1 ("Default Gateway"). 1006 7.9. Route Distinguisher Assignment per MAC-VRF 1008 The Route Distinguisher (RD) MUST be set to the RD of the MAC-VRF 1009 that is advertising the NLRI. An RD MUST be assigned for a given 1010 MAC-VRF on a PE. This RD MUST be unique across all MAC-VRFs on a PE. 1011 It is RECOMMENDED to use the Type 1 RD [RFC4364]. The value field 1012 comprises an IP address of the PE (typically, the loopback address) 1013 followed by a number unique to the PE. This number may be generated 1014 by the PE. In case of VLAN-based or VLAN Bundle services, this 1015 number may also be generated out of the Ethernet Tag ID for the BD as 1016 long as the value does not exceed a length of 16 bits. Or, in the 1017 Unique VLAN EVPN case, the low-order 12 bits may be the 12-bit VLAN 1018 ID, with the remaining high-order 4 bits set to 0. 1020 7.10. Route Targets 1022 The EVPN route MAY carry one or more Route Target (RT) extended 1023 communities. RTs may be configured (as in IP VPNs) or may be derived 1024 automatically. 1026 If a PE uses RT Constraint, the PE advertises all such RTs using RT 1027 Constraints per [RFC4684]. The use of RT Constraints allows each 1028 EVPN route to reach only those PEs that are configured to import at 1029 least one RT from the set of RTs carried in the EVPN route. 1031 7.10.1. Auto-derivation from the Ethernet Tag (VLAN ID) 1033 For the "Unique VLAN EVPN" scenario (Section 4), it is highly 1034 desirable to auto-derive the RT from the Ethernet Tag (VLAN ID). The 1035 procedure for performing such auto-derivation is as follows: 1037 * The Global Administrator field of the RT MUST be set to the 1038 Autonomous System (AS) number with which the PE is associated. 1040 * The 12-bit VLAN ID MUST be encoded in the lowest 12 bits of the 1041 Local Administrator field, with the remaining bits set to zero. 1043 For VLAN-based and VLAN Bundle services, the RT may also be auto- 1044 derived as per the above rules but replacing the 12-bit VLAN ID with 1045 a 16-bit Ethernet Tag ID configured for the BD. If the Ethernet Tag 1046 ID length is 24 bits, the RT for the MAC-VRF can be auto-derived as 1047 per [RFC8365] section 5.1.2.1. 1049 7.11. EVPN Layer 2 Attributes Extended Community 1051 [RFC8214] defines this extended community ("L2-Attr"), to be included 1052 with per-EVI Ethernet A-D routes and mandatory if multihoming is 1053 enabled. 1055 Usage and applicability of this Extended community to Bridging is 1056 clarified here. 1058 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1059 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1060 | MBZ |RSV|RSV|F|C|P|B| (MBZ = MUST Be Zero) 1061 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1063 The following bits in Control Flags are defined in [RFC8214]: 1065 Name Meaning 1066 --------------------------------------------------------------- 1067 P If set to 1 in multihoming Single-Active scenarios, 1068 this flag indicates that the advertising PE is the 1069 primary PE. MUST be set to 1 for multihoming 1070 All-Active scenarios by all active PE(s). 1072 B If set to 1 in multihoming Single-Active scenarios, 1073 this flag indicates that the advertising PE is the 1074 backup PE. 1076 C If set to 1, a control word [RFC4448] MUST be present 1077 when sending EVPN packets to this PE. It is 1078 recommended that the control word be included in the 1079 absence of an entropy label [RFC6790]. 1081 The bits in Control Flags are extended, and [RFC8214] updated, by the 1082 following additional bits: 1084 Name Meaning 1085 --------------------------------------------------------------- 1086 F If set to 1, a Flow Label SHOULD be present 1087 when sending EVPN packets to this PE. 1088 If set to 0, a Flow Label MUST NOT be present 1089 when sending EVPN packets to this PE. 1091 For procedures and usage of this extended community, with respect to 1092 Control Word and Flow Label, please see Section 18. ("Frame 1093 Ordering"). 1095 For procedures and usage of this extended community, with respect to 1096 Primary-Backup bits, please see Section 8.5. ("Designated Forwarder 1097 Election"). 1099 7.11.1. EVPN Layer 2 Attributes Partitioning 1101 The information carried in the L2-Attr Extended Community may be ESI- 1102 specific or BD/MAC-VRF-specific. In order to minimize the processing 1103 overhead of configuration-time items such as MTU not expected to 1104 change at runtime based on failures, the Extended Community from 1105 [RFC8214] is partitioned, with a subset of information carried over 1106 each Ethernet A-D per EVI and Inclusive Multicast routes. 1108 The EVPN Layer 2 Attributes Extended Community, when added to 1109 Inclusive Multicast route: 1111 * BD/MAC-VRF attributes MTU, Control Word and Flow Label are 1112 conveyed, and; 1114 * per-ESI attributes P, B MUST be zero. 1116 +-------------------------------------------+ 1117 | Type (0x06) / Sub-type (0x04) (2 octets) | 1118 +-------------------------------------------+ 1119 | Control Flags (2 octets) | 1120 +-------------------------------------------+ 1121 | L2 MTU (2 octets) | 1122 +-------------------------------------------+ 1123 | Reserved (2 octets) | 1124 +-------------------------------------------+ 1126 1 1 1 1 1 1127 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1128 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1129 | MBZ | MBZ |F|C|MBZ| (MBZ = MUST Be Zero) 1130 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1132 The EVPN Layer 2 Attributes Extended Community is included on 1133 Ethernet A-D per EVI route and: 1135 * per-ESI attributes P, B are conveyed, and; 1137 * BD/MAC-VRF attributes MTU, Control Word and Flow Label MUST be 1138 zero. 1140 +-------------------------------------------+ 1141 | Type (0x06) / Sub-type (0x04) (2 octets) | 1142 +-------------------------------------------+ 1143 | Control Flags (2 octets) | 1144 +-------------------------------------------+ 1145 | MBZ (2 octets) | 1146 +-------------------------------------------+ 1147 | Reserved (2 octets) | 1148 +-------------------------------------------+ 1150 1 1 1 1 1 1151 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1152 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1153 | MBZ | MBZ |P|B| (MBZ = MUST Be Zero) 1154 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1156 Note that in both of the above cases, the values conveyed in this 1157 extended community are at the granularity of an individual EVI (or 1158 [EVI, BD] for VLAN-aware bundle) and hence may vary for different 1159 EVIs. 1161 7.12. Route Prioritization 1163 In order to achieve the Fast Convergence referred to in (Section 8.2 1164 ("Fast Convergence")), BGP speakers SHOULD prioritise advertisement, 1165 processing and redistribution of routes based on relative scale of 1166 priority vs. expected or average scale. 1168 1. Ethernet A-D per ES (Mass-Withdraw Route Type 1) and Ethernet 1169 Segment (Route Type 4) are lower scale and highly convergence 1170 affecting, and SHOULD be handled in first order of priority 1172 2. Ethernet A-D per EVI, Inclusive Multicast Ethernet Tag route, and 1173 IP Prefix route defined in [RFC9136] are sent for each Bridge or 1174 AC at medium scale and may be convergence affecting, and SHOULD 1175 be handled in second order of priority 1177 3. MAC advertisement route (zero and nonzero IP portion), Multicast 1178 Join Sync and Multicast Leave Sync routes defined in 1179 [I-D.ietf-bess-evpn-igmp-mld-proxy] are considered 'individual 1180 routes' and very-high scale or of relatively low importance for 1181 fast convergence and SHOULD be handled in last order of priority. 1183 8. Multihoming Functions 1185 This section discusses the functions, procedures, and associated BGP 1186 routes used to support multihoming in EVPN. This covers both 1187 multihomed device (MHD) and multihomed network (MHN) scenarios. 1189 8.1. Multihomed Ethernet Segment Auto-discovery 1191 PEs connected to the same Ethernet segment can automatically discover 1192 each other with minimal to no configuration through the exchange of 1193 the Ethernet Segment route. 1195 8.1.1. Constructing the Ethernet Segment Route 1197 The Route Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The 1198 value field comprises an IP address of the PE (typically, the 1199 loopback address) followed by a number unique to the PE. 1201 The Ethernet Segment Identifier (ESI) MUST be set to the 10-octet 1202 value described in Section 5. 1204 The BGP advertisement that advertises the Ethernet Segment route MUST 1205 also carry an ES-Import Route Target, as defined in Section 7.6. 1207 The Ethernet Segment route filtering MUST be done such that the 1208 Ethernet Segment route is imported only by the PEs that are 1209 multihomed to the same Ethernet segment. To that end, each PE that 1210 is connected to a particular Ethernet segment constructs an import 1211 filtering rule to import a route that carries the ES-Import Route 1212 Target, constructed from the ESI. 1214 8.2. Fast Convergence 1216 In EVPN, MAC address reachability is learned via the BGP control 1217 plane over the MPLS network. As such, in the absence of any fast 1218 protection mechanism, the network convergence time is a function of 1219 the number of MAC/IP Advertisement routes that must be withdrawn by 1220 the PE encountering a failure. For highly scaled environments, this 1221 scheme yields slow convergence. 1223 To alleviate this, EVPN defines a mechanism to efficiently and 1224 quickly signal, to remote PE nodes, the need to update their 1225 forwarding tables upon the occurrence of a failure in connectivity to 1226 an Ethernet segment. This is done by having each PE advertise a set 1227 of one or more Ethernet A-D per ES routes for each locally attached 1228 Ethernet segment (refer to Section 8.2.1 below for details on how 1229 these routes are constructed). A PE may need to advertise more than 1230 one Ethernet A-D per ES route for a given ES because the ES may be in 1231 a multiplicity of EVIs and the RTs for all of these EVIs may not fit 1232 into a single route. Advertising a set of Ethernet A-D per ES routes 1233 for the ES allows each route to contain a subset of the complete set 1234 of RTs. Each Ethernet A-D per ES route is differentiated from the 1235 other routes in the set by a different Route Distinguisher (RD). 1237 Upon a failure in connectivity to the attached segment, the PE 1238 withdraws the corresponding set of Ethernet A-D per ES routes. This 1239 triggers all PEs that receive the withdrawal to update their next-hop 1240 adjacencies for all MAC addresses associated with the Ethernet 1241 segment in question. If no other PE had advertised an Ethernet A-D 1242 per ES route for the same segment, then the PE that received the 1243 withdrawal simply invalidates the MAC entries for that segment. 1244 Otherwise, the PE updates its next-hop adjacencies accordingly. 1246 8.2.1. Constructing Ethernet A-D per Ethernet Segment Route 1248 This section describes the procedures used to construct the Ethernet 1249 A-D per ES route, which is used for fast convergence (as discussed 1250 above) and for advertising the ESI label used for split-horizon 1251 filtering (as discussed in Section 8.3). Support of this route is 1252 REQUIRED. 1254 The Route Distinguisher (RD) MUST be a Type 1 RD [RFC4364]. The 1255 value field comprises an IP address of the PE (typically, the 1256 loopback address) followed by a number unique to the PE. 1258 The Ethernet Segment Identifier MUST be a 10-octet entity as 1259 described in Section 5 ("Ethernet Segment"). The Ethernet A-D route 1260 is not needed when the Segment Identifier is set to 0 (e.g., single- 1261 homed scenarios). An exception to this rule is described in 1262 [RFC8317]. 1264 The Ethernet Tag ID MUST be set to MAX-ET. 1266 The MPLS label in the NLRI MUST be set to 0. 1268 The ESI Label extended community MUST be included in the route. If 1269 All-Active redundancy mode is desired, then the "Single-Active" bit 1270 in the flags of the ESI Label extended community MUST be set to 0 and 1271 the MPLS label in that Extended Community MUST be set to a valid MPLS 1272 label value. The MPLS label in this Extended Community is referred 1273 to as the ESI label and MUST have the same value in each Ethernet A-D 1274 per ES route advertised for the ES. This label MUST be a downstream 1275 assigned MPLS label if the advertising PE is using ingress 1276 replication for receiving multicast, broadcast, or unknown unicast 1277 traffic from other PEs. If the advertising PE is using P2MP MPLS 1278 LSPs for sending multicast, broadcast, or unknown unicast traffic, 1279 then this label MUST be an upstream assigned MPLS label, unless DCB 1280 allocated labels are used. The usage of this label is described in 1281 Section 8.3. 1283 If Single-Active redundancy mode is desired, then the "Single-Active" 1284 bit in the flags of the ESI Label extended community MUST be set to 1 1285 and the ESI label SHOULD be set to a valid MPLS label value. 1287 8.2.1.1. Ethernet A-D Route Targets 1289 Each Ethernet A-D per ES route MUST carry one or more Route Target 1290 (RT) extended communities. The set of Ethernet A-D routes per ES 1291 MUST carry the entire set of RTs for all the EVPN instances to which 1292 the Ethernet segment belongs. 1294 8.3. Split Horizon 1296 Consider a CE that is multihomed to two or more PEs on an Ethernet 1297 segment ES1 operating in All-Active redundancy mode. If the CE sends 1298 a broadcast, unknown unicast, or multicast (BUM) packet to one of the 1299 Non-Designated Forwarder (Non-DF) PEs, say PE1, then PE1 will forward 1300 that packet to all or a subset of the other PEs in that EVPN 1301 instance, including the DF PE for that Ethernet segment. In this 1302 case, the DF PE to which the CE is multihomed MUST drop the packet 1303 and not forward back to the CE. This filtering is referred to as 1304 "split-horizon filtering" in this document. 1306 When a set of PEs are operating in Single-Active redundancy mode, the 1307 use of this split-horizon filtering mechanism is highly recommended 1308 because it prevents transient loops at the time of failure or 1309 recovery that would impact the Ethernet segment -- e.g., when two PEs 1310 think that both are DFs for that segment before the DF election 1311 procedure settles down. 1313 In order to achieve this split-horizon function, every BUM packet 1314 originating from a Non-DF PE is encapsulated with an MPLS label that 1315 identifies the Ethernet segment of origin (i.e., the segment from 1316 which the frame entered the EVPN network). This label is referred to 1317 as the ESI label and MUST be distributed by all PEs when operating in 1318 All-Active redundancy mode using a set of Ethernet A-D per ES routes, 1319 per Section 8.2.1 above. The ESI label SHOULD be distributed by all 1320 PEs when operating in Single-Active redundancy mode using a set of 1321 Ethernet A-D per ES routes. These routes are imported by the PEs 1322 connected to the Ethernet segment and also by the PEs that have at 1323 least one EVPN instance in common with the Ethernet segment in the 1324 route. As described in Section 8.1.1, the route MUST carry an ESI 1325 Label extended community with a valid ESI label. The disposition PE 1326 relies on the value of the ESI label to determine whether or not a 1327 BUM frame is allowed to egress a specific Ethernet segment. 1329 8.3.1. ESI Label Assignment 1331 The following subsections describe the assignment procedures for the 1332 ESI label, which differ depending on the type of tunnels being used 1333 to deliver multi-destination packets in the EVPN network. 1335 8.3.1.1. Ingress Replication 1337 Each PE that operates in All-Active or Single-Active redundancy mode 1338 and that uses ingress replication to receive BUM traffic advertises a 1339 downstream assigned ESI label in the set of Ethernet A-D per ES 1340 routes for its attached ES. This label MUST be programmed in the 1341 platform label space by the advertising PE, and the forwarding entry 1342 for this label must result in NOT forwarding packets received with 1343 this label onto the Ethernet segment for which the label was 1344 distributed. 1346 The rules for the inclusion of the ESI label in a BUM packet by the 1347 ingress PE operating in All-Active redundancy mode are as follows: 1349 * A Non-DF ingress PE MUST include the ESI label distributed by the 1350 DF egress PE in the copy of a BUM packet sent to it. 1352 * An ingress PE (DF or Non-DF) SHOULD include the ESI label 1353 distributed by each Non-DF egress PE in the copy of a BUM packet 1354 sent to it. 1356 The rule for the inclusion of the ESI label in a BUM packet by the 1357 ingress PE operating in Single-Active redundancy mode is as follows: 1359 * An ingress DF PE SHOULD include the ESI label distributed by the 1360 egress PE in the copy of a BUM packet sent to it. 1362 In both All-Active and Single-Active redundancy mode, an ingress PE 1363 MUST NOT include an ESI label in the copy of a BUM packet sent to an 1364 egress PE that is not attached to the ES through which the BUM packet 1365 entered the EVI. 1367 As an example, consider PE1 and PE2, which are multihomed to CE1 on 1368 ES1 and operating in All-Active multihoming mode. Further, consider 1369 that PE1 is using P2P or MP2P LSPs to send packets to PE2. Consider 1370 that PE1 is the Non-DF for VLAN1 and PE2 is the DF for VLAN1, and PE1 1371 receives a BUM packet from CE1 on VLAN1 on ES1. In this scenario, 1372 PE2 distributes an Inclusive Multicast Ethernet Tag route for VLAN1 1373 corresponding to an EVPN instance. So, when PE1 sends a BUM packet 1374 that it receives from CE1, it MUST first push onto the MPLS label 1375 stack the ESI label that PE2 has distributed for ES1. It MUST then 1376 push onto the MPLS label stack the MPLS label distributed by PE2 in 1377 the Inclusive Multicast Ethernet Tag route for VLAN1. The resulting 1378 packet is further encapsulated in the P2P or MP2P LSP label stack 1379 required to transmit the packet to PE2. When PE2 receives this 1380 packet, it determines, from the top MPLS label, the set of ESIs to 1381 which it will replicate the packet after any P2P or MP2P LSP labels 1382 have been removed. If the next label is the ESI label assigned by 1383 PE2 for ES1, then PE2 MUST NOT forward the packet onto ES1. If the 1384 next label is an ESI label that has not been assigned by PE2, then 1385 PE2 MUST drop the packet. It should be noted that in this scenario, 1386 if PE2 receives a BUM packet for VLAN1 from CE1, then it SHOULD 1387 encapsulate the packet with an ESI label received from PE1 when 1388 sending it to PE1 in order to avoid any transient loops during a 1389 failure scenario that would impact ES1 (e.g., port or link failure). 1391 8.3.1.2. P2MP MPLS LSPs 1393 The Non-DF PEs that operate in All-Active redundancy mode and that 1394 use P2MP LSPs to send BUM traffic advertise an upstream assigned ESI 1395 label in the set of Ethernet A-D per ES routes for their common 1396 attached ES. This label is upstream assigned by the PE that 1397 advertises the route. This label MUST be programmed by the other PEs 1398 that are connected to the ESI advertised in the route, in the context 1399 label space for the advertising PE. Further, the forwarding entry 1400 for this label must result in NOT forwarding packets received with 1401 this label onto the Ethernet segment for which the label was 1402 distributed. This label MUST also be programmed by the other PEs 1403 that import the route but are not connected to the ESI advertised in 1404 the route, in the context label space for the advertising PE. 1405 Further, the forwarding entry for this label must be a label pop with 1406 no other associated action. 1408 The DF PE that operates in Single-Active redundancy mode and that 1409 uses P2MP LSPs to send BUM traffic should advertise an upstream 1410 assigned ESI label in the set of Ethernet A-D per ES routes for its 1411 attached ES, just as described in the previous paragraph. 1413 As an example, consider PE1 and PE2, which are multihomed to CE1 on 1414 ES1 and operating in All-Active multihoming mode. Also, consider 1415 that PE3 belongs to one of the EVPN instances of ES1. Further, 1416 assume that PE1, which is the Non-DF, is using P2MP MPLS LSPs to send 1417 BUM packets. When PE1 sends a BUM packet that it receives from CE1, 1418 it MUST first push onto the MPLS label stack the ESI label that it 1419 has assigned for the ESI on which the packet was received. The 1420 resulting packet is further encapsulated in the P2MP MPLS label stack 1421 necessary to transmit the packet to the other PEs. Penultimate hop 1422 popping MUST be disabled on the P2MP LSPs used in the MPLS transport 1423 infrastructure for EVPN. When PE2 receives this packet, it 1424 decapsulates the top MPLS label and forwards the packet using the 1425 context label space determined by the top label. If the next label 1426 is the ESI label assigned by PE1 to ES1, then PE2 MUST NOT forward 1427 the packet onto ES1. When PE3 receives this packet, it decapsulates 1428 the top MPLS label and forwards the packet using the context label 1429 space determined by the top label. If the next label is the ESI 1430 label assigned by PE1 to ES1 and PE3 is not connected to ES1, then 1431 PE3 MUST pop the label and flood the packet over all local ESIs in 1432 that EVPN instance. It should be noted that when PE2 sends a BUM 1433 frame over a P2MP LSP, it should encapsulate the frame with an ESI 1434 label even though it is the DF for that VLAN, in order to avoid any 1435 transient loops during a failure scenario that would impact ES1 1436 (e.g., port or link failure). 1438 8.3.1.3. MP2MP MPLS LSPs 1440 The procedures for MP2MP tunnels follow Section 8.3.1.2, with the 1441 exceptions described in this section. 1443 When MP2MP tunnels are used, ESI-labels MUST be allocated from a DCB 1444 and the same label must be used by all the PEs attached to the same 1445 Ethernet Segment. 1447 In that way, any egress PE with local Ethernet Segments can identify 1448 the source ES of the received BUM packets. 1450 8.4. Aliasing and Backup Path 1452 In the case where a CE is multihomed to multiple PE nodes, using a 1453 Link Aggregation Group (LAG) with All-Active redundancy, it is 1454 possible that only a single PE learns a set of the MAC addresses 1455 associated with traffic transmitted by the CE. This leads to a 1456 situation where remote PE nodes receive MAC/IP Advertisement routes 1457 for these addresses from a single PE, even though multiple PEs are 1458 connected to the multihomed segment. As a result, the remote PEs are 1459 not able to effectively load balance traffic among the PE nodes 1460 connected to the multihomed Ethernet segment. This could be the 1461 case, for example, when the PEs perform data-plane learning on the 1462 access, and the load-balancing function on the CE hashes traffic from 1463 a given source MAC address to a single PE. 1465 Another scenario where this occurs is when the PEs rely on control- 1466 plane learning on the access (e.g., using ARP), since ARP traffic 1467 will be hashed to a single link in the LAG. 1469 To address this issue, EVPN introduces the concept of 'aliasing', 1470 which is the ability of a PE to signal that it has reachability to an 1471 EVPN instance on a given ES even when it has learned no MAC addresses 1472 from that EVI/ES. The Ethernet A-D per EVI route is used for this 1473 purpose. A remote PE that receives a MAC/IP Advertisement route with 1474 a non-reserved ESI SHOULD consider the advertised MAC address to be 1475 reachable via all PEs that have advertised reachability to that MAC 1476 address's EVI/ES/Ethernet Tag ID via the combination of an Ethernet 1477 A-D per EVI route for that EVI/ES/Ethernet Tag ID AND Ethernet A-D 1478 per ES routes for that ES with the "Single-Active" bit in the flags 1479 of the ESI Label extended community set to 0. 1481 Note that the Ethernet A-D per EVI route may be received by a remote 1482 PE before it receives the set of Ethernet A-D per ES routes. 1483 Therefore, in order to handle corner cases and race conditions, the 1484 Ethernet A-D per EVI route MUST NOT be used for traffic forwarding by 1485 a remote PE until it also receives the associated set of Ethernet A-D 1486 per ES routes. 1488 The backup path is a closely related function, but it is used in 1489 Single-Active redundancy mode. In this case, a PE also advertises 1490 that it has reachability to a given EVI/ES using the same combination 1491 of Ethernet A-D per EVI route and Ethernet A-D per ES route as 1492 discussed above, but with the "Single-Active" bit in the flags of the 1493 ESI Label extended community set to 1. A remote PE that receives a 1494 MAC/IP Advertisement route with a non-reserved ESI SHOULD consider 1495 the advertised MAC address to be reachable via any PE that has 1496 advertised this combination of Ethernet A-D routes, and it SHOULD 1497 install a backup path for that MAC address. 1499 Please see Section 14.1.1 for a description of the backup paths 1500 operation. 1502 8.4.1. Constructing Ethernet A-D per EVPN Instance Route 1504 This section describes the procedures used to construct the Ethernet 1505 A-D per EVPN instance (EVI) route, which is used for aliasing (as 1506 discussed above). Support of this route is OPTIONAL. 1508 The Route Distinguisher (RD) MUST be set per Section 7.9. 1510 The Ethernet Segment Identifier MUST be a 10-octet entity as 1511 described in Section 5 ("Ethernet Segment"). The Ethernet A-D route 1512 is not needed when the Segment Identifier is set to 0. 1514 The Ethernet Tag ID is set as defined in Section 6. 1516 Note that the above allows the Ethernet A-D per EVI route to be 1517 advertised with one of the following granularities: 1519 * One Ethernet A-D route per tuple per 1520 MAC-VRF. This is applicable when the PE uses MPLS-based 1521 disposition with VID translation or may be applicable when the PE 1522 uses MAC-based disposition with VID translation. 1524 * One Ethernet A-D route for each per MAC-VRF (where the 1525 Ethernet Tag ID is set to 0). This is applicable when the PE uses 1526 MAC-based disposition or MPLS-based disposition without VID 1527 translation. 1529 The usage of the MPLS label is described in Section 14 ("Load 1530 Balancing of Unicast Packets"). 1532 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1533 be set to the IPv4 or IPv6 address of the advertising PE. 1535 The Ethernet A-D per EVI route MUST carry one or more Route Target 1536 (RT) extended communities, per Section 7.10. 1538 8.5. Designated Forwarder Election 1540 Consider a CE that is a host or a router that is multihomed directly 1541 to more than one PE in an EVPN instance on a given Ethernet segment. 1542 In this scenario, only one of the PEs, referred to as the Designated 1543 Forwarder (DF), is responsible for certain actions: 1545 * Sending broadcast and multicast traffic for a given EVI to that 1546 CE. 1548 * If the flooding of unknown unicast traffic (i.e., traffic for 1549 which a PE does not know the destination MAC address, see 1550 Section 12) is allowed, sending unknown unicast traffic for a 1551 given EVI to that CE. 1553 * If the multihoming mode is Single-Active, sending (known) unicast 1554 traffic for a given EVI to that CE. 1556 Note that this behavior, which allows selecting a DF at the 1557 granularity of for is the default behavior in this 1558 specification. 1560 In this same scenario, a second PE referred to as the 1561 Backup-Designated Forwarder (Backup-DF or BDF), is responsible for 1562 assuming the role of DF in the event of DF's failure. Until this 1563 occurs, the Backup-DF PE is a subset of, and behaves like, a Non-DF 1564 PE for all forwarding considerations. 1566 All other PEs, referred to as Non-Designated Forwarder (Non-DF or 1567 NDF) are not responsible for any forwarding nor of assuming any 1568 functionality from the DF in the event of its failure. 1570 The default procedure for DF election at the granularity of 1571 is referred to as "service carving". With service carving, it is 1572 possible to perform load-balancing of traffic destined to a given 1573 segment. The load-balancing procedure carves the set of EVIs on that 1574 ES among the PEs nodes evenly such that every PE is the DF for a 1575 disjoint and distinct set of EVIs for that ES. The procedure for 1576 service carving is as follows according to the DF Election Finite 1577 State Machine as defined in Section 2.1 of [RFC8584]: 1579 1. When a PE discovers the ESI of the attached Ethernet segment, it 1580 advertises an Ethernet Segment route with the associated 1581 ES-Import extended community. 1583 2. The PE then starts a timer (default value = 3 seconds) to allow 1584 the reception of Ethernet Segment routes from other PE nodes 1585 connected to the same Ethernet segment. This timer value should 1586 be the same across all PEs connected to the same Ethernet 1587 segment. 1589 3. When the timer expires, each PE builds an ordered list of the IP 1590 addresses of all the PE nodes connected to the Ethernet segment 1591 (including itself), in increasing numeric value. Each IP address 1592 in this list is extracted from the "IP Address length" and 1593 "Originating Router's IP address" fields of the advertised 1594 Ethernet Segment route. Every PE is then given an ordinal 1595 indicating its position in the ordered list, starting with 0 as 1596 the ordinal for the PE with the lowest IP address length and 1597 numeric value tuple. The tuple list is ordered by the IP address 1598 length first and IP address value second. The ordinals are used 1599 to determine which PE node will be the DF for a given EVPN 1600 instance on the Ethernet segment, using the following rule: 1601 Assuming a redundancy group of N PE nodes, the PE with ordinal i 1602 is the DF for an when (V mod N) = i, where V is the 1603 Ethernet tag for that EVI. For VLAN-Aware Bundle service, then 1604 the numerically lowest Ethernet tag in that EVI MUST be used in 1605 the modulo function. 1606 It should be noted that using the "Originating Router's IP 1607 address" field in the Ethernet Segment route to get the PE IP 1608 address needed for the ordered list allows for a CE to be 1609 multihomed across different ASes if such a need ever arises. 1611 4. For each EVPN instance, a second list of the IP addresses of all 1612 the PE nodes connected to the Ethernet segment is built. The PE 1613 which was determined as DF above is removed from that ordered 1614 candidate list, forming a backup redundancy group of M PE nodes. 1615 Every remaining PE is then given a second ordinal indicating its 1616 position in the secondary ordered list according to the same 1617 criteria as in step 3 above. 1618 The second ordinals are used to determine which PE nodes will be 1619 the BDF for a given EVPN instance on the Ethernet segment, using 1620 the same modulo rule as above, (V mod M) = i. 1622 5. The PE that is elected as a DF for a given will unblock 1623 BUM traffic, or all traffic if in Single-Active mode, for that 1624 EVI on the corresponding ES. Note that the DF PE unblocks BUM 1625 traffic in the egress direction towards the segment. All Non-DF 1626 PEs, including the Backup-DF PE, continue to drop 1627 multi-destination traffic in the egress direction towards that 1628 . 1629 In the case of link or port failure, the affected PE withdraws 1630 its Ethernet Segment route. This will re-trigger the service 1631 carving procedures on all the PEs in the redundancy group: the 1632 expected new-DF will be BDF previously calculated in step 5. For 1633 PE node failure, or upon PE commissioning or decommissioning, the 1634 PEs re-trigger the service carving. In the case of Single-Active 1635 multihoming, when a service moves from one PE in the redundancy 1636 group to another PE as a result of re-carving, the PE, which ends 1637 up being the elected DF for the service, SHOULD trigger a MAC 1638 address flush notification towards the associated Ethernet 1639 segment. This can be done, for example, using the IEEE 802.1ak 1640 Multiple VLAN Registration Protocol (MVRP) 'new' declaration. 1642 It is RECOMMENDED that all future DF Election algorithms specify an 1643 algorithm to select one Designated Forwarder (DF) PE, one Backup-DF 1644 PE and a residual number of Non-DF PE(s). 1646 8.6. Signaling Primary and Backup DF Elected PEs 1648 Once the Primary and Backup DF Elected PEs for a given are 1649 determined, the multi-homed PEs for that ES will each advertise an 1650 Ethernet A-D per EVI route for that EVI and each will include an 1651 L2-Attr extended community with the P and B bits set to reflect the 1652 advertising PE's role for that EVI. 1654 It should be noted if L2-Attr extended community is included for All- 1655 Active mode, then the P bit must be set for all PEs in the redundancy 1656 group. 1658 8.7. Interoperability with Single-Homing PEs 1660 Let's refer to PEs that only support single-homed CE devices as 1661 single-homing PEs. For single-homing PEs, all the above multihoming 1662 procedures can be omitted; however, to allow for single-homing PEs to 1663 fully interoperate with multihoming PEs, some of the multihoming 1664 procedures described above SHOULD be supported even by single- homing 1665 PEs: 1667 * procedures related to processing Ethernet A-D routes for the 1668 purpose of fast convergence (Section 8.2 ("Fast Convergence")), to 1669 let single-homing PEs benefit from fast convergence 1671 * procedures related to processing Ethernet A-D routes for the 1672 purpose of aliasing (Section 8.4 ("Aliasing and Backup Path")), to 1673 let single-homing PEs benefit from load balancing 1675 * procedures related to processing Ethernet A-D routes for the 1676 purpose of a backup path (Section 8.4 ("Aliasing and Backup 1677 Path")), to let single-homing PEs benefit from the corresponding 1678 convergence improvement 1680 9. Determining Reachability to Unicast MAC Addresses 1682 PEs forward packets that they receive based on the destination MAC 1683 address. This implies that PEs must be able to learn how to reach a 1684 given destination unicast MAC address. 1686 There are two components to MAC address learning -- "local learning" 1687 and "remote learning": 1689 9.1. Local Learning 1691 A particular PE must be able to learn the MAC addresses from the CEs 1692 that are connected to it. This is referred to as local learning. 1694 The PEs in a particular EVPN instance MUST support local data-plane 1695 learning using standard IEEE Ethernet learning procedures. A PE must 1696 be capable of learning MAC addresses in the data plane when it 1697 receives packets such as the following from the CE network: 1699 * DHCP requests 1700 * An ARP Request for its own MAC 1701 * An ARP Request for a peer 1703 Alternatively, PEs MAY learn the MAC addresses of the CEs in the 1704 control plane or via management-plane integration between the PEs and 1705 the CEs. 1707 There are applications where a MAC address that is reachable via a 1708 given PE on a locally attached segment (e.g., with ESI X) may move, 1709 such that it becomes reachable via another PE on another segment 1710 (e.g., with ESI Y). This is referred to as "MAC Mobility". 1711 Procedures to support this are described in Section 15 ("MAC 1712 Mobility"). 1714 9.2. Remote Learning 1716 A particular PE must be able to determine how to send traffic to MAC 1717 addresses that belong to or are behind CEs connected to other PEs, 1718 i.e., to remote CEs or hosts behind remote CEs. We call such MAC 1719 addresses "remote" MAC addresses. 1721 This document requires a PE to learn remote MAC addresses in the 1722 control plane. In order to achieve this, each PE advertises the MAC 1723 addresses it learns from its locally attached CEs in the control 1724 plane, to all the other PEs in that EVPN instance, using MP-BGP and, 1725 specifically, the MAC/IP Advertisement route. 1727 9.2.1. Constructing MAC/IP Address Advertisement 1729 BGP is extended to advertise these MAC addresses using the MAC/IP 1730 Advertisement route type in the EVPN NLRI. 1732 The RD MUST be set per Section 7.9. 1734 The Ethernet Segment Identifier is set to the 10-octet ESI described 1735 in Section 5 ("Ethernet Segment"). 1737 The Ethernet Tag ID is set as defined in Section 6. 1739 The MAC Address Length field is in bits, and it is set to 48. MAC 1740 address length values other than 48 bits are outside the scope of 1741 this document. The encoding of a MAC address MUST be the 6-octet MAC 1742 address specified by [IEEE.802.1Q_2014] and [IEEE.802.1D_2004]. 1744 The IP Address field is optional. By default, the IP Address Length 1745 field is set to 0, and the IP Address field is omitted from the 1746 route. When a valid IP address needs to be advertised, it is then 1747 encoded in this route. When an IP address is present, the IP Address 1748 Length field is in bits, and it is set to 32 or 128 bits. Other IP 1749 Address Length values are outside the scope of this document. The 1750 encoding of an IP address MUST be either 4 octets for IPv4 or 1751 16 octets for IPv6. The Length field of the EVPN NLRI (which is in 1752 octets and is described in Section 7) is sufficient to determine 1753 whether an IP address is encoded in this route and, if so, whether 1754 the encoded IP address is IPv4 or IPv6. 1756 The MPLS Label1 field is encoded as 3 octets, where the high-order 1757 20 bits contain the label value. The MPLS Label1 MUST be downstream 1758 assigned, and it is associated with the MAC address being advertised 1759 by the advertising PE. The advertising PE uses this label when it 1760 receives an MPLS-encapsulated packet to perform forwarding based on 1761 the destination MAC address toward the CE. The forwarding procedures 1762 are specified in Sections 13 and 14. 1764 A PE may advertise the same single EVPN label for all MAC addresses 1765 in a given MAC-VRF. This label assignment is referred to as a per 1766 MAC-VRF label assignment. Alternatively, a PE may advertise a unique 1767 EVPN label per combination. This label 1768 assignment is referred to as a per label 1769 assignment. As a third option, a PE may advertise a unique EVPN 1770 label per combination. This label assignment is 1771 referred to as a per label assignment. As a 1772 fourth option, a PE may advertise a unique EVPN label per MAC 1773 address. This label assignment is referred to as a per MAC label 1774 assignment. All of these label assignment methods have their 1775 trade-offs. The choice of a particular label assignment methodology 1776 is purely local to the PE that originates the route. 1778 An assignment per MAC-VRF label requires the least number of EVPN 1779 labels but requires a MAC lookup in addition to an MPLS lookup on an 1780 egress PE for forwarding. On the other hand, a unique label per 1781 or a unique label per MAC allows an egress PE to 1782 forward a packet that it receives from another PE, to the connected 1783 CE, after looking up only the MPLS labels without having to perform a 1784 MAC lookup. This includes the capability to perform appropriate VLAN 1785 ID translation on egress to the CE. 1787 The MPLS Label2 field is an optional field. If it is present, then 1788 it is encoded as 3 octets, where the high-order 20 bits contain the 1789 label value. 1791 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 1792 be set to the IPv4 or IPv6 address of the advertising PE. 1794 The BGP advertisement for the MAC/IP Advertisement route MUST also 1795 carry one or more Route Target (RT) extended communities. RTs may be 1796 configured (as in IP VPNs) or may be derived automatically in the 1797 "Unique VLAN EVPN" case from the Ethernet Tag (VLAN ID), as described 1798 in Section 7.10.1. 1800 It is to be noted that this document does not require PEs to create 1801 forwarding state for remote MACs when they are learned in the control 1802 plane. When this forwarding state is actually created is a local 1803 implementation matter. 1805 9.2.2. Route Resolution 1807 If the Ethernet Segment Identifier field in a received MAC/IP 1808 Advertisement route is set to the reserved ESI value of 0 or MAX-ESI, 1809 then if the receiving PE decides to install forwarding state for the 1810 associated MAC address, it MUST be based on the MAC/IP Advertisement 1811 route alone. 1813 If the Ethernet Segment Identifier field in a received MAC/IP 1814 Advertisement route is set to a non-reserved ESI, and the receiving 1815 PE is locally attached to the same ESI, then the PE does not alter 1816 its forwarding state based on the received route. This ensures that 1817 local routes are preferred to remote routes. 1819 If the Ethernet Segment Identifier field in a received MAC/IP 1820 Advertisement route is set to a non-reserved ESI, then if the 1821 receiving PE decides to install forwarding state for the associated 1822 MAC address, it MUST be when both the MAC/IP Advertisement route AND 1823 the associated set of Ethernet A-D per ES routes have been received. 1824 The dependency of MAC route installation on Ethernet A-D per ES 1825 routes is to ensure that MAC routes don't get accidentally installed 1826 during a mass withdraw period. 1828 To illustrate this with an example, consider two PEs (PE1 and PE2) 1829 connected to a multihomed Ethernet segment ES1. All-Active 1830 redundancy mode is assumed. A given MAC address M1 is learned by PE1 1831 but not PE2. On PE3, the following states may arise: 1833 T1 When the MAC/IP Advertisement route from PE1 and the set of 1834 Ethernet A-D per ES routes and Ethernet A-D per EVI routes from 1835 PE1 and PE2 are received, PE3 can forward traffic destined to 1836 M1 to both PE1 and PE2. 1838 T2 If after T1 PE1 withdraws its set of Ethernet A-D per ES 1839 routes, then PE3 forwards traffic destined to M1 to PE2 only. 1841 T2' If after T1 PE2 withdraws its set of Ethernet A-D per ES 1842 routes, then PE3 forwards traffic destined to M1 to PE1 only. 1844 T2'' If after T1 PE1 withdraws its MAC/IP Advertisement route, then 1845 PE3 treats traffic to M1 as unknown unicast. 1847 T3 PE2 also advertises a MAC route for M1, and then PE1 withdraws 1848 its MAC route for M1. PE3 continues forwarding traffic 1849 destined to M1 to both PE1 and PE2. In other words, despite M1 1850 withdrawal by PE1, PE3 forwards the traffic destined to M1 to 1851 both PE1 and PE2. This is because a flow from the CE, 1852 resulting in M1 traffic getting hashed to PE1, can get 1853 terminated, resulting in M1 being aged out in PE1; however, M1 1854 can be reachable by both PE1 and PE2. 1856 10. ARP and ND 1858 The IP Address field in the MAC/IP Advertisement route may optionally 1859 carry one of the IP addresses associated with the MAC address. This 1860 provides an option that can be used to minimize the flooding of ARP 1861 or Neighbor Discovery (ND) messages over the MPLS network and to 1862 remote CEs. This option also minimizes ARP (or ND) message 1863 processing on end-stations/hosts connected to the EVPN network. A PE 1864 may learn the IP address associated with a MAC address in the control 1865 or management plane between the CE and the PE. Or, it may learn this 1866 binding by snooping certain messages to or from a CE. When a PE 1867 learns the IP address associated with a MAC address of a locally 1868 connected CE, it may advertise this address to other PEs by including 1869 it in the MAC/IP Advertisement route. The IP address may be an IPv4 1870 address encoded using 4 octets or an IPv6 address encoded using 1871 16 octets. For ARP and ND purposes, the IP Address Length field MUST 1872 be set to 32 for an IPv4 address or 128 for an IPv6 address. 1874 If there are multiple IP addresses associated with a MAC address, 1875 then multiple MAC/IP Advertisement routes MUST be generated, one for 1876 each IP address. For instance, this may be the case when there are 1877 both an IPv4 and an IPv6 address associated with the same MAC address 1878 for dual-IP-stack scenarios. When the IP address is dissociated with 1879 the MAC address, then the MAC/IP Advertisement route with that 1880 particular IP address MUST be withdrawn. 1882 Note that a MAC-only route can be advertised along with, but 1883 independent from, a MAC/IP route for scenarios where the MAC learning 1884 over an access network/node is done in the data plane and independent 1885 from ARP snooping that generates a MAC/IP route. In such scenarios, 1886 when the ARP entry times out and causes the MAC/IP to be withdrawn, 1887 then the MAC information will not be lost. In scenarios where the 1888 host MAC/IP is learned via the management or control plane, then the 1889 sender PE may only generate and advertise the MAC/IP route. If the 1890 receiving PE receives both the MAC-only route and the MAC/IP route, 1891 then when it receives a withdraw message for the MAC/IP route, it 1892 MUST delete the corresponding entry from the ARP table but not the 1893 MAC entry from the MAC-VRF table, unless it receives a withdraw 1894 message for the MAC-only route. 1896 When a PE receives an ARP Request for an IP address from a CE, and if 1897 the PE has the MAC address binding for that IP address, the PE SHOULD 1898 perform ARP proxy by responding to the ARP Request. 1900 In the same way, when a PE receives a Neighbor Solicitation for an IP 1901 address from a CE, the PE SHOULD perform ND proxy and respond if the 1902 PE has the binding information for the IP. 1904 10.1. Default Gateway 1906 When a PE needs to perform inter-subnet forwarding where each subnet 1907 is represented by a different broadcast domain (e.g., a different 1908 VLAN), the inter-subnet forwarding is performed at Layer 3, and the 1909 PE that performs such a function is called the default gateway for 1910 the EVPN instance. In this case, when the PE receives an ARP Request 1911 for the IP address configured as the default gateway address, the PE 1912 originates an ARP Reply. 1914 Each PE that acts as a default gateway for a given EVPN instance MAY 1915 advertise in the EVPN control plane its default gateway MAC address 1916 using the MAC/IP Advertisement route, and each such PE indicates that 1917 such a route is associated with the default gateway. This is 1918 accomplished by requiring the route to carry the Default Gateway 1919 extended community defined in Section 7.8 ("Default Gateway Extended 1920 Community"). The ESI field is set to zero when advertising the MAC 1921 route with the Default Gateway extended community. 1923 The IP Address field of the MAC/IP Advertisement route is set to the 1924 default gateway IP address for that subnet (e.g., an EVPN instance). 1925 For a given subnet (e.g., a VLAN or EVPN instance), the default 1926 gateway IP address is the same across all the participant PEs. The 1927 inclusion of this IP address enables the receiving PE to check its 1928 configured default gateway IP address against the one received in the 1929 MAC/IP Advertisement route for that subnet (or EVPN instance), and if 1930 there is a discrepancy, then the PE SHOULD notify the operator and 1931 log an error message. 1933 Unless it is known a priori (by means outside of this document) that 1934 all PEs of a given EVPN instance act as a default gateway for that 1935 EVPN instance, the MPLS label MUST be set to a valid downstream 1936 assigned label. 1938 Furthermore, even if all PEs of a given EVPN instance do act as a 1939 default gateway for that EVPN instance, but only some, but not all, 1940 of these PEs have sufficient (routing) information to provide 1941 inter-subnet routing for all the inter-subnet traffic originated 1942 within the subnet associated with the EVPN instance, then when such a 1943 PE advertises in the EVPN control plane its default gateway MAC 1944 address using the MAC/IP Advertisement route and indicates that such 1945 a route is associated with the default gateway, the route MUST carry 1946 a valid downstream assigned label. 1948 If all PEs of a given EVPN instance act as a default gateway for that 1949 EVPN instance, and the same default gateway MAC address is used 1950 across all gateway devices, then no such advertisement is needed. 1951 However, if each default gateway uses a different MAC address, then 1952 each default gateway needs to be aware of other gateways' MAC 1953 addresses and thus the need for such an advertisement. This is 1954 called MAC address aliasing, since a single default gateway can be 1955 represented by multiple MAC addresses. 1957 Each PE that receives this route and imports it as per procedures 1958 specified in this document follows the procedures in this section 1959 when replying to ARP Requests that it receives. 1961 Each PE that acts as a default gateway for a given EVPN instance that 1962 receives this route and imports it as per procedures specified in 1963 this document MUST create MAC forwarding state that enables it to 1964 apply IP forwarding to the packets destined to the MAC address 1965 carried in the route. 1967 10.1.1. Best Path selection for Default Gateway 1969 Default gateway MAC address that is assigned to an IRB interface (for 1970 a subnet) in a PE MUST be unique in context of that subnet. In other 1971 words, the same MAC address cannot be used by a host either 1972 intentionally or accidently. Therefore, in case such conflicts 1973 arises, there needs to be scheme to detect it and resolve it. In 1974 order to properly detect such conflicts, the following BGP best path 1975 selection MUST be applied. 1977 * When comparing two routes, the route which has Default Gateway 1978 extended community is preferred over a route which does not have 1979 the extended comunity. The PE that has advertised the MAC route 1980 without Default Gateway extended community, upon receiving the 1981 route with Default Gateway extended community, SHALL withdraw its 1982 route and raise an alarm. 1984 * When comparing two routes where both routes have the Default 1985 Gateway extended community, normal BGP best path processing is be 1986 applied. 1988 * When comparing local and remote routes with Default Gateway 1989 extended community, the local route is always preferred. 1991 * MAC Mobility extended community SHALL NOT be attached to routes 1992 which also have Default Gateway extended community on the sending 1993 side and SHALL be ignored on the receiving side. 1995 11. Handling of Multi-destination Traffic 1997 Procedures are required for a given PE to flood broadcast or 1998 multicast traffic received from a CE and with a given Ethernet tag to 1999 the other PEs in the associated [EVI, BD] (EVPN instance). In 2000 certain scenarios, as described in Section 12 ("Processing of Unknown 2001 Unicast Packets"), a given PE may also need to flood unknown unicast 2002 traffic to other PEs. 2004 The PEs in a particular EVPN instance may use ingress replication, 2005 P2MP LSPs, or MP2MP LSPs to send unknown unicast, broadcast, or 2006 multicast traffic to other PEs. 2008 Each PE MUST advertise an "Inclusive Multicast Ethernet Tag route" to 2009 enable the above. The following subsection provides the procedures 2010 to construct the Inclusive Multicast Ethernet Tag route. Subsequent 2011 subsections describe its usage in further detail. 2013 11.1. Constructing Inclusive Multicast Ethernet Tag Route 2015 The RD MUST be set per Section 7.9. 2017 The Ethernet Tag ID is set as defined in Section 6. 2019 The Originating Router's IP Address field value MUST be set to an IP 2020 address of the PE that should be common for all the EVIs on the PE 2021 (e.g., this address may be the PE's loopback address). The IP 2022 Address Length field is in bits. 2024 The Next Hop field of the MP_REACH_NLRI attribute of the route MUST 2025 be set to the IPv4 or IPv6 address of the advertising PE. 2027 The BGP advertisement for the Inclusive Multicast Ethernet Tag route 2028 MUST also carry one or more Route Target (RT) extended communities. 2029 The assignment of RTs as described in Section 7.10 MUST be followed. 2031 11.2. P-Tunnel Identification 2033 In order to identify the P-tunnel used for sending broadcast, unknown 2034 unicast, or multicast traffic, the Inclusive Multicast Ethernet Tag 2035 route MUST carry a Provider Multicast Service Interface (PMSI) Tunnel 2036 attribute as specified in [RFC6514]. 2038 Depending on the technology used for the P-tunnel for the EVPN 2039 instance on the PE, the PMSI Tunnel attribute of the Inclusive 2040 Multicast Ethernet Tag route is constructed as follows. 2042 * If the PE that originates the advertisement uses a P-multicast 2043 tree for the P-tunnel for EVPN, the PMSI Tunnel attribute MUST 2044 contain the identity of the tree (note that the PE could create 2045 the identity of the tree prior to the actual instantiation of the 2046 tree). 2048 * A PE that uses a P-multicast tree for the P-tunnel MAY aggregate 2049 two or more Broadcast Domains (BDs) present on the PE onto the 2050 same tree. In this case, in addition to carrying the identity of 2051 the tree, the PMSI Tunnel attribute MUST carry an MPLS label, 2052 which the PE has bound uniquely to the BD associated with this 2053 update (as determined by its RTs and Ethernet Tag ID). The 2054 assigned MPLS label is upstream allocated unless the procedures in 2055 section 19 (Use of Domain-wide Common Block (DCB) Labels) are 2056 followed. If the PE has already advertised Inclusive Multicast 2057 Ethernet Tag routes for two or more BDs that it now desires to 2058 aggregate, then the PE MUST re-advertise those routes. The 2059 re-advertised routes MUST be the same as the original ones, except 2060 for the PMSI Tunnel attribute and the label carried in that 2061 attribute. 2063 * If the PE that originates the advertisement uses ingress 2064 replication for the P-tunnel for EVPN, the route MUST include the 2065 PMSI Tunnel attribute with the Tunnel Type set to Ingress 2066 Replication and the Tunnel Identifier set to a routable address of 2067 the PE. The PMSI Tunnel attribute MUST carry a downstream 2068 assigned MPLS label. This label is used to demultiplex the 2069 broadcast, multicast, or unknown unicast EVPN traffic received 2070 over an MP2P tunnel by the PE. 2072 12. Processing of Unknown Unicast Packets 2074 The procedures in this document do not require the PEs to flood 2075 unknown unicast traffic to other PEs. If PEs learn CE MAC addresses 2076 via a control-plane protocol, the PEs can then distribute MAC 2077 addresses via BGP, and all unicast MAC addresses will be learned 2078 prior to traffic to those destinations. 2080 However, if a destination MAC address of a received packet is not 2081 known by the PE, the PE may have to flood the packet. When flooding, 2082 one must take into account "split-horizon forwarding" as follows: The 2083 principles behind the following procedures are borrowed from the 2084 split-horizon forwarding rules in VPLS solutions [RFC4761] [RFC4762]. 2085 When a PE capable of flooding (say PEx) receives an unknown 2086 destination MAC address, it floods the frame. If the frame arrived 2087 from an attached CE, PEx must send a copy of that frame on every 2088 Ethernet segment (belonging to that EVI) for which it is the DF, 2089 other than the Ethernet segment on which it received the frame. In 2090 addition, the PE must flood the frame to all other PEs participating 2091 in that EVPN instance. If, on the other hand, the frame arrived from 2092 another PE (say PEy), PEx must send a copy of the packet on each 2093 Ethernet segment (belonging to that EVI) for which it is the DF. PEx 2094 MUST NOT send the frame to other PEs, since PEy would have already 2095 done so. Split-horizon forwarding rules apply to unknown MAC 2096 addresses. 2098 Whether or not to flood packets to unknown destination MAC addresses 2099 should be an administrative choice, depending on how learning happens 2100 between CEs and PEs. 2102 The PEs in a particular EVPN instance may use ingress replication 2103 using RSVP-TE P2P LSPs or LDP MP2P LSPs for sending unknown unicast 2104 traffic to other PEs. Or, they may use RSVP-TE P2MP or LDP P2MP for 2105 sending such traffic to other PEs. 2107 12.1. Ingress Replication 2109 If ingress replication is in use, the P-tunnel attribute, carried in 2110 the Inclusive Multicast Ethernet Tag routes for the EVPN instance, 2111 specifies the downstream label that the other PEs can use to send 2112 unknown unicast, multicast, or broadcast traffic for that EVPN 2113 instance to this particular PE. 2115 The PE that receives a packet with this particular MPLS label MUST 2116 treat the packet as a broadcast, multicast, or unknown unicast 2117 packet. Further, if the MAC address is a unicast MAC address, the PE 2118 MUST treat the packet as an unknown unicast packet. 2120 12.2. P2MP MPLS LSPs 2122 The procedures for using P2MP or MP2MP LSPs are very similar to the 2123 VPLS procedures described in [RFC7117]. The P-tunnel attribute used 2124 by a PE for sending unknown unicast, broadcast, or multicast traffic 2125 for a particular EVPN instance is advertised in the Inclusive 2126 Multicast Ethernet Tag route as described in Section 11 ("Handling of 2127 Multi-destination Traffic"). 2129 The P-tunnel attribute specifies the P2MP or MP2MP LSP identifier. 2130 This is the equivalent of an Inclusive tree as described in 2131 [RFC7117]. Note that multiple BDs in the same or different EVIs may 2132 use the same P2MP or MP2MP LSP, using upstream labels [RFC7117] or 2133 DCB labels [I-D.ietf-bess-mvpn-evpn-aggregation-label]. This is the 2134 equivalent of an Aggregate Inclusive tree [RFC7117]. When P2MP or 2135 MP2MP LSPs are used for flooding unknown unicast traffic, packet 2136 reordering is possible. 2138 The PE that receives a packet on the P2MP or MP2MP LSP specified in 2139 the PMSI Tunnel attribute MUST treat the packet as a broadcast, 2140 multicast, or unknown unicast packet. Further, if the MAC address is 2141 a unicast MAC address, the PE MUST treat the packet as an unknown 2142 unicast packet. 2144 13. Forwarding Unicast Packets 2146 This section describes procedures for forwarding unicast packets by 2147 PEs, where such packets are received from either directly connected 2148 CEs or some other PEs. 2150 13.1. Forwarding Packets Received from a CE 2152 When a PE receives a packet from a CE with a given Ethernet Tag, it 2153 must first look up the packet's source MAC address. In certain 2154 environments that enable MAC security, the source MAC address MAY be 2155 used to validate the host identity and determine that traffic from 2156 the host can be allowed into the network. Source MAC lookup MAY also 2157 be used for local MAC address learning. 2159 If the PE decides to forward the packet, the destination MAC address 2160 of the packet must be looked up. If the PE has received MAC address 2161 advertisements for this destination MAC address from one or more 2162 other PEs or has learned it from locally connected CEs, the MAC 2163 address is considered a known MAC address. Otherwise, it is 2164 considered an unknown MAC address. 2166 For known MAC addresses, the PE forwards this packet to one of the 2167 remote PEs or to a locally attached CE. When forwarding to a remote 2168 PE, the packet is encapsulated in the EVPN MPLS label advertised by 2169 the remote PE, for that MAC address, and in the MPLS LSP label stack 2170 to reach the remote PE. 2172 If the MAC address is unknown and if the administrative policy on the 2173 PE requires flooding of unknown unicast traffic, then: 2175 * The PE MUST flood the packet to other PEs. The PE MUST first 2176 encapsulate the packet in the ESI MPLS label as described in 2177 Section 8.3. 2178 If ingress replication is used, the packet MUST be replicated to 2179 each remote PE, with the VPN label being the MPLS label advertised 2180 by the remote PE in a PMSI Tunnel attribute in the Inclusive 2181 Multicast Ethernet Tag route for the [EVI, BD] associated with the 2182 received packet's Ethernet tag. 2183 If P2MP LSPs are being used, the packet MUST be sent on the P2MP 2184 LSP of which the PE is the root, for the [EVI, BD] associated with 2185 the received packet's Ethernet tag. If the same P2MP LSP is used 2186 for all the BD's in the EVI, then all the PEs in the EVI MUST be 2187 the leaves of the P2MP LSP. If a different P2MP LSP is used for a 2188 given BD in the EVI, then only the PEs in that BD MUST be the 2189 leaves of the P2MP LSP. The packet MUST be encapsulated in the 2190 P2MP LSP label stack. 2192 If the MAC address is unknown, then, if the administrative policy on 2193 the PE does not allow flooding of unknown unicast traffic: 2195 * the PE MUST drop the packet. 2197 13.2. Forwarding Packets Received from a Remote PE 2199 This section describes the procedures for forwarding known and 2200 unknown unicast packets received from a remote PE. 2202 13.2.1. Unknown Unicast Forwarding 2204 When a PE receives an MPLS packet from a remote PE, then, after 2205 processing the MPLS label stack, if the top MPLS label ends up being 2206 a P2MP LSP label associated with an EVPN instance or -- in the case 2207 of ingress replication -- the downstream label advertised in the 2208 P-tunnel attribute, and after performing the split-horizon procedures 2209 described in Section 8.3: 2211 * If the PE is the designated forwarder of BUM traffic on a 2212 particular set of ESes for the [EVI, BD], the default behavior is 2213 for the PE to flood that traffic to these ESes. In other words, 2214 the default behavior is for the PE to assume that for BUM traffic 2215 it is not required to perform a destination MAC address lookup. 2216 As an option, the PE may perform a destination MAC lookup to flood 2217 the packet to only a subset of these ESes. For instance, the PE 2218 may decide to not flood a BUM packet on certain Ethernet segments 2219 even if it is the DF on the Ethernet segment, based on 2220 administrative policy. 2222 * If the PE is not the designated forwarder for any ES associated 2223 with the [EVI, BD], the default behavior is for it to drop the BUM 2224 traffic. 2226 13.2.2. Known Unicast Forwarding 2228 If the top MPLS label ends up being an EVPN label that was advertised 2229 in the unicast MAC advertisements, then the PE either forwards the 2230 packet based on CE next-hop forwarding information associated with 2231 the label or does a destination MAC address lookup to forward the 2232 packet to a CE. 2234 14. Load Balancing of Unicast Packets 2236 This section specifies the load-balancing procedures for sending 2237 known unicast packets to a multihomed CE. 2239 14.1. Load Balancing of Traffic from a PE to Remote CEs 2241 When a remote PE imports a MAC/IP Advertisement route for a given ES 2242 in a MAC-VRF, it MUST examine all imported Ethernet A-D routes for 2243 that ESI in order to determine the load- balancing characteristics of 2244 the Ethernet segment. 2246 14.1.1. Single-Active Redundancy Mode 2248 For a given ES, if a remote PE has imported the set of Ethernet A-D 2249 per ES routes from at least one PE, where the "Single-Active" flag in 2250 the ESI Label extended community is set, then that remote PE MUST 2251 deduce that the ES is operating in Single-Active redundancy mode. 2253 This means that for a given [EVI, BD], a given MAC address is only 2254 reachable only via the PE announcing the associated MAC/IP 2255 Advertisement route - this PE will also have advertised an Ethernet 2256 A-D per EVI route for that [EVI, BD] with an L2-Attr extended 2257 community in which the P bit is set. I.e., the Primary DF Elected PE 2258 is also responsible for sending known unicast frames to the CE and 2259 receiving unicast and BUM frames from it. Similarly, the Backup DF 2260 Elected PE will have advertised an Ethernet AD per EVI route for 2261 [EVI, BD] with an L2-Attr extended community in which the B bit is 2262 set. 2264 If the Primary DF Elected PE loses connectivity to the CE it SHOULD 2265 withdraw its set of Ethernet A-D per ES routes for the affected ES 2266 prior to withdrawing the affected MAC/IP Advertisement routes. The 2267 Backup DF Elected PE (which is now the Primary DF Elected PE) needs 2268 to advertise an Ethernet A-D per EVI route for [EVI, BD] with an 2269 L2-Attr extended community in which the P bit is set. Furthermore, 2270 the new Backup DF Elected PE needs to advertise an Ethernet A-D per 2271 EVI route for [EVI, BD] with an L2-Attr extended community in which 2272 the B bit is set. 2274 A remote PE SHOULD use the Primary DF Elected PE's withdrawal of its 2275 set of Ethernet A-D per ES routes as a trigger to update its 2276 forwarding entries for the associated MAC addresses to point at the 2277 Backup DF Elected PE. As the Backup DF Elected PE starts learning 2278 the MAC addresses over its attached ES, it will start sending MAC/IP 2279 Advertisement routes while the failed PE withdraws its routes. This 2280 mechanism minimizes the flooding of traffic during fail-over events. 2282 14.1.2. All-Active Redundancy Mode 2284 For a given ES, if the remote PE has imported the set of Ethernet A-D 2285 per ES routes from one or more PEs and none of them have the 2286 "Single-Active" flag in the ESI Label extended community set, then 2287 the remote PE MUST deduce that the ES is operating in All-Active 2288 redundancy mode. A remote PE that receives a MAC/IP Advertisement 2289 route with a non-reserved ESI SHOULD consider the advertised MAC 2290 address to be reachable via all PEs that have advertised reachability 2291 to that MAC address's EVI/ES/Ethernet Tag ID via the combination of 2292 an Ethernet A-D per EVI route for that EVI/ES/Ethernet Tag ID AND an 2293 Ethernet A-D per ES route for that ES. The remote PE MUST use 2294 received MAC/IP Advertisement routes and Ethernet A-D per EVI/per ES 2295 routes to construct the set of next hops for the advertised MAC 2296 address. 2298 Each next hop comprises an MPLS label stack that is to be used to 2299 reach a given egress PE and allow it to forward a packet. The 2300 portion of the MPLS label stack that is to be used by that egress PE 2301 to forward a packet is constructed by the remote PE as follows: 2303 * If a MAC/IP Advertisement route was received from that PE, then 2304 its label stack MUST be used in the next hop. 2306 * Otherwise, the label stack from the Ethernet A-D per EVI route 2307 that matches the MAC address' EVI/ES/Ethernet Tag ID MUST be used 2308 in the next hop. 2310 The following example explains the above. 2312 Consider a CE (CE1) that is dual-homed to two PEs (PE1 and PE2) on a 2313 LAG interface (ES1), and is sending packets with source MAC address 2314 MAC1 on VLAN1 (mapped to EVI1). A remote PE, say PE3, is able to 2315 learn that MAC1 is reachable via PE1 and PE2. Both PE1 and PE2 may 2316 advertise MAC1 if they receive packets with MAC1 from CE1. If this 2317 is not the case, and if MAC1 is advertised only by PE1, PE3 still 2318 considers MAC1 as reachable via both PE1 and PE2, as both PE1 and PE2 2319 advertise a set of Ethernet A-D per ES routes for ES1 as well as an 2320 Ethernet A-D per EVI route for . 2322 The MPLS label stack to send the packets to PE1 is the MPLS LSP stack 2323 to get to PE1 (at the top of the stack) followed by the EVPN label 2324 advertised by PE1 for CE1's MAC. 2326 The MPLS label stack to send packets to PE2 is the MPLS LSP stack to 2327 get to PE2 (at the top of the stack) followed by the MPLS label in 2328 the Ethernet A-D route advertised by PE2 for , if PE2 has 2329 not advertised MAC1 in BGP. 2331 We will refer to these label stacks as MPLS next hops. 2333 The remote PE (PE3) can now load balance the traffic it receives from 2334 its CEs, destined for CE1, between PE1 and PE2. PE3 may use N-tuple 2335 flow information to hash traffic into one of the MPLS next hops for 2336 load balancing of IP traffic. Alternatively, PE3 may rely on the 2337 source MAC addresses for load balancing. 2339 Note that once PE3 decides to send a particular packet to PE1 or PE2, 2340 it can pick one out of multiple possible paths to reach the 2341 particular remote PE using regular MPLS procedures. For instance, if 2342 the tunneling technology is based on RSVP-TE LSPs and PE3 decides to 2343 send a particular packet to PE1, then PE3 can choose from multiple 2344 RSVP-TE LSPs that have PE1 as their destination. 2346 When PE1 or PE2 receives the packet destined for CE1 from PE3, if the 2347 packet is a known unicast, it is forwarded to CE1. 2349 14.2. Load Balancing of Traffic between a PE and a Local CE 2351 A CE may be configured with more than one interface connected to 2352 different PEs or the same PE for load balancing, using a technology 2353 such as a LAG. The PE(s) and the CE can load balance traffic onto 2354 these interfaces using one of the following mechanisms. 2356 14.2.1. Data-Plane Learning 2358 Consider that the PEs perform data-plane learning for local MAC 2359 addresses learned from local CEs. This enables the PE(s) to learn a 2360 particular MAC address and associate it with one or more interfaces, 2361 if the technology between the PE and the CE supports multipathing. 2362 The PEs can now load balance traffic destined to that MAC address on 2363 the multiple interfaces. 2365 Whether the CE can load balance traffic that it generates on the 2366 multiple interfaces is dependent on the CE implementation. 2368 14.2.2. Control-Plane Learning 2370 The CE can be a host that advertises the same MAC address using a 2371 control protocol on all interfaces. This enables the PE(s) to learn 2372 the host's MAC address and associate it with all interfaces. The PEs 2373 can now load balance traffic destined to the host on all these 2374 interfaces. The host can also load balance the traffic it generates 2375 onto these interfaces, and the PE that receives the traffic employs 2376 EVPN forwarding procedures to forward the traffic. 2378 15. MAC Mobility 2380 It is possible for a given host or end-station (as defined by its MAC 2381 address) to move from one Ethernet segment to another; this is 2382 referred to as 'MAC Mobility' or 'MAC move', and it is different from 2383 the multihoming situation in which a given MAC address is reachable 2384 via multiple PEs for the same Ethernet segment. In a MAC move, there 2385 would be two sets of MAC/IP Advertisement routes -- one set with the 2386 new Ethernet segment and one set with the previous Ethernet segment 2387 -- and the MAC address would appear to be reachable via each of these 2388 segments. 2390 In order to allow all of the PEs in the EVPN instance to correctly 2391 determine the current location of the MAC address, all advertisements 2392 of it being reachable via the previous Ethernet segment MUST be 2393 withdrawn by the PEs, for the previous Ethernet segment, that had 2394 advertised it. 2396 If local learning is performed using the data plane, these PEs will 2397 not be able to detect that the MAC address has moved to another 2398 Ethernet segment, and the receipt of MAC/IP Advertisement routes, 2399 with the MAC Mobility extended community, from other PEs serves as 2400 the trigger for these PEs to withdraw their advertisements. If local 2401 learning is performed using the control or management planes, these 2402 interactions serve as the trigger for these PEs to withdraw their 2403 advertisements. 2405 In a situation where there are multiple moves of a given MAC, 2406 possibly between the same two Ethernet segments, there may be 2407 multiple withdrawals and re-advertisements. In order to ensure that 2408 all PEs in the EVPN instance receive all of these correctly through 2409 the intervening BGP infrastructure, introducing a sequence number 2410 into the MAC Mobility extended community is necessary. 2412 In order to process mobility events correctly, an implementation MUST 2413 handle scenarios in which sequence number wraparound occurs. 2415 Every MAC mobility event for a given MAC address will contain a 2416 sequence number that is set using the following rules: 2418 * A PE advertising a MAC address for the first time advertises it 2419 with no MAC Mobility extended community. 2421 * A PE detecting a locally attached MAC address for which it had 2422 previously received a MAC/IP Advertisement route with a different 2423 Ethernet segment identifier advertises the MAC address in a MAC/IP 2424 Advertisement route tagged with a MAC Mobility extended community 2425 with a sequence number one greater than the sequence number in the 2426 MAC Mobility extended community of the received MAC/IP 2427 Advertisement route. In the case of the first mobility event for 2428 a given MAC address, where the received MAC/IP Advertisement route 2429 does not carry a MAC Mobility extended community, the value of the 2430 sequence number in the received route is assumed to be 0 for the 2431 purpose of this processing. 2433 * A PE detecting a locally attached MAC address for which it had 2434 previously received a MAC/IP Advertisement route with the same 2435 non-zero Ethernet segment identifier advertises it with: 2437 1. no MAC Mobility extended community, if the received route did 2438 not carry said extended community. 2440 2. a MAC Mobility extended community with the sequence number 2441 equal to the highest of the sequence number(s) in the received 2442 MAC/IP Advertisement route(s), if the received route(s) is 2443 (are) tagged with a MAC Mobility extended community. 2445 * A PE detecting a locally attached MAC address for which it had 2446 previously received a MAC/IP Advertisement route with the same 2447 zero Ethernet segment identifier (single-homed scenarios) 2448 advertises it with a MAC Mobility extended community with the 2449 sequence number set properly. In the case of single-homed 2450 scenarios, there is no need for ESI comparison. ESI comparison is 2451 done for multihoming in order to prevent false detection of MAC 2452 moves among the PEs attached to the same multihomed site. 2454 A PE receiving a MAC/IP Advertisement route for a MAC address with a 2455 different Ethernet segment identifier and a higher sequence number 2456 than that which it had previously advertised withdraws its MAC/IP 2457 Advertisement route. If two (or more) PEs advertise the same MAC 2458 address with the same sequence number but different Ethernet segment 2459 identifiers, a PE that receives these routes selects the route 2460 advertised by the PE with the lowest IP address as the best route. 2461 If the PE is the originator of the MAC route and it receives the same 2462 MAC address with the same sequence number that it generated, it will 2463 compare its own IP address with the IP address of the remote PE and 2464 will select the lowest IP. If its own route is not the best one, it 2465 will withdraw the route. 2467 15.1. MAC Duplication Issue 2469 A situation may arise where the same MAC address is learned by 2470 different PEs in the same VLAN because of two (or more) hosts being 2471 misconfigured with the same (duplicate) MAC address. In such a 2472 situation, the traffic originating from these hosts would trigger 2473 continuous MAC moves among the PEs attached to these hosts. It is 2474 important to recognize such a situation and avoid incrementing the 2475 sequence number (in the MAC Mobility extended community) to infinity. 2476 In order to remedy such a situation, a PE that detects a MAC mobility 2477 event via local learning starts an M-second timer (with a default 2478 value of M = 180), and if it detects N MAC moves before the timer 2479 expires (with a default value of N = 5), it concludes that a 2480 duplicate-MAC situation has occurred. The PE MUST alert the operator 2481 and stop sending and processing any BGP MAC/IP Advertisement routes 2482 for that MAC address until a corrective action is taken by the 2483 operator. The values of M and N MUST be configurable to allow for 2484 flexibility in operator control. Note that the other PEs in the EVPN 2485 instance will forward the traffic for the duplicate MAC address to 2486 one of the PEs advertising the duplicate MAC address. 2488 15.2. Sticky MAC Addresses 2490 There are scenarios in which it is desired to configure some MAC 2491 addresses as static so that they are not subjected to MAC moves. In 2492 such scenarios, these MAC addresses are advertised with a MAC 2493 Mobility extended community where the static flag is set to 1 and the 2494 sequence number is set to zero. If a PE receives such advertisements 2495 and later learns the same MAC address(es) via local learning, then 2496 the PE MUST alert the operator. 2498 15.3. Loop Protection 2500 The EVPN MAC Duplication procedure in Section 15.1 prevents an 2501 endless EVPN MAC/IP route advertisement exchange for a duplicate MAC 2502 between two (or more) PEs. While this helps the control plane 2503 settle, in case there is backdoor link (loop) between two or more PEs 2504 attached to the same BD, BUM frames being sent by a CE are still 2505 endlessly looped within the BD through the backdoor link and among 2506 the PEs. This may cause unpredictable issues in the CEs connected to 2507 the affected BD. 2509 The EVPN MAC Duplication Mechanism in Section 15.1 MAY be extended 2510 with a Loop-protection action that is applied on the duplicate-MAC 2511 addresses. This additional mechanism resolves loops created by 2512 accidental or intentional backdoor links and SHOULD be enabled in all 2513 the PEs attached to the BD. 2515 After following the procedure in Section 15.1, when a PE detects a 2516 MAC M as duplicate, the PE behaves as follows: 2518 a) Stops advertising M and logs a duplicate event. 2520 b) Initializes a retry-timer, R seconds. 2522 c) Since Loop Protection is enabled, the PE executes a Loop 2523 Protection action, which we refer to as "Black-Holing" M. 2525 When the PE programs M as a Black-Hole MAC in the Bridge Table, M is 2526 no longer associated to the backdoor Attachment Circuit (AC), but to 2527 a Black-Hole destination. 2529 At this point and while M is in Black-Hole state: 2531 a) If a new frame is received (from the EVPN network or the backdoor 2532 AC) with MAC SA = M, the PE identifies M to be Black-Holed and 2533 discards the frame, ending the loop. 2535 b) Optionally, instead of simply discarding the frame with MAC SA = 2536 M, the PE MAY bring down the AC on which the offending frame is 2537 seen last. 2539 c) Optionally, any frame that arrives at the PE with MAC DA = M 2540 SHOULD be discarded too. 2542 When the retry-timer R for M expires, the PE flushes M from the 2543 Bridge Table and the process is restarted. In general, a Black-Hole 2544 MAC M can be flushed from the Bridge Table if any of the following 2545 events occur: 2547 * Retry-timer R for duplicate-MAC M expires (as discussed). R is 2548 initialized when M is detected as duplicate-MAC. Its value is 2549 configurable and SHOULD be at least three times the EVPN MAC 2550 Duplication M-timer window. 2552 * The operator manually flushes a Black-Hole MAC M. This should be 2553 done only if the conditions under which M was identified as 2554 duplicate have been cleared. 2556 * The remote PE withdraws the MAC/IP route for M and there are no 2557 other remote MAC/IP routes for M. 2559 * The remote PE sends a MAC/IP route update for M with the 2560 sticky-bit set (in the MAC Mobility extended community). 2562 16. Multicast and Broadcast 2564 The PEs in a particular EVPN instance may use ingress replication or 2565 P2MP or MP2MP LSPs to send multicast traffic to other PEs. 2567 16.1. Ingress Replication 2569 The PEs may use ingress replication for flooding BUM traffic as 2570 described in Section 11 ("Handling of Multi-destination Traffic"). A 2571 given broadcast packet must be sent to all the remote PEs. However, 2572 a given multicast packet for a multicast flow may be sent to only a 2573 subset of the PEs. Specifically, a given multicast flow may be sent 2574 to only those PEs that have receivers that are interested in the 2575 multicast flow. Determining which of the PEs have receivers for a 2576 given multicast flow is done using the procedures of 2577 [I-D.ietf-bess-evpn-igmp-mld-proxy]. 2579 16.2. P2MP or MP2MP LSPs 2581 A PE may use an "Inclusive" tree for sending a BUM packet. This 2582 terminology is borrowed from [RFC7117]. 2584 A variety of transport technologies may be used in the service 2585 provider (SP) network. For Inclusive P-multicast trees, these 2586 transport technologies include point-to-multipoint LSPs created by 2587 RSVP-TE or Multipoint LDP (mLDP) or BIER. 2589 16.2.1. Inclusive Trees 2591 An Inclusive tree allows the use of a single multicast distribution 2592 tree, referred to as an Inclusive P-multicast tree, in the SP network 2593 to carry all the multicast traffic from a specified set of EVPN 2594 instances on a given PE. A particular P-multicast tree can be set up 2595 to carry the traffic originated by sites belonging to a single EVPN 2596 instance, or to carry the traffic originated by sites belonging to 2597 several EVPN instances. The ability to carry the traffic of more 2598 than one EVPN instance on the same tree is termed 'Aggregation', and 2599 the tree is called an Aggregate Inclusive P-multicast tree or 2600 Aggregate Inclusive tree for short. The Aggregate Inclusive tree 2601 needs to include every PE that is a member of any of the EVPN 2602 instances that are using the tree. This implies that a PE may 2603 receive BUM traffic even if it doesn't have any receivers that are 2604 interested in receiving that traffic. 2606 An Inclusive or Aggregate Inclusive tree as defined in this document 2607 is a P2MP tree. A P2MP or MP2MP tree is used to carry traffic only 2608 for EVPN CEs that are connected to the PE that is the root of the 2609 tree. 2611 The procedures for signaling an Inclusive tree are the same as those 2612 in [RFC7117], with the VPLS A-D route replaced with the Inclusive 2613 Multicast Ethernet Tag route. The P-tunnel attribute [RFC7117] for 2614 an Inclusive tree is advertised with the Inclusive Multicast Ethernet 2615 Tag route as described in Section 11 ("Handling of Multi-destination 2616 Traffic"). Note that for an Aggregate Inclusive tree, a PE can 2617 "aggregate" multiple EVPN instances on the same P2MP LSP using 2618 upstream labels or DCB allocated labels 2619 [I-D.ietf-bess-mvpn-evpn-aggregation-label]. The procedures for 2620 aggregation are the same as those described in [RFC7117], with VPLS 2621 A-D routes replaced by EVPN Inclusive Multicast Ethernet Tag routes. 2623 17. Convergence 2625 This section describes failure recovery from different types of 2626 network failures. 2628 17.1. Transit Link and Node Failures between PEs 2630 The use of existing MPLS fast-reroute mechanisms can provide failure 2631 recovery on the order of 50 ms, in the event of transit link and node 2632 failures in the infrastructure that connects the PEs. 2634 17.2. PE Failures 2636 Consider a host CE1 that is dual-homed to PE1 and PE2. If PE1 fails, 2637 a remote PE, PE3, can discover this based on the failure of the BGP 2638 session. This failure detection can be in the sub-second range if 2639 Bidirectional Forwarding Detection (BFD) is used to detect BGP 2640 session failures. PE3 can update its forwarding state to start 2641 sending all traffic for CE1 to only PE2. 2643 17.3. PE-to-CE Network Failures 2645 If the connectivity between the multihomed CE and one of the PEs to 2646 which it is attached fails, the PE MUST withdraw the set of Ethernet 2647 A-D per ES routes that had been previously advertised for that ES. 2648 This enables the remote PEs to remove the MPLS next hop to this 2649 particular PE from the set of MPLS next hops that can be used to 2650 forward traffic to the CE. When the MAC entry on the PE ages out, 2651 the PE MUST withdraw the MAC address from BGP. 2653 When an EVI is decommissioned on an Ethernet segment the PE MUST 2654 withdraw the Ethernet A-D per EVI route(s) announced for that . In addition, the PE MUST also withdraw the MAC/IP Advertisement 2656 routes that are impacted by the decommissioning. 2658 The Ethernet A-D per ES routes should be used by an implementation to 2659 optimize the withdrawal of MAC/IP Advertisement routes. When a PE 2660 receives a withdrawal of a particular Ethernet A-D route from an 2661 advertising PE, it SHOULD consider all the MAC/IP Advertisement 2662 routes that are learned from the same ESI as in the Ethernet A-D 2663 route from the advertising PE as having been withdrawn. This 2664 optimizes the network convergence times in the event of PE-to-CE 2665 failures. 2667 18. Frame Ordering 2669 In a MAC address, if the value of the first nibble (bits 8 through 5) 2670 of the most significant octet of the destination MAC address (which 2671 follows the last MPLS label) happens to be 0x4 or 0x6, then the 2672 Ethernet frame can be misinterpreted as an IPv4 or IPv6 packet by 2673 intermediate P nodes performing ECMP based on deep packet inspection, 2674 thus resulting in load balancing packets belonging to the same flow 2675 on different ECMP paths and subjecting those packets to different 2676 delays. Therefore, packets belonging to the same flow can arrive at 2677 the destination out of order. This out-of-order delivery can happen 2678 during steady state in the absence of any failures, resulting in 2679 significant impact on network operations. 2681 In order to avoid frame misordering described in Section 18, the 2682 following network-wide rules are applied: 2684 * If a network uses deep packet inspection for its ECMP, then the 2685 "Preferred PW MPLS Control Word" [RFC4385] MUST be used with the 2686 value 0 (e.g., a 4-octet field with a value of zero) when sending 2687 unicast EVPN-encapsulated packets over an MP2P LSP. 2689 * When sending EVPN-encapsulated packets over a P2MP or P2P RSVP-TE 2690 LSP, then the control word SHOULD NOT be used. 2692 * When sending EVPN-encapsulated packets over a P2MP LSP (e.g., 2693 using mLDP signaling), then the control word SHOULD be used. 2695 * If a network uses entropy labels per [RFC6790], then the control 2696 word SHOULD NOT be used when sending EVPN-encapsulated packets 2697 over an MP2P LSP. 2699 18.1. Flow Label 2701 Flow label is used to add entropy to divisible flows, and creates 2702 ECMP load-balancing in the network. The Flow label MAY be used in 2703 EVPN networks to achieve better load-balancing in the network, when 2704 transit nodes perform deep packet inspection for ECMP hashing. The 2705 following rules apply: 2707 * When F-bit is set to 1, the PE announces the capability of both 2708 sending and receiving flow label for known unicast. 2710 If the PE is capable itself of supporting Flow Label, then: 2712 - upon receiving the F-bit set (F=1) from a remote PE, it MUST 2713 send known unicast packets to that PE with Flow labels; 2715 - alternately, upon receiving the F-bit unset (F=0) from a 2716 remote PE, it MUST NOT send known unicast packets to that PE 2717 with Flow labels. 2719 Receiving the F-bit set (F=1) from a remote PE has no effect when 2720 the PE itself does not support Flow label. 2722 * The Flow Label MUST NOT be used for EVPN-encapsulated BUM packets. 2724 * An ingress PE will push the Flow Label at the bottom of the stack 2725 of the EVPN-encapsulated known unicast packets sent to an egress 2726 PE that previously signaled F-bit set to 1. 2728 * If a PE receives a unicast packet with two labels, then it can 2729 differentiate between [VPN label + ESI label] and [VPN label + 2730 Flow label] and there should be no ambiguity between ESI and Flow 2731 labels even if they overlap. The reason for this is that the 2732 downstream assigned VPN label for known unicast is different than 2733 for BUM traffic and ESI label (if present) comes after BUM VPN 2734 label. Therefore, from the VPN label, the receiving PE knows 2735 whether the next label is a ESI label or a Flow label - i.e., if 2736 the VPN label is for known unicast, then the next label MUST be a 2737 flow label and if the VPN label is for BUM traffic, then the next 2738 label MUST be an ESI label because BUM packets are not sent with 2739 Flow labels. 2741 * When sending EVPN-encapsulated packets over a P2MP LSP (either 2742 RSVP-TE or mLDP), flow label SHOULD NOT be used. This is 2743 independant of any F-bit signalling in the L2-Attr Extended 2744 Community which would still apply to unicast. 2746 * This document updates the procedures in [RFC8214] to include 2747 optional use of the F-bit defined in Section 7.11 thus adding 2748 support for flow-aware transport of EVPN-VPWS signaled 2749 pseudowires. 2751 19. Use of Domain-wide Common Block (DCB) Labels 2753 The use of DCB labels as in 2754 [I-D.ietf-bess-mvpn-evpn-aggregation-label] is RECOMMENDED in the 2755 following cases: 2757 * Aggregate P-multicast trees: A P-multicast tree MAY aggregate the 2758 traffic of two or more BDs on a given ingress PE. When 2759 aggregation is needed, DCB Labels 2760 [I-D.ietf-bess-mvpn-evpn-aggregation-label] MAY be used in the 2761 MPLS label field of the Inclusive Multicast Ethernet Tag routes 2762 PMSI Tunnel Attribute. The use of DCB Labels, instead of upstream 2763 allocated labels, can greatly reduce the number of labels that the 2764 egress PEs need to process when P-multicast tunnel aggregation is 2765 used in a network with a large number of BDs. 2767 * BIER tunnels: As described in [I-D.ietf-bier-evpn], the use of 2768 labels with BIER tunnels in EVPN networks is similar to aggregate 2769 tunnels, since the ingress PE uses upstream allocated labels to 2770 identify the BD. As described in [I-D.ietf-bier-evpn], DCB labels 2771 can be allocated instead of upstream labels in the PMSI Tunnel 2772 Attribute so that the number of labels required on the egress PEs 2773 can be reduced. 2775 * ESI labels: The ESI labels advertised with EVPN A-D per ES routes 2776 MAY be allocated as DCB labels in general, and are RECOMMENDED to 2777 be allocated as DCB labels when used in combination with P2MP/BIER 2778 tunnels. 2780 When MP2MP tunnels are used, ESI-labels MUST be allocated from a DCB 2781 and the same label must be used by all the PEs attached to the same 2782 Ethernet Segment. In that way, any egress PE with local Ethernet 2783 Segments can identify the source ES of the received BUM packets. 2785 20. Security Considerations 2787 Security considerations discussed in [RFC4761] and [RFC4762] apply to 2788 this document for MAC learning in the data plane over an Attachment 2789 Circuit (AC) and for flooding of unknown unicast and ARP messages 2790 over the MPLS/IP core. Security considerations discussed in 2791 [RFC4364] apply to this document for MAC learning in the control 2792 plane over the MPLS/IP core. This section describes additional 2793 considerations. 2795 As mentioned in [RFC4761], there are two aspects to achieving data 2796 privacy and protecting against denial-of-service attacks in a VPN: 2797 securing the control plane and protecting the forwarding path. 2798 Compromise of the control plane could result in a PE sending customer 2799 data belonging to some EVPN to another EVPN, or black-holing EVPN 2800 customer data, or even sending it to an eavesdropper, none of which 2801 are acceptable from a data privacy point of view. In addition, 2802 compromise of the control plane could provide opportunities for 2803 unauthorized EVPN data usage (e.g., exploiting traffic replication 2804 within a multicast tree to amplify a denial-of-service attack based 2805 on sending large amounts of traffic). 2807 The mechanisms in this document use BGP for the control plane. 2808 Hence, techniques such as those discussed in [RFC5925] help 2809 authenticate BGP messages, making it harder to spoof updates (which 2810 can be used to divert EVPN traffic to the wrong EVPN instance) or 2811 withdrawals (denial-of-service attacks). In the multi-AS backbone 2812 options (b) and (c) [RFC4364], this also means protecting the 2813 inter-AS BGP sessions between the Autonomous System Border Routers 2814 (ASBRs), the PEs, or the Route Reflectors. 2816 Further discussion of security considerations for BGP may be found in 2817 the BGP specification itself [RFC4271] and in the security analysis 2818 for BGP [RFC4272]. The original discussion of the use of the TCP MD5 2819 signature option to protect BGP sessions is found in [RFC5925], while 2820 [RFC6952] includes an analysis of BGP keying and authentication 2821 issues. 2823 Note that [RFC5925] will not help in keeping MPLS labels private -- 2824 knowing the labels, one can eavesdrop on EVPN traffic. Such 2825 eavesdropping additionally requires access to the data path within an 2826 SP network. Users of VPN services are expected to take appropriate 2827 precautions (such as encryption) to protect the data exchanged over a 2828 VPN. 2830 One of the requirements for protecting the data plane is that the 2831 MPLS labels be accepted only from valid interfaces. For a PE, valid 2832 interfaces comprise links from other routers in the PE's own AS. For 2833 an ASBR, valid interfaces comprise links from other routers in the 2834 ASBR's own AS, and links from other ASBRs in ASes that have instances 2835 of a given EVPN. It is especially important in the case of multi-AS 2836 EVPN instances that one accept EVPN packets only from valid 2837 interfaces. 2839 It is also important to help limit malicious traffic into a network 2840 for an impostor MAC address. The mechanism described in Section 15.1 2841 shows how duplicate MAC addresses can be detected and continuous 2842 false MAC mobility can be prevented. The mechanism described in 2843 Section 15.2 shows how MAC addresses can be pinned to a given 2844 Ethernet segment, such that if they appear behind any other Ethernet 2845 segments, the traffic for those MAC addresses can be prevented from 2846 entering the EVPN network from the other Ethernet segments. 2848 21. IANA Considerations 2850 This document defines a new NLRI, called "EVPN", to be carried in BGP 2851 using multiprotocol extensions. This NLRI uses the existing AFI of 2852 25 (L2VPN). IANA has assigned BGP EVPNs a SAFI value of 70. 2854 IANA has allocated the following EVPN Extended Community sub-types in 2855 [RFC7153], and this document is the only reference for them, in 2856 addition to [RFC7432]. 2858 0x00 MAC Mobility [RFC7432] 2859 0x01 ESI Label [RFC7432] 2860 0x02 ES-Import Route Target [RFC7432] 2862 This document creates a registry called "EVPN Route Types". New 2863 registrations will be made through the "RFC Required" procedure 2864 defined in [RFC8126]. The registry has a maximum value of 255. 2865 Registrations carried forward from [RFC7432] are as follows: 2867 0 Reserved [RFC7432] 2868 1 Ethernet Auto-discovery [RFC7432] 2869 2 MAC/IP Advertisement [RFC7432] 2870 3 Inclusive Multicast Ethernet Tag [RFC7432] 2871 4 Ethernet Segment [RFC7432] 2873 This document creates a registry called "EVPN ESI Multihoming 2874 Attributes" for the 1-octet Flags field in the ESI Label Extended 2875 Community. New registrations will be made through the "RFC Required" 2876 procedure defined in [RFC8126]. 2878 Initial registrations are as follows: 2880 RED Multihomed site redundancy mode 2881 00 = All-Active 2882 01 = Single-Active 2884 This document requests allocation of bit 3 in the "EVPN Layer 2 2885 Attributes Control Flags" registry with name F: 2887 F Flow Label MUST be present 2889 22. References 2890 22.1. Normative References 2892 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2893 Requirement Levels", BCP 14, RFC 2119, 2894 DOI 10.17487/RFC2119, March 1997, 2895 . 2897 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 2898 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 2899 DOI 10.17487/RFC4271, January 2006, 2900 . 2902 [RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended 2903 Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, 2904 February 2006, . 2906 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 2907 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 2908 2006, . 2910 [RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter, 2911 "Multiprotocol Extensions for BGP-4", RFC 4760, 2912 DOI 10.17487/RFC4760, January 2007, 2913 . 2915 [RFC4761] Kompella, K., Ed. and Y. Rekhter, Ed., "Virtual Private 2916 LAN Service (VPLS) Using BGP for Auto-Discovery and 2917 Signaling", RFC 4761, DOI 10.17487/RFC4761, January 2007, 2918 . 2920 [RFC4762] Lasserre, M., Ed. and V. Kompella, Ed., "Virtual Private 2921 LAN Service (VPLS) Using Label Distribution Protocol (LDP) 2922 Signaling", RFC 4762, DOI 10.17487/RFC4762, January 2007, 2923 . 2925 [RFC7153] Rosen, E. and Y. Rekhter, "IANA Registries for BGP 2926 Extended Communities", RFC 7153, DOI 10.17487/RFC7153, 2927 March 2014, . 2929 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 2930 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 2931 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2932 2015, . 2934 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 2935 Rabadan, "Virtual Private Wire Service Support in Ethernet 2936 VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017, 2937 . 2939 [RFC8584] Rabadan, J., Ed., Mohanty, S., Ed., Sajassi, A., Drake, 2940 J., Nagaraj, K., and S. Sathappan, "Framework for Ethernet 2941 VPN Designated Forwarder Election Extensibility", 2942 RFC 8584, DOI 10.17487/RFC8584, April 2019, 2943 . 2945 22.2. Informative References 2947 [I-D.ietf-bess-evpn-igmp-mld-proxy] 2948 Sajassi, A., Thoria, S., Mishra, M. P., Drake, J., and W. 2949 Lin, "IGMP and MLD Proxy for EVPN", Work in Progress, 2950 Internet-Draft, draft-ietf-bess-evpn-igmp-mld-proxy-18, 16 2951 February 2022, . 2954 [I-D.ietf-bess-evpn-mh-split-horizon] 2955 Rabadan, J., Nagaraj, K., Lin, W., and A. Sajassi, "EVPN 2956 Multi-Homing Extensions for Split Horizon Filtering", Work 2957 in Progress, Internet-Draft, draft-ietf-bess-evpn-mh- 2958 split-horizon-02, 15 October 2021, 2959 . 2962 [I-D.ietf-bess-mvpn-evpn-aggregation-label] 2963 Zhang, Z. J., Rosen, E. C., Lin, W., Li, Z., and I. 2964 Wijnands, "MVPN/EVPN Tunnel Aggregation with Common 2965 Labels", Work in Progress, Internet-Draft, draft-ietf- 2966 bess-mvpn-evpn-aggregation-label-06, 19 April 2021, 2967 . 2970 [I-D.ietf-bier-evpn] 2971 Zhang, Z., Przygienda, T., Sajassi, A., and J. Rabadan, 2972 "EVPN BUM Using BIER", Work in Progress, Internet-Draft, 2973 draft-ietf-bier-evpn-04, 2 December 2020, 2974 . 2977 [IEEE.802.1D_2004] 2978 IEEE, "IEEE Standard for Local and metropolitan area 2979 networks: Media Access Control (MAC) Bridges", IEEE 2980 802.1D-2004, DOI 10.1109/ieeestd.2004.94569, 6 July 2004, 2981 . 2983 [IEEE.802.1Q_2014] 2984 IEEE, "IEEE Standard for Local and metropolitan area 2985 networks--Bridges and Bridged Networks", IEEE 802.1Q-2014, 2986 DOI 10.1109/ieeestd.2014.6991462, 18 December 2014, 2987 . 2990 [RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis", 2991 RFC 4272, DOI 10.17487/RFC4272, January 2006, 2992 . 2994 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 2995 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 2996 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 2997 February 2006, . 2999 [RFC4664] Andersson, L., Ed. and E. Rosen, Ed., "Framework for Layer 3000 2 Virtual Private Networks (L2VPNs)", RFC 4664, 3001 DOI 10.17487/RFC4664, September 2006, 3002 . 3004 [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 3005 R., Patel, K., and J. Guichard, "Constrained Route 3006 Distribution for Border Gateway Protocol/MultiProtocol 3007 Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual 3008 Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, 3009 November 2006, . 3011 [RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP 3012 Authentication Option", RFC 5925, DOI 10.17487/RFC5925, 3013 June 2010, . 3015 [RFC6514] Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP 3016 Encodings and Procedures for Multicast in MPLS/BGP IP 3017 VPNs", RFC 6514, DOI 10.17487/RFC6514, February 2012, 3018 . 3020 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 3021 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 3022 RFC 6790, DOI 10.17487/RFC6790, November 2012, 3023 . 3025 [RFC6952] Jethanandani, M., Patel, K., and L. Zheng, "Analysis of 3026 BGP, LDP, PCEP, and MSDP Issues According to the Keying 3027 and Authentication for Routing Protocols (KARP) Design 3028 Guide", RFC 6952, DOI 10.17487/RFC6952, May 2013, 3029 . 3031 [RFC7117] Aggarwal, R., Ed., Kamite, Y., Fang, L., Rekhter, Y., and 3032 C. Kodeboniya, "Multicast in Virtual Private LAN Service 3033 (VPLS)", RFC 7117, DOI 10.17487/RFC7117, February 2014, 3034 . 3036 [RFC7209] Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N., 3037 Henderickx, W., and A. Isaac, "Requirements for Ethernet 3038 VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014, 3039 . 3041 [RFC7991] Hoffman, P., "The "xml2rfc" Version 3 Vocabulary", 3042 RFC 7991, DOI 10.17487/RFC7991, December 2016, 3043 . 3045 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 3046 Writing an IANA Considerations Section in RFCs", BCP 26, 3047 RFC 8126, DOI 10.17487/RFC8126, June 2017, 3048 . 3050 [RFC8317] Sajassi, A., Ed., Salam, S., Drake, J., Uttaro, J., 3051 Boutros, S., and J. Rabadan, "Ethernet-Tree (E-Tree) 3052 Support in Ethernet VPN (EVPN) and Provider Backbone 3053 Bridging EVPN (PBB-EVPN)", RFC 8317, DOI 10.17487/RFC8317, 3054 January 2018, . 3056 [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., 3057 Uttaro, J., and W. Henderickx, "A Network Virtualization 3058 Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, 3059 DOI 10.17487/RFC8365, March 2018, 3060 . 3062 [RFC9136] Rabadan, J., Ed., Henderickx, W., Drake, J., Lin, W., and 3063 A. Sajassi, "IP Prefix Advertisement in Ethernet VPN 3064 (EVPN)", RFC 9136, DOI 10.17487/RFC9136, October 2021, 3065 . 3067 Appendix A. Acknowledgments for This Document (2021) 3069 Appendix B. Contributors for This Document (2021) 3071 In addition to the authors listed on the front page, the following 3072 co-authors have also contributed to this document: 3074 Appendix C. Acknowledgments from the First Edition (2015) 3076 Special thanks to Yakov Rekhter for reviewing this document several 3077 times and providing valuable comments, and for his very engaging 3078 discussions on several topics of this document that helped shape this 3079 document. We would also like to thank Pedro Marques, Kaushik Ghosh, 3080 Nischal Sheth, Robert Raszuk, Amit Shukla, and Nadeem Mohammed for 3081 discussions that helped shape this document. We would also like to 3082 thank Han Nguyen for his comments and support of this work. We would 3083 also like to thank Steve Kensil and Reshad Rahman for their reviews. 3084 We would like to thank Jorge Rabadan for his contribution to 3085 Section 5 of this document. We would like to thank Thomas Morin for 3086 his review of this document and his contribution of Section 8.7. 3087 Many thanks to Jakob Heitz for his help to improve several sections 3088 of this document. 3090 We would also like to thank Clarence Filsfils, Dennis Cai, Quaizar 3091 Vohra, Kireeti Kompella, and Apurva Mehta for their contributions to 3092 this document. 3094 Last but not least, special thanks to Giles Heron (our WG chair) for 3095 his detailed review of this document in preparation for WG Last Call 3096 and for making many valuable suggestions. 3098 C.1. Contributors from the First Edition (2015) 3100 In addition to the authors listed on the front page, the following 3101 co-authors have also contributed to this document: 3103 Keyur Patel 3104 Samer Salam 3105 Sami Boutros 3106 Cisco 3108 Yakov Rekhter 3109 Ravi Shekhar 3110 Juniper Networks 3112 Florin Balus 3113 Nuage Networks 3115 C.2. Authors from the First Edition (2015) 3117 Original Authors: 3119 Ali Sajassi 3120 Cisco 3121 EMail: sajassi@cisco.com 3123 Rahul Aggarwal 3124 Arktan 3126 EMail: raggarwa_1@yahoo.com 3128 Nabil Bitar 3129 Verizon Communications 3131 EMail : nabil.n.bitar@verizon.com 3133 Aldrin Isaac 3134 Bloomberg 3136 EMail: aisaac71@bloomberg.net 3138 James Uttaro 3139 AT&T 3141 EMail: uttaro@att.com 3143 John Drake 3144 Juniper Networks 3146 EMail: jdrake@juniper.net 3148 Wim Henderickx 3149 Alcatel-Lucent 3151 EMail: wim.henderickx@alcatel-lucent.com 3153 Authors' Addresses 3155 Ali Sajassi 3156 Cisco 3157 Email: sajassi@cisco.com 3159 Luc Andre Burdet (editor) 3160 Cisco 3161 Email: lburdet@cisco.com 3163 John Drake 3164 Juniper 3165 Email: jdrake@juniper.net 3166 Jorge Rabadan 3167 Nokia 3168 Email: jorge.rabadan@nokia.com