idnits 2.17.1 draft-ietf-bess-evpn-mvpn-seamless-interop-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? RFC 2119 keyword, line 335: '... The solution SHALL support optimum ...' RFC 2119 keyword, line 337: '...ple LATAs. The solution SHALL support...' RFC 2119 keyword, line 343: '...ability, the solution SHALL use only a...' RFC 2119 keyword, line 347: '... solution MUST support optimum repli...' RFC 2119 keyword, line 350: '... - Non-IP traffic SHALL be forwarded per EVPN baseline [RFC7432] or...' (42 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 22, 2021) is 917 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2119' is mentioned on line 235, but not defined == Missing Reference: 'IEEE802.1Q' is mentioned on line 483, but not defined == Unused Reference: 'RFC4389' is defined on line 1653, but no explicit reference was found in the text == Unused Reference: 'RFC4761' is defined on line 1657, but no explicit reference was found in the text == Unused Reference: 'RFC7080' is defined on line 1662, but no explicit reference was found in the text == Unused Reference: 'RFC7209' is defined on line 1667, but no explicit reference was found in the text == Outdated reference: A later version (-21) exists of draft-ietf-bess-evpn-igmp-mld-proxy-13 Summary: 2 errors (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS WorkGroup A. Sajassi 3 Internet-Draft K. Thiruvenkatasamy 4 Intended status: Standards Track S. Thoria 5 Expires: April 25, 2022 Cisco 6 A. Gupta 7 VMware 8 L. Jalil 9 Verizon 10 October 22, 2021 12 Seamless Multicast Interoperability between EVPN and MVPN PEs 13 draft-ietf-bess-evpn-mvpn-seamless-interop-03 15 Abstract 17 Ethernet Virtual Private Network (EVPN) solution is becoming 18 pervasive for Network Virtualization Overlay (NVO) services in data 19 center (DC) networks and as the next generation VPN services in 20 service provider (SP) networks. 22 As service providers transform their networks in their Central 23 Offices (COs) towards the next generation data center with Software 24 Defined Networking (SDN) based fabric and Network Function 25 Virtualization (NFV), they want to be able to maintain their offered 26 services including Multicast VPN (MVPN) service between their 27 existing network and their new Service Provider Data Center (SPDC) 28 network seamlessly without the use of gateway devices. They want to 29 have such seamless interoperability between their new SPDCs and their 30 existing networks for a) reducing cost, b) having optimum forwarding, 31 and c) reducing provisioning. This document describes a unified 32 solution based on RFCs 6513 & 6514 for seamless interoperability of 33 Multicast VPN between EVPN and MVPN PEs. Furthermore, it describes 34 how the proposed solution can be used as a routed multicast solution 35 in data centers with only EVPN PEs. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at https://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 50 Internet-DrafSeamless Multicast Interoperability between EV October 2021 52 time. It is inappropriate to use Internet-Drafts as reference 53 material or to cite them other than as "work in progress." 55 This Internet-Draft will expire on April 25, 2022. 57 Copyright Notice 59 Copyright (c) 2021 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (https://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the Simplified BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 75 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 5 76 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 77 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 7 78 4.1. Optimum Forwarding . . . . . . . . . . . . . . . . . . . 7 79 4.2. Optimum Replication . . . . . . . . . . . . . . . . . . . 7 80 4.3. All-Active and Single-Active Multi-Homing . . . . . . . . 8 81 4.4. Inter-AS Tree Stitching . . . . . . . . . . . . . . . . . 8 82 4.5. EVPN Service Interfaces . . . . . . . . . . . . . . . . . 8 83 4.6. Distributed Anycast Gateway . . . . . . . . . . . . . . . 8 84 4.7. Selective & Aggregate Selective Tunnels . . . . . . . . 8 85 4.8. Tenants' (S,G) or (*,G) states . . . . . . . . . . . . . 9 86 4.9. Zero Disruption upon BD/Subnet Addition . . . . . . . . . 9 87 4.10. No Changes to Existing EVPN Service Interface Models . . 9 88 4.11. External source and receivers . . . . . . . . . . . . . . 9 89 4.12. Tenant RP placement . . . . . . . . . . . . . . . . . . . 9 90 5. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 9 91 5.1. IRB Unicast versus IRB Multicast . . . . . . . . . . . . 10 92 5.1.1. IRB multicast in seamless interop mode . . . . . . . 10 93 5.2. Operational Model for EVPN IRB PEs . . . . . . . . . . . 11 94 5.3. Unicast Route Advertisements for IP multicast Source . . 13 95 5.4. Multi-homing of IP Multicast Source and Receivers . . . . 15 96 5.4.1. Single-Active Multi-Homing . . . . . . . . . . . . . 15 97 5.4.2. All-Active Multi-Homing . . . . . . . . . . . . . . . 16 98 5.5. Mobility for Tenant's Sources and Receivers . . . . . . . 18 99 6. Control Plane Operation . . . . . . . . . . . . . . . . . . . 18 101 Internet-DrafSeamless Multicast Interoperability between EV October 2021 103 6.1. Intra-ES Subnet Tunnel . . . . . . . . . . . . . . . . . 18 104 6.2. Intra-Subnet BUM Tunnel . . . . . . . . . . . . . . . . . 20 105 6.3. Inter-Subnet IP Multicast Tunnel . . . . . . . . . . . . 20 106 6.4. IGMP Hosts as TSes . . . . . . . . . . . . . . . . . . . 21 107 6.5. PIM Routers as TSes . . . . . . . . . . . . . . . . . . . 21 108 7. Data Plane Operation . . . . . . . . . . . . . . . . . . . . 22 109 7.1. Intra-Subnet L2 Switching . . . . . . . . . . . . . . . . 23 110 7.2. Inter-Subnet L3 Routing . . . . . . . . . . . . . . . . . 23 111 8. DCs with only EVPN PEs . . . . . . . . . . . . . . . . . . . 23 112 8.1. Setup of overlay multicast delivery . . . . . . . . . . . 24 113 8.2. Handling of different encapsulations . . . . . . . . . . 25 114 8.2.1. MPLS Encapsulation . . . . . . . . . . . . . . . . . 26 115 8.2.2. VxLAN Encapsulation . . . . . . . . . . . . . . . . . 26 116 8.2.3. Other Encapsulation . . . . . . . . . . . . . . . . . 26 117 9. DCI with MPLS in WAN and VxLAN in DCs . . . . . . . . . . . . 26 118 9.1. Control plane inter-connect . . . . . . . . . . . . . . . 27 119 9.2. Data plane inter-connect . . . . . . . . . . . . . . . . 28 120 10. Interop with L2 EVPN PEs . . . . . . . . . . . . . . . . . . 28 121 10.1. Interaction with L2EVPN PE and Seamless interop capable 122 PE . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 123 10.2. Network having L2EVPN PE, Seamless interop capable PE 124 and MVPN PE . . . . . . . . . . . . . . . . . . . . . . 31 125 11. Connecting external Multicast networks or PIM routers. . . . 31 126 12. TS RP options . . . . . . . . . . . . . . . . . . . . . . . . 32 127 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 128 14. Security Considerations . . . . . . . . . . . . . . . . . . . 32 129 15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 32 130 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 33 131 16.1. Normative References . . . . . . . . . . . . . . . . . . 33 132 16.2. Informative References . . . . . . . . . . . . . . . . . 34 133 Appendix A. Supporting application with TTL value 1 . . . . . . 34 134 A.1. Policy based model . . . . . . . . . . . . . . . . . . . 34 135 A.2. Exercising BUM procedure for VLAN/BD . . . . . . . . . . 35 136 A.3. Intra-subnet bridging . . . . . . . . . . . . . . . . . . 35 137 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 139 1. Introduction 141 Ethernet Virtual Private Network (EVPN) solution is becoming 142 pervasive for Network Virtualization Overlay (NVO) services in data 143 center (DC) networks and as the next generation VPN services in 144 service provider (SP) networks. 146 As service providers transform their networks in their Central 147 Offices (COs) towards the next generation data center with Software 148 Defined Networking (SDN) based fabric and Network Function 149 Virtualization (NFV), they want to be able to maintain their offered 150 services including Multicast VPN (MVPN) service between their 152 Internet-DrafSeamless Multicast Interoperability between EV October 2021 154 existing network and their new SPDC network seamlessly without the 155 use of gateway devices. There are several reasons for having such 156 seamless interoperability between their new DCs and their existing 157 networks: 159 - Lower Cost: gateway devices need to have very high scalability to 160 handle VPN services for their DCs and as such need to handle large 161 number of VPN instances (in tens or hundreds of thousands) and very 162 large number of routes (e.g., in tens of millions). For the same 163 speed and feed, these high scale gateway boxes are relatively much 164 more expensive than the edge devices (e.g., PEs and TORs) that 165 support much lower number of routes and VPN instances. 167 - Optimum Forwarding: in a given Central Office(CO), both EVPN PEs 168 and MVPN PEs can be connected to the same fabric/network (e.g., same 169 IGP domain). In such scenarios, the service providers want to have 170 optimum forwarding among these PE devices without the use of gateway 171 devices. Because if gateway devices are used, then the IP multicast 172 traffic between an EVPN and MVPN PEs can no longer be optimum and in 173 some case, it may even get tromboned. Furthermore, when an SPDC 174 network spans across multiple LATA (multiple geographic areas) and 175 gateways are used between EVPN and MVPN PEs, then with respect to IP 176 multicast traffic, only one GW can be designated forwarder (DF) 177 between EVPN and MVPN PEs. Such scenarios not only result in non- 178 optimum forwarding but also it can result in tromboning of IP 179 multicast traffic between the two LATAs when both source and 180 destination PEs are in the same LATA and the DF gateway is elected to 181 be in a different LATA. 183 - Less Provisioning: If gateways are used, then the operator need to 184 configure per-tenant info on the gateways. In other words, for each 185 tenant that is configured, one (or maybe two) additional touch points 186 are needed. 188 In datacenter deployments, inter-subnet multicast traffic within an 189 EVPN based fabric/data center is unoptimized. When there are 190 multiple receivers in different bridge domains of the same tenant 191 system, a router attached to an EVPN PE would send multiple copies 192 into the EVPN fabric resulting bandwidth wastage. [RFC9135] only 193 covers procedures for efficient inter-subnet connectivity among these 194 Tenant Systems and End Devices while maintaining the multi-homing 195 capabilities of EVPN only for unicast traffic. There is a need to 196 support efficient inter-subnet multicast forwarding within the data 197 center. 199 This document describes a unified solution based on [RFC6513] and 200 [RFC6514] for seamless interoperability of multicast VPN between EVPN 201 and MVPN PEs. Furthermore, it describes how the proposed solution 203 Internet-DrafSeamless Multicast Interoperability between EV October 2021 205 can be used as a routed multicast solution in data centers with only 206 EVPN PEs (e.g., routed multicast VPN only among EVPN PEs) to do 207 optimized multicast forwarding. 209 The document is organized such that seamless interop mode covered 210 first followed by how the same model can be used as an optimized 211 multicast forwarding solution for data center networks. 213 Section 5 provides the solution overview in detail. This section 214 assumes that all EVPN PEs have IRB capability and operating in IRB 215 mode for both unicast and multicast traffic (e.g., all EVPN PEs are 216 homogenous in terms of their capabilities and operational modes). 217 Section 6 and 7 covers control plane and data plane respectively. 219 Section 8 describes how the proposed solution can be used to achieve 220 optimized multicast forwarding within the EVPN domain/Data center 221 only networks. Section 9 discusses DCI usecases. 223 An EVPN network can consist of a mix of L2 and L3 PEs. The multicast 224 operation of such heterogeneous EVPN network will be an extension of 225 an EVPN homogenous network. Section 10 discusses the multicast IRB 226 solution description for the EVPN heterogeneous network. 228 2. Requirements Language 230 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 231 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to 232 be interpreted as described in [RFC2119] only when they appear in all 233 upper case. They may also appear in lower or mixed case as English 234 words, without any normative meaning. 236 3. Terminology 238 Most of the terminology used in this document comes from [RFC8365] 240 Broadcast Domain (BD): In a bridged network, the broadcast domain 241 corresponds to a Virtual LAN (VLAN), where a VLAN is typically 242 represented by a single VLAN ID (VID) but can be represented by 243 several VIDs where Shared VLAN Learning (SVL) is used per [802.1Q]. 245 Bridge Table (BT): An instantiation of a broadcast domain on a MAC- 246 VRF. 248 VXLAN: Virtual Extensible LAN 250 PoD: Point of Delivery 252 NV: Network Virtualization 254 Internet-DrafSeamless Multicast Interoperability between EV October 2021 256 NVO: Network Virtualization Overlay 258 NVE: Network Virtualization Endpoint 260 NVGRE: Network Virtualization using Generic Routing Encapsulation 262 GENEVE: Generic Network Virtualization Encapsulation 264 VNI: Virtual Network Identifier (for VXLAN) 266 EVPN: Ethernet VPN 268 EVI: An EVPN instance spanning the Provider Edge (PE) devices 269 participating in that EVPN 271 MAC-VRF: A Virtual Routing and Forwarding table for Media Access 272 Control (MAC) addresses on a PE 274 IP-VRF: A Virtual Routing and Forwarding table for Internet Protocol 275 (IP) addresses on a PE 277 Ethernet Segment (ES): When a customer site (device or network) is 278 connected to one or more PEs via a set of Ethernet links, then that 279 set of links is referred to as an 'Ethernet segment'. 281 Ethernet Segment Identifier (ESI): A unique non-zero identifier that 282 identifies an Ethernet segment is called an 'Ethernet Segment 283 Identifier'. 285 Ethernet Tag: An Ethernet tag identifies a particular broadcast 286 domain, e.g., a VLAN. An EVPN instance consists of one or more 287 broadcast domains. 289 PE: Provider Edge device. 291 Single-Active Redundancy Mode: When only a single PE, among all the 292 PEs attached to an Ethernet segment, is allowed to forward traffic 293 to/from that Ethernet segment for a given VLAN, then the Ethernet 294 segment is defined to be operating in Single-Active redundancy mode. 296 All-Active Redundancy Mode: When all PEs attached to an Ethernet 297 segment are allowed to forward known unicast traffic to/from that 298 Ethernet segment for a given VLAN, then the Ethernet segment is 299 defined to be operating in All-Active redundancy mode. 301 PIM-SM: Protocol Independent Multicast - Sparse-Mode 303 PIM-SSM: Protocol Independent Multicast - Source Specific Multicast 305 Internet-DrafSeamless Multicast Interoperability between EV October 2021 307 Bidir PIM: Bidirectional PIM 309 FHR: First Hop Router 311 LHR: Last Hop Router 313 CO: Central Office of a service provider 315 SPDC: Service Provider Data Center 317 LATA: Local Access and Transport Area 319 Border Leafs: A set of EVPN-PE acting as exit point for EVPN fabric. 321 EC: BGP Extended Community 323 UMH: Upstream Multicast Hop 325 TS: Tenant Systems 327 4. Requirements 329 This section describes the requirements specific in providing 330 seamless multicast VPN service between MVPN and EVPN capable 331 networks. 333 4.1. Optimum Forwarding 335 The solution SHALL support optimum multicast forwarding between EVPN 336 and MVPN PEs within a network. The network can be confined to a CO 337 or it can span across multiple LATAs. The solution SHALL support 338 optimum multicast forwarding with both ingress replication tunnels 339 and P2MP tunnels. 341 4.2. Optimum Replication 343 For EVPN PEs with IRB capability, the solution SHALL use only a 344 single multicast tunnel among EVPN and MVPN PEs for IP multicast 345 traffic, when both PEs use the same tunnel type. Multicast tunnels 346 can be either ingress replication tunnels or P2MP tunnels. The 347 solution MUST support optimum replication for both Intra-subnet and 348 Inter-subnet IP multicast traffic: 350 - Non-IP traffic SHALL be forwarded per EVPN baseline [RFC7432] or 351 [RFC8365] 353 - If a Multicast VPN spans across both Intra and Inter subnets, then 354 for Ingress replication regardless of whether the traffic is Intra or 356 Internet-DrafSeamless Multicast Interoperability between EV October 2021 358 Inter subnet, only a single copy of IP multicast traffic SHALL be 359 sent from the source PE to the destination PE. 361 - If a Multicast VPN spans across both Intra and Inter subnets, then 362 for P2MP tunnels regardless of whether the traffic is Intra or Inter 363 subnet, only a single copy of multicast data SHALL be transmitted by 364 the source PE. Source PE can be either EVPN or MVPN PE and receiving 365 PEs can be a mix of EVPN and MVPN PEs - i.e., a multicast VPN can be 366 spread across both EVPN and MVPN PEs. 368 4.3. All-Active and Single-Active Multi-Homing 370 The solution MUST support multi-homing of source devices and 371 receivers that are sitting in the same subnet (e.g., VLAN) and are 372 multi-homed to EVPN PEs. The solution SHALL allow for both Single- 373 Active and All-Active multi-homing. 375 4.4. Inter-AS Tree Stitching 377 The solution SHALL support multicast tree stitching when the tree 378 spans across multiple Autonomous Systems. 380 4.5. EVPN Service Interfaces 382 The solution MUST support all EVPN service interfaces listed in 383 section 6 of [RFC7432]: 385 o VLAN-based service interface 387 o VLAN-bundle service interface 389 o VLAN-aware bundle service interface. 391 4.6. Distributed Anycast Gateway 393 The solution SHALL support distributed anycast gateways for tenant 394 workloads on NVE devices operating in EVPN-IRB mode. 396 4.7. Selective & Aggregate Selective Tunnels 398 The solution SHALL support selective and aggregate selective P- 399 tunnels as well as inclusive and aggregate inclusive P-tunnels. When 400 selective tunnels are used, multicast traffic SHOULD only be 401 forwarded to the remote PEs that have receivers - i.e., if there are 402 no receivers at a remote PE, the multicast traffic SHOULD NOT be 403 forwarded to that PE. If there are no receivers on any remote PEs, 404 then the multicast traffic SHOULD NOT be forwarded to the core. 406 Internet-DrafSeamless Multicast Interoperability between EV October 2021 408 4.8. Tenants' (S,G) or (*,G) states 410 The solution SHOULD store (C-S,C-G) and (C-*,C-G) states only on PE 411 devices that have interest in such states hence reducing memory and 412 processing requirements - i.e., PE devices that have sources and/or 413 receivers interested in such multicast groups. 415 4.9. Zero Disruption upon BD/Subnet Addition 417 In DC environments, various Bridge Domains are provisioned and 418 removed on regular basis due to host mobility, policy and tenant 419 changes. Such change in BD configuration should not affect existing 420 flows within the same BD or any other BD in the network. 422 4.10. No Changes to Existing EVPN Service Interface Models 424 VLAN-aware bundle service as defined in [RFC7432] typically does not 425 require any VLAN ID translation from one tenant site to another - 426 i.e., the same set of VLAN IDs are configured consistently on all 427 tenant segments. In such scenarios, EVPN-IRB multicast service MUST 428 maintain the same mode of operation and SHALL NOT require any VLAN ID 429 translation. 431 4.11. External source and receivers 433 The solution SHALL support sources and receivers external to the 434 tenant domain. i.e., multicast source inside the tenant domain can 435 have receiver outside the tenant domain and vice versa. 437 4.12. Tenant RP placement 439 The solution SHALL support a tenant to have RP anywhere in the 440 network. RP can be placed inside the EVPN network or MVPN network or 441 external domain. 443 5. Solution Overview 445 This section describes a multicast VPN solution based on [RFC6513] 446 and [RFC6514] for EVPN PEs operating in IRB mode that want to perform 447 seamless interoperability with their counterparts MVPN PEs. 449 In order to enable seamless integration of EVPN and MVPN Pes, traffic 450 originated/received from EVPN PE needs to be modelled very similar to 451 MVPN PE. Hence, there are some differences in handling IRB multicast 452 defined in this document in comparison to IRB unicast defined in 453 [RFC9135]. The next section covers differences. 455 Internet-DrafSeamless Multicast Interoperability between EV October 2021 457 5.1. IRB Unicast versus IRB Multicast 459 [RFC9135] describes the operation for EVPN PEs in IRB mode for 460 unicast traffic. The same IRB model used for unicast traffic, where 461 an IP-VRF in an EVPN PE is attached to one or more bridge tables 462 (BTs) via virtual IRB interfaces, is also applicable for multicast 463 traffic. 465 For unicast traffic, the intra-subnet traffic is bridged within the 466 MAC-VRF associated with that subnet (i.e., a lookup based on MAC-DA 467 is performed); whereas, the inter-subnet traffic is routed in the 468 corresponding IP-VRF (i.e. a lookup based on IP-DA is performed). 470 A given tenant can have one or more IP-VRFs; however, without loss of 471 generality, this document assumes one IP-VRF per tenant. In context 472 of a given tenant's multicast traffic, the intra-subnet traffic is 473 bridged for non-IP traffic and it is Layer-2 switched for IP traffic. 474 Whereas, the tenant's inter-subnet multicast traffic is always routed 475 in the corresponding IP-VRF. The difference between bridging and 476 L2-switching for multicast traffic is that the former uses MAC-DA 477 lookup for forwarding the multicast traffic; whereas, the latter uses 478 IP-DA lookup for such forwarding where the forwarding states are 479 built in the MAC-VRF using IGMP/MLD or PIM snooping. 481 5.1.1. IRB multicast in seamless interop mode 483 EVPN does not provide a Virtual LAN (VLAN) service per [IEEE802.1Q] 484 but rather an emulated VLAN service. This VLAN service emulation is 485 not only done for unicast traffic but also is extended for intra- 486 subnet multicast traffic described in 487 [I-D.ietf-bess-evpn-igmp-mld-proxy]. For intra-subnet multicast, an 488 EVPN PE builds multicast forwarding states in its bridge table (BT) 489 based on snooping of IGMP/MLD and/or PIM messages and the forwarding 490 is performed based on destination IP multicast address of the 491 Ethernet frame rather than destination MAC address as noted above. 493 In order to enable seamless integration of EVPN and MVPN PEs, this 494 document extends the concept of an emulated VLAN service for 495 multicast IRB applications such that the intra-subnet IP multicast 496 traffic can get treated same as inter- subnet IP multicast traffic 497 which means intra-subnet IP multicast traffic destined to remote PEs 498 gets routed instead of being L2- switched - i.e., TTL value gets 499 decremented and the Ethernet header of the L2 frame is de-capsulated 500 and encapsulated at both ingress and egress PEs. 502 It should be noted that the non-IP multicast or L2 broadcast traffic 503 still gets bridged and frames get forwarded based on their 504 destination MAC addresses. 506 Internet-DrafSeamless Multicast Interoperability between EV October 2021 508 Link local IP multicast traffic consists IPv4 traffic with a 509 destination address prefix of 224/8 and IPv6 traffic with a 510 destination address prefix of FF02/16. Such IP multicast traffic 511 along with non-IP multicast/broadcast traffic are sent per EVPN 512 [RFC7432] BUM procedures and does not get routed via IP-VRF for 513 multicast addresses. So, such BUM traffic will be limited to a given 514 EVI/VLAN (e.g., a given subnet); whereas, IP multicast traffic, will 515 be locally L2 switched for local interfaces attached on the same 516 subnet and will be routed for local interfaces attached on a 517 different subnet or for forwarding traffic to other EVPN PEs (refer 518 to section 7 for data plane operation). 520 5.2. Operational Model for EVPN IRB PEs 522 Without the loss of generality, this section assumes that all EVPN 523 PEs have IRB capability and operating in IRB mode for both unicast 524 and multicast traffic (e.g., all EVPN PEs are homogenous in terms of 525 their capabilities and operational modes). As it will be seen later, 526 an EVPN network can consist of a mix of PEs where some are capable of 527 multicast IRB and some are not and the multicast operation of such 528 heterogeneous EVPN network will be an extension of an EVPN homogenous 529 network. Therefore, we start with the multicast IRB solution 530 description for the EVPN homogenous network. 532 The EVPN PEs terminate IGMP/MLD messages from tenant host devices or 533 PIM messages from tenant routers on their IRB interfaces, thus avoid 534 sending these messages over MPLS/IP core. A tenant virtual/physical 535 router (e.g., CE) attached to an EVPN PE becomes a multicast routing 536 adjacency of that PE. Furthermore, the PE uses MVPN BGP protocol and 537 procedures per [RFC6513] and [RFC6514]. With respect to multicast 538 routing protocol between tenant's virtual/physical router and the PE 539 that it is attached to, any of the following PIM protocols is 540 supported per [RFC6513]: PIM-SM with Any Source Multicast (ASM) mode, 541 PIM-SM with Source Specific Multicast (SSM) mode, and PIM 542 Bidirectional (BIDIR) mode. Support of PIM-DM (Dense Mode) is 543 excluded in this document per [RFC6513]. 545 The EVPN PEs use MVPN BGP routes defined in [RFC6514] to convey 546 tenant (S,G) or (*,G) states to other MVPN or EVPN PEs and to set up 547 overlay trees (inclusive or selective) for a given MVPN instance. 548 The root or a leaf of such an overlay tree is terminated on an EVPN 549 or MVPN PE. Furthermore, this inclusive or selective overlay tree is 550 terminated on a single IP-VRF of the EVPN or MVPN PE. In case of 551 EVPN PE, these overlay trees never get terminated on MAC-VRFs of that 552 PE. 554 Overlay trees are instantiated by underlay provider tunnels (P- 555 tunnels) - e.g., P2MP, MP2MP, or unicast tunnels per [RFC6513]. When 557 Internet-DrafSeamless Multicast Interoperability between EV October 2021 559 there are several overlay trees mapped to a single underlay P-tunnel, 560 the tunnel is referred to as an aggregate tunnel. 562 Figure-1 below depicts a scenario where a tenant's MVPN spans across 563 both EVPN and MVPN PEs; where all EVPN PEs have multicast IRB 564 capability. An EVPN PE (with multicast IRB capability) can be 565 modeled as a MVPN PE where the virtual IRB interface of an EVPN PE 566 (virtual interface between a BT and IP-VRF) can be considered a 567 routed interface for the MVPN PE. 569 EVPN PE1 570 +------------+ 571 Src1 +----|(MAC-VRF1) | MVPN PE3 572 Rcvr1 +----| \ | +---------+ +--------+ 573 | (IP-VRF)|----| |---|(IP-VRF)|--- Rcvr5 574 | / | | | +--------+ 575 Rcvr2 +---|(MAC-VRF2) | | | 576 +------------+ | | 577 | MPLS/ | 578 EVPN PE2 | IP | 579 +------------+ | | 580 Rcvr3 +---|(MAC-VRF1) | | | MVPN PE4 581 | \ | | | +--------+ 582 | (IP-VRF)|----| |---|(IP-VRF)|--- Rcvr6 583 | / | +---------+ +--------+ 584 Rcvr4 +---|(MAC-VRF3) | 585 +------------+ 587 Figure-1: EVPN & MVPN PEs Seamless Interop 589 Figure 2 depicts the modeling of EVPN PEs based on MVPN PEs where an 590 EVPN PE can be modeled as a PE that consists of a MVPN PE whose 591 routed interfaces (e.g., attachment circuits) are replaced with IRB 592 interfaces connecting each IP-VRF of the MVPN PE to a set of BTs. 593 Similar to a MVPN PE where an attachment circuit serves as a routed 594 multicast interface for an IP-VRF associated with a MVPN instance, an 595 IRB interface serves as a routed multicast interface for the IP-VRF 596 associated with the MVPN instance. Since EVPN PEs run MVPN protocols 597 (e.g., [RFC6513] and [RFC6514] ), for all practical purposes, they 598 look just like MVPN PEs to other PE devices. Such modeling of EVPN 599 PEs, transforms the multicast VPN operation of EVPN PEs to that of 600 MVPN and thus simplifies the interoperability between EVPN and MVPN 601 PEs to that of running a single unified solution based on MVPN. 603 Internet-DrafSeamless Multicast Interoperability between EV October 2021 605 EVPN PE1 606 +------------+ 607 Src1 +----|(MAC-VRF1) | 608 | \ | 609 Rcvr1 +----| +--------+| +---------+ +--------+ 610 | |MVPN PE1||----| |---|MVPN PE3|--- Rcvr5 611 | +--------+| | | +--------+ 612 | / | | | 613 Rcvr2 +---|(MAC-VRF2) | | | 614 +------------+ | | 615 | MPLS/ | 616 EVPN PE2 | IP | 617 +------------+ | | 618 Rcvr3 +---|(MAC-VRF1) | | | 619 | \ | | | 620 | +--------+| | | +--------+ 621 | |MVPN PE2||----| |---|MVPN PE4|--- Rcvr6 622 | +--------+| | | +--------+ 623 | / | +---------+ 624 Rcvr4 +---|(MAC-VRF3) | 625 +------------+ 627 Figure-2: Modeling EVPN PEs as MVPN PEs 629 Although modeling an EVPN PE as a MVPN PE, conceptually simplifies 630 the operation to that of a solution based on MVPN, the following 631 operational aspects of EVPN need to be factored in when considering 632 seamless integration between EVPN and MVPN PEs. 634 o Unicast route advertisements for IP multicast source 636 o Multi-homing of IP multicast sources and receivers 638 o Mobility for Tenant's sources and receivers 640 5.3. Unicast Route Advertisements for IP multicast Source 642 When an IP multicast source is attached to an EVPN PE, the unicast 643 route for that IP multicast source needs to be advertised. When the 644 source is attached to a Single-Active multi-homed ES, then the EVPN 645 DF PE is the PE that advertises a unicast route corresponding to the 646 source IP address with VRF Route Import extended community which in 647 turn is used as the Route Target for Join (S,G) messages sent toward 648 the source PE by the remote PEs. The EVPN PE advertises this unicast 649 route using EVPN route type 2 and IPVPN unicast route along with VRF 650 Route Import extended community. EVPN route type 2 is advertised 651 with the Route Targets corresponding to both IP-VRF and MAC-VRF/BT; 652 whereas, IPVPN unicast route is advertised with RT corresponding to 654 Internet-DrafSeamless Multicast Interoperability between EV October 2021 656 the IP-VRF. When unicast routes are advertised by MVPN PEs, they are 657 advertised using IPVPN unicast route along with VRF Route Import 658 extended community per [RFC6514]. 660 When the source is attached to an All-Active multi-homed ES, then the 661 PE that learns the source advertises the unicast route for that 662 source using EVPN route type 2 and IPVPN unicast route along with VRF 663 Route Import extended community. EVPN route type 2 is advertised 664 with the Route Targets corresponding to both IP-VRF and MAC-VRF/BT; 665 whereas, IPVPN unicast route is advertised with RT corresponding to 666 the IP-VRF. When the other multi-homing EVPN PEs for that ES receive 667 this unicast EVPN route, they import the route and check to see if 668 they have learned the route locally for that ES, if they have, then 669 they do nothing. But if they have not, then they add the IP and MAC 670 addresses to their IP-VRF and MAC-VRF/BT tables respectively with the 671 local interface corresponding to that ES as the corresponding route 672 adjacency. Furthermore, these PEs advertise an IPVPN unicast route 673 along with VRF Route Import extended community and Route Target 674 corresponding to IP-VRF to other remote PEs for that MVPN. 675 Therefore, the remote PEs learn the unicast route corresponding to 676 the source from all multi-homing PEs associated with that All- Active 677 Ethernet Segment even though one of the multi-homing PEs may only 678 have directly learned the IP address of the source. 680 EVPN-PEs advertise unicast routes as host routes using EVPN route 681 type 2 for sources that are directly attached to a tenant BD that has 682 been extended in the EVPN fabric. EVPN-PE may summarize sources (IP 683 networks) behind a router that are attached to EVPN-PE or sources 684 that are connected to a BD, which is not extended across EVPN fabric 685 and advertises those routes with EVPN route type 5. EVPN host-routes 686 are advertised as IPVPN host-routes to MVPN-PEs only incase of 687 seamless interop mode. 689 Section 8 extends seamless interop procedures to EVPN only fabrics as 690 an IRB solution for multicast. L3VPN provisioning is not needed 691 between EVPN-PEs. EVPN-PEs only need to advertise unicast routes 692 using EVPN route-type 2 or route-type 5 with VRF Route Import 693 extended community and don't need to advertise IPVPN routes within 694 EVPN only fabric. 696 Section 9 discusses DCI usecases, where EVPN and MVPN networks are 697 connected using gateway model. In gateway model, EVPN-PE advertises 698 unicast routes as IPVPN routes along with VRI extended community for 699 all multicast sources attached behind EVPN-PEs. All IPVPN routes 700 SHOULD be summarized while adverting to MVPN-PEs. 702 Internet-DrafSeamless Multicast Interoperability between EV October 2021 704 5.4. Multi-homing of IP Multicast Source and Receivers 706 EVPN [RFC7432] has extensive multi-homing capabilities that allows 707 TSes to be multi-homed to two or more EVPN PEs in Single-Active or 708 All-Active mode. In Single-Active mode, only one of the multi-homing 709 EVPN PEs can receive/transmit traffic for a given subnet (a given BD) 710 for that multi-homed Ethernet Segment (ES). In All-Active mode, any 711 of the multi-homing EVPN PEs can receive/transmit unicast traffic but 712 only one of them (the DF PE) can send BUM traffic to the multi-homed 713 ES for a given subnet. 715 The multi-homing mode (Single-Active versus All-Active) of a TS 716 source can impact the MVPN procedures as described below. 718 5.4.1. Single-Active Multi-Homing 720 When a TS source reside on an ES that is multi-homed to two or more 721 EVPN PEs operating in Single-Active mode, only one of the EVPN PEs 722 can be active for the source subnet on that ES. Therefore, only one 723 of the multi-homing PE learns the unicast route of the TS source and 724 advertises that using EVPN and IPVPN to other PEs as described 725 previously. 727 A downstream PE that receives a Join/Prune message from a TS host/ 728 router, selects an Upstream Multicast Hop (UMH) which is the upstream 729 PE that receives the IP multicast flow in case of Singe- Active 730 multi-homing. An IP multicast flow belongs to either a source- 731 specific tree (S,G) or to a shared tree (*,G). We use the notation 732 (X,G) to refer to either (S,G) or (*,G); where X refers to S in case 733 of (S,G) and X refers to the Rendezvous Point (RP) for G in case of 734 (*,G). Since the active PE (which is also the UMH PE) has advertised 735 unicast route for X along with the VRF Route Import EC, the 736 downstream PEs selects the UMH without any ambiguity based on MVPN 737 procedures described in section 5.1 of [RFC6513]. 739 The multi-homing PE that receives the IP multicast flow on its local 740 AC, performs the following tasks: 742 - L2 switches the multicast traffic in its BT associated with the 743 local AC over which it received the flow if there are any interested 744 receivers for that subnet. 746 - L3 routes the multicast traffic to other BTs for other subnets if 747 there are any interested receivers for those subnets. 749 - L3 routes the multicast traffic to other PEs per MVPN procedures. 751 Internet-DrafSeamless Multicast Interoperability between EV October 2021 753 The multicast traffic can be sent on Inclusive, Selective, or 754 Aggregate-Selective tree. Regardless of what type of tree is used, 755 only a single copy of the multicast traffic is received by the 756 downstream PEs and the multicast traffic is forwarded optimally from 757 the upstream PE to the downstream PEs. 759 5.4.2. All-Active Multi-Homing 761 When a TS source reside on an ES that is multi-homed to two or more 762 EVPN PEs operating in All-Active mode, then any of the multi-homing 763 PEs can learn the TS source's unicast route; however, that PE may not 764 be the same PE that receives the IP multicast flow. Therefore, the 765 procedures for Single-Active Multi-homing need to be augmented for 766 All-Active scenario as below. 768 The multi-homing EVPN PE that receives the IP multicast flow on its 769 local AC, needs to do the following task in additions to the ones 770 listed in the previous section for Single-Active multi-homing: L2 771 switch the multicast traffic to other multi-homing EVPN PEs for that 772 ES via a multicast tunnel which it is called intra-ES subnet tunnel. 773 There will be a dedicated tunnel for this purpose which is different 774 from inter-subnet overlay tree/tunnel setup by MVPN procedures. 776 When the multi-homing EVPN PEs receive the IP multicast flow via this 777 tunnel, they treat it as if they receive the flow via their local ACs 778 and thus perform the tasks mentioned in the previous section for 779 Single-Active multi-homing. The tunnel type for this intra-ES subnet 780 tunnel can be any of the supported tunnel types such as ingress- 781 replication, P2MP tunnel, BIER, and Assisted Replication; however, 782 given that vast majority of multi-homing ESes are just dual-homing, a 783 simple ingress replication tunnel can serve well. For a given ES, 784 since multicast traffic that is locally received by one multi-homing 785 PE is sent to other multi-homing PEs via this intra-ES subnet tunnel, 786 there is no need for sending the multicast tunnel via MVPN tunnel to 787 these multi-homing PEs - i.e., MVPN multicast tunnels are used only 788 for remote EVPN and MVPN PEs. Multicast traffic sent over this 789 intra-ES subnet tunnel to other multi-homing PEs for a given ES can 790 be either fixed or on demand basis. 792 By feeding IP multicast flow received on one of the EVPN multi-homing 793 PEs to the interested EVPN PEs in the same multi-homing group, we 794 have essentially enabled all the EVPN PEs in the multi-homing group 795 to serve as UMH for that IP multicast flow. Each of these UMH PEs 796 advertises unicast route for X in (X,G) along with the VRF Route 797 Import EC to all PEs for that MVPN instance. The downstream PEs 798 build a candidate UMH set based on procedures described in section 799 5.1 of [RFC6513] and pick a UMH from the set. It should be noted 800 that both the default UMH selection procedure based on highest UMH PE 802 Internet-DrafSeamless Multicast Interoperability between EV October 2021 804 IP address and the UMH selection algorithm based on hash function 805 specified in section 5.1.3 of [RFC6513] (which is also a MUST 806 implement algorithm) result in the same UMH PE be selected by all 807 downstream PEs running the same algorithm. However, in order to 808 allow a form of "equal cost load balancing", the hash algorithm is 809 recommended to be used among all EVPN and MVPN PEs. This hash 810 algorithm distributes UMH selection for different IP multicast flows 811 among the multi-homing PEs for a given ES. 813 Since all downstream PEs (EVPN and MVPN) use the same hash-based 814 algorithm for UMH determination, they all choose the same upstream PE 815 as their UMH for a given (X,G) flow and thus they all send their 816 (X,G) join message via BGP to the same upstream PE. This results in 817 one of the multi-homing PEs to receive the join message and thus send 818 the IP multicast flow for (X,G) over its associated overlay tree even 819 though all of the multi-homing PEs in the All-Active redundancy group 820 have received the IP multicast flow (one of them directly via its 821 local AC and the rest indirectly via the associated intra-ES subnet 822 tunnel). Therefore, only a single copy of routed IP multicast flow 823 is sent over the network regardless of overlay tree type supported by 824 the PEs - i.e., the overlay tree can be of type selective or 825 aggregate selective or inclusive tree. This gives the network 826 operator the maximum flexibility for choosing any overlay tree type 827 that is suitable for its network operation and still be able to 828 deliver only a single copy of the IP multicast flows to the egress 829 PEs. In other words, an egress PE only receives a single copy of the 830 IP multicast flow over the network, because it either receives it via 831 the EVPN intra-ES subnet tunnel or MVPN inter-subnet tunnel. 832 Furthermore, if it receives it via MVPN inter-subnet tunnel, then 833 only one of the multi- homing PEs associated with the source ES, 834 sends the IP multicast traffic. 836 Since the network of interest for seamless interoperability between 837 EVPN and MVPN PEs is MPLS, the EVPN handling of BUM traffic for MPLS 838 network needs to be considered. EVPN [RFC7432] uses ESI MPLS label 839 for split-horizon filtering of Broadcast/Unknown unicast/multicast 840 (BUM) traffic from an All-Active multi-homing Ethernet Segment to 841 ensure that BUM traffic doesn't get loop back to the same Ethernet 842 Segment that it came from. This split-horizon filtering mechanism 843 applies as-is for multicast IRB scenario because of using the intra- 844 ES tunnel among multi-homing PEs. Since the multicast traffic 845 received from a TS source on an All-Active ES by a multi-homing PE is 846 bridged to all other multi-homing PEs in that group, the standard 847 EVPN split-horizon filtering described in [RFC7432] applies as-is. 849 Internet-DrafSeamless Multicast Interoperability between EV October 2021 851 5.5. Mobility for Tenant's Sources and Receivers 853 When a tenant system (TS), source or receiver, is multi-homed behind 854 a group of multi-homing EVPN PEs, then TS mobility SHALL be supported 855 among EVPN PEs. Furthermore, such TS mobility SHALL only cause an 856 temporary disruption to the related multicast service among EVPN and 857 MVPN PEs. If a source is moved from one EVPN PE to another one, then 858 the EVPN mobility procedure SHALL discover this move and a new 859 unicast route advertisement (using both EVPN and IPVPN routes) is 860 made by the EVPN PE where the source has moved to per section 5.3 861 above and unicast route withdraw (for both EVPN and IPVPN routes) is 862 performed by the EVPN PE where the source has moved from. 864 The move of a source results in disruption of the IP multicast flow 865 for the corresponding (S,G) flow till the new unicast route 866 associated with the source is advertised by the new PE along with the 867 VRF Route Import EC, the join messages sent by the egress PEs are 868 received by the new PE, the multicast state for that flow is 869 installed in the new PE and a new overlay tree is built for that 870 source from the new PE to the egress PEs that are interested in 871 receiving that IP multicast flow. 873 The move of a receiver results in disruption of the IP multicast flow 874 to that receiver only till the new PE for that receiver discovers the 875 source and joins the overlay tree for that flow. 877 6. Control Plane Operation 879 In seamless interop between EVPN and MVPN PEs, the control plane need 880 to setup the following three types of multicast tunnels. The first 881 two are among EVPN PEs and are associated with attached BD, but the 882 third one is among EVPN and MVPN PEs and is associated with tenant- 883 VRF 885 1) Intra-ES subnet tunnel 887 2) Intra-subnet BUM tunnel 889 3) Inter-subnet IP multicast tunnel 891 6.1. Intra-ES Subnet Tunnel 893 As described in section 5.4.2, when a multicast source is sitting 894 behind an All-Active ES, then an intra-subnet multicast tunnel is 895 needed among the multi-homing EVPN PEs for that ES to carry multicast 896 flow received by one of the multi-homing PEs to the other PEs in that 897 ES. We refer to this multicast tunnel as Intra-ES subnet tunnel. 898 Vast majority of All-Active multi-homing for TOR devices in DC 900 Internet-DrafSeamless Multicast Interoperability between EV October 2021 902 networks are just dual-homing which means the multicast flow received 903 by one of the dual-homing PE only needs to be sent to the other dual- 904 homing PE. Therefore, a simple ingress replication tunnel is all 905 that is needed. In case of multi-homing to three or more EVPN PEs, 906 then other tunnel types such as P2MP, MP2MP, BIER, and Assisted 907 Replication can be considered. It should be noted that this intra-ES 908 subnet tunnel is only needed for All-Active multi-homing and it is 909 not required for Single- Active multi-homing. 911 The EVPN PEs belonging to a given All-Active ES discover each other 912 using EVPN Ethernet Segment route per procedures described in 913 [RFC7432]. These EVPN PEs perform DF election per [RFC7432], 914 [RFC8584], or other DF election algorithms to decide who is a DF for 915 a given BD. If the BD belongs to a tenant that has IRB IP multicast 916 enabled for it, then for fixed-mode, each PE sets up an intra-ES 917 subnet tunnel to forward IP multicast traffic received locally on 918 that BD to other multi-homing PE(s) for that ES. Therefore, IP 919 multicast traffic received via a local attachment circuit is sent on 920 this tunnel and on the associated IRB interface for that BT and other 921 local attachment circuits if there are interested receivers for them. 922 The other multi-homing EVPN PEs treat this intra-ES subnet tunnel 923 just like their local ACs - i.e., the multicast traffic received over 924 this tunnel is treated as if it is received via its local AC. Thus, 925 the multi-homing PEs cannot receive the same IP multicast flow from 926 an MVPN tunnel (e.g., over an IRB interface for that BD) because 927 between a source behind a local AC versus a source behind a remote 928 PE, the PE always chooses its local AC. 930 When all multihomed PE support [I-D.ietf-bess-evpn-igmp-mld-proxy], 931 traffic may be forwarded on demand basis. Based on IGMP 932 synchronization procedure specified in 933 [I-D.ietf-bess-evpn-igmp-mld-proxy], join state may be synchronized 934 between all multihomed PEs. Multihomed PE which receives the 935 multicast traffic from its attached circuit, may send the traffic 936 towards intra-ES subnet tunnel, only if it has received IGMP sync 937 message from one of the multihomed PEs. Such extension is outside 938 the scope of this document and may be covered in a separate document, 939 if required. 941 If a source exists behind inter-subnet tunnel, it is possible that 942 more than one multihomed PEs send MVPN join towards remote PE based 943 on incoming join on their local interfaces. When the traffic is 944 received on the inter-subnet tunnel, it is sent towards locally 945 attached receivers. Only DF sends traffic towards multihomed 946 ethernet segment. Traffic received on the inter-subnet tunnel, 947 should not be sent towards Intra-ES subnet tunnel. 949 Internet-DrafSeamless Multicast Interoperability between EV October 2021 951 When ingress replication is used for intra-ES subnet tunnel, every PE 952 in the All-Active multi-homing ES has all the information to setup 953 these tunnels - i.e., a) each PE knows what are the other multi- 954 homing PEs for that ES via EVPN Ethernet Segment route and can use 955 this information to setup intra-ES subnet tunnel among themselves. 957 6.2. Intra-Subnet BUM Tunnel 959 As the name implies, this tunnel is setup to carry BUM traffic for a 960 given subnet/BD among EVPN PEs. In [RFC7432], this overlay tunnel is 961 used for transmission of all BUM traffic including tenant IP 962 multicast traffic. 964 When an EVPN IRB PE operates in seamless interop mode, this tunnel is 965 used for all broadcast, unknown-unicast, non-IP multicast traffic, 966 and link-local IP multicast traffic - i.e., it is used for all BUM 967 traffic except tenant IP multicast traffic. This tunnel is setup 968 using IMET route for a given EVI/BD. The composition and 969 advertisement of IMET routes are exactly per [RFC7432]. It should be 970 noted that when an EVPN All-Active multi-homing PE uses both this 971 tunnel as well as intra-ES subnet tunnel, there SHALL be no 972 duplication of multicast traffic over the network because they carry 973 different types of multicast traffic - i.e., intra-ES subnet tunnel 974 among multi-homing PEs carries only tenant IP multicast traffic; 975 whereas, intra-subnet BUM tunnel carries link-local IP multicast 976 traffic and BUM traffic (w/ non-IP multicast). 978 6.3. Inter-Subnet IP Multicast Tunnel 980 As its name implies, this tunnel is setup to carry IP-only multicast 981 traffic for a given tenant across all its subnets (BDs) among EVPN 982 and MVPN PEs. 984 The following NLRIs from [RFC6514] is used for setting up this inter- 985 subnet tunnel in the network. 987 Intra-AS I-PMSI A-D route is used for the setup of default 988 underlay tunnel (also called inclusive tunnel) for a tenant IP- 989 VRF. The tunnel attributes are indicated using PMSI attribute 990 with this route. 992 S-PMSI A-D route is used for the setup of Customer flow specific 993 underlay tunnels. This enables selective delivery of data to PEs 994 having active receivers and optimizes fabric bandwidth 995 utilization. The tunnel attributes are indicated using PMSI 996 attribute with this route. 998 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1000 Each EVPN PE supporting a specific MVPN instance discovers the set of 1001 other PEs in its AS that are attached to sites of that MVPN using 1002 Intra-AS I-PMSI A-D route (route type 1) per [RFC6514]. It can also 1003 discover the set of other ASes that have PEs attached to sites of 1004 that MVPN using Inter-AS I-PMSI A-D route (route type 2) per 1005 [RFC6514]. After the discovery of PEs that are attached to sites of 1006 the MVPN, an inclusive overlay tree (I-PMSI) can be setup for 1007 carrying tenant multicast flows for that MVPN; however, this is not a 1008 requirement per [RFC6514] and it is possible to adopt a policy in 1009 which all tenant flows are carried on S-PMSIs. 1011 An EVPN-IRB PE sends a tenant IP multicast flow to other EVPN and 1012 MVPN PEs over this inter-subnet tunnel that is instantiated using 1013 MVPN I- PMSI or S-PMSI. This tunnel can be considered as being 1014 originated and terminated from/to among IP-VRFs of EVPN/MVPN PEs; 1015 whereas, intra- subnet tunnel is originated/terminated among MAC-VRFs 1016 of EVPN PEs. 1018 6.4. IGMP Hosts as TSes 1020 IGMP messages are terminated by the EVPN-IRB PE and tenant (*,G) or 1021 (S,G) join messages are sent via MVPN Shared Tree Join route (route 1022 type 6) or Source Tree Join route (route type 7) respectively of 1023 MCAST-VPN NLRI per [RFC6514]. 1025 Here, IGMP states are terminated at IRB interfaces and local interest 1026 are expressed in the context of IP-VRF to remote PEs. Hence, If a 1027 tenant system which is an IGMP host is multi-homed to two or more 1028 EVPN PEs using All-Active multi-homing, there is no need to sync IGMP 1029 join and leave messages between these EVPN PEs using EVPN IGMP Join 1030 Synch route (route type 7) and EVPN IGMP Leave Synch route (route 1031 type 8) per [I-D.ietf-bess-evpn-igmp-mld-proxy]. 1033 In case of a network with only IGMP hosts, the preferred mode of 1034 operation is that of Shortest Path Tree(SPT) per section 14 of 1035 [RFC6514]. This mode is only supported for PIM-SM and avoids the RP 1036 configuration overhead. Such mode is chosen by provisioning/ 1037 configuration. 1039 6.5. PIM Routers as TSes 1041 Just like a MVPN PE, an EVPN PE runs a separate tenant multicast 1042 routing instance (VPN-specific) per MVPN instance and the following 1043 tenant multicast routing instances are supported: 1045 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1047 - PIM Sparse Mode (PIM-SM) with the ASM service model 1048 - PIM Sparse Mode with the SSM service model 1049 - PIM Bidirectional Mode (BIDIR-PIM), which uses bidirectional 1050 tenant-trees to support the ASM service model 1052 A given tenant's PIM join messages for (*,G) or (S, G) are processed 1053 by the corresponding tenant multicast routing protocol and they are 1054 advertised over MPLS/IP network using Shared Tree Join route (route 1055 type 6) and Source Tree Join route (route type 7) respectively of 1056 MCAST-VPN NLRI per [RFC6514]. 1058 7. Data Plane Operation 1060 When an EVPN-IRB PE receives an IGMP/MLD join message over one of its 1061 Attachment Circuits (ACs), it adds that AC to its Layer-2 (L2) OIF 1062 list. This L2 OIF list is associated with the MAC-VRF/BT 1063 corresponding to the subnet of the tenant device that sent the IGMP/ 1064 MLD join. Therefore, tenant (S,G) or (*,G) forwarding entries are 1065 created/updated for the corresponding MAC-VRF/BT based on these 1066 source and group IP addresses. Furthermore, the IGMP/MLD join 1067 message is propagated over the corresponding IRB interface and it is 1068 processed by the tenant multicast routing instance which creates the 1069 corresponding tenant (S,G) or (*,G) Layer-3 (L3) forwarding entries. 1070 It adds this IRB interface to the L3 OIF list. An IRB is removed as 1071 a L3 OIF when all L2 tenant (S,G) or (*,G) forwarding states is 1072 removed for the MAC-VRF/BT associated with that IRB. Furthermore, 1073 tenant (S,G) or (*,G) L3 forwarding state is removed when all of its 1074 L3 OIFs are removed - i.e., all the IRB and L3 interfaces associated 1075 with that tenant (S,G) or (*,G) are removed. 1077 When an EVPN PE receives IP multicast traffic from one of its AC, if 1078 it has any attached receivers for that subnet, it performs L2 1079 switching of the intra-subnet traffic within the BT attached to that 1080 AC. If the multicast flow is received over an AC that belongs to an 1081 All-Active ES, then the multicast flow is also sent over the intra- 1082 ES subnet tunnel among multi-homing PEs. The EVPN PE then sends the 1083 multicast traffic over the corresponding IRB interface. The 1084 multicast traffic then gets routed in the corresponding IP-VRF and it 1085 gets forwarded to interfaces in the L3 OIF list which can include 1086 other IRB interfaces, other L3 interfaces directly connected to TSes, 1087 and the MVPN Inter-Subnet tunnel which is instantiated by an I-PMSI 1088 or S-PMSI tunnel. When the multicast packet is routed within the IP- 1089 VRF of the EVPN PE, its Ethernet header is stripped and its TTL gets 1090 decremented as the result of this IP routing. Remote multicast 1091 traffic that is received from MVPN Inter-Subnet tunnel gets routed 1092 towards all L3 OIFs. When the multicast traffic is received on an 1093 IRB interface by the BT corresponding to that interface, it gets L2 1094 switched and sent over ACs that belong to the L2 OIF list. 1096 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1098 7.1. Intra-Subnet L2 Switching 1100 Rcvr1 in Figure 1 is connected to PE1 in MAC-VRF1 (same as Src1) and 1101 sends IGMP join for (C-S, C-G), IGMP snooping will record this state 1102 in local bridging entry. A routing entry will be formed as well 1103 which will point to MAC-VRF1 as RPF for Src1. We assume that Src1 is 1104 known via ARP or similar procedures. Rcvr1 will get a locally 1105 bridged copy of multicast traffic from Src1. Rcvr3 is also connected 1106 in MAC-VRF1 but to PE2 and hence would send IGMP join which will be 1107 recorded at PE2. PE2 will also form routing entry and RPF will be 1108 assumed as Tenant Tunnel "Tenant1" formed beforehand using MVPN 1109 procedures. Also this would cause multicast control plane to 1110 initiate a BGP MCAST-VPN type 7 route which would include VRI for PE1 1111 and hence be accepted on PE1. PE1 will include Tenant1 tunnel as 1112 Outgoing Interface (OIF) in the routing entry. Now, since it has 1113 knowledge of remote receivers via MVPN control plane it will 1114 encapsulate original multicast traffic in Tenant1 tunnel towards 1115 core. 1117 7.2. Inter-Subnet L3 Routing 1119 Rcvr2 in Figure 1 is connected to PE1 in MAC-VRF2 and hence PE1 will 1120 record its membership in MAC-VRF2. Since MAC-VRF2 is enabled with 1121 IRB, it gets added as another OIF to routing entry formed for (C-S, 1122 C-G). Rcvr2 and Rcvr4 are also in different MAC-VRFs than multicast 1123 speaker Src1 and hence need Inter-subnet forwarding. PE2 now adds 1124 another OIF 'MAC-VRF2' to its existing routing entry. But there is 1125 no change in control plane states since it is already sent MVPN route 1126 and no further signaling is required. Traffic received on the tenant 1127 tunnel interface gets routed towards both MAC-VRF1 and MAC-VRF3. PE3 1128 forms routing entry very similar to PE2. It is to be noted that PE3 1129 does not have MAC-VRF1 configured locally but still can receive the 1130 multicast data traffic over Tenant1 tunnel formed due to MVPN 1131 procedures and routes traffic towards its L3 OIFs for that (C-S,C-G). 1133 8. DCs with only EVPN PEs 1135 As mentioned earlier, the proposed solution can be used as a routed 1136 multicast solution in data center networks with only EVPN PEs (e.g., 1137 routed multicast VPN only among EVPN PEs). 1139 As per section 5.2, EVPN PE is modeled as a PE that consists of a 1140 MVPN PE whose routed interfaces (e.g., attachment circuits) are 1141 replaced with IRB interfaces connecting each IP-VRF of the MVPN PE to 1142 a set of BTs. Due to this, the IP multicast traffic that needs to be 1143 forwarded from the source PE to remote PEs is routed to remote PEs 1144 regardless of whether the traffic is intra-subnet or inter-subnet. 1146 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1148 As the result, the TTL value for intra-subnet traffic that spans 1149 across two or more PEs get decremented. 1151 However, if there are applications that require intra-subnet 1152 multicast traffic to be L2 forwarded, Appendix A discusses some 1153 options to support applications having TTL value 1. The procedure 1154 discussed in Appendix A may be used to support applications that 1155 require intra-subnet multicast traffic to be L2 forwarded. 1157 8.1. Setup of overlay multicast delivery 1159 It must be emphasized that this solution poses no restriction on the 1160 setup of the tenant BDs and that neither the source PE, nor the 1161 receiver PEs do not need to know/learn about the BD configuration on 1162 other PEs in the tenant VRF ( Since EVPN PE is modelled as MVPN PE, 1163 source and receivers are announced to remote PE in the context of 1164 tenant VRF(MVPN) as opposed to BD context). The Reverse Path 1165 Forwarder (RPF) is selected per the tenant multicast source and the 1166 IP-VRF in compliance with the procedures in [RFC6514], using the 1167 incoming EVPN route type 2 or 5 NLRI per [RFC7432]. 1169 The VRF Route Import (VRI) extended community that is carried with 1170 the IPVPN routes in [RFC6514] MUST be carried with the EVPN unicast 1171 routes when these routes are used. The construction and processing 1172 of the VRI are consistent with [RFC6514]. The VRI MUST uniquely 1173 identify the PE which is advertising a multicast source and the IP- 1174 VRF it resides in. 1176 VRI is constructed as following: 1178 - The 4-octet Global Administrator field MUST be set to an IP 1179 address of the PE. This address SHOULD be common for all the 1180 IP-VRFs on the PE (e.g., this address may be the PE's loopback 1181 address or VTEP address). 1183 - The 2-octet Local Administrator field associated with a given 1184 IP-VRF contains a number that uniquely identifies that IP-VRF 1185 within the PE that contains the IP-VRF. 1187 EVPN PE MUST have Route Target Extended Community to import/export 1188 MVPN routes. In data center environment, it is desirable to have 1189 this RT configured using auto-generated method than static 1190 configuration. 1192 The following is one recommended model to auto-generate MVPN RT: 1194 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1196 - The Global Administrator field of the MVPN RT MAY be set 1197 to BGP AS Number. 1199 - The Local Administrator field of the MVPN RT MAY be set to 1200 the VNI associated with the tenant VRF. 1202 Every PE which detects a local receiver via a local IGMP join or a 1203 local PIM join for a specific source (overlay SSM mode) MUST 1204 terminate the IGMP/PIM signaling at the IP-VRF and generate a (C-S,C- 1205 G) via the BGP MCAST-VPN route type 7 per [RFC6514] if and only if 1206 the RPF for the source points to the fabric. If the RPF points to a 1207 local multicast source on the same MAC-VRF or a different MAC-VRF on 1208 that PE, the MCAST-VPN MUST NOT be advertised and data traffic will 1209 be locally routed/bridged to the receiver. 1211 The VRI received with EVPN route type 2 or 5 NLRI from source PE will 1212 be appended as an export route-target extended community. The PE 1213 which has advertised the unicast route with VRI, will import the 1214 incoming MCAST-VPN NLRI in the IP-VRF with the same import route- 1215 target extended-community and other PEs SHOULD ignore it. Following 1216 such procedure the source PE learns about the existence of at least 1217 one remote receiver in the tenant overlay and programs data plane 1218 accordingly so that a single copy of multicast data is forwarded into 1219 the fabric using tenant VRF tunnel(i.e. inter-subnet tunnel/mvpn 1220 tunnel). 1222 If the multicast source is unknown (overlay ASM mode), the MCAST-VPN 1223 route type 6 (C-*,C-G) join SHOULD be targeted towards the designated 1224 overlay Rendezvous Point (RP) by appending the received RP VRI as an 1225 export route-target extended community. Every PE which detects a 1226 local source, registers with its RP PE. That is how the RP learns 1227 about the tenant source(s) and group(s) within the MVPN. Once the 1228 overlay RP PE receives either the first remote (C-RP,C-G) join or a 1229 local IGMP/PIM join, it will trigger an MCAST-VPN route type 7 (C- 1230 S,C-G) towards the actual source PE for which it has received PIM 1231 register message in full compliance with regular PIM procedures. 1232 This involves the source PE to advertise the MCAST-VPN Source Active 1233 A-D route (MCAST-VPN route-type 5) towards all PEs. The Source 1234 Active A- D route is used to inform all PEs in a given MVPN about the 1235 active multicast source for switching from RPT to SPT when MVPNs use 1236 tenant RP-shared trees (i.e., rooted at tenant's RP) per section 13 1237 of [RFC6514]. 1239 8.2. Handling of different encapsulations 1241 Just as in [RFC6514] the MVPN I-PMSI and S-PMSI A-D routes are used 1242 to form the overlay multicast tunnels and signal the tunnel type 1244 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1246 using the P-Multicast Service Interface Tunnel (PMSI Tunnel) 1247 attribute. 1249 8.2.1. MPLS Encapsulation 1251 The [RFC6514] assumes MPLS/IP core and there is no modification to 1252 the signaling procedures and encoding for PMSI tunnel formation 1253 therein. Also, there is no need for a gateway to inter-operate with 1254 non-EVPN PEs supporting [RFC6514] based MVPN over IP/MPLS. 1256 8.2.2. VxLAN Encapsulation 1258 In order to signal VXLAN, the corresponding BGP encapsulation 1259 extended community [RFC9012] SHOULD be appended to the MVPN I- PMSI 1260 and S-PMSI A-D routes. The MPLS label in the PMSI Tunnel Attribute 1261 MUST be the Virtual Network Identifier (VNI) associated with the 1262 customer MVPN. The supported PMSI tunnel types with VXLAN 1263 encapsulation are: PIM-SSM Tree, PIM-SM Tree, BIDIR-PIM Tree, Ingress 1264 Replication [RFC6514]. Further details are in [RFC8365]. 1266 In this case, a gateway is needed for inter-operation between the 1267 EVPN PEs and non-EVPN MVPN PEs. The gateway should re-originate the 1268 control plane signaling with the relevant tunnel encapsulation on 1269 either side. In the data plane, the gateway terminates the tunnels 1270 formed on either side and performs the relevant stitching/re- 1271 encapsulation on data packets. 1273 8.2.3. Other Encapsulation 1275 In order to signal a different tunneling encapsulation such as NVGRE, 1276 GPE, or GENEVE the corresponding BGP encapsulation extended community 1277 [RFC9012] SHOULD be appended to the MVPN I-PMSI and S-PMSI A-D 1278 routes. If the Tunnel Type field in the encapsulation extended- 1279 community is set to a type which requires Virtual Network Identifier 1280 (VNI), e.g., VXLAN-GPE or NVGRE [RFC9012], then the MPLS label in the 1281 PMSI Tunnel Attribute MUST be the VNI associated with the customer 1282 MVPN. Same as in VXLAN case, a gateway is needed for inter- 1283 operation between the EVPN-IRB PEs and non-EVPN MVPN PEs. 1285 9. DCI with MPLS in WAN and VxLAN in DCs 1287 This section describes the inter-operation between MVPN PEs in WAN 1288 using MPLS encapsulation with EVPN PEs in a DC network using VxLAN 1289 encapsulation. Since the tunnel encapsulation between these networks 1290 are different, we must have at least one gateway in between. 1291 Usually, two or more are required for redundancy and load balancing 1292 purpose. In such scenarios, a DC network can be represented as a 1293 customer network that is multi-homed to two or more MVPN PEs via L3 1295 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1297 interfaces and thus standard MVPN multi-homing procedures are 1298 applicable here. It should be noted that a MVPN overlay tunnel over 1299 the DC network is terminated on the IP-VRF of the gateway and not the 1300 MAC-VRF/BTs. Therefore, the considerations for loop prevention and 1301 split-horizon filtering described in [RFC9014] are not applicable 1302 here. . 1304 9.1. Control plane inter-connect 1306 The gateway(s) MUST be setup with the inclusive set of all the IP- 1307 VRFs that span across the two domains. On each gateway, there will 1308 be at least two BGP sessions: one towards the DC side and the other 1309 towards the WAN side. Usually for redundancy purpose, more sessions 1310 are setup on each side. The unicast route propagation follows the 1311 exact same procedures in [RFC9014]. Hence, a multicast host located 1312 in either domain, is advertised with the gateway IP address as the 1313 next-hop to the other domain. As a result, PEs view the hosts in the 1314 other domain as directly attached to the gateway and all inter-domain 1315 multicast signaling is directed towards the gateway(s). Received 1316 MVPN routes type 1-7 from either side of the gateway(s), MUST NOT be 1317 reflected back to the same side but processed locally and re- 1318 advertised (if needed) to the other side: 1320 o Intra-AS/Inter-AS I-PMSI A-D Route: these are distributed within 1321 each domain to form the overlay tunnels which terminate at 1322 gateway(s). They are not passed to the other side of the 1323 gateway(s). 1325 o C-Multicast Route: joins are imported into the corresponding IP- 1326 VRF on each gateway and advertised as a new route to the other 1327 side with the following modifications (the rest of NLRI fields and 1328 path attributes remain on-touched): 1330 * Route-Distinguisher is set to that of the IP-VRF 1332 * Route-target is set to the exported route-target list on IP-VRF 1334 * The PMSI tunnel attribute and BGP Encapsulation extended 1335 community will be modified according to section 8 1337 * Next-hop will be set to the IP address which represents the 1338 gateway on either domain 1340 o Source Active A-D Route: same as joins 1342 o S-PMSI A-D Route: these are passed to the other side to form 1343 selective PMSI tunnels per every (C-S,C-G) from the gateway to the 1344 PEs in the other domain provided it contains receivers for the 1346 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1348 given (C-S, C-G). Similar modifications made to joins are made to 1349 the newly originated S-PMSI. 1351 In addition, the Originating Router's IP address is set to GW's IP 1352 address. Multicast signaling from/to hosts on local ACs on the 1353 gateway(s) are generated and propagated in both domains (if needed) 1354 per the procedures in section 6 in this document and in [RFC6514] 1355 with no change. It must be noted that for a locally attached source, 1356 the gateway will program an OIF per every domain from which it 1357 receives a remote join in its forwarding plane and different 1358 encapsulation will be used on the data packets. 1360 9.2. Data plane inter-connect 1362 Traffic forwarding procedures on gateways are same as those described 1363 for PEs in section 5 except that, unlike a non-border leaf PE, the 1364 gateway will not only route the incoming traffic from one side to its 1365 local receivers, but will also send it to the remote receivers in the 1366 other domain after de-capsulation and appending the right 1367 encapsulation. The OIF and IIF are programmed in FIB based on the 1368 received joins from either side and the RPF calculation to the source 1369 or RP. The de-capsulation and encapsulation actions are programmed 1370 based on the received I-PMSI or S-PMSI A-D routes from either side. 1372 The multicast traffic from local sources on each gateway may flow to 1373 the other gateway with either of the tunnel encapsulation. But, it 1374 is recommended to use VxLAN tunnel than MPLS in this case. 1376 10. Interop with L2 EVPN PEs 1378 A gateway device is needed to do interop between EVPN PEs that 1379 support seamless interop procedure specified in this document and 1380 L2EVPN-PEs. A tenant domain can be provisioned with one or more such 1381 gateway devices known as "Seamless interop EVPN Multicast Gateway 1382 (SEMG)". PE that is configured as SEMG must be provisioned with all 1383 BDs that are available in the tenant domain. 1385 When advertising IMET route for a BD, PE configured as SEMG 1386 advertises EVPN Multicast Flags Extended Community with SEMG flag 1387 set. Given set of eligible PEs, one PE is selected as the SEMG 1388 designated forwarder (SEMG-DF). PE should use procedure specified in 1389 [RFC8584] for the SEMG DF election. 1391 There are multiple possibilities that need to be considered here. 1393 o L2EVPN PE may or may not have support for 1394 [I-D.ietf-bess-evpn-igmp-mld-proxy] 1396 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1398 o Seamless interop PE may or may not support 1399 [I-D.ietf-bess-evpn-igmp-mld-proxy] 1401 o Network may only have L2EVPN PE and Seamless interop capable PE 1403 o Network may have L2EVPN PE, Seamless interop capable PE and MVPN 1404 PE. 1406 Multicast sources and receivers can exist anywhere in the network. 1407 These usecases are discussed below. 1409 10.1. Interaction with L2EVPN PE and Seamless interop capable PE 1411 The following cases are considered in this section. 1413 o Case1: [I-D.ietf-bess-evpn-igmp-mld-proxy] is supported both at 1414 seamless interop capable PE and L2EVPN PE. 1416 o Case2: [I-D.ietf-bess-evpn-igmp-mld-proxy] is supported only at 1417 seamless interop capable PE. 1419 o Case3: [I-D.ietf-bess-evpn-igmp-mld-proxy] is not supported at 1420 interop capable PE. 1422 [I-D.ietf-bess-evpn-igmp-mld-proxy] support is recommended for 1423 seamless interop capable PE. SEMG can group L2 EVPN PEs into two 1424 separate groups ( one that supports the 1425 [I-D.ietf-bess-evpn-igmp-mld-proxy] and another that doesn't) from 1426 IMET routes that it receives from the remote peers. The interop 1427 procedure for handling these two different sets of remote L2 EVPN PEs 1428 are captured in case 1 and 2. 1430 Case 1: [I-D.ietf-bess-evpn-igmp-mld-proxy] is supported both at 1431 seamless interop capable PE and L2EVPN PE 1433 This may be the most common usecase. 1435 SEMG-DF has the following special responsibilities on a BD for which 1436 it is the DF. 1438 o Process EVPN SMET routes from the remote L2 EVPN PEs that support 1439 [I-D.ietf-bess-evpn-igmp-mld-proxy] and creates L2 multicast 1440 state. SMET route in-turn triggers the creation of L3 multicast 1441 state similar to IGMP join received on the local AC. SEMG-DF 1442 exercises the MVPN procedures for the join. 1444 o It should not process IGMP control packets from L2EVPN PE that 1445 supports [I-D.ietf-bess-evpn-igmp-mld-proxy]. 1447 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1449 o Originate SMET(*,*) route towards L2 EVPN PEs. This is to receive 1450 traffic from multicast sources that are connected behind L2 EVPN 1451 PEs. 1453 o When SEMG-DF receives traffic from L2 EVPN PE on the intra-subnet 1454 tunnel on BD-X, it does the following 1456 * Performs FHR functionality 1458 * Advertises the host route with L3 label and VRF Route-Import 1459 corresponds to the tenant domain. 1461 * Sends the traffic towards the locally attached receivers. 1463 * Sends the traffic towards L2EVPN receiver on BDs other than 1464 incoming BD(after multicast routing) 1466 * Sends the traffic towards remote seamless interop capable PEs, 1467 where receivers are attached/connected behind that PE. 1469 o When SEMG-DF receives traffic from the MVPN tunnel, it does the 1470 following 1472 * Sends the traffic towards the IRB interfaces, where receiver 1473 exists 1475 * BD corresponding to the IRB interfaces may have local receivers 1476 or remote receivers behind L2 EVPN PE. SEMG-DF sends the 1477 traffic on the intra-subnet tunnel for remote receivers. 1479 Case 2: [I-D.ietf-bess-evpn-igmp-mld-proxy] is not supported at L2 1480 EVPN PE 1482 This case only differs from case 1 in terms of the way it learns 1483 receivers behind L2 EVPN PEs and how SEMG-DF attracts traffic from 1484 sources behind L2 EVPN PE. Rest of procedures specified above is 1485 applicable for this case. 1487 SEMG-DF has the following special responsibilities on a BD for which 1488 it is the DF 1490 o Process IGMP control packets from remote L2 EVPN PEs that doesn't 1491 support [I-D.ietf-bess-evpn-igmp-mld-proxy] and create L2 and L3 1492 state. 1494 o When an IGMP query is received on the intra-subnet tunnel on BD-X, 1495 SEMG-DF needs to send proxy IGMP reports for all groups that it 1496 has learned from remote L2-EVPN PEs on that BD. 1498 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1500 o Connecting multicast router behind L2 EVPN PE is not recommended. 1501 If a multicast router is connected behind L2 EVPN PE, the BD 1502 corresponds to VRF tunnel needs to be configured in the L2 EVPN PE 1503 so that PIM router may get all joins that are received in the BD 1504 corresponds to MVPN tunnel interface at SEMG-DF. 1506 o SEMG-DF should get all multicast traffic from L2EVPN PEs. This 1507 may be achieved by sending IGMP query or PIM hello on the intra- 1508 subnet tunnel 1510 Case 3: [I-D.ietf-bess-evpn-igmp-mld-proxy] is not supported at 1511 seamless interop capable PE 1513 The procedure of handling this use case is exactly the same as case 1514 2. 1516 All seamless interop capable PEs other than SEMG should discard SMET 1517 routes that are coming from L2EVPN PEs and must discard all IGMP 1518 control packets, if any received on the intra-subnet tunnel. SEMG 1519 should discard incoming SMET routes and IGMP joins from L2EVPN PEs, 1520 if it is not the DF for the incoming BD. 1522 When [I-D.ietf-bess-evpn-igmp-mld-proxy] is supported both at 1523 seamless interop capable PE and L2EVPN PE, selective forwarding is 1524 done based on receiver interest at the egress-PE, when overlay tunnel 1525 type is Ingress-replication or selective tunnel. 1527 10.2. Network having L2EVPN PE, Seamless interop capable PE and MVPN PE 1529 Since MVPN-PE can only interact with Seamless interop capable PEs, 1530 SEMG-DF acts as FHR and LHR for sources and receivers behind L2 EVPN 1531 PE. Only SEMG-DF advertises IPVPN unicast route along with VRF Route 1532 Import extended community for hosts behind L2 EVPN PE. No additional 1533 procedures are required, when they all co-exist. 1535 11. Connecting external Multicast networks or PIM routers. 1537 External multicast networks or PIM routers can be attached to any 1538 seamless interop capable EVPN-PEs or set of EVPN-PEs or MVPN-PEs. 1539 Multicast network or PIM router can also be attached to any IRB 1540 enabled interface or set of interfaces . The fabric can be used as a 1541 Transit network for connecting the external multicast networks. All 1542 PIM signaling is terminated at PE's IRB interfaces. 1544 No additional procedures are required while connecting external 1545 multicast networks. 1547 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1549 12. TS RP options 1551 RP can be configured in the EVPN-PE itself in the tenant VRF or in 1552 the external multicast networks connected behind an EVPN PE or in the 1553 MVPN network. When RPF is not local to EVPN-PE, EVPN-PE operates in 1554 rpt-spt mode as PER procedures specified in section 13 of [RFC6514]. 1556 EVPN fabric without having any external multicast network/attached 1557 MVPN network, doesn't need RP configuration. A configuration option 1558 SHALL be provided to the end user to operate the fabric in RP less 1559 mode. When an EVPN-PE is operating in RP-less mode, EVPN-PE MUST 1560 advertise all attached sources to remote EVPN PEs using procedure 1561 specified in [RFC6514]. 1563 In RP less mode, (C-*,C-G) RPF may be set to NULL or may be set to 1564 wild card interface( Any interface on the tenant VRF). In RP-less 1565 mode, traffic is always forwarded based on (C-S,C-G) state. 1567 13. IANA Considerations 1569 IANA has allocated the codepoint for Multicast Flags Extended 1570 Community which is defined in [I-D.ietf-bess-evpn-igmp-mld-proxy]. 1572 The Multicast Flags Extended Community contains a 16-bit Flags field. 1573 The bits are numbered 0-15, from high-order to low-order. IANA is 1574 requested to assign the following new flags in the "Multicast Flags 1575 Extended Community Flags" registry. 1577 Bit Name Reference 1578 ---- -------------- ------------- 1579 0-11 Unassigned 1580 12 SEMG This document 1581 13 Seamless interop capable PE This document 1582 14 MLD Proxy Support [I-D.ietf-bess-evpn-igmp-mld-proxy] 1583 15 IGMP Proxy Support [I-D.ietf-bess-evpn-igmp-mld-proxy] 1585 14. Security Considerations 1587 All the security considerations in [RFC7432], [RFC6513] and [RFC6514] 1588 apply directly to this document because this document leverages these 1589 RFCs control plane and their associated procedures. 1591 15. Acknowledgements 1593 The authors would like to thank Niloofar Fazlollahi, Aamod 1594 Vyavaharkar, Raunak Banthia, and Swadesh Agrawal for their 1595 discussions and contributions. 1597 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1599 16. References 1601 16.1. Normative References 1603 [I-D.ietf-bess-evpn-igmp-mld-proxy] 1604 Sajassi, A., Thoria, S., Mishra, M., Drake, J., and W. 1605 Lin, "IGMP and MLD Proxy for EVPN", draft-ietf-bess-evpn- 1606 igmp-mld-proxy-13 (work in progress), September 2021. 1608 [RFC6513] Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/ 1609 BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February 1610 2012, . 1612 [RFC6514] Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP 1613 Encodings and Procedures for Multicast in MPLS/BGP IP 1614 VPNs", RFC 6514, DOI 10.17487/RFC6514, February 2012, 1615 . 1617 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 1618 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 1619 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 1620 2015, . 1622 [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., 1623 Uttaro, J., and W. Henderickx, "A Network Virtualization 1624 Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, 1625 DOI 10.17487/RFC8365, March 2018, 1626 . 1628 [RFC8584] Rabadan, J., Ed., Mohanty, S., Ed., Sajassi, A., Drake, 1629 J., Nagaraj, K., and S. Sathappan, "Framework for Ethernet 1630 VPN Designated Forwarder Election Extensibility", 1631 RFC 8584, DOI 10.17487/RFC8584, April 2019, 1632 . 1634 [RFC9012] Patel, K., Van de Velde, G., Sangli, S., and J. Scudder, 1635 "The BGP Tunnel Encapsulation Attribute", RFC 9012, 1636 DOI 10.17487/RFC9012, April 2021, 1637 . 1639 [RFC9014] Rabadan, J., Ed., Sathappan, S., Henderickx, W., Sajassi, 1640 A., and J. Drake, "Interconnect Solution for Ethernet VPN 1641 (EVPN) Overlay Networks", RFC 9014, DOI 10.17487/RFC9014, 1642 May 2021, . 1644 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1646 [RFC9135] Sajassi, A., Salam, S., Thoria, S., Drake, J., and J. 1647 Rabadan, "Integrated Routing and Bridging in Ethernet VPN 1648 (EVPN)", RFC 9135, DOI 10.17487/RFC9135, October 2021, 1649 . 1651 16.2. Informative References 1653 [RFC4389] Thaler, D., Talwar, M., and C. Patel, "Neighbor Discovery 1654 Proxies (ND Proxy)", RFC 4389, DOI 10.17487/RFC4389, April 1655 2006, . 1657 [RFC4761] Kompella, K., Ed. and Y. Rekhter, Ed., "Virtual Private 1658 LAN Service (VPLS) Using BGP for Auto-Discovery and 1659 Signaling", RFC 4761, DOI 10.17487/RFC4761, January 2007, 1660 . 1662 [RFC7080] Sajassi, A., Salam, S., Bitar, N., and F. Balus, "Virtual 1663 Private LAN Service (VPLS) Interoperability with Provider 1664 Backbone Bridges", RFC 7080, DOI 10.17487/RFC7080, 1665 December 2013, . 1667 [RFC7209] Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N., 1668 Henderickx, W., and A. Isaac, "Requirements for Ethernet 1669 VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014, 1670 . 1672 Appendix A. Supporting application with TTL value 1 1674 It is possible that some deployments may have a host on the tenant 1675 domain that sends multicast traffic with TTL value 1. The interested 1676 receiver for that traffic flow may be attached to different PEs on 1677 the same subnet. The procedures specified in section 5 always routes 1678 the traffic between PEs for both intra and inter subnet traffic. 1679 Hence traffic with TTL value 1 is dropped due to the nature of 1680 routing. 1682 This section discusses few possible ways to support traffic having 1683 TTL value 1 or traffic that require L2 bridging behavior. 1684 Implementation MAY support any of the following model. 1686 A.1. Policy based model 1688 Policies may be used to enforce EVPN BUM procedure for traffic flows 1689 with TTL value 1. Traffic flow that matches the policy is excluded 1690 from seamless interop procedure specified in this document, hence TTL 1691 decrement issue will not apply. 1693 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1695 A.2. Exercising BUM procedure for VLAN/BD 1697 Servers/hosts sending the traffic with TTL value 1 may be attached to 1698 a separate VLAN/BD, where multicast routing is disabled. When 1699 multicast routing is disabled, EVPN BUM procedure may be applied to 1700 all traffic ingressing on that VLAN/BD. On the Egress PE, the RPF 1701 for such traffic may be set to BD interface, where the source is 1702 attached. 1704 A.3. Intra-subnet bridging 1706 The procedure specified in the section enables a PE to detect an 1707 attached subnet source (i.e., source that is directly attached in the 1708 tenant BD/VLAN). By applying the following procedure for the 1709 attached source, Traffic flows having TTL value 1 can be supported. 1711 - On the ingress PE, do the bridging on the interface towards the 1712 core interface 1713 - On the egress side, make a decision whether to bridge or route 1714 at the outgoing interface (OIF) based on whether the source is 1715 attached to the OIF's BD/VLAN or not. 1717 Recent ASIC supports single lookup forwarding for bridging and 1718 routing (L2+L3). The procedure mentioned here leverages this ASIC 1719 capability. 1721 PE1 1722 +------------+ 1723 S11 +---+(BD1) | +---------+ 1724 | \ | | | 1725 |(IP-VRF)-(CORE)| | 1726 | / | | | 1727 R12 +---+(BD2) | | | 1728 +------------+ | | 1729 | | 1730 PE2 | VXLAN. | 1731 +------------+ | | 1732 R21 +---+(BD1) | | | 1733 | \ | | | 1734 |(IP-VRF)-(CORE)| | 1735 | / | | | 1736 R22+----+(BD3) | +---------+ 1737 +------------+ 1739 Figure 3 Intra-subnet bridging 1741 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1743 Consider the above picture. In the picture 1745 - PE1 and PE2 are seamless interop capable PEs 1746 - S11 is a multicast host directly attached to PE1 in BD1 1747 - Source S11 sends traffic to Group G11 1748 - R21, R22 are IGMP receivers for group G11 1749 - R21 and R22 are attached to BD1 and BD3 respectively at PE2. 1751 When source S11 starts sending the traffic, PE1 learns the source and 1752 announces the source using MVPN procedures to the remote PEs. 1754 At PE2, IGMP joins from R21, R22 result the creation of (*,G11) entry 1755 with outgoing OIF as IRB interface of BD1 and BD3. When PE2 learns 1756 the source information from PE1, it installs the route (S11, G11) at 1757 the tenant VRF with RPF as CORE interface. 1759 PE2 inherits (*, G11) OIFs to (S11, G11) entry. While inheriting 1760 OIF, PE2 checks whether source is attached to OIF's subnet. OIF 1761 matching source subnet is added with flag indicating bridge only 1762 interface. In case of (S11, G11) entry, BD1 is added as the bridge 1763 only OIF, while BD3 is added as normal OIF(L3 OIF). PEs (PE2) sends 1764 MVPN join (S11, G11) towards PE1, since it has local receivers. 1766 At Ingress PE(PE1), CORE interface is added to (S11, G11) entry as an 1767 OIF (outgoing interface) with a flag indicating that bridge only 1768 interface. With this procedure, ingress PE(PE1) bridges the traffic 1769 on CORE interface. (PE1 retains the TTL and source-MAC). The 1770 traffic is encapsulated with VNI associated with CORE interface. PE1 1771 also routes the traffic for R12 which is attached to BD2 on the same 1772 device. 1774 PE2 decapsulates the traffic from PE1 and does inner lookup on the 1775 tenant VRF associated with incoming VNI. Traffic lookup on the 1776 tenant VRF yields (S11, G11) entry as the matching entry. Traffic 1777 gets bridged on BD1 (PE2 retains the TTL and source-MAC) since the 1778 OIF is marked as bridge only interface. Traffic gets routed on BD2. 1780 Authors' Addresses 1782 Ali Sajassi 1783 Cisco 1784 170 West Tasman Drive 1785 San Jose, CA 95134, US 1787 Email: sajassi@cisco.com 1789 Internet-DrafSeamless Multicast Interoperability between EV October 2021 1791 Kesavan Thiruvenkatasamy 1792 Cisco 1793 170 West Tasman Drive 1794 San Jose, CA 95134, US 1796 Email: kethiruv@cisco.com 1798 Samir Thoria 1799 Cisco 1800 170 West Tasman Drive 1801 San Jose, CA 95134, US 1803 Email: sthoria@cisco.com 1805 Ashutosh Gupta 1806 VMware 1807 3401 Hillview Ave, Palo Alto, CA 94304 1809 Email: ashutoshgupta@vmware.com 1811 Luay Jalil 1812 Verizon 1814 Email: luay.jalil@verizon.com