idnits 2.17.1 draft-ietf-bess-evpn-optimized-ir-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** There are 2 instances of lines with control characters in the document. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 166: '...(optimized-IR hereafter) MUST meet the...' RFC 2119 keyword, line 169: '... a) The solution MUST provide an IR op...' RFC 2119 keyword, line 171: '...d unknown unicast traffic SHALL follow...' RFC 2119 keyword, line 174: '... b) The solution MUST be compatible with [RFC7432] and [EVPN-OVERLAY]...' RFC 2119 keyword, line 176: '...ar, the solution SHOULD support the fo...' (63 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: o AR-LEAF nodes SHALL send service-level BM control plane packets following regular IR procedures. An example would be IGMP, MLD or PIM multicast packets. The AR-REPLICATORs MUST not replicate these control plane packets to other overlay tunnels since they will use the regular IR-IP Address. -- The document date (February 23, 2018) is 2253 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC7432' is mentioned on line 1028, but not defined == Missing Reference: 'RFC6514' is mentioned on line 255, but not defined == Missing Reference: 'RFC2119' is mentioned on line 979, but not defined == Missing Reference: 'RFC8174' is mentioned on line 979, but not defined == Outdated reference: A later version (-14) exists of draft-ietf-bess-evpn-bum-procedure-updates-01 == Outdated reference: A later version (-12) exists of draft-ietf-bess-evpn-overlay-07 Summary: 3 errors (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Workgroup J. Rabadan, Ed. 3 Internet Draft S. Sathappan 4 Intended status: Standards Track W. Henderickx 5 Nokia 6 R. Shekhar 7 N. Sheth A. Sajassi 8 W. Lin Cisco 9 Juniper 10 A. Isaac 11 Juniper 12 M. Tufail 13 Citibank M. Katiyar 14 Versa Networks 16 Expires: August 27, 2018 February 23, 2018 18 Optimized Ingress Replication solution for EVPN 19 draft-ietf-bess-evpn-optimized-ir-03 21 Abstract 23 Network Virtualization Overlay (NVO) networks using EVPN as control 24 plane may use ingress replication (IR) or PIM-based trees to convey 25 the overlay BUM traffic. PIM provides an efficient solution to avoid 26 sending multiple copies of the same packet over the same physical 27 link, however it may not always be deployed in the NVO core network. 28 IR avoids the dependency on PIM in the NVO network core. While IR 29 provides a simple multicast transport, some NVO networks with 30 demanding multicast applications require a more efficient solution 31 without PIM in the core. This document describes a solution to 32 optimize the efficiency of IR in NVO networks. 34 Status of this Memo 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. 39 Internet-Drafts are working documents of the Internet Engineering 40 Task Force (IETF), its areas, and its working groups. Note that 41 other groups may also distribute working documents as Internet- 42 Drafts. 44 Internet-Drafts are draft documents valid for a maximum of six months 45 and may be updated, replaced, or obsoleted by other documents at any 46 time. It is inappropriate to use Internet-Drafts as reference 47 material or to cite them other than as "work in progress." 49 The list of current Internet-Drafts can be accessed at 50 http://www.ietf.org/ietf/1id-abstracts.txt 52 The list of Internet-Draft Shadow Directories can be accessed at 53 http://www.ietf.org/shadow.html 55 This Internet-Draft will expire on August 27, 2018. 57 Copyright Notice 59 Copyright (c) 2018 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (http://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the Simplified BSD License. 72 Table of Contents 74 1. Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 3 75 2. Solution requirements . . . . . . . . . . . . . . . . . . . . . 4 76 3. EVPN BGP Attributes for optimized-IR . . . . . . . . . . . . . 5 77 4. Non-selective Assisted-Replication (AR) Solution Description . 7 78 4.1. Non-selective AR-REPLICATOR procedures . . . . . . . . . . 8 79 4.2. Non-selective AR-LEAF procedures . . . . . . . . . . . . . 9 80 4.3. RNVE procedures . . . . . . . . . . . . . . . . . . . . . . 11 81 4.4. Forwarding behavior in non-selective AR EVIs . . . . . . . 11 82 4.4.1. Broadcast and Multicast forwarding behavior . . . . . . 11 83 4.4.1.1. Non-selective AR-REPLICATOR BM forwarding . . . . . 11 84 4.4.1.2. Non-selective AR-LEAF BM forwarding . . . . . . . . 12 85 4.4.1.3. RNVE BM forwarding . . . . . . . . . . . . . . . . 12 86 4.4.2. Unknown unicast forwarding behavior . . . . . . . . . . 13 87 4.4.2.1. Non-selective AR-REPLICATOR/LEAF Unknown unicast 88 forwarding . . . . . . . . . . . . . . . . . . . . 13 89 4.4.2.2. RNVE Unknown unicast forwarding . . . . . . . . . . 13 90 5. Selective Assisted-Replication (AR) Solution Description . . . 13 91 5.1. Selective AR-REPLICATOR procedures . . . . . . . . . . . . 14 92 5.2. Selective AR-LEAF procedures . . . . . . . . . . . . . . . 15 93 5.3. Forwarding behavior in selective AR EVIs . . . . . . . . . 16 94 5.3.1. Selective AR-REPLICATOR BM forwarding . . . . . . . . . 16 95 5.3.2. Selective AR-LEAF BM forwarding . . . . . . . . . . . . 17 96 6. Pruned-Flood-Lists (PFL) . . . . . . . . . . . . . . . . . . . 18 97 6.1. A PFL example . . . . . . . . . . . . . . . . . . . . . . . 18 98 7. AR Procedures for single-IP AR-REPLICATORS . . . . . . . . . . 19 99 8. AR Procedures and EVPN Multi-homing Split-Horizon . . . . . . . 20 100 9. Out-of-band distribution of Broadcast/Multicast traffic . . . . 21 101 10. Benefits of the optimized-IR solution . . . . . . . . . . . . 21 102 11. Conventions used in this document . . . . . . . . . . . . . . 21 103 12. Security Considerations . . . . . . . . . . . . . . . . . . . 21 104 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 105 14. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 22 106 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 107 15.1 Normative References . . . . . . . . . . . . . . . . . . . 23 108 15.2 Informative References . . . . . . . . . . . . . . . . . . 23 109 16. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 110 17. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 23 112 1. Problem Statement 114 EVPN may be used as the control plane for a Network Virtualization 115 Overlay (NVO) network. Network Virtualization Edge (NVE) devices and 116 PEs that are part of the same EVI use Ingress Replication (IR) or 117 PIM-based trees to transport the tenant's BUM traffic. In NVO 118 networks where PIM-based trees cannot be used, IR is the only 119 alternative. Examples of these situations are NVO networks where the 120 core nodes don't support PIM or the network operator does not want to 121 run PIM in the core. 123 In some use-cases, the amount of replication for BUM (Broadcast, 124 Unknown unicast and Multicast traffic) is kept under control on the 125 NVEs due to the following fairly common assumptions: 127 a) Broadcast is greatly reduced due to the proxy-ARP and proxy-ND 128 capabilities supported by EVPN on the NVEs. Some NVEs can even 129 provide DHCP-server functions for the attached Tenant Systems (TS) 130 reducing the broadcast even further. 132 b) Unknown unicast traffic is greatly reduced in virtualized NVO 133 networks where all the MAC and IP addresses are learnt in the 134 control plane. 136 c) Multicast applications are not used. 138 If the above assumptions are true for a given NVO network, then IR 139 provides a simple solution for multi-destination traffic. However, 140 the statement c) above is not always true and multicast applications 141 are required in many use-cases. 143 When the multicast sources are attached to NVEs residing in 144 hypervisors or low-performance-replication TORs, the ingress 145 replication of a large amount of multicast traffic to a significant 146 number of remote NVEs/PEs can seriously degrade the performance of 147 the NVE and impact the application. 149 This document describes a solution that makes use of two IR 150 optimizations: 152 i) Assisted-Replication (AR) 153 ii) Pruned-Flood-Lists (PFL) 155 Both optimizations may be used together or independently so that the 156 performance and efficiency of the network to transport multicast can 157 be improved. Both solutions require some extensions to [RFC7432] that 158 are described in section 3. 160 Section 2 lists the requirements of the combined optimized-IR 161 solution, whereas sections 4 and 5 describe the Assisted-Replication 162 (AR) solution, and section 6 the Pruned-Flood-Lists (PFL) solution. 164 2. Solution requirements 166 The IR optimization solution (optimized-IR hereafter) MUST meet the 167 following requirements: 169 a) The solution MUST provide an IR optimization for BM (Broadcast and 170 Multicast) traffic, while preserving the packet order for unicast 171 applications, i.e. known and unknown unicast traffic SHALL follow 172 the same path. 174 b) The solution MUST be compatible with [RFC7432] and [EVPN-OVERLAY] 175 and have no impact on the EVPN procedures for BM traffic. In 176 particular, the solution SHOULD support the following EVPN 177 functions: 179 o All-active multi-homing, including the split-horizon and 180 Designated Forwarder (DF) functions. 182 o Single-active multi-homing, including the DF function. 184 o Handling of multi-destination traffic and processing of 185 broadcast and multicast as per [RFC7432]. 187 c) The solution MUST be backwards compatible with existing NVEs using 188 a non-optimized version of IR. A given EVI can have NVEs/PEs 189 supporting regular-IR and optimized-IR. 191 d) The solution MUST be independent of the NVO specific data plane 192 encapsulation and the virtual identifiers being used, e.g.: VXLAN 193 VNIs, NVGRE VSIDs or MPLS labels. 195 3. EVPN BGP Attributes for optimized-IR 197 This solution proposes some changes to the [RFC7432] Inclusive 198 Multicast Ethernet Tag routes and attributes so that an NVE/PE can 199 signal its optimized-IR capabilities. 201 The Inclusive Multicast Ethernet Tag route (RT-3) and its PMSI Tunnel 202 Attribute's (PTA) general format used in [RFC7432] are shown below: 204 +---------------------------------+ 205 | RD (8 octets) | 206 +---------------------------------+ 207 | Ethernet Tag ID (4 octets) | 208 +---------------------------------+ 209 | IP Address Length (1 octet) | 210 +---------------------------------+ 211 | Originating Router's IP Addr | 212 | (4 or 16 octets) | 213 +---------------------------------+ 215 +---------------------------------+ 216 | Flags (1 octet) | 217 +---------------------------------+ 218 | Tunnel Type (1 octets) | 219 +---------------------------------+ 220 | MPLS Label (3 octets) | 221 +---------------------------------+ 222 | Tunnel Identifier (variable) | 223 +---------------------------------+ 225 The Flags field is defined as follows: 227 0 1 2 3 4 5 6 7 228 +-+-+-+-+-+--+-+-+ 229 |rsved| T |BM|U|L| 230 +-+-+-+-+-+--+-+-+ 232 Where a new type field (for AR) and two new flags (for PFL signaling) 233 are defined: 235 - T is the AR Type field (2 bits) that defines the AR role of the 236 advertising router: 238 + 00 (decimal 0) = RNVE (non-AR support) 240 + 01 (decimal 1) = AR-REPLICATOR 242 + 10 (decimal 2) = AR-LEAF 244 + 11 (decimal 3) = RESERVED 246 - The PFL (Pruned-Flood-Lists) flags defined the desired behavior of 247 the advertising router for the different types of traffic: 249 + BM= Broadcast and Multicast (BM) flag. BM=1 means "prune-me" from 250 the BM flooding list. BM=0 means regular behavior. 252 + U= Unknown flag. U=1 means "prune-me" from the Unknown flooding 253 list. U=0 means regular behavior. 255 - Flag L is an existing flag defined in [RFC6514] (L=Leaf Information 256 Required) and it will be used only in the Selective AR Solution. 258 Please refer to section 10 for the IANA considerations related to the 259 PTA flags. 261 In this document, the above RT-3 and PTA can be used in two different 262 modes for the same EVI/Ethernet Tag: 264 o Regular-IR route: in this route, Originating Router's IP Address, 265 Tunnel Type (0x06), MPLS Label, Tunnel Identifier and Flags MUST be 266 used as described in [RFC7432]. The Originating Router's IP Address 267 and Tunnel Identifier are set to an IP address that we denominate 268 IR-IP in this document. 270 o Replicator-AR route: this route is used by the AR-REPLICATOR to 271 advertise its AR capabilities, with the fields set as follows. 273 + Originating Router's IP Address as well as the Tunnel Identifier 274 are set to the same routable IP address that we denominate AR-IP 275 and SHOULD be different than the IR-IP for a given PE/NVE. 277 + Tunnel Type = Assisted-Replication (AR). Section 11 provides the 278 allocated type value. 280 + T (AR role type) = 01 (AR-REPLICATOR). 282 + L (Leaf Information Required) = 0 (for non-selective AR) or 1 283 (for selective AR). 285 In addition, this document also uses the Leaf-AD route (RT-11) 286 defined in [EVPN-BUM] in case the selective AR mode is used. The 287 Leaf-AD route MAY be used by the AR-LEAF in response to a Replicator- 288 AR route (with the L flag set) to advertise its desire to receive the 289 multicast traffic from a specific AR-REPLICATOR. It is only used for 290 selective AR and its fields are set as follows: 292 + Originating Router's IP Address is set to the advertising IR-IP 293 (same IP used by the AR-LEAF in regular-IR routes). 295 + Route Key is the "Route Type Specific" NLRI of the Replicator-AR 296 route for which this Leaf-AD route is generated. 298 + The AR-LEAF constructs an IP-address-specific route-target as 299 indicated in [EVPN-BUM], by placing the IP address carried in the 300 Next Hop field of the received Replicator-AR route in the Global 301 Administrator field of the Community, with the Local 302 Administrator field of this Community set to 0. Note that the 303 same IP-address-specific import route-target is auto-configured 304 by the AR-REPLICATOR that sent the Replicator-AR, in order to 305 control the acceptance of the Leaf-AD routes. 307 + The leaf-AD route MUST include the PMSI Tunnel attribute with the 308 Tunnel Type set to AR, type set to AR-LEAF and the Tunnel 309 Identifier set to the IR-IP of the advertising AR-LEAF. The PMSI 310 Tunnel attribute MUST carry a downstream-assigned MPLS label that 311 is used by the AR-REPLICATOR to send traffic to the AR-LEAF. 313 Each AR-enabled node MUST understand and process the AR type field in 314 the PTA (Flags field) of the routes, and MUST signal the 315 corresponding type (1 or 2) according to its administrative choice. 317 Each node, part of the EVI, MAY understand and process the BM/U 318 flags. Note that these BM/U flags may be used to optimize the 319 delivery of multi-destination traffic and its use SHOULD be an 320 administrative choice, and independent of the AR role. 322 Non-optimized-IR nodes will be unaware of the new PMSI attribute flag 323 definition as well as the new Tunnel Type (AR), i.e. they will ignore 324 the information contained in the flags field for any RT-3 and will 325 ignore the RT-3 routes with an unknown Tunnel Type (type AR in this 326 case). 328 4. Non-selective Assisted-Replication (AR) Solution Description 329 The following figure illustrates an example NVO network where the 330 non-selective AR function is enabled. Three different roles are 331 defined for a given EVI: AR-REPLICATOR, AR-LEAF and RNVE (Regular 332 NVE). The solution is called "non-selective" because the chosen AR- 333 REPLICATOR for a given flow MUST replicate the multicast traffic to 334 'all' the NVE/PEs in the EVI except for the source NVE/PE. 336 ( ) 337 (_ WAN _) 338 +---(_ _)----+ 339 | (_ _) | 340 PE1 | PE2 | 341 +------+----+ +----+------+ 342 TS1--+ (EVI-1) | | (EVI-1) +--TS2 343 |REPLICATOR | |REPLICATOR | 344 +--------+--+ +--+--------+ 345 | | 346 +--+----------------+--+ 347 | | 348 | | 349 +----+ VXLAN/nvGRE/MPLSoGRE +----+ 350 | | IP Fabric | | 351 | | | | 352 NVE1 | +-----------+----------+ | NVE3 353 Hypervisor| TOR | NVE2 |Hypervisor 354 +---------+-+ +-----+-----+ +-+---------+ 355 | (EVI-1) | | (EVI-1) | | (EVI-1) | 356 | LEAF | | RNVE | | LEAF | 357 +--+-----+--+ +--+-----+--+ +--+-----+--+ 358 | | | | | | 359 VM11 VM12 TS3 TS4 VM31 VM32 361 Figure 1 Optimized-IR scenario 363 4.1. Non-selective AR-REPLICATOR procedures 365 An AR-REPLICATOR is defined as an NVE/PE capable of replicating 366 ingress BM (Broadcast and Multicast) traffic received on an overlay 367 tunnel to other overlay tunnels and local Attachment Circuits (ACs). 368 The AR-REPLICATOR signals its role in the control plane and 369 understands where the other roles (AR-LEAF nodes, RNVEs and other AR- 370 REPLICATORs) are located. A given AR-enabled EVI service may have 371 zero, one or more AR-REPLICATORs. In our example in figure 1, PE1 and 372 PE2 are defined as AR-REPLICATORs. The following considerations apply 373 to the AR-REPLICATOR role: 375 a) The AR-REPLICATOR role SHOULD be an administrative choice in any 376 NVE/PE that is part of an AR-enabled EVI. This administrative 377 option to enable AR-REPLICATOR capabilities MAY be implemented as 378 a system level option as opposed to as a per-MAC-VRF option. 380 b) An AR-REPLICATOR MUST advertise a Replicator-AR route and MAY 381 advertise a Regular-IR route. The AR-REPLICATOR MUST NOT generate 382 a Regular-IR route if it does not have local attachment circuits 383 (AC). 385 c) The Replicator-AR and Regular-IR routes will be generated 386 according to section 3. The AR-IP and IR-IP used by the 387 Replicator-AR will be different routable IP addresses. 389 d) When a node defined as AR-REPLICATOR receives a packet on an 390 overlay tunnel, it will do a tunnel destination IP lookup and 391 apply the following procedures: 393 o If the destination IP is the AR-REPLICATOR IR-IP Address the 394 node will process the packet normally as in [RFC7432]. 396 o If the destination IP is the AR-REPLICATOR AR-IP Address the 397 node MUST replicate the packet to local ACs and overlay 398 tunnels (excluding the overlay tunnel to the source of the 399 packet). When replicating to remote AR-REPLICATORs the tunnel 400 destination IP will be an IR-IP. That will be an indication 401 for the remote AR-REPLICATOR that it MUST NOT replicate to 402 overlay tunnels. The tunnel source IP will be the AR-IP of the 403 AR-REPLICATOR. 405 4.2. Non-selective AR-LEAF procedures 407 AR-LEAF is defined as an NVE/PE that - given its poor replication 408 performance - sends all the BM traffic to an AR-REPLICATOR that can 409 replicate the traffic further on its behalf. It MAY signal its AR- 410 LEAF capability in the control plane and understands where the other 411 roles are located (AR-REPLICATOR and RNVEs). A given service can have 412 zero, one or more AR-LEAF nodes. Figure 1 shows NVE1 and NVE3 (both 413 residing in hypervisors) acting as AR-LEAF. The following 414 considerations apply to the AR-LEAF role: 416 a) The AR-LEAF role SHOULD be an administrative choice in any NVE/PE 417 that is part of an AR-enabled EVI. This administrative option to 418 enable AR-LEAF capabilities MAY be implemented as a system level 419 option as opposed to as per-MAC-VRF option. 421 b) In this non-selective AR solution, the AR-LEAF MUST advertise a 422 single Regular-IR inclusive multicast route as in [RFC7432]. The 423 AR-LEAF SHOULD set the AR Type field to AR-LEAF. Note that 424 although this flag does not make any difference for the egress 425 nodes when creating an EVPN destination to the the AR-LEAF, it is 426 RECOMMENDED the use of this flag for an easy operation and 427 troubleshooting of the EVI. 429 c) In a service where there are no AR-REPLICATORs, the AR-LEAF MUST 430 use regular ingress replication. This will happen when a new 431 update from the last former AR-REPLICATOR is received and contains 432 a non-REPLICATOR AR type, or when the AR-LEAF detects that the 433 last AR-REPLICATOR is down (next-hop tracking in the IGP or any 434 other detection mechanism). Ingress replication MUST use the 435 forwarding information given by the remote Regular-IR Inclusive 436 Multicast Routes as described in [RFC7432]. 438 d) In a service where there is one or more AR-REPLICATORs (based on 439 the received Replicator-AR routes for the EVI), the AR-LEAF can 440 locally select which AR-REPLICATOR it sends the BM traffic to: 442 o A single AR-REPLICATOR MAY be selected for all the BM packets 443 received on the AR-LEAF attachment circuits (ACs) for a given 444 EVI. This selection is a local decision and it does not have 445 to match other AR-LEAF's selection within the same EVI. 447 o An AR-LEAF MAY select more than one AR-REPLICATOR and do 448 either per-flow or per-EVI load balancing. 450 o In case of a failure on the selected AR-REPLICATOR, another 451 AR-REPLICATOR will be selected. 453 o When an AR-REPLICATOR is selected, the AR-LEAF MUST send all 454 the BM packets to that AR-REPLICATOR using the forwarding 455 information given by the Replicator-AR route for the chosen 456 AR-REPLICATOR, with tunnel type = 0x0A (AR tunnel). The 457 underlay destination IP address MUST be the AR-IP advertised 458 by the AR-REPLICATOR in the Replicator-AR route. 460 o AR-LEAF nodes SHALL send service-level BM control plane 461 packets following regular IR procedures. An example would be 462 IGMP, MLD or PIM multicast packets. The AR-REPLICATORs MUST 463 not replicate these control plane packets to other overlay 464 tunnels since they will use the regular IR-IP Address. 466 e) The use of an AR-REPLICATOR-activation-timer (in seconds) on the 467 AR-LEAF nodes is RECOMMENDED. Upon receiving a new Replicator-AR 468 route where the AR-REPLICATOR is selected, the AR-LEAF will run a 469 timer before programming the new AR-REPLICATOR. This will give the 470 AR-REPLICATOR some time to program the AR-LEAF nodes before the 471 AR-LEAF sends BM traffic. 473 4.3. RNVE procedures 475 RNVE (Regular Network Virtualization Edge node) is defined as an 476 NVE/PE without AR-REPLICATOR or AR-LEAF capabilities that does IR as 477 described in [RFC7432]. The RNVE does not signal any AR role and is 478 unaware of the AR-REPLICATOR/LEAF roles in the EVI. The RNVE will 479 ignore the Flags in the Regular-IR routes and will ignore the 480 Replicator-AR routes (due to an unknown tunnel type in the PTA) and 481 the Leaf-AD routes (due to the IP-address-specific route-target). 483 This role provides EVPN with the backwards compatibility required in 484 optimized-IR EVIs. Figure 1 shows NVE2 as RNVE. 486 4.4. Forwarding behavior in non-selective AR EVIs 488 In AR EVIs, BM (Broadcast and Multicast) traffic between two NVEs may 489 follow a different path than unicast traffic. This solution proposes 490 the replication of BM through the AR-REPLICATOR node, whereas 491 unknown/known unicast will be delivered directly from the source node 492 to the destination node without being replicated by any intermediate 493 node. Unknown unicast SHALL follow the same path as known unicast 494 traffic in order to avoid packet reordering for unicast applications 495 and simplify the control and data plane procedures. Section 4.4.1. 496 describes the expected forwarding behavior for BM traffic in nodes 497 acting as AR-REPLICATOR, AR-LEAF and RNVE. Section 4.4.2. describes 498 the forwarding behavior for unknown unicast traffic. 500 Note that known unicast forwarding is not impacted by this solution. 502 4.4.1. Broadcast and Multicast forwarding behavior 504 The expected behavior per role is described in this section. 506 4.4.1.1. Non-selective AR-REPLICATOR BM forwarding 508 The AR-REPLICATORs will build a flooding list composed of ACs and 509 overlay tunnels to remote nodes in the EVI. Some of those overlay 510 tunnels MAY be flagged as non-BM receivers based on the BM flag 511 received from the remote nodes in the EVI. 513 o When an AR-REPLICATOR receives a BM packet on an AC, it will 514 forward the BM packet to its flooding list (including local ACs and 515 remote NVE/PEs), skipping the non-BM overlay tunnels. 517 o When an AR-REPLICATOR receives a BM packet on an overlay tunnel, it 518 will check the destination IP of the underlay IP header and: 520 - If the destination IP matches its AR-IP, the AR-REPLICATOR will 521 forward the BM packet to its flooding list (ACs and overlay 522 tunnels) excluding the non-BM overlay tunnels. The AR-REPLICATOR 523 will do source squelching to ensure the traffic is not sent back 524 to the originating AR-LEAF. If the encapsulation is MPLSoGRE (or 525 MPLSoUDP) and the EVI label is not the bottom of the stack, the 526 AR-REPLICATOR MUST copy the rest of the labels and forward them 527 to the egress overlay tunnels. 529 - If the destination IP matches its IR-IP, the AR-REPLICATOR will 530 skip all the overlay tunnels from the flooding list, i.e. it 531 will only replicate to local ACs. This is the regular IR 532 behavior described in [RFC7432]. 534 4.4.1.2. Non-selective AR-LEAF BM forwarding 536 The AR-LEAF nodes will build two flood-lists: 538 1) Flood-list #1 - composed of ACs and an AR-REPLICATOR-set of 539 overlay tunnels. The AR-REPLICATOR-set is defined as one or more 540 overlay tunnels to the AR-IP Addresses of the remote AR- 541 REPLICATOR(s) in the EVI. The selection of more than one AR- 542 REPLICATOR is described in section 4.2. and it is a local AR- 543 LEAF decision. 545 2) Flood-list #2 - composed of ACs and overlay tunnels to the 546 remote IR-IP Addresses. 548 When an AR-LEAF receives a BM packet on an AC, it will check the 549 AR-REPLICATOR-set: 551 o If the AR-REPLICATOR-set is empty, the AR-LEAF will send the packet 552 to flood-list #2. 554 o If the AR-REPLICATOR-set is NOT empty, the AR-LEAF will send the 555 packet to flood-list #1, where only one of the overlay tunnels of 556 the AR-REPLICATOR-set is used. 558 When an AR-LEAF receives a BM packet on an overlay tunnel, will 559 forward the BM packet to its local ACs and never to an overlay 560 tunnel. This is the regular IR behavior described in [RFC7432]. 562 4.4.1.3. RNVE BM forwarding 564 The RNVE is completely unaware of the AR-REPLICATORs, AR-LEAF nodes 565 and BM/U flags (that information is ignored). Its forwarding behavior 566 is the regular IR behavior described in [RFC7432]. Any regular non-AR 567 node is fully compatible with the RNVE role described in this 568 document. 570 4.4.2. Unknown unicast forwarding behavior 572 The expected behavior is described in this section. 574 4.4.2.1. Non-selective AR-REPLICATOR/LEAF Unknown unicast forwarding 576 While the forwarding behavior in AR-REPLICATORs and AR-LEAF nodes is 577 different for BM traffic, as far as Unknown unicast traffic 578 forwarding is concerned, AR-LEAF nodes behave exactly in the same way 579 as AR-REPLICATORs do. 581 The AR-REPLICATOR/LEAF nodes will build a flood-list composed of ACs 582 and overlay tunnels to the IR-IP Addresses of the remote nodes in the 583 EVI. Some of those overlay tunnels MAY be flagged as non-U (Unknown 584 unicast) receivers based on the U flag received from the remote nodes 585 in the EVI. 587 o When an AR-REPLICATOR/LEAF receives an unknown packet on an AC, it 588 will forward the unknown packet to its flood-list, skipping the 589 non-U overlay tunnels. 591 o When an AR-REPLICATOR/LEAF receives an unknown packet on an overlay 592 tunnel will forward the unknown packet to its local ACs and never 593 to an overlay tunnel. This is the regular IR behavior described in 594 [RFC7432]. 596 4.4.2.2. RNVE Unknown unicast forwarding 598 As described for BM traffic, the RNVE is completely unaware of the 599 REPLICATORs, LEAF nodes and BM/U flags (that information is ignored). 600 Its forwarding behavior is the regular IR behavior described in 601 [RFC7432], also for Unknown unicast traffic. Any regular non-AR node 602 is fully compatible with the RNVE role described in this document. 604 5. Selective Assisted-Replication (AR) Solution Description 606 Figure 1 is also used to describe the selective AR solution, however 607 in this section we consider NVE2 as one more AR-LEAF for EVI-1. The 608 solution is called "selective" because a given AR-REPLICATOR MUST 609 replicate the BM traffic to only the AR-LEAF that requested the 610 replication (as opposed to all the AR-LEAF nodes) and MAY replicate 611 the BM traffic to the RNVEs. The same AR roles defined in section 4 612 are used here, however the procedures are slightly different. 614 The following sub-sections describe the differences in the procedures 615 of AR-REPLICATOR/LEAFs compared to the non-selective AR solution. 616 There is no change on the RNVEs. 618 5.1. Selective AR-REPLICATOR procedures 620 In our example in figure 1, PE1 and PE2 are defined as Selective AR- 621 REPLICATORs. The following considerations apply to the Selective AR- 622 REPLICATOR role: 624 a) The Selective AR-REPLICATOR capability SHOULD be an administrative 625 choice in any NVE/PE that is part of an AR-enabled EVI, as the AR 626 role itself. This administrative option MAY be implemented as a 627 system level option as opposed to as a per-MAC-VRF option. 629 b) Each AR-REPLICATOR will build a list of AR-REPLICATOR, AR-LEAF and 630 RNVE nodes (AR-LEAF nodes that sent only a regular-IR route are 631 accounted as RNVEs by the AR-REPLICATOR). In spite of the 632 'Selective' administrative option, an AR-REPLICATOR MUST NOT 633 behave as a Selective AR-REPLICATOR if at least one of the AR- 634 REPLICATORs has the L flag NOT set. If at least one AR-REPLICATOR 635 sends a Replicator-AR route with L=0 (in the EVI context), the 636 rest of the AR-REPLICATORs will fall back to non-selective AR 637 mode. 639 b) The Selective AR-REPLICATOR MUST follow the procedures described 640 in section 4.1, except for the following differences: 642 o The Replicator-AR route MUST include L=1 (Leaf Information 643 Required) in the Replicator-AR route. This flag is used by the 644 AR-REPLICATORs to advertise their 'selective' AR-REPLICATOR 645 capabilities. In addition, the AR-REPLICATOR auto-configures 646 its IP-address-specific import route-target as described in 647 section 3. 649 o The AR-REPLICATOR will build a 'selective' AR-LEAF-set with 650 the list of nodes that requested replication to its own AR-IP. 651 For instance, assuming NVE1 and NVE2 advertise a Leaf-AD route 652 with PE1's IP-address-specific route-target and NVE3 653 advertises a Leaf-AD route with PE2's IP-address-specific 654 route-target, PE1 MUST only add NVE1/NVE2 to its selective AR- 655 LEAF-set for EVI-1, and exclude NVE3. 657 o When a node defined and operating as Selective AR-REPLICATOR 658 receives a packet on an overlay tunnel, it will do a tunnel 659 destination IP lookup and if the destination IP is the AR- 660 REPLICATOR AR-IP Address, the node MUST replicate the packet 661 to: 663 + local ACs 664 + overlay tunnels in the Selective AR-LEAF-set (excluding the 665 overlay tunnel to the source AR-LEAF). 666 + overlay tunnels to the RNVEs if the tunnel source IP is the 667 IR-IP of an AR-LEAF (in any other case, the AR-REPLICATOR 668 MUST NOT replicate the BM traffic to remote RNVEs). In other 669 words, the first-hop selective AR-REPLICATOR will replicate 670 to all the RNVEs. 671 + overlay tunnels to the remote Selective AR-REPLICATORs if 672 the tunnel source IP is an IR-IP of its own AR-LEAF-set (in 673 any other case, the AR-REPLICATOR MUST NOT replicate the BM 674 traffic to remote AR-REPLICATORs), where the tunnel 675 destination IP is the AR-IP of the remote Selective AR- 676 REPLICATOR. The tunnel destination IP AR-IP will be an 677 indication for the remote Selective AR-REPLICATOR that the 678 packet needs further replication to its AR-LEAFs. 680 5.2. Selective AR-LEAF procedures 682 A Selective AR-LEAF chooses a single Selective AR-REPLICATOR per EVI 683 and: 685 o Sends all the EVI BM traffic to that AR-REPLICATOR and 686 o Expects to receive the BM traffic for a given EVI from the same AR- 687 REPLICATOR. 689 In the example of Figure 1, we consider NVE1/NVE2/NVE3 as Selective 690 AR-LEAFs. NVE1 selects PE1 as its Selective AR-REPLICATOR. If that is 691 so, NVE1 will send all its BM traffic for EVI-1 to PE1. If other AR- 692 LEAF/REPLICATORs send BM traffic, NVE1 will receive that traffic from 693 PE1. These are the differences in the behavior of a Selective AR-LEAF 694 compared to a non-selective AR-LEAF: 696 a) The AR-LEAF role selective capability SHOULD be an administrative 697 choice in any NVE/PE that is part of an AR-enabled EVI. This 698 administrative option to enable AR-LEAF capabilities MAY be 699 implemented as a system level option as opposed to as per-MAC-VRF 700 option. 702 b) The AR-LEAF MAY advertise a Regular-IR route if there are RNVEs in 703 the EVI. The Selective AR-LEAF MUST advertise a Leaf-AD route 704 after receiving a Replicator-AR route with L=1. It is recommended 705 that the Selective AR-LEAF waits for a timer t before sending the 706 Leaf-AD route, so that the AR-LEAF receives all the Replicator-AR 707 routes for the EVI. 709 c) In a service where there is more than one Selective AR-REPLICATORs 710 the Selective AR-LEAF MUST locally select a single Selective AR- 711 REPLICATOR for the EVI. Once selected: 713 o The Selective AR-LEAF will send a Leaf-AD route including the 714 Route-key and IP-address-specific route-target of the selected 715 AR-REPLICATOR. 717 o The Selective AR-LEAF will send all the BM packets received on 718 the attachment circuits (ACs) for a given EVI to that AR- 719 REPLICATOR. 721 o In case of a failure on the selected AR-REPLICATOR, another 722 AR-REPLICATOR will be selected and a new Leaf-AD update will 723 be issued for the new AR-REPLICATOR. This new route will 724 update the selective list in the new Selective AR-REPLICATOR. 725 In case of failure on the active Selective AR-REPLICATOR, it 726 is recommended for the Selective AR-LEAF to revert to IR 727 behavior for a timer t to speed up the convergence. When the 728 timer expires, the Selective AR-LEAF will resume its AR mode 729 with the new Selective AR-REPLICATOR. 731 All the AR-LEAFs in an EVI are expected to be configured as either 732 selective or non-selective. A mix of selective and non-selective AR- 733 LEAFs SHOULD NOT coexist in the same EVI. In case there is a non- 734 selective AR-LEAF, its BM traffic sent to a selective AR-REPLICATOR 735 will not be replicated to other AR-LEAFs that are not in its 736 Selective AR-LEAF-set. 738 5.3. Forwarding behavior in selective AR EVIs 740 This section describes the differences of the selective AR forwarding 741 mode compared to the non-selective mode. Compared to section 4.4, 742 there are no changes for the forwarding behavior in RNVEs or for 743 unknown unicast traffic. 745 5.3.1. Selective AR-REPLICATOR BM forwarding 747 The Selective AR-REPLICATORs will build two flood-lists: 749 1) Flood-list #1 - composed of ACs and overlay tunnels to the 750 remote nodes in the EVI, always using the IR-IPs in the tunnel 751 destination IP addresses. Some of those overlay tunnels MAY be 752 flagged as non-BM receivers based on the BM flag received from 753 the remote nodes in the EVI. 755 2) Flood-list #2 - composed of ACs, a Selective AR-LEAF-set and a 756 Selective AR-REPLICATOR-set, where: 758 o The Selective AR-LEAF-set is composed of the overlay tunnels 759 to the AR-LEAFs that advertise a Leaf-AD route for the local 760 AR-REPLICATOR. This set is updated with every Leaf-AD route 761 received/withdrawn from a new AR-LEAF. 763 o The Selective AR-REPLICATOR-set is composed of the overlay 764 tunnels to all the AR-REPLICATORs that send a Replicator-AR 765 route with L=1. The AR-IP addresses are used as tunnel 766 destination IP. 768 When a Selective AR-REPLICATOR receives a BM packet on an AC, it will 769 forward the BM packet to its flood-list #1, skipping the non-BM 770 overlay tunnels. 772 When a Selective AR-REPLICATOR receives a BM packet on an overlay 773 tunnel, it will check the destination and source IPs of the underlay 774 IP header and: 776 - If the destination IP matches its AR-IP and the source IP 777 matches an IP of its own Selective AR-LEAF-set, the AR- 778 REPLICATOR will forward the BM packet to its flood-list #2, as 779 long as the list of AR-REPLICATORs for the EVI matches the 780 Selective AR-REPLICATOR-set. If the Selective AR-REPLICATOR-set 781 does not match the list of AR-REPLICATORs, the node reverts back 782 to non-selective mode and flood-list #1 is used. 784 - If the destination IP matches its AR-IP and the source IP does 785 not match any IP of its Selective AR-LEAF-set, the AR-REPLICATOR 786 will forward the BM packet to flood-list #2 but skipping the AR- 787 REPLICATOR-set. 789 - If the destination IP matches its IR-IP, the AR-REPLICATOR will 790 use flood-list #1 but MUST skip all the overlay tunnels from the 791 flooding list, i.e. it will only replicate to local ACs. This is 792 the regular-IR behavior described in [RFC7432]. 794 In any case, non-BM overlay tunnels are excluded from flood-lists 795 and, also, source squelching is always done in order to ensure the 796 traffic is not sent back to the originating source. If the 797 encapsulation is MPLSoGRE (or MPLSoUDP) and the EVI label is not the 798 bottom of the stack, the AR-REPLICATOR MUST copy the rest of the 799 labels when forwarding them to the egress overlay tunnels. 801 5.3.2. Selective AR-LEAF BM forwarding 802 The Selective AR-LEAF nodes will build two flood-lists: 804 1) Flood-list #1 - composed of ACs and the overlay tunnel to the 805 selected AR-REPLICATOR (using the AR-IP as the tunnel 806 destination IP). 808 2) Flood-list #2 - composed of ACs and overlay tunnels to the 809 remote IR-IP Addresses. 811 When an AR-LEAF receives a BM packet on an AC, it will check if there 812 is any selected AR-REPLICATOR. If there is, flood-list #1 will be 813 used. Otherwise, flood-list #2 will. 815 When an AR-LEAF receives a BM packet on an overlay tunnel, will 816 forward the BM packet to its local ACs and never to an overlay 817 tunnel. This is the regular IR behavior described in [RFC7432]. 819 6. Pruned-Flood-Lists (PFL) 821 In addition to AR, the second optimization supported by this solution 822 is the ability for the all the EVI nodes to signal Pruned-Flood-Lists 823 (PFL). As described in section 3, an EVPN node can signal a given 824 value for the BM and U PFL flags in the IR Inclusive Multicast 825 Routes, where: 827 + BM= Broadcast and Multicast (BM) flag. BM=1 means "prune-me" from 828 the BM flood-list. BM=0 means regular behavior. 830 + U= Unknown flag. U=1 means "prune-me" from the Unknown flood-list. 831 U=0 means regular behavior. 833 The ability to signal these PFL flags is an administrative choice. 834 Upon receiving a non-zero PFL flag, a node MAY decide to honor the 835 PFL flag and remove the sender from the corresponding flood-list. A 836 given EVI node receiving BUM traffic on an overlay tunnel MUST 837 replicate the traffic normally, regardless of the signaled PFL 838 flags. 840 This optimization MAY be used along with the AR solution. 842 6.1. A PFL example 844 In order to illustrate the use of the solution described in this 845 document, we will assume that EVI-1 in figure 1 is optimized-IR 846 enabled and: 848 o PE1 and PE2 are administratively configured as AR-REPLICATORs, due 849 to their high-performance replication capabilities. PE1 and PE2 850 will send a Replicator-AR route with BM/U flags = 00. 852 o NVE1 and NVE3 are administratively configured as AR-LEAF nodes, due 853 to their low-performance software-based replication capabilities. 854 They will advertise a Regular-IR route with type AR-LEAF. Assuming 855 both NVEs advertise all the attached VMs in EVPN as soon as they 856 come up and don't have any VMs interested in multicast 857 applications, they will be configured to signal BM/U flags = 11 for 858 EVI-1. 860 o NVE2 is optimized-IR unaware; therefore it takes on the RNVE role 861 in EVI-1. 863 Based on the above assumptions the following forwarding behavior will 864 take place: 866 (1) Any BM packets sent from VM11 will be sent to VM12 and PE1. PE1 867 will forward further the BM packets to TS1, WAN link, PE2 and 868 NVE2, but not to NVE3. PE2 and NVE2 will replicate the BM packets 869 to their local ACs but we will avoid NVE3 having to replicate 870 unnecessarily those BM packets to VM31 and VM32. 872 (2) Any BM packets received on PE2 from the WAN will be sent to PE1 873 and NVE2, but not to NVE1 and NVE3, sparing the two hypervisors 874 from replicating unnecessarily to their local VMs. PE1 and NVE2 875 will replicate to their local ACs only. 877 (3) Any Unknown unicast packet sent from VM31 will be forwarded by 878 NVE3 to NVE2, PE1 and PE2 but not NVE1. The solution avoids the 879 unnecessary replication to NVE1, since the destination of the 880 unknown traffic cannot be at NVE1. 882 (4) Any Unknown unicast packet sent from TS1 will be forwarded by PE1 883 to the WAN link, PE2 and NVE2 but not to NVE1 and NVE3, since the 884 target of the unknown traffic cannot be at those NVEs. 886 7. AR Procedures for single-IP AR-REPLICATORS 888 The procedures explained in sections 4 (Non-selective AR) and 5 889 (Selective AR) assume that the AR-REPLICATOR can use two local 890 routable IP addresses to terminate and originate NVO tunnels, i.e. 891 IR-IP and AR-IP addresses. This is usually the case for PE-based AR- 892 REPLICATOR nodes. 894 In some cases, the AR-REPLICATOR node does not support more than one 895 IP address to terminate and originate NVO tunnels, i.e. the IR-IP and 896 AR-IP are the same IP addresses. This may be the case in some 897 software-based or low-end AR-REPLICATOR nodes. If this is the case, 898 the procedures in sections 4 and 5 must be modified in the following 899 way: 901 o The Replicator-AR routes generated by the AR-REPLICATOR use an AR- 902 IP that will match its IR-IP. In order to differentiate the data 903 plane packets that need to use IR from the packets that must use AR 904 forwarding mode, the Replicator-AR route must advertise a different 905 VNI/VSID than the one used by the Regular-IR route. For instance, 906 the AR-REPLICATOR will advertise AR-VNI along with the Replicator- 907 AR route and IR-VNI along with the Regular-IR route. Since both 908 routes have the same key, different RDs are needed for both routes. 910 o An AR-REPLICATOR will perform IR or AR forwarding mode for the 911 incoming Overlay packets based on an ingress VNI lookup, as opposed 912 to the tunnel IP DA lookup described in sections 4 and 5. Note 913 that, when replicating to remote AR-REPLICATOR nodes, the use of 914 the IR-VNI or AR-VNI advertised by the egress node will determine 915 the IR or AR forwarding mode at the subsequent AR-REPLICATOR. 917 The rest of the procedures will follow what is described in sections 918 4 and 5. 920 8. AR Procedures and EVPN Multi-homing Split-Horizon 922 If VXLAN or NVGRE are used, and if the Split-horizon is based on the 923 tunnel IP SA and "Local-Bias" as described in [EVPN-OVERLAY], the 924 Split-horizon check will not work if there is an Ethernet-Segment 925 shared between two AR-LEAF nodes, and the AR-REPLICATOR changes the 926 tunnel IP SA of the packets with its own AR-IP. 928 In order to be compatible with the IP SA split-horizon check, the AR- 929 REPLICATOR MAY keep the original received tunnel IP SA when 930 replicating packets to a remote AR-LEAF or AR-REPLICATOR. This will 931 allow DF (Designated Forwarder) AR-LEAF nodes to apply Split-horizon 932 check procedures for BM packets, before sending them to the local 933 Ethernet-Segment. 935 When EVPN is used for MPLS over GRE (or UDP), the ESI-label based 936 split-horizon procedure as in [RFC7432] will not work for multi-homed 937 Ethernet-Segments defined on AR-LEAF nodes. "Local-Bias" is 938 recommended in this case, as in the case of VXLAN or NVGRE explained 939 above. The "Local-Bias" and tunnel IP SA preservation mechanisms 940 provide the required split-horizon behavior in non-selective or 941 selective AR. 943 Note that if the AR-REPLICATOR implementation keeps the received 944 tunnel IP SA, the use of uRPF in the IP fabric based on the tunnel IP 945 SA MUST be disabled. 947 9. Out-of-band distribution of Broadcast/Multicast traffic 949 The use of out-of-band mechanisms to distribute BM traffic between 950 AR-REPLICATORS MAY be used. 952 10. Benefits of the optimized-IR solution 954 A solution for the optimization of Ingress Replication in EVPN is 955 described in this document (optimized-IR). The solution brings the 956 following benefits: 958 o Optimizes the multicast forwarding in low-performance NVEs, by 959 relaying the replication to high-performance NVEs (AR-REPLICATORs) 960 and while preserving the packet ordering for unicast applications. 962 o Reduces the flooded traffic in NVO networks where some NVEs do not 963 need broadcast/multicast and/or unknown unicast traffic. 965 o It is fully compatible with existing EVPN implementations and EVPN 966 functions for NVO overlay tunnels. Optimized-IR NVEs and regular 967 NVEs can be even part of the same EVI. 969 o It does not require any PIM-based tree in the NVO core of the 970 network. 972 11. Conventions used in this document 974 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 975 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 976 "OPTIONAL" in this document are to be interpreted as described in BCP 977 14 [RFC2119] [RFC8174] when, and only when, they appear in all 978 capitals, as shown here. 980 12. Security Considerations 982 This section will be added in future versions. 984 13. IANA Considerations 986 IANA has allocated the following Border Gateway Protocol (BGP) 987 Parameters: 989 1) Allocation in the P-Multicast Service Interface Tunnel (PMSI 990 Tunnel) Tunnel Types registry: 992 Value Meaning Reference 993 0x0A Assisted-Replication Tunnel [This document] 995 2) Allocations in the P-Multicast Service Interface (PMSI) Tunnel 996 Attribute Flags registry: 998 Value Name Reference 999 3-4 Assisted-Replication Type (T) [This document] 1000 5 Broadcast and Multicast (BM) [This document] 1001 6 Unknown (U) [This document] 1003 14. Terminology 1005 Regular-IR: Refers to Regular Ingress Replication, where the source 1006 NVE/PE sends a copy to each remote NVE/PE part of the EVI. 1008 AR-IP: IP address owned by the AR-REPLICATOR and used to 1009 differentiate the ingress traffic that must follow the AR 1010 procedures. 1012 IR-IP: IP address used for Ingress Replication as in [RFC7432]. 1014 AR-VNI: VNI advertised by the AR-REPLICATOR along with the 1015 Replicator-AR route. It is used to identify the ingress 1016 packets that must follow AR procedures ONLY in the Single-IP 1017 AR-REPLICATOR case. 1019 IR-VNI: VNI advertised along with the RT-3 for IR. 1021 AR forwarding mode: for an AR-LEAF, it means sending an AC BM packet 1022 to a single AR-REPLICATOR with tunnel destination IP AR-IP. 1023 For an AR-REPLICATOR, it means sending a BM packet to a 1024 selective number or all the overlay tunnels when the packet 1025 was previously received from an overlay tunnel. 1027 IR forwarding mode: it refers to the Ingress Replication behavior 1028 explained in [RFC7432]. It means sending an AC BM packet copy 1029 to each remote PE/NVE in the EVI and sending an overlay BM 1030 packet only to the ACs and not other overlay tunnels. 1032 PTA: PMSI Tunnel Attribute 1034 RT-3: EVPN Route Type 3, Inclusive Multicast Ethernet Tag route 1035 RT-11: EVPN Route Type 11, Leaf Auto-Discovery (AD) route 1037 15. References 1039 15.1 Normative References 1041 [RFC6514]Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP 1042 Encodings and Procedures for Multicast in MPLS/BGP IP VPNs", 1043 RFC 6514, DOI 10.17487/RFC6514, February 2012, . 1046 [RFC7432]Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 1047 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet 1048 VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, 1049 . 1051 [RFC7902]Rosen, E. and T. Morin, "Registry and Extensions for P- 1052 Multicast Service Interface Tunnel Attribute Flags", RFC 7902, DOI 1053 10.17487/RFC7902, June 2016, . 1056 [RFC7902]Rosen, E. and Morin, T., "Registry and Extensions for P- 1057 Multicast Service Interface Tunnel Attribute Flags", June 2016, 1058 . 1060 [EVPN-BUM] Zhang et al., "Updates on EVPN BUM Procedures", draft- 1061 ietf-bess-evpn-bum-procedure-updates-01.txt, work in progress, 1062 December 2016. 1064 15.2 Informative References 1066 [EVPN-OVERLAY] Sajassi-Drake et al., "A Network Virtualization 1067 Overlay Solution using EVPN", draft-ietf-bess-evpn-overlay-07.txt, 1068 work in progress, December 2016. 1070 16. Acknowledgments 1072 The authors would like to thank Neil Hart, David Motz, Kiran Nagaraj, 1073 Dai Truong, Thomas Morin, Jeffrey Zhang and Shankar Murthy for their 1074 valuable feedback and contributions. 1076 17. Authors' Addresses 1078 Jorge Rabadan (Editor) 1079 Nokia 1080 777 E. Middlefield Road 1081 Mountain View, CA 94043 USA 1082 Email: jorge.rabadan@nokia.com 1084 Senthil Sathappan 1085 Nokia 1086 Email: senthil.sathappan@nokia.com 1088 Mukul Katiyar 1089 Versa Networks 1090 Email: mukul@versa-networks.com 1092 Wim Henderickx 1093 Nokia 1094 Email: wim.henderickx@nokia.com 1096 Ravi Shekhar 1097 Juniper Networks 1098 Email: rshekhar@juniper.net 1100 Nischal Sheth 1101 Juniper Networks 1102 Email: nsheth@juniper.net 1104 Wen Lin 1105 Juniper Networks 1106 Email: wlin@juniper.net 1108 Ali Sajassi 1109 Cisco 1110 Email: sajassi@cisco.com 1112 Aldrin Isaac 1113 Juniper 1114 Email: aisaac@juniper.net 1116 Mudassir Tufail 1117 Citibank 1118 mudassir.tufail@citi.com