idnits 2.17.1 draft-nr-bess-evpn-mh-split-horizon-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 9, 2020) is 1318 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-06) exists of draft-ietf-bess-evpn-geneve-01 -- Obsolete informational reference (is this intentional?): RFC 5512 (Obsoleted by RFC 9012) == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-17 == Outdated reference: A later version (-15) exists of draft-ietf-bess-srv6-services-04 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Workgroup J. Rabadan, Ed. 3 Internet-Draft K. Nagaraj 4 Intended status: Standards Track Nokia 5 Expires: March 13, 2021 W. Lin 6 Juniper 7 A. Sajassi 8 Cisco 9 September 9, 2020 11 EVPN Multi-Homing Extensions for Split Horizon Filtering 12 draft-nr-bess-evpn-mh-split-horizon-03 14 Abstract 16 Ethernet Virtual Private Network (EVPN) is commonly used along with 17 Network Virtualization Overlay (NVO) tunnels. The EVPN multi-homing 18 procedures may be different depending on the NVO tunnel type used in 19 the EVPN Broadcast Domain. In particular, there are two multi-homing 20 Split Horizon procedures to avoid looped frames on the multi-homed 21 CE: ESI Label based and Local Bias. ESI Label based Split Horizon is 22 used for MPLSoX tunnels, E.g., MPLSoUDP, whereas Local Bias is used 23 for others, E.g., VXLAN tunnels. The current specifications do not 24 allow the operator to decide which Split Horizon procedure to use for 25 tunnel encapsulations that could support both. Examples of tunnels 26 that may support both procedures are MPLSoGRE, MPLSoUDP, GENEVE or 27 SRv6. This document extends the EVPN Multi-Homing procedures so that 28 an operator can decide the Split Horizon procedure for a given NVO 29 tunnel depending on their own requirements. 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at https://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on March 13, 2021. 48 Copyright Notice 50 Copyright (c) 2020 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (https://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 66 1.1. Conventions and Terminology . . . . . . . . . . . . . . . 5 67 2. BGP EVPN Extensions . . . . . . . . . . . . . . . . . . . . . 7 68 2.1. The Split Horizon Type (SHT) . . . . . . . . . . . . . . 7 69 2.2. Use of the Split Horizon Type In A-D Per ES Routes . . . 8 70 2.3. ESI Label Value In A-D Per ES Routes . . . . . . . . . . 9 71 2.4. Backwards Compatibility With [RFC8365] NVEs . . . . . . . 10 72 3. Procedures for NVEs Supporting Multiple Encapsulations . . . 10 73 4. Security Considerations . . . . . . . . . . . . . . . . . . . 12 74 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 75 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 76 6.1. Normative References . . . . . . . . . . . . . . . . . . 13 77 6.2. Informative References . . . . . . . . . . . . . . . . . 13 78 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 14 79 Appendix B. Contributors . . . . . . . . . . . . . . . . . . . . 15 80 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 82 1. Introduction 84 Ethernet Virtual Private Network (EVPN) is commonly used along with 85 Network Virtualization Overlay (NVO) tunnels and specified in 86 [RFC8365]. The EVPN multi-homing procedures may be different 87 depending on the NVO tunnel type used in the EVPN Broadcast Domain. 88 In particular, there are two Multi-Homing Split Horizon procedures to 89 avoid looped frames on the multi-homed CE: ESI Label based and Local 90 Bias. ESI Label based Split Horizon is used for MPLSoX tunnels, 91 E.g., MPLSoUDP [RFC7510], and its procedures described in [RFC7432]. 92 Local Bias is used by non-MPLS NVO tunnels, E.g., VXLAN tunnels, and 93 it is described in [RFC8365]. 95 As a refresher: 97 o ESI Label based Split-Horizon filtering [RFC7432] 99 When EVPN is used for MPLS transport tunnels, an MPLS label 100 enables the Split Horizon filtering capability to support All- 101 Active multi-homing. The ingress NVE adds a label corresponding 102 to the source Ethernet Segment (aka an ESI label) when 103 encapsulating the packet. The egress NVE checks the ESI label 104 when attempting to forward a multi-destination frame out of a 105 local ES interface, and if the label corresponds to the same site 106 identifier (ESI) associated with that ES interface, the packet is 107 not forwarded. This prevents the occurrence of forwarding loops 108 for BUM traffic. 110 The ESI Label Split Horizon filtering SHOULD also be used with 111 Single-Active multi-homing to avoid transient loops for in-flight 112 packets when the egress NVE takes over as DF for an Ethernet 113 Segment. 115 o Local Bias for non-MPLS NVO tunnels [RFC8365] 117 Since non-MPLS NVO tunnels (such as VXLAN or NVGRE) do not support 118 the ESI label (or any MPLS label at all), a different Split 119 Horizon filtering procedure must be used for All-Active multi- 120 homing. This mechanism is called Local Bias and relies on the NVO 121 tunnel source IP address to decide whether to forward BUM traffic 122 to a local ES interface at the egress NVE. 124 In a nutshell, every NVE tracks the IP address(es) associated with 125 the other NVE(s) with which it has shared multi-homed ESs. When 126 the egress NVE receives a BUM frame encapsulated in a non-MPLS NVO 127 packet, it examines the source IP address in the tunnel header 128 (which identifies the ingress NVE) and filters out the frame on 129 all local interfaces connected to ESes that are shared with the 130 ingress NVE. 132 Due to this behavior at the egress NVE, the ingress NVE's behavior 133 is also changed to perform replication locally to all directly 134 attached Ethernet Segments (regardless of the DF election state) 135 for all BUM ingress from the access ACs. Because of this "local" 136 replication at the ingress NVE, this approach is referred to as 137 Local Bias. 139 Local Bias cannot be used for Single-Active multi-homing, since 140 the ingress NVE brings operationally down the ACs for which it is 141 non-DF (hence local replication to non-DF ACs cannot be done). 142 This means transient in-flight BUM packets may be looped back to 143 the originating site by new elected DF egress NVEs. 145 [RFC8365] states that Local Bias is used only for non-MPLS NVO 146 tunnels, and ESI Label based Split Horizon for MPLS NVO tunnels. 147 However, MPLS NVO tunnels, such as MPLSoGRE or MPLSoUDP, can 148 potentially support both procedures, since they can carry ESI Labels 149 and they also use a tunnel IP header where the source IP address 150 identifies the ingress NVE. 152 Similarly, some non-MPLS NVO tunnels that carry an identifier of the 153 source ES in the tunnel header, may potentially follow either 154 procedure too. Some examples are GENEVE or SRv6: 156 o In a GENEVE tunnel the source IP address identifies the ingress 157 NVE therefore local bias is possible. Also, 158 [I-D.ietf-bess-evpn-geneve] defines an Ethernet option TLV (Type 159 Length Value) to encode an ESI label value. 161 o In an SRv6 tunnel, the source IP address also identifies the 162 ingress NVE, however, by default and as described in 163 [I-D.ietf-bess-srv6-services] the ingress PE will add information 164 in the SRv6 packet so that the egress PE can identify the source 165 ES of the BUM packet. That information is the ESI filtering 166 argument of the service SID received on an A-D per ES route from 167 the egress PE. 169 Table 1 shows different tunnel encapsulations and their supported and 170 default Split Horizon method. In the case of GENEVE, the default 171 Split Horizon Type (SHT) depends on whether the Ethernet Option with 172 Source ID TLV is negotiated. In the case of SRv6, the default SHT is 173 listed as ESI label filtering in the Table, since the behavior is 174 equivalent to that of ESI Label filtering. In this document, ESI 175 Label filtering refers to the Split Horizon filtering based on the 176 existence of a source ES identifier in the tunnel header. 178 +---------------+------------------------+-------------+------------+ 179 | Tunnel | Default Split Horizon | Supports | Supports | 180 | Encapsulation | Type (SHT) | Local Bias | ESI Label | 181 +---------------+------------------------+-------------+------------+ 182 | VXLAN | Local Bias | Yes | No | 183 +---------------+------------------------+-------------+------------+ 184 | NVGRE | Local Bias | Yes | No | 185 +---------------+------------------------+-------------+------------+ 186 | MPLS | ESI Label filtering | No | Yes | 187 +---------------+------------------------+-------------+------------+ 188 | MPLSoGRE | ESI Label filtering | Yes | Yes | 189 +---------------+------------------------+-------------+------------+ 190 | MPLSoUDP | ESI Label filtering | Yes | Yes | 191 +---------------+------------------------+-------------+------------+ 192 | GENEVE | Local Bias (no ESI Lb) | Yes | Yes | 193 | | ESI Label (if ESI lb) | | | 194 +---------------+------------------------+-------------+------------+ 195 | SRv6 | ESI Label filtering | Yes | Yes | 196 +---------------+------------------------+-------------+------------+ 198 Table 1: Tunnel Encapsulations and Split Horizon Types 200 The ESI Label method works for All-Active and Single-Active, while 201 Local Bias only works for All-Active. In addition, the ESI Label 202 method works across different network domains, whereas Local Bias is 203 limited to networks with no next hop change between the NVEs attached 204 to the same Ethernet Segment. However, some operators prefer the 205 Local Bias method, since it simplifies the encapsulation, consumes 206 less resources on the NVEs and the ingress NVE always forwards 207 locally to other interfaces, reducing the delay to reach multi-homed 208 hosts. 210 This document extends the EVPN Multi-Homing procedures so that an 211 operator can decide the Split Horizon procedure for a given NVO 212 tunnel depending on their own specific requirements. The choice of 213 Local Bias or ESI Label Split Horizon is now allowed for NVO tunnels 214 that support both methods. Non-MPLS NVO tunnels that do not support 215 both methods, E.g., VXLAN or NVGRE, will keeo following [RFC8365] 216 procedures. 218 1.1. Conventions and Terminology 220 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 221 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 222 "OPTIONAL" in this document are to be interpreted as described in BCP 223 14 [RFC2119] [RFC8174] when, and only when, they appear in all 224 capitals, as shown here. 226 o BUM: Broadcast, Unknown unicast and Multicast traffic. 228 o ES and ESI: Ethernet Segment and Ethernet Segment Identifier. 230 o A-D per ES route: refers to the EVPN Ethernet Auto-Discovery per 231 Ethernet Segment route defined in [RFC7432]. 233 o AC: Attachment Circuit. 235 o NVE: Network Virtualization Edge device. 237 o EVI and EVI-RT: EVPN Instance and EVI Route Target. A group of 238 NVEs attached to the same EVI will share the same EVI-RT. 240 o MPLS and non-MPLS NVO tunnels: refer to Multi-Protocol Label 241 Switching (or the absence of it) Network Virtualization Overlay 242 tunnels. Network Virtualization Overlay tunnels use an IP 243 encapsulation for overlay frames, where the source IP address 244 identifies the ingress NVE and the destination IP address the 245 egress NVE. 247 o MPLSoUDP: Multi-Protocol Label Switching over User Datagram 248 Protocol, [RFC7510] 250 o MPLSoGRE: Multi-Protocol Label Switching over Generic Network 251 Encapsulation, [RFC4023]. 253 o MPLSoX: refers to MPLS over any IP encapsulation. Examples are 254 MPLSoUDP or MPLSoGRE. 256 o GENEVE: Generic Network Virtualization Encapsulation, 257 [I-D.ietf-nvo3-geneve]. 259 o VXLAN: Virtual eXtensible Local Area Network, [RFC7348]. 261 o NVGRE: Network Virtualization Using Generic Routing Encapsulation, 262 [RFC7637]. 264 o VNI: Virtual Network Identifier. A 24-bit identifier used by 265 Network Virtualization Overlay (NVO) over IP encapsulations. 266 Examples are VXLAN (Virtual Extended Local Area Network) or GENEVE 267 (Generic Network Virtualization Encapsulation). 269 o Broadcast Domain (BD): an emulated ethernet, such that two systems 270 on the same BD will receive each other's link-local broadcasts. 271 In this document, BD also refers to the instantiation of a 272 Broadcast Domain on an EVPN PE. An EVPN PE can be attached to one 273 or multiple BDs of the same tenant. 275 o Designated Forwarder (DF): as defined in [RFC7432], an ethernet 276 segment may be multi-homed (attached to more than one PE). An 277 ethernet segment may also contain multiple BDs, of one or more 278 EVIs. For each such EVI, one of the PEs attached to the segment 279 becomes that EVI's DF for that segment. Since a BD may belong to 280 only one EVI, we can speak unambiguously of the BD's DF for a 281 given segment. 283 o SHT: Split Horizon Type, it refers to the Split Horizon method 284 that a PE intends to use and advertises in an A-D per ES route. 286 This document also assumes familiarity with the terminology of 287 [RFC7432] and [RFC8365]. 289 2. BGP EVPN Extensions 291 EVPN extensions are needed so that NVEs can advertise their 292 preference for the Split Horizon method to be used in the Ethernet 293 Segment. Figure 1 shows the ESI Label extended community that is 294 always advertised along with the EVPN A-D per ES route. All the NVEs 295 attached to an Ethernet Segment advertise an A-D per ES route for the 296 ES, including this extended community that conveys the information 297 for the multi-homing mode (All-active or Single-Active), as well as 298 the ESI Label to be used (if needed). 300 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 301 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 302 | Type=0x06 | Sub-Type=0x01 | Flags(1 octet)| Reserved=0 | 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 | Reserved=0 | ESI Label | 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 307 Figure 1: ESI Label extended community 309 [RFC7432] defines the low-order bit of the Flags octet (bit 0) as the 310 "Single-Active" bit: 312 o A value of 0 means that the multi-homed Ethernet Segment is 313 operating in All-Active mode. 315 o A value of 1 means that the multi-homed Ethernet Segment is 316 operating in Single-Active mode. 318 2.1. The Split Horizon Type (SHT) 320 [RFC8365] does not add any explicit indication about the Split 321 Horizon method in the A-D per ES route. In this document, the 322 [RFC8365] Split Horizon procedure is the default behavior and assumes 323 that Local Bias is used only for non-MPLS NVO tunnels, and ESI Label 324 based Split Horizon for MPLS NVO tunnels. This document defines the 325 two high-order bits in the Flags octet (bits 6 and 7) as the "Split 326 Horizon Type" (SHT) field, where: 328 SHT bit 7 6 329 ----------- 330 0 0 --> Default SHT. Backwards compatible with [RFC8365] 331 0 1 --> Local Bias 332 1 0 --> ESI Label based filtering 333 1 1 --> reserved for future use 335 o SHT = 00 is backwards compatible with [RFC8365] and indicates that 336 the advertising NVE intends to use the default or native SHT. The 337 default SHT is shown in Table 1 for each NVO encapsulation. An 338 egress NVE that follows the [RFC8365] behavior and does not 339 support this specification will ignore the SHT bits (which is 340 equivalent to process them as value of 00). 342 o SHT = 01 indicates that the advertising NVE intends to use Local 343 Bias procedures in the Ethernet Segment for which the AD per-ES 344 route is advertised. 346 o SHT = 10 indicates that the advertising NVE intends to use the ESI 347 Label based Split Horizon method procedures in the Ethernet 348 Segment for which the AD per-ES route is advertised. 350 2.2. Use of the Split Horizon Type In A-D Per ES Routes 352 The following must be observed: 354 o An SHT value of 01 or 10 MUST NOT be used with encapsulations that 355 support only one SHT in Table 1, and MAY be used by encapsulations 356 that support the two SHTs in Table 1. 358 o An SHT value different than 00 expresses the intend to use a 359 specific Split Horizon method, but does not reflect the actual 360 operational SHT used by the advertising NVE, unless all the NVEs 361 attached to the ES advertise the same SHT. 363 o In case of inconsistency in the SHT value advertised by the NVEs 364 attached to the same ES for a given EVI, all the NVEs MUST revert 365 to the [RFC8365] behavior, and use the default SHT in Table 1, 366 irrespective of the advertised SHT. 368 o An SHT different from 00 MUST NOT be set if the Single-Active bit 369 is set. A received A-D per ES route where Single-Active and SHT 370 bits are different from zero MUST be treat-as-withdraw [RFC7606]. 372 o The SHT MUST have the same value in each Ethernet A-D per ES route 373 that an NVE advertises for a given ES and a given encapsulation 374 (see Section 3 for NVEs supporting multiple encapsulations). 376 As an example, egress NVEs that support MPLS NVO tunnels, E.g., 377 MPLSoGRE or MPLSoUDP, will advertise A-D per ES route(s) for the ES 378 along with the [RFC5512] BGP Encapsulation extended community 379 [I-D.ietf-idr-tunnel-encaps]indicating the encapsulation (MPLSoGRE or 380 MPLSoUDP) and MAY use the SHT = 01 or 10 to indicate the intend to 381 use Local Bias or ESI Label, respectively. 383 An egress NVE MUST NOT use an SHT value different from 00 when 384 advertising an A-D per ES route with encapsulation VXLAN, NVGRE, MPLS 385 or no [I-D.ietf-idr-tunnel-encaps] BGP tunnel encapsulation extended 386 community. We assume that, in all these cases, there is no Split 387 Horizon method choice, and therefore the SHT value MUST be 00. A 388 received route with one of the above encapsulation options and SHT 389 value different from 00 SHOULD be treat-as-withdraw. 391 An egress NVE advertising A-D per ES route(s) for an ES with 392 encapsulation GENEVE MAY use an SHT value of 01 or 10. A value of 01 393 indicates the intend to use Local Bias, irrespective of the presence 394 of an Ethernet option TLV with a non-zero Source-ID 395 [I-D.ietf-bess-evpn-geneve]. A value of 10 indicates the intend to 396 use ESI Label based Split Horizon. A value of 00 indicates the 397 default behavior in Table 1, that is, use Local Bias if no ESI-Label 398 exists in the Ethernet option TLV or no Ethernet option TLV 399 whatsoever. Otherwise the ESI Label Split Horizon method is used. 401 The above procedures assume a single encapsulation supported in the 402 egress NVE. Section 3 describes additional procedures for NVEs 403 supporting multiple encapsulations. 405 2.3. ESI Label Value In A-D Per ES Routes 407 This document also updates [RFC8365] in the value that is advertised 408 in the ESI Label field of the ESI Label extended community, as 409 follows: 411 o The A-D per ES route(s) for an ES MAY have an ESI Label value of 412 zero if the SHT value is 01. Section 2.2 specifies the cases 413 where the SHT can be 01. An ESI Label value of zero avoids the 414 allocation of Labels in the cases where they are not used (Local 415 Bias). 417 o The A-D per ES route(s) for an ES MAY have an ESI Label value of 418 zero for VXLAN or NVGRE encapsulations. 420 2.4. Backwards Compatibility With [RFC8365] NVEs 422 As discussed in Section 2.2 this specification is backwards 423 compatible with the Split Horizon filtering behavior in [RFC8365] and 424 a non-upgraded NVE can be attached to the same ES as other NVEs 425 supporting this specification. 427 An NVE has an administrative SHT value for an ES (the one that is 428 advertised along with the A-D per ES route) and an operational SHT 429 value (the one that is actually used irrespective of what the NVE 430 advertised). The administrative SHT matches the operational SHT if 431 all the NVEs attached to the ES have the same administrative SHT. 433 This document assumes that an [RFC7432] or [RFC8365] implementation 434 that does not support this document, ignores the value of all the 435 Flags in the ESI Label extended community except for the Single- 436 Active bit. Based on this assumption, a non-upgraded NVE will ignore 437 an SHT different from 00. As soon as an upgraded NVE receives at 438 least one A-D per ES route for the ES with SHT value of 00, it MUST 439 revert its operational SHT to the default Split Horizon method, as in 440 Table 1, and irrespective of its administrative SHT. 442 As an example, consider an NVE attached to Ethernet Segment N that 443 receives two A-D per ES routes for N from different NVEs, NVE1 and 444 NVE2. If the route from NVE1 has SHT = 00 and the one from NVE2 an 445 SHT = 01, the NVE MUST use the default Split Horizon method in 446 Table 1 as operational SHT, irrespective of its administrative SHT. 448 All the NVEs attached to an ES with operational SHT value of 10 MUST 449 advertise a valid non-zero ESI Label. If the operational SHT value 450 is 01, the ESI Label MAY be zero. If the operational SHT value is 451 00, the ESI Label MAY be zero only if the default encapsulation 452 supports Local Bias only and the NVEs do not check the presence of a 453 valid non-zero ESI Label. 455 If an NVE changes its operational SHT value from 01 to 00 (as a 456 result of a new non-upgraded NVE present in the ES) and it previously 457 advertised a zero ESI Label, it MUST send an update with a non-zero 458 valid ESI Label, unless all the non-upgraded NVEs in the ES support 459 Local Bias only. 461 3. Procedures for NVEs Supporting Multiple Encapsulations 463 As specified by [RFC8365], an egress NVE that supports multiple data 464 plane encapsulations (I.e., VXLAN, NVGRE, MPLS, MPLSoUDP, GENEVE) 465 needs to indicate all the supported encapsulations using BGP 466 Encapsulation extended communities defined in 467 [I-D.ietf-idr-tunnel-encaps] with all EVPN routes. This section 468 clarifies the multi-homing Split Horizon behavior for NVEs 469 advertising and receiving multiple BGP Encapsulation extended 470 communities along with the A-D per ES routes. This section uses a 471 notation of {x,y} to indicate the encapsulations advertised in 472 [I-D.ietf-idr-tunnel-encaps] BGP Encapsulation extended communities, 473 with x and y being different encapsulation values. 475 It is important to remember that an NVE MAY advertise multiple A-D 476 per ES routes for the same ES (and not only one), each route 477 conveying a number of Route Targets (RT). We refer to the total 478 number of Route Targets in a given ES as RT-set for that ES. Any of 479 the EVIs represented in the RT-set will have its RT included in one 480 (and only one) A-D per ES route for the ES. When multiple A-D per ES 481 routes are advertised for the same ES, each route MUST have a 482 different Route Distinguisher. 484 As per [RFC8365], an NVE that advertises multiple encapsulations in 485 the A-D per ES route(s) for an ES, MUST advertise encapsulations that 486 use the same Split Horizon filtering method in the same route. For 487 example: 489 o An A-D per ES route for ES-x may be advertised with {VXLAN, NVGRE} 490 encapsulations. 492 o An A-D per ES route for ES-y may be advertised with {MPLS, 493 MPLSoUDP, MPLSoGRE} encapsulations (or a subset). 495 o But an A-D per ES route for ES-z MUST NOT be advertised with 496 {MPLS, VXLAN} encapsulations. 498 This document extends this behavior as follows: 500 a. An A-D per ES route for ES-x may be advertised with multiple 501 encapsulations where some support a single Split Horizon method. 502 In this case, the SHT value MUST be 00. As an example, {VXLAN, 503 NVGRE}, {VXLAN, GENEVE} or {MPLS, MPLSoGRE, MPLSoUDP} can be 504 advertised in an A-D per ES route. In all those cases SHT MUST 505 be 00. 507 b. An A-D per ES route for ES-y may be advertised with multiple 508 encapsulations where all of them support both Split Horizon 509 methods. In this case the SHT value MAY be 01 if the desired 510 method is Local Bias, or 10 if ESI Label based. For example, 511 {MPLSoGRE, MPLSoUDP, GENEVE} (or a subset) may be advertised in 512 an A-D per ES route with SHT value of 01. The ESI Label value in 513 this case MAY be zero. 515 c. If ES-z with RT-set composed of (RT1, RT2, RT3.. RTn) supports 516 multiple encapsulations that require a different Split Horizon 517 method, a different A-D per ES route (or group of routes) per 518 Split Horizon method MUST be advertised. For example, consider n 519 RTs in ES-z and: 521 o the EVIs corresponding to (RT1..RTi) support VXLAN, 523 o the ones for (RTi+1..RTm) (with i. 580 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 581 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 582 May 2017, . 584 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 585 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 586 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 587 2015, . 589 [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., 590 Uttaro, J., and W. Henderickx, "A Network Virtualization 591 Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, 592 DOI 10.17487/RFC8365, March 2018, 593 . 595 6.2. Informative References 597 [I-D.ietf-bess-evpn-geneve] 598 Boutros, S., Sajassi, A., Drake, J., Rabadan, J., and S. 599 Aldrin, "EVPN control plane for Geneve", draft-ietf-bess- 600 evpn-geneve-01 (work in progress), June 2020. 602 [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, 603 L., Sridhar, T., Bursell, M., and C. Wright, "Virtual 604 eXtensible Local Area Network (VXLAN): A Framework for 605 Overlaying Virtualized Layer 2 Networks over Layer 3 606 Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014, 607 . 609 [RFC5512] Mohapatra, P. and E. Rosen, "The BGP Encapsulation 610 Subsequent Address Family Identifier (SAFI) and the BGP 611 Tunnel Encapsulation Attribute", RFC 5512, 612 DOI 10.17487/RFC5512, April 2009, 613 . 615 [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., 616 "Encapsulating MPLS in IP or Generic Routing Encapsulation 617 (GRE)", RFC 4023, DOI 10.17487/RFC4023, March 2005, 618 . 620 [RFC7637] Garg, P., Ed. and Y. Wang, Ed., "NVGRE: Network 621 Virtualization Using Generic Routing Encapsulation", 622 RFC 7637, DOI 10.17487/RFC7637, September 2015, 623 . 625 [RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black, 626 "Encapsulating MPLS in UDP", RFC 7510, 627 DOI 10.17487/RFC7510, April 2015, 628 . 630 [I-D.ietf-nvo3-geneve] 631 Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic 632 Network Virtualization Encapsulation", draft-ietf- 633 nvo3-geneve-16 (work in progress), March 2020. 635 [I-D.ietf-idr-tunnel-encaps] 636 Patel, K., Velde, G., Sangli, S., and J. Scudder, "The BGP 637 Tunnel Encapsulation Attribute", draft-ietf-idr-tunnel- 638 encaps-17 (work in progress), July 2020. 640 [RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K. 641 Patel, "Revised Error Handling for BGP UPDATE Messages", 642 RFC 7606, DOI 10.17487/RFC7606, August 2015, 643 . 645 [I-D.ietf-bess-srv6-services] 646 Dawra, G., Filsfils, C., Raszuk, R., Decraene, B., Zhuang, 647 S., and J. Rabadan, "SRv6 BGP based Overlay services", 648 draft-ietf-bess-srv6-services-04 (work in progress), July 649 2020. 651 Appendix A. Acknowledgments 652 Appendix B. Contributors 654 Authors' Addresses 656 Jorge Rabadan (editor) 657 Nokia 658 777 Middlefield Road 659 Mountain View, CA 94043 660 USA 662 Email: jorge.rabadan@nokia.com 664 Kiran Nagaraj 665 Nokia 666 701 Middlefield Road 667 Mountain View, CA 94043 668 USA 670 Email: kiran.nagaraj@nokia.com 672 Wen Lin 673 Juniper Networks 675 Email: wlin@juniper.net 677 Ali Sajassi 678 Cisco Systems, Inc. 680 Email: sajassi@cisco.com