idnits 2.17.1 draft-wang-bess-evpn-cmac-overload-reduction-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC7623], [RFC7348], [RFC7041], [RFC7432]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. == There are 1 instance of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 344: '...pectively. The EGD and GEIs MUST have...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (1 July 2020) is 1395 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'M1' is mentioned on line 813, but not defined == Missing Reference: 'M4' is mentioned on line 859, but not defined == Missing Reference: 'M2' is mentioned on line 822, but not defined == Missing Reference: 'M3' is mentioned on line 828, but not defined == Missing Reference: 'M5' is mentioned on line 847, but not defined == Missing Reference: '6A' is mentioned on line 864, but not defined == Missing Reference: '6B' is mentioned on line 877, but not defined == Missing Reference: '6C' is mentioned on line 880, but not defined == Missing Reference: '6D' is mentioned on line 894, but not defined == Missing Reference: 'CMAC-Reduction' is mentioned on line 1036, but not defined == Missing Reference: 'EVPN-IRB' is mentioned on line 1039, but not defined == Missing Reference: 'Unified-Encap' is mentioned on line 1042, but not defined == Missing Reference: 'ESI-Retained' is mentioned on line 1045, but not defined == Missing Reference: 'Flexible-MH' is mentioned on line 1048, but not defined == Missing Reference: 'Learn-Confined' is mentioned on line 1051, but not defined == Missing Reference: 'AA-no-flush' is mentioned on line 1054, but not defined == Missing Reference: 'SA-independent' is mentioned on line 1057, but not defined == Outdated reference: A later version (-14) exists of draft-ietf-bess-mvpn-evpn-aggregation-label-03 ** Downref: Normative reference to an Informational RFC: RFC 7348 Summary: 3 errors (**), 0 flaws (~~), 20 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS WG Y. Wang 3 Internet-Draft ZTE Corporation 4 Intended status: Standards Track 1 July 2020 5 Expires: 2 January 2021 7 Reduction of EVPN C-MAC Overload 8 draft-wang-bess-evpn-cmac-overload-reduction-01 10 Abstract 12 When there are too many customer-MACs (C-MACs), the RRs and/or ASBRs 13 will be overloaded by the RT-2 routes for these MACs according to 14 [RFC7432]. This issue can be simply solved by making the remote 15 C-MAC entries learnt via data-plane MAC learning (like what PBB VPLS 16 have done since [RFC7041]) rather than received from RT-2 routes. 17 This simplified solution will works as well as PBB VPLS. But this 18 simplified solution will lose many important features that based on 19 the ESI concept. Because the ingress-ESI can't be learnt via data- 20 plane MAC learning at the egress PE. So when the data packets is 21 forwarded following these MAC entries, they can't benefit from the 22 EAD/EVI routes as per RFC7432. So the All-Active Redundancy mode for 23 ES can't be supported. This make the simplified solution can't work 24 as well as PBB EVPN ([RFC7623]). 26 This document proposes some new extensions to [RFC7432] to achieve 27 all-active mode ES redundancy on TPEs and reduce the C-MAC loads for 28 RRs and ASBRs at the same time. The new solution will work even more 29 better than PBB EVPN under the help of these extensions, especially 30 when there is no deployment of MPLS dataplane. 32 In Section 6.2, this draft is compared with PBB EVPN and other 33 solutions . These solutions are PBB EVPN, PBB VPLS, VXLAN([RFC7348]). 35 Status of This Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at https://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on 2 January 2021. 51 Copyright Notice 53 Copyright (c) 2020 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 58 license-info) in effect on the date of publication of this document. 59 Please review these documents carefully, as they describe your rights 60 and restrictions with respect to this document. Code Components 61 extracted from this document must include Simplified BSD License text 62 as described in Section 4.e of the Trust Legal Provisions and are 63 provided without warranty as described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 68 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 69 2. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 5 70 2.1. No C-MAC Awareness in the Backbone . . . . . . . . . . . 6 71 2.2. EVPN IRB Support . . . . . . . . . . . . . . . . . . . . 6 72 2.3. Unified Encapsulation per Scenario . . . . . . . . . . . 6 73 2.4. ESI Features Remain Supported . . . . . . . . . . . . . . 6 74 2.5. Flexible Multi-homing Remains Supported . . . . . . . . . 7 75 2.6. C-MAC Address Learning and Confinement . . . . . . . . . 7 76 2.7. No C-MAC Flushing for All-Active ESes . . . . . . . . . . 7 77 2.8. Independent C-MAC Flushing for Single-Active ESes . . . . 7 78 2.9. Independent Convergency per . . . . . . . . . 7 79 2.10. Route Aggregation and Default Route in Backbone . . . . . 8 80 3. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 8 81 3.1. Common Overview . . . . . . . . . . . . . . . . . . . . . 8 82 3.2. Pseudo PBB EVPN . . . . . . . . . . . . . . . . . . . . . 8 83 3.3. VXLAN Solution Overview . . . . . . . . . . . . . . . . . 10 84 3.4. MPLS Solution Overview . . . . . . . . . . . . . . . . . 12 85 3.5. SRv6 Solution Overview . . . . . . . . . . . . . . . . . 13 86 3.6. Comparisons of Relative Concepts . . . . . . . . . . . . 14 87 4. Dataplane-specific Procedures . . . . . . . . . . . . . . . . 15 88 4.1. Packet Walkthrough . . . . . . . . . . . . . . . . . . . 15 89 4.2. Pseudo PBB-EVPN . . . . . . . . . . . . . . . . . . . . . 16 90 4.3. VXLAN over IP-VRF . . . . . . . . . . . . . . . . . . . . 17 91 4.4. NVO3-specific EVPN-lite Procedures . . . . . . . . . . . 17 92 4.5. MPLS-specific EVPN-lite Procedures . . . . . . . . . . . 18 93 4.6. SRv6-specific EVPN-lite Procedures . . . . . . . . . . . 19 94 5. Other Considerations . . . . . . . . . . . . . . . . . . . . 20 95 5.1. ESI Indicator Advertisement Optimization . . . . . . . . 20 96 5.2. C-MAC Flush Notification Procedure . . . . . . . . . . . 20 97 5.3. E-Tree Support Considerations . . . . . . . . . . . . . . 20 98 5.4. EVPN IRB Support Considerations . . . . . . . . . . . . . 21 99 5.5. Use End.ESI SID in MAC/IP Advertisement Routes . . . . . 21 100 5.6. Hierarchical VPLS in EVPN-lite . . . . . . . . . . . . . 21 101 6. Comparison with Other Solutions . . . . . . . . . . . . . . . 22 102 6.1. Questions . . . . . . . . . . . . . . . . . . . . . . . . 22 103 6.2. Summary Comparisons . . . . . . . . . . . . . . . . . . . 23 104 6.3. Detailed Comparisons with PBB EVPN . . . . . . . . . . . 24 105 6.4. Detailed Comparisons with PBB VPLS . . . . . . . . . . . 24 106 6.5. Detailed Comparisons with EVPN VXLAN . . . . . . . . . . 24 107 7. Security Considerations . . . . . . . . . . . . . . . . . . . 24 108 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 109 8.1. END.ESI SID . . . . . . . . . . . . . . . . . . . . . . . 24 110 8.2. Global Unique ESI-label in EAD per ES Route . . . . . . . 25 111 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 112 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 113 10.1. Normative References . . . . . . . . . . . . . . . . . . 25 114 10.2. Informative References . . . . . . . . . . . . . . . . . 26 115 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 26 117 1. Introduction 119 In [RFC7432], the C-MACs is advertised via RT-2 route. This behavior 120 is inheritted by [RFC8365] and [I-D.dawra-bess-srv6-services]. but 121 in order to solve the C-MAC overload problem for RRs and ASBRs, we 122 have to return to a PBB-like dataplane C-MAC learning procedures. 124 We discuss all the requirements for a light-weight EVPN solution 125 which pushes no C-MAC entries into the backbone network in Section 2. 126 Note that some of these requirements is not supported well by PBB 127 EVPN. 129 In this document, the light-weight EVPN solutions are also called as 130 EVPN-lite for short. A total of five EVPN-lite solutions are 131 proposed in Section 3. These solutions are Pseudo PBB-EVPN, VXLAN 132 over IP-VRF, VXLAN-based EVPN-lite, MPLS-based EVPN-lite, SRv6-based 133 EVPN-lite. 135 Note that the Pseudo PBB-EVPN solution is not a pragmatic solution, 136 it is just a techniquely practicable solution for us to easily 137 understand the equivalency between PBB EVPN and VXLAN over IP-VRF. 138 Because the Pseudo PBB-EVPN is exactly the same as PBB EVPN in the 139 control-plane, at the same time, the Pseudo PBB-EVPN is exactly the 140 same as VXLAN over IP-VRF in the data-plane. 142 Note that the VXLAN-based EVPN-lite and MPLS-based EVPN-lite are both 143 a evolution of VXLAN over IP-VRF. The former takes the MPLS factors 144 off the VXLAN over IP-VRF solution, and becomes a pure VXLAN 145 solution. The latter takes the VXLAN factors off the VXLAN over IP- 146 VRF solution, and becomes a pure MPLS solution. 148 In order to compare these five solutions with [RFC7348] and [RFC7623] 149 whose C-MAC entries are also not pushed into the backbone network, 150 two terms are introduced in this document, because the comparisons 151 need to be done in unified terminology. One term is "Global ESI 152 Indicator (GEI)", which is called as B-MAC in PBB EVPN. The other 153 term is "EVI's Global Dicreminator (EGD)", which is called as I-SID 154 in PBB EVPN. 156 Note that the EVI here corresponds to the I-Component of [RFC7623], 157 not the B-Component. In fact, there will be no B-components in some 158 of the above seven solutions. 160 Note that the GEI and EGD in different EVPN-lite solutions are very 161 different. The details will be described in Section 3. 163 On the basis of GEI concept, then we define two route-types for EVPN- 164 lite: The first route type is GEI/ES route, which is called as RT-2 165 route in PBB EVPN. The second route type is GEI/EVI route, which is 166 called as EAD/EVI roue in [RFC7432]. 168 The details of these terms are described in Section 1.1. 170 1.1. Terminology 172 Most of the terminology used in this documents comes from [RFC7432] 173 and [I-D.dawra-bess-srv6-services] except for the following: 175 * C-MAC: Customer MAC, it is the same as the C-MAC of PBB EVPN. 177 * ISID: a broadcast domain identifier in PBB I-Component. 179 * LDV: Local Discreminating Value. It is similar to the Local 180 Discreminating Value of type 3 ESI. 182 * GDV: Global Discreminating Value. An identifier with global 183 uniqueness. 185 * EGD: EVI-GDV, an EVI's Global Discreminator, it is a GDV for an 186 EVI instance. A EGD is used to idenfify an EVPN Instance (EVI) in 187 data plane. The EGD is a Global Discreminating Value (GDV) of 188 that EVI, so it is also the abbreviation of EVI-GDV. e.g. The 189 EGD of [RFC7348] is a global VNI. 191 * ESI Indicator: A Global ID for an ESI. Note that different PE may 192 assign different ESI-indicator for the same ESI, espacially when 193 the ES redundancy mode is single-active. e.g. The ESI indicator 194 of [RFC7623] is B-MAC. 196 * GEI: Global ESI Indicator. It is the same as the "ESI Indicator" 197 except for the emphasization to its global uniqueness. A GEI is 198 used in data plane to identify an ESI, because it have global 199 uniqueness across the service domain of a corresponding EVPN 200 Instance (EVI). But an ESI may have a few GEIs, each for a TPE, 201 espacially in the single-active mode of ES redundancy. And in 202 E-Tree scenarios, an ESI may have two GEIs on the same PE, one for 203 Root ACs, one for Leaf ACs. e.g. The GEIs for an ESI of 204 [RFC8317] is two B-MACs, one for root ACs, one for Leaf ACs. 206 * GEI/ES: The EVPN route which is used to advertise the relation 207 between ESE and its GEI. Note that the GEI/ES route is advertised 208 per ESI basis on a specified PE. In PBB EVPN, the GEI/ES route is 209 the MAC Advertisement Route. Note that different solutions may 210 have different GEI/ES routes. 212 * EAD/EVI: An Ethernet A-D route per EVI. 214 * GEI/EVI: The EVPN route which is used to advertise the relation 215 between and its EVPN label and MPLS nexthops. Note 216 that in PBB EVPN, such route is not used. Note that different 217 solutions may have different GEI/EVI routes. 219 * ARG.EGD: The argument part of a SID of the End.ESI function is 220 called as ARG.EGD, because the value of that argument will be a 221 EGD. 223 * RT-2: MAC/IP Advertise Route. 225 * MAC Entry: An entry in the EVPN MAC table in data-plane. 227 * ESI IP: An End.ESI SID with its Argument part being set to zero. 229 * VXLAN EVPN: EVPN per [RFC8365]. 231 * EVPN VXLAN: A broadcast domain per [RFC7348], but use IMET routes 232 of [RFC8365] to construct VXLAN tunnels. 234 2. Requirements 236 EVPN C-MAC Reduction should be provided together with the following 237 requirements: 239 2.1. No C-MAC Awareness in the Backbone 241 In typical operation, an EVPN PE sends a BGP MAC Advertisement route 242 per C-MAC address. In certain applications, this poses scalability 243 challenges, as is the case in data center interconnect (DCI) 244 scenarios where the number of virtual machines (VMs), and hence the 245 number of C-MAC addresses, can be in the millions. This is called as 246 C-MAC overload of DC Backbone. In such scenarios, it is required to 247 reduce the number of BGP MAC Advertisement routes by relying on a 248 'EVPN-lite' scheme, as is provided by ESI and its equivalents (e.g. 249 Pseudo B-MAC, ESI IP). 251 2.2. EVPN IRB Support 253 The PBB-VPLS/PBB-EVPN is not friendly to IRB usecase because of its 254 complicated Protocol Stack, so it is used just in pure L2VPN usecase 255 up to now in the industry. 257 The solution should provider efficient forwarding performance in EVPN 258 IRB use cases. 260 2.3. Unified Encapsulation per Scenario 262 PBB EVPN, especially the MPLS encapsulation of its B-VPLS, is 263 typically not used in DC Scenario. So we bring PBB and MPLS 264 encapsulation to DC Backbone just due to the C-MAC overload problem. 265 EVPN IRB is widely deplyed in DC scenarios, but PBB EVPN is not 266 friendly for EVPN IRB use cases. So we have to use different 267 solutions in EVPN IRB and C-MAC reduction use cases. We believe that 268 if we choose VXLAN/Geneve data-plane, we will prefer to use the same 269 data-plane in all use cases, e.g. EVPN IRB, C-MAC reduction. So it 270 is necessary to make NVO3/MPLS/SRv6 EVPN to support Section 2.1 in 271 order to provider a unified solution for data center and other 272 secenarios. 274 2.4. ESI Features Remain Supported 276 Two redundancy modes are defined in [RFC7432]. They are All-Active 277 mode and Single-Active mode. 279 In All-active mode, the C-MAC movement among the different adjacent 280 PE nodes of the same ESI should not be considered as C-MAC mobility. 281 In Single-Active mode, such movements can be considered as C-MAC 282 mobility. 284 2.5. Flexible Multi-homing Remains Supported 286 Flexible multi-homing means that different ES instances can have 287 different adjacent-PEs. We call all the adjacent-PEs of the same ES 288 instances as that ES's location-set in this document. Flexible 289 multi-homing means that different ES can have different location-set. 291 For example, ES1's location-set is {PE1}, ES2's location-set is {PE2, 292 PE3}, ES3's location-set is {PE1, PE3}, and ES4's location-set is 293 {PE2,PE4}. 295 2.6. C-MAC Address Learning and Confinement 297 In EVPN, all the PE nodes participating in the same EVPN instance are 298 exposed to all the C-MAC addresses learned by any one of these PE 299 nodes because a C-MAC learned by one of the PE nodes is advertised in 300 BGP to other PE nodes in that EVPN instance. This is the case even 301 if some of the PE nodes for that EVPN instance are not involved in 302 forwarding traffic to, or from, these C-MAC addresses. Even if an 303 implementation does not install hardware forwarding entries for C-MAC 304 addresses that are not part of active traffic flows on that PE, the 305 device memory is still consumed by keeping record of the C-MAC 306 addresses in the routing information base (RIB) table. In network 307 applications with millions of C-MAC addresses, this introduces a non- 308 trivial waste of PE resources. As such, it is required to confine 309 the scope of visibility of C-MAC addresses to only those PE nodes 310 that are actively involved in forwarding traffic to, or from, these 311 addresses. 313 2.7. No C-MAC Flushing for All-Active ESes 315 Just as in [RFC7432], it is required to avoid C-MAC address flushing 316 upon link, port, or node failure for All-Active multihomed segments. 318 2.8. Independent C-MAC Flushing for Single-Active ESes 320 Just as in [RFC7432], upon singel-active ES1's link or port failure, 321 the C-MACs of other single-active ESes from the same PE will not be 322 flushed. 324 2.9. Independent Convergency per 326 When the physical port of an All-Active ES works well, but a single 327 Ethernet Tag ID (ETI) of that ES fails, The traffic to that ETI of 328 that ES will be re-routed to other adjacent PE of the same ES, but 329 the traffic to other ETIs of the same ES will not be affected. 331 2.10. Route Aggregation and Default Route in Backbone 333 The routes per ESIs can be aggregated in Backbone network. Even the 334 default route should be supported. 336 3. Solution Overview 338 3.1. Common Overview 340 We assign a Global Discreminator EGD1 to an EVI instance EVI1, the 341 EGD1 is a number consists of N bits. We assign an ESI-indicator GEI1 342 to ESI1 on PE1, and we assign an ESI-indicator GEI2 to ESI1 on PE2. 343 We call the relationship between ESI1 and its two ESI-indicators as 344 ESI1_GEI1 and ESI1_GEI2 respectively. The EGD and GEIs MUST have 345 global uniqueness in EVI1's service domain. 347 +----------+ 348 PE1 | | 349 +-------------+ | | 350 | ESI1_GEI1 | | | PE3 351 /| |----| | +-------------+ 352 / | | | IP/MPLS | | | 353 LAG / +-------------+ | Backbone | | ESI2_GEI3 |---CE2 354 CE1===== | with | | | 355 \ +-------------+ | EVPN |---| | 356 \ | | | RRs | +-------------+ 357 \| |----| and | 358 | ESI1_GEI2 | | SPEs | 359 +-------------+ | | 360 PE2 | | 361 +----------+ 363 Figure 1: EVPN MAC Reduction Usecase 365 We use IMET routes to build a broadcast-list. The broadcast-list is 366 used to forward BUM traffics. The data-plane MAC learning for BUM 367 traffics produces the first batch of C-MAC entries. The subsequent 368 C-MAC entries can be learnt from Unicast traffics and/or BUM 369 traffics. It is clear that we don't use MAC/IP routes to advertise 370 C-MAC entries as usual, that is for fear that the RRs and/or ASBRs 371 are overloaded by these C-MACs. 373 3.2. Pseudo PBB EVPN 375 Pseudo PBB EVPN means that we use the management and control plane of 376 PBB EVPN but use the data plane of EVPN VXLAN. 378 The BMACs of Pseudo PBB EVPN is called as Pseudo BMACs in this draft, 379 the format of a pseudo-BMAC is illustrated as the following: 381 | 24 bits | 24 bits | 382 +----+----+----+----+----+----+----+----+----+----+----+----+ 383 | OUI = PBB2IP | NIC-ID = ESI IP's Host ID | 384 +----+----+----+----+----+----+----+----+----+----+----+----+ 386 Figure 2: GEI Format of Pseudo PBB-EVPN 388 The OUI part of a pseudo-BMAC is a constant value called PBB2IP. We 389 suppose that PBB2IP is 0x20000A, so that the lower four bytes of the 390 pseudo-BMAC is exactly right a IP address of the IPv4 prefix 391 10.0.0.0/8. When the Pseudo-BMAC is for an ESI x, that IP address is 392 called as ESI x's ESI IP in Pseudo PBB-EVPN. When the Pseudo-BMAC is 393 for a PE a, that IP address is called as PE a's PE IP in Pseudo PBB- 394 EVPN. The Pseudo-BMAC is not used in the dataplane, that's why it is 395 called as pseudo-BMAC. 397 Note that when the NIC-specific part of a pseudo-BMAC has the same 398 value with the host ID part of an IPv4 address, We say that this IPv4 399 address equals that pseudo-BMAC, or this IPv4 address in data plane 400 is in the name of that pseudo-BMAC, or just "IP=BMAC". 402 The ESI/PE IP will be used in the dataplane in the name of their 403 Pseudo-BMAC. So it won't be so surprising that we don't use PBB 404 encapsulation in dataplane. In fact we use VXLAN over IP-VRF data 405 plane in Pseudo PBB-EVPN, the encapsulation is illustrated as the 406 following: 408 +----------------------------------------+ 409 | IP/MPLS Header (Backbone IP-VRF label) | 410 +----------------------------------------+ 411 | IPv4 Header (SIP=S-BMAC, DIP=D-BMAC) | 412 +----------------------------------------+ 413 | UDP Header | 414 +----------------------------------------+ 415 | VXLAN Header (VNI=EGD=I-SID) | 416 +----------------------------------------+ 417 | Ethernet Header | 418 +----------------------------------------+ 419 | Ethernet Payload | 420 +----------------------------------------+ 421 | Ethernet FCS | 422 +----------------------------------------+ 424 Figure 3: Pseudo PBB-EVPN Encapsulation 426 It is clearly illustrated that the VXLAN header is in the name of old 427 PBB Header, the VNI is in the name of old I-SID, the underlay source/ 428 destination IP of VXLAN header is in the name of source/destination 429 BMAC, the backbone IP-VRF is in the name of old B-VPLS instance. 431 As a sequence of the above settings, the B-MAC based forwarding in 432 old B-VPLS is replaced with IP forwarding in IP-VRF instances. 434 Note that the encapsulation in Figure 3 is completely the same as 435 [RFC7348] except for the MPLS encapsulation. 437 Note that the GEI in the control-plane is B-MAC, the GEI in the data- 438 plane is ESI-IP. And the EGD in control-plaen is I-SID, the EGD in 439 data-plane is VNI. 441 The GEI/ES route is the RT-2 route of PBB-EVPN. There is no GEI/EVI 442 route in Pseudo PBB-EVPN, just as there is no GEI/EVI route in PBB 443 EVPN. 445 3.3. VXLAN Solution Overview 447 Although the VXLAN encapsulation is used in Pseudo PBB-EVPN, the MPLS 448 data plane is also needed, because that the B-Component of Pseudo 449 PBB-EVPN is an IP-VRF. So we introduce a completely VXLAN 450 encapsulation in this section, it is called as VXLAN solution. 452 In VXLAN solution, the GEI is composed of a VTEP IP and an ESI label. 453 It is illustrated as the following: 455 | 32 bits | 16 bits | 456 +----+----+----+----+----+----+----+----+----+----+----+----+ 457 | VTEP IP | ESI Label | 458 +----+----+----+----+----+----+----+----+----+----+----+----+ 460 Figure 4: VXLAN GEI Format 462 The VTEP IP is PE1/PE2/PE3's IP address, the ESI label is a local 463 discreminating value (LDV) that is used to identify a ESI. So the 464 GEI has global uniqueness in the EVPN domain. 466 Note that the GEIs (on PE1 and PE2) of ESI1 don't have to be the 467 same. But if we let them to be the same through configuration, it 468 will work well too. 470 We use the following encapsulation in this solution: 472 +----------------------------------------+ 473 | IPv4 Header (SIP=VTEP IP) | 474 +----------------------------------------+ 475 | UDP Header (Source Port ^= ESI label) | 476 +----------------------------------------+ 477 | VXLAN Header (VNI=EGD) | 478 +----------------------------------------+ 479 | Ethernet Header | 480 +----------------------------------------+ 481 | Ethernet Payload | 482 +----------------------------------------+ 483 | Ethernet FCS | 484 +----------------------------------------+ 486 Figure 5: VXLAN Encapsulation for EVPN-lite 488 Note that only the ingress GEI will be encapsulated in data-plane, 489 the VTEP IP of ingress GEI is encapsulated as source IP, the ESI 490 label is encrypted into the source port. 492 Note that the cryptographic key for ESI label of the inner packet's 493 intrinsic entropy. The intrinsic entropy is a 16 bits unsigned 494 integer that is computed from nothing but the inner packet's fields. 495 When the ESI label is encrypted into the source port, then the source 496 port will contain both the context entropy and the intrinsic entropy 497 of the inner packet. The source port is now called as the 498 compositive entropy of the inner packet. 500 The GEI/ES route of VXLAN-based EVPN-lite is the RT-1 per ES route. 501 The ESI-label attribute is used to carry the GEI1's ESI-label field. 502 The OPE TLV or nexthop is used to carry the GEI1's VTEP IP field. 504 On receiving the RT-1 per ESI1 route R1 from PE1, PE3 will install a 505 GEI mapping entry ME1 into the data-plane. On receiving a VXLAN 506 packet VP1 of Figure 5 format, PE3 recomputes the intrinsic entropy 507 of VP1 by the same algorithm, the PE3 decrypts GEI1's ESI label part 508 from VP1's UDP source port using the intrinsic entropy. Then PE3 509 learn's VP1's inner S-MAC MAC1 whose destination ESI is ESI1 510 according to the GEI mapping entry ME1. 512 Note that the simplest encryption algorithm may be the bitwise XOR. 513 And it is good enough for our use case. 515 On receiving a ethernet packet EP1 whose D-MAC is MAC1, PE3 will 516 forwarded EP1 to PE1/PE2 following ESI1's RT-1 per EVI routes for 517 EVI1. 519 On receiving the RT-1 per ESI route R2 from PE1, PE2 will install a 520 GEI mapping entry ME2 into the data-plane. Then, on receiving a 521 VXLAN packet VP2 of Figure 5 format, when VP2 is about to be 522 forwarded to ESI1, PE2 will drop VP2 because that its ingress GEI is 523 ESI1 (according to the GEI mapping entry ME2) too. 525 3.4. MPLS Solution Overview 527 In MPLS EVPN control plane, we use a 24 bits unsigned number as the 528 EGD of EVI1, and it has global uniqueness in EVI1's service domain. 529 In data plane, we use QinQ tags to carry the EGD. 531 We use a Global Unique Label (GUL) to identify a ESI in EVI1's 532 service domain. So the ESI-GUL is also its Global ESI Indicator. 533 The ESI-GULs are avertised through RT-1 per ES routes, and they are 534 considered to be an ESI-label by these routes. The label in RT-3 535 route's PMSI-Tunnel Attribute (PTA-Label) whose tunnel type is 536 ingress replication is called as Ingress Replication Multicast Label 537 (IRML) in this document. 539 We use the following encapsulation in MPLS-based EVPN-lite: 541 Format #1 Format #2 542 +-----------------------+ +----------------------------+ 543 | PSN Labels | | PSN Labels | 544 +-----------------------+ +----------------------------+ 545 | IRML (EVI1) | | Destination-ESI GUL (ESI1) | 546 +-----------------------+ +----------------------------+ 547 | Source-ESI GUL (ESI1) | | Source-ESI GUL (ESI2) | 548 +-----------------------+ +----------------------------+ 549 | Ethernet Header | | Ethernet Header (EVI1) | 550 +-----------------------+ +----------------------------+ 551 | Ethernet Payload | | Ethernet Payload | 552 +-----------------------+ +----------------------------+ 553 | Ethernet FCS | | Ethernet FCS | 554 +-----------------------+ +----------------------------+ 556 Figure 6: MPLS Encapsulation for EVPN-lite 558 Note that the GUL can be a single Label Stack Entry (LSE), in such 559 case, it should be allocated in DCB label space. Given that the ESIs 560 and vESIs may be too many to be allocated in DCB in certain 561 scenarios, so the GUL should be allocated in a few context-specific 562 label spaces, each identified by a Context Label Space ID (CLS-ID) 563 per [I-D.ietf-bess-mvpn-evpn-aggregation-label] in such case. In 564 such case, the ESI-GUL is the entirety of ESI-label and its Context 565 Label Space ID (CLS-ID), so it means two LSEs in the Label Stack at 566 that time. 568 Note that the ESI GULs are assigned by a center authority, which may 569 be a DC controller or an administrator. 571 Note that the ESI-label (ESI-GUL) must be pushed onto the Label Stack 572 whether the packet is BUM or not. The ESI-GUL can't identify the 573 EVPN Instance EVI1, so we have to use the EGD in the inner ethernet 574 header of "Format #2" to find EVI1 out. 576 Note that the GUL concept is very different with the "upstream- 577 assigned label (UAL)" concept. Because that when we receives a GUL 578 from a remote PE, the GUL is considered as an outgoing-label to that 579 PE, and the GUL is also considered as a incoming-label of the current 580 PE, and the label operation for the GUL will be a "swap", to be 581 precise, The SPE will swap it to itself and then push the MPLS Label 582 Stack to that remote PE. When the same GUL is received from 583 different remote PEs, MPLS ECMP or FRR procedures will be applied. 585 So when the GUL is two LSEs in the label stack, we can say that the 586 Context-specific Label Space (CLS) of the ESI-label (inside the GUL) 587 takes the role of B-VPLS of PBB EVPN, and the CLS-ID label inside the 588 GUL takes the role of the B-VPLS label of PBB EVPN. So no B-VPLS 589 instances will be found here. 591 Note that the GEI/ES route of MPLS-based EVPN-lite is the RT-1 per ES 592 route. 594 3.5. SRv6 Solution Overview 596 We introduce a SRv6 function named End.ESI to carry the ESI-indicator 597 in SRv6 dataplane. A SID with the End.ESI function is called as an 598 "ESI SID" in this document. The GEI is the locator and fuction part 599 of its corresponding ESI SID. The argument part of the ESI SID is 600 the EGD for an EVI. The EGD works like the function part of an 601 End.DT2U/DT2M SID. But the EGD has a global meaning like a global 602 VNI or an PBB ISID but the function part for an End.DT2U/DT2M SID 603 typically is only a local discreminator on the egress PE. The 604 argument part of the ESI SID is called as ARG.EGD in this document, 605 where the EGD is the abbreviation of EVI-GDV. 607 The SRv6 SID in IMET route is an End.DT2M SID with a zero argument 608 length. The GEI1 and GEI2 are SRv6 SID of End.ESI function that is 609 defined in the following figure. We use IGP protocols to advertise 610 GEI1 and GEI2 to PE3 respectively in SRv6 underlay. So we don't use 611 EAD/ES route or EAD/EVI route in SRv6 EVPN in this section. If ESI1 612 is single-active mode, GEI1 is different from GEI2, but if ESI1 is 613 all-active mode, GEI1 is the same as GEI2. 615 | ESI-Indicator(128-N bits) | N bits | 616 +------------+------------+-----------+-------------------------+ 617 | Block | Node | ESI.LDV | ARG.EGD | 618 +------------+------------+-----------+-------------------------+ 620 Figure 7: End.ESI SID Format 622 Note that an ESI-indicator is composed of Locator and Function, an 623 ESI IP is an 128 bits SID with a zero argument. The function part is 624 a Local Discreminating Value (LDV) on that PE for the ESI. The 625 argument part is a EVI-GDV (EGD) for the EVPN Instance. The argument 626 part is also called ARG.EGD in this document. 628 3.6. Comparisons of Relative Concepts 630 In this section, we compare the concepts of these four solutions with 631 PBB EVPN. These solutions are: 633 * PBB EVPN: Standard PBB MPLS EVPN per [RFC7623]. 634 * Pseudo PBB EVPN: PBB EVPN Control Plane with IP Dataplane, see 635 Section 3.2 for details. 636 * NVO3-based EVPN-lite: C-MAC overload Reduction of NVO3 EVPN. 637 * MPLS-based EVPN-lite: C-MAC overload Reduction of MPLS EVPN. 638 * SRv6-based EVPN-lite: C-MAC overload Reduction of SRv6 EVPN. 640 We place the detailed comparisons for each solution in separated 641 sections, but we place the brief comparisons in the following table: 643 +==========+================+=============+============+============+ 644 | PBB-EVPN | Pseudo PBB | NVO3-based | MPLS-based | SRv6-based | 645 +==========+================+=============+============+============+ 646 | I-VPLS | VXLAN | EVI | EVI | EVI | 647 | | Instance | | | | 648 +----------+----------------+-------------+------------+------------+ 649 | I-SID | VNI | VNI | QinQ | ARG.EGD | 650 +----------+----------------+-------------+------------+------------+ 651 | C-MAC | C-MAC | MAC | MAC | MAC | 652 +----------+----------------+-------------+------------+------------+ 653 | B-VPLS | IP-VRF | IP Underlay | Label | SRv6 | 654 | | | | Space | Underlay | 655 +----------+----------------+-------------+------------+------------+ 656 | ESI-BMAC | ESI IP | VTEP+S-Port | ESI-GUL | ESI SID | 657 +----------+----------------+-------------+------------+------------+ 658 | BUM-BMAC | DIP | DIP | N/A | End.PBB | 659 +----------+----------------+-------------+------------+------------+ 660 | S-BMAC | SIP | SIP | S-ESI GUL | SIP | 661 +----------+----------------+-------------+------------+------------+ 662 | D-BMAC | DIP | DIP | D-ESI GUL | DIP | 663 +----------+----------------+-------------+------------+------------+ 664 | MPLS | MPLS | VXLAN | MPLS | SRv6 | 665 | Tunnel | Tunnel | Tunnel | Tunnel | Tunnel | 666 +----------+----------------+-------------+------------+------------+ 668 Table 1: Concepts Comparisons 670 * I-VPLS - The I-Component of PBB EVPN. 671 * I-SID - The Identifier of I-VPLS, and the EGD of PBB EVPN. 672 * B-VPLS - The B-Component of PBB EVPN 673 * DIP - Destination IP. 674 * SIP - Source IP . 675 * BMAC - Backbone MAC, including Source-BMAC (S-BMAC) and 676 Destination-BMAC (D-BMAC). 678 4. Dataplane-specific Procedures 680 4.1. Packet Walkthrough 682 #1 [PE1 forward ARP Request to PE2/PE3] 684 * When CE1 requests CE2's ARP, PE1 will receive the ARP Request BUM1 685 from a AC (say AC1) of ESI1. PE1 will forward the ARP Request 686 following the broadcast-list of AC1's EVI instance(say EVI1. The 687 broadcast-list is constructed by IMET routes from PE2/PE3. 689 PE1 will forward the ARP Request to PE2/PE3. The ARP Request is 690 encapsulated with GEI1 and EVI1_GDV1. The inner SMAC of the ARP 691 request is M1 which is CE1's MAC address. 693 #2 [PE2/PE3's Dataplane MAC Learning] 695 * When PE2/PE3 receives the ARP Request packet BUM1, they do 696 dataplane MAC learning independently. They will learn that M1 is 697 behind GEI1. 699 Note that when PE2 learns that M1 is behind GEI1, it will assume 700 that M1 is behind the local AC whose ESI-indicator is GEI1 too. 701 The local AC may have more higher priority than the remote one. 703 After the dataplane MAC learning, the ARP request packet BUM1 is 704 broadcasted to the local ACs, behind one of which is CE2. 706 #3 [PE2 Discard ARP Request to CE1] 707 * On receiving BUM1 from PE1, PE2 use the ingress GEI information in 708 BUM1 to determine its ingress ESI ESI1, When ESI1 is all-active 709 mode and PE2 is about to forward the ARP request to CE1, PE2 will 710 find that the ESI for the outgoing AC is also ESI1, so PE2 711 discards it for ESI loop-free considerations. 713 When ESI1 is single-active mode, the outgoing AC may be in 714 blocking state, otherwise its corresponding sub-interface on CE1 715 will take charge of packet-drop behavior instead. So alghough the 716 ESI for the outgoing AC is not the same as ESI1, no loop will 717 arise in the Ethernet Segment. 719 #4 [PE3 Forward ARP Replay to PE1/PE2] 721 * When CE2 replies to CE1 for the ARP request, PE3 will forward the 722 ARP reply U1 according to the MAC entry M1 learned previously as 723 above. 725 PE3 will forward the ARP reply U1 to PE1 or PE2 according to 726 ESI1's RT-1 per EVI routes and RT-1 per ES routes: 728 When ESI1 is all-active mode, GEI1 may be the same as GEI2, in 729 such case, we call both of them GEI21 instead. The traffics to M1 730 will be load-balanced between PE1 and PE2. Because that GEI21 is 731 advertised by both PE1 and PE2l. 733 #5 [PE1 Forward ARP Replay to CE1] 735 * Whe PE1 received the ARP reply packet U1 from PE3, PE1 first match 736 the packet to the its EVI instance EVI1 by U1's EGD information. 737 And PE1 will not discard it because the egress ESI is not the same 738 as the ingress ESI which is determined by U1's GEI information. 740 4.2. Pseudo PBB-EVPN 742 The data plane of Pseudo PBB-EVPN is the same as VXLAN over IP-VRF 743 except for a few notable differences. 745 Different data packets from PE1 to PE2 will have different SIPs and 746 DIPs. When their egress ESIs are different their DIP will be 747 different too. When their ingress ESIs are different their SIP will 748 be different too. So we can't do (as known as VXLAN 749 tunnel) filtering like what have been done in normal VXLAN EVPN 750 implements. 752 Fortunately it is not necessary for us to do such filtering, because 753 that the ESI IPs are installed into an IP-VRF not into GRT. ESI IP 754 in the backbone IP-VRF will not have the security issues that normal 755 VXLAN EVPNs will have. 757 Note that the multicast B-MAC address of BUM packet will be replaced 758 with multicast group IP address in the dataplane. And this requires 759 the control-plane to set up the mVPN infrastructures in the backbone 760 IP-VRF instance. 762 It is a bit weird to use PBB EVPN management plane together with 763 VXLAN over IP-VRF data-plane. So we can use VXLAN over IP-VRF 764 management plane instead. 766 4.3. VXLAN over IP-VRF 768 We configure ESI-IP and VTEP IP instead of B-MAC, VXLAN instance 769 instead of I-VPLS, Backbone IP-VRF instead of B-VPLS, VNI instead of 770 I-SID. 772 The ESI-IP and VTEP IP are advertised by RT-5 routes, not RT-2 773 routes. Although these IPs can be carried in the RT-2 route's IP 774 address field too, it may be a bit weird. So we choose RT-5 route to 775 do the advertisement. 777 Note that the GEI/ES route in VXLAN over IP-VRF will be RT-5 route, 778 And the ESI may be not advertised together with its GEI. 780 We don't use multicast IP address as the underlay destination IP 781 address of the BUM packets. We use the VTEP IP address per each 782 egress PE instead. But the IMET route is not advertised for the 783 backbone IP-VRF instance, they are advertised for the VXLAN instance. 784 We use the Originator Router's IP (ORIP) field of the IMET route as 785 the underlay destination IP address instead of the nexthop. It means 786 that ORIP should be advertised for the backbone IP-VRF instance, and 787 the ORIP should be the VTEP IP address of corresponding PE. 789 There are no other changes from Pseudo PBB-EVPN in data-plane. So it 790 is also a MPLS-based solution. 792 4.4. NVO3-specific EVPN-lite Procedures 794 In Section 3.3, We use VXLAN encapsulation as an example for NVO3 795 EVPN. But when GENEVE or MPLSoGRE encapsulation is used, the ESI- 796 label will have its own space in packet headers, so we don't have to 797 encapsulate ESI-label in UDP Source port. 799 Note that in step #4 the egress GEI is not encapsulated in U1. U1's 800 underlay DIP will be determined by these RT-1 per EVI routes. 802 4.5. MPLS-specific EVPN-lite Procedures 804 According to [RFC7432], When the IMET route's PTA's tunnel type is 805 ingress replication, the ESI-label is considered to be downstream- 806 assigned too. Because that nothing of RT-1 per ES route will 807 indicate whether the ESI-label is upstream-assigned or not. 809 Alghough ESI-GUL can be a single LSE or two LSEs in the Label Stack, 810 we assume that it is a single LSE by default in this section, it is 811 for simplification purpose. 813 [M1] In Step #1, "Format #1" of Figure 6 will be used. 815 Although the Ingress Replication Multicat Label (IRML) of 816 "Format #1" can identify EVI1 by itself, we suppose that the 817 ethernet header of it should also carry EGD as what [M4] does. 819 Note that there isn't a B-VPLS here, so the IRML identifies the 820 EVI1 itself. The EVI1 here equals C-VPLS of PBB EVPN. 822 [M2] In Step #2, "Format #1" of Figure 6 will be received. PE3 823 knows the packet is for EVI1 with the help of the IRML label. 824 Then PE3 can learn the relation between the ingress-GEI 825 (ingress-ESI GUL) and S-MAC of BUM1 directly, no GEI to ESI 826 lookup needed. 828 [M3] In Step #3, PE2 can compare the ingress-GEI (ingress-ESI GUL) 829 of BUM1 and the egress-GEI (ESI-GUL of outgoing AC) directly, 830 no GEI to ESI lookup needed. 832 [M4] In Step #4, "Format #2" of Figure 6 will be used. The source- 833 ESI GUL, from which the corresponding MAC entry M1 is 834 previously learnt, will be encapsulated as the destination-ESI 835 GUL directly. No GEI to ESI lookup needed only if we don't 836 care the requirements of Section 2.9. Otherwise we should 837 refer the corresponding RT-1 per EVI routes of ESI1 to forward 838 the packet. These RT-1 per EVI routes are advertised for EVI1, 839 so the Ethernet Tag ID (ETI) of these routes don't have to be 840 the EGD. 842 Note that when ESI1 is single-active mode, ESI-GUL of ESI1 will 843 be different on PE1 and PE2. But the MAC entry M1 will use the 844 newest one only, the swithover between them is called as MAC- 845 move. 847 [M5] In Step #5, Whe PE1 received the ARP reply packet from PE3, PE1 848 first match the packet to ESI1 by Destination-ESI GUL, then 849 match the packet to the EVI instance EVI1 by the QinQ tags of 850 Ethernet header. 852 Note that we suppose that the original tags from ingress AC 853 will be processed following the Raw mode per [RFC4448]. 854 Although the tagged mode can be used technically. Note that 855 the original tags (if they are kept in the packet) will be the 856 inner tags of the EGD. 858 Note that when RT-1 per EVI route are used, as specified in 859 [M4]. There is no need to carry EGD in unicast data-packets 860 too. 862 4.6. SRv6-specific EVPN-lite Procedures 864 [6A] In Step #1, PE1 will forward the ARP Request to PE2/PE3 with 865 the following SRv6 BE encapsulation: It's underlay Source IP is 866 the End.ESI SID on PE1 for ESI1; It's underlay Destination IP 867 is the End.DT2M SID on PE2/PE3. The locator and function part 868 of the End.ESI SID is GEI1. The Argument part of the End.ESI 869 SID is 0. 871 Note that the underlay SIP will be the End.DT2U SID for the 872 single-homed ingress ACs. The multi-homed ingress ACs with 873 single-active behavior may not be assigned with an ESI- 874 indicator either. In such situations, the underlay SIP will be 875 the End.DT2U SID too. 877 [6B] In Step #3, PE2 can compare the ingress-GEI of BUM1 and the GEI 878 of outgoing AC directly, no GEI to ESI lookup needed. 880 [6C] In Step #4, PE3 will forward the ARP reply to PE1 with the 881 following SRv6 BE encapsulation: It's underlay Source IP is the 882 End.ESI SID on PE3 for ESI2; It's underlay Destination IP is 883 the End.ESI SID on PE1 for ESI1 according to the MAC entry M1. 884 The ARG.EGD for the End.ESI SID in DIP is the EGD configured on 885 PE3. Note that the EGD for the same EVI is configured with the 886 same value on PE1/PE2/PE3. 888 When ESI1 is all-active mode, GEI1 will be the same as GEI2, so 889 we call both of them GEI21 instead. The traffics to M1 will be 890 load-balanced between PE1 and PE2 by the underlay network on 891 PE3. Because GEI21 is advertised by both PE1 and PE2 in the 892 underlay IGP protocol. 894 [6D] In Step #5, Whe PE1 received the SRv6 encapsulated ARP reply 895 packet from PE3, PE1 first match the packet to the End.ESI SID 896 of ESI1 by DIP, then match the packet to the EVI instance EVI1 897 by ARG.EGD. 899 5. Other Considerations 901 5.1. ESI Indicator Advertisement Optimization 903 Although we can advertise End.ESI SID in underlay IGP protocols, But 904 it is better to use the SRv6 SID Structure Sub-Sub-TLV to indicate 905 the length of the ARG.EGD in the End.ESI SID. 907 So we can use EAD/EVI route to advertise Global ESI Indicator (GEI), 908 these EAD/EVI routes is called as GEI/EVI route in this document. 909 When the GEI/EVI route is used to advertise GEI, the End.ESI SID is 910 advertised in its SRv6 L2 Service TLV, not in its nexthop. 912 Either GEI/EVI routes or GEI/ES routes will be advertised/imported 913 for Global Routing Table (GRT), so their Route-Targets (RT) will be 914 configured with GRT. Because there isn't a dedicated B-component 915 like PBB VPLS and PBB EVPN. 917 Although GEIs is imported to GRT, they are awared only on PE nodes, 918 the transit nodes in underlay network won't be aware of GEIs in order 919 to reduce the FIB consumption. We can use the argument length in the 920 SRv6 SID Structure Sub-Sub-TLV to check whether the EGD is too big 921 for the End.ESI SID, So we can avoid the destruction to the function 922 part of the End.ESI and we can use flexible EGD length. 924 5.2. C-MAC Flush Notification Procedure 926 The withdraw of GEI Advertisement can be used as C-MAC flush 927 notification like what have been done by [RFC8317] and 928 [I-D.snr-bess-pbb-evpn-isid-cmacflush]. 930 5.3. E-Tree Support Considerations 932 E-tree Supprot extensions is similar to [RFC8317] section 5 except 933 for the following notable differences: The leaf B-MACs are replaced 934 by leaf GEIs, the root B-MACs are replaced by root GEIs. the PBB 935 encapsulation is replaced by other encapsulations, the B-component is 936 replaced by an IP-VRF or the underlay GRT. The B-MAC Advertisement 937 Route is replaced by GEI/EVI route or ESI/IP Route. 939 5.4. EVPN IRB Support Considerations 941 The dataplane in this draft is no more complex with typical SRv6 942 EVPN. So it will work as efficient as we should expect in SRv6 EVPN 943 IRB usecase. 945 5.5. Use End.ESI SID in MAC/IP Advertisement Routes 947 In [I-D.dawra-bess-srv6-services] the downstream assigned ESI label 948 is encapsulated in the Arg.FE2 part of End.DT2M SID, And the ESI 949 label present as Arg.FE2 only when the egress PE is adjacent with the 950 ingress ESI. So it is difficult (if not impossible) to do data-plane 951 C-MAC learning via End.DT2M SID and its unwarranted Arg.FE2 presence. 952 Alghough upstream assigned ESI label may be used to learn ingress 953 ESI-indicator on egress PE node, other issues will arise together. 955 But the End.ESI SID can be used in MAC/IP advertisement route, only 956 if C-MAC overload is not a real threat. By doing this, the data- 957 plane can be unified among these usecases. The details for using 958 End.ESI SID in MAC/IP Advertisement Route will be described in future 959 versions. 961 5.6. Hierarchical VPLS in EVPN-lite 963 In hierachical topology (as illustrated in the following figure), the 964 PEs are separated into two groups, the Target PEs (TPEs) and the 965 Superstratum PEs (SPEs). 967 ___TPE5___ SPE3 ___TPE4_____ 968 /AC5 \ / \ / \AC4 969 CE3 \ / \ / >=====CE2 970 \___ \ / \ / ____/AC2 971 ___TPE3----SPE1-------SPE2-------TPE2 972 /AC3 / \ 973 CE1 ____/ \ 974 \____TPE1 \___CE6 975 AC1 977 Figure 8: EVPN-lite H-VPLS 979 The TPEs works like the IB-BEB-PE in PBB VPLS, the SPE works like the 980 BCB-PE in PBB VPLS. The BCB-PEs in PBB VPLS do BUM replication based 981 on the PBB header. There are no PBB hearder in EVPN-lite solutions, 982 but the SPEs won't learn the C-MACs, which is the same as BCB-PEs in 983 PBB VPLS. The forwarding behaviors of these EVPN-lite solutions are 984 very different from each other: 986 * The SPE in Pseudo PBB-EVPN do BUM replication based on the 987 Multicast Group IP address. 989 * The SPEs in VXLAN over IP-VRF needn't aware of the BUM packets, 990 because the destination IP address of the BUM packets will be an 991 ingress replication tunnel address to the egress TPE. 993 * The SPEs in MPLS-based EVPN-lite don't have to aware of the BUM 994 packets, because that, for IMET routes, they work like the ASBRs 995 in inter-AS option B. In such case, the TPEs do ingress- 996 replication for all other TPEs by themselves. 998 The SPEs in MPLS-based EVPN-lite may terminate the IMET routes 999 that were received from their TPEs. These IMET routes are 1000 imported into an corresponding BD, but may not be passed through 1001 other SPEs, so as not to cause duplicated BUM packets. In such 1002 case, take SPE1 for example, there are two split-horizon-groups, 1003 one group is TPE1/TPE3/TPE5, another split-horizon-group is SPE1/ 1004 SPE2. The BUM packets are replicated between different split- 1005 horizon-groups. In such case, the TPEs do ingress-replication for 1006 its directly connected TPEs and SPEs, not for the indirectly 1007 connected TPEs and SPEs. But the unicast packet will not be 1008 forwarded by that BD on the SPEs. The unicast packets will be 1009 label-swapped in the context-specific label-space for the 1010 corresponding GULs. 1012 Note that the BCB-PE in PBB VPLS is typically supported in the 1013 industry, But it seems that the BCB-PE in PBB EVPN is typically 1014 not supported in the industry up to now. Because the BCB-PE 1015 function can be replaced in MPLS EVPN by a label-swapping 1016 operation which is like the inter-AS option B scenarios. 1018 Note that the BUM packets here are defined based on the destination 1019 C-MAC addresses. 1021 6. Comparison with Other Solutions 1023 6.1. Questions 1025 We compare EVPN-lite with three other solutions in this section. 1026 These solutions are: 1028 * PBB EVPN - PBB MPLS EVPN per [RFC7623]. 1029 * PBB VPLS - PBB VPLS (PW-based) per [RFC7041]. 1030 * EVPN VXLAN - [RFC7348] plus IMET routes of [RFC8365], the IMET 1031 routes are used to discover EVIs and establish VXLAN tunnels. 1033 We use the following questions for these solutions to do the 1034 comparison: 1036 [CMAC-Reduction] 1037 #1 No C-MAC Awareness in the Backbone ? 1039 [EVPN-IRB] 1040 #2 EVPN IRB Support ? 1042 [Unified-Encap] 1043 #3 Unified Encapsulation per Scenario ? 1045 [ESI-Retained] 1046 #4 ESI Features Remain Supported ? 1048 [Flexible-MH] 1049 #5 Flexible Multi-homing Remains Supported ? 1051 [Learn-Confined] 1052 #6 C-MAC Address Learning and Confinement ? 1054 [AA-no-flush] 1055 #7 No C-MAC Flushing for All-Active ESes ? 1057 [SA-independent] 1058 #8 Independent C-MAC Flushing for Single-Active ESes ? 1060 [ Converge] 1061 #9 Independent Convergency per ? 1063 [Route Aggregation] 1064 #10 Route Aggregation and Default Route in Backbone ? 1066 6.2. Summary Comparisons 1068 We place the detailed comparisons about the answers of these 1069 questions for each solution in separated sections, but we place the 1070 brief comparisons in the following table: 1072 +===================+===========+==========+==========+============+ 1073 | Questions | EVPN-lite | PBB-EVPN | PBB-VPLS | EVPN-VXLAN | 1074 +===================+===========+==========+==========+============+ 1075 | CMAC-Reduction | Yes | Yes | Yes | Yes | 1076 +-------------------+-----------+----------+----------+------------+ 1077 | EVPN-IRB | Yes | No | No | Yes | 1078 +-------------------+-----------+----------+----------+------------+ 1079 | Unified-Encap | Yes | No | No | Yes | 1080 +-------------------+-----------+----------+----------+------------+ 1081 | ESI-Retained | Yes | Yes | No | No | 1082 +-------------------+-----------+----------+----------+------------+ 1083 | Flexible-MH | Yes | Yes | N/A | No | 1084 +-------------------+-----------+----------+----------+------------+ 1085 | Learn-Confined | Yes | Yes | Yes | Yes | 1086 +-------------------+-----------+----------+----------+------------+ 1087 | AA-no-flush | Yes | Yes | No | Yes | 1088 +-------------------+-----------+----------+----------+------------+ 1089 | SA-independent | Yes | Yes | No | No | 1090 +-------------------+-----------+----------+----------+------------+ 1091 | ESI-EVI-Converge | Yes | No | No | No | 1092 +-------------------+-----------+----------+----------+------------+ 1093 | Route-Aggregation | Yes | No | NO | N/A | 1094 +-------------------+-----------+----------+----------+------------+ 1096 Table 2: Solution Comparisons 1098 6.3. Detailed Comparisons with PBB EVPN 1100 TBD. 1102 6.4. Detailed Comparisons with PBB VPLS 1104 TBD. 1106 6.5. Detailed Comparisons with EVPN VXLAN 1108 TBD. 1110 7. Security Considerations 1112 Detailed in Section 4.2. Other considerations will be added in 1113 future versions. 1115 8. IANA Considerations 1117 8.1. END.ESI SID 1119 IANA is requested to allocate a new code points for the new SRv6 1120 Endpoint Behaviors defined in this document. 1122 +------+-------------+---------------+ 1123 | Type | Description | Reference | 1124 +------+-------------+---------------+ 1125 | TBD1 | END.ESI | This Document | 1126 +------+-------------+---------------+ 1127 Figure 9: END.ESI 1129 8.2. Global Unique ESI-label in EAD per ES Route 1131 When we use Global Unique ESI-label in EAD per ES route, especially 1132 in ingress-replication use case, It should be explicitly indicated in 1133 the EAD per ES route. The details will be added in future versions. 1135 9. Acknowledgements 1137 The authors would like to thank the following for their comments and 1138 review of this document: 1140 Ye Shu. 1142 10. References 1144 10.1. Normative References 1146 [I-D.dawra-bess-srv6-services] 1147 Dawra, G., Filsfils, C., Brissette, P., Agrawal, S., 1148 Leddy, J., daniel.voyer@bell.ca, d., 1149 daniel.bernier@bell.ca, d., Steinberg, D., Raszuk, R., 1150 Decraene, B., Matsushima, S., Zhuang, S., and J. Rabadan, 1151 "SRv6 BGP based Overlay services", Work in Progress, 1152 Internet-Draft, draft-dawra-bess-srv6-services-02, 8 July 1153 2019, . 1156 [I-D.ietf-bess-mvpn-evpn-aggregation-label] 1157 Zhang, Z., Rosen, E., Lin, W., Li, Z., and I. Wijnands, 1158 "MVPN/EVPN Tunnel Aggregation with Common Labels", Work in 1159 Progress, Internet-Draft, draft-ietf-bess-mvpn-evpn- 1160 aggregation-label-03, 24 October 2019, 1161 . 1164 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1165 "Encapsulation Methods for Transport of Ethernet over MPLS 1166 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1167 . 1169 [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, 1170 L., Sridhar, T., Bursell, M., and C. Wright, "Virtual 1171 eXtensible Local Area Network (VXLAN): A Framework for 1172 Overlaying Virtualized Layer 2 Networks over Layer 3 1173 Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014, 1174 . 1176 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 1177 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 1178 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 1179 2015, . 1181 [RFC7623] Sajassi, A., Ed., Salam, S., Bitar, N., Isaac, A., and W. 1182 Henderickx, "Provider Backbone Bridging Combined with 1183 Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623, 1184 September 2015, . 1186 [RFC8317] Sajassi, A., Ed., Salam, S., Drake, J., Uttaro, J., 1187 Boutros, S., and J. Rabadan, "Ethernet-Tree (E-Tree) 1188 Support in Ethernet VPN (EVPN) and Provider Backbone 1189 Bridging EVPN (PBB-EVPN)", RFC 8317, DOI 10.17487/RFC8317, 1190 January 2018, . 1192 [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., 1193 Uttaro, J., and W. Henderickx, "A Network Virtualization 1194 Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, 1195 DOI 10.17487/RFC8365, March 2018, 1196 . 1198 10.2. Informative References 1200 [I-D.snr-bess-pbb-evpn-isid-cmacflush] 1201 Rabadan, J., Sathappan, S., Nagaraj, K., Miyake, M., and 1202 T. Matsuda, "PBB-EVPN ISID-based CMAC-Flush", Work in 1203 Progress, Internet-Draft, draft-snr-bess-pbb-evpn-isid- 1204 cmacflush-06, 26 July 2019, . 1207 [RFC7041] Balus, F., Ed., Sajassi, A., Ed., and N. Bitar, Ed., 1208 "Extensions to the Virtual Private LAN Service (VPLS) 1209 Provider Edge (PE) Model for Provider Backbone Bridging", 1210 RFC 7041, DOI 10.17487/RFC7041, November 2013, 1211 . 1213 Author's Address 1215 Yubao Wang 1216 ZTE Corporation 1217 No. 50 Software Ave, Yuhuatai Distinct 1218 Nanjing 1219 China 1221 Email: yubao.wang2008@hotmail.com