idnits 2.17.1 draft-sajassi-bess-evpn-virtual-eth-segment-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 26, 2018) is 2250 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'EVPN-VPWS' is mentioned on line 183, but not defined == Missing Reference: 'EVPN-REQ' is mentioned on line 279, but not defined == Missing Reference: 'ETH-OAM' is mentioned on line 578, but not defined == Missing Reference: 'MPLS-OAM' is mentioned on line 584, but not defined == Missing Reference: 'PW-OAM' is mentioned on line 584, but not defined == Missing Reference: 'RFC 7432' is mentioned on line 616, but not defined == Unused Reference: 'PBB' is defined on line 883, but no explicit reference was found in the text == Unused Reference: 'RFC7209' is defined on line 890, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'PBB' == Outdated reference: A later version (-11) exists of draft-ietf-l2vpn-evpn-07 == Outdated reference: A later version (-10) exists of draft-ietf-l2vpn-pbb-evpn-07 Summary: 0 errors (**), 0 flaws (~~), 11 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Working Group A. Sajassi 3 Internet Draft P. Brissette 4 Category: Standards Track Cisco 5 R. Schell 6 Verizon 7 J. Drake 8 Juniper 9 J. Rabadan 10 Nokia 12 Expires: August 26, 2016 February 26, 2018 14 EVPN Virtual Ethernet Segment 15 draft-sajassi-bess-evpn-virtual-eth-segment-03 17 Status of this Memo 19 This Internet-Draft is submitted to IETF in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF), its areas, and its working groups. Note that 24 other groups may also distribute working documents as 25 Internet-Drafts. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 The list of current Internet-Drafts can be accessed at 33 http://www.ietf.org/1id-abstracts.html 35 The list of Internet-Draft Shadow Directories can be accessed at 36 http://www.ietf.org/shadow.html 38 Copyright and License Notice 40 Copyright (c) 2014 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Abstract 55 EVPN and PBB-EVPN introduce a family of solutions for multipoint 56 Ethernet services over MPLS/IP network with many advanced 57 capabilities among which their multi-homing capabilities. These 58 solutions define two types of multi-homing for an Ethernet Segment 59 (ES): 1) Single-Active and 2) All-Active, where an Ethernet Segment 60 is defined as a set of links between the multi-homed device/network 61 and the set of PE devices that they are connected to. 63 Some Service Providers want to extend the concept of the physical 64 links in an ES to Ethernet Virtual Circuits (EVCs) where many of such 65 EVCs can be aggregated on a single physical External Network-to- 66 Network Interface (ENNI). An ES that consists of a set of EVCs 67 instead of physical links is referred to as a virtual ES (vES). This 68 draft describes the requirements and the extensions needed to support 69 vES in EVPN and PBB-EVPN. 71 Conventions 73 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 74 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 75 document are to be interpreted as described in RFC 2119. 77 Table of Contents 79 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 80 1.1 Virtual Ethernet Segments in Access Ethernet Networks . . . 4 81 1.2 Virtual Ethernet Segments in Access MPLS Networks . . . . . 5 82 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 7 83 3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 8 84 3.1. Single-Homed & Multi-Homed Virtual Ethernet Segments . . . 8 85 3.2. Scalability . . . . . . . . . . . . . . . . . . . . . . . . 8 86 3.3. Local Switching . . . . . . . . . . . . . . . . . . . . . . 9 87 3.4. EVC Service Types . . . . . . . . . . . . . . . . . . . . . 9 88 3.5. Designated Forwarder (DF) Election . . . . . . . . . . . . 10 89 3.6. OAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 90 3.7. Failure & Recovery . . . . . . . . . . . . . . . . . . . . 10 91 3.8. Fast Convergence . . . . . . . . . . . . . . . . . . . . . 11 92 4. Solution Overview . . . . . . . . . . . . . . . . . . . . . . . 11 93 4.1. EVPN DF Election for vES . . . . . . . . . . . . . . . . . 12 94 5. Failure Handling & Recovery . . . . . . . . . . . . . . . . . . 14 95 5.1. Failure Handling for Single-Active vES in EVPN . . . . . . 15 96 5.2. EVC Failure Handling for Single-Active vES in PBB-EVPN . . 15 97 5.3. Port Failure Handling for Single-Active vES's in EVPN . . . 16 98 5.4. Port Failure Handling for Single-Active vES's in PBB-EVPN . 17 99 5.5. Fast Convergence in PBB-EVPN . . . . . . . . . . . . . . . 18 100 6. BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . 20 101 6.1. I-SID Extended Community . . . . . . . . . . . . . . . . . 20 102 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 20 103 8. Security Considerations . . . . . . . . . . . . . . . . . . . . 21 104 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 21 105 10. Intellectual Property Considerations . . . . . . . . . . . . . 21 106 11. Normative References . . . . . . . . . . . . . . . . . . . . . 21 107 12. Informative References . . . . . . . . . . . . . . . . . . . . 21 108 13. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 21 110 1. Introduction 112 [EVPN] and [PBB-EVPN] introduce a family of solutions for multipoint 113 Ethernet services over MPLS/IP network with many advanced 114 capabilities among which their multi-homing capabilities. These 115 solutions define two types of multi-homing for an Ethernet Segment 116 (ES): 1) Single-Active and 2) All-Active, where an Ethernet Segment 117 is defined as a set of links between the multi-homed device/network 118 and the set of PE devices that they are connected to. 120 This document extends the Ethernet Segment concept so that an ES can 121 be associated to a set of EVCs or other objects such as MPLS Label 122 Switch Paths (LSP) or Pseudowires (PW). 124 1.1 Virtual Ethernet Segments in Access Ethernet Networks 126 Some Service Providers (SPs) want to extend the concept of the 127 physical links in an ES to Ethernet Virtual Circuits (EVCs) where 128 many of such EVCs can be aggregated on a single physical External 129 Network-to-Network Interface (ENNI). An ES that consists of a set of 130 EVCs instead of physical links is referred to as a virtual ES (vES). 131 Figure below depicts two PE devices (PE1 and PE2) each with an ENNI 132 where a number of vES's are aggregated on - each of which through its 133 associated EVC. 135 Carrier 136 Ethernet 137 +-----+ Network 138 | CE11|EVC1 +---------+ 139 +-----+ \ | | +---+ 140 Cust. A \-0=========0--ENNI1| | 141 +-----+ | | ENNI1| | +-------+ +---+ 142 | CE12|EVC2--0=========0--ENNI1|PE1|---| | | | 143 +-----+ | | ENNI1| | | |---|PE3|- 144 | ==0--ENNI1| | |IP/MPLS| | | \ +---+ 145 +-----+ | / | +---+ |Network| +---+ \-| | 146 | CE22|EVC3--0==== / | | | |CE4| 147 +-----+ | X | | | +---+ | | 148 | / \ | +---+ | | | | /-| | 149 +-----+ -0=== ===0--ENNI2| | | |---|PE4|-/ +---+ 150 | CE3 |EVC4/ | | ENNI2|PE2|---| | | | 151 | |EVC5--0=========0--ENNI2| | +-------+ +---+ 152 +-----+ | | +---+ 153 Cust. C +---------+ /\ 154 /\ || 155 || ENNI 156 EVCs Interface 157 <--------802.1Q----------> <-802.1Q-> 159 Figure 1: DHD/DHN (both SA/AA) and SH on same ENNI 161 E-NNIs are commonly used to reach off-network / out-of-franchise 162 customer sites via independent Ethernet access networks or third- 163 party Ethernet Access Providers (EAP) (see above figure). E-NNIs can 164 aggregate traffic from hundreds to thousands of vES's; where, each 165 vES is represented by its associated EVC on that ENNI. As a result, 166 ENNIs and their associated EVCs are a key element of SP off-networks 167 that are carefully designed and closely monitored. 169 In order to meet customer's Service Level Agreements (SLA), SPs build 170 redundancy via multiple E-PEs / ENNIs (as shown in figure above) 171 where a given vES can be multi-homed to two or more PE devices (on 172 two or more ENNIs) via their associated EVCs. Just like physical ES's 173 in [EVPN] and [PBB-EVPN] solutions, these vES's can be single-homed 174 or multi-homed ES's and when multi-homed, then can operate in either 175 Single-Active or All-Active redundancy modes. In a typical SP off- 176 network scenario, an ENNI can be associated with several thousands of 177 single-homed vES's, several hundreds of Single-Active vES's and it 178 may also be associated with tens or hundreds of All-Active vES's. 180 1.2 Virtual Ethernet Segments in Access MPLS Networks 181 Other Service Providers (SPs) want to extend the concept of the 182 physical links in an ES to individual Pseudowires (PW) or to MPLS 183 Label Switched Paths (LSPs) per [EVPN-VPWS] in Access MPLS networks. 184 Figure 2 illustrates this concept. 186 MPLS Aggregation 187 Network 188 +-----+ +----------------+ <----EVPN Network-----> 189 | CE11|EVC1 | | 190 +-----+ \+AG1--+ PW1 +-----+ 191 Cust. A -0----|===========| | 192 +-----+ | ---+===========| | +-------+ +---+ 193 | CE12|EVC2-0/ | PW2 /\ | PE1 +---+ | | | 194 +-----+ ++---+ ==||=| | | +---+PE3+- 195 | //=||=| | |IP/MPLS| | | \ +---+ 196 | // \/ ++----+ |Network| +---+ \-+ | 197 +-----+EVC3 | PW3// LSP1 | | | |CE4| 198 | CE13| +AG2--+===/PW4 | | | +---+ | | 199 +-----+ 0 |=== /\ ++----+ | | | | /-+ | 200 0 |==PW5===||=| | | +---+PE4+-/ +---+ 201 +-----+ /++---+==PW6===||=| PE2 +---+ | | | 202 | CE14|EVC4 | \/ | | +-------+ +---+ 203 +-----+ | LSP2+-----+ 204 Cust. C +----------------+ 205 /\ 206 || 207 EVCs 208 <--802.1Q---><------MPLS-------> <-802.1Q-> 210 Figure 2: DHN and SH on Access MPLS networks 212 In some cases, Service Providers use Access MPLS Networks that belong 213 to separate administrative entities or third parties as a way to get 214 access to the their own IP/MPLS network infrastructure. This is the 215 case illustrated in Figure 2. 217 An ES is defined as a set of individual PWs if they cannot be 218 aggregated into a common LSP. If the aggregation of PWs is possible, 219 the ES can be associated to an LSP in a given PE. In the example of 220 Figure 2, EVC3 is connected to a VPWS instance in AG2 that is 221 connected to PE1 and PE2 via PW3 and PW5 respectively. EVC4 is 222 connected to a separate VPWS instance on AG2 that gets connected to 223 an EVI on PE1 and PE2 via PW4 and PW6, respectively. Since the PWs 224 for the two VPWS instances can be aggregated into the same LSPs going 225 to the EVPN network, a common virtual ES can be defined for LSP1 and 226 LSP2. This ES will be shared by two separate EVIs in the EVPN 227 network. 229 In some cases, this aggregation of PWs into common LSPs may not be 230 possible. For instance, if PW3 were terminated into a third PE, e.g. 231 PE3, instead of PE1, the ES would need to be defined on a per 232 individual PW on each PE, i.e. PW3 and PW5 would belong to ES-1, 233 whereas PW4 and PW6 would be associated to ES-2. 235 An ES that consists of a set of LSPs or individual PWs is also 236 referred as virtual ES (vES) in this document." 238 This draft describes requirements and the extensions needed to 239 support vES in [EVPN] and [PBB-EVPN]. Section 3 lists the set of 240 requirements for Virtual ES's. Section 4 describes the solution for 241 [PBB-EVPN] to meet these requirements. Section 5 describes the 242 failure handling and recovery for Virtual ES's in [PBB-EVPN]. Section 243 6 covers scalability and fast convergence required for Virtual ES's 244 in [PBB-EVPN]. 246 2. Terminology 248 AC: Attachment Circuit 249 BEB: Backbone Edge Bridge 250 B-MAC: Backbone MAC Address 251 CE: Customer Edge 252 CFM: Connectivity Fault Management 253 C-MAC: Customer/Client MAC Address 254 DHD: Dual-homed Device 255 DHN: Dual-homed Network 256 ENNI: External Network-Network Interface 257 ES: Ethernet Segment 258 ESI: Ethernet-Segment Identifier 259 EVC: Ethernet Virtual Circuit 260 EVPN: Ethernet VPN 261 LACP: Link Aggregation Control Protocol 262 PE: Provider Edge 263 SH: Single-Homed 265 Single-Active Redundancy Mode (SA): When only a single PE, among a 266 group of PEs attached to an Ethernet-Segment, is allowed to forward 267 traffic to/from that Ethernet Segment, then the Ethernet segment is 268 defined to be operating in Single-Active redundancy mode. 270 All-Active Redundancy Mode (AA): When all PEs attached to an Ethernet 271 segment are allowed to forward traffic to/from that Ethernet-Segment, 272 then the Ethernet segment is defined to be operating in All-Active 273 redundancy mode. 275 3. Requirements 277 This section describes the requirements specific to virtual Ethernet 278 Segment (vES) for (PBB-)EVPN solutions. These requirements are in 279 addition to the ones described in [EVPN-REQ], [EVPN], and [PBB-EVPN]. 281 3.1. Single-Homed & Multi-Homed Virtual Ethernet Segments 283 A PE needs to support the following types of vES's: 285 (R1a) A PE MUST handle single-homed vES's on a single physical port 286 (e.g., single ENNI) 288 (R1b) A PE MUST handle a mix of Single-Homed vES's and Single-Active 289 multi-homed vES's simultaneously on a single physical port (e.g., 290 single ENNI). Single-Active multi-homed vES's will be simply referred 291 to as Single-Active vES's through the rest of this document. 293 (R1c) A PE MAY handle All-Active multi-homed vES's on a single 294 physical port. All-Active multi-homed vES's will be simply referred 295 to as All-Active vES's through the rest of this document. 297 (R1d) A PE MAY handle a mixed of All-Active vES's along with other 298 types of vES's on a single physical port 300 (R1e) A Multi-Homed vES (Single-Active or All-Active) can be spread 301 across any two or more PEs (on two or more ENNIs) 303 3.2. Scalability 305 A single physical port (e.g., ENNI) can be associated with many 306 vES's. The following requirements give a quantitative measure for 307 each vES type. 309 (R2a) A PE MUST handle thousands or tens of thousands of Single-homed 310 vES's on a single physical port (e.g., single ENNI) 312 (R2b) A PE MUST handle hundreds of Single-Active vES's on a single 313 physical port (e.g., single ENNI) 315 (R2c) A PE MAY handle tens or hundreds of All-Active Multi-Homed 316 vES's on a single physical port (e.g., single ENNI) 318 (R2d) A PE MUST handle the above scale for a mix of Single-homed 319 vES's and Single-Active vES's simultaneously on a single physical 320 port (e.g., single ENNI) 322 (R4e) A PE MAY handle the above sale for a mixed of All-Active Multi- 323 Homed vES's along with other types of vES's on a single physical port 325 3.3. Local Switching 327 Many vES's of different types can be aggregated on a single physical 328 port on a PE device and some of these vES can belong to the same 329 service instance (or customer). This translates into the need for 330 supporting local switching among the vES's of the same service 331 instance on the same physical port (e.g., ENNI) of the PE. 333 (R3a) A PE MUST support local switching among different vES's 334 belonging to the same service instance (or customer) on a single 335 physical port. For example, in the above figure (1), PE1 MUST 336 support local switching between CE11 and CE12 (both belonging to 337 customer A) that are mapped to two Single-homed vES's on ENNI1. 339 In case of Single-Active vES's, the local switching is performed 340 among active EVCs belonging to the same service instance on the same 341 ENNI. 343 3.4. EVC Service Types 345 A physical port (e.g., ENNI) of a PE can aggregate many EVCs each of 346 which is associated with a vES. Furthermore, an EVC may carry one or 347 more VLANs. Typically, an EVC carries a single VLAN and thus it is 348 associated with a single broadcast domain. However, there is no 349 restriction on an EVC to carry more than one VLANs. 351 (R4a) An EVC can be associated with a single broadcast domain - e.g., 352 VLAN-based service or VLAN bundle service 354 (R4b) An EVC MAY be associated with several broadcast domains - e.g., 355 VLAN-aware bundle service 357 In the same way, a PE can aggregated many LSPs and PWs. In the case 358 of individual PWs per vES, typically a PW is associated with a single 359 broadcast domain, but there is no restriction on the PW to carry more 360 than one VLAN if the PW is defined as vc-type VLAN. 362 (R4c) A PW can be associated with a single broadcast domain - e.g., 363 VLAN-based service or VLAN bundle service. 365 (R4b) An PW MAY be associated with several broadcast domains - e.g., 366 VLAN-aware bundle service." 368 3.5. Designated Forwarder (DF) Election 370 Section 8.5 of [EVPN] describes the default procedure for DF election 371 in EVPN which is also used in [PBB-EVPN]. This default DF election 372 procedure is performed at the granularity of . In case of a 373 vES, the same EVPN default procedure for DF election also applies; 374 however, at the granularity of ; where vESI is the virtual 375 Ethernet Segment Identifier. As in [EVPN], this default procedure for 376 DF election at the granularity of is also referred to as 377 "service carving"; where, EVI is represented by an I-SID in PBB-EVPN 378 and by a EVI service-id/vpn-id in EVPN. With service carving, it is 379 possible to evenly distribute the DFs for different vES's among 380 different PEs, thus distributing the traffic among different PEs. The 381 following list the requirements apply to DF election of vES's for 382 EVPN. 384 (R5a) A vES with m EVCs can be distributed among n ENNIs belonging to 385 p PEs in any arbitrary oder; where n >= P >= m. For example, if there 386 is an vES with 2 EVCs and there are 5 ENNIs on 5 PEs (PE1 through 387 PE5), then vES can be dual-homed to PE2 and PE4 and the DF election 388 must be performed between PE2 and PE4. 390 (R5b) Each vES MUST be identified by its own virtual ESI (vESI) 392 3.6. OAM 394 In order to detect the failure of individual EVC and perform DF 395 election for its associated vES as the result of this failure, each 396 EVC should be monitored independently. 398 (R6a) Each EVC SHOULD be monitored for its health independently 400 (R6b) A single EVC failure (among many aggregated on a single 401 physical port/ENNI) MUST trigger DF election for its associated vES. 403 3.7. Failure & Recovery 405 (R7a) Failure and failure recovery of an EVC for a Single-homed vES 406 SHALL NOT impact any other EVCs for its own service instance or any 407 other service instances. In other words, for PBB-EVPN, it SHALL NOT 408 trigger any MAC flushing both within its own I-SID as well as other 409 I-SIDs. 411 (R7b) In case of All-Active Multi-Homed vES, failure and failure 412 recovery of an EVC for that vES SHALL NOT impact any other EVCs for 413 its own service instance or any other service instances. In other 414 words, for PBB-EVPN, it SHALL NOT trigger any MAC flushing both 415 within its own I-SID as well as other I-SIDs. 417 (R7c) Failure & failure recovery of an EVC for a Single-Active vES 418 SHALL only impact its own service instance. In other words, for PBB- 419 EVPN, MAC flushing SHALL be limited to the associated I-SID only and 420 SHALL NOT impact any other I-SIDs. 422 (R7d) Failure & failure recovery of an EVC for a Single-Active vES 423 MAY only impact C-MACs associated with MHD/MHNs for that service 424 instance. In other words, MAC flushing SHOULD be limited to single 425 service instance (I-SID in the case of PBB-EVPN) and only CMACs for 426 Single-Active MHD/MHNs. 428 3.8. Fast Convergence 430 Since large number of EVCs (and their associated vES's) are 431 aggregated via a single physical port (e.g., ENNI), then the failure 432 of that physical port impacts large number of vES's and triggers 433 large number of ES route withdrawals. Formulating, sending, 434 receiving, and processing such large number of BGP messages can 435 introduce delay in DF election and convergence time. As such, it is 436 highly desirable to have a mass-withdraw mechanism similar to the one 437 in the [EVPN] for withdrawing large number of Ethernet A-D routes. 439 (R8a) There SHOULD be a mechanism equivalent to EVPN mass-withdraw 440 such that upon an ENNI failure, only a single BGP message is needed 441 to indicate to the remote PEs to trigger DF election for all impacted 442 vES associated with that ENNI. 444 4. Solution Overview 446 The solutions described in [EVPN] and [PBB-EVPN] are leveraged as is 447 with one simple modification and that is the ESI assignment is 448 performed for a group of EVCs instead of a group of links. In other 449 words, the ESI is associated with a virtual ES (vES) and that's why 450 it will be referred to as vESI. 452 For EVPN solution, everything basically remains the same except for 453 the handling of physical port failure where many vES's can be 454 impacted. Section 5.1 and 5.3 below describe the handling of physical 455 port/link failure for EVPN. In a typical multi-homed operation, MAC 456 addresses are learned behind a vES are advertised with the ESI 457 corresponding to the vES (i.e., vESI). EVPN aliasing and mass- 458 withdraw operations are performed with respect to vES. In other 459 words, the Ethernet A-D routes for these operations are advertised 460 with vESI instead of ESI. 462 For PBB-EVPN solution, the main change is with respect to the BMAC 463 address assignment which is performed similar to what is described in 464 section 7.2.1.1 of [PBB-EVPN] with the following refinements: 466 - One shared BMAC address is used per PE for the single-homed vES's. 467 In other words, a single BMAC is shared for all single-homed vES's on 468 that PE. 470 - One shared BMAC address should be used per PE per physical port 471 (e.g., ENNI) for the Single-Active vES's. In other words, a single 472 BMAC is shared for all Single-Active vES's that shared the same ENNI. 474 - One shared BMAC address can be used for all Single-Active vES's on 475 that PE. 477 - One BMAC address is used per EVC per physical port per PE for each 478 All-Active multi-homed vES. In other words, a single BMAC address is 479 used per vES for All-Active multi-homing scenarios. 481 - A single BMAC address may also be used per vES per PE for Single- 482 Active multi-homing scenarios. 484 BEB +--------------+ BEB 485 || | | || 486 \/ | | \/ 487 +----+ EVC1 +----+ | | +----+ +----+ 488 | CE1|------| | | | | |---| CE2| 489 +----+\ | PE1| | IP/MPLS | | PE3| +----+ 490 \ +----+ | Network | +----+ 491 \ | | 492 EVC2\ +----+ | | 493 \ | | | | 494 \| PE2| | | 495 +----+ | | 496 /\ +--------------+ 497 || 498 BEB 499 <--802.1Q---> <---PBB over MPLS---> <--802.1Q-> 501 Figure 2: PBB-EVPN Network 503 4.1. EVPN DF Election for vES 504 The procedure for service carving for virtual Ethernet Segments is 505 the same as the one outlined in section 8.5 of [EVPN] except for the 506 fact that ES is replaced with vES. For the sake of clarity and 507 completeness, this procedure is repeated below: 509 1. When a PE discovers the ESI or is configured with the ESI 510 associated with its attached vES, it advertises an Ethernet Segment 511 route with the associated ES-Import extended community attribute. 513 2. The PE then starts a timer (default value = 3 seconds) to allow 514 the reception of Ethernet Segment routes from other PE nodes 515 connected to the same vES. This timer value MUST be same across all 516 PEs connected to the same vES. 518 3. When the timer expires, each PE builds an ordered list of the IP 519 addresses of all the PE nodes connected to the vES (including 520 itself), in increasing numeric value. Each IP address in this list is 521 extracted from the "Originator Router's IP address" field of the 522 advertised Ethernet Segment route. Every PE is then given an ordinal 523 indicating its position in the ordered list, starting with 0 as the 524 ordinal for the PE with the numerically lowest IP address. The 525 ordinals are used to determine which PE node will be the DF for a 526 given EVPN instance on the vES using the following rule: Assuming a 527 redundancy group of N PE nodes, the PE with ordinal i is the DF for 528 an EVPN instance with an associated EVI ID value of V when (V mod N) 529 = i. 531 It should be noted that using "Originator Router's IP address" field 532 in the Ethernet Segment route to get the PE IP address needed for the 533 ordered list, allows for a CE to be multi-homed across different ASes 534 if such need ever arises. 536 4. The PE that is elected as a DF for a given EVPN instance will 537 unblock traffic for that EVPN instance. Note that the DF PE unblocks 538 all traffic in both ingress and egress directions for Single-Active 539 vES and unblocks multi-destination in egress direction for All-Active 540 Multi-homed vES. All non-DF PEs block all traffic in both ingress and 541 egress directions for Single-Active vES and block multi-destination 542 traffic in the egress direction for All-Active multi-homed vES. 544 In the case of an EVC failure, the affected PE withdraws its Ethernet 545 Segment route. This will re-trigger the service carving procedures on 546 all the PEs in the RG. For PE node failure, or upon PE commissioning 547 or decommissioning, the PEs re-trigger the service carving across all 548 affected vES's. In case of a Single-Active multi-homing, when a 549 service moves from one PE in the RG to another PE as a result of re- 550 carving, the PE, which ends up being the elected DF for the service, 551 SHOULD trigger a MAC address flush notification towards the 552 associated vES. This can be done, for e.g. using IEEE 802.1ak MVRP 553 'new' declaration. 555 For LSP and PW based vES, the non-DF PE SHOULD signal PW-status 556 'standby' signaling to the AG PE, and the new DF MAY send an LDP MAC 557 withdraw message as a MAC address flush notification. 559 5. Failure Handling & Recovery 561 There are a number of failure scenarios to consider such as: 563 A: CE Uplink Port Failure 564 B: Ethernet Access Network Failure 565 C: PE Access-facing Port or link Failure 566 D: PE Node Failure 567 E: PE isolation from IP/MPLS network 569 [EVPN] and [PBB-EVPN] solutions provide protection against such 570 failures as described in the corresponding references. In the 571 presence of virtual Ethernet Segments (vES's) in these solutions, 572 besides the above failure scenarios, there is one more scenario to 573 consider and that is EVC failure. This implies that individual EVCs 574 need to be monitored and upon their failure detection, appropriate DF 575 election procedures and failure recovery mechanism need to be 576 executed. 578 [ETH-OAM] is used for monitoring EVCs and upon failure detection of a 579 given EVC, DF election procedure per section [4.1] is executed. For 580 PBB-EVPN, some addition extensions are needed to failure handling and 581 recovery procedures of [PBB-EVPN] in order to meet the above 582 requirements. These extensions are describe in the next section. 584 [MPLS-OAM] and [PW-OAM] are used for monitoring the status of LSPs 585 and/or PWs associated to vES. 587 B D 588 || || 589 \/ \/ 590 +-----+ 591 +-----+ | | +---+ 592 | CE1 |EVC2--0=====0--ENNI1| | +-------+ 593 +-----+ | =0--ENNI1|PE1|---| | +---+ +---+ 594 Cust. A | / | | | |IP/MPLS|--|PE3|--|CE4| 595 +-----+ | / | +---+ |Network| | | +---+ 596 | |EVC2--0== | | | +---+ 597 | CE2 | | | +---+ | | 598 | |EVC3--0=====0--ENNI2|PE2|---| | 599 +-----+ | | | | +-------+ 600 +-----+ +---+ 601 /\ /\ /\ 602 || || || 603 A C E 605 Figure 3: Failure Scenarios A,B,C,D and E 607 5.1. Failure Handling for Single-Active vES in EVPN 609 When a PE connected to a Single-Active multi-homed Ethernet Segment 610 loses connectivity to the segment, due to link or port failure, it 611 signals the remote PE to flush all CMAC addresses associated with 612 that Ethernet Segment. This is done by advertising a mass-withdraw 613 message using Ethernet A-D per-ES route. To be precise, there is no 614 MAC flush per-se if there is only one backup PE for a given ES - 615 i.e., only an update of the forwarding entries per backup-path 616 procedure in [RFC 7432]. 618 In case of an EVC failure that impacts a single vES, the exact same 619 EVPN procedure is used. In this case, the message using Ethernet A-D 620 per ES route carries the vESI representing the vES which is in turn 621 associated with the failed EVC. The remote PEs upon receiving this 622 message perform the same procedures outlined in section 8.2 of 623 [EVPN]. 625 5.2. EVC Failure Handling for Single-Active vES in PBB-EVPN 627 When a PE connected to a Single-Active multi-homed Ethernet Segment 628 loses connectivity to the segment, due to link or port failure, it 629 signals the remote PE to flush all CMAC addresses associated with 630 that Ethernet Segment. This is done by advertising a BMAC route along 631 with MAC Mobility Extended community. 633 In case of an EVC failure that impacts a single vES, if the above 634 PBB-EVPN procedure is used, it results in excessive CMAC flushing 635 because a single physical port can support large number of EVCs (and 636 their associated vES's) and thus advertising a BMAC corresponding to 637 the physical port with MAC mobility Extended community will result in 638 flushing CMAC addresses not just for the impacted EVC but for all 639 other EVCs on that port. 641 In order to reduce the scope of CMAC flushing to only the impacted 642 service instances (the service instance(s) impacted by the EVC 643 failure), the BGP flush message is sent along with a list of impacted 644 I-SID(s) represented by the new EVPN I-SID Extended Community as 645 defined in section 6. Since typically an EVC maps to a single 646 broadcast domain and thus a single service instance, the list only 647 contains a single I-SID. However, if the failed EVC carries multiple 648 VLANs each with its own broadcast domain, then the list contains 649 several I-SIDs - one for each broadcast domain. This new BGP flush 650 message basically instructs the remote PE to perform flushing for 651 CMACs corresponding to the advertised BMAC only across the advertised 652 list of I-ISIDs (which is typically one). 654 The above BMAC route that is advertised with the MAC Mobility 655 Extended Community, can either represent the MAC address of the 656 physical port that the failed EVC is associated with, or it can 657 represent the MAC address of the PE. In the latter case, this is the 658 dedicated MAC address used for all Single-Active vES's on that PE. 659 The former one performs better than the latter one in terms of 660 reducing the scope of flushing as described below and thus it is the 661 recommended approach. 663 Advertising the BMAC route that represent the physical port (e.g., 664 ENNI) on which the failed EVC reside along with MAC Mobility and I- 665 SID extended communities provide the most optimum mechanism for CMAC 666 flushing upon EVC failure in PBB-EVPN for Single-Active vES because: 668 1) Only CMAC addresses for the impacted service instances are 669 flushed. 671 2) Only a subset of CMAC addresses for the impacted service 672 instances are flushed - only the ones that are learned over the BMAC 673 associated with the failed EVC. In other words, only a small fraction 674 of the CMACs for the impacted service instance(s) are flushed. 676 5.3. Port Failure Handling for Single-Active vES's in EVPN 678 When a large number of EVCs are aggregated via a single physical port 679 on a PE; where each EVC corresponds to a vES, then the port failure 680 impacts all the associated EVCs and their corresponding vES's. If the 681 number of EVCs corresponding to the Single-Active vES's for that 682 physical port is in thousands, then thousands of service instances 683 are impacted. Therefore, the BGP flush message need to be inclusive 684 of all these impacted service instances. In order to achieve this, 685 the following extensions are added to the baseline EVPN mechanism: 687 1) A PE when advertises an Ether-AD per ES route for a given vES, it 688 colors it with the MAC address of the physical port which is 689 associated with that vES. The receiving PEs take note of this color 690 and create a list of vES's for this color. 692 2) Upon a port failure (e.g., ENNI failure), the PE advertise a 693 special mass-withdraw message with the MAC address of the failed port 694 (i.e., the color of the port) encoded in the ESI field. For this 695 encoding, type 3 ESI is used with the MAC field set to the MAC 696 address of the port and the 3-octet local discriminator field set to 697 0xFFFFFF. This mass-withdraw route is advertised with a list of Route 698 Targets corresponding to the impacted service instances. If the 699 number of Route Targets is more than they can fit into a single 700 attribute, then a set of Ethernet A-D per ES routes are advertised. 701 The remote PEs upon receiving this message, realize that this is a 702 special mass-withdraw message and they access the list of the vES's 703 for the specified color. Next, they initiate mass-withdraw procedure 704 for each of the vES's in the list. 706 5.4. Port Failure Handling for Single-Active vES's in PBB-EVPN 708 When a large number of EVCs are aggregated via a single physical port 709 on a PE; where each EVC corresponds to a vES, then the port failure 710 impacts all the associated EVCs and their corresponding vES's. If the 711 number of EVCs corresponding to the Single-Active vES's for that 712 physical port is in thousands, then thousands of service instances 713 (I-SIDs) are impacted. Therefore, the BGP flush message need to be 714 sent with a list of thousands of I-SIDs. The new I-SID Extended 715 Community provides a way to encode upto 24 I-SIDs in each Extended 716 Community if the impacted I-SIDs are sequential (the base I-SID value 717 plus the next 23 I-SID values). So, the packing efficiency can range 718 from 1 to 24 and there can be up to 400 such Extended Community sent 719 along with a BGP flush message for a total of 400 to 9600 I-SIDs. If 720 the number of I-SIDs is large enough to not fit in a single 721 Attribute, then either a number of BGP flush messages (with different 722 RDs) can be transmitted or a single BGP flush message without the I- 723 SID list can be transmitted. If the BGP flush message is transmitted 724 without the I-SID list, then it instructs the receiving PEs to flush 725 CMACs associated with that BMAC across all I-SIDs. For simplicity, we 726 opt for the latter option in this document. In other words, if the 727 number of impacted I-SIDs exceed that of a single BGP flush message, 728 then the flush message is sent without the I-SID list. 730 As also described in [PBB-EVPN], there are two ways to signal flush 731 message upon a physical port failure: 733 1) If the MAC address of the physical port is used for PBB 734 encapsulation as BMAC SA, then upon the port failure, the PE MUST use 735 the EVPN MAC route withdrawal message to signal the flush 737 2) If the PE shared MAC address is used for PBB encapsulation as BMAC 738 SA, then upon the port failure, the PE MUST re-advertise this MAC 739 route with the MAC Mobility Extended Community to signal the flush 741 The first method is recommended because it reduces the scope of 742 flushing the most. 744 5.5. Fast Convergence in PBB-EVPN 746 As described above, when a large number of EVCs are aggregated via a 747 physical port on a PE; where each EVC corresponds to a vES, then the 748 port failure impacts all the associated EVCs and their corresponding 749 vES's. Two actions must be taken as the result of such port failure: 751 - Flushing of all CMACs associated with the BMAC of the failed port 752 for the impacted I-SIDs 754 - DF election for all impacted vES's associated with the failed port 756 Section 5.4 describes how to flush CMAC address in the most optimum 757 way - e.g., to flush least number of CMAC addresses for the impacted 758 I-SIDs. This section describes how to perform DF election in the most 759 optimum way - e.g., to trigger DF election for all impacted vES's 760 (which can be in thousands) among the participating PEs via a single 761 BGP message as opposed to sending thousands of BGP messages - one per 762 vES. 764 In order to devise such fast convergence mechanism that can be 765 triggered via a single BGP message, all vES's associated with a given 766 physical port (e.g., ENNI) are colored with the same color 767 representing that physical port. The MAC address of the physical port 768 is used for this coloring purposes and when the PE advertises an ES 769 route for a vES associated with that physical port, it advertises it 770 with an EVPN MAC Extended Community indicating the color of that 771 port. 773 The receiving PEs take note of this color and for each such color, 774 they create a list of vES's associated with this color (with this MAC 775 address). Now, when a port failure occurs, the impacted PE needs to 776 notify the other PEs of this color so that these PEs can identify all 777 the impacted vES's associated with that color (from the above list) 778 and re-execute DF election procedures for all the impacted vES's. 780 In PBB-EVPN, there are two ways to convey this color to other PEs 781 upon a port failure - one corresponding to each method for signaling 782 flush message as described in section 5.4. If for PBB encapsulation, 783 the MAC address of the physical port is used as BMAC SA, then upon 784 the port failure, the PE sends MAC withdrawal message with the MAC 785 address of the failed port as the color. However, if for PBB 786 encapsulation, the shared MAC address of the PE (dedicated for all 787 Single-Active vES's) is used as BMAC SA, then upon the port failure, 788 the PE re-advertises the MAC route (that carries the shared BMAC) 789 along with this new EVPN MAC Extended Community to indicate the color 790 along with MAC Mobility Extended Community. 792 +-----+ 793 +----+ | | +---+ 794 | CE1|AC1--0=====0--ENNI1| | +-------+ 795 | |AC2--0 | |PE1|--| | 796 +----+ |\ ==0--ENNI2| | | | 797 | \/ | +---+ | | 798 | /\ | |IP/MPLS| 799 +----+ |/ \ | +---+ |Network| +---+ +---+ 800 | CE2|AC4--0 =0--ENNI3| | | |---|PE4|--|CE4| 801 | |AC4--0=====0--ENNI3|PE2|--| | +---+ +---+ 802 +----+ | ====0--ENNI3| | | | 803 |/ | +---+ | | 804 0 | | | 805 +----+ /| | +---+ | | 806 | CE3|AC5- | | |PE3|--| | 807 | |AC6--0=====0--ENNI4| | +-------+ 808 +----+ | | +---+ 809 +-----+ 811 Figure 4: Fast Convergence Upon ENNI Failure 813 The following describes the procedure for coloring vES's and fast 814 convergence using this color in more details: 816 1- When a vES is configured, the PE colors the vES with the MAC 817 address of the corresponding physical port and advertises the 818 Ethernet Segment route for this vES with this color. 820 2- All other PEs (in the redundancy group) take note of this color 821 and add the vES to the list for this color. 823 3- Upon the occurrence of a port failure (e.g., an ENNI failure), the 824 PE sends the flush message in one of the two ways described above 825 indicating this color. 827 4- On reception of the flush message, other PEs use this info to 828 flush their impacted CMACs and to initiate DF election procedures 829 across all their affected vES's. 831 5- The PE with the physical port failure (ENNI failure), also send ES 832 route withdrawal for every impacted vES's. The other PEs upon 833 receiving these messages, clear up their BGP tables. It should be 834 noted the ES route withdrawal messages are not used for executing DF 835 election procedures by the receiving PEs. 837 6. BGP Encoding 839 This document defines one new BGP Extended Community for EVPN. 841 6.1. I-SID Extended Community 843 A new EVPN BGP Extended Community called I-SID is introduced. This 844 new extended community is a transitive extended community with the 845 Type field of 0x06 (EVPN) and the Sub-Type of 0x04. 847 The I-SID Extended Community is encoded as an 8-octet value as 848 follows: 850 0 1 2 3 851 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 853 | Type=0x06 | Sub-Type=0x03 | Base I-SID | 854 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 855 | Cont. | Bit Map (24 bits) | 856 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 858 This extended community is used to indicate the list of I-SIDs 859 associated with a given Ethernet Segment. 861 24-bit map represents the next 24 I-SID after the base I-SID. For 862 example based I-SID of 10025 with 24-bit map of zero means, only a 863 single I-SID of 10025. I-SID of 10025 with bit map of 0x000001 means 864 there are two I-SIDs, 10025 and 10026. 866 7. Acknowledgements 867 TBD 869 8. Security Considerations This document does not introduce any 870 additional security constraints. 872 9. IANA Considerations 874 TBD 876 10. Intellectual Property Considerations 878 This document is being submitted for use in IETF standards 879 discussions. 881 11. Normative References 883 [PBB] Clauses 25 and 26 of "IEEE Standard for Local and metropolitan 884 area networks - Media Access Control (MAC) Bridges and 885 Virtual Bridged Local Area Networks", IEEE Std 802.1Q, 886 2013. 888 12. Informative References 890 [RFC7209] Sajassi, et al., "Requirements for Ethernet VPN (EVPN)", 891 RFC7209, May 2014. 893 [EVPN] Sajassi, et al., "BGP MPLS Based Ethernet VPN", draft-ietf- 894 l2vpn-evpn-07.txt, work in progress, May 7, 2014. 896 [PBB-EVPN] Sajassi, et al., "PBB-EVPN", draft-ietf-l2vpn-pbb-evpn- 897 07.txt, work in progress, June 18, 2014. 899 13. Authors' Addresses 901 Ali Sajassi 902 Cisco Systems 903 Email: sajassi@cisco.com 905 Patrice Brissette 906 Cisco Systems 907 Email: pbrisset@cisco.com 909 Rick Schell 910 Verizon 911 Email: richard.schell@verizon.com 912 John E Drake 913 Juniper 914 Email: jdrake@juniper.net 916 Tapraj Singh 917 Juniper 918 Email: tsingh@juniper.net 920 Jorge Rabadan 921 ALU 922 Email: jorge.rabadan@alcatel-lucent.com