idnits 2.17.1 draft-ietf-bess-evpn-df-election-framework-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 4, 2018) is 1941 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'HRW1999' == Outdated reference: A later version (-05) exists of draft-ietf-bess-vpls-multihoming-02 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Workgroup J. Rabadan, Ed. 3 Internet Draft Nokia 4 S. Mohanty, Ed. 5 Intended status: Standards Track A. Sajassi 6 Cisco 7 J. Drake 8 Juniper 9 K. Nagaraj 10 S. Sathappan 11 Nokia 13 Expires: June 7, 2019 December 4, 2018 15 Framework for EVPN Designated Forwarder Election Extensibility 16 draft-ietf-bess-evpn-df-election-framework-06 18 Abstract 20 The Designated Forwarder (DF) in EVPN networks is the Provider Edge 21 (PE) router responsible for sending broadcast, unknown unicast and 22 multicast (BUM) traffic to a multi-homed Customer Equipment (CE) 23 device, on a given VLAN on a particular Ethernet Segment (ES). The DF 24 is selected out of a list of candidate PEs that advertise the same 25 Ethernet Segment Identifier (ESI) to the EVPN network. By default, 26 EVPN uses a DF Election algorithm referred to as "Service Carving" 27 and it is based on a modulus function (V mod N) that takes the number 28 of PEs in the ES (N) and the VLAN value (V) as input. This default DF 29 Election algorithm has some inefficiencies that this document 30 addresses by defining a new DF Election algorithm and a capability to 31 influence the DF Election result for a VLAN, depending on the state 32 of the associated Attachment Circuit (AC). In addition, this document 33 creates a registry with IANA, for future DF Election Algorithms and 34 Capabilities. It also presents a formal definition and clarification 35 of the DF Election Finite State Machine. 37 Status of this Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF), its areas, and its working groups. Note that 44 other groups may also distribute working documents as Internet- 45 Drafts. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 The list of current Internet-Drafts can be accessed at 53 http://www.ietf.org/ietf/1id-abstracts.txt 55 The list of Internet-Draft Shadow Directories can be accessed at 56 http://www.ietf.org/shadow.html 58 This Internet-Draft will expire on June 7, 2019. 60 Copyright Notice 62 Copyright (c) 2018 IETF Trust and the persons identified as the 63 document authors. All rights reserved. 65 This document is subject to BCP 78 and the IETF Trust's Legal 66 Provisions Relating to IETF Documents 67 (http://trustee.ietf.org/license-info) in effect on the date of 68 publication of this document. Please review these documents 69 carefully, as they describe your rights and restrictions with respect 70 to this document. Code Components extracted from this document must 71 include Simplified BSD License text as described in Section 4.e of 72 the Trust Legal Provisions and are provided without warranty as 73 described in the Simplified BSD License. 75 Table of Contents 77 1. Conventions and Terminology . . . . . . . . . . . . . . . . . . 3 78 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 79 2.1. Default Designated Forwarder (DF) Election in EVPN . . . . 4 80 2.2. Problem Statement . . . . . . . . . . . . . . . . . . . . . 5 81 2.2.1. Unfair Load-Balancing and Service Disruption . . . . . 6 82 2.2.2. Traffic Black-Holing on Individual AC Failures . . . . 7 83 2.3. The Need for Extending the Default DF Election in EVPN . . 9 84 3. Designated Forwarder Election Protocol and BGP Extensions . . . 10 85 3.1 The DF Election Finite State Machine (FSM) . . . . . . . . . 10 86 3.2 The DF Election Extended Community . . . . . . . . . . . . . 13 87 3.3 Auto-Derivation of ES-Import Route Target . . . . . . . . . 15 88 4. The Highest Random Weight DF Election Type . . . . . . . . . . 15 89 4.1. HRW and Consistent Hashing . . . . . . . . . . . . . . . . 16 90 4.2. HRW Algorithm for EVPN DF Election . . . . . . . . . . . . 16 91 5. The Attachment Circuit Influenced DF Election Capability . . . 17 92 5.1. AC-Influenced DF Election Capability For VLAN-Aware 93 Bundle Services . . . . . . . . . . . . . . . . . . . . . . 19 94 6. Solution Benefits . . . . . . . . . . . . . . . . . . . . . . . 20 95 7. Security Considerations . . . . . . . . . . . . . . . . . . . . 21 96 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 21 97 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 98 9.1. Normative References . . . . . . . . . . . . . . . . . . . 21 99 9.2. Informative References . . . . . . . . . . . . . . . . . . 22 100 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 101 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 23 102 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 104 1. Conventions and Terminology 106 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 107 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 108 "OPTIONAL" in this document are to be interpreted as described in BCP 109 14 [RFC2119] [RFC8174] when, and only when, they appear in all 110 capitals, as shown here. 112 o AC and ACS - Attachment Circuit and Attachment Circuit Status. An 113 AC has an Ethernet Tag associated to it. 115 o BUM - refers to the Broadcast, Unknown unicast and Multicast 116 traffic. 118 o DF, NDF and BDF - Designated Forwarder, Non-Designated Forwarder 119 and Backup Designated Forwarder 121 o Ethernet A-D per ES route - refers to [RFC7432] route type 1 or 122 Auto-Discovery per Ethernet Segment route. 124 o Ethernet A-D per EVI route - refers to [RFC7432] route type 1 or 125 Auto-Discovery per EVPN Instance route. 127 o ES and ESI - Ethernet Segment and Ethernet Segment Identifier. 129 o EVI - EVPN Instance. 131 o BD - Broadcast Domain. An EVI may be comprised of one (VLAN-Based 132 or VLAN-Bundle services) or multiple (VLAN-Aware Bundle services) 133 Broadcast Domains. 135 o HRW - Highest Random Weight 137 o VID and CE-VID - VLAN Identifier and Customer Equipment VLAN 138 Identifier. 140 o Ethernet Tag - used to represent a Broadcast Domain that is 141 configured on a given ES for the purpose of DF election. Note that 142 any of the following may be used to represent a Broadcast Domain: 143 VIDs (including double Q-in-Q tags), configured IDs, VNI, 144 normalized VID, I-SIDs, etc., as long as the representation of the 145 broadcast domains is configured consistently across the multi-homed 146 PEs attached to that ES. The Ethernet Tag value MUST be different 147 from zero. 149 o Ethernet Tag ID - refers to the identifier used in the EVPN routes 150 defined in [RFC7432]. Its value may be the same as the Ethernet Tag 151 value (see Ethernet Tag definition) when advertising routes for 152 VLAN-aware bundle services. Note that in case of VLAN-based or VLAN 153 Bundle services, the Ethernet Tag ID is zero. 155 o DF Election Procedure and DF Algorithm - The Designated Forwarder 156 Election Procedure or simply DF Election, refers to the process in 157 its entirety, including the discovery of the PEs in the ES, the 158 creation and maintenance of the PE candidate list and the selection 159 of a PE. The Designated Forwarder Algorithm is just a component of 160 the DF Election Procedure and strictly refers to the selection of a 161 PE for a given . 163 This document also assumes familiarity with the terminology of 164 [RFC7432]. 166 2. Introduction 168 2.1. Default Designated Forwarder (DF) Election in EVPN 170 [RFC7432] defines the Designated Forwarder (DF) as the EVPN PE 171 responsible for: 173 o Flooding Broadcast, Unknown unicast and Multicast traffic (BUM), on 174 a given Ethernet Tag on a particular Ethernet Segment (ES), to the 175 CE. This is valid for single-active and all-active EVPN 176 multi-homing. 178 o Sending unicast traffic on a given Ethernet Tag on a particular ES 179 to the CE. This is valid for single-active multi-homing. 181 Figure 1 illustrates an example that we will use to explain the 182 Designated Forwarder function. 184 +---------------+ 185 | IP/MPLS | 186 | CORE | 187 +----+ ES1 +----+ +----+ 188 | CE1|-----| |-----------| |____ES2 189 +----+ | PE1| | PE2| \ 190 | |-------- +----+ \+----+ 191 +----+ | | | CE2| 192 | | +----+ /+----+ 193 | |__| |____/ | 194 | | PE3| ES2 / 195 | +----+ / 196 | | / 197 +-------------+----+ / 198 | PE4|____/ES2 199 | | 200 +----+ 202 Figure 1 Multi-homing Network of EVPN 204 Figure 1 illustrates a case where there are two Ethernet Segments, 205 ES1 and ES2. PE1 is attached to CE1 via Ethernet Segment ES1 whereas 206 PE2, PE3 and PE4 are attached to CE2 via ES2 i.e. PE2, PE3 and PE4 207 form a redundancy group. Since CE2 is multi-homed to different PEs on 208 the same Ethernet Segment, it is necessary for PE2, PE3 and PE4 to 209 agree on a DF to satisfy the above mentioned requirements. 211 Layer-2 devices are particularly susceptible to forwarding loops 212 because of the broadcast nature of the Ethernet traffic. Therefore it 213 is very important that, in case of multi-homing, only one of the 214 links be used to direct traffic to/from the core. 216 One of the pre-requisites for this support is that participating PEs 217 must agree amongst themselves as to who would act as the Designated 218 Forwarder (DF). This needs to be achieved through a distributed 219 algorithm in which each participating PE independently and 220 unambiguously selects one of the participating PEs as the DF, and the 221 result should be consistent and unanimous. 223 The default algorithm for DF election defined by [RFC7432] at the 224 granularity of (ESI,EVI) is referred to as "service carving". In this 225 document, service carving or default DF Election algorithm are used 226 interchangeably. With service carving, it is possible to elect 227 multiple DFs per Ethernet Segment (one per EVI) in order to perform 228 load-balancing of traffic destined to a given Segment. The objective 229 is that the load-balancing procedures should carve up the BD space 230 among the redundant PE nodes evenly, in such a way that every PE is 231 the DF for a distinct set of EVIs. 233 The DF Election algorithm as described in [RFC7432] (Section 8.5) is 234 based on a modulus operation. The PEs to which the ES (for which DF 235 election is to be carried out per EVI) is multi-homed form an ordered 236 (ordinal) list in ascending order of the PE IP address values. For 237 example, there are N PEs: PE0, PE1,... PEN-1 ranked as per increasing 238 IP addresses in the ordinal list; then for each VLAN with Ethernet 239 Tag V, configured on the Ethernet Segment ES1, PEx is the DF for VLAN 240 V on ES1 when x equals (V mod N). In the case of VLAN-Bundle only the 241 lowest VLAN is used. In the case when the planned density is high 242 (meaning there are significant number of VLANs and the Ethernet Tags 243 are uniformly distributed), the thinking is that the DF Election will 244 be spread across the PEs hosting that Ethernet Segment and good load- 245 balancing can be achieved. 247 However, the described default DF Election algorithm has some 248 undesirable properties and in some cases can be somewhat disruptive 249 and unfair. This document describes some of those issues and proposes 250 a mechanism for dealing with them. These mechanisms do involve 251 changes to the default DF Election algorithm, but they do not require 252 any changes to the EVPN Route exchange and have minimal changes to 253 their content per se. 255 In addition, there is a need to extend the DF Election procedures so 256 that new algorithms and capabilities are possible. A single algorithm 257 (the default DF Election algorithm) may not meet the requirements in 258 all the use-cases. 260 Note that while [RFC7432] elects a DF per , this document 261 elects a DF per . This means that unlike [RFC7432], where for 262 a VLAN Aware Bundle service EVI there is only one DF for the EVI, 263 this document specifies that there will be multiple DFs, one for each 264 BD configured in that EVI. 266 2.2. Problem Statement 268 This section describes some potential issues with the default DF 269 Election algorithm. 271 2.2.1. Unfair Load-Balancing and Service Disruption 273 There are three fundamental problems with the current default DF 274 Election algorithm. 276 1- First, the algorithm will not perform well when the Ethernet Tag 277 follows a non-uniform distribution, for instance when the Ethernet 278 Tags are all even or all odd. In such a case let us assume that 279 the ES is multi-homed to two PEs; one of the PEs will be elected 280 as DF for all of the VLANs. This is very sub-optimal. It defeats 281 the purpose of service carving as the DFs are not really evenly 282 spread across. In fact, in this particular case, one of the PEs 283 does not get elected as DF at all, so it does not participate in 284 the DF responsibilities at all. Consider another example where, 285 referring to Figure 1, lets assume that PE2, PE3, PE4 are in 286 ascending order of the IP address; and each VLAN configured on ES2 287 is associated with an Ethernet Tag of the form (3x+1), where x is 288 an integer. This will result in PE3 always be selected as the DF. 290 2- Even in the case when the Ethernet Tag distribution is uniform the 291 instance of a PE being up or down results in re-computation ((v 292 mod N-1) or (v mod N+1) as is the case); the resulting modulus 293 value need not be uniformly distributed because it can be subject 294 to the primality of N-1 or N+1 as may be the case. 296 3- The third problem is one of disruption. Consider a case when the 297 same Ethernet Segment is multi homed to a set of PEs. When the ES 298 is down in one of the PEs, say PE1, or PE1 itself reboots, or the 299 BGP process goes down or the connectivity between PE1 and an RR 300 goes down, the effective number of PEs in the system now becomes 301 N-1, and DFs are computed for all the VLANs that are configured on 302 that Ethernet Segment. In general, if the DF for a VLAN v happens 303 not to be PE1, but some other PE, say PE2, it is likely that some 304 other PE will become the new DF. This is not desirable. Similarly 305 when a new PE hosts the same Ethernet Segment, the mapping again 306 changes because of the modulus operation. This results in needless 307 churn. Again referring to Figure 1, say v1, v2 and v3 are VLANs 308 configured on ES2 with associated Ethernet Tags of value 999, 1000 309 and 1001 respectively. So PE1, PE2 and PE3 are the DFs for v1, v2 310 and v3 respectively. Now when PE3 goes down, PE2 will become the 311 DF for v1 and PE1 will become the DF for v2. 313 One point to note is that the default DF election algorithm assumes 314 that all the PEs who are multi-homed to the same Ethernet Segment 315 (and interested in the DF Election by exchanging EVPN routes) use an 316 Originating Router's IP Address of the same family. This does not 317 need to be the case as the EVPN address-family can be carried over a 318 v4 or v6 peering, and the PEs attached to the same ES may use an 319 address of either family. 321 Mathematically, a conventional hash function maps a key k to a number 322 i representing one of m hash buckets through a function h(k) i.e. 323 i=h(k). In the EVPN case, h is simply a modulo-m hash function viz. 324 h(v) = v mod N, where N is the number of PEs that are multi-homed to 325 the Ethernet Segment in discussion. It is well-known that for good 326 hash distribution using the modulus operation, the modulus N should 327 be a prime-number not too close to a power of 2 [CLRS2009]. When the 328 effective number of PEs changes from N to N-1 (or vice versa); all 329 the objects (VLAN V) will be remapped except those for which V mod N 330 and V mod (N-1) refer to the same PE in the previous and subsequent 331 ordinal rankings respectively. From a forwarding perspective, this is 332 a churn, as it results in re-programming the PE ports as either 333 blocking or non-blocking at potentially all PEs when the DF changes. 335 This document addresses this problem and furnishes a solution to this 336 undesirable behavior. 338 2.2.2. Traffic Black-Holing on Individual AC Failures 340 As discussed in section 2.1 the default DF Election algorithm defined 341 by [RFC7432] takes into account only two variables in the modulus 342 function for a given ES: the existence of the PE's IP address on the 343 candidate list and the locally provisioned Ethernet Tags. 345 If the DF for an fails (due to physical link/node 346 failures) an ES route withdrawal will make the Non-DF (NDF) PEs re- 347 elect the DF for that and the service will be recovered. 349 However, the default DF election procedure does not provide a 350 protection against "logical" failures or human errors that may occur 351 at service level on the DF, while the list of active PEs for a given 352 ES does not change. These failures may have an impact not only on the 353 local PE where the issue happens, but also on the rest of the PEs of 354 the ES. Some examples of such logical failures are listed below: 356 a) A given individual Attachment Circuit (AC) defined in an ES is 357 accidentally shutdown or even not provisioned yet (hence the 358 Attachment Circuit Status - ACS - is DOWN), while the ES is 359 operationally active (since the ES route is active). 361 b) A given MAC-VRF - with a defined ES - is shutdown or not 362 provisioned yet, while the ES is operationally active (since the 363 ES route is active). In this case, the ACS of all the ACs defined 364 in that MAC-VRF is considered to be DOWN. 366 Neither (a) nor (b) will trigger the DF re-election on the remote 367 multi-homed PEs for a given ES since the ACS is not taken into 368 account in the DF election procedures. While the ACS is used as a DF 369 election tie-breaker and trigger in VPLS multi-homing procedures 370 [VPLS-MH], there is no procedure defined in EVPN [RFC7432] to trigger 371 the DF re-election based on the ACS change on the DF. 373 Figure 2 illustrates the described issue with an example. 375 +---+ 376 |CE4| 377 +---+ 378 | 379 PE4 | 380 +-----+-----+ 381 +---------------| +-----+ |---------------+ 382 | | | BD-1| | | 383 | +-----------+ | 384 | | 385 | EVPN | 386 | | 387 | PE1 PE2 PE3 | 388 | (NDF) (DF) (NDF)| 389 +-----------+ +-----------+ +-----------+ 390 | | BD-1| | | | BD-1| | | | BD-1| | 391 | +-----+ |-------| +-----+ |-------| +-----+ | 392 +-----------+ +-----------+ +-----------+ 393 AC1\ ES12 /AC2 AC3\ ES23 /AC4 394 \ / \ / 395 \ / \ / 396 +----+ +----+ 397 |CE12| |CE23| 398 +----+ +----+ 400 Figure 2 Default DF Election and Traffic Black-Holing 402 BD-1 is defined in PE1, PE2, PE3 and PE4. CE12 is a multi-homed CE 403 connected to ES12 in PE1 and PE2. Similarly CE23 is multi-homed to 404 PE2 and PE3 using ES23. Both, CE12 and CE23, are connected to BD-1 405 through VLAN-based service interfaces: CE12-VID 1 (VLAN ID 1 on CE12) 406 is associated to AC1 and AC2 in BD-1, whereas CE23-VID 1 is 407 associated to AC3 and AC4 in BD-1. Assume that, although not 408 represented, there are other ACs defined on these ES mapped to 409 different BDs. 411 After running the [RFC7432] default DF election algorithm, PE2 turns 412 out to be the DF for ES12 and ES23 in BD-1. The following issues may 413 arise: 415 a) If AC2 is accidentally shutdown or even not configured, CE12 416 traffic will be impacted. In case of all-active multi-homing, the 417 BUM traffic to CE12 will be "black-holed", whereas for single- 418 active multi-homing, all the traffic to/from CE12 will be 419 discarded. This is due to the fact that a logical failure in PE2's 420 AC2 may not trigger an ES route withdrawn for ES12 (since there 421 are still other ACs active on ES12) and therefore PE1 will not re- 422 run the DF election procedures. 424 b) If the Bridge Table for BD-1 is administratively shutdown or even 425 not configured yet on PE2, CE12 and CE23 will both be impacted: 426 BUM traffic to both CEs will be discarded in case of all-active 427 multi-homing and all traffic will be discarded to/from the CEs in 428 case of single-active multi-homing. This is due to the fact that 429 PE1 and PE3 will not re-run the DF election procedures and will 430 keep assuming PE2 is the DF. 432 Quoting [RFC7432], "when an Ethernet Tag is decommissioned on an 433 Ethernet Segment, then the PE MUST withdraw the Ethernet A-D per EVI 434 route(s) announced for the that are impacted by 435 the decommissioning", however, while this A-D per EVI route 436 withdrawal is used at the remote PEs performing aliasing or backup 437 procedures, it is not used to influence the DF election for the 438 affected EVIs. 440 This document adds an optional modification of the DF Election 441 procedure so that the ACS may be taken into account as a variable in 442 the DF election, and therefore EVPN can provide protection against 443 logical failures. 445 2.3. The Need for Extending the Default DF Election in EVPN 447 Section 2.2 describes some of the issues that exist in the default DF 448 Election procedures. In order to address those issues, this document 449 introduces a new DF Election framework. This framework allows the PEs 450 to agree on a common DF election algorithm, as well as the 451 capabilities to enable during the DF Election procedure Generally, 452 'DF election algorithm' refers to the algorithm by which a number of 453 input parameters are used to determine the DF PE, while 'DF election 454 capability' refers to an additional feature that can be used prior to 455 the invocation of the DF election algorithm, such as modifying the 456 inputs (or list of candidate PEs). 458 Within this framework, this document defines a new DF Election 459 algorithm and a new capability that can influence the DF Election 460 result: 462 o The new DF Election algorithm is referred to as "Highest Random 463 Weight" (HRW). The HRW procedures are described in section 4. 465 o The new DF Election capability is referred to as "AC-Influenced DF 466 Election" (AC-DF). The AC-DF procedures are described in section 5. 468 o HRW and AC-DF mechanisms are independent of each other. Therefore, 469 a PE MAY support either HRW or AC-DF independently or MAY support 470 both of them together. A PE MAY also support AC-DF capability along 471 with the default DF election algorithm per [RFC7432]. 473 In addition, this document defines a way to indicate the support of 474 HRW and/or AC-DF along with the EVPN ES routes advertised for a given 475 ES. Refer to section 3.2 for more details. 477 3. Designated Forwarder Election Protocol and BGP Extensions 479 This section describes the BGP extensions required to support the new 480 DF Election procedures. In addition, since the specification in EVPN 481 [RFC7432] does leave several questions open as to the precise final 482 state machine behavior of the DF election, section 3.1 describes 483 precisely the intended behavior. 485 3.1 The DF Election Finite State Machine (FSM) 487 Per [RFC7432], the FSM described in Figure 3 is executed per 488 in case of VLAN-based service or in case of VLAN-Bundle on each participating PE. 491 Observe that currently the VLANs are derived from local configuration 492 and the FSM does not provide any protection against misconfiguration 493 where the same (EVI,ESI) combination has different set of VLANs on 494 different participating PEs or one of the PEs elects to consider 495 VLANs as VLAN-Bundle and another as separate VLANs for election 496 purposes (service type mismatch). 498 The FSM is conceptual and any design or implementation MUST comply 499 with a behavior equivalent to the one outlined in this FSM. 501 VLAN_CHANGE 502 VLAN_CHANGE RCVD_ES 503 RCVD_ES LOST_ES 504 LOST_ES +----+ 505 +----+ | v 506 | | ++----++ 507 | +-+----+ ES_UP | DF | 508 +->+ INIT +---------------> WAIT | 509 ++-----+ +----+-+ 510 ^ | 511 +-----------+ | |DF_TIMER 512 | ANY STATE +-------+ VLAN_CHANGE | 513 +-----------+ ES_DOWN +-----------------+ | 514 | RCVD_ES v v 515 +-----++ LOST_ES ++---+-+ 516 | DF | | DF | 517 | DONE +<--------------+ CALC +<-+ 518 +------+ CALCULATED +----+-+ | 519 | | 520 +----+ 521 VLAN_CHANGE 522 RCVD_ES 523 LOST_ES 525 Figure 3 DF Election Finite State Machine 527 States: 529 1. INIT: Initial State 531 2. DF WAIT: State in which the participant waits for enough 532 information to perform the DF election for the EVI/ESI/VLAN 533 combination. 535 3. DF CALC: State in which the new DF is recomputed. 537 4. DF DONE: State in which the according DF for the EVI/ESI/VLAN 538 combination has been elected. 540 Events: 542 1. ES_UP: The ESI has been locally configured as 'up'. 544 2. ES_DOWN: The ESI has been locally configured as 'down'. 546 3. VLAN_CHANGE: The VLANs configured in a bundle (that uses the ESI) 547 changed. This event is necessary for VLAN-Bundles only. 549 4. DF_TIMER: DF Wait timer has expired. 551 5. RCVD_ES: A new or changed Ethernet Segment Route is received in a 552 BGP REACH UPDATE. Receiving an unchanged UPDATE MUST NOT trigger 553 this event. 555 6. LOST_ES: A BGP UNREACH UPDATE for a previously received Ethernet 556 Segment route has been received. If an UNREACH is seen for a 557 route that has not been advertised previously, the event MUST NOT 558 be triggered. 560 7. CALCULATED: DF has been successfully calculated. 562 According actions when transitions are performed or states 563 entered/exited: 565 1. ANY STATE on ES_DOWN: (i) stop DF timer (ii) assume non-DF for 566 local PE. 568 2. INIT on ES_UP: transition to DF_WAIT. 570 3. INIT on VLAN_CHANGE, RCVD_ES, LOST_ES: do nothing. 572 4. DF_WAIT on entering the state: (i) start DF timer if not started 573 already or expired (ii) assume non-DF for local PE. 575 5. DF_WAIT on VLAN_CHANGE, RCVD_ES, LOST_ES: do nothing. 577 6. DF_WAIT on DF_TIMER: transition to DF_CALC. 579 7. DF_CALC on entering or re-entering the state: (i) rebuild 580 candidate list, hash and perform election (ii) Afterwards FSM 581 generates CALCULATED event against itself. 583 8. DF_CALC on VLAN_CHANGE, RCVD_ES, LOST_ES: do nothing. 585 9. DF_CALC on CALCULATED: mark election result for VLAN or bundle, 586 and transition to DF_DONE. 588 11. DF_DONE on exiting the state: if there is a new DF election 589 triggered and the current DF is lost, then assume non-DF for 590 local PE for VLAN or VLAN-Bundle. 592 12. DF_DONE on VLAN_CHANGE, RCVD_ES or LOST_ES: transition to 593 DF_CALC. 595 3.2 The DF Election Extended Community 597 For the DF election procedures to be consistent and unanimous, it is 598 necessary that all the participating PEs agree on the DF Election 599 algorithm and capabilities to be used. For instance, it is not 600 possible that some PEs continue to use the default DF Election 601 algorithm and some PEs use HRW. For brown-field deployments and for 602 interoperability with legacy PEs, it is important that all PEs need 603 to have the capability to fall back on the Default DF Election. A PE 604 can indicate its willingness to support HRW and/or AC-DF by signaling 605 a DF Election Extended Community along with the Ethernet Segment 606 Route (Type-4). 608 The DF Election Extended Community is a new BGP transitive extended 609 community attribute [RFC4360] that is defined to identify the DF 610 election procedure to be used for the Ethernet Segment. Figure 4 611 shows the encoding of the DF Election Extended Community. 613 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 614 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 615 | Type=0x06 | Sub-Type(0x06)| RSV | DF Alg | Bitmap | 616 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 617 | Bitmap | Reserved | 618 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 620 Figure 4 DF Election Extended Community 622 Where: 624 o Type is 0x06 as registered with IANA for EVPN Extended Communities. 626 o Sub-Type is 0x06 - "DF Election Extended Community" as requested by 627 this document to IANA. 629 o RSV - Reserved bits for future use. 631 o DF Alg (5 bits) - Encodes the DF Election algorithm values (between 632 0 and 31) that the advertising PE desires to use for the ES. This 633 document requests IANA to set up a registry called "DF Alg 634 Registry" and solicits the following values: 636 - Type 0: Default DF Election algorithm, or modulus-based algorithm 637 as in [RFC7432]. 639 - Type 1: HRW algorithm (explained in this document). 641 - Types 2-30: Unassigned. 643 - Type 31: Reserved for Experimental Use. 645 o Bitmap (2 octets) - Encodes "capabilities" to use with the DF 646 Election algorithm in the field "DF Alg". This document requests 647 IANA to create a registry for the Bitmap field, with values 0-15, 648 called "DF Election Capabilities" and solicits the following 649 values: 651 1 1 1 1 1 1 652 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 653 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 654 | |A| | 655 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 657 Figure 5 Bitmap field in the DF Election Extended Community 659 - Bit 0 (corresponds to Bit 24 of the DF Election Extended 660 Community): Unassigned. 662 - Bit 1: AC-DF (AC-Influenced DF Election, explained in this 663 document). When set to 1, it indicates the desire to use AC- 664 Influenced DF Election with the rest of the PEs in the ES. 666 - Bits 2-15: Unassigned. 668 The DF Election Extended Community is used as follows: 670 o A PE SHOULD attach the DF Election Extended Community to any 671 advertised ES route and the Extended Community MUST be sent if the 672 ES is locally configured with a DF election algorithm other than 673 the Default Election algorithm or if a capability is required to be 674 used. In the Extended Community, the PE indicates the desired "DF 675 Alg" algorithm and "Bitmap" capabilities to be used for the ES. 677 - Only one DF Election Extended Community can be sent along with an 678 ES route. Note that the intent is not for the advertising PE to 679 indicate all the supported DF election algorithms and 680 capabilities, but signal the preferred one. 682 - DF Algs 0 and 1 can be both used with bit AC-DF set to 0 or 1. 684 - In general, a specific DF Alg MAY determine the use of the 685 reserved bits in the Extended Community, which may be used in a 686 different way for a different DF Alg. 688 o When a PE receives the ES Routes from all the other PEs for the ES 689 in question, it checks to see if all the advertisements have the 690 extended community with the same DF Alg and Bitmap: 692 - In the case that they do, this particular PE MUST follow the 693 procedures for the advertised DF Alg and capabilities. For 694 instance, if all ES routes for a given ES indicate DF Alg HRW and 695 AC-DF set to 1, the receiving PE and by induction all the other 696 PEs in the ES will proceed to do DF Election as per the HRW 697 Algorithm and following the AC-DF procedures. 699 - Otherwise if even a single advertisement for the type-4 route is 700 not received with the locally configured DF Alg and capability, 701 the default DF Election algorithm (modulus) algorithm MUST be 702 used as in [RFC7432]. 704 - The absence of the DF Election Extended Community MUST be 705 interpreted by a receiving PE as an indication of the default DF 706 Election algorithm on the sending PE, that is, DF Alg 0 and no DF 707 Election capabilities. 709 o When all the PEs in an ES advertise DF Type 31, they will rely on 710 the local policy to decide how to proceed with the DF Election. 712 o For any new capability defined in the future, the 713 applicability/compatibility of this new capability to the existing 714 DF Algs must be assessed on a case by case basis. 716 o Likewise, for any new DF Alg defined in future, its 717 applicability/compatibility to the existing capabilities must be 718 assessed on a case by case basis. 720 3.3 Auto-Derivation of ES-Import Route Target 722 Section 7.6 of [RFC7432] describes how the value of the ES-Import 723 Route Target for ESI types 1, 2, and 3 can be auto-derived by using 724 the high-order six bytes of the nine byte ESI value. The same auto- 725 derivation procedure can be extended to ESI types 0, 4, and 5 as long 726 as it is ensured that the auto-derived values for ES-Import RT among 727 different ES types don't overlap. 729 4. The Highest Random Weight DF Election Algorithm 731 The procedure discussed in this section is applicable to the DF 732 Election in EVPN Services [RFC7432] and EVPN Virtual Private Wire 733 Services [RFC8214]. 735 Highest Random Weight (HRW) as defined in [HRW1999] is originally 736 proposed in the context of Internet Caching and proxy Server load 737 balancing. Given an object name and a set of servers, HRW maps a 738 request to a server using the object-name (object-id) and server-name 739 (server-id) rather than the state of the server states. HRW forms a 740 hash out of the server-id and the object-id and forms an ordered list 741 of the servers for the particular object-id. The server for which the 742 hash value is highest, serves as the primary responsible for that 743 particular object, and the server with the next highest value in that 744 hash serves as the backup server. HRW always maps a given object name 745 to the same server within a given cluster; consequently it can be 746 used at client sites to achieve global consensus on object-server 747 mappings. When that server goes down, the backup server becomes the 748 responsible designate. 750 Choosing an appropriate hash function that is statistically oblivious 751 to the key distribution and imparts a good uniform distribution of 752 the hash output is an important aspect of the algorithm. Fortunately 753 many such hash functions exist. [HRW1999] provides pseudo-random 754 functions based on Unix utilities rand and srand and easily 755 constructed XOR functions that perform considerably well. This 756 imparts very good properties in the load balancing context. Also each 757 server independently and unambiguously arrives at the primary server 758 selection. HRW already finds use in multicast and ECMP [RFC2991], 759 [RFC2992]. 761 4.1. HRW and Consistent Hashing 763 HRW is not the only algorithm that addresses the object to server 764 mapping problem with goals of fair load distribution, redundancy and 765 fast access. There is another family of algorithms that also 766 addresses this problem; these fall under the umbrella of the 767 Consistent Hashing Algorithms [CHASH]. These will not be considered 768 here. 770 4.2. HRW Algorithm for EVPN DF Election 772 This section describes the application of HRW to DF election. Let 773 DF(v) denote the Designated Forwarder and BDF(v) the Backup 774 Designated forwarder for the Ethernet Tag v, where v is the VLAN, Si 775 is the IP address of server i, Es denotes the Ethernet Segment 776 Identifier and weight is a function of v, Si, and Es. 778 Note that while the DF election algorithm in [RFC7432] uses PE 779 address and vlan as inputs, this document uses Ethernet Tag, PE 780 address and ESI as inputs. This is because if the same set of PEs are 781 multi-homed to the same set of ESes, then the DF election algorithm 782 used in [RFC7432] would result in the same PE being elected DF for 783 the same set of broadcast domains on each ES, which can have adverse 784 side-effects on both load balancing and redundancy. Including ESI in 785 the DF election algorithm introduces additional entropy which 786 significantly reduces the probability of the same PE being elected DF 787 for the same set of broadcast domains on each ES. Therefore, the ESI 788 value in the Weight function below SHOULD be set to that of 789 corresponding ES. The ESI value MAY be set to all 0's in the Weight 790 function below if the operator chooses so. 792 In case of a VLAN-Bundle service, v denotes the lowest VLAN similar 793 to the 'lowest VLAN in bundle' logic of [RFC7432]. 795 1. DF(v) = Si: Weight(v, Es, Si) >= Weight(v, Es, Sj), for all j. In 796 case of a tie, choose the PE whose IP address is numerically the 797 least. Note 0 <= i,j <= Number of PEs in the redundancy group. 799 2. BDF(v) = Sk: Weight(v, Es, Si) >= Weight(V, Es, Sk) and Weight(v, 800 Es, Sk) >= Weight(v, Es, Sj). In case of tie choose the PE whose 801 IP address is numerically the least. 803 Since the Weight is a Pseudo-random function with domain as the 804 three-tuple (v, Es, S), it is an efficient deterministic algorithm 805 that is independent of the Ethernet Tag v sample space distribution. 806 Choosing a good hash function for the pseudo-random function is an 807 important consideration for this algorithm to perform better than the 808 default algorithm. As mentioned previously, such functions are 809 described in the HRW paper. We take as candidate hash functions two 810 of the ones that are preferred in [HRW1999]. 812 1. Wrand(v, Es, Si) = (1103515245((1103515245.Si+12345)XOR 813 D(v,Es))+12345)(mod 2^31) and 815 2. Wrand2(v, Es, Si) = (1103515245((1103515245.D(v,Es)+12345)XOR 816 Si)+12345)(mod 2^31) 818 Here D(v,Es) is the 31-bit digest (CRC-32 and discarding the MSB as 819 in [HRW1999]) of the 14-byte stream, the Ethernet Tag v (4 bytes) 820 followed by the Ethernet Segment Identifier (10 bytes). It is 821 mandated that the 14-byte stream is formed by concatenation of the 822 Ethernet tag and the Ethernet Segment identifier in network byte 823 order. The CRC should proceed as if the stream is in network byte 824 order (big-endian). Si is address of the ith server. The server's IP 825 address length does not matter as only the low-order 31 bits are 826 modulo significant. Although both the above hash functions perform 827 similarly, we select the first hash function (1) of choice, as the 828 hash function has to be the same in all the PEs participating in the 829 DF election. 831 A point to note is that the Weight function takes into consideration 832 the combination of the Ethernet Tag, Ethernet Segment and the PE IP- 833 address, and the actual length of the server IP address (whether V4 834 or V6) is not really relevant. The default algorithm in [RFC7432] 835 cannot employ both V4 and V6 PE addresses, since [RFC7432] does not 836 specify how to decide on the ordering (the ordinal list) when both V4 837 and V6 PEs are present. 839 HRW solves the disadvantage pointed out in Section 2.2.1 and ensures: 841 o with very high probability that the task of DF election for the 842 VLANs configured on an ES is more or less equally distributed among 843 the PEs even for the 2 PE case. 845 o If a PE that is not the DF or the BDF for that VLAN, goes down or 846 its connection to the ES goes down, it does not result in a DF or 847 BDF reassignment. This saves computation, especially in the case 848 when the connection flaps. 850 o More importantly it avoids the needless disruption case of Section 851 2.2.1 (3), that is inherent in the existing default DF Election. 853 o In addition to the DF, the algorithm also furnishes the BDF, which 854 would be the DF if the current DF fails. 856 5. The Attachment Circuit Influenced DF Election Capability 858 The procedure discussed in this section is applicable to the DF 859 Election in EVPN Services [RFC7432] and EVPN Virtual Private Wire 860 Services [RFC8214]. 862 The AC-DF capability MAY be used with any "DF Alg" algorithm. It MUST 863 modify the DF Election procedures by removing from consideration any 864 candidate PE in the ES that cannot forward traffic on the AC that 865 belongs to the BD. This section is applicable to VLAN-Based and VLAN- 866 Bundle service interfaces. Section 5.1 describes the procedures for 867 VLAN-Aware Bundle interfaces. 869 In particular, when used with the default DF Alg, the AC-DF 870 capability modifies the Step 3 in the DF Election procedure described 871 in [RFC7432] Section 8.5, as follows: 873 3. When the timer expires, each PE builds an ordered "candidate" list 874 of the IP addresses of all the PE nodes attached to the Ethernet 875 Segment (including itself), in increasing numeric value. The 876 candidate list is based on the Originator Router's IP addresses of 877 the ES routes, but excludes any PE from whom no Ethernet A-D per 878 ES route has been received, or from whom the route has been 879 withdrawn. Afterwards, the DF Election algorithm is applied on a 880 per or , however, the IP address for a 881 PE will not be considered candidate for a given or 882 until the corresponding Ethernet A-D per EVI 883 route has been received from that PE. In other words, the ACS on 884 the ES for a given PE must be UP so that the PE is considered as 885 candidate for a given BD. 887 The above paragraph differs from [RFC7432] Section 8.5, Step 3, in 888 two aspects: 890 o Any DF Alg algorithm can be used, and not only the modulus-based 891 one (which is the default DF Election, or DF Alg 0 in this 892 document). 894 o The candidate list is pruned based upon non-receipt of Ethernet A-D 895 routes: a PE's IP address MUST be removed from the ES candidate 896 list if its Ethernet A-D per ES route is withdrawn. A PE's IP 897 address MUST NOT be considered as candidate DF for a or 898 , if its Ethernet A-D per EVI route for the 899 or respectively, is withdrawn. 901 The following example illustrates the AC-DF behavior applied to the 902 Default DF election algorithm, assuming the network in Figure 2: 904 a) When PE1 and PE2 discover ES12, they advertise an ES route for 905 ES12 with the associated ES-import extended community and the DF 906 Election Extended Community indicating AC-DF=1; they start a timer 907 at the same time. Likewise, PE2 and PE3 advertise an ES route for 908 ES23 with AC-DF=1 and start a timer. 910 b) PE1/PE2 advertise an Ethernet A-D per ES route for ES12, and 911 PE2/PE3 advertise an Ethernet A-D per ES route for ES23. 913 c) In addition, PE1/PE2/PE3 advertise an Ethernet A-D per EVI route 914 for AC1, AC2, AC3 and AC4 as soon as the ACs are enabled. Note 915 that the AC can be associated to a single customer VID (e.g. VLAN- 916 based service interfaces) or a bundle of customer VIDs (e.g. VLAN- 917 Bundle service interfaces). 919 d) When the timer expires, each PE builds an ordered "candidate" list 920 of the IP addresses of all the PE nodes connected to the Ethernet 921 Segment (including itself) as explained above in [RFC7432] Step 3. 923 Any PE from which an Ethernet A-D per ES route has not been 924 received is pruned from the list. 926 e) When electing the DF for a given BD, a PE will not be considered 927 candidate until an Ethernet A-D per EVI route has been received 928 from that PE. In other words, the ACS on the ES for a given PE 929 must be UP so that the PE is considered as candidate for a given 930 BD. For example, PE1 will not consider PE2 as candidate for DF 931 election for until an Ethernet A-D per EVI route is 932 received from PE2 for . 934 f) Once the PEs with ACS = DOWN for a given BD have been removed from 935 the candidate list, the DF Election can be applied for the 936 remaining N candidates. 938 Note that this procedure only modifies the existing EVPN control 939 plane by adding and processing the DF Election Extended Community, 940 and by pruning the candidate list of PEs that take part in the DF 941 election. 943 In addition to the events defined in the FSM in Section 3.1, the 944 following events SHALL modify the candidate PE list and trigger the 945 DF re-election in a PE for a given or . In 946 the FSM of Figure 3, the events below MUST trigger a transition from 947 DF_DONE to DF_CALC: 949 i. Local AC going DOWN/UP. 951 ii. Reception of a new Ethernet A-D per EVI update/withdraw for the 952 or . 954 iii. Reception of a new Ethernet A-D per ES update/withdraw for the 955 ES. 957 5.1. AC-Influenced DF Election Capability For VLAN-Aware Bundle Services 959 The procedure described section 5 works for VLAN-based and 960 VLAN-Bundle service interfaces since, for those service types, a PE 961 advertises only one Ethernet A-D per EVI route per or 962 . The withdrawal of such route means that the PE 963 cannot forward traffic on that particular or 964 , therefore the PE can be removed from consideration 965 for DF. 967 According to [RFC7432], in VLAN-aware bundle services, the PE 968 advertises multiple Ethernet A-D per EVI routes per 969 (one route per Ethernet Tag), while the DF Election is still 970 performed per . The withdrawal of an individual route 971 only indicates the unavailability of a specific AC but not 972 necessarily all the ACs in the . 974 This document modifies the DF Election for VLAN-Aware Bundle services 975 in the following way: 977 o After confirming that all the PEs in the ES advertise the AC-DF 978 capability, a PE will perform a DF Election per , as 979 opposed to per in [RFC7432]. Now, the withdrawal 980 of an Ethernet A-D per EVI route for a VLAN will indicate that the 981 advertising PE's ACS is DOWN and the rest of the PEs in the ES can 982 remove the PE from consideration for DF in the . 984 o The PEs will now follow the procedures in section 5. 986 For example, assuming three bridge tables in PE1 for the same MAC-VRF 987 (each one associated to a different Ethernet Tag, e.g. VLAN-1, VLAN-2 988 and VLAN-3), PE1 will advertise three Ethernet A-D per EVI routes for 989 ES12. Each of the three routes will indicate the status of each of 990 the three ACs in ES12. PE1 will be considered as a valid candidate PE 991 for DF election in , , as 992 long as its three routes are active. For instance, if PE1 withdraws 993 the Ethernet A-D per EVI routes for , the PEs in ES12 994 will not consider PE1 as a suitable DF candidate for . 995 PE1 will still be considered for and 996 since its routes are active. 998 6. Solution Benefits 1000 The solution described in this document provides the following 1001 benefits: 1003 a) Extends the DF Election in [RFC7432] to address the unfair load- 1004 balancing and potential black-holing issues of the default DF 1005 Election algorithm. The solution is applicable to the DF Election 1006 in EVPN Services [RFC7432] and EVPN Virtual Private Wire Services 1007 [RFC8214]. 1009 b) It defines a way to signal the DF Election algorithm and 1010 capabilities intended by the advertising PE. This is done by 1011 defining the DF Election Extended Community, which allow signaling 1012 of the capabilities supported by this document as well as any 1013 other future DF Election algorithms and capabilities. 1015 c) The solution is backwards compatible with the procedures defined 1016 in [RFC7432]. If one or more PEs in the ES do not support the new 1017 procedures, they will all follow the [RFC7432] DF Election. 1019 7. Security Considerations 1021 This document addresses some identified issues in the DF Election 1022 procedures described in [RFC7432] by defining a new DF Election 1023 framework. In general, this framework allows the PEs that are part of 1024 the same Ethernet Segment to exchange additional information and 1025 agree on the DF Election Type and Capabilities to be used. 1027 Following the procedures in this document, the operator will minimize 1028 undesired situations such as unfair load-balancing, service 1029 disruption and traffic black-holing. Since those situations may have 1030 been purposely created by a malicious user with access to the 1031 configuration of one PE, this document enhances also the security of 1032 the network. In addition, the new framework is extensible and allows 1033 for future new security enhancements that are out of the scope of 1034 this document. Finally, since this document extends the procedures in 1035 [RFC7432], the same Security Considerations described in [RFC7432] 1036 are valid for this document. 1038 8. IANA Considerations 1040 IANA is requested to: 1042 o Allocate Sub-Type value 0x06 in the "EVPN Extended Community Sub- 1043 Types" registry defined in [RFC7153] as follows: 1045 SUB-TYPE VALUE NAME Reference 1046 -------------- ------------------------- ------------- 1047 0x06 DF Election Extended Community This document 1049 o Set up a registry called "DF Alg" for the DF Alg octet in the 1050 Extended Community. New registrations will be made through the "RFC 1051 Required" procedure defined in [RFC8126]. The following initial 1052 values in that registry are requested: 1054 Alg Name Reference 1055 ---- -------------- ------------- 1056 0 Default DF Election This document 1057 1 HRW algorithm This document 1058 2-30 Unassigned 1059 31 Reserved for Experimental use This document 1061 o Set up a registry called "DF Election Capabilities" for the two- 1062 octet Bitmap field in the Extended Community. New registrations 1063 will be made through the "RFC Required" procedure defined in 1065 [RFC8126]. The following initial value in that registry is 1066 requested: 1068 Bit Name Reference 1069 ---- -------------- ------------- 1070 0 Unassigned 1071 1 AC-DF capability This document 1072 2-15 Unassigned 1074 9. References 1076 9.1. Normative References 1078 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 1079 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet 1080 VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, 1081 . 1083 [RFC8214] Boutros, S., Sajassi, A., Salam, S., Drake, J., and J. 1084 Rabadan, "Virtual Private Wire Service Support in Ethernet VPN", RFC 1085 8214, DOI 10.17487/RFC8214, August 2017, . 1088 [HRW1999] Thaler, D. and C. Ravishankar, "Using Name-Based Mappings 1089 to Increase Hit Rates", IEEE/ACM Transactions in networking Volume 6 1090 Issue 1, February 1998. 1092 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1093 Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1094 1997, . 1096 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1097 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, 1098 . 1100 [RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended 1101 Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, February 1102 2006, . 1104 [RFC7153] Rosen, E. and Y. Rekhter, "IANA Registries for BGP 1105 Extended Communities", RFC 7153, DOI 10.17487/RFC7153, March 2014, 1106 . 1108 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 1109 Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, 1110 DOI 10.17487/RFC8126, June 2017, . 1113 9.2. Informative References 1115 [VPLS-MH] Kothari, Henderickx et al., "BGP based Multi-homing in 1116 Virtual Private LAN Service", draft-ietf-bess-vpls-multihoming- 1117 02.txt, work in progress, September, 2018. 1119 [CHASH] Karger, D., Lehman, E., Leighton, T., Panigrahy, R., Levine, 1120 M., and D. Lewin, "Consistent Hashing and Random Trees: Distributed 1121 Caching Protocols for Relieving Hot Spots on the World Wide Web", ACM 1122 Symposium on Theory of Computing ACM Press New York, May 1997. 1124 [CLRS2009] Cormen, T., Leiserson, C., Rivest, R., and C. Stein, 1125 "Introduction to Algorithms (3rd ed.)", MIT Press and McGraw-Hill 1126 ISBN 0-262-03384-4., February 2009. 1128 [RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and 1129 Multicast Next-Hop Selection", RFC 2991, DOI 10.17487/RFC2991, 1130 November 2000, . 1132 [RFC2992] Hopps, C., "Analysis of an Equal-Cost Multi-Path 1133 Algorithm", RFC 2992, DOI 10.17487/RFC2992, November 2000, 1134 . 1136 10. Acknowledgments 1138 The authors want to thank Sriram Venkateswaran, Laxmi Padakanti, 1139 Ranganathan Boovaraghavan, Tamas Mondal, Sami Boutros, Jakob Heitz, 1140 Mrinmoy Ghosh, Leo Mermelstein, Mankamana Mishra and Samir Thoria for 1141 their review and contributions. Special thanks to Stephane Litkowski 1142 for his thorough review and detailed contributions. 1144 11. Contributors 1146 In addition to the authors listed on the front page, the following 1147 coauthors have also contributed to this document: 1149 Antoni Przygienda 1150 Juniper Networks, Inc. 1151 1194 N. Mathilda Drive 1152 Sunnyvale, CA 95134 1153 USA 1154 Email: prz@juniper.net 1155 Vinod Prabhu 1156 Nokia 1157 Email: vinod.prabhu@nokia.com 1159 Wim Henderickx 1160 Nokia 1161 Email: wim.henderickx@nokia.com 1163 Wen Lin 1164 Juniper Networks, Inc. 1165 Email: wlin@juniper.net 1167 Patrice Brissette 1168 Cisco Systems 1169 Email: pbrisset@cisco.com 1171 Keyur Patel 1172 Arrcus, Inc 1173 Email: keyur@arrcus.com 1175 Autumn Liu 1176 Ciena 1177 Email: hliu@ciena.com 1179 Authors' Addresses 1181 Jorge Rabadan 1182 Nokia 1183 777 E. Middlefield Road 1184 Mountain View, CA 94043 USA 1185 Email: jorge.rabadan@nokia.com 1187 Satya Mohanty 1188 Cisco Systems, Inc. 1189 225 West Tasman Drive 1190 San Jose, CA 95134 1191 USA 1192 Email: satyamoh@cisco.com 1194 Ali Sajassi 1195 Cisco Systems, Inc. 1196 225 West Tasman Drive 1197 San Jose, CA 95134 1198 USA 1199 Email: sajassi@cisco.com 1200 John Drake 1201 Juniper Networks, Inc. 1202 1194 N. Mathilda Drive 1203 Sunnyvale, CA 95134 1204 USA 1205 Email: jdrake@juniper.net 1207 Kiran Nagaraj 1208 Nokia 1209 701 E. Middlefield Road 1210 Mountain View, CA 94043 USA 1211 Email: kiran.nagaraj@nokia.com 1213 Senthil Sathappan 1214 Nokia 1215 701 E. Middlefield Road 1216 Mountain View, CA 94043 USA 1217 Email: senthil.sathappan@nokia.com