idnits 2.17.1 draft-ietf-bess-evpn-irb-extended-mobility-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 375 has weird spacing: '...E2] and adver...' -- The document date (Nov 2, 2019) is 1636 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'EVPN-INTER-SUBNET' is mentioned on line 170, but not defined == Missing Reference: 'RFC2119' is mentioned on line 190, but not defined == Missing Reference: 'RFC 826' is mentioned on line 215, but not defined == Missing Reference: 'RFC 4861' is mentioned on line 217, but not defined == Missing Reference: 'RFC 7432' is mentioned on line 986, but not defined == Missing Reference: 'VM-IP1' is mentioned on line 322, but not defined == Missing Reference: 'VM-IP2' is mentioned on line 322, but not defined == Missing Reference: 'VM-IP3' is mentioned on line 322, but not defined == Missing Reference: 'VM-IP4' is mentioned on line 322, but not defined == Missing Reference: 'VM-IP5' is mentioned on line 322, but not defined == Missing Reference: 'VM-IP6' is mentioned on line 322, but not defined == Missing Reference: 'IP1' is mentioned on line 395, but not defined == Missing Reference: 'MAC1' is mentioned on line 327, but not defined == Missing Reference: 'PE1' is mentioned on line 375, but not defined == Missing Reference: 'PE2' is mentioned on line 375, but not defined == Missing Reference: 'PE3' is mentioned on line 449, but not defined == Missing Reference: 'PE4' is mentioned on line 449, but not defined == Missing Reference: 'MAC2' is mentioned on line 331, but not defined == Missing Reference: 'VM-IP1-M1' is mentioned on line 372, but not defined == Missing Reference: 'VM-IP2-M2' is mentioned on line 372, but not defined == Missing Reference: 'VM-IP3-M3' is mentioned on line 372, but not defined == Missing Reference: 'VM-IP4-M4' is mentioned on line 372, but not defined == Missing Reference: 'VM-IP5-M5' is mentioned on line 372, but not defined == Missing Reference: 'VM-IP6-M6' is mentioned on line 372, but not defined == Missing Reference: 'IP7' is mentioned on line 393, but not defined == Missing Reference: 'M1' is mentioned on line 746, but not defined == Missing Reference: 'I1' is mentioned on line 746, but not defined == Missing Reference: 'M2' is mentioned on line 748, but not defined == Missing Reference: 'I3' is mentioned on line 748, but not defined == Missing Reference: 'RFC 7814' is mentioned on line 818, but not defined == Unused Reference: 'RFC7814' is defined on line 1042, but no explicit reference was found in the text == Outdated reference: A later version (-16) exists of draft-ietf-bess-evpn-proxy-arp-nd-06 == Outdated reference: A later version (-15) exists of draft-ietf-bess-evpn-inter-subnet-forwarding-08 ** Downref: Normative reference to an Informational RFC: RFC 7814 Summary: 1 error (**), 0 flaws (~~), 35 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft N. Malhotra, Ed. 3 BESS Working Group A. Sajassi 4 Intended Status: Proposed Standard A. Pattekar 5 Cisco 7 A. Lingala 8 AT&T 10 J. Rabadan 11 Nokia 13 J. Drake 14 Juniper Networks 16 Expires: May 5, 2020 Nov 2, 2019 18 Extended Mobility Procedures for EVPN-IRB 19 draft-ietf-bess-evpn-irb-extended-mobility-02 21 Abstract 23 Procedure to handle host mobility in a layer 2 Network with EVPN 24 control plane is defined as part of RFC 7432. EVPN has since evolved 25 to find wider applicability across various IRB use cases that include 26 distributing both MAC and IP reachability via a common EVPN control 27 plane. MAC Mobility procedures defined in RFC 7432 are extensible to 28 IRB use cases if a fixed 1:1 mapping between VM IP and MAC is assumed 29 across VM moves. Generic mobility support for IP and MAC that allows 30 these bindings to change across moves is required to support a 31 broader set of EVPN IRB use cases, and requires further 32 consideration. EVPN all-active multi-homing further introduces 33 scenarios that require additional consideration from mobility 34 perspective. This document enumerates a set of design considerations 35 applicable to mobility across these EVPN IRB use cases and defines 36 generic sequence number assignment procedures to address these IRB 37 use cases. 39 Status of this Memo 41 This Internet-Draft is submitted to IETF in full conformance with the 42 provisions of BCP 78 and BCP 79. 44 Internet-Drafts are working documents of the Internet Engineering 45 Task Force (IETF), its areas, and its working groups. Note that 46 other groups may also distribute working documents as 47 Internet-Drafts. 49 Internet-Drafts are draft documents valid for a maximum of six months 50 and may be updated, replaced, or obsoleted by other documents at any 51 time. It is inappropriate to use Internet-Drafts as reference 52 material or to cite them other than as "work in progress." 54 The list of current Internet-Drafts can be accessed at 55 http://www.ietf.org/1id-abstracts.html 57 The list of Internet-Draft Shadow Directories can be accessed at 58 http://www.ietf.org/shadow.html 60 Copyright and License Notice 62 Copyright (c) 2017 IETF Trust and the persons identified as the 63 document authors. All rights reserved. 65 This document is subject to BCP 78 and the IETF Trust's Legal 66 Provisions Relating to IETF Documents 67 (http://trustee.ietf.org/license-info) in effect on the date of 68 publication of this document. Please review these documents 69 carefully, as they describe your rights and restrictions with respect 70 to this document. Code Components extracted from this document must 71 include Simplified BSD License text as described in Section 4.e of 72 the Trust Legal Provisions and are provided without warranty as 73 described in the Simplified BSD License. 75 Table of Contents 77 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 78 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 79 2. Optional MAC only RT-2 . . . . . . . . . . . . . . . . . . . . 6 80 3. Mobility Use Cases . . . . . . . . . . . . . . . . . . . . . . 6 81 3.1 Host MAC+IP Move . . . . . . . . . . . . . . . . . . . . . 6 82 3.2 Host IP Move to new MAC . . . . . . . . . . . . . . . . . . 6 83 3.2.1 VM Reload . . . . . . . . . . . . . . . . . . . . . . . 7 84 3.2.2 MAC Sharing . . . . . . . . . . . . . . . . . . . . . . 7 85 3.2.3 Problem . . . . . . . . . . . . . . . . . . . . . . . . 7 86 3.3 Host MAC move to new IP . . . . . . . . . . . . . . . . . . 8 87 3.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . 8 88 4. EVPN All Active multi-homed ES . . . . . . . . . . . . . . . . 11 89 5. Design Considerations . . . . . . . . . . . . . . . . . . . . 12 90 6. Solution Components . . . . . . . . . . . . . . . . . . . . . 13 91 6.1 Sequence Number Inheritance . . . . . . . . . . . . . . . . 13 92 6.2 MAC Sharing . . . . . . . . . . . . . . . . . . . . . . . . 14 93 6.3 Multi-homing Mobility Synchronization . . . . . . . . . . . 15 95 7. Requirements for Sequence Number Assignment . . . . . . . . . 15 96 7.1 LOCAL MAC-IP learning . . . . . . . . . . . . . . . . . . . 15 97 7.2 LOCAL MAC learning . . . . . . . . . . . . . . . . . . . . 16 98 7.3 Remote MAC OR MAC-IP Update . . . . . . . . . . . . . . . . 16 99 7.4 REMOTE (SYNC) MAC update . . . . . . . . . . . . . . . . . 16 100 7.5 REMOTE (SYNC) MAC-IP update . . . . . . . . . . . . . . . . 17 101 7.6 Inter-op . . . . . . . . . . . . . . . . . . . . . . . . . 17 102 7.7 MAC Sharing Race Condition . . . . . . . . . . . . . . . . 18 103 7.8 Mobility Convergence . . . . . . . . . . . . . . . . . . . 18 104 7.8.1 Generalized Probing Logic . . . . . . . . . . . . . . . 19 105 8. Routed Overlay . . . . . . . . . . . . . . . . . . . . . . . . 19 106 9. Duplicate Host Detection . . . . . . . . . . . . . . . . . . . 20 107 9.1 Scenario A . . . . . . . . . . . . . . . . . . . . . . . . . 20 108 9.2 Scenario B . . . . . . . . . . . . . . . . . . . . . . . . . 20 109 9.2.1 Duplicate IP Detection Procedure for Scenario B . . . . 21 110 9.3 Scenario C . . . . . . . . . . . . . . . . . . . . . . . . . 22 111 9.4 Duplicate Host Recovery . . . . . . . . . . . . . . . . . . 22 112 9.4.1 Route Un-freezing Configuration . . . . . . . . . . . . 22 113 9.4.2 Route Clearing Configuration . . . . . . . . . . . . . 23 114 10. Security Considerations . . . . . . . . . . . . . . . . . . . 23 115 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 116 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 117 12.1 Normative References . . . . . . . . . . . . . . . . . . . 24 118 12.2 Informative References . . . . . . . . . . . . . . . . . . 24 119 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24 120 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24 121 Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 123 1 Introduction 125 EVPN-IRB enables capability to advertise both MAC and IP routes via a 126 single MAC+IP RT-2 advertisement. MAC is imported into local bridge 127 MAC table and enables L2 bridged traffic across the network overlay. 128 IP is imported into the local ARP table in an asymmetric IRB design 129 OR imported into the IP routing table in a symmetric IRB design, and 130 enables routed traffic across the layer 2 network overlay. Please 131 refer to [EVPN-IRB] for more background on EVPN IRB forwarding modes. 133 To support EVPN mobility procedure, a single sequence number mobility 134 attribute is advertised with the combined MAC+IP route. A single 135 sequence number advertised with the combined MAC+IP route to resolve 136 both MAC and IP reachability implicitly assumes a 1:1 fixed mapping 137 between IP and MAC. While a fixed 1:1 mapping between IP and MAC is a 138 common use case that could be addressed via existing MAC mobility 139 procedure, additional IRB scenarios need to be considered, that don't 140 necessarily adhere to this assumption. Following IRB mobility 141 scenarios are considered: 143 o VM move results in VM IP and MAC moving together 145 o VM move results in VM IP moving to a new MAC association 147 o VM move results in VM MAC moving to a new IP association 149 While existing MAC mobility procedure can be leveraged for MAC+IP 150 move in the first scenario, subsequent scenarios result in a new MAC- 151 IP association. As a result, a single sequence number assigned 152 independently per-[MAC, IP] is not sufficient to determine most 153 recent reachability for both MAC and IP, unless the sequence number 154 assignment algorithm is designed to allow for changing MAC-IP 155 bindings across moves. 157 Purpose of this draft is to define additional sequence number 158 assignment and handling procedures to adequately address generic 159 mobility support across EVPN-IRB overlay use cases that allow MAC-IP 160 bindings to change across VM moves and can support mobility for both 161 MAC and IP components carried in an EVPN RT-2 for these use cases. 163 In addition, for hosts on an ESI multi-homed to multiple GW devices, 164 additional procedure is proposed to ensure synchronized sequence 165 number assignments across the multi-homing devices. 167 Content presented in this draft is independent of data plane 168 encapsulation used in the overlay being MPLS or NVO Tunnels. It is 169 also largely independent of the EVPN IRB solution being based on 170 symmetric OR asymmetric IRB design as defined in [EVPN-INTER-SUBNET]. 172 In addition to symmetric and asymmetric IRB, mobility solution for a 173 routed overlay, where traffic to an end host in the overlay is always 174 IP routed using EVPN RT-5 is also presented in section 8. 176 To summarize, this draft covers mobility mobility for the following 177 independent of the overlay encapsulation being MPLS or an NVO Tunnel: 179 o Symmetric EVPN IRB overlay 181 o Asymmetric EVPN IRB overlay 183 o Routed EVPN overlay 185 1.1 Terminology 187 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 188 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 189 document are to be interpreted as described in RFC 2119 [RFC2119]. 191 o EVPN-IRB: A BGP-EVPN distributed control plane based integrated 192 routing and bridging fabric overlay discussed in [EVPN-IRB] 193 o Underlay: IP or MPLS fabric core network that provides IP or 194 MPLS routed reachability between EVPN PEs. 195 o Overlay: VPN or service layer network consisting of EVPN PEs 196 OR VPN provider-edge (PE) switch-router devices that runs on top 197 of an underlay routed core. 198 o EVPN PE: A PE switch-router in a data-center fabric that 199 runs overlay BGP-EVPN control plane and connects to overlay CE 200 host devices. An EVPN PE may also be the first-hop layer-3 201 gateway for CE/host devices. This document refers to EVPN PE as a 202 logical function in a data-center fabric. This EVPN PE function 203 may be physically hosted on a top-of-rack switching device (ToR) 204 OR at layer(s) above the ToR in the Clos fabric. An EVPN PE is 205 typically also an IP or MPLS tunnel end-point for overlay VPN 206 flow 207 o Symmetric EVPN-IRB: An overlay fabric first-hop routing 208 architecture as defined in [EVPN-IRB], wherein, overlay host-to- 209 host routed inter-subnet flows are routed at both ingress and 210 egress EVPN PEs. 211 o Asymmetric EVPN-IRB: An overlay fabric first-hop routing 212 architecture as defined in [EVPN-IRB], wherein, overlay host-to- 213 host routed inter-subnet flows are routed and bridged at ingress 214 PE and bridged at egress PEs. 215 o ARP: Address Resolution Protocol [RFC 826]. ARP references in 216 this document are equally applicable to ND as well. 217 o ND: IPv6 Neighbor Discovery Protocol [RFC 4861]. 218 o Ethernet-Segment: physical Ethernet or LAG port that connects an 219 access device to an EVPN PE, as defined in [RFC 7432]. 221 o ESI: Ethernet Segment Identifier as defined in [RFC 7432]. 222 o LAG: Layer-2 link-aggregation, also known as layer-2 bundle 223 port-channel, or bond interface. 224 o EVPN all-active multi-homing: PE-CE all-active multi-homing 225 achieved via a multi-homed layer-2 LAG interface on a CE with 226 member links to multiple PEs and related EVPN procedures on the 227 PEs. 228 o RT-2: EVPN route type 2 carrying both MAC and IP reachability. 229 o RT-5: EVPN route type 5 carrying IP prefix reachability. 230 o MAC-IP: IP association for a MAC, referred to in this document 231 may be IPv4, IPv6 or both. 233 2. Optional MAC only RT-2 235 In an EVPN IRB scenario, where a single MAC+IP RT-2 advertisement 236 carries both IP and MAC routes, a MAC only RT-2 advertisement is 237 redundant for host MACs that are advertised via MAC+IP RT-2. As a 238 result, a MAC only RT-2 is an optional route that may not be 239 advertised from or received at an EVPN PE. This is an important 240 consideration for mobility scenarios discussed in subsequent 241 sections. 243 MAC only RT-2 may still be advertised for non-IP host MACs that are 244 not advertised via MAC+IP RT-2. 246 3. Mobility Use Cases 248 This section describes the IRB mobility use cases considered in this 249 document. Procedures to address them are covered later in section 6 250 and section 7. 252 o Host move results in Host IP and MAC moving together 254 o Host move results in Host IP moving to a new MAC association 256 o Host move results in Host MAC moving to a new IP association 258 3.1 Host MAC+IP Move 260 This is the baseline case, wherein a host move results in both host 261 MAC and IP moving together with no change in MAC-IP binding across a 262 move. Existing MAC mobility defined in RFC 7432 may be leveraged to 263 apply to corresponding MAC+IP route to support this mobility 264 scenario. 266 3.2 Host IP Move to new MAC 268 This is the case, where a host move results in VM IP moving to a new 269 MAC binding. 271 3.2.1 VM Reload 273 A host reload or an orchestrated host move that results in host being 274 re-spawned at a new location may result in host getting a new MAC 275 assignment, while maintaining existing IP address. This results in a 276 host IP move to a new MAC binding: 278 IP-a, MAC-a ---> IP-a, MAC-b 280 3.2.2 MAC Sharing 282 This takes into account scenarios, where multiple hosts, each with a 283 unique IP, may share a common MAC binding, and a host move results in 284 a new MAC binding for the host IP. 286 As an example, hosts running on a single physical server, each with a 287 unique IP, may share the same physical server MAC. In yet another 288 scenario, an L2 access network may be behind a firewall, such that 289 all hosts IPs on the access network are learnt with a common firewall 290 MAC. In all such "shared MAC" use cases, multiple local MAC-IP ARP 291 entries may be learnt with the same MAC. A host IP move, in such 292 scenarios (for e.g., to a new physical server), could result in new 293 MAC association for the host IP. 295 3.2.3 Problem 297 In both of the above scenarios, a combined MAC+IP EVPN RT-2 298 advertised with a single sequence number attribute implicitly assumes 299 a fixed IP to MAC mapping. A host IP move to a new MAC breaks this 300 assumption and results in a new MAC+IP route. If this new MAC+IP 301 route is independently assigned a new sequence number, the sequence 302 number can no longer be used to determine most recent host IP 303 reachability in a symmetric EVPN-IRB design OR the most recent IP to 304 MAC binding in an asymmetric EVPN-IRB design. 306 +------------------------+ 307 | Underlay Network Fabric| 308 +------------------------+ 310 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 311 | PE1 | | PE2 | | PE3 | | PE4 | | PE5 | | PE6 | 312 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 313 \ / \ / \ / 314 \ ESI-1 / \ ESI-2 / \ ESI-3 / 315 \ / \ / \ / 316 +\---/+ +\---/+ +\---/+ 317 | \ / | | \ / | | \ / | 318 +--+--+ +--+--+ +--+--+ 319 | | | 320 Server-MAC1 Server-MAC2 Server-MAC3 321 | | | 322 [VM-IP1, VM-IP2] [VM-IP3, VM-IP4] [VM-IP5, VM-IP6] 324 Figure 1 326 As an example, consider a topology shown in Figure 1, with host VMs 327 sharing the physical server MAC. In steady state, [IP1, MAC1] route 328 is learnt at [PE1, PE2] and advertised to remote PEs with a sequence 329 number N. Now, VM-IP1 is moved to Server-MAC2. ARP or ND based local 330 learning at [PE3, PE4] would now result in a new [IP1, MAC2] route 331 being learnt. If route [IP1, MAC2] is learnt as a new MAC+IP route 332 and assigned a new sequence number of say 0, mobility procedure for 333 VM-IP1 will not trigger across the overlay network. 335 A sequence number assignment procedure needs to be defined to 336 unambiguously determine the most recent IP reachability, IP to MAC 337 binding, and MAC reachability for such a MAC sharing scenario. 339 3.3 Host MAC move to new IP 341 This is a scenario where host move or re-provisioning behind a new 342 gateway location may result in host getting a new IP address 343 assigned, while keeping the same MAC. 345 3.3.1 Problem 347 Complication with this scenario is that MAC reachability could be 348 carried via a combined MAC+IP route while a MAC only route may not be 349 advertised at all. A single sequence number association with the 350 MAC+IP route again implicitly assumes a fixed mapping between MAC and 351 IP. A MAC move resulting in a new IP association for the host MAC 352 breaks this assumption and results in a new MAC+IP route. If this new 353 MAC+IP route independently assumes a new sequence number, this 354 mobility attribute can no longer be used to determine most recent 355 host MAC reachability. 357 +------------------------+ 358 | Underlay Network Fabric| 359 +------------------------+ 360 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 361 | PE1 | | PE2 | | PE3 | | PE4 | | PE5 | | PE6 | 362 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 363 \ / \ / \ / 364 \ ESI-1 / \ ESI-2 / \ ESI-3 / 365 \ / \ / \ / 366 +\---/+ +\---/+ +\---/+ 367 | \ / | | \ / | | \ / | 368 +--+--+ +--+--+ +--+--+ 369 | | | 370 Server1 Server2 Server3 371 | | | 372 [VM-IP1-M1, VM-IP2-M2] [VM-IP3-M3, VM-IP4-M4] [VM-IP5-M5, VM-IP6-M6] 374 As an example, consider a host VM IP1-M1 that is learnt locally at 375 [PE1, PE2] and advertised to remote hosts with a sequence number N. 376 Consider a scenario where this VM with MAC M1 is re-provisioned at 377 server 2, however, as part of this re-provisioning, assigned a 378 different IP address say IP7. [IP7, M1] is learnt as a new route at 379 [PE3, PE4] and advertised to remote PEs with a sequence number of 0. 380 As a result, L3 reachability to IP7 would be established across the 381 overlay, however, MAC mobility procedure for MAC1 will not trigger as 382 a result of this MAC-IP route advertisement. If an optional MAC only 383 route is also advertised, sequence number associated with the MAC 384 only route would trigger MAC mobility as per [RFC7432]. However, in 385 the absence of an additional MAC only route advertisement, a single 386 sequence number advertised with a combined MAC+IP route would not be 387 sufficient to update MAC reachability across the overlay. 389 A MAC-IP sequence number assignment procedure needs to be defined to 390 unambiguously determine the most recent MAC reachability in such a 391 scenario without a MAC only route being advertised. 393 Further, PE1/PE2, on learning new reachability for [IP7, M1] via 394 PE3/PE4 MUST probe and delete any local IPs associated with MAC M1, 395 such as [IP1, M1] in the above example. 397 Arguably, MAC mobility sequence number defined in [RFC7432], could be 398 interpreted to apply only to the MAC part of MAC-IP route, and would 399 hence cover this scenario. It could hence be interpreted as a 400 clarification to [RFC7432] and one of the considerations for a common 401 sequence number assignment procedure across all MAC-IP mobility 402 scenarios detailed in this document. 404 4. EVPN All Active multi-homed ES 406 +------------------------+ 407 | Underlay Network Fabric| 408 +------------------------+ 410 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 411 | PE1 | | PE2 | | PE3 | | PE4 | | PE5 | | PE6 | 412 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 413 \ / \ / \ / 414 \ ESI-1 / \ ESI-2 / \ ESI-3 / 415 \ / \ / \ / 416 +\---/+ +\---/+ +\---/+ 417 | \ / | | \ / | | \ / | 418 +--+--+ +--+--+ +--+--+ 419 | | | 420 Server-1 Server-2 Server-3 422 Figure 2 424 Consider an EVPN-IRB overlay network shown in Figure 2, with hosts 425 multi-homed to two or more PE devices via an all-active multi-homed 426 ES. MAC and ARP entries learnt on a local ES may also be synchronized 427 across the multi-homing PE devices sharing this ES. This MAC and ARP 428 SYNC enables local switching of intra and inter subnet ECMP traffic 429 flows from remote hosts. In other words, local MAC and ARP entries on 430 a given ES may be learnt via local learning and / or via sync from 431 another PE device sharing the same ES. 433 For a host that is multi-homed to multiple PE devices via an all- 434 active ES interface, local learning of host MAC and MAC-IP at each PE 435 device is an independent asynchronous event, that is dependent on 436 traffic flow and or ARP / ND response from the host hashing to a 437 directly connected PE on the MC-LAG interface. As a result, sequence 438 number mobility attribute value assigned to a locally learnt MAC or 439 MAC-IP route at each device may not always be the same, depending on 440 transient states on the device at the time of local learning. 442 As an example, consider a host VM that is deleted from ESI-2 and 443 moved to ESI-1. It is possible for host to be learnt on say, PE1 444 following deletion of the remote route from [PE3, PE4], while being 445 learnt on PE2 prior to deletion of remote route from [PE3, PE4]. If 446 so, PE1 would process local host route learning as a new route and 447 assign a sequence number of 0, while PE2 would process local host 448 route learning as a remote to local move and assign a sequence number 449 of N+1, N being the existing sequence number assigned at [PE3, PE4]. 451 Inconsistent sequence numbers advertised from multi-homing devices 452 introduces: 454 o Ambiguity with respect to how the remote PEs should handle 455 paths with same ESI and different sequence numbers. A remote PE 456 may not program ECMP paths if it receives routes with different 457 sequence numbers from a set of multi-homing PEs sharing the same 458 ESI. 460 o Breaks consistent route versioning across the network overlay 461 that is needed for EVPN mobility procedures to work. 463 As an example, in this inconsistent state, PE2 would drop a remote 464 route received for the same host with sequence number N (as its local 465 sequence number is N+1), while PE1 would install it as the best route 466 (as its local sequence number is 0). 468 There is need for a mechanism to ensure consistency of sequence 469 numbers advertised from a set of multi-homing devices for EVPN 470 mobility to work reliably. 472 In order to support mobility for multi-homed hosts using the sequence 473 number mobility attribute, local MAC and MAC-IP routes learnt on a 474 multi-homed ES MUST be advertised with the same sequence number by 475 all PE devices that the ES is multi-homed to. There is need for a 476 mechanism to ensure consistency of sequence numbers assigned across 477 these PEs. 479 5. Design Considerations 481 To summarize, sequence number assignment scheme and implementation 482 must take following considerations into account: 484 o MAC+IP may be learnt on an ES multi-homed to multiple PE 485 devices, hence requires sequence numbers to be synchronized 486 across multi-homing PE devices. 488 o MAC only RT-2 is optional in an IRB scenario and may not 489 necessarily be advertised in addition to MAC+IP RT-2 491 o Single MAC may be associated with multiple IPs, i.e., multiple 492 host IPs may share a common MAC 494 o Host IP move could result in host moving to a new MAC, resulting 495 in a new IP to MAC association and a new MAC+IP route. 497 o Host MAC move to a new location could result in host MAC being 498 associated with a different IP address, resulting in a new MAC to 499 IP association and a new MAC+IP route 501 o LOCAL MAC-IP learn via ARP would always accompanied by a LOCAL 502 MAC learn event resulting from the ARP packet. MAC and MAC-IP 503 learning, however, could happen in any order 505 o Use cases discussed earlier that do not maintain a constant 1:1 506 MAC-IP mapping across moves could potentially be addressed by 507 using separate sequence numbers associated with MAC and IP 508 components of MAC+IP route. Maintaining two separate sequence 509 numbers however adds significant overhead with respect to 510 complexity, debugability, and backward compatibility. Hence, this 511 document addresses these requirements via a single sequence 512 number attribute. 514 6. Solution Components 516 This section goes over main components of the EVPN IRB mobility 517 solution proposed in this draft. Later sections will go over exact 518 sequence number assignment procedures resulting from concepts 519 described in this section. 521 6.1 Sequence Number Inheritance 523 Main idea presented here is to view a LOCAL MAC-IP route as a child 524 of the corresponding LOCAL MAC only route that inherits the sequence 525 number attribute from the parent LOCAL MAC only route: 527 Mx-IPx -----> Mx (seq# = N) 529 As a result, both parent MAC and child MAC-IP routes share one common 530 sequence number associated with the parent MAC route. Doing so 531 ensures that a single sequence number attribute carried in a combined 532 MAC+IP route represents sequence number for both a MAC only route as 533 well as a MAC+IP route, and hence makes the MAC only route truly 534 optional. As a result, optional MAC only route with its own sequence 535 number is not required to establish most recent reachability for a 536 MAC in the overlay network. Specifically, this enables a MAC to 537 assume a different IP address on a move, and still be able to 538 establish most recent reachability to the MAC across the overlay 539 network via mobility attribute associated with the MAC+IP route 540 advertisement. As an example, when Mx moves to a new location, it 541 would result in LOCAL Mx being assigned a higher sequence number at 542 its new location as per RFC 7432. If this move results in Mx assuming 543 a different IP address, IPz, LOCAL Mx+IPz route would inherit the new 544 sequence number from Mx. 546 LOCAL MAC and LOCAL MAC-IP routes would typically be sourced from 547 data plane learning and ARP learning respectively, and could get 548 learnt in control plane in any order. Implementation could either 549 replicate inherited sequence number in each MAC-IP entry OR maintain 550 a single attribute in the parent MAC by creating a forward reference 551 LOCAL MAC object for cases where a LOCAL MAC-IP is learnt before the 552 LOCAL MAC. 554 Arguably, this inheritance may be assumed from RFC 7432, in which 555 case, the above may be interpreted as a clarification with respect to 556 interpretation of a MAC sequence number in a MAC-IP route. 558 6.2 MAC Sharing 560 Further, for the shared MAC scenario, this would result in multiple 561 LOCAL MAC-IP siblings inheriting sequence number attribute from a 562 common parent MAC route: 564 Mx-IP1 ----- 565 | | 566 Mx-IP2 ----- 567 . | 568 . +---> Mx (seq# = N) 569 . | 570 Mx-IPw ----- 571 | | 572 Mx-IPx ----- 574 In such a case, a host-IP move to a different physical server would 575 result in IP moving to a new MAC binding. A new MAC-IP route 576 resulting from this move must now be advertised with a sequence 577 number that is higher than the previous MAC-IP route for this IP, 578 advertised from the prior location. As an example, consider a route 579 Mx-IPx that is currently advertised with sequence number N from PE1. 580 IPx moving to a new physical server behind PE2 results in IPx being 581 associated with MAC Mz. A new local Mz-IPx route resulting from this 582 move at PE2 must now be advertised with a sequence number higher than 583 N. This is so that PE devices, including PE1, PE2, and other remote 584 PE devices that are part of the overlay can clearly determine and 585 program the most recent MAC binding and reachability for the IP. PE1, 586 on receiving this new Mz-IPx route with sequence number say, N+1, for 587 symmetric IRB case, would update IPx reachability via PE2 in 588 forwarding, for asymmetric IRB case, would update IPx's ARP binding 589 to Mz. In addition, PE1 would clear and withdraw the stale Mx-IPx 590 route with the lower sequence number. 592 This also implies that sequence number associated with local MAC Mz 593 and all local MAC-IP children of Mz at PE2 must now be incremented to 594 N+1, and re-advertised across the overlay. While this re- 595 advertisement of all local MAC-IP children routes affected by the 596 parent MAC route is an overhead, it avoids the need for two separate 597 sequence number attributes to be maintained and advertised for IP and 598 MAC components of MAC+IP RT-2. Implementation would need to be able 599 to lookup MAC-IP routes for a given IP and update sequence number for 600 it's parent MAC and its MAC-IP children. 602 6.3 Multi-homing Mobility Synchronization 604 In order to support mobility for multi-homed hosts, local MAC and 605 MAC-IP routes learnt on a shared ES MUST be advertised with the same 606 sequence number by all PE devices that the ES is multi-homed to. This 607 also applies to local MAC only routes. LOCAL MAC and MAC-IP may be 608 learnt natively via data plane and ARP/ND respectively as well as via 609 SYNC from another multi-homing PE to achieve local switching. Local 610 and SYNC route learning can happen in any order. Local MAC-IP routes 611 advertised by all multi-homing PE devices sharing the ES must carry 612 the same sequence number, independent of the order in which they are 613 learnt. This implies: 615 o On local or sync MAC-IP route learning, sequence number for the 616 local MAC-IP route MUST be compared and updated to the higher 617 value. 619 o On local or sync MAC route learning, sequence number for the 620 local MAC route MUST be compared and updated to the higher value. 622 If an update to local MAC-IP sequence number is required as a result 623 of above comparison with sync MAC-IP route, it would essentially 624 amount to a sequence number update on the parent local MAC, resulting 625 in inherited sequence number update on the MAC-IP route. 627 7. Requirements for Sequence Number Assignment 629 Following sections summarize sequence number assignment procedure 630 needed on local and sync MAC and MAC-IP route learning events in 631 order to accomplish the above. 633 7.1 LOCAL MAC-IP learning 635 A local Mx-IPx learning via ARP or ND should result in computation OR 636 re-computation of parent MAC Mx's sequence number, following which 637 the MAC-IP route Mx-IPx would simply inherit parent MAC's sequence 638 number. Parent MAC Mx Sequence number should be computed as follows: 640 o MUST be higher than any existing remote MAC route for Mx, as per 641 RFC 7432. 643 o MUST be at least equal to corresponding SYNC MAC sequence number 644 if one is present. 646 o If the IP is also associated with a different remote MAC "Mz", 647 MUST be higher than "Mz" sequence number 649 Once new sequence number for MAC route Mx is computed as per above, 650 all LOCAL MAC-IPs associated with MAC Mx MUST inherit the updated 651 sequence number. 653 7.2 LOCAL MAC learning 655 Local MAC Mx Sequence number should be computed as follows: 657 o MUST be higher than any existing remote MAC route for Mx, as per 658 RFC 7432. 660 o MUST be at least equal to corresponding SYNC MAC sequence number 661 if one is present. 663 o Once new sequence number for MAC route Mx is computed as per 664 above, all LOCAL MAC-IPs associated with MAC Mx MUST inherit the 665 updated sequence number. 667 Note that the local MAC sequence number might already be present if 668 there was a local MAC-IP learnt prior to the local MAC, in which case 669 the above may not result in any change in local MAC's sequence 670 number. 672 7.3 Remote MAC OR MAC-IP Update 674 On receiving a remote MAC OR MAC-IP route update associated with a 675 MAC Mx with a sequence number that is higher than or equal to 676 sequence number assigned to a LOCAL route for MAC Mx: 678 o PE MUST trigger probe and deletion procedure for all LOCAL IPs 679 associated with MAC Mx 681 o PE MUST trigger deletion procedure for LOCAL MAC route for Mx 683 7.4 REMOTE (SYNC) MAC update 685 Corresponding local MAC Mx (if present) sequence number should be re- 686 computed as follows: 688 o If the current sequence number is less than the received SYNC 689 MAC sequence number, it MUST be increased to be equal to received 690 SYNC MAC sequence number. 692 o If a LOCAL MAC sequence number is updated as a result of the 693 above, all LOCAL MAC-IPs associated with MAC Mx MUST inherit the 694 updated sequence number. 696 7.5 REMOTE (SYNC) MAC-IP update 698 If this is a SYNCed MAC-IP on a local ES, it would also result in a 699 derived SYNC MAC Mx route entry, as MAC only RT-2 advertisement is 700 optional. Corresponding local MAC Mx (if present) sequence number 701 should be re-computed as follows: 703 o If the current sequence number is less than the received SYNC 704 MAC sequence number, it MUST be increased to be equal to received 705 SYNC MAC sequence number. 707 o If a LOCAL MAC sequence number is updated as a result of the 708 above, all LOCAL MAC-IPs associated with MAC Mx MUST inherit the 709 updated sequence number. 711 7.6 Inter-op 713 In general, if all PE nodes in the overlay network follow the above 714 sequence number assignment procedure, and the PE is advertising both 715 MAC+IP and MAC routes, sequence number advertised with the MAC and 716 MAC+IP routes with the same MAC would always be the same. However, an 717 inter-op scenario with a different implementation could arise, where 718 a PE implementation non-compliant with this document or with RFC 7432 719 assigns and advertises independent sequence numbers to MAC and MAC+IP 720 routes. To handle this case, if different sequence numbers are 721 received for remote MAC+IP and corresponding remote MAC routes from a 722 remote PE, sequence number associated with the remote MAC route 723 should be computed as: 725 o Highest of the all received sequence numbers with remote MAC+IP 726 and MAC routes with the same MAC. 728 o MAC sequence number would be re-computed on a MAC or MAC+IP 729 route withdraw as per above. 731 A MAC and / or IP move to the local PE would now result in the MAC 732 (and hence all MAC-IP) sequence numbers incremented from the above 733 computed remote MAC sequence number. 735 7.7 MAC Sharing Race Condition 737 In a MAC sharing use case described in section 6.2, a race condition 738 is possible with simultaneous host moves between a pair of PEs. As an 739 example, consider PE1 with local host IPs I1 and I2 sharing MAC M1, 740 and PE2 with local host IPs I3 and I4 sharing MAC M2. A simultaneous 741 move of I1 from PE1 to PE2 and of I3 from PE2 to PE1, such that I3 is 742 learnt on PE1 before I1's local entry has been probed out on PE1 743 and/or I1 is learnt on PE2 before I3's local entry has been probed 744 out on PE2 may trigger a race condition. This race condition together 745 with MAC sequence number assignment rules defined in section 7.1 can 746 cause new mac-ip routes [I1, M2] and [I3, M1] to bounce a couple of 747 times with an incremented sequence number until stale entries [I1, 748 M1] and [I3, M2] have been probed out from PE1 and PE2 respectively. 749 An implementation MUST ensure proper probing procedures to remove 750 stale ARP, ND, and local MAC entries, following a move, on learning 751 remote routes as defined in section 7.3 (and as per [EVPN-IRB]) to 752 minimize exposure to this race condition. 754 7.8 Mobility Convergence 756 This sections is to be treated as optional and details ARP and ND 757 probing procedures that MAY be implemented to achieve faster host re- 758 learning and convergence on mobility events. 760 o Following a host move from PE1 to PE2, the host's MAC is 761 discovered at PE2 as a local MAC via a data frames received from 762 the host. If PE2 has a prior REMOTE MAC-IP host route for this 763 MAC from PE1, an ARP/ND probe MAY be triggered at PE2 to learn 764 the MAC-IP as a local adjacency and trigger EVPN RT-2 765 advertisement for this MAC-IP across the overlay with new 766 reachability via PE2. This results in a reliable "event based" 767 host IP learning triggered by a "MAC learning event" across the 768 overlay, and hence faster convergence of overlay routed flows to 769 the host. 771 o Following a host move from PE1 to PE2, once PE1 receives a MAC 772 or MAC-IP route from PE2 with a higher sequence number, an ARP/ND 773 probe MAY be triggered at PE1 to clear the stale local MAC-IP 774 neighbor adjacency OR re-learn the local MAC-IP in case the host 775 has moved back or is duplicate. 777 o Following a local MAC age-out, if there is a local IP adjacency 778 with this MAC, an ARP/ND probe MAY be triggered for this IP to 779 either re-learn the local MAC and maintain local l3 and l2 780 reachability to this host OR to clear the ARP/ND entry in case 781 the host is indeed no longer local. Note that this accomplishes 782 clearing of stale ARP entries, triggered by a MAC age-out event 783 even when the ARP refresh timer was longer than the MAC age-out 784 timer. Clearing of stale IP neighbor entries in-turn facilitates 785 traffic convergence in the event that the host was silent and not 786 discovered at its new location. Once stale neighbor entry for the 787 host is cleared, routed traffic flow destined for the host can 788 re-trigger ARP/ND discovery for this host at the new location. 790 7.8.1 Generalized Probing Logic 792 Above probing logic may be generalized as probing for an IP neighbor 793 anytime a resolving parent MAC route is "inconsistent" with the MAC- 794 IP neighbor route, where being inconsistent is defined as being not 795 present OR conflicting in terms of the route source being local OR 796 remote. MAC-IP to MAC parent relationship described earlier in this 797 document in section 6.1 MAY be used to achieve this logic. 799 8. Routed Overlay 801 An additional use case is possible, such that traffic to an end host 802 in the overlay is always IP routed. In a purely routed overlay such 803 as this: 805 o A host MAC is never advertised in EVPN overlay control plane 807 o Host /32 or /128 IP reachability is distributed across the 808 overlay via EVPN route type 5 (RT-5) along with a zero or non- 809 zero ESI 811 o An overlay IP subnet may still be stretched across the underlay 812 fabric, however, intra-subnet traffic across the stretched 813 overlay is never bridged 815 o Both inter-subnet and intra-subnet traffic, in the overlay is 816 IP routed at the EVPN PE. 818 Please refer to [RFC 7814] for more details. 820 Host mobility within the stretched subnet would still need to be 821 supported for this use. In the absence of any host MAC routes, 822 sequence number mobility EXT-COMM specified in [RFC7432], section 7.7 823 may be associated with a /32 OR /128 host IP prefix advertised via 824 EVPN route type 5. MAC mobility procedures defined in RFC 7432 can 825 now be applied as is to host IP prefixes: 827 o On LOCAL learning of a host IP, on a new ESI, host IP MUST be 828 advertised with a sequence number attribute that is higher than 829 what is currently advertised with the old ESI 831 o on receiving a host IP route advertisement with a higher 832 sequence number, a PE MUST trigger ARP/ND probe and deletion 833 procedure on any LOCAL route for that IP with a lower sequence 834 number. A PE would essentially move the forwarding entry to point 835 to the remote route with a higher sequence number and send an 836 ARP/ND PROBE for the local IP route. If the IP has indeed moved, 837 PROBE would timeout and the local IP host route would be deleted. 839 Note that there is still only one sequence number associated with a 840 host route at any time. For earlier use cases where a host MAC is 841 advertised along with the host IP, a sequence number is only 842 associated with a MAC. Only if the MAC is not advertised at all, as 843 in this use case, is a sequence number associated with a host IP. 845 Note that this mobility procedure would not apply to "anycast IPv6" 846 hosts advertised via NA messages with 0-bit=0. Please refer to [EVPN- 847 PROXY-ARP]. 849 9. Duplicate Host Detection 851 Duplicate host detection scenarios across EVPN IRB can be classified 852 as follows: 854 o Scenario A: where two hosts have the same MAC (host IPs may or 855 may not be duplicate) 857 o Scenario B: where two hosts have the same IP but different MACs 859 o Scenario C: where two hosts have the same IP and host MAC is not 860 advertised at all 862 Duplicate detection procedures for scenario B and C would not apply 863 to "anycast IPv6" hosts advertised via NA messages with 0-bit=0. 864 Please refer to [EVPN-PROXY-ARP]. 866 9.1 Scenario A 868 For all use cases where duplicate hosts have the same MAC, MAC is 869 detected as duplicate via duplicate MAC detection procedure described 870 in RFC 7432. Corresponding MAC-IP routes with the same MAC do not 871 require duplicate detection and MUST simply inherit the DUPLICATE 872 property from the corresponding MAC route. In other words, if a MAC 873 route is in DUPLICATE state, all corresponding MAC-IP routes MUST 874 also be treated as DUPLICATE. Duplicate detection procedure need only 875 be applied to MAC routes. 877 9.2 Scenario B 878 Due to misconfiguration, a situation may arise where hosts with 879 different MACs are configured with the same IP. This scenario would 880 not be detected by existing duplicate MAC detection procedure and 881 would result in incorrect forwarding of routed traffic destined to 882 this IP. 884 Such a situation, on LOCAL MAC-IP learning, would be detected as a 885 move scenario via the following local MAC sequence number computation 886 procedure described earlier in section 5.1: 888 o If the IP is also associated with a different remote MAC "Mz", 889 MUST be higher than "Mz" sequence number 891 Such a move that results in sequence number increment on local MAC 892 because of a remote MAC-IP route associated with a different MAC MUST 893 be counted as an "IP move" against the "IP" independent of MAC. 894 Duplicate detection procedure described in RFC 7432 can now be 895 applied to an "IP" entity independent of MAC. Once an IP is detected 896 as DUPLICATE, corresponding MAC-IP route should be treated as 897 DUPLICATE. Associated MAC routes and any other MAC-IP routes 898 associated with this MAC should not be affected. 900 9.2.1 Duplicate IP Detection Procedure for Scenario B 902 Duplicate IP detection procedure for such a scenario is specified in 903 [EVPN-PROXY-ARP]. What counts as an "IP move" in this scenario is 904 further clarified as follows: 906 o On learning a LOCAL MAC-IP route Mx-IPx, check if there is an 907 existing REMOTE OR LOCAL route for IPx with a different MAC 908 association, say, Mz-IPx. If so, count this as an "IP move" count 909 for IPx, independent of the MAC 911 o On learning a REMOTE MAC-IP route Mz-IPx, check if there is an 912 existing LOCAL route for IPx with a different MAC association, 913 say, Mx-IPx. If so, count this as an "IP move" count for IPx, 914 independent of the MAC 916 A MAC-IP route SHOULD be treated as DUPLICATE if either of the 917 following two conditions are met: 919 o Corresponding MAC route is marked as DUPLICATE via existing 920 duplicate detection procedure 922 o Corresponding IP is marked as DUPLICATE via extended procedure 923 described above 925 9.3 Scenario C 927 For a purely routed overlay scenario described in section 8, where 928 only a host IP is advertised via EVPN RT-5, together with a sequence 929 number mobility attribute, duplicate MAC detection procedures 930 specified in RFC 7432 can be intuitively applied to IP only host 931 routes for the purpose of duplicate IP detection. 933 o On learning a LOCAL host IP route IPx, check if there is an 934 existing REMOTE OR LOCAL route for IPx with a different ESI 935 association. If so, count this as an "IP move" count for IPx. 937 o On learning a REMOTE host IP route IPx, check if there is an 938 existing LOCAL route for IPx with a different ESI association. If 939 so, count this as an "IP move" count for IPx 941 o With configurable parameters "N" and "M", If "N" IP moves are 942 detected within "M" seconds for IPx, treat IPx as DUPLICATE 944 9.4 Duplicate Host Recovery 946 Once a MAC or IP is marked as DUPLICATE and FROZEN, corrective action 947 must be taken to un-provision one of the duplicate MAC or IP. Un- 948 provisioning a duplicate MAC or IP in this context refers to a 949 corrective action taken on the host side. Once one of the duplicate 950 MAC or IP is un-provisioned, normal operation would not resume until 951 the duplicate MAC or IP ages out, following this correction, unless 952 additional action is taken to speed up recovery. 954 This section lists possible additional corrective actions that could 955 be taken to achieve faster recovery to normal operation. 957 9.4.1 Route Un-freezing Configuration 959 Unfreezing the DUPLICATE OR FROZEN MAC or IP via a CLI can be 960 leveraged to recover from DUPLICATE and FROZEN state following 961 corrective un-provisioning of the duplicate MAC or IP. 963 Unfreezing the frozen MAC or IP via a CLI at a PE should result in 964 that MAC OR IP being advertised with a sequence number that is higher 965 than the sequence number advertised from the other location of that 966 MAC or IP. 968 Two possible corrective un-provisioning scenarios exist: 970 o Scenario A: A duplicate MAC or IP may have been un-provisioned 971 at the location where it was NOT marked as DUPLICATE and FROZEN 973 o Scenario B: A duplicate MAC or IP may have been un-provisioned 974 at the location where it was marked as DUPLICATE and FROZEN 976 Unfreezing the DUPLICATE and FROZEN MAC or IP, following the above 977 corrective un-provisioning scenarios would result in recovery to 978 steady state as follows: 980 o Scenario A: If the duplicate MAC or IP was un-provisioned at 981 the location where it was NOT marked as DUPLICATE, unfreezing the 982 route at the FROZEN location will result in the route being 983 advertised with a higher sequence number. This would in-turn 984 result in automatic clearing of local route at the PE location, 985 where the host was un-provisioned via ARP/ND PROBE and DELETE 986 procedure specified earlier in section 8 and in [RFC 7432]. 988 o Scenario B: If the duplicate host is un-provisioned at the 989 location where it was marked as DUPLICATE, unfreezing the route 990 will trigger an advertisement with a higher sequence number to 991 the other location. This would in-turn trigger re-learning of 992 local route at the remote location, resulting in another 993 advertisement with a higher sequence number from the remote 994 location. Route at the local location would now be cleared on 995 receiving this remote route advertisement, following the ARP/ND 996 PROBE. 998 9.4.2 Route Clearing Configuration 1000 In addition to the above, route clearing CLIs may also be leveraged 1001 to clear the local MAC or IP route, to be executed AFTER the 1002 duplicate host is un-provisioned: 1004 o clear mac CLI: A clear MAC CLI can be leveraged to clear a 1005 DUPLICATE MAC route, to recover from a duplicate MAC scenario 1007 o clear ARP/ND: A clear ARP/ND CLI may be leveraged to clear a 1008 DUPLICATE IP route to recover from a duplicate IP scenario 1010 Note that the route unfreeze CLI may still need to be run if the 1011 route was un-provisioned and cleared from the NON-DUPLICATE / NON- 1012 FROZEN location. Given that unfreezing of the route via the un-freeze 1013 CLI would any ways result in auto-clearing of the route from the "un- 1014 provisioned" location, as explained in the prior section, need for a 1015 route clearing CLI for recovery from DUPLICATE / FROZEN state is 1016 truly optional. 1018 10. Security Considerations 1019 11. IANA Considerations 1021 12. References 1023 12.1 Normative References 1025 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 1026 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 1027 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 1028 2015, . 1030 [EVPN-PROXY-ARP] Rabadan et al., "Operational Aspects of Proxy- 1031 ARP/ND in EVPN Networks", draft-ietf-bess-evpn-proxy-arp- 1032 nd-06, work in progress, April 2019, 1033 . 1036 [EVPN-IRB] Sajassi et al., "Integrated Routing and Bridging in 1037 EVPN", draft-ietf-bess-evpn-inter-subnet-forwarding-08, 1038 work in progress, March 2019, 1039 . 1042 [RFC7814] Xu, X., Jacquenet, C., Raszuk, R., Boyes, T., Fee, B., 1043 "Virtual Subnet: A BGP/MPLS IP VPN-Based Subnet Extension 1044 Solution", RFC 7814, March 2016, 1045 . 1047 12.2 Informative References 1049 13. Acknowledgements 1051 Authors would like to thank Vibov Bhan and Patrice Brisset for 1052 feedback the process of design and implementation of procedures 1053 defined in this document. Authors would like to thank Wen Lin for a 1054 detailed review and valuable comments related to MAC sharing race 1055 conditions. 1057 Authors' Addresses 1059 Neeraj Malhotra (Editor) 1060 Cisco 1061 EMail: neeraj.ietf@gmail.com 1063 Ali Sajassi 1064 Cisco 1065 EMail: sajassi@cisco.com 1066 Aparna Pattekar 1067 Cisco 1068 Email: apjoshi@cisco.com 1070 Jorge Rabadan 1071 Nokia 1072 Email: jorge.rabadan@nokia.com 1074 Avinash Lingala 1075 AT&T 1076 Email: ar977m@att.com 1078 John Drake 1079 Juniper Networks 1080 EMail: jdrake@juniper.net 1082 Appendix A 1084 An alternative approach considered was to associate two independent 1085 sequence number attributes with MAC and IP components of a MAC-IP 1086 route. However, the approach of enabling IRB mobility procedures 1087 using a single sequence number associated with a MAC, as specified in 1088 this document was preferred for the following reasons: 1090 o Procedural overhead and complexity associated with maintaining 1091 two separate sequence numbers all the time, only to address 1092 scenarios with changing MAC-IP bindings is a big overhead for 1093 topologies where MAC-IP bindings never change. 1095 o Using a single sequence number associated with MAC is much 1096 simpler and adds no overhead for topologies where MAC-IP bindings 1097 never change. 1099 o Using a single sequence number associated with MAC is aligned 1100 with existing MAC mobility implementations. On other words, it is 1101 an easier implementation extension to existing MAC mobility 1102 procedure.