idnits 2.17.1 draft-ietf-bess-evpn-irb-extended-mobility-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 27, 2019) is 1850 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2119' is mentioned on line 188, but not defined == Missing Reference: 'VM-IP1' is mentioned on line 295, but not defined == Missing Reference: 'VM-IP2' is mentioned on line 295, but not defined == Missing Reference: 'VM-IP3' is mentioned on line 295, but not defined == Missing Reference: 'VM-IP4' is mentioned on line 295, but not defined == Missing Reference: 'VM-IP5' is mentioned on line 295, but not defined == Missing Reference: 'VM-IP6' is mentioned on line 295, but not defined == Missing Reference: 'IP1' is mentioned on line 369, but not defined == Missing Reference: 'MAC1' is mentioned on line 300, but not defined == Missing Reference: 'GW1' is mentioned on line 348, but not defined == Missing Reference: 'GW2' is mentioned on line 348, but not defined == Missing Reference: 'GW3' is mentioned on line 425, but not defined == Missing Reference: 'GW4' is mentioned on line 425, but not defined == Missing Reference: 'MAC2' is mentioned on line 304, but not defined == Missing Reference: 'VM-IP1-M1' is mentioned on line 346, but not defined == Missing Reference: 'VM-IP2-M2' is mentioned on line 346, but not defined == Missing Reference: 'VM-IP3-M3' is mentioned on line 346, but not defined == Missing Reference: 'VM-IP4-M4' is mentioned on line 346, but not defined == Missing Reference: 'VM-IP5-M5' is mentioned on line 346, but not defined == Missing Reference: 'VM-IP6-M6' is mentioned on line 346, but not defined == Missing Reference: 'IP7' is mentioned on line 367, but not defined == Missing Reference: 'M1' is mentioned on line 369, but not defined == Missing Reference: 'RFC 7814' is mentioned on line 729, but not defined == Missing Reference: 'RFC 7432' is mentioned on line 898, but not defined == Unused Reference: 'RFC7814' is defined on line 955, but no explicit reference was found in the text == Outdated reference: A later version (-16) exists of draft-ietf-bess-evpn-proxy-arp-nd-02 == Outdated reference: A later version (-15) exists of draft-ietf-bess-evpn-inter-subnet-forwarding-03 ** Downref: Normative reference to an Informational RFC: RFC 7814 Summary: 1 error (**), 0 flaws (~~), 28 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Working Group N. Malhotra, Ed. 3 INTERNET-DRAFT Arrcus 5 Intended Status: Proposed Standard A. Sajassi 6 A. Pattekar 7 (Cisco) 9 A. Lingala 10 AT&T 12 J. Rabadan 13 Nokia 15 J. Drake 16 Juniper Networks 18 Expires: Sept 28, 2019 March 27, 2019 20 Extended Mobility Procedures for EVPN-IRB 21 draft-ietf-bess-evpn-irb-extended-mobility-00 23 Abstract 25 The procedure to handle host mobility in a layer 2 Network with EVPN 26 control plane is defined as part of RFC 7432. EVPN has since evolved 27 to find wider applicability across various IRB use cases that include 28 distributing both MAC and IP reachability via a common EVPN control 29 plane. MAC Mobility procedures defined in RFC 7432 are extensible to 30 IRB use cases if a fixed 1:1 mapping between VM IP and MAC is assumed 31 across VM moves. Generic mobility support for IP and MAC that allows 32 these bindings to change across moves is required to support a 33 broader set of EVPN IRB use cases, and requires further 34 consideration. EVPN all-active multi-homing further introduces 35 scenarios that require additional consideration from mobility 36 perspective. Intent of this draft is to enumerate a set of design 37 considerations applicable to mobility across EVPN IRB use cases and 38 define generic sequence number assignment procedures to address these 39 IRB use cases. 41 Status of this Memo 43 This Internet-Draft is submitted to IETF in full conformance with the 44 provisions of BCP 78 and BCP 79. 46 Internet-Drafts are working documents of the Internet Engineering 47 Task Force (IETF), its areas, and its working groups. Note that 48 other groups may also distribute working documents as 49 Internet-Drafts. 51 Internet-Drafts are draft documents valid for a maximum of six months 52 and may be updated, replaced, or obsoleted by other documents at any 53 time. It is inappropriate to use Internet-Drafts as reference 54 material or to cite them other than as "work in progress." 56 The list of current Internet-Drafts can be accessed at 57 http://www.ietf.org/1id-abstracts.html 59 The list of Internet-Draft Shadow Directories can be accessed at 60 http://www.ietf.org/shadow.html 62 Copyright and License Notice 64 Copyright (c) 2017 IETF Trust and the persons identified as the 65 document authors. All rights reserved. 67 This document is subject to BCP 78 and the IETF Trust's Legal 68 Provisions Relating to IETF Documents 69 (http://trustee.ietf.org/license-info) in effect on the date of 70 publication of this document. Please review these documents 71 carefully, as they describe your rights and restrictions with respect 72 to this document. Code Components extracted from this document must 73 include Simplified BSD License text as described in Section 4.e of 74 the Trust Legal Provisions and are provided without warranty as 75 described in the Simplified BSD License. 77 Table of Contents 79 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 80 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 81 2. Optional MAC only RT-2 . . . . . . . . . . . . . . . . . . . . 5 82 3. Mobility Use Cases . . . . . . . . . . . . . . . . . . . . . . 6 83 3.1 VM MAC+IP Move . . . . . . . . . . . . . . . . . . . . . . 6 84 3.2 VM IP Move to new MAC . . . . . . . . . . . . . . . . . . . 6 85 3.2.1 VM Reload . . . . . . . . . . . . . . . . . . . . . . . 6 86 3.2.2 MAC Sharing . . . . . . . . . . . . . . . . . . . . . . 6 87 3.2.3 Problem . . . . . . . . . . . . . . . . . . . . . . . . 7 88 3.3 VM MAC move to new IP . . . . . . . . . . . . . . . . . . . 8 89 3.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . 8 90 4. EVPN All Active multi-homed ES . . . . . . . . . . . . . . . . 10 91 5. Design Considerations . . . . . . . . . . . . . . . . . . . . 11 92 6. Solution Components . . . . . . . . . . . . . . . . . . . . . 12 93 6.1 Sequence Number Inheritance . . . . . . . . . . . . . . . . 12 94 6.2 MAC Sharing . . . . . . . . . . . . . . . . . . . . . . . . 13 95 6.3 Multi-homing Mobility Synchronization . . . . . . . . . . . 14 96 7. Requirements for Sequence Number Assignment . . . . . . . . . 14 97 7.1 LOCAL MAC-IP learning . . . . . . . . . . . . . . . . . . . 14 98 7.2 LOCAL MAC learning . . . . . . . . . . . . . . . . . . . . 15 99 7.3 Remote MAC OR MAC-IP Update . . . . . . . . . . . . . . . . 15 100 7.4 REMOTE (SYNC) MAC update . . . . . . . . . . . . . . . . . 15 101 7.5 REMOTE (SYNC) MAC-IP update . . . . . . . . . . . . . . . . 16 102 7.6 Inter-op . . . . . . . . . . . . . . . . . . . . . . . . . 16 103 8. Routed Overlay . . . . . . . . . . . . . . . . . . . . . . . . 16 104 9. Duplicate Host Detection . . . . . . . . . . . . . . . . . . . 18 105 9.1 Scenario A . . . . . . . . . . . . . . . . . . . . . . . . . 18 106 9.2 Scenario B . . . . . . . . . . . . . . . . . . . . . . . . . 18 107 9.2.1 Duplicate IP Detection Procedure for Scenario B . . . . 19 108 9.3 Scenario C . . . . . . . . . . . . . . . . . . . . . . . . . 19 109 9.4 Duplicate Host Recovery . . . . . . . . . . . . . . . . . . 20 110 9.4.1 Route Un-freezing Configuration . . . . . . . . . . . . 20 111 9.4.2 Route Clearing Configuration . . . . . . . . . . . . . 21 112 10. Security Considerations . . . . . . . . . . . . . . . . . . . 21 113 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 114 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 115 12.1 Normative References . . . . . . . . . . . . . . . . . . . 21 116 12.2 Informative References . . . . . . . . . . . . . . . . . . 22 117 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 118 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22 119 Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 121 1 Introduction 123 EVPN-IRB enables capability to advertise both MAC and IP routes via a 124 single MAC+IP RT-2 advertisement. MAC is imported into local bridge 125 MAC table and enables L2 bridged traffic across the network overlay. 126 IP is imported into the local ARP table in an asymmetric IRB design 127 OR imported into the IP routing table in a symmetric IRB design, and 128 enables routed traffic across the layer 2 network overlay. Please 129 refer to [EVPN-INTER-SUBNET] more background on EVPN IRB forwarding 130 modes. 132 To support EVPN mobility procedure, a single sequence number mobility 133 attribute is advertised with the combined MAC+IP route. A single 134 sequence number advertised with the combined MAC+IP route to resolve 135 both MAC and IP reachability implicitly assumes a 1:1 fixed mapping 136 between IP and MAC. While a fixed 1:1 mapping between IP and MAC is a 137 common use case that could be addressed via existing MAC mobility 138 procedure, additional IRB scenarios need to be considered, that don't 139 necessarily adhere to this assumption. Following IRB mobility 140 scenarios are considered: 142 o VM move results in VM IP and MAC moving together 144 o VM move results in VM IP moving to a new MAC association 146 o VM move results in VM MAC moving to a new IP association 148 While existing MAC mobility procedure can be leveraged for MAC+IP 149 move in the first scenario, subsequent scenarios result in a new MAC- 150 IP association. As a result, a single sequence number assigned 151 independently per-[MAC, IP] is not sufficient to determine most 152 recent reachability for both MAC and IP, unless the sequence number 153 assignment algorithm is designed to allow for changing MAC-IP 154 bindings across moves. 156 Purpose of this draft is to define additional sequence number 157 assignment and handling procedures to adequately address generic 158 mobility support across EVPN-IRB overlay use cases that allow MAC-IP 159 bindings to change across VM moves and can support mobility for both 160 MAC and IP components carried in an EVPN RT-2 for these use cases. 162 In addition, for hosts on an ESI multi-homed to multiple GW devices, 163 additional procedure is proposed to ensure synchronized sequence 164 number assignments across the multi-homing devices. 166 Content presented in this draft is independent of data plane 167 encapsulation used in the overlay being MPLS or NVO Tunnels. It is 168 also largely independent of the EVPN IRB solution being based on 169 symmetric OR asymmetric IRB design as defined in [EVPN-INTER-SUBNET]. 170 In addition to symmetric and asymmetric IRB, mobility solution for a 171 routed overlay, where traffic to an end host in the overlay is always 172 IP routed using EVPN RT-5 is also presented in section 8. 174 To summarize, this draft covers mobility mobility for the following 175 independent of the overlay encapsulation being MPLS or an NVO Tunnel: 177 o Symmetric EVPN IRB overlay 179 o Asymmetric EVPN IRB overlay 181 o Routed EVPN overlay 183 1.1 Terminology 185 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 186 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 187 document are to be interpreted as described in RFC 2119 [RFC2119]. 189 o ARP is widely referred to in this document. This is simply for 190 ease of reading, and as such, these references are equally 191 applicable to ND (neighbor discovery) as well. 193 o GW: used widely in the document refers to an IRB GW that is 194 doing routing and bridging between an access network and an EVPN 195 enabled overlay network. 197 o RT-2: EVPN route type 2 carrying both MAC and IP reachability 199 o RT-5: EVPN route type 5 carrying IP prefix reachability 201 o ES: EVPN Ethernet Segment 203 o MAC-IP: IP association for a MAC, referred to in this document 204 may be IPv4, IPv6 or both. 206 2. Optional MAC only RT-2 208 In an EVPN IRB scenario, where a single MAC+IP RT-2 advertisement 209 carries both IP and MAC routes, a MAC only RT-2 advertisement is 210 redundant for host MACs that are advertised via MAC+IP RT-2. As a 211 result, a MAC only RT-2 is an optional route that may not be 212 advertised from or received at an IRB GW. This is an important 213 consideration for mobility scenarios discussed in subsequent 214 sections. 216 MAC only RT-2 may still be advertised for non-IP host MACs that are 217 not advertised via MAC+IP RT-2. 219 3. Mobility Use Cases 221 This section describes the IRB mobility use cases considered in this 222 document. Procedures to address them are covered later in section 6 223 and section 7. 225 o VM move results in VM IP and MAC moving together 227 o VM move results in VM IP moving to a new MAC association 229 o VM move results in VM MAC moving to a new IP association 231 3.1 VM MAC+IP Move 233 This is the baseline case, wherein a VM move results in both VM MAC 234 and IP moving together with no change in MAC-IP binding across a 235 move. Existing MAC mobility defined in RFC 7432 may be leveraged to 236 apply to corresponding MAC+IP route to support this mobility 237 scenario. 239 3.2 VM IP Move to new MAC 241 This is the case, where a VM move results in VM IP moving to a new 242 MAC binding. 244 3.2.1 VM Reload 246 A VM reload or an orchestrated VM move that results in VM being re- 247 spawned at a new location may result in VM getting a new MAC 248 assignment, while maintaining existing IP address. This results in a 249 VM IP move to a new MAC binding: 251 IP-a, MAC-a ---> IP-a, MAC-b 253 3.2.2 MAC Sharing 255 This takes into account scenarios, where multiple hosts, each with a 256 unique IP, may share a common MAC binding, and a host move results in 257 a new MAC binding for the host IP. 259 As an example, host VMs running on a single physical server, each 260 with a unique IP, may share the same physical server MAC. In yet 261 another scenario, an L2 access network may be behind a firewall, such 262 that all hosts IPs on the access network are learnt with a common 263 firewall MAC. In all such "shared MAC" use cases, multiple local MAC- 264 IP ARP entries may be learnt with the same MAC. A VM IP move, in such 265 scenarios (for e.g., to a new physical server), could result in new 266 MAC association for the VM IP. 268 3.2.3 Problem 270 In both of the above scenarios, a combined MAC+IP EVPN RT-2 271 advertised with a single sequence number attribute implicitly assumes 272 a fixed IP to MAC mapping. A host IP move to a new MAC breaks this 273 assumption and results in a new MAC+IP route. If this new MAC+IP 274 route is independently assigned a new sequence number, the sequence 275 number can no longer be used to determine most recent host IP 276 reachability in a symmetric EVPN-IRB design OR the most recent IP to 277 MAC binding in an asymmetric EVPN-IRB design. 279 +------------------------+ 280 | Underlay Network Fabric| 281 +------------------------+ 283 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 284 | GW1 | | GW2 | | GW3 | | GW4 | | GW5 | | GW6 | 285 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 286 \ / \ / \ / 287 \ ESI-1 / \ ESI-2 / \ ESI-3 / 288 \ / \ / \ / 289 +\---/+ +\---/+ +\---/+ 290 | \ / | | \ / | | \ / | 291 +--+--+ +--+--+ +--+--+ 292 | | | 293 Server-MAC1 Server-MAC2 Server-MAC3 294 | | | 295 [VM-IP1, VM-IP2] [VM-IP3, VM-IP4] [VM-IP5, VM-IP6] 297 Figure 1 299 As an example, consider a topology shown in Figure 1, with host VMs 300 sharing the physical server MAC. In steady state, [IP1, MAC1] route 301 is learnt at [GW1, GW2] and advertised to remote GWs with a sequence 302 number N. Now, VM-IP1 is moved to Server-MAC2. ARP or ND based local 303 learning at [GW3, GW4] would now result in a new [IP1, MAC2] route 304 being learnt. If route [IP1, MAC2] is learnt as a new MAC+IP route 305 and assigned a new sequence number of say 0, mobility procedure for 306 VM-IP1 will not trigger across the overlay network. 308 A clear sequence number assignment procedure needs to be defined to 309 unambiguously determine the most recent IP reachability, IP to MAC 310 binding, and MAC reachability for such a MAC sharing scenario. 312 3.3 VM MAC move to new IP 314 This is a scenario where host move or re-provisioning behind a new 315 gateway location may result in the same VM MAC getting a new IP 316 address assigned. 318 3.3.1 Problem 320 Complication with this scenario is that MAC reachability could be 321 carried via a combined MAC+IP route while a MAC only route may not be 322 advertised at all. A single sequence number association with the 323 MAC+IP route again implicitly assumes a fixed mapping between MAC and 324 IP. A MAC move resulting in a new IP association for the host MAC 325 breaks this assumption and results in a new MAC+IP route. If this new 326 MAC+IP route independently assumes a new sequence number, this 327 mobility attribute can no longer be used to determine most recent 328 host MAC reachability as opposed to the older existing MAC 329 reachability. 331 +------------------------+ 332 | Underlay Network Fabric| 333 +------------------------+ 334 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 335 | GW1 | | GW2 | | GW3 | | GW4 | | GW5 | | GW6 | 336 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 337 \ / \ / \ / 338 \ ESI-1 / \ ESI-2 / \ ESI-3 / 339 \ / \ / \ / 340 +\---/+ +\---/+ +\---/+ 341 | \ / | | \ / | | \ / | 342 +--+--+ +--+--+ +--+--+ 343 | | | 344 Server1 Server2 Server3 345 | | | 346 [VM-IP1-M1, VM-IP2-M2] [VM-IP3-M3, VM-IP4-M4] [VM-IP5-M5, VM-IP6-M6] 348 As an example, IP1-M1 is learnt locally at [GW1, GW2] and currently 349 advertised to remote hosts with a sequence number N. Consider a 350 scenario where a VM with MAC M1 is re-provisioned at server 2, 351 however, as part of this re-provisioning, assigned a different IP 352 address say IP7. [IP7, M1] is learnt as a new route at [GW3, GW4] and 353 advertised to remote GWs with a sequence number of 0. As a result, L3 354 reachability to IP7 would be established across the overlay, however, 355 MAC mobility procedure for MAC1 will not trigger as a result of this 356 MAC-IP route advertisement. If an optional MAC only route is also 357 advertised, sequence number associated with the MAC only route would 358 trigger MAC mobility as per [RFC7432]. However, in the absence of an 359 additional MAC only route advertisement, a single sequence number 360 advertised with a combined MAC+IP route would not be sufficient to 361 update MAC reachability across the overlay. 363 A MAC-IP sequence number assignment procedure needs to be defined to 364 unambiguously determine the most recent MAC reachability in such a 365 scenario without a MAC only route being advertised. 367 Further, GW1/GW2, on learning new reachability for [IP7, M1] via 368 GW3/GW4 MUST probe and delete any local IPs associated with MAC M1, 369 such as [IP1, M1] in the above example. 371 Arguably, MAC mobility sequence number defined in [RFC7432], could be 372 interpreted to apply only to the MAC part of MAC-IP route, and would 373 hence cover this scenario. It could hence be interpreted as a 374 clarification to [RFC7432] and one of the considerations for a common 375 sequence number assignment procedure across all MAC-IP mobility 376 scenarios detailed in this document. 378 4. EVPN All Active multi-homed ES 380 +------------------------+ 381 | Underlay Network Fabric| 382 +------------------------+ 384 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 385 | GW1 | | GW2 | | GW3 | | GW4 | | GW5 | | GW6 | 386 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 387 \ / \ / \ / 388 \ ESI-1 / \ ESI-2 / \ ESI-3 / 389 \ / \ / \ / 390 +\---/+ +\---/+ +\---/+ 391 | \ / | | \ / | | \ / | 392 +--+--+ +--+--+ +--+--+ 393 | | | 394 Server-1 Server-2 Server-3 396 Figure 2 398 Consider an EVPN-IRB overlay network shown in Figure 2, with hosts 399 multi-homed to two or more leaf GW devices via an all-active multi- 400 homed ES. MAC and ARP entries learnt on a local ESI may also be 401 synchronized across the multi-homing GW devices sharing this ESI. 402 This MAC and ARP SYNC enables local switching of intra and inter 403 subnet ECMP traffic flows from remote hosts. In other words, local 404 MAC and ARP entries on a given Ethernet segment (ES) may be learnt 405 via local learning and / or sync from another GW device sharing the 406 same ES. 408 For a host that is multi-homed to multiple GW devices via an all- 409 active ES interface, local learning of host MAC and MAC-IP at each GW 410 device is an independent asynchronous event, that is dependent on 411 traffic flow and or ARP / ND response from the host hashing to a 412 directly connected GW on the MC-LAG interface. As a result, sequence 413 number mobility attribute value assigned to a locally learnt MAC or 414 MAC-IP route (as per RFC 7432) at each device may not always be the 415 same, depending on transient states on the device at the time of 416 local learning. 418 As an example, consider a host VM that is deleted from ESI-2 and 419 moved to ESI-1. It is possible for host to be learnt on say, GW1 420 following deletion of the remote route from [GW3, GW4], while being 421 learnt on GW2 prior to deletion of remote route from [GW3, GW4]. If 422 so, GW1 would process local host route learning as a new route and 423 assign a sequence number of 0, while GW2 would process local host 424 route learning as a remote to local move and assign a sequence number 425 of N+1, N being the existing sequence number assigned at [GW3, GW4]. 426 Inconsistent sequence numbers advertised from multi-homing devices 427 introduces ambiguity with respect to sequence number based mobility 428 procedures across the overlay. 430 o Ambiguity with respect to how the remote ToRs should handle 431 paths with same ESI and different sequence numbers. A remote ToR 432 may not program ECMP paths if it receives routes with different 433 sequence numbers from a set of multi-homing GWs sharing the same 434 ESI. 436 o Breaks consistent route versioning across the network overlay 437 that is needed for EVPN mobility procedures to work. 439 As an example, in this inconsistent state, GW2 would drop a remote 440 route received for the same host with sequence number N (as its local 441 sequence number is N+1), while GW1 would install it as the best route 442 (as its local sequence number is 0). 444 There is need for a mechanism to ensure consistency of sequence 445 numbers advertised from a set of multi-homing devices for EVPN 446 mobility to work reliably. 448 In order to support mobility for multi-homed hosts using the sequence 449 number mobility attribute, local MAC and MAC-IP routes MUST be 450 advertised with the same sequence number by all GW devices that the 451 ESI is multi-homed to. In other words, there is need for a mechanism 452 to ensure consistency of sequence numbers advertised from a set of 453 multi-homing devices for EVPN mobility to work reliably. 455 5. Design Considerations 457 To summarize, sequence number assignment scheme and implementation 458 must take following considerations into account: 460 o MAC+IP may be learnt on an ESI multi-homed to multiple GW 461 devices, hence requires sequence numbers to be synchronized 462 across multi-homing GW devices. 464 o MAC only RT-2 is optional in an IRB scenario and may not 465 necessarily be advertised in addition to MAC+IP RT-2 467 o Single MAC may be associated with multiple IPs, i.e., multiple 468 host IPs may share a common MAC 470 o Host IP move could result in host moving to a new MAC, resulting 471 in a new IP to MAC association and a new MAC+IP route. 473 o Host MAC move to a new location could result in host MAC being 474 associated with a different IP address, resulting in a new MAC to 475 IP association and a new MAC+IP route 477 o LOCAL MAC-IP learn via ARP would always accompanied by a LOCAL 478 MAC learn event resulting from the ARP packet. MAC and MAC-IP 479 learning, however, could happen in any order 481 o Use cases discussed earlier that do not maintain a constant 1:1 482 MAC-IP mapping across moves could potentially be addressed by 483 using separate sequence numbers associated with MAC and IP 484 components of MAC+IP route. Maintaining two separate sequence 485 numbers however adds significant overhead with respect to 486 complexity, debugability, and backward compatibility. It is 487 therefore goal of solution presented here to address these 488 requirements via a single sequence number attribute. 490 6. Solution Components 492 This section goes over main components of the EVPN IRB mobility 493 solution proposed in this draft. Later sections will go over exact 494 sequence number assignment procedures resulting from concepts 495 described in this section. 497 6.1 Sequence Number Inheritance 499 Main idea presented here is to view a LOCAL MAC-IP route as a child 500 of the corresponding LOCAL MAC only route that inherits the sequence 501 number attribute from the parent LOCAL MAC only route: 503 Mx-IPx -----> Mx (seq# = N) 505 As a result, both parent MAC and child MAC-IP routes share one common 506 sequence number associated with the parent MAC route. Doing so 507 ensures that a single sequence number attribute carried in a combined 508 MAC+IP route represents sequence number for both a MAC only route as 509 well as a MAC+IP route, and hence makes the MAC only route truly 510 optional. As a result, optional MAC only route with its own sequence 511 number is not required to establish most recent reachability for a 512 MAC in the overlay network. Specifically, this enables a MAC to 513 assume a different IP address on a move, and still be able to 514 establish most recent reachability to the MAC across the overlay 515 network via mobility attribute associated with the MAC+IP route 516 advertisement. As an example, when Mx moves to a new location, it 517 would result in LOCAL Mx being assigned a higher sequence number at 518 its new location as per RFC 7432. If this move results in Mx assuming 519 a different IP address, IPz, LOCAL Mx+IPz route would inherit the new 520 sequence number from Mx. 522 LOCAL MAC and LOCAL MAC-IP routes would typically be sourced from 523 data plane learning and ARP learning respectively, and could get 524 learnt in control plane in any order. Implementation could either 525 replicate inherited sequence number in each MAC-IP entry OR maintain 526 a single attribute in the parent MAC by creating a forward reference 527 LOCAL MAC object for cases where a LOCAL MAC-IP is learnt before the 528 LOCAL MAC. 530 Arguably, this inheritance may be assumed from RFC 7432, in which 531 case, the above may be interpreted as a clarification with respect to 532 interpretation of a MAC sequence number in a MAC-IP route. 534 6.2 MAC Sharing 536 Further, for the shared MAC scenario, this would result in multiple 537 LOCAL MAC-IP siblings inheriting sequence number attribute from a 538 common parent MAC route: 540 Mx-IP1 ----- 541 | | 542 Mx-IP2 ----- 543 . | 544 . +---> Mx (seq# = N) 545 . | 546 Mx-IPw ----- 547 | | 548 Mx-IPx ----- 550 In such a case, a host-IP move to a different physical server would 551 result in IP moving to a new MAC binding. A new MAC-IP route 552 resulting from this move must now be advertised with a sequence 553 number that is higher than the previous MAC-IP route for this IP, 554 advertised from the prior location. As an example, consider a route 555 Mx-IPx that is currently advertised with sequence number N from GW1. 556 IPx moving to a new physical server behind GW2 results in IPx being 557 associated with MAC Mz. A new local Mz-IPx route resulting from this 558 move at GW2 must now be advertised with a sequence number higher than 559 N. This is so that GW devices, including GW1, GW2, and other remote 560 GW devices that are part of the overlay can clearly determine and 561 program the most recent MAC binding and reachability for the IP. GW1, 562 on receiving this new Mz-IPx route with sequence number say, N+1, for 563 symmetric IRB case, would update IPx reachability via GW2 in 564 forwarding, for asymmetric IRB case, would update IPx's ARP binding 565 to Mz. In addition, GW1 would clear and withdraw the stale Mx-IPx 566 route with the lower sequence number. 568 This also implies that sequence number associated with local MAC Mz 569 and all local MAC-IP children of Mz at GW2 must now be incremented to 570 N+1, and re-advertised across the overlay. While this re- 571 advertisement of all local MAC-IP children routes affected by the 572 parent MAC route is an overhead, it avoids the need for two separate 573 sequence number attributes to be maintained and advertised for IP and 574 MAC components of MAC+IP RT-2. Implementation would need to be able 575 to lookup MAC-IP routes for a given IP and update sequence number for 576 it's parent MAC and its MAC-IP children. 578 6.3 Multi-homing Mobility Synchronization 580 In order to support mobility for multi-homed hosts, local MAC and 581 MAC-IP routes learnt on the shared ESI MUST be advertised with the 582 same sequence number by all GW devices that the ESI is multi-homed 583 to. This also applies to local MAC only routes. LOCAL MAC and MAC-IP 584 may be learnt natively via data plane and ARP/ND respectively as well 585 as via SYNC from another multi-homing GW to achieve local switching. 586 Local and SYNC route learning can happen in any order. Local MAC-IP 587 routes advertised by all multi-homing GW devices sharing the ESI must 588 carry the same sequence number, independent of the order in which 589 they are learnt. This implies: 591 o On local or sync MAC-IP route learning, sequence number for the 592 local MAC-IP route MUST be compared and updated to the higher 593 value. 595 o On local or sync MAC route learning, sequence number for the 596 local MAC route MUST be compared and updated to the higher value. 598 If an update to local MAC-IP sequence number is required as a result 599 of above comparison with sync MAC-IP route, it would essentially 600 amount to a sequence number update on the parent local MAC, resulting 601 in the inherited sequence number update on the MAC-IP route. 603 7. Requirements for Sequence Number Assignment 605 Following sections summarize sequence number assignment procedure 606 needed on local and sync MAC and MAC-IP route learning events in 607 order to accomplish the above. 609 7.1 LOCAL MAC-IP learning 611 A local Mx-IPx learning via ARP or ND should result in computation OR 612 re-computation of parent MAC Mx's sequence number, following which 613 the MAC-IP route Mx-IPx would simply inherit parent MAC's sequence 614 number. Parent MAC Mx Sequence number should be computed as follows: 616 o MUST be higher than any existing remote MAC route for Mx, as per 617 RFC 7432. 619 o MUST be at least equal to corresponding SYNC MAC sequence number 620 if one is present. 622 o If the IP is also associated with a different remote MAC "Mz", 623 MUST be higher than "Mz" sequence number 625 Once new sequence number for MAC route Mx is computed as per above, 626 all LOCAL MAC-IPs associated with MAC Mx MUST inherit the updated 627 sequence number. 629 7.2 LOCAL MAC learning 631 Local MAC Mx Sequence number should be computed as follows: 633 o MUST be higher than any existing remote MAC route for Mx, as per 634 RFC 7432. 636 o MUST be at least equal to corresponding SYNC MAC sequence number 637 if one is present. 639 o Once new sequence number for MAC route Mx is computed as per 640 above, all LOCAL MAC-IPs associated with MAC Mx MUST inherit the 641 updated sequence number. 643 Note that the local MAC sequence number might already be present if 644 there was a local MAC-IP learnt prior to the local MAC, in which case 645 the above may not result in any change in local MAC's sequence 646 number. 648 7.3 Remote MAC OR MAC-IP Update 650 On receiving a remote MAC OR MAC-IP route update associated with a 651 MAC Mx with a sequence number that is higher than a LOCAL route for 652 MAC Mx: 654 o GW MUST trigger probe and deletion procedure for all LOCAL IPs 655 associated with MAC Mx 657 o GW MUST trigger deletion procedure for LOCAL MAC route for Mx 659 7.4 REMOTE (SYNC) MAC update 661 Corresponding local MAC Mx (if present) Sequence number should be re- 662 computed as follows: 664 o If the current sequence number is less than the received SYNC 665 MAC sequence number, it MUST be increased to be equal to received 666 SYNC MAC sequence number. 668 o If a LOCAL MAC sequence number is updated as a result of the 669 above, all LOCAL MAC-IPs associated with MAC Mx MUST inherit the 670 updated sequence number. 672 7.5 REMOTE (SYNC) MAC-IP update 674 If this is a SYNCed MAC-IP on a local ESI, it would also result in a 675 derived SYNC MAC Mx route entry, as MAC only RT-2 advertisement is 676 optional. Corresponding local MAC Mx (if present) Sequence number 677 should be re-computed as follows: 679 o If the current sequence number is less than the received SYNC 680 MAC sequence number, it MUST be increased to be equal to received 681 SYNC MAC sequence number. 683 o If a LOCAL MAC sequence number is updated as a result of the 684 above, all LOCAL MAC-IPs associated with MAC Mx MUST inherit the 685 updated sequence number. 687 7.6 Inter-op 689 In general, if all GW nodes in the overlay network follow the above 690 sequence number assignment procedure, and the GW is advertising both 691 MAC+IP and MAC routes, sequence number advertised with the MAC and 692 MAC+IP routes with the same MAC would always be the same. However, an 693 inter-op scenario with a different implementation could arise, where 694 a GW implementation non-compliant with this document or with RFC 7432 695 assigns and advertises independent sequence numbers to MAC and MAC+IP 696 routes. To handle this case, if different sequence numbers are 697 received for remote MAC+IP and corresponding remote MAC routes from a 698 remote GW, sequence number associated with the remote MAC route 699 should be computed as: 701 o Highest of the all received sequence numbers with remote MAC+IP 702 and MAC routes with the same MAC. 704 o MAC sequence number would be re-computed on a MAC or MAC+IP 705 route withdraw as per above. 707 A MAC and / or IP move to the local GW would now result in the MAC 708 (and hence all MAC-IP) sequence numbers incremented from the above 709 computed remote MAC sequence number. 711 8. Routed Overlay 712 An additional use case is possible, such that traffic to an end host 713 in the overlay is always IP routed. In a purely routed overlay such 714 as this: 716 o A host MAC is never advertised in EVPN overlay control plane 718 o Host /32 or /128 IP reachability is distributed across the 719 overlay via EVPN route type 5 (RT-5) along with a zero or non- 720 zero ESI 722 o An overlay IP subnet may still be stretched across the underlay 723 fabric, however, intra-subnet traffic across the stretched 724 overlay is never bridged 726 o Both inter-subnet and intra-subnet traffic, in the overlay is 727 IP routed at the EVPN GW. 729 Please refer to [RFC 7814] for more details. 731 Host mobility within the stretched subnet would still need to be 732 supported for this use. In the absence of any host MAC routes, 733 sequence number mobility EXT-COMM specified in [RFC7432], section 7.7 734 may be associated with a /32 OR /128 host IP prefix advertised via 735 EVPN route type 5. MAC mobility procedures defined in RFC 7432 can 736 now be applied as is to host IP prefixes: 738 o On LOCAL learning of a host IP, on a new ESI, host IP MUST be 739 advertised with a sequence number attribute that is higher than 740 what is currently advertised with the old ESI 742 o on receiving a host IP route advertisement with a higher 743 sequence number, a PE MUST trigger ARP/ND probe and deletion 744 procedure on any LOCAL route for that IP with a lower sequence 745 number. A PE would essentially move the forwarding entry to point 746 to the remote route with a higher sequence number and send an 747 ARP/ND PROBE for the local IP route. If the IP has indeed moved, 748 PROBE would timeout and the local IP host route would be deleted. 750 Note that there is still only one sequence number associated with a 751 host route at any time. For earlier use cases where a host MAC is 752 advertised along with the host IP, a sequence number is only 753 associated with a MAC. Only if the MAC is not advertised at all, as 754 in this use case, is a sequence number associated with a host IP. 756 Note that this mobility procedure would not apply to "anycast IPv6" 757 hosts advertised via NA messages with 0-bit=0. Please refer to [EVPN- 758 PROXY-ARP]. 760 9. Duplicate Host Detection 762 Duplicate host detection scenarios across EVPN IRB can be classified 763 as follows: 765 o Scenario A: where two hosts have the same MAC (host IPs may or 766 may not be duplicate) 768 o Scenario B: where two hosts have the same IP but different MACs 770 o Scenario C: where two hosts have the same IP and host MAC is not 771 advertised at all 773 Duplicate detection procedures for scenario B and C would not apply 774 to "anycast IPv6" hosts advertised via NA messages with 0-bit=0. 775 Please refer to [EVPN-PROXY-ARP]. 777 9.1 Scenario A 779 For all use cases where duplicate hosts have the same MAC, MAC is 780 detected as duplicate via duplicate MAC detection procedure described 781 in RFC 7432. Corresponding MAC-IP routes with the same MAC do not 782 require duplicate detection and MUST simply inherit the DUPLICATE 783 property from the corresponding MAC route. In other words, if a MAC 784 route is in DUPLICATE state, all corresponding MAC-IP routes MUST 785 also be treated as DUPLICATE. Duplicate detection procedure need only 786 be applied to MAC routes. 788 9.2 Scenario B 790 Due to misconfiguration, a situation may arise where hosts with 791 different MACs are configured with the same IP. This scenario would 792 not be detected by existing duplicate MAC detection procedure and 793 would result in incorrect forwarding of routed traffic destined to 794 this IP. 796 Such a situation, on LOCAL MAC-IP learning, would be detected as a 797 move scenario via the following local MAC sequence number computation 798 procedure described earlier in section 5.1: 800 o If the IP is also associated with a different remote MAC "Mz", 801 MUST be higher than "Mz" sequence number 803 Such a move that results in sequence number increment on local MAC 804 because of a remote MAC-IP route associated with a different MAC MUST 805 be counted as an "IP move" against the "IP" independent of MAC. 806 Duplicate detection procedure described in RFC 7432 can now be 807 applied to an "IP" entity independent of MAC. Once an IP is detected 808 as DUPLICATE, corresponding MAC-IP route should be treated as 809 DUPLICATE. Associated MAC routes and any other MAC-IP routes 810 associated with this MAC should not be affected. 812 9.2.1 Duplicate IP Detection Procedure for Scenario B 814 Duplicate IP detection procedure for such a scenario is specified in 815 [EVPN-PROXY-ARP]. What counts as an "IP move" in this scenario is 816 further clarified as follows: 818 o On learning a LOCAL MAC-IP route Mx-IPx, check if there is an 819 existing REMOTE OR LOCAL route for IPx with a different MAC 820 association, say, Mz-IPx. If so, count this as an "IP move" count 821 for IPx, independent of the MAC 823 o On learning a REMOTE MAC-IP route Mz-IPx, check if there is an 824 existing LOCAL route for IPx with a different MAC association, 825 say, Mx-IPx. If so, count this as an "IP move" count for IPx, 826 independent of the MAC 828 A MAC-IP route SHOULD be treated as DUPLICATE if either of the 829 following two conditions are met: 831 o Corresponding MAC route is marked as DUPLICATE via existing 832 duplicate detection procedure 834 o Corresponding IP is marked as DUPLICATE via extended procedure 835 described above 837 9.3 Scenario C 839 For a purely routed overlay scenario described in section 8, where 840 only a host IP is advertised via EVPN RT-5, together with a sequence 841 number mobility attribute, duplicate MAC detection procedures 842 specified in RFC 7432 can be intuitively applied to IP only host 843 routes for the purpose of duplicate IP detection. 845 o On learning a LOCAL host IP route IPx, check if there is an 846 existing REMOTE OR LOCAL route for IPx with a different ESI 847 association. If so, count this as an "IP move" count for IPx. 849 o On learning a REMOTE host IP route IPx, check if there is an 850 existing LOCAL route for IPx with a different ESI association. If 851 so, count this as an "IP move" count for IPx 853 o With configurable parameters "N" and "M", If "N" IP moves are 854 detected within "M" seconds for IPx, treat IPx as DUPLICATE 856 9.4 Duplicate Host Recovery 858 Once a MAC or IP is marked as DUPLICATE and FROZEN, corrective action 859 must be taken to un-provision one of the duplicate MAC or IP. Un- 860 provisioning a duplicate MAC or IP in this context refers to a 861 corrective action taken on the host side. Once one of the duplicate 862 MAC or IP is un-provisioned, normal operation would not resume until 863 the duplicate MAC or IP ages out, following this correction, unless 864 additional action is taken to speed up recovery. 866 This section lists possible additional corrective actions that could 867 be taken to achieve faster recovery to normal operation. 869 9.4.1 Route Un-freezing Configuration 871 Unfreezing the DUPLICATE OR FROZEN MAC or IP via a CLI can be 872 leveraged to recover from DUPLICATE and FROZEN state following 873 corrective un-provisioning of the duplicate MAC or IP. 875 Unfreezing the frozen MAC or IP via a CLI at a GW should result in 876 that MAC OR IP being advertised with a sequence number that is higher 877 than the sequence number advertised from the other location of that 878 MAC or IP. 880 Two possible corrective un-provisioning scenarios exist: 882 o Scenario A: A duplicate MAC or IP may have been un-provisioned 883 at the location where it was NOT marked as DUPLICATE and FROZEN 885 o Scenario B: A duplicate MAC or IP may have been un-provisioned 886 at the location where it was marked as DUPLICATE and FROZEN 888 Unfreezing the DUPLICATE and FROZEN MAC or IP, following the above 889 corrective un-provisioning scenarios would result in recovery to 890 steady state as follows: 892 o Scenario A: If the duplicate MAC or IP was un-provisioned at 893 the location where it was NOT marked as DUPLICATE, unfreezing the 894 route at the FROZEN location will result in the route being 895 advertised with a higher sequence number. This would in-turn 896 result in automatic clearing of local route at the GW location, 897 where the host was un-provisioned via ARP/ND PROBE and DELETE 898 procedure specified earlier in section 8 and in [RFC 7432]. 900 o Scenario B: If the duplicate host is un-provisioned at the 901 location where it was marked as DUPLICATE, unfreezing the route 902 will trigger an advertisement with a higher sequence number to 903 the other location. This would in-turn trigger re-learning of 904 local route at the remote location, resulting in another 905 advertisement with a higher sequence number from the remote 906 location. Route at the local location would now be cleared on 907 receiving this remote route advertisement, following the ARP/ND 908 PROBE. 910 9.4.2 Route Clearing Configuration 912 In addition to the above, route clearing CLIs may also be leveraged 913 to clear the local MAC or IP route, to be executed AFTER the 914 duplicate host is un-provisioned: 916 o clear mac CLI: A clear MAC CLI can be leveraged to clear a 917 DUPLICATE MAC route, to recover from a duplicate MAC scenario 919 o clear ARP/ND: A clear ARP/ND CLI may be leveraged to clear a 920 DUPLICATE IP route to recover from a duplicate IP scenario 922 Note that the route unfreeze CLI may still need to be run if the 923 route was un-provisioned and cleared from the NON-DUPLICATE / NON- 924 FROZEN location. Given that unfreezing of the route via the un-freeze 925 CLI would any ways result in auto-clearing of the route from the "un- 926 provisioned" location, as explained in the prior section, need for a 927 route clearing CLI for recovery from DUPLICATE / FROZEN state is 928 truly optional. 930 10. Security Considerations 932 11. IANA Considerations 934 12. References 936 12.1 Normative References 938 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 939 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 940 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 941 2015, . 943 [EVPN-PROXY-ARP] Rabadan et al., "Operational Aspects of Proxy- 944 ARP/ND in EVPN Networks", draft-ietf-bess-evpn-proxy-arp- 945 nd-02, work in progress, April 2017, 946 . 949 [EVPN-INTER-SUBNET] Sajassi et al., "Integrated Routing and Bridging 950 in EVPN", draft-ietf-bess-evpn-inter-subnet-forwarding-03, 951 work in progress, Feb 2017, 952 . 955 [RFC7814] Xu, X., Jacquenet, C., Raszuk, R., Boyes, T., Fee, B., 956 "Virtual Subnet: A BGP/MPLS IP VPN-Based Subnet Extension 957 Solution", RFC 7814, March 2016, 958 . 960 12.2 Informative References 962 13. Acknowledgements 964 Authors would like to thank Vibov Bhan and Patrice Brisset for 965 feedback and comments through the process. 967 Authors' Addresses 969 Neeraj Malhotra (Editor) 970 Arrcus 971 EMail: neeraj.ietf@gmail.com 973 Ali Sajassi 974 Cisco 975 EMail: sajassi@cisco.com 977 Aparna Pattekar 978 Cisco 979 Email: apjoshi@cisco.com 981 Jorge Rabadan 982 Nokia 983 Email: jorge.rabadan@nokia.com 985 Avinash Lingala 986 AT&T 987 Email: ar977m@att.com 989 John Drake 990 Juniper Networks 991 EMail: jdrake@juniper.net 993 Appendix A 995 An alternative approach considered was to associate two independent 996 sequence number attributes with MAC and IP components of a MAC-IP 997 route. However, the approach of enabling IRB mobility procedures 998 using a single sequence number associated with a MAC, as specified in 999 this document was preferred for the following reasons: 1001 o Procedural overhead and complexity associated with maintaining 1002 two separate sequence numbers all the time, only to address 1003 scenarios with changing MAC-IP bindings is a big overhead for 1004 topologies where MAC-IP bindings never change. 1006 o Using a single sequence number associated with MAC is much 1007 simpler and adds no overhead for topologies where MAC-IP bindings 1008 never change. 1010 o Using a single sequence number associated with MAC is aligned 1011 with existing MAC mobility implementations. On other words, it is 1012 an easier implementation extension to existing MAC mobility 1013 procedure.