idnits 2.17.1 draft-snr-bess-evpn-proxy-arp-nd-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC6820], [RFC7432]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 6, 2015) is 3216 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC7432' is mentioned on line 604, but not defined == Missing Reference: 'RFC6820' is mentioned on line 175, but not defined == Missing Reference: 'RFC4861' is mentioned on line 493, but not defined == Missing Reference: 'RFC5227' is mentioned on line 209, but not defined == Missing Reference: 'EVPN-NA-FLAGS' is mentioned on line 447, but not defined == Missing Reference: 'RFC0826' is mentioned on line 493, but not defined == Missing Reference: 'RFC3971' is mentioned on line 500, but not defined == Missing Reference: 'RFC2119' is mentioned on line 840, but not defined == Unused Reference: 'EVPN-ND-FLAGS' is defined on line 913, but no explicit reference was found in the text == Unused Reference: 'Euro-IX BCP' is defined on line 917, but no explicit reference was found in the text == Outdated reference: A later version (-07) exists of draft-snr-bess-evpn-na-flags-02 Summary: 1 error (**), 0 flaws (~~), 12 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Workgroup J. Rabadan, Ed. 3 Internet Draft S. Sathappan 4 K. Nagaraj 5 Intended status: Informational W. Henderickx 6 G. Hankins 7 Alcatel-Lucent 9 T. King 10 D. Melzer 11 DE-CIX 13 Expires: January 7, 2016 July 6, 2015 15 Operational Aspects of Proxy-ARP/ND in EVPN Networks 16 draft-snr-bess-evpn-proxy-arp-nd-01 18 Abstract 20 The MAC/IP Advertisement route specified in [RFC7432] can optionally 21 carry IPv4 and IPv6 addresses associated with a MAC address. Remote 22 PEs can use this information to reply locally (act as proxy) to IPv4 23 ARP requests and IPv6 Neighbor Solicitation messages and 24 reduce/suppress the flooding produced by the Address Resolution 25 procedure. This EVPN capability is extremely useful in Internet 26 Exchange Points (IXPs) and Data Centers (DCs) with large broadcast 27 domains, where the amount of ARP/ND flooded traffic causes issues on 28 routers and CEs, as explained in [RFC6820]. This document describes 29 how the [RFC7432] EVPN proxy-ARP/ND function may be implemented to 30 help IXPs and other operators deal with the issues derived from 31 Address Resolution in large broadcast domains. 33 Status of this Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF), its areas, and its working groups. Note that 40 other groups may also distribute working documents as Internet- 41 Drafts. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." The list 47 of current Internet-Drafts can be accessed at 48 http://www.ietf.org/ietf/1id-abstracts.txt 50 The list of Internet-Draft Shadow Directories can be accessed at 51 http://www.ietf.org/shadow.html 53 This Internet-Draft will expire on January 7, 2015. 55 Copyright Notice 57 Copyright (c) 2015 IETF Trust and the persons identified as the 58 document authors. All rights reserved. 60 This document is subject to BCP 78 and the IETF Trust's Legal 61 Provisions Relating to IETF Documents 62 (http://trustee.ietf.org/license-info) in effect on the date of 63 publication of this document. Please review these documents 64 carefully, as they describe your rights and restrictions with respect 65 to this document. Code Components extracted from this document must 66 include Simplified BSD License text as described in Section 4.e of 67 the Trust Legal Provisions and are provided without warranty as 68 described in the Simplified BSD License. 70 Table of Contents 72 1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3 73 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 74 2.1. The DC Use-Case . . . . . . . . . . . . . . . . . . . . . . 4 75 2.2. The IXP Use-Case . . . . . . . . . . . . . . . . . . . . . 4 76 3. Solution Requirements . . . . . . . . . . . . . . . . . . . . . 5 77 4. Solution Description . . . . . . . . . . . . . . . . . . . . . 6 78 4.1. Learning Sub-Function . . . . . . . . . . . . . . . . . . . 8 79 4.1.1. Proxy-ND and the NA Flags . . . . . . . . . . . . . . . 10 80 4.2. Reply Sub-Function . . . . . . . . . . . . . . . . . . . . 11 81 4.3. Maintenance Sub-Function . . . . . . . . . . . . . . . . . 12 82 4.4. Flooding (to Remote PEs) Reduction/Suppression . . . . . . 13 83 4.5. Duplicate IP Detection . . . . . . . . . . . . . . . . . . 13 84 5. Solution Benefits . . . . . . . . . . . . . . . . . . . . . . . 15 85 6. Deployment Scenarios . . . . . . . . . . . . . . . . . . . . . 16 86 6.1. All Dynamic Learning . . . . . . . . . . . . . . . . . . . 16 87 6.2. Dynamic Learning with Proxy-ARP/ND . . . . . . . . . . . . 16 88 6.3. Hybrid Dynamic Learning and Static Provisioning with 89 Proxy-ARP/ND . . . . . . . . . . . . . . . . . . . . . . . 16 90 6.4 All Static Provisioning with Proxy-ARP/ND . . . . . . . . . 17 91 6.5 Deployment Scenarios in IXPs . . . . . . . . . . . . . . . . 17 92 6.6 Deployment Scenarios in DCs . . . . . . . . . . . . . . . . 18 93 7. Conventions Used in this Document . . . . . . . . . . . . . . . 18 94 8. Security Considerations . . . . . . . . . . . . . . . . . . . . 19 95 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 19 96 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 19 97 10.1. Normative References . . . . . . . . . . . . . . . . . . . 19 98 10.2. Informative References . . . . . . . . . . . . . . . . . . 20 99 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 20 100 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 102 1. Terminology 104 BUM: Broadcast, Unknown unicast and Multicast layer-2 traffic. 106 ARP: Address Resolution Protocol. 108 GARP: Gratuitous ARP message. 110 ND: Neighbor Discovery Protocol. 112 NS: Neighbor Solicitation message. 114 NA: Neighbor Advertisement. 116 IXP: Internet eXchange Point. 118 IXP-LAN: it refers to the IXP's large Broadcast Domain to where 119 Internet routers are connected. 121 DC: Data Center. 123 IP->MAC: it refers to an IP address associated to a MAC address. The 124 entries may be of three different types: dynamic, static or EVPN- 125 learned. 127 SN-multicast address: Refers to the Solicited-Node IPv6 multicast 128 address used by NS messages. 130 NUD: Neighbor Unreachability Detection, as per [RFC4861]. 132 DAD: Duplicate Address Detection, as per [RFC4861]. 134 SLLA: Source Link Layer Address, as per [RFC4861]. 136 TLLA: Target Link Layer Address, as per [RFC4861]. 138 R-bit: Router Flag in NA messages, as per [RFC4861]. 140 O-bit: Override Flag in NA messages, as per [RFC4861]. 142 S-bit: Solicited Flag in NA messages, as per [RFC4861]. 144 RT2: EVPN Route type 2 or MAC/IP Advertisement route, as per 145 [RFC7432]. 147 MAC or IP DA: MAC or IP Destination Address. 149 MAC or IP SA: MAC or IP Source Address. 151 AS-MAC: Anti-spoofing MAC. 153 2. Introduction 155 As specified in [RFC7432] the IP Address field in the MAC/IP 156 Advertisement route may optionally carry one of the IP addresses 157 associated with the MAC address. A PE may learn local IP->MAC pairs 158 and advertise them in EVPN MAC/IP routes. The remote PEs may add 159 those IP->MAC pairs to their Proxy-ARP/ND tables and reply to local 160 ARP requests or Neighbor Solicitations, reducing and even suppressing 161 in some cases the flooding in the EVPN network. 163 EVPN and its associated Proxy-ARP/ND function are extremely useful in 164 Data Centers (DCs) or Internet Exchange Points (IXPs) with large 165 broadcast domains, where the amount of ARP/ND flooded traffic causes 166 issues on routers and CEs. [RFC6820] describes the Address Resolution 167 problems in Large Data Center networks. 169 This document describes how the [RFC7432] proxy-ARP/ND function may 170 be implemented to help IXPs, DCs and other operators deal with the 171 issues derived from Address Resolution in large broadcast domains. 173 2.1. The DC Use-Case 175 As described in [RFC6820] the IPv4 and IPv6 Address Resolution can 176 create a lot of issues in large DCs. The amount of flooding that 177 Address Resolution creates, as well as other associated issues can be 178 mitigated with the use of EVPN and its proxy-ARP/ND function. 180 2.2. The IXP Use-Case 182 The implementation described in this document is especially useful in 183 IXP networks. 185 A typical IXP provides access to a large layer-2 peering network, 186 where (hundreds of) Internet routers are connected. Because of the 187 requirement to connect all routers to a single layer-2 network the 188 peering networks use IPv4 layer-3 addresses in length ranges from /21 189 to /24, which can create very large broadcast domains. This peering 190 network is transparent to the Customer Edge (CE) devices and 191 therefore floods any ARP request or NS messages to all the CEs in the 192 network. Unsolicited GARP and NA messages are flooded to all the CEs 193 too. 195 In these IXP networks, most of the CEs are typically peering routers 196 and roughly all the BUM traffic is originated by the ARP and ND 197 address resolution procedures. This ARP/ND BUM traffic causes 198 significant data volumes that reach every single router in the 199 peering network. Since the ARP/ND messages are processed in software 200 processors and they take high priority in the routers, heavy loads of 201 ARP/ND traffic can cause some routers to run out of resources. CEs 202 disappearing from the network may cause Address Resolution explosions 203 that can make a router with limited processing power fail to keep BGP 204 sessions running. 206 The issue may be better in IPv6 routers, since ND uses SN-multicast 207 address in NS messages, however ARP uses broadcast and has to be 208 processed by all the routers in the network. Some routers may also be 209 configured to broadcast periodic GARPs [RFC5227]. The amount of 210 ARP/ND flooded traffic grows exponentially with the number of IXP 211 participants, therefore the issue can only go worse as new CEs are 212 added. 214 In order to deal with this issue, IXPs have developed certain 215 solutions over the past years. One example is the ARP-Sponge daemon 216 [ARP-Sponge]. While these solutions may mitigate the issues of 217 Address Resolution in large broadcasts domains, EVPN provides new 218 more efficient possibilities to IXPs. EVPN and its proxy-ARP/ND 219 function may help solve the issue in a distributed and scalable way, 220 fully integrated with the PE network. 222 3. Solution Requirements 224 The distributed EVPN proxy-ARP/ND function described in this document 225 SHOULD meet the following requirements: 227 o The solution SHOULD support the learning of the CE IP->MAC entries 228 on the EVPN PEs via the management, control or data planes. An 229 implementation SHOULD allow to intentionally enable or disable 230 those possible learning mechanisms. 232 o The solution MAY suppress completely the flooding of the ARP/ND 233 messages in the EVPN network, assuming that all the CE IP->MAC 234 addresses local to the PEs are known or provisioned on the PEs from 235 a management system. Note that in this case, the unknown unicast 236 traffic can also be suppressed, since all the expected unicast 237 traffic will be destined to known MAC addresses in the PE MAC-VRFs. 239 o The solution MAY reduce significantly the flooding of the ARP/ND 240 messages in the EVPN network, assuming that some or all the CE 241 IP->MAC addresses are learned on the data plane by snooping ARP/ND 242 messages issued by the CEs. 244 o The solution MAY provide a way to refresh periodically the CE 245 IP->MAC entries learned through the data plane, so that the IP->MAC 246 entries are not withdrawn by EVPN when they age out unless the CE 247 is not active anymore. This option helps reducing the EVPN control 248 plane overhead in a network with active CEs that do not send 249 packets frequently. 251 o The solution SHOULD provide a mechanism to detect duplicate IP 252 addresses. In case of duplication, the detecting PE should not 253 reply to requests for the duplicate IP. Instead, the PE should 254 alert the operator and may optionally prevent any other CE from 255 sending traffic to the duplicate IP. 257 o The solution MUST NOT change any existing behavior in the CEs 258 connected to the EVPN PEs. 260 4. Solution Description 262 Figure 1 illustrates an example EVPN network where the Proxy-ARP/ND 263 function is enabled. 265 MAC-VRF1 266 Proxy-ARP/ND 267 +------------+ 268 IP1/M1 +----------------------------+ |IP1->M1 EVPN| 269 GARP --->Proxy-ARP/ND | |IP2->M2 EVPN| 270 +---+ +----+---+ RT2(IP1/M1) | |IP3->M3 sta | 271 |CE1+------+MAC-VRF1| ------> +------+---|IP4->M4 dyn | 272 +---+ +--------+ | +------------+ 273 PE1 | +--------+ Who has IP1? 274 | EVPN | |MAC-VRF1| <----- +---+ 275 | EVI1 | | | | |CE3| 276 IP2/M2 | | | | -----> +---+ 277 GARP --->Proxy-ARP/ND | +--------+ | IP1->M1 278 +---+ +--------+ RT2(IP2/M2) | | 279 |CE2+----+MAC-VRF1| ------> +--------------+ 280 +---+ +--------+ PE3| +---+ 281 PE2 | +----+CE4| 282 +----------------------------+ +---+ 283 <---IP4/M4 GARP 285 Figure 1 Proxy-ARP/ND network example 287 When the Proxy-ARP/ND function is enabled in the MAC-VRFs of the EVPN 288 PEs, each PE creates a Proxy table specific to that MAC-VRF that can 289 contain three types of Proxy-ARP/ND entries: 291 a) Dynamic entries: learned by snooping CE's ARP and ND messages. For 292 instance, IP4->M4 in Figure 1. 294 b) Static entries: provisioned on the PE by the management system. 295 For instance, IP3->M3 in Figure 1. 297 c) EVPN-learned entries: learned from the IP/MAC information encoded 298 in the received RT2's coming from remote PEs. For instance, IP1- 299 >M1 and IP2->M2 in Figure 1. 301 As a high level example, the operation of the EVPN Proxy-ARP/ND 302 function in the network of Figure 1 is described below. In this 303 example we assume IP1, IP2 and IP3 are IPv4 addresses: 305 1. Proxy-ARP/ND is enabled in MAC-VRF1 of PE1, PE2 and PE3. 307 2. The PEs start adding dynamic, static and EVPN-learned entries to 308 their Proxy tables: 310 a. PE3 adds IP1->M1 and IP2->M2 based on the EVPN routes received 311 from PE1 and PE2. Those entries were previously learned as 312 dynamic entries in PE1 and PE2 respectively, and advertised in 313 BGP EVPN. 314 b. PE3 adds IP4->M4 as dynamic. This entry is learned by snooping 315 the corresponding ARP messages sent by CE4. 316 c. An operator also provisions the static entry IP3->M3. 318 3. When CE3 sends an ARP Request asking for IP1, PE3 will: 320 a. Intercept the ARP Request and perform a Proxy-ARP lookup for 321 IP1. 322 b. If the lookup is successful (as in Figure 1), PE3 will send an 323 ARP Reply with IP1->M1. The ARP Request will not be flooded to 324 the EVPN network or any other local CEs. 325 c. If the lookup is not successful, PE3 will flood the ARP Request 326 in the EVPN network and the other local CEs. 328 As PE3 learns more and more host entries in the Proxy-ARP/ND table, 329 the flooding of ARP Request messages is reduced and in some cases it 330 can even be suppressed. In a network where most of the participant 331 CEs are not moving between PEs and they advertise their presence with 332 GARPs or unsolicited NA messages, the ARP/ND flooding as well as the 333 unknown unicast flooding can practically be suppressed. In an EVPN- 334 based IXP network, where all the entries are Static, the ARP/ND 335 flooding is in fact totally suppressed. 337 The Proxy-ARP/ND function can be structured in five sub-functions or 338 procedures: 340 1. Learning sub-function 341 2. Reply sub-function 342 3. Maintenance sub-function 343 4. Flooding reduction/suppression sub-function 344 5. Duplicate IP detection sub-function 346 A Proxy-ARP/ND implementation MAY support all those sub-functions or 347 only a subset of them. The following sections describe each 348 individual sub-function. 350 4.1. Learning Sub-Function 352 A Proxy-ARP/ND implementation SHOULD support static, dynamic and 353 EVPN-learned entries. 355 Static entries are provisioned from the management plane. The 356 provisioned static IP->MAC entry SHOULD be advertised in EVPN with a 357 MAC Mobility extended community where the static flag is set to 1, as 358 per [RFC7432]. A static entry MAY associate and IP to a list of 359 potential MACs, i.e. IP1->(MAC1,MAC2..MACN). When there is more than 360 one MAC in the list of allowed MACs, the PE will not advertise any 361 IP->MAC in EVPN until a local ARP/NA message or any other frame is 362 received from the CE. Upon receiving traffic from the CE, the PE will 363 check that the source MAC is included in the list of allowed MACs. 364 Only in that case, the PE will activate the IP->MAC and advertise it 365 in EVPN. 367 EVPN-learned entries MUST be learned from received valid EVPN MAC/IP 368 Advertisement routes containing a MAC and IP address. 370 Dynamic entries are learned in different ways depending on whether 371 the entry contains an IPv4 or IPv6 address: 373 a) Proxy-ARP dynamic entries: 375 They SHOULD be learned by snooping any ARP packet (Ethertype 376 0x0806) received from the CEs attached to the MAC-VRF. The 377 Learning function will add the Sender MAC and Sender IP of the 378 snooped ARP packet to the Proxy-ARP table. 380 b) Proxy-ND dynamic entries: 382 They SHOULD be learned out of the Target Address and TLLA 383 information in NA messages (Ethertype 0x86DD, ICMPv6 type 136) 384 received from the CEs attached to the MAC-VRF. A Proxy-ND 385 implementation SHOULD NOT learn IP->MAC entries from NS messages, 386 since they don't contain the R-bit Flag required by the Proxy-ND 387 reply function. See section 4.1.1 for more information about the 388 R-bit flag. 390 Note that if the O-bit is zero in the received NA message, the 391 IP->MAC SHOULD only be learned in case IPv6 'anycast' is enabled 392 in the EVI. 394 The following procedure associated to the Learning sub-function is 395 recommended: 397 o When a new Proxy-ARP/ND EVPN or static active entry is learned (or 398 provisioned), the PE SHOULD send an unsolicited GARP or NA message 399 to the access CEs. The PE SHOULD send an unsolicited GARP/NA 400 message for dynamic entries only if the ARP/NA message creating the 401 entry was NOT flooded before. This unsolicited GARP/NA message 402 makes sure the CE ARP/ND caches are updated even if the ARP/NS/NA 403 messages from remote CEs are not flooded in the EVPN network. 405 Note that if a Static entry is provisioned with the same IP as an 406 existing EVPN-learned or Dynamic entry, the Static entry takes 407 precedence. 409 4.1.1. Proxy-ND and the NA Flags 411 [RFC4861] describes the use of the R-bit flag in IPv6 Address 412 Resolution: 414 o Nodes capable of routing IPv6 packets must reply to NS messages 415 with NA messages where the R-bit flag is set (R-bit=1). 417 o Hosts that are not able to route IPv6 packets must indicate that 418 inability by replying with NA messages that contain R-bit=0. 420 The use of the R-bit flag in NA messages has an impact on how hosts 421 select their default gateways when sending packets off-link: 423 o Hosts build a Default Router List based on the received RAs and NAs 424 with R-bit=1. Each cache entry has an IsRouter flag, which must be 425 set based on the R-bit flag in the received NAs. A host can choose 426 one or more Default Routers when sending packets off-link. 428 o In those cases where the IsRouter flag changes from TRUE to FALSE 429 as a result of a NA update, the node MUST remove that router from 430 the Default Router List and update the Destination Cache entries 431 for all destinations using that neighbor as a router, as specified 432 in [RFC4861] section 7.3.3. This is needed to detect when a node 433 that is used as a router stops forwarding packets due to being 434 configured as a host. 436 The R-bit and O-bit will be learned in the following ways: 438 o Static entries SHOULD have the R-bit information added by the 439 management interface. The O-bit information MAY also be added by 440 the management interface. 442 o Dynamic entries SHOULD learn the R-bit and MAY learn the O-bit from 443 the snooped NA messages used to learn the IP->MAC itself. 445 o EVPN-learned entries SHOULD learn the R-bit and MAY learn the O-bit 446 from the ND Extended Community received from EVPN along with the 447 RT2 used to learn the IP->MAC itself. Please refer to [EVPN-NA- 448 FLAGS]. If no ND extended community is received, the PE will add 449 the default R-bit/O-bit to the entry. The default R-bit SHOULD be 450 an administrative choice. The default O-bit SHOULD be 1. 452 Note that the O-bit SHOULD only be learned if 'anycast' is enabled in 453 the EVI. If so, Duplicate IP Detection must be disabled so that the 454 PE is able to learn the same IP mapped to different MACs in the same 455 Proxy-ND table. If 'anycast' is disabled, NA messages with O-bit = 0 456 will not create a proxy-ND entry, hence no EVPN advertisement with ND 457 extended community will be generated. 459 4.2. Reply Sub-Function 461 This sub-function will reply to Address Resolution 462 requests/solicitations upon successful lookup in the Proxy-ARP/ND 463 table for a given IP address. The following considerations should be 464 taken into account: 466 a) When replying to ARP Request or NS messages, the PE SHOULD use the 467 Proxy-ARP/ND entry MAC address as MAC SA. This is recommended so 468 that the resolved MAC can be learned in the MAC FIB of potential 469 Layer-2 switches seating between the PE and the CE requesting the 470 Address Resolution. 472 b) A PE SHOULD NOT reply to a request/solicitation received on the 473 same attachment circuit over which the IP->MAC is learned. In this 474 case the requester and the requested IP are assumed to be 475 connected to the same layer-2 switch/access network linked to the 476 PE's attachment circuit, and therefore the requested IP owner will 477 receive the request directly. 479 c) A PE SHOULD reply to broadcast/multicast Address Resolution 480 messages, that is, ARP-Request, NS messages as well as DAD NS 481 messages. A PE SHOULD NOT reply to unicast Address Resolution 482 requests (for instance, NUD NS messages). 484 d) A PE SHOULD include the R-bit learned for the IP->MAC entry in the 485 NA messages (see section 4.1.1). The S-bit will be set/unset as 486 per [RFC4861]. The O-bit will be included if IPv6 'anycast' is 487 enabled in the EVI and it is learned for the IP->MAC entry. If 488 'anycast' is enabled and there are more than one MAC for a given 489 IP, the PE will reply to NS messages with as many NA responses as 490 'anycast' entries are in the proxy-ND table. 492 e) A PE SHOULD only reply to ARP-Request and NS messages with the 493 format specified in [RFC0826] and [RFC4861] respectively. Received 494 ARP-Requests and NS messages with unknown options SHOULD be either 495 forwarded (as unicast packets) to the owner of the requested IP 496 (assuming the MAC is known in the proxy-ARP/ND table and MAC-VRF) 497 or discarded. An administrative option SHOULD control whether to 498 'unicast-forward' or 'discard' these frames with unknown options. 499 Note that, as an example, this would allow to enable proxy-ND and 500 Secure ND [RFC3971] in the same EVI. The 'unicast-forward' option 501 allows the support of new unknown options in the EVI while 502 reducing the flooding at the same time. 504 4.3. Maintenance Sub-Function 506 The Proxy-ARP/ND tables SHOULD follow a number of maintenance 507 procedures so that the dynamic IP->MAC entries are kept if the owner 508 is active and flushed if the owner is no longer in the network. The 509 following procedures are recommended: 511 a) Age-time 513 A dynamic Proxy-ARP/ND entry SHOULD be flushed out of the table if 514 the IP->MAC has not been refreshed within a given age-time. The 515 entry is refreshed if an ARP or NA message is received for the 516 same IP->MAC entry. The age-time is an administrative option and 517 its value should be carefully chosen depending on the specific 518 use-case: in IXP networks (where the CE routers are fairly static) 519 the age-time may normally be longer than in DC networks (where 520 mobility is required). 522 b) Send-refresh option 524 The PE MAY send periodic refresh messages (ARP/ND "probes") to the 525 owners of the dynamic Proxy-ARP/ND entries, so that the entries 526 can be refreshed before they age out. The owner of the IP->MAC 527 entry would reply to the ARP/ND probe and the corresponding entry 528 age-time reset. The periodic send-refresh timer is an 529 administrative option and is recommended to be a third of the age- 530 time or a half of the age-time in scaled networks. 532 An ARP refresh issued by the PE will be an ARP-Request message 533 with the Sender's IP = 0 sent from the PE's MAC SA. An ND refresh 534 will be a NS message issued from the PE's MAC SA and a Link Local 535 Address associated to the PE's MAC. 537 The refresh request messages should be sent only for dynamic 538 entries and not for static or EVPN-learned entries. Even though 539 the refresh request messages are broadcast or multicast, the PE 540 SHOULD only send the message to the attachment circuit associated 541 to the MAC in the IP->MAC entry. 543 The age-time and send-refresh options are used in EVPN networks to 544 avoid unnecessary EVPN RT2 withdrawals: if refresh messages are sent 545 before the corresponding MAC-VRF FIB and Proxy-ARP/ND age-time for a 546 given entry expires, inactive but existing hosts will reply, 547 refreshing the entry and therefore avoiding unnecessary MAC and MAC- 548 IP withdrawals in EVPN. Both entries (MAC in the MAC-VRF and IP->MAC 549 in Proxy-ARP/ND) are reset when the owner replies to the ARP/ND 550 probe. If there is no response to the ARP/ND probe, the MAC and 551 IP->MAC entries will be legitimately flushed and the RT2s withdrawn. 553 4.4. Flooding (to Remote PEs) Reduction/Suppression 555 The Proxy-ARP/ND function implicitly helps reducing the flooding of 556 ARP Request and NS messages to remote PEs in an EVPN network. 557 However, in certain use-cases, the flooding of ARP/NS/NA messages 558 (and even the unknown unicast flooding) to remote PEs can be 559 suppressed completely in an EVPN network. 561 For instance, in an IXP network, since all the participant CEs are 562 well known and will not move to a different PE, the IP->MAC entries 563 may be all provisioned by a management system. Assuming the entries 564 for the CEs are all provisioned on the local PE, a given Proxy-ARP/ND 565 table will only contain static and EVPN-learned entries. In this 566 case, the operator may choose to suppress the flooding of ARP/NS/NA 567 to remote PEs completely. 569 The flooding may also be suppressed completely in IXP networks with 570 dynamic Proxy-ARP/ND entries assuming that all the CEs are directly 571 connected to the PEs and they all advertise their presence with a 572 GARP/unsolicited-NA when they connect to the network. 574 In networks where fast mobility is expected (DC use-case), it is not 575 recommended to suppress the flooding of unknown ARP-Requests/NS or 576 GARPs/unsolicited-NAs. Unknown ARP-Requests/NS refer to those 577 ARP-Request/NS messages for which the Proxy-ARP/ND lookups for the 578 requested IPs do not succeed. 580 In order to give the operator the choice to suppress/allow the 581 flooding to remote PEs, a PE MAY support administrative options to 582 individually suppress/allow the flooding of: 584 o Unknown ARP-Request and NS messages. 585 o GARP and unsolicited-NA messages. 587 The operator will use these options based on the expected behavior in 588 the CEs. 590 4.5. Duplicate IP Detection 592 The Proxy-ARP/ND function SHOULD support duplicate IP detection so 593 that ARP/ND-spoofing attacks or duplicate IPs due to human errors can 594 be detected. 596 ARP/ND spoofing is a technique whereby an attacker sends "fake" 597 ARP/ND messages onto a broadcast domain. Generally the aim is to 598 associate the attacker's MAC address with the IP address of another 599 host causing any traffic meant for that IP address to be sent to the 600 attacker instead. 602 The distributed nature of EVPN and proxy-ARP/ND allows the easy 603 detection of duplicated IPs in the network, in a similar way to the 604 MAC duplication function supported by [RFC7432] for MAC addresses. 606 Duplicate IP detection monitors "IP-moves" in the Proxy-ARP/ND table 607 in the following way: 609 o When an existing active IP1->MAC1 entry is modified, a PE starts an 610 M-second timer (default value of M=180), and if it detects N IP 611 moves before the timer expires (default value of N=5), it concludes 612 that a duplicate IP situation has occurred. An IP move is 613 considered when, for instance, IP1->MAC1 is replaced by IP1->MAC2 614 in the Proxy-ARP/ND table. 616 o In order to detect the duplicate IP faster, the PE MAY send a 617 CONFIRM message to the former owner of the IP. A CONFIRM message is 618 a unicast ARP-Request/NS message sent by the PE to the MAC 619 addresses that previously owned the IP, when the MAC changes in the 620 Proxy-ARP/ND table. If the PE does not receive an answer within a 621 given timer, the new entry will be confirmed and activated. For 622 instance, if IP1->MAC1 moves to IP1->MAC2, the PE may send a 623 unicast ARP-Request/NS message for IP1 with MAC DA= MAC1 and MAC 624 SA= PE's MAC. This will force the legitimate owner and the spoofer 625 to reply so that the PE can detect the duplicate IP within the M 626 timer: 628 - If the IP1->MAC1 pair was previously owned by the spoofer and the 629 new IP1->MAC2 was from a valid CE, then the issued CONFIRM 630 message would trigger a response from the spoofer. 632 - If it were the other way around, that is, IP1->MAC1 was 633 previously owned by a valid CE, the CONFIRM message would trigger 634 a response from the CE. 636 Either way, if this process continues, then duplicate detection 637 will kick in. 639 o Upon detecting a duplicate IP situation: 641 a) The entry in duplicate detected state cannot be updated with new 642 dynamic or EVPN-learned entries for the same IP. The operator 643 MAY override the entry though with a static IP->MAC. 645 b) The PE SHOULD alert the operator and stop responding ARP/NS for 646 the duplicate IP until a corrective action is taken. 648 c) Optionally the PE MAY associate an "anti-spoofing-mac" (AS-MAC) 649 to the duplicate IP. The PE will send a GARP/unsolicited-NA 650 message with IP1->AS-MAC to the local CEs as well as an RT2 651 (with IP1->AS-MAC) to the remote PEs. This will force all the 652 CEs in the EVI to use the AS-MAC as MAC DA for IP1, and prevent 653 the spoofer from attracting any traffic for IP1. Since the AS- 654 MAC is a managed MAC address known by all the PEs in the EVI, 655 all the PEs MAY apply filters to drop and/or log any frame with 656 MAC DA= AS-MAC. The advertisement of the AS-MAC as a "black-hole 657 MAC" that can be used directly in the MAC-VRF to drop frames is 658 for further study. 660 o The duplicate IP situation will be cleared when a corrective action 661 is taken by the operator, or alternatively after a HOLD-DOWN timer 662 (default value of 540 seconds). 664 The values of M, N and HOLD-DOWN timer SHOULD be a configurable 665 administrative option to allow for the required flexibility in 666 different scenarios. 668 For Proxy-ND, Duplicate IP Detection SHOULD only monitor IP moves for 669 IP->MACs learned from NA messages with O-bit=1. NA messages with 670 O-bit=0 would not override the ND cache entries for an existing IP. 671 Duplicate IP Detection for IPv6 SHOULD be disabled when IPv6 672 'anycast' is activated in a given EVI. 674 5. Solution Benefits 676 The solution described in this document provides the following 677 benefits: 679 a) The solution may suppress completely the flooding of the ARP/ND 680 and unknown-unicast messages in the EVPN network, in cases where 681 all the CE IP->MAC addresses local to the PEs are known and 682 provisioned on the PEs from a management system. 684 b) The solution reduces significantly the flooding of the ARP/ND 685 messages in the EVPN network, in cases where some or all the CE 686 IP->MAC addresses are learned on the data plane by snooping ARP/ND 687 messages issued by the CEs. 689 c) The solution reduces the control plane overhead and unnecessary 690 BGP MAC/IP Advertisements and Withdrawals in a network with active 691 CEs that do not send packets frequently. 693 d) The solution provides a mechanism to detect duplicate IP addresses 694 and avoid ARP/ND-spoof attacks or the effects of duplicate 695 addresses due to human errors. 697 6. Deployment Scenarios 699 Four deployment scenarios with different levels of ARP/ND control are 700 available to operators using this solution, depending on their 701 requirements to manage ARP/ND: all dynamic learning, all dynamic 702 learning with proxy-ARP/ND, hybrid dynamic learning and static 703 provisioning with proxy-ARP/ND, and all static provisioning with 704 proxy-ARP/ND. 706 6.1. All Dynamic Learning 708 In this scenario for minimum security and mitigation, EVPN is 709 deployed in the peering network with the proxy-ARP/ND function 710 shutdown. PEs do not intercept ARP/ND requests and flood all 711 requests, as in a conventional layer-2 network. While no ARP/ND 712 mitigation is used in this scenario, the IXP can still take advantage 713 of EVPN features such as control plane learning and all-active 714 multihoming in the peering network. Existing mitigation solutions, 715 such as the ARP-Sponge daemon [ARP-Sponge] MAY also be used in this 716 scenario. 718 Although this option does not require any of the procedures described 719 in this document, it is added as baseline/default option for 720 completeness. 722 6.2. Dynamic Learning with Proxy-ARP/ND 724 This scenario minimizes flooding while enabling dynamic learning of 725 IP->MAC entries. The Proxy-ARP/ND function is enabled in the MAC-VRFs 726 of the EVPN PEs, so that the PEs intercept and respond to CE 727 requests. 729 The solution MAY further reduce the flooding of the ARP/ND messages 730 in the EVPN network by snooping ARP/ND messages issued by the CEs. 732 PEs will flood requests if the entry is not in their Proxy table. Any 733 unknown source MAC->IP entries will be learnt and advertised in EVPN, 734 and traffic to unknown entries is discarded at the ingress PE. 736 6.3. Hybrid Dynamic Learning and Static Provisioning with Proxy-ARP/ND 738 Some IXPs want to protect particular hosts on the peering network 739 while allowing dynamic learning of peering router addresses. For 740 example, an IXP may want to configure static MAC->IP entries for 741 management and infrastructure hosts that provide critical services. 742 In this scenario, static entries are provisioned from the management 743 plane for protected MAC->IP addresses, and dynamic learning with 744 Proxy-ARP/ND is enabled as described in section 6.2 on the peering 745 network. 747 6.4 All Static Provisioning with Proxy-ARP/ND 749 For a solution that maximizes security and eliminates flooding and 750 unknown unicast in the peering network, all MAC-IP entries are 751 provisioned from the management plane. The Proxy-ARP/ND function is 752 enabled in the MAC-VRFs of the EVPN PEs, so that the PEs intercept 753 and respond to CE requests. Dynamic learning and ARP/ND snooping is 754 disabled so that traffic to unknown entries is discarded at the 755 ingress PE. This scenario provides and IXP the most control over 756 MAC->IP entries and allows an IXP to manage all entries from a 757 management system. 759 6.5 Deployment Scenarios in IXPs 761 Nowadays, almost all IXPs installed some security rules in order to 762 protect the IXP-LAN. These rules are often called port security. Port 763 security summarizes different operational steps that limit the access 764 to the IXP-LAN, to the customer router and controls the kind of 765 traffic that the routers are allowed to be exchange (e.g., Ethernet, 766 IPv4, IPv6). Due to this, the deployment scenario as described in 6.4 767 "All Static Provisioning with Proxy-ARP/ND" is the predominant 768 scenario for IXPs. 770 In addition to the "All Static Provisioning" behavior, in IXP 771 networks it is recommended to configure the Reply Sub-Function to 772 'discard' ARP-Requests/NS messages with unrecognized options. 774 At IXPs, customers usually follow a certain operational life-cycle. 775 For each step of the operational life-cycle specific operational 776 procedures are executed. 778 The following describes the operational procedures that are needed to 779 guarantee port security throughout the life-cycle of a customer with 780 focus on EVPN features: 782 1. A new customer is connected the first time to the IXP: 784 Before the connection between the customer router and the IXP-LAN 785 is activated, the MAC of the router is white-listed on the IXP's 786 switch port. All other MAC addresses are blocked. Pre-defined IPv4 787 and IPv6 addresses of the IXP's peering network space are 788 configured at the customer router. The IP->MAC static entries 789 (IPv4 and IPv6) are configured in the management system of the IXP 790 for the customer's port in order to support Proxy-ARP/ND. 792 In case a customer uses multiple ports aggregated to a single 793 logical port (LAG) some vendors randomly select the MAC address of 794 the LAG from the different MAC addresses assigned to the ports. In 795 this case the static entry will be used associated to a list of 796 allowed MACs. 798 2. Replacement of customer router: 800 If a customer router is about to be replaced, the new MAC 801 address(es) must be installed in the management system besides the 802 MAC address(es) of the currently connected router. This allows the 803 customer to replace the router without any active involvement of 804 the IXP operator. For this, static entries are also used. After 805 the replacement takes place, the MAC address(es) of the replaced 806 router can be removed. 808 3. Decommissioning a customer router 810 If a customer router is decommissioned, the router is disconnected 811 from the IXP PE. Right after that, the MAC address(es) of the 812 router and IP->MAC bindings can be removed from the management 813 system. 815 6.6 Deployment Scenarios in DCs 817 DCs normally have different requirements than IXPs in terms of Proxy- 818 ARP/ND. Some differences are listed below: 820 a) The required mobility in virtualized DCs makes the "Dynamic 821 Learning" or "Hybrid Dynamic and Static Provisioning" models more 822 appropriate than the "All Static Provisioning" model. 824 b) IPv6 'anycast' may be required in DCs, while it is not a 825 requirement in IXP networks. Therefore if the DC needs IPv6 826 'anycast' it will be explicitly enabled in the proxy-ND function, 827 hence the proxy-ND sub-functions modified accordingly. For 828 instance, if IPv6 'anycast' is enabled in the proxy-ND function, 829 Duplicate IP Detection must be disabled. 831 c) DCs may require special options on ARP/ND as opposed to the 832 Address Resolution function, which is the only one typically 833 required in IXPs. Based on that, the Reply Sub-function may be 834 modified to forward or discard unknown options. 836 7. Conventions Used in this Document 837 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 838 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 839 document are to be interpreted as described in RFC-2119 [RFC2119]. 841 In this document, these words will appear with that interpretation 842 only when in ALL CAPS. Lower case uses of these words are not to be 843 interpreted as carrying RFC-2119 significance. 845 In this document, the characters ">>" preceding an indented line(s) 846 indicates a compliance requirement statement using the key words 847 listed above. This convention aids reviewers in quickly identifying 848 or finding the explicit compliance requirements of this RFC. 850 8. Security Considerations 852 When EVPN and its associated Proxy-ARP/ND function are used in IXP 853 networks, they only provide ARP/ND security and mitigation. IXPs MUST 854 still employ security mechanisms that protect the peering network and 855 SHOULD follow established BCPs such as the ones described in [Euro-IX 856 BCP]. 858 For example, IXPs should disable all unneeded control protocols, and 859 block unwanted protocols from CEs so that only IPv4, ARP and IPv6 860 Ethertypes are permitted on the peering network. In addition, port 861 security features and ACLs can provide an additional level of 862 security. 864 9. IANA Considerations 866 No IANA considerations. 868 10. References 870 10.1. Normative References 872 [RFC7432]Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 873 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet 874 VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . 877 [RFC4861]Narten, T., Nordmark, E., Simpson, W., and H. Soliman, 878 "Neighbor Discovery for IP version 6 (IPv6)", RFC 4861, DOI 879 10.17487/RFC4861, September 2007, . 882 [RFC0826]Plummer, D., "Ethernet Address Resolution Protocol: Or 883 Converting Network Protocol Addresses to 48.bit Ethernet Address for 884 Transmission on Ethernet Hardware", STD 37, RFC 826, DOI 885 10.17487/RFC0826, November 1982, . 888 [RFC6820]Narten, T., Karir, M., and I. Foo, "Address Resolution 889 Problems in Large Data Center Networks", RFC 6820, DOI 890 10.17487/RFC6820, January 2013, . 893 [RFC7342]Dunbar, L., Kumari, W., and I. Gashinsky, "Practices for 894 Scaling ARP and Neighbor Discovery (ND) in Large Data Centers", 895 RFC 7342, DOI 10.17487/RFC7342, August 2014, . 898 [RFC3971]Arkko, J., Ed., Kempf, J., Zill, B., and P. Nikander, 899 "SEcure Neighbor Discovery (SEND)", RFC 3971, DOI 10.17487/RFC3971, 900 March 2005, . 902 [RFC7432]Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 903 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet 904 VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . 907 10.2. Informative References 909 [ARP-Sponge] Wessel M. and Sijm N., Universiteit van Amsterdam, 910 "Effects of IPv4 and IPv6 address resolution on AMS-IX and the ARP 911 Sponge", July 2009. 913 [EVPN-ND-FLAGS] Sathappan S., Nagaraj K. and Rabadan J., "Propagation 914 of IPv6 Neighbor Advertisement Flags in EVPN", draft-snr-bess-evpn- 915 na-flags-02, Work in Progress, July 2015. 917 [Euro-IX BCP] https://www.euro-ix.net/pages/28/1/bcp_ixp.html 919 11. Acknowledgments 921 The authors want to thank Ranganathan Boovaraghavan, Sriram 922 Venkateswaran, Manish Krishnan, Seshagiri Venugopal, Tony Przygienda, 923 Erik Nordmark and Robert Raszuk for their review and contributions. 924 Thank you to Oliver Knapp as well, for his detailed review. 926 Authors' Addresses 928 Jorge Rabadan (Editor) 929 Alcatel-Lucent 930 777 E. Middlefield Road 931 Mountain View, CA 94043 USA 932 Email: jorge.rabadan@alcatel-lucent.com 934 Senthil Sathappan 935 Alcatel-Lucent 936 Email: senthil.sathappan@alcatel-lucent.com 938 Kiran Nagaraj 939 Alcatel-Lucent 940 Email: kiran.nagaraj@alcatel-lucent.com 942 Wim Henderickx 943 Alcatel-Lucent 944 Email: wim.henderickx@alcatel-lucent.com 946 Greg Hankins 947 Alcatel-Lucent 948 Email: greg.hankins@alcatel-lucent.com 950 Thomas King 951 DE-CIX 952 Email: thomas.king@de-cix.net 954 Daniel Melzer 955 DE-CIX 956 Email: daniel.melzer@de-cix.net