idnits 2.17.1 draft-ietf-bess-dci-evpn-overlay-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 16 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 18, 2017) is 2475 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC4761' is mentioned on line 986, but not defined == Missing Reference: 'RFC4762' is mentioned on line 987, but not defined == Missing Reference: 'RFC6074' is mentioned on line 395, but not defined == Missing Reference: 'RFC7432' is mentioned on line 986, but not defined == Missing Reference: 'RFC7041' is mentioned on line 452, but not defined == Missing Reference: 'RFC5512' is mentioned on line 549, but not defined ** Obsolete undefined reference: RFC 5512 (Obsoleted by RFC 9012) == Missing Reference: 'RFC2119' is mentioned on line 947, but not defined == Outdated reference: A later version (-12) exists of draft-ietf-bess-evpn-overlay-08 -- No information found for draft-ietf-bess-evpn-vpls-integration - is the name correct? Summary: 2 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS Workgroup J. Rabadan (Ed.) 3 Internet Draft S. Sathappan 4 Intended status: Standards Track W. Henderickx 5 Nokia 7 A. Sajassi 8 Cisco 10 J. Drake 11 Juniper 13 Expires: January 19, 2018 July 18, 2017 15 Interconnect Solution for EVPN Overlay networks 16 draft-ietf-bess-dci-evpn-overlay-05 18 Abstract 20 This document describes how Network Virtualization Overlays (NVO) can 21 be connected to a Wide Area Network (WAN) in order to extend the 22 layer-2 connectivity required for some tenants. The solution analyzes 23 the interaction between NVO networks running EVPN and other L2VPN 24 technologies used in the WAN, such as VPLS/PBB-VPLS or EVPN/PBB-EVPN, 25 and proposes a solution for the interworking between both. 27 Status of this Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF), its areas, and its working groups. Note that 34 other groups may also distribute working documents as Internet- 35 Drafts. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 The list of current Internet-Drafts can be accessed at 43 http://www.ietf.org/ietf/1id-abstracts.txt 45 The list of Internet-Draft Shadow Directories can be accessed at 46 http://www.ietf.org/shadow.html 48 This Internet-Draft will expire on January 19, 2018. 50 Copyright Notice 52 Copyright (c) 2017 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 68 2. Decoupled Interconnect solution for EVPN overlay networks . . . 3 69 2.1. Interconnect requirements . . . . . . . . . . . . . . . . . 4 70 2.2. VLAN-based hand-off . . . . . . . . . . . . . . . . . . . . 5 71 2.3. PW-based (Pseudowire-based) hand-off . . . . . . . . . . . 5 72 2.4. Multi-homing solution on the GWs . . . . . . . . . . . . . 6 73 2.5. Gateway Optimizations . . . . . . . . . . . . . . . . . . . 6 74 2.5.1. MAC Address Advertisement Control . . . . . . . . . . . 6 75 2.5.2. ARP flooding control . . . . . . . . . . . . . . . . . 7 76 2.5.3. Handling failures between GW and WAN Edge routers . . . 7 77 3. Integrated Interconnect solution for EVPN overlay networks . . 8 78 3.1. Interconnect requirements . . . . . . . . . . . . . . . . . 8 79 3.2. VPLS Interconnect for EVPN-Overlay networks . . . . . . . . 9 80 3.2.1. Control/Data Plane setup procedures on the GWs . . . . 9 81 3.2.2. Multi-homing procedures on the GWs . . . . . . . . . . 10 82 3.3. PBB-VPLS Interconnect for EVPN-Overlay networks . . . . . . 10 83 3.3.1. Control/Data Plane setup procedures on the GWs . . . . 10 84 3.3.2. Multi-homing procedures on the GWs . . . . . . . . . . 11 85 3.4. EVPN-MPLS Interconnect for EVPN-Overlay networks . . . . . 11 86 3.4.1. Control Plane setup procedures on the GWs . . . . . . . 11 87 3.4.2. Data Plane setup procedures on the GWs . . . . . . . . 13 88 3.4.3. Multi-homing procedure extensions on the GWs . . . . . 14 89 3.4.4. Impact on MAC Mobility procedures . . . . . . . . . . . 16 90 3.4.5. Gateway optimizations . . . . . . . . . . . . . . . . . 16 91 3.4.6. Benefits of the EVPN-MPLS Interconnect solution . . . . 17 92 3.5. PBB-EVPN Interconnect for EVPN-Overlay networks . . . . . . 18 93 3.5.1. Control/Data Plane setup procedures on the GWs . . . . 18 94 3.5.2. Multi-homing procedures on the GWs . . . . . . . . . . 18 95 3.5.3. Impact on MAC Mobility procedures . . . . . . . . . . . 18 96 3.5.4. Gateway optimizations . . . . . . . . . . . . . . . . . 19 97 3.6. EVPN-VXLAN Interconnect for EVPN-Overlay networks . . . . . 19 98 3.6.1. Globally unique VNIs in the Interconnect network . . . 20 99 3.6.2. Downstream assigned VNIs in the Interconnect network . 20 100 5. Conventions and Terminology . . . . . . . . . . . . . . . . . . 20 101 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 21 102 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 21 103 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 104 8.1. Normative References . . . . . . . . . . . . . . . . . . . 22 105 8.2. Informative References . . . . . . . . . . . . . . . . . . 22 106 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 23 107 10. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 23 108 11. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 23 110 1. Introduction 112 [EVPN-Overlays] discusses the use of EVPN as the control plane for 113 Network Virtualization Overlays (NVO), where VXLAN, NVGRE or MPLS 114 over GRE can be used as possible data plane encapsulation options. 116 While this model provides a scalable and efficient multi-tenant 117 solution within the Data Center, it might not be easily extended to 118 the WAN in some cases due to the requirements and existing deployed 119 technologies. For instance, a Service Provider might have an already 120 deployed (PBB-)VPLS or (PBB-)EVPN network that has to be used to 121 interconnect Data Centers and WAN VPN users. A Gateway (GW) function 122 is required in these cases. 124 This document describes a Interconnect solution for EVPN overlay 125 networks, assuming that the NVO Gateway (GW) and the WAN Edge 126 functions can be decoupled in two separate systems or integrated into 127 the same system. The former option will be referred as "Decoupled 128 Interconnect solution" throughout the document, whereas the latter 129 one will be referred as "Integrated Interconnect solution". 131 2. Decoupled Interconnect solution for EVPN overlay networks 133 This section describes the interconnect solution when the GW and WAN 134 Edge functions are implemented in different systems. Figure 1 depicts 135 the reference model described in this section. 137 +--+ 138 |CE| 139 +--+ 140 | 141 +----+ 142 +----| PE |----+ 143 +---------+ | +----+ | +---------+ 144 +----+ | +---+ +----+ +----+ +---+ | +----+ 145 |NVE1|--| | | |WAN | |WAN | | | |--|NVE3| 146 +----+ | |GW1|--|Edge| |Edge|--|GW3| | +----+ 147 | +---+ +----+ +----+ +---+ | 148 | NVO-1 | | WAN | | NVO-2 | 149 | +---+ +----+ +----+ +---+ | 150 | | | |WAN | |WAN | | | | 151 +----+ | |GW2|--|Edge| |Edge|--|GW4| | +----+ 152 |NVE2|--| +---+ +----+ +----+ +---+ |--|NVE4| 153 +----+ +---------+ | | +---------+ +----+ 154 +--------------+ 156 |<-EVPN-Overlay-->|<-VLAN->|<-WAN L2VPN->|<--PW-->|<--EVPN-Overlay->| 157 hand-off hand-off 159 Figure 1 Decoupled Interconnect model 161 The following section describes the interconnect requirements for 162 this model. 164 2.1. Interconnect requirements 166 This proposed Interconnect architecture will be normally deployed in 167 networks where the EVPN-Overlay and WAN providers are different 168 entities and a clear demarcation is needed. The solution needs to 169 observe the following requirements: 171 o A simple connectivity hand-off needs to be provided between the 172 EVPN-Overlay network provider and the WAN provider so that QoS and 173 security enforcements are easily accomplished. 175 o The solution has to be independent of the L2VPN technology deployed 176 in the WAN. 178 o Multi-homing between GW and WAN Edge routers is required. Per- 179 service load balancing MUST be supported. Per-flow load balancing 180 MAY be supported but it is not a strong requirement since a 181 deterministic path per service is usually required for an easy QoS 182 and security enforcement. 184 o Ethernet OAM and Connectivity Fault Management (CFM) functions 185 needs to be supported between the EVPN-Overlay network and the WAN 186 network. 188 o The following optimizations MAY be supported at the GW: 189 + Flooding reduction of unknown unicast traffic sourced from the DC 190 Network Virtualization Edge devices (NVEs). 191 + Control of the WAN MAC addresses advertised to the DC. 192 + ARP flooding control for the requests coming from the WAN. 194 2.2. VLAN-based hand-off 196 In this option, the hand-off between the GWs and the WAN Edge routers 197 is based on 802.1Q VLANs. This is illustrated in Figure 1 (between 198 the GWs in NVO-1 and the WAN Edge routers). Each MAC-VRF in the GW is 199 connected to a different VSI/MAC-VRF instance in the WAN Edge router 200 by using a different C-TAG VLAN ID or a different combination of 201 S/C-TAG VLAN IDs that matches at both sides. 203 This option provides the best possible demarcation between the DC and 204 WAN providers and it does not require control plane interaction 205 between both providers. The disadvantage of this model is the 206 provisioning overhead since the service has to be mapped to a C-TAG 207 or S/C-TAG VLAN ID combination at both GW and WAN Edge routers. 209 In this model, the GW acts as a regular Network Virtualization Edge 210 (NVE) towards the DC. Its control plane, data plane procedures and 211 interactions are described in [EVPN-Overlays]. 213 The WAN Edge router acts as a (PBB-)VPLS or (PBB-)EVPN PE with 214 attachment circuits (ACs) to the GWs. Its functions are described in 215 [RFC4761], [RFC4762], [RFC6074] or [RFC7432], [RFC7623]. 217 2.3. PW-based (Pseudowire-based) hand-off 219 If MPLS can be enabled between the GW and the WAN Edge router, a PW- 220 based Interconnect solution can be deployed. In this option the 221 hand-off between both routers is based on FEC128-based PWs or FEC129- 222 based PWs (for a greater level of network automation). Note that this 223 model still provides a clear demarcation boundary between DC and WAN 224 (since there is a single PW between each MAC-VRF and peer VSI), and 225 security/QoS policies may be applied on a per PW basis. This model 226 provides better scalability than a C-TAG based hand-off and less 227 provisioning overhead than a combined C/S-TAG hand-off. The PW-based 228 hand-off interconnect is illustrated in Figure 1 (between the NVO-2 229 GWs and the WAN Edge routers). 231 In this model, besides the usual MPLS procedures between GW and WAN 232 Edge router, the GW MUST support an interworking function in each 233 MAC-VRF that requires extension to the WAN: 235 o If a FEC128-based PW is used between the MAC-VRF (GW) and the VSI 236 (WAN Edge), the provisioning of the VCID for such PW MUST be 237 supported on the MAC-VRF and MUST match the VCID used in the peer 238 VSI at the WAN Edge router. 240 o If BGP Auto-discovery [RFC6074] and FEC129-based PWs are used 241 between the GW MAC-VRF and the WAN Edge VSI, the provisioning of 242 the VPLS-ID MUST be supported on the MAC-VRF and MUST match the 243 VPLS-ID used in the WAN Edge VSI. 245 2.4. Multi-homing solution on the GWs 247 As already discussed, single-active multi-homing, i.e. per-service 248 load-balancing multi-homing MUST be supported in this type of 249 interconnect. 251 The GWs will be provisioned with a unique ESI per WAN interconnect 252 and the hand-off attachment circuits or PWs between the GW and the 253 WAN Edge router will be assigned to such ESI. The ESI will be 254 administratively configured on the GWs according to the procedures in 255 [RFC7432]. This Interconnect ESI will be referred as "I-ESI" 256 hereafter. 258 The solution (on the GWs) MUST follow the single-active multi-homing 259 procedures as described in [EVPN-Overlays] for the provisioned I-ESI, 260 i.e. Ethernet A-D routes per ESI and per EVI will be advertised to 261 the DC NVEs. The MAC addresses learned (in the data plane) on the 262 hand-off links will be advertised with the I-ESI encoded in the ESI 263 field. 265 2.5. Gateway Optimizations 267 The following features MAY be supported on the GW in order to 268 optimize the control plane and data plane in the DC. 270 2.5.1. MAC Address Advertisement Control 272 The use of EVPN in the NVO networks brings a significant number of 273 benefits as described in [EVPN-Overlays]. However, if multiple DCs 274 are interconnected into a single EVI, each DC will have to import all 275 of the MAC addresses from each of the other DCs. 277 Even if optimized BGP techniques like RT-constraint are used, the 278 number of MAC addresses to advertise or withdraw (in case of failure) 279 by the GWs of a given DC could overwhelm the NVEs within that DC, 280 particularly when the NVEs reside in the hypervisors. 282 The solution specified in this document uses the 'Unknown MAC' route 283 which is advertised into a given DC by each of the DC's GWs. This 284 route is a regular EVPN MAC/IP Advertisement route in which the MAC 285 Address Length is set to 48, the MAC address is set to 286 00:00:00:00:00:00, the IP length is set to 0, and the ESI field is 287 set to the DC GW's I-ESI. 289 An NVE within that DC that understands the Unknown MAC route will 290 send (unicast) a packet with an unknown unicast MAC address to one of 291 the DCs GWs which will then forward that packet to the correct egress 292 PE. I.e., because the ESI is set to the DC GW's I-ESI, all-active 293 multi-homing can be applied to unknown unicast MAC addresses. 295 This document proposes that local policy determines whether MAC 296 addresses and/or the Unknown MAC route are advertised into a given 297 DC. As an example, when all the DC MAC addresses are learned in the 298 control/management plane, it may be appropriate to advertise only the 299 Unknown MAC route. 301 2.5.2. ARP flooding control 303 Another optimization mechanism, naturally provided by EVPN in the 304 GWs, is the Proxy ARP/ND function. The GWs SHOULD build a Proxy 305 ARP/ND cache table as per [RFC7432]. When the active GW receives an 306 ARP/ND request/solicitation coming from the WAN, the GW does a Proxy 307 ARP/ND table lookup and replies as long as the information is 308 available in its table. 310 This mechanism is especially recommended on the GWs since it protects 311 the DC network from external ARP/ND-flooding storms. 313 2.5.3. Handling failures between GW and WAN Edge routers 315 Link/PE failures are handled on the GWs as specified in [RFC7432]. 316 The GW detecting the failure will withdraw the EVPN routes as per 317 [RFC7432]. 319 Individual AC/PW failures MAY be detected by OAM mechanisms. For 320 instance: 322 o If the Interconnect solution is based on a VLAN hand-off, 323 802.1ag/Y.1731 Ethernet-CFM MAY be used to detect individual AC 324 failures on both, the GW and WAN Edge router. An individual AC 325 failure will trigger the withdrawal of the corresponding A-D per 326 EVI route as well as the MACs learned on that AC. 328 o If the Interconnect solution is based on a PW hand-off, the LDP PW 329 Status bits TLV MAY be used to detect individual PW failures on 330 both, the GW and WAN Edge router. 332 3. Integrated Interconnect solution for EVPN overlay networks 334 When the DC and the WAN are operated by the same administrative 335 entity, the Service Provider can decide to integrate the GW and WAN 336 Edge PE functions in the same router for obvious CAPEX and OPEX 337 saving reasons. This is illustrated in Figure 2. Note that this model 338 does not provide an explicit demarcation link between DC and WAN 339 anymore. 341 +--+ 342 |CE| 343 +--+ 344 | 345 +----+ 346 +----| PE |----+ 347 +---------+ | +----+ | +---------+ 348 +----+ | +---+ +---+ | +----+ 349 |NVE1|--| | | | | |--|NVE3| 350 +----+ | |GW1| |GW3| | +----+ 351 | +---+ +---+ | 352 | NVO-1 | WAN | NVO-2 | 353 | +---+ +---+ | 354 | | | | | | 355 +----+ | |GW2| |GW4| | +----+ 356 |NVE2|--| +---+ +---+ |--|NVE4| 357 +----+ +---------+ | | +---------+ +----+ 358 +--------------+ 360 |<--EVPN-Overlay--->|<-----VPLS--->|<---EVPN-Overlay-->| 361 |<--PBB-VPLS-->| 362 Interconnect -> |<-EVPN-MPLS-->| 363 options |<--EVPN-Ovl-->| 364 |<--PBB-EVPN-->| 366 Figure 2 Integrated Interconnect model 368 3.1. Interconnect requirements 370 The solution needs to observe the following requirements: 372 o The GW function MUST provide control plane and data plane 373 interworking between the EVPN-overlay network and the L2VPN 374 technology supported in the WAN, i.e. (PBB-)VPLS or (PBB-)EVPN, as 375 depicted in Figure 2. 377 o Multi-homing MUST be supported. Single-active multi-homing with 378 per-service load balancing MUST be implemented. All-active multi- 379 homing, i.e. per-flow load-balancing, SHOULD be implemented as long 380 as the technology deployed in the WAN supports it. 382 o If EVPN is deployed in the WAN, the MAC Mobility, Static MAC 383 protection and other procedures (e.g. proxy-arp) described in 384 [RFC7432] MUST be supported end-to-end. 386 o Any type of inclusive multicast tree MUST be independently 387 supported in the WAN as per [RFC7432], and in the DC as per [EVPN- 388 Overlays]. 390 3.2. VPLS Interconnect for EVPN-Overlay networks 392 3.2.1. Control/Data Plane setup procedures on the GWs 394 Regular MPLS tunnels and TLDP/BGP sessions will be setup to the WAN 395 PEs and RRs as per [RFC4761], [RFC4762], [RFC6074] and overlay 396 tunnels and EVPN will be setup as per [EVPN-Overlays]. Note that 397 different route-targets for the DC and for the WAN are normally 398 required. A single type-1 RD per service may be used. 400 In order to support multi-homing, the GWs will be provisioned with an 401 I-ESI (see section 2.4), that will be unique per interconnection. All 402 the [RFC7432] procedures are still followed for the I-ESI, e.g. any 403 MAC address learned from the WAN will be advertised to the DC with 404 the I-ESI in the ESI field. 406 A MAC-VRF per EVI will be created in each GW. The MAC-VRF will have 407 two different types of tunnel bindings instantiated in two different 408 split-horizon-groups: 410 o VPLS PWs will be instantiated in the "WAN split-horizon-group". 412 o Overlay tunnel bindings (e.g. VXLAN, NVGRE) will be instantiated 413 in the "DC split-horizon-group". 415 Attachment circuits are also supported on the same MAC-VRF, but they 416 will not be part of any of the above split-horizon-groups. 418 Traffic received in a given split-horizon-group will never be 419 forwarded to a member of the same split-horizon-group. 421 As far as BUM flooding is concerned, a flooding list will be created 422 with the sub-list created by the inclusive multicast routes and the 423 sub-list created for VPLS in the WAN. BUM frames received from a 424 local attachment circuit will be forwarded to the flooding list. BUM 425 frames received from the DC or the WAN will be forwarded to the 426 flooding list observing the split-horizon-group rule described above. 428 Note that the GWs are not allowed to have an EVPN binding and a PW to 429 the same far-end within the same MAC-VRF in order to avoid loops and 430 packet duplication. This is described in [EVPN-VPLS-INTEGRATION]. 432 The optimizations procedures described in section 2.5 can also be 433 applied to this model. 435 3.2.2. Multi-homing procedures on the GWs 437 Single-active multi-homing MUST be supported on the GWs. All-active 438 multi-homing is not supported by VPLS. 440 All the single-active multi-homing procedures as described by [EVPN- 441 Overlays] will be followed for the I-ESI. 443 The non-DF GW for the I-ESI will block the transmission and reception 444 of all the bindings in the "WAN split-horizon-group" for BUM and 445 unicast traffic. 447 3.3. PBB-VPLS Interconnect for EVPN-Overlay networks 449 3.3.1. Control/Data Plane setup procedures on the GWs 451 In this case, there is no impact on the procedures described in 452 [RFC7041] for the B-component. However the I-component instances 453 become EVI instances with EVPN-Overlay bindings and potentially local 454 attachment circuits. A number of MAC-VRF instances can be multiplexed 455 into the same B-component instance. This option provides significant 456 savings in terms of PWs to be maintained in the WAN. 458 The I-ESI concept described in section 3.2.1 will also be used for 459 the PBB-VPLS-based Interconnect. 461 B-component PWs and I-component EVPN-overlay bindings established to 462 the same far-end will be compared. The following rules will be 463 observed: 465 o Attempts to setup a PW between the two GWs within the B- 466 component context will never be blocked. 468 o If a PW exists between two GWs for the B-component and an 469 attempt is made to setup an EVPN binding on an I-component linked 470 to that B-component, the EVPN binding will be kept operationally 471 down. Note that the BGP EVPN routes will still be valid but not 472 used. 474 o The EVPN binding will only be up and used as long as there is no 475 PW to the same far-end in the corresponding B-component. The EVPN 476 bindings in the I-components will be brought down before the PW in 477 the B-component is brought up. 479 The optimizations procedures described in section 2.5 can also be 480 applied to this Interconnect option. 482 3.3.2. Multi-homing procedures on the GWs 484 Single-active multi-homing MUST be supported on the GWs. All-active 485 multi-homing is not supported by this scenario. 487 All the single-active multi-homing procedures as described by [EVPN- 488 Overlays] will be followed for the I-ESI for each EVI instance 489 connected to B-component. 491 3.4. EVPN-MPLS Interconnect for EVPN-Overlay networks 493 If EVPN for MPLS tunnels, EVPN-MPLS hereafter, is supported in the 494 WAN, an end-to-end EVPN solution can be deployed. The following 495 sections describe the proposed solution as well as the impact 496 required on the [RFC7432] procedures. 498 3.4.1. Control Plane setup procedures on the GWs 500 The GWs MUST establish separate BGP sessions for sending/receiving 501 EVPN routes to/from the DC and to/from the WAN. Normally each GW will 502 setup one (two) BGP EVPN session(s) to the DC RR(s) and one(two) 503 session(s) to the WAN RR(s). 505 In order to facilitate separate BGP processes for DC and WAN, EVPN 506 routes sent to the WAN SHOULD carry a different route-distinguisher 507 (RD) than the EVPN routes sent to the DC. In addition, although 508 reusing the same value is possible, different route-targets are 509 expected to be handled for the same EVI in the WAN and the DC. Note 510 that the EVPN service routes sent to the DC RRs will normally include 511 a [RFC5512] BGP encapsulation extended community with a different 512 tunnel type than the one sent to the WAN RRs. 514 As in the other discussed options, an I-ESI will be configured on the 515 GWs for multi-homing. This I-ESI represents the WAN to the DC but 516 also the DC to the WAN. Optionally, different I-ESI values MAY be 517 configured for representing the WAN and the DC. If different EVPN- 518 Overlay networks are connected to the same group of GWs, each EVPN- 519 Overlay network MUST get assigned a different I-ESI. 521 Received EVPN routes will never be reflected on the GWs but consumed 522 and re-advertised (if needed): 524 o Ethernet A-D routes, ES routes and Inclusive Multicast routes 525 are consumed by the GWs and processed locally for the 526 corresponding [RFC7432] procedures. 528 o MAC/IP advertisement routes will be received, imported and if 529 they become active in the MAC-VRF, the information will be re- 530 advertised as new routes with the following fields: 532 + The RD will be the GW's RD for the MAC-VRF. 534 + The ESI will be set to the I-ESI. 536 + The Ethernet-tag value will be kept from the received NLRI. 538 + The MAC length, MAC address, IP Length and IP address values 539 will be kept from the received NLRI. 541 + The MPLS label will be a local 20-bit value (when sent to the 542 WAN) or a DC-global 24-bit value (when sent to the DC). 544 + The appropriate Route-Targets (RTs) and [RFC5512] BGP 545 Encapsulation extended community will be used according to 546 [EVPN-Overlays]. 548 The GWs will also generate the following local EVPN routes that will 549 be sent to the DC and WAN, with their corresponding RTs and [RFC5512] 550 BGP Encapsulation extended community values: 552 o ES route(s) for the I-ESI(s). 554 o Ethernet A-D routes per ESI and EVI for the I-ESI(s). The A-D 555 per-EVI routes sent to the WAN and the DC will have consistent 556 Ethernet-Tag values. 558 o Inclusive Multicast routes with independent tunnel type value 559 for the WAN and DC. E.g. a P2MP LSP may be used in the WAN 560 whereas ingress replication may be used in the DC. The routes 561 sent to the WAN and the DC will have a consistent Ethernet-Tag. 563 o MAC/IP advertisement routes for MAC addresses learned in local 564 attachment circuits. Note that these routes will not include the 565 I-ESI, but ESI=0 or different from 0 for local multi-homed 566 Ethernet Segments (ES). The routes sent to the WAN and the DC 567 will have a consistent Ethernet-Tag. 569 Assuming GW1 and GW2 are peer GWs of the same DC, each GW will 570 generate two sets of local service routes: Set-DC will be sent to the 571 DC RRs and will include A-D per EVI, Inclusive Multicast and MAC/IP 572 routes for the DC encapsulation and RT. Set-WAN will be sent to the 573 WAN RRs and will include the same routes but using the WAN RT and 574 encapsulation. GW1 and GW2 will receive each other's set-DC and set- 575 WAN. This is the expected behavior on GW1 and GW2 for locally 576 generated routes: 578 o Inclusive multicast routes: when setting up the flooding lists 579 for a given MAC-VRF, each GW will include its DC peer GW only in 580 the EVPN-MPLS flooding list (by default) and not the EVPN- 581 Overlay flooding list. That is, GW2 will import two Inclusive 582 Multicast routes from GW1 (from set-DC and set-WAN) but will 583 only consider one of the two, having the set-WAN route higher 584 priority. An administrative option MAY change this preference so 585 that the set-DC route is selected first. 587 o MAC/IP advertisement routes for local attachment circuits: as 588 above, the GW will select only one, having the route from the 589 set-WAN a higher priority. As for the Inclusive multicast 590 routes, an administrative option MAY change this priority. 592 Note that, irrespective of the encapsulation, EVPN routes always have 593 higher priority than VPLS AD routes as per [EVPN-VPLS-INTEGRATION]. 595 3.4.2. Data Plane setup procedures on the GWs 597 The procedure explained at the end of the previous section will make 598 sure there are no loops or packet duplication between the GWs of the 599 same EVPN-Overlay network (for frames generated from local ACs) since 600 only one EVPN binding per EVI (or per Ethernet Tag in case of VLAN- 601 aware bundle services) will be setup in the data plane between the 602 two nodes. That binding will by default be added to the EVPN-MPLS 603 flooding list. 605 As for the rest of the EVPN tunnel bindings, they will be added to 606 one of the two flooding lists that each GW sets up for the same MAC- 607 VRF: 609 o EVPN-overlay flooding list (composed of bindings to the remote 610 NVEs or multicast tunnel to the NVEs). 612 o EVPN-MPLS flooding list (composed of MP2P or LSM tunnel to the 613 remote PEs) 615 Each flooding list will be part of a separate split-horizon-group: 616 the WAN split-horizon-group or the DC split-horizon-group. Traffic 617 generated from a local AC can be flooded to both 618 split-horizon-groups. Traffic from a binding of a split-horizon-group 619 can be flooded to the other split-horizon-group and local ACs, but 620 never to a member of its own split-horizon-group. 622 When either GW1 or GW2 receive a BUM frame on an MPLS tunnel 623 including an ESI label at the bottom of the stack, they will perform 624 an ESI label lookup and split-horizon filtering as per [RFC7432] in 625 case the ESI label identifies a local ESI (I-ESI or any other non- 626 zero ESI). 628 3.4.3. Multi-homing procedure extensions on the GWs 630 Single-active as well as all-active multi-homing MUST be supported. 632 All the [RFC7432] multi-homing procedures for the DF election on I- 633 ESI(s) as well as the backup-path (single-active) and aliasing (all- 634 active) procedures will be followed on the GWs. Remote PEs in the 635 EVPN-MPLS network will follow regular [RFC7432] aliasing or backup- 636 path procedures for MAC/IP routes received from the GWs for the same 637 I-ESI. So will NVEs in the EVPN-Overlay network for MAC/IP routes 638 received with the same I-ESI. 640 As far as the forwarding plane is concerned, by default, the EVPN- 641 Overlay network will have an analogous behavior to the access ACs in 642 [RFC7432] multi-homed Ethernet Segments. 644 The forwarding behavior on the GWs is described below: 646 o Single-active multi-homing; assuming a WAN split-horizon-group 647 (comprised of EVPN-MPLS bindings), a DC split-horizon-group 648 (comprised of EVPN-Overlay bindings) and local ACs on the GWs: 650 + Forwarding behavior on the non-DF: the non-DF MUST block 651 ingress and egress forwarding on the EVPN-Overlay bindings 652 associated to the I-ESI. The EVPN-MPLS network is considered 653 to be the core network and the EVPN-MPLS bindings to the 654 remote PEs and GWs will be active. 656 + Forwarding behavior on the DF: the DF MUST NOT forward BUM or 657 unicast traffic received from a given split-horizon-group to a 658 member of his own split-horizon group. Forwarding to other 659 split-horizon-groups and local ACs is allowed (as long as the 660 ACs are not part of an ES for which the node is non-DF). As 661 per [RFC7432] and for split-horizon purposes, when receiving 662 BUM traffic on the EVPN-Overlay bindings associated to an I- 663 ESI, the DF GW SHOULD add the I-ESI label when forwarding to 664 the peer GW over EVPN-MPLS. 666 + When receiving EVPN MAC/IP routes from the WAN, the non-DF 667 MUST NOT re-originate the EVPN routes and advertise them to 668 the DC peers. In the same way, EVPN MAC/IP routes received 669 from the DC MUST NOT be advertised to the WAN peers. This is 670 consistent with [RFC7432] and allows the remote PE/NVEs know 671 who the primary GW is, based on the reception of the MAC/IP 672 routes. 674 o All-active multi-homing; assuming a WAN split-horizon-group 675 (comprised of EVPN-MPLS bindings), a DC split-horizon-group 676 (comprised of EVPN-Overlay bindings) and local ACs on the GWs: 678 + Forwarding behavior on the non-DF: the non-DF follows the same 679 behavior as the non-DF in the single-active case but only for 680 BUM traffic. Unicast traffic received from a split-horizon- 681 group MUST NOT be forwarded to a member of its own split- 682 horizon-group but can be forwarded normally to the other 683 split-horizon-groups and local ACs. If a known unicast packet 684 is identified as a "flooded" packet, the procedures for BUM 685 traffic MUST be followed. 687 + Forwarding behavior on the DF: the DF follows the same 688 behavior as the DF in the single-active case but only for BUM 689 traffic. Unicast traffic received from a split-horizon-group 690 MUST NOT be forwarded to a member of its own split-horizon- 691 group but can be forwarded normally to the other split- 692 horizon-group and local ACs. If a known unicast packet is 693 identified as a "flooded" packet, the procedures for BUM 694 traffic MUST be followed. As per [RFC7432] and for split- 695 horizon purposes, when receiving BUM traffic on the EVPN- 696 Overlay bindings associated to an I-ESI, the DF GW MUST add 697 the I-ESI label when forwarding to the peer GW over EVPN-MPLS. 699 + Contrary to the single-active multi-homing case, both DF and 700 non-DF re-originate and advertise MAC/IP routes received from 701 the WAN/DC peers, adding the corresponding I-ESI so that the 702 remote PE/NVEs can perform regular aliasing as per [RFC7432]. 704 The example in Figure 3 illustrates the forwarding of BUM traffic 705 originated from an NVE on a pair of all-active multi-homing GWs. 707 |<--EVPN-Overlay--->|<--EVPN-MPLS-->| 709 +---------+ +--------------+ 710 +----+ BUM +---+ | 711 |NVE1+----+----> | +-+-----+ | 712 +----+ | | DF |GW1| | | | 713 | | +-+-+ | | ++--+ 714 | | | | +--> |PE1| 715 | +--->X +-+-+ | ++--+ 716 | NDF| | | | 717 +----+ | |GW2<-+ | 718 |NVE2+--+ +-+-+ | 719 +----+ +--------+ | +------------+ 720 v 721 +--+ 722 |CE| 723 +--+ 725 Figure 3 Multi-homing BUM forwarding 727 GW2 is the non-DF for the I-ESI and blocks the BUM forwarding. GW1 is 728 the DF and forwards the traffic to PE1 and GW2. GW2 will only forward 729 the packets to local ACs (CE in the example). 731 3.4.4. Impact on MAC Mobility procedures 733 MAC Mobility procedures described in [RFC7432] are not modified by 734 this document. 736 Note that an intra-DC MAC move still leaves the MAC attached to the 737 same I-ESI, so under the rules of [RFC7432] this is not considered a 738 MAC mobility event. Only when the MAC moves from the WAN domain to 739 the DC domain (or from one DC to another) the MAC will be learned 740 from a different ES and the MAC Mobility procedures will kick in. 742 The sticky bit indication in the MAC Mobility extended community MUST 743 be propagated between domains. 745 3.4.5. Gateway optimizations 747 All the Gateway optimizations described in section 2.5 MAY be applied 748 to the GWs when the Interconnect is based on EVPN-MPLS. 750 In particular, the use of the Unknown MAC route, as described in 751 section 2.5.1, solves some transient packet duplication issues in 752 cases of all-active multi-homing, as explained below. 754 Consider the diagram in Figure 2 for EVPN-MPLS Interconnect and all- 755 active multi-homing, and the following sequence: 757 a) MAC Address M1 is advertised from NVE3 in EVI-1. 759 b) GW3 and GW4 learn M1 for EVI-1 and re-advertise M1 to the WAN 760 with I-ESI-2 in the ESI field. 762 c) GW1 and GW2 learn M1 and install GW3/GW4 as next-hops following 763 the EVPN aliasing procedures. 765 d) Before NVE1 learns M1, a packet arrives at NVE1 with 766 destination M1. If the Unknown MAC route had not been 767 advertised into the DC, NVE1 would have flooded the packet 768 throughout the DC, in particular to both GW1 and GW2. If the 769 same VNI/VSID is used for both known unicast and BUM traffic, 770 as is typically the case, there is no indication in the packet 771 that it is a BUM packet and both GW1 and GW2 would have 772 forwarded it. However, because the Unknown MAC route had been 773 advertised into the DC, NVE1 will unicast the packet to either 774 GW1 or GW2. 776 e) Since both GW1 and GW2 know M1, the GW receiving the packet 777 will forward it to either GW3 or GW4. 779 3.4.6. Benefits of the EVPN-MPLS Interconnect solution 781 Besides retaining the EVPN attributes between Data Centers and 782 throughout the WAN, the EVPN-MPLS Interconnect solution on the GWs 783 has some benefits compared to pure BGP EVPN RR or Inter-AS model B 784 solutions without a gateway: 786 o The solution supports the connectivity of local attachment 787 circuits on the GWs. 789 o Different data plane encapsulations can be supported in the DC 790 and the WAN. 792 o Optimized multicast solution, with independent inclusive 793 multicast trees in DC and WAN. 795 o MPLS Label aggregation: for the case where MPLS labels are 796 signaled from the NVEs for MAC/IP Advertisement routes, this 797 solution provides label aggregation. A remote PE MAY receive a 798 single label per GW MAC-VRF as opposed to a label per NVE/MAC- 799 VRF connected to the GW MAC-VRF. For instance, in Figure 2, PE 800 would receive only one label for all the routes advertised for a 801 given MAC-VRF from GW1, as opposed to a label per NVE/MAC-VRF. 803 o The GW will not propagate MAC mobility for the MACs moving 804 within a DC. Mobility intra-DC is solved by all the NVEs in the 805 DC. The MAC Mobility procedures on the GWs are only required in 806 case of mobility across DCs. 808 o Proxy-ARP/ND function on the DC GWs can be leveraged to reduce 809 ARP/ND flooding in the DC or/and in the WAN. 811 3.5. PBB-EVPN Interconnect for EVPN-Overlay networks 813 PBB-EVPN [RFC7623] is yet another Interconnect option. It requires 814 the use of GWs where I-components and associated B-components are 815 part of EVI instances. 817 3.5.1. Control/Data Plane setup procedures on the GWs 819 EVPN will run independently in both components, the I-component MAC- 820 VRF and B-component MAC-VRF. Compared to [RFC7623], the DC C-MACs are 821 no longer learned in the data plane on the GW but in the control 822 plane through EVPN running on the I-component. Remote C-MACs coming 823 from remote PEs are still learned in the data plane. B-MACs in the B- 824 component will be assigned and advertised following the procedures 825 described in [RFC7623]. 827 An I-ESI will be configured on the GWs for multi-homing, but it will 828 only be used in the EVPN control plane for the I-component EVI. No 829 non-reserved ESIs will be used in the control plane of the B- 830 component EVI as per [RFC7623]. 832 The rest of the control plane procedures will follow [RFC7432] for 833 the I-component EVI and [RFC7623] for the B-component EVI. 835 From the data plane perspective, the I-component and B-component EVPN 836 bindings established to the same far-end will be compared and the I- 837 component EVPN-overlay binding will be kept down following the rules 838 described in section 3.3.1. 840 3.5.2. Multi-homing procedures on the GWs 842 Single-active as well as all-active multi-homing MUST be supported. 844 The forwarding behavior of the DF and non-DF will be changed based on 845 the description outlined in section 3.4.3, only replacing the "WAN 846 split-horizon-group" for the B-component. 848 3.5.3. Impact on MAC Mobility procedures 850 C-MACs learned from the B-component will be advertised in EVPN within 851 the I-component EVI scope. If the C-MAC was previously known in the 852 I-component database, EVPN would advertise the C-MAC with a higher 853 sequence number, as per [RFC7432]. From a Mobility perspective and 854 the related procedures described in [RFC7432], the C-MACs learned 855 from the B-component are considered local. 857 3.5.4. Gateway optimizations 859 All the considerations explained in section 3.4.5 are applicable to 860 the PBB-EVPN Interconnect option. 862 3.6. EVPN-VXLAN Interconnect for EVPN-Overlay networks 864 If EVPN for Overlay tunnels is supported in the WAN and a GW function 865 is required, an end-to-end EVPN solution can be deployed. This 866 section focuses on the specific case of EVPN for VXLAN (EVPN-VXLAN 867 hereafter) and the impact on the [RFC7432] procedures. 869 This use-case assumes that NVEs need to use the VNIs or VSIDs as a 870 globally unique identifiers within a data center, and a Gateway needs 871 to be employed at the edge of the data center network to translate 872 the VNI or VSID when crossing the network boundaries. This GW 873 function provides VNI and tunnel IP address translation. The use-case 874 in which local downstream assigned VNIs or VSIDs can be used (like 875 MPLS labels) is described by [EVPN-Overlays]. 877 While VNIs are globally significant within each DC, there are two 878 possibilities in the Interconnect network: 880 a) Globally unique VNIs in the Interconnect network: 881 In this case, the GWs and PEs in the Interconnect network will 882 agree on a common VNI for a given EVI. The RT to be used in the 883 Interconnect network can be auto-derived from the agreed 884 Interconnect VNI. The VNI used inside each DC MAY be the same 885 as the Interconnect VNI. 887 b) Downstream assigned VNIs in the Interconnect network. 888 In this case, the GWs and PEs MUST use the proper RTs to 889 import/export the EVPN routes. Note that even if the VNI is 890 downstream assigned in the Interconnect network, and unlike 891 option B, it only identifies the pair and 892 not the pair. The VNI used inside 893 each DC MAY be the same as the Interconnect VNI. GWs SHOULD 894 support multiple VNI spaces per EVI (one per Interconnect 895 network they are connected to). 897 In both options, NVEs inside a DC only have to be aware of a single 898 VNI space, and only GWs will handle the complexity of managing 899 multiple VNI spaces. In addition to VNI translation above, the GWs 900 will provide translation of the tunnel source IP for the packets 901 generated from the NVEs, using their own IP address. GWs will use 902 that IP address as the BGP next-hop in all the EVPN updates to the 903 Interconnect network. 905 The following sections provide more details about these two options. 907 3.6.1. Globally unique VNIs in the Interconnect network 909 Considering Figure 2, if a host H1 in NVO-1 needs to communicate with 910 a host H2 in NVO-2, and assuming that different VNIs are used in each 911 DC for the same EVI, e.g. VNI-10 in NVO-1 and VNI-20 in NVO-2, then 912 the VNIs MUST be translated to a common Interconnect VNI (e.g. VNI- 913 100) on the GWs. Each GW is provisioned with a VNI translation 914 mapping so that it can translate the VNI in the control plane when 915 sending BGP EVPN route updates to the Interconnect network. In other 916 words, GW1 and GW2 MUST be configured to map VNI-10 to VNI-100 in the 917 BGP update messages for H1's MAC route. This mapping is also used to 918 translate the VNI in the data plane in both directions, that is, VNI- 919 10 to VNI-100 when the packet is received from NVO-1 and the reverse 920 mapping from VNI-100 to VNI-10 when the packet is received from the 921 remote NVO-2 network and needs to be forwarded to NVO-1. 923 The procedures described in section 3.4 will be followed, considering 924 that the VNIs advertised/received by the GWs will be translated 925 accordingly. 927 3.6.2. Downstream assigned VNIs in the Interconnect network 929 In this case, if a host H1 in NVO-1 needs to communicate with a host 930 H2 in NVO-2, and assuming that different VNIs are used in each DC for 931 the same EVI, e.g. VNI-10 in NVO-1 and VNI-20 in NVO-2, then the VNIs 932 MUST be translated as in section 3.6.1. However, in this case, there 933 is no need to translate to a common Interconnect VNI on the GWs. Each 934 GW can translate the VNI received in an EVPN update to a locally 935 assigned VNI advertised to the Interconnect network. Each GW can use 936 a different Interconnect VNI, hence this VNI does not need to be 937 agreed on all the GWs and PEs of the Interconnect network. 939 The procedures described in section 3.4 will be followed, taking the 940 considerations above for the VNI translation. 942 5. Conventions and Terminology 944 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 945 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 946 document are to be interpreted as described in RFC-2119 [RFC2119]. 948 AC: Attachment Circuit 950 BUM: it refers to the Broadcast, Unknown unicast and Multicast 951 traffic 953 DF: Designated Forwarder 955 GW: Gateway or Data Center Gateway 957 DCI: Data Center Interconnect 959 ES: Ethernet Segment 961 ESI: Ethernet Segment Identifier 963 I-ESI: Interconnect ESI defined on the GWs for multi-homing to/from 964 the WAN 966 EVI: EVPN Instance 968 MAC-VRF: it refers to an EVI instance in a particular node 970 NVE: Network Virtualization Edge 972 PW: Pseudowire 974 RD: Route-Distinguisher 976 RT: Route-Target 978 TOR: Top-Of-Rack switch 980 VNI/VSID: refers to VXLAN/NVGRE virtual identifiers 982 VSI: Virtual Switch Instance or VPLS instance in a particular PE 984 6. Security Considerations 986 Security considerations included in [RFC7432], [RFC4761] and 987 [RFC4762] apply to this document. 989 7. IANA Considerations 991 8. References 992 8.1. Normative References 994 [RFC4761]Kompella, K., Ed., and Y. Rekhter, Ed., "Virtual Private LAN 995 Service (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 4761, 996 DOI 10.17487/RFC4761, January 2007, . 999 [RFC4762]Lasserre, M., Ed., and V. Kompella, Ed., "Virtual Private 1000 LAN Service (VPLS) Using Label Distribution Protocol (LDP) 1001 Signaling", RFC 4762, DOI 10.17487/RFC4762, January 2007, 1002 . 1004 [RFC6074]Rosen, E., Davie, B., Radoaca, V., and W. Luo, 1005 "Provisioning, Auto-Discovery, and Signaling in Layer 2 Virtual 1006 Private Networks (L2VPNs)", RFC 6074, DOI 10.17487/RFC6074, January 1007 2011, . 1009 [RFC7041]Balus, F., Ed., Sajassi, A., Ed., and N. Bitar, Ed., 1010 "Extensions to the Virtual Private LAN Service (VPLS) Provider Edge 1011 (PE) Model for Provider Backbone Bridging", RFC 7041, DOI 1012 10.17487/RFC7041, November 2013, . 1015 [RFC7432]Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 1016 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet 1017 VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . 1020 [RFC2119]Bradner, S., "Key words for use in RFCs to Indicate 1021 Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1022 1997, . 1024 [RFC5512]Mohapatra, P. and E. Rosen, "The BGP Encapsulation 1025 Subsequent Address Family Identifier (SAFI) and the BGP Tunnel 1026 Encapsulation Attribute", RFC 5512, DOI 10.17487/RFC5512, April 2009, 1027 . 1029 [RFC7623] Sajassi et al., "Provider Backbone Bridging Combined with 1030 Ethernet VPN (PBB-EVPN)", RFC 7623, September, 2015, . 1033 8.2. Informative References 1035 [EVPN-Overlays] Sajassi-Drake et al., "A Network Virtualization 1036 Overlay Solution using EVPN", draft-ietf-bess-evpn-overlay-08.txt, 1037 work in progress, March, 2017 1039 [EVPN-VPLS-INTEGRATION] Sajassi et al., "(PBB-)EVPN Seamless 1040 Integration with (PBB-)VPLS", draft-ietf-bess-evpn-vpls-integration- 1041 00.txt, work in progress, February, 2015 1043 9. Acknowledgments 1045 The authors would like to thank Neil Hart, Vinod Prabhu and Kiran 1046 Nagaraj for their valuable comments and feedback. 1048 10. Contributors 1050 In addition to the authors listed on the front page, the following 1051 co-authors have also contributed to this document: 1053 Ravi Shekhar 1054 Anil Lohiya 1055 Wen Lin 1056 Juniper Networks 1058 Florin Balus 1059 Patrice Brissette 1060 Cisco 1062 Senad Palislamovic 1063 Nokia 1065 Dennis Cai 1066 Alibaba 1068 11. Authors' Addresses 1070 Jorge Rabadan 1071 Nokia 1072 777 E. Middlefield Road 1073 Mountain View, CA 94043 USA 1074 Email: jorge.rabadan@nokia.com 1076 Senthil Sathappan 1077 Nokia 1078 Email: senthil.sathappan@nokia.com 1080 Wim Henderickx 1081 Nokia 1082 Email: wim.henderickx@nokia.com 1084 Ali Sajassi 1085 Cisco 1086 Email: sajassi@cisco.com 1088 John Drake 1089 Juniper 1090 Email: jdrake@juniper.net