idnits 2.17.1 draft-ietf-rtgwg-net2cloud-gap-analysis-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 8, 2020) is 1507 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC2332' is mentioned on line 167, but not defined == Missing Reference: 'BGP-SDWAN-PORT' is mentioned on line 200, but not defined == Missing Reference: 'SECURE-EVPN' is mentioned on line 348, but not defined == Missing Reference: 'Tunnel-Encap' is mentioned on line 301, but not defined == Missing Reference: 'RFC4364' is mentioned on line 328, but not defined == Missing Reference: 'MEF-Cloud' is mentioned on line 413, but not defined == Missing Reference: 'RFC6325' is mentioned on line 556, but not defined == Unused Reference: 'RFC2119' is defined on line 601, but no explicit reference was found in the text == Unused Reference: 'RFC8192' is defined on line 606, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 609, but no explicit reference was found in the text == Unused Reference: 'ITU-T-X1036' is defined on line 639, but no explicit reference was found in the text -- No information found for draft-dunbar-idr-sdwan-framework - is the name correct? == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 == Outdated reference: A later version (-07) exists of draft-dm-net2cloud-problem-statement-02 Summary: 0 errors (**), 0 flaws (~~), 14 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft Futurewei 3 Intended status: Informational A. Malis 4 Expires: September 8, 2020 Independent 6 C. Jacquenet 7 Orange 8 March 8, 2020 10 Gap Analysis of Dynamic Networks to Hybrid Cloud DCs 11 draft-ietf-rtgwg-net2cloud-gap-analysis-04 13 Abstract 15 This document analyzes the technological gaps, especially IETF 16 protocols gaps, to achieve dynamically interconnecting workloads and 17 applications hosted in Hybrid Cloud Data Centers. 19 Status of this Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF), its areas, and its working groups. Note that 26 other groups may also distribute working documents as Internet- 27 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other documents 31 at any time. It is inappropriate to use Internet-Drafts as 32 reference material or to cite them other than as "work in progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt 37 The list of Internet-Draft Shadow Directories can be accessed at 38 http://www.ietf.org/shadow.html 40 This Internet-Draft will expire on September 8, 2020. 42 Copyright Notice 44 Copyright (c) 2020 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with 52 respect to this document. Code Components extracted from this 53 document must include Simplified BSD License text as described in 54 Section 4.e of the Trust Legal Provisions and are provided without 55 warranty as described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction...................................................3 60 2. Conventions used in this document..............................3 61 3. Gap Analysis for Accessing Cloud Resources.....................4 62 4. Gap Analysis of Overlay Edge Node's WAN Ports Management.......4 63 5. Aggregating VPN paths and Internet paths.......................6 64 5.1. Control Plane for Overlay over Heterogeneous Networks.....7 65 5.2. Using BGP UPDATE Messages.................................8 66 5.3. SECURE-L3VPN/EVPN.........................................9 67 5.4. Preventing attacks from Internet-facing ports............10 68 6. C-PEs not directly connected to VPN PEs.......................10 69 6.1. Floating PEs to connect to Remote CPEs...................13 70 6.2. NAT Traversal............................................13 71 6.3. Complexity of using BGP between PEs and remote CPEs via 72 Internet......................................................13 73 6.4. Designated Forwarder to the remote edges.................14 74 6.5. Traffic Path Management..................................15 75 7. Manageability Considerations..................................15 76 8. Security Considerations.......................................15 77 9. IANA Considerations...........................................16 78 10. References...................................................16 79 10.1. Normative References....................................16 80 10.2. Informative References..................................16 81 11. Acknowledgments..............................................17 83 1. Introduction 85 [Net2Cloud-Problem] describes the problems enterprises face today 86 when interconnecting their branch offices with dynamic workloads in 87 third party data centers (a.k.a. Cloud DCs). This document analyzes 88 the IETF routing protocols to identify if there are gaps or if 89 protocol extension might be needed. 91 For the sake of readability, an edge, an endpoint, C-PE, or CPE are 92 used interchangeably throughout this document. However, each term 93 has some minor emphasis, especially when used in other related 94 documents: 96 . Edge: could include multiple devices (virtual or physical); 97 . endpoint: to refer to a WAN port of an Edge device; 98 . C-PE: more for provider owned edge, e.g. for SECURE-EVPN's PE 99 based VPN, where PE is the edge node; 100 . CPE: more for enterprise owned edge. 102 2. Conventions used in this document 104 Cloud DC: Third party Data Centers that usually host applications 105 and workload owned by different organizations or 106 tenants. 108 Controller: Used interchangeably with Overlay controller to manage 109 overlay path creation/deletion and monitor the path 110 conditions between sites. 112 CPE-Based VPN: Virtual Private Network designed and deployed from 113 CPEs. This is to differentiate from most commonly used 114 PE-based VPNs a la RFC 4364. 116 OnPrem: On Premises data centers and branch offices 118 SDWAN: Software Defined Wide Area Network, "SDWAN" refers to 119 the solutions of pooling WAN bandwidth from multiple 120 underlay networks to get better WAN bandwidth 121 management, visibility & control. When the underlay is a 122 private network, traffic may be forwarded without any 123 additional encryption; when the underlay networks are 124 public, such as the Internet, some traffic needs to be 125 encrypted when passing through (depending on user- 126 provided policies). 128 3. Gap Analysis for Accessing Cloud Resources 130 Many problems described in the [Net2Cloud-Problem] are not in the 131 scope of IETF, let alone IETF Routing area. Therefore, this document 132 will not cover the detailed protocol gaps analysis for security, 133 identity management or DNS for Cloud Resources. 135 4. Gap Analysis of Overlay Edge Node's WAN Ports Management 137 Very often the Hybrid Cloud DCs are interconnected by overlay 138 networks that arch over many different types of networks, such as 139 VPN, public internet, wireless, etc. Sometimes the enterprises' VPN 140 providers do not have direct access to the Cloud DCs that are 141 optimal for some specific applications or workloads. 143 Under those circumstances, the overlay network' edges can have WAN 144 ports facing networks provided by different ISPs, some can be 145 untrusted public internet, some can be trusted provider VPN, some 146 can be Cloud internal networks, and some can be others. 148 If all WAN ports of an edge node are facing untrusted network, then 149 all sensitive data to/from this edge have to be encrypted, usually 150 by IPsec tunnels which can be terminated at the WAN port address, at 151 the edge node's loopback address if the loopback address is routable 152 in the wide area network, or even at the ingress ports of the edge 153 node. 155 If an edge node has some WAN ports facing trusted VPN and some 156 facing untrusted networks, sensitive data can be forwarded through 157 ports facing VPN natively without encryption and forwarded through 158 ports facing public network with encryption. To achieve this 159 flexibility, it is necessary to have the IPsec tunnels terminated at 160 the WAN ports facing the untrusted networks. 162 In order to establish pair-wise secure encrypted connection among 163 those WAN ports, it is necessary for peers to be informed of the WAN 164 port properties. 166 Some of those overlay networks (such as some deployed SDWAN 167 networks) use the modified NHRP protocol [RFC2332] to register WAN 168 ports of the edges with their "Controller" (or NHRP server), which 169 then map a private VPN address to a public IP address of the 170 destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to 171 establish tunnels between WAN ports of SDWAN edge nodes. 173 NHRP was originally intended for ATM address resolution, and as a 174 result, it misses many attributes that are necessary for dynamic 175 endpoint C-PE registration to the controller, such as: 177 - Interworking with the MPLS VPN control plane. An overlay edge can 178 have some ports facing the MPLS VPN network over which packets can 179 be forwarded without any encryption and some ports facing the 180 public Internet over which sensitive traffic needs to be 181 encrypted. 182 - Scalability: NHRP/DSVPN/DMVPN works fine with small numbers of 183 edge nodes. When a network has more than 100 nodes, these 184 protocols do not scale well. 185 - NHRP does not have the IPsec attributes, which are needed for 186 peers to build Security Associations over the public internet. 187 - NHRP messages do not have any field to encode the C-PE supported 188 encapsulation types, such as IPsec-GRE or IPsec-VxLAN. 189 - NHRP messages do not have any field to encode C-PE Location 190 identifiers, such as Site Identifier, System ID, and/or Port ID. 191 - NHRP messages do not have any field to describe the gateway(s) to 192 which the C-PE is attached. When a C-PE is instantiated in a Cloud 193 DC, it is desirable for C-PE's owner to be informed of how/where 194 the C-PE is attached. 195 - NHRP messages do not have any field to describe C-PE's NAT 196 properties if the C-PE is using private addresses, such as the NAT 197 type, Private address, Public address, Private port, Public port, 198 etc. 200 [BGP-SDWAN-PORT] describes how to use BGP to distribute SDWAN edge 201 properties to peers. There is need to extend the protocol to 202 register WAN ports properties to the overlay controller, which then 203 propagates the information to other overlay edge nodes that are 204 authenticated and authorized to communicate with them. 206 5. Aggregating VPN paths and Internet paths 208 Most likely, enterprises (especially the largest ones) already have 209 their C-PEs interconnected by providers VPNs, such as EVPN, L2VPN, 210 or L3VPN, which can be PE-based or CPE-based. The commonly used PE- 211 based VPNs have C-PE directly attached to PEs, therefore the 212 communication between C-PEs and PEs is considered as secure. MP-BGP 213 is used to learn & distribute routes among C-PEs, even though 214 sometimes routes among C-PEs are statically configured on the C-PEs. 216 For enterprises already interconnected by VPNs, it is desirable to 217 aggregate the bandwidth among VPN paths and Internet paths by C-PEs 218 adding additional ports facing public internet. Under this scenario, 219 which is referred to as Overlay throughout this document, it is 220 necessary for the C-PEs to manage and communicate with controller on 221 how traffic are distributed among multiple heterogenous WAN underlay 222 networks, and manage secure tunnels over untrusted networks 223 independently from the attached clients routes. 225 When using NHRP for WAN ports registration purposes, C-PEs need to 226 run two separate control planes: EVPN&BGP for CPE-based VPNs, and 227 NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate 228 control planes not only add complexity to C-PEs, but also increase 229 operational cost. 231 +---+ 232 +--------------|RR |----------+ 233 / Untrusted +-+-+ \ 234 / \ 235 / \ 236 +----+ +---------+ packets encrypted over +------+ +----+ 237 | TN3|--| A1-----+ Untrusted +------ B1 |--| TN1| 238 +----+ | C-PE A2-\ | C-PE | +----+ 239 +----+ | A A3--+--+ +---+---B2 B | +----+ 240 | TN2|--| | |PE+--------------+PE |---B3 |--| TN3| 241 +----+ +---------+ +--+ trusted +---+ +------+ +----+ 242 | WAN | 243 +----+ +---------+ +--+ packets +---+ +------+ +----+ 244 | TN1|--| C1--|PE| go natively |PE |-- D1 |--| TN1| 245 +----+ | C-PE C2--+--+ without encry+---+ | C-PE | +----+ 246 | C | +--------------+ | D | 247 | | | | 248 +----+ | C3--| without encrypt over | | +----+ 249 | TN2|--| C4--+---- Untrusted --+------D2 |--| TN2| 250 +----+ +---------+ +------+ +----+ 251 Figure 1: CPEs interconnected by VPN paths and Internet Paths 253 5.1. Control Plane for Overlay over Heterogeneous Networks 255 As described in [BGP-SDWAN-Usage], the Control Plane for Overlay 256 network over heterogenous networks has three distinct properties: 258 - WAN Port Property registration to the Overlay Controller. 259 o To inform the Overlay controller and authorized peers of 260 the WAN port properties of the Edge nodes. When the WAN 261 ports are assigned private addresses, this step can 262 register the type of NAT that translates private addresses 263 into public ones. 265 - Controller facilitated IPsec SA management and NAT information 266 distribution 267 o It is for Overlay controller to facilitate or manage the 268 IPsec configuration and peer authentication for all IPsec 269 tunnels terminated at the edge nodes. 271 - Establishing and Managing the topology and reachability for 272 services attached to the client ports of overlay edge nodes. 274 o This is for the overlay layer's route distribution, so 275 that a C-PE can populate its overlay routing table with 276 entries that identify the next hop for reaching a specific 277 route/service attached to remote nodes. [SECURE-EVPN] 278 describes EVPN and other options. 280 5.2. Using BGP UPDATE Messages 282 [Tunnel-Encap] describe the BGP UPDATE Tunnel Path Attribute that 283 advertise endpoints' tunnel encapsulation capability for the 284 respective attached client routes encoded in the MP-NLRI Path 285 Attribute. The receivers of the BGP UPDATE can use any of the 286 supported encapsulations encoded in the Tunnel Path Attribute for 287 the routes encoded in the MP-NLRI Path Attribute. 289 Here are some of the gaps using [Tunnel-Encap] to distribute Edge 290 WAN port properties: 292 - [Tunnel-Encap] doesn't yet have the encoding to describe the NAT 293 information for WAN ports that have private addresses. The NAT 294 information needs to be propagated to the trusted peers via 295 Controllers, such as the virtual C-PEs instantiated in public 296 Cloud DCs. 297 - It is not easy using the current mechanism in [Tunnel-Encap] to 298 exchange IPsec SA specific parameters independently from 299 advertising the attached clients' routes, even after adding a new 300 IPsec tunnel type. 301 [Tunnel-Encap] requires all tunnels updates are associated with 302 routes. There can be many client routes associated with the IPsec 303 tunnel between two C-PEs' WAN ports; the corresponding destination 304 prefixes (as announced by the aforementioned routes) may also be 305 reached through the VPN underlay without any encryption. 307 The establishment of an IPsec tunnel can fail, such as due to the 308 two endpoints supporting different encryption algorithms or other 309 reasons. There can be multiple negotiations messages for the IPsec 310 SA parameters between two end points. That is why IPsec SA 311 association establishment between end points is independent from 312 the policies on mapping routes to specific IPSec SA. 314 If C-PEs need to establish WAN Port based IPsec SA, the 315 information encoded in Tunnel Path Attribute should only apply to 316 the WAN ports and should be independent of the clients' routes. 318 In addition, the Overlay IPsec SA Tunnel may need to be 319 established before clients' routes are attached. 321 - C-PEs tend to communicate with a subset of the other C-PEs, not 322 all the C-PEs need to be connected through a mesh topology. 323 Therefore, the distribution of the Overlay Edge WAN ports 324 information need be be scoped to the authorized peers. 326 5.3. SECURE-L3VPN/EVPN 328 [SECURE-L3VPN] describes how to extend the BGP/MPLS VPN [RFC4364] 329 capabilities to allow some PEs to connect to other PEs via public 330 networks. [SECURE-L3VPN] introduces the concept of Red Interface & 331 Black Interface used by PEs, where the RED interfaces are used to 332 forward traffic into the VPN, and the Black Interfaces are used 333 between WAN ports through which only IPsec-protected packets are 334 forwarded to the Internet or to other backbone network thereby 335 eliminating the need for MPLS transport in the backbone. 337 [SECURE-L3VPN] assumes PEs using MPLS over IPsec when sending 338 traffic through the Black Interfaces. 340 [SECURE-EVPN] describes a solution where point-to-multipoint BGP 341 signaling is used in the control plane for the Scenario #1 described 342 in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to 343 facilitate the key and policy exchange among PE devices to create 344 private pair-wise IPsec Security Associations without IKEv2 point- 345 to-point signaling or any other direct peer-to-peer session 346 establishment messages. 348 Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both 349 miss the aspects of aggregating VPN and Internet underlays. In 350 summary: 352 - Both documents assume a client traffic is either forwarded all 353 encrypted through an IPsec tunnel, or not encrypted at all through 354 a different tunnel regardless which WAN ports the traffic egress 355 the PEs towards WAN. For Overlay arch over trusted VPN and 356 untrusted Internet, one client traffic can be forwarded encrypted 357 at one time through a WAN port towards untrusted network and be 358 forwarded unencrypted at different time through a WAN port to MPLS 359 VPN. 361 - The [SECURE-L3VPN] assumes that a CPE "registers" with the RR. 362 However, it does not say how. It assumes that the remote CPEs are 363 pre-configured with the IPsec SA manually. In Overlay network to 364 connect Hybrid Cloud DCs, Zero Touch Provisioning is expected. 365 Manual configuration is not an option, especially for the edge 366 devices that are deployed in faraway places. 368 - The [SECURE-L3VPN] assumes that C-PEs and RR are connected via an 369 IPsec tunnel. Missing TLS/DTLS. The following assumption by 370 [SECURE-L3VPN] becomes invalid for the Overlay network to connect 371 Hybrid Cloud DCs where automatic synchronization of IPsec SA 372 between C-PEs and RR is needed: 373 A CPE must also be provisioned with whatever additional 374 information is needed in order to set up an IPsec SA with 375 each of the red RRs 377 - IPsec requires periodic refreshment of the keys. The draft does 378 not provide any information about how to synchronize the 379 refreshment among multiple nodes. 380 - IPsec usually sends configuration parameters to two endpoints only 381 and lets these endpoints negotiate the key. The [SECURE-L3VPN] 382 assumes that the RR is responsible for creating/managing the key 383 for all endpoints. When one endpoint is compromised, all other 384 connections will be impacted. 386 5.4. Preventing attacks from Internet-facing ports 388 When C-PEs have Internet-facing ports, additional security risks are 389 raised. 391 To mitigate security risks, in addition to requiring Anti-DDoS 392 features on C-PEs, it is necessary for C-PEs to support means to 393 determine whether traffic sent by remote peers is legitimate to 394 prevent spoofing attacks. 396 6. C-PEs not directly connected to VPN PEs 398 Because of the ephemeral property of the selected Cloud DCs for 399 specific workloads/Apps, an enterprise or its network service 400 provider may not have direct physical connections to the Cloud DCs 401 that are optimal for hosting the enterprise's specific 402 workloads/Apps. Under those circumstances, Overlay is a very 403 flexible choice to interconnect the enterprise on-premises data 404 centers & branch offices to its desired Cloud DCs. 406 However, Overlay paths established over the public Internet can have 407 unpredictable performance, especially over long distances and across 408 operators' domains. Therefore, it is highly desirable to steer as 409 much as possible the portion of Overlay paths over the enterprise's 410 existing VPN that has guaranteed SLA to minimize the distance or the 411 number of segments over the public Internet. 413 MEF Cloud Service Architecture [MEF-Cloud] also describes a use case 414 of network operators using Overlay path over LTE or the public 415 Internet for last mile access where the VPN service providers cannot 416 necessarily provide the required physical infrastructure. 418 Under those scenarios, one or two of the Overlay endpoints may not 419 be directly attached to the PEs of a VPN Domain. 421 When using Overlay to connect the enterprise's existing sites to the 422 workloads hosted in Cloud DCs, the corresponding C-PEs have to be 423 upgraded to support the desired Overlay. If the workloads hosted in 424 Cloud DCs need to be connected to many sites, the upgrade process 425 can be very expensive. 427 [Net2Cloud-Problem] describes a hybrid network approach that extend 428 the existing MPLS-based VPNs to the Cloud DC Workloads over the 429 access paths that are not under the VPN provider's control. To make 430 it work properly, a small number of the PEs of the MPLS VPN can be 431 designated to connect to the remote workloads via secure IPsec 432 tunnels. Those designated PEs are shown as fPE (floating PE or 433 smart PE) in Figure 3. Once the secure IPsec tunnels are 434 established, the workloads hosted in Cloud DCs can be reached by the 435 enterprise's VPN without upgrading all of the enterprise's existing 436 CPEs. The only CPE that needs to support the Overlay would be a 437 virtualized CPE instantiated within the cloud DC. 439 +--------+ +--------+ 440 | Host-a +--+ +----| Host-b | 441 | | | (') | | 442 +--------+ | +-----------+ ( ) +--------+ 443 | +-+--+ ++-+ ++-+ +--+-+ (_) 444 | | CPE|--|PE| |PE+--+ CPE| | 445 +--| | | | | | | |---+ 446 +-+--+ ++-+ ++-+ +----+ 447 / | | 448 / | MPLS +-+---+ +--+-++--------+ 449 +------+-+ | Network |fPE-1| |CPE || Host | 450 | Host | | | |- --| || d | 451 | c | +-----+ +-+---+ +--+-++--------+ 452 +--------+ |fPE-2|-----+ 453 +---+-+ (|) 454 (|) (|) Overlay 455 (|) (|) over any access 456 +=\======+=========+ 457 // \ | Cloud DC \\ 458 // \ ++-----+ \\ 459 +Remote| 460 | CPE | 461 +-+----+ 462 ----+-------+-------+----- 463 | | 464 +---+----+ +---+----+ 465 | Remote | | Remote | 466 | App-1 | | App-2 | 467 +--------+ +--------+ 469 Figure 3: VPN Extension to Cloud DC 471 In Figure 3, the optimal Cloud DC to host the workloads (as a 472 function of the proximity, capacity, pricing, or other criteria 473 chosen by the enterprises) does not have a direct connection to the 474 PEs of the MPLS VPN that interconnects the enterprise's existing 475 sites. 477 6.1. Floating PEs to connect to Remote CPEs 479 To extend MPLS VPNs to remote CPEs, it is necessary to establish 480 secure tunnels (such as IPsec tunnels) between the Floating PEs and 481 the remote CPEs. 483 Even though a set of PEs can be manually selected to act as the 484 floating PEs for a specific cloud data center, there are no standard 485 protocols for those PEs to interact with the remote CPEs (most 486 likely virtualized) instantiated in the third party cloud data 487 centers (such as exchanging performance or route information). 489 When there is more than one fPE available for use (as there should 490 be for resiliency purposes or the ability to support multiple cloud 491 DCs geographically scattered), it is not straightforward to 492 designate an egress fPE to remote CPEs based on applications. There 493 is too much applications' traffic traversing PEs, and it is not 494 feasible for PEs to recognize applications from the payload of 495 packets. 497 6.2. NAT Traversal 499 Cloud DCs that only assign private IPv4 addresses to the 500 instantiated workloads assume that traffic to/from the workload 501 usually needs to traverse NATs. 502 An overlay edge node can solicit a STUN (Session Traversal of UDP 503 Through Network Address Translation RFC 3489) Server to get the NAT 504 property, the public IP address and the Public Port number so that 505 such information can be communicated to the relevant peers. 507 6.3. Complexity of using BGP between PEs and remote CPEs via Internet 509 Even though an EBGP (external BGP) Multi-hop design can be used to 510 connect peers that are not directly connected to each other, there 511 are still some complications in extending BGP from MPLS VPN PEs to 512 remote CPEs via any access path (e.g., Internet). 514 The path between the remote CPEs and VPN PEs that maintain VPN 515 routes may very well traverse untrusted nodes. 517 EBGP Multi-hop design requires static configuration on both peers. 518 To use EBGP between a PE and remote CPEs, the PE has to be manually 519 configured with the "next-hop" set to the IP address of the CPEs. 520 When remote CPEs, especially remote virtualized CPEs are dynamically 521 instantiated or removed, the configuration of Multi-Hop EBGP on the 522 PE has to be changed accordingly. 524 Egress peering engineering (EPE) is not sufficient. Running BGP on 525 virtualized CPEs in Cloud DCs requires GRE tunnels to be 526 established first, which requires the remote CPEs to support 527 address and key management capabilities. RFC 7024 (Virtual Hub & 528 Spoke) and Hierarchical VPN do not support the required 529 properties. 531 Also, there is a need for a mechanism to automatically trigger 532 configuration changes on PEs when remote CPEs' are instantiated or 533 moved (leading to an IP address change) or deleted. 535 EBGP Multi-hop design does not include a security mechanism by 536 default. The PE and remote CPEs need secure communication channels 537 when connecting via the public Internet. 539 Remote CPEs, if instantiated in Cloud DCs, might have to traverse 540 NATs to reach PEs. It is not clear how BGP can be used between 541 devices located beyond the NAT and the devices located behind the 542 NAT. It is not clear how to configure the Next Hop on the PEs to 543 reach private IPv4 addresses. 545 6.4. Designated Forwarder to the remote edges 547 Among the multiple floating PEs that are reachable from a remote 548 CPE, multicast traffic sent by the remote CPE towards the MPLS VPN 549 can be forwarded back to the remote CPE due to the PE receiving the 550 multicast packets forwarding the multicast/broadcast frame to other 551 PEs that in turn send to all attached CPEs. This process may cause 552 traffic loops. 554 Therefore, it is necessary to designate one floating PE as the CPE's 555 Designated Forwarder, similar to TRILL's Appointed Forwarders 556 [RFC6325]. 558 MPLS VPNs do not have features like TRILL's Appointed Forwarders. 560 6.5. Traffic Path Management 562 When there are multiple floating PEs that have established IPsec 563 tunnels with the remote CPE, the remote CPE can forward outbound 564 traffic to the Designated Forwarder PE, which in turn forwards 565 traffic to egress PEs and then to the final destinations. However, 566 it is not straightforward for the egress PE to send back the return 567 traffic to the Designated Forwarder PE. 569 Example of Return Path management using Figure 3 above. 571 - fPE-1 is DF for communication between App-1 <-> Host-a due to 572 latency, pricing or other criteria. 573 - fPE-2 is DF for communication between App-1 <-> Host-b. 575 7. Manageability Considerations 577 Zero touch provisioning of Overlay networks to interconnect Hybrid 578 Clouds is highly desired. It is necessary for a newly powered up 579 edge node to establish a secure connection (by means of TLS, DTLS, 580 etc.) with its controller. 582 8. Security Considerations 584 Cloud Services is built upon shared infrastructure, therefore not 585 secure by nature. 587 Secure user identity management, authentication, and access 588 control mechanisms are important. Developing appropriate security 589 measurements can enhance the confidence needed by enterprises to 590 fully take advantage of Cloud Services. 592 9. IANA Considerations 594 This document requires no IANA actions. RFC Editor: Please remove 595 this section before publication. 597 10. References 599 10.1. Normative References 601 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 602 Requirement Levels", BCP 14, RFC 2119, March 1997. 604 10.2. Informative References 606 [RFC8192] S. Hares, et al, "Interface to Network Security Functions 607 (I2NSF) Problem Statement and Use Cases", July 2017 609 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 610 Address Family Identifier (SAFI) and the BGP Tunnel 611 Encapsulation Attribute", April 2009. 613 [BGP-SDWAN-PORT]L. Dunbar, et al, "Subsequent Address Family 614 Indicator for SDWAN Ports", draft-dunbar-idr-sdwan-port- 615 safi-00, Work-in-progress, March 2019. 617 [BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for 618 SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework- 619 00, work-in-progress, Feb 2019. 621 [Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation 622 Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018. 624 [SECURE-EVPN A. Sajassi, et al, draft-sajassi-bess-secure-evpn-01, 625 work in progress, March 2019. 627 [SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public 628 Infrastructure", draft-rosen-bess-secure-l3vpn-00, work- 629 in-progress, July 2018 631 [DMVPN] Dynamic Multi-point VPN: 632 https://www.cisco.com/c/en/us/products/security/dynamic- 633 multipoint-vpn-dmvpn/index.html 635 [DSVPN] Dynamic Smart VPN: 636 http://forum.huawei.com/enterprise/en/thread-390771-1- 637 1.html 639 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 640 storage, distribution and enforcement of policies for 641 network security", Nov 2007. 643 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect 644 Underlay to Cloud Overlay Problem Statement", draft-dm- 645 net2cloud-problem-statement-02, June 2018 647 11. Acknowledgments 649 Acknowledgements to John Drake for his review and contributions. 650 Many thanks to John Scudder for stimulating the clarification 651 discussion on the Tunnel-Encap draft so that our gap analysis can be 652 more accurate. 654 This document was prepared using 2-Word-v2.0.template.dot. 656 Authors' Addresses 658 Linda Dunbar 659 Futurewei 660 Email: ldunbar@futurewei.com 662 Andrew G. Malis 663 Independent 664 Email: agmalis@gmail.com 666 Christian Jacquenet 667 Orange 668 Rennes, 35000 669 France 670 Email: Christian.jacquenet@orange.com