idnits 2.17.1 draft-ietf-rtgwg-net2cloud-gap-analysis-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 2) being 104 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 336 has weird spacing: '...t carry the I...' -- The document date (March 18, 2020) is 1500 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC2332' is mentioned on line 173, but not defined == Missing Reference: 'BGP-SDWAN-PORT' is mentioned on line 206, but not defined == Missing Reference: 'SECURE-EVPN' is mentioned on line 378, but not defined == Missing Reference: 'Tunnel-Encap' is mentioned on line 327, but not defined == Missing Reference: 'RFC4364' is mentioned on line 358, but not defined == Missing Reference: 'MEF-Cloud' is mentioned on line 442, but not defined == Missing Reference: 'RFC3489' is mentioned on line 533, but not defined ** Obsolete undefined reference: RFC 3489 (Obsoleted by RFC 5389) == Missing Reference: 'RFC6325' is mentioned on line 588, but not defined == Unused Reference: 'RFC2119' is defined on line 634, but no explicit reference was found in the text == Unused Reference: 'RFC8192' is defined on line 639, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 642, but no explicit reference was found in the text == Unused Reference: 'ITU-T-X1036' is defined on line 672, but no explicit reference was found in the text -- No information found for draft-dunbar-idr-sdwan-framework - is the name correct? == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 == Outdated reference: A later version (-07) exists of draft-dm-net2cloud-problem-statement-02 Summary: 1 error (**), 0 flaws (~~), 17 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft Futurewei 3 Intended status: Informational A. Malis 4 Expires: September 18, 2020 Independent 6 C. Jacquenet 7 Orange 8 March 18, 2020 10 Networks Connecting to Hybrid Cloud DCs: Gap Analysis 11 draft-ietf-rtgwg-net2cloud-gap-analysis-05 13 Abstract 15 This document analyzes the technical gaps that may affect the 16 dynamic connection to workloads and applications hosted in hybrid 17 Cloud Data Centers from enterprise premises. 19 Status of this Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF), its areas, and its working groups. Note that 26 other groups may also distribute working documents as Internet- 27 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other documents 31 at any time. It is inappropriate to use Internet-Drafts as 32 reference material or to cite them other than as "work in progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt 37 The list of Internet-Draft Shadow Directories can be accessed at 38 http://www.ietf.org/shadow.html 40 This Internet-Draft will expire on September 18, 2020. 42 xxx, et al. Expires September 18, 2020 [Page 43 1] 44 Copyright Notice 46 Copyright (c) 2020 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with 54 respect to this document. Code Components extracted from this 55 document must include Simplified BSD License text as described in 56 Section 4.e of the Trust Legal Provisions and are provided without 57 warranty as described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction...................................................3 62 2. Conventions used in this document..............................3 63 3. Gap Analysis for Accessing Cloud Resources.....................4 64 4. Gap Analysis of Overlay Edge Node's WAN Port Management........4 65 5. Aggregating VPN paths and Internet paths.......................6 66 5.1. Control Plane for Overlay over Heterogeneous Networks.....7 67 5.2. Using BGP UPDATE Messages.................................8 68 5.2.1. Lacking SD-WAN Segments Identifier...................8 69 5.2.2. Missing attributes in Tunnel-Encap...................8 70 5.3. SECURE-L3VPN/EVPN.........................................9 71 5.4. Preventing attacks from Internet-facing ports............11 72 6. C-PEs not directly connected to VPN PEs.......................11 73 6.1. Floating PEs to connect to Remote CPEs...................14 74 6.2. NAT Traversal............................................14 75 6.3. Complexity of using BGP between PEs and remote CPEs via 76 Internet......................................................14 77 6.4. Designated Forwarder to the remote edges.................15 78 6.5. Traffic Path Management..................................16 79 7. Manageability Considerations..................................16 80 8. Security Considerations.......................................16 81 9. IANA Considerations...........................................17 82 10. References...................................................17 83 10.1. Normative References....................................17 84 10.2. Informative References..................................17 85 11. Acknowledgments..............................................18 87 1. Introduction 89 [Net2Cloud-Problem] describes the problems enterprises face today 90 when interconnecting their branch offices with dynamic workloads 91 hosted in third party data centers (a.k.a. Cloud DCs). In 92 particular, this document analyzes the routing protocols to identify 93 whether there are any gaps that may impede such interconnection 94 which may for example justify additional specification effort to 95 define proper protocol extensions. 97 For the sake of readability, an edge, an endpoint, C-PE, or CPE are 98 used interchangeably throughout this document. More precisely: 100 . Edge: may include multiple devices (virtual or physical); 101 . endpoint: refers to a WAN port of device located in the edge; 102 . C-PE: provider-owned edge, e.g. for SECURE-EVPN's PE-based 103 BGP/MPLS VPN, where PE is the edge node; 104 . CPE: device located in enterprise premises. 106 2. Conventions used in this document 108 Cloud DC: Third party Data Centers that usually host applications 109 and workload owned by different organizations or 110 tenants. 112 Controller: Used interchangeably with Overlay controller to manage 113 overlay path creation/deletion and monitor the path 114 conditions between sites. 116 CPE-Based VPN: Virtual Private Network designed and deployed from 117 CPEs. This is to differentiate from most commonly used 118 PE-based VPNs a la RFC 4364. 120 OnPrem: On Premises data centers and branch offices 122 SDWAN: Software Defined Wide Area Network, "SDWAN" refers to 123 the solutions of pooling WAN bandwidth from multiple 124 underlay networks to get better WAN bandwidth 125 management, visibility & control. When the underlay is a 126 private network, traffic may be forwarded without any 127 additional encryption; when the underlay networks are 128 public, such as the Internet, some traffic needs to be 129 encrypted when passing through (depending on user- 130 provided policies). 132 3. Gap Analysis for Accessing Cloud Resources 134 Many problems described in the [Net2Cloud-Problem] are not in the 135 scope of IETF, let alone IETF Routing area. This document primarily 136 focuses on the gap analysis for protocols in IETF Routing area. 138 4. Gap Analysis of Overlay Edge Node's WAN Port Management 140 Very often the Hybrid Cloud DCs are interconnected by overlay 141 networks that arch over many different types of networks, such as 142 VPN, public Internet, wireless and wired infrastructures, etc. 143 Sometimes the enterprises' VPN providers do not have direct access 144 to the Cloud DCs that host some specific applications or workloads 145 operated by the enterprise. 147 Under those circumstances, the overlay network's edge nodes can have 148 WAN ports facing networks provided by different ISPs, some of these 149 networks may not be trustable, some others can be trusted like VPNs 150 (to some extent), etc. 152 If all WAN ports of an edge node are facing an untrusted network, 153 then all sensitive data to/from this edge node have to be encrypted, 154 usually by means of IPsec tunnels which can be terminated at the WAN 155 port address, at the edge node's loopback address if the loopback 156 address is routable in the wide area network, or even at the ingress 157 ports of the edge node. 159 If an edge node has some WAN ports facing trusted networks and 160 others facing untrusted networks, sensitive data can be forwarded 161 through ports facing the trusted networks natively (i.e., without 162 encryption) and forwarded through ports facing untrusted networks 163 assuming encryption. To achieve this flexibility of sending traffic 164 either encrypted or not encrypted depending on egress WAN ports, it 165 is necessary to have the IPsec tunnels terminated at the WAN ports 166 facing the untrusted networks. 168 In order to establish peer-wise secure encrypted communications 169 among those WAN ports of two edge nodes, it is necessary for the 170 edge nodes (peers) to be informed of the WAN port properties. 172 Some of those overlay networks (such as some deployed SDWAN 173 networks) use the modified NHRP protocol [RFC2332] to register WAN 174 ports of the edge nodes with their Controller (or NHRP server), 175 which then maps a private VPN address to a public IP address of the 176 destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to 177 establish tunnels between WAN ports of SDWAN edge nodes. 179 NHRP was originally intended for ATM address resolution, and as a 180 result, it misses many attributes that are necessary for dynamic 181 endpoint C-PE registration to the controller, such as: 183 - Interworking with the MPLS VPN control plane. An overlay edge can 184 have some ports facing the MPLS VPN network over which packets can 185 be forwarded without any encryption and some ports facing the 186 public Internet over which sensitive traffic needs to be 187 encrypted. 188 - Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of edge 189 nodes. When a network has more than 100 nodes, these protocols do 190 not scale well. 191 - NHRP does not have the IPsec attributes, which are needed for 192 peers to build Security Associations over the public Internet. 193 - NHRP messages do not have any field to encode the C-PE supported 194 encapsulation types, such as IPsec-GRE or IPsec-VxLAN. 195 - NHRP messages do not have any field to encode C-PE Location 196 identifiers, such as Site Identifier, System ID, and/or Port ID. 197 - NHRP messages do not have any field to describe the gateway(s) to 198 which the C-PE is attached. When a C-PE is instantiated in a Cloud 199 DC, it is desirable for the C-PE's owner to be informed about how 200 and where the C-PE is attached. 201 - NHRP messages do not have any field to describe C-PE's NAT 202 properties if the C-PE is using private IPv4 addresses, such as 203 the NAT type, Private address, Public address, Private port, 204 Public port, etc. 206 [BGP-SDWAN-PORT] describes how to use BGP to distribute SDWAN edge 207 properties to peers. SDWAN is an overlay network with specific 208 properties, such as application-based forwarding, augmented 209 transport, and user specified policies. There is a need to extend 210 the protocol to register WAN port properties of an edge node to the 211 overlay controller, which then propagates the information to other 212 overlay edge nodes that are authenticated and authorized to 213 communicate with them. 215 5. Aggregating VPN paths and Internet paths 217 Most likely, enterprises (especially the largest ones) already have 218 their C-PEs interconnected by VPN service providers, based upon VPN 219 techniques like EVPN, L2VPN, or L3VPN, and which can be lead to PE- 220 based or CPE-based VPN service designs. The commonly used PE-based 221 BGP/MPLS VPNs have C-PEs directly attached to PEs, the communication 222 between C-PEs and PEs is considered as secure as they are connected 223 by direct physical links albeit there could be routes leaking or 224 unauthorized routes being injected. MP-BGP can be used to learn & 225 distribute routes among C-PEs, but sometimes routes among C-PEs are 226 statically configured on the C-PEs. 228 For enterprises already interconnected by VPNs, if there are short 229 term high traffic volume that can't justify increasing the VPNs 230 capacity, it is desirable for the CPE to aggregate the bandwidth 231 that pertains to VPN paths and Internet paths by adding ports that 232 connect the CPE to the public Internet. Under this scenario, which 233 is referred to as the Overlay scenario throughout this document, it 234 is necessary for the C-PEs to manage and communicate with the 235 controller on how traffic is distributed among multiple 236 heterogeneous underlay networks, and also to manage secure tunnels 237 over untrusted networks. 239 When using NHRP for WAN port registration purposes, C-PEs need to 240 run two separate control planes: EVPN&BGP for CPE-based VPNs, and 241 NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate 242 control planes not only add complexity to C-PEs, but also increase 243 operational costs. 245 +---+ 246 +--------------|RR |----------+ 247 / Untrusted +-+-+ \ 248 / \ 249 / \ 250 +----+ +---------+ packets encrypted over +------+ +----+ 251 | TN3|--| A1-----+ Untrusted +------ B1 |--| TN1| 252 +----+ | C-PE A2-\ | C-PE | +----+ 253 +----+ | A A3--+--+ +---+---B2 B | +----+ 254 | TN2|--| | |PE+--------------+PE |---B3 |--| TN3| 255 +----+ +---------+ +--+ trusted +---+ +------+ +----+ 256 | WAN | 257 +----+ +---------+ +--+ packets +---+ +------+ +----+ 258 | TN1|--| C1--|PE| go natively |PE |-- D1 |--| TN1| 259 +----+ | C-PE C2--+--+ without encry+---+ | C-PE | +----+ 260 | C | +--------------+ | D | 261 | | | | 262 +----+ | C3--| without encrypt over | | +----+ 263 | TN2|--| C4--+---- Untrusted --+------D2 |--| TN2| 264 +----+ +---------+ +------+ +----+ 265 Figure 1: CPEs interconnected by VPN paths and Internet Paths 267 5.1. Control Plane for Overlay over Heterogeneous Networks 269 As described in [BGP-SDWAN-Usage], the Control Plane for Overlay 270 network over heterogeneous networks has three distinct properties: 272 - WAN Port Property registration to the Overlay Controller. 273 o To inform the Overlay controller and authorized peers of 274 the WAN port properties of the Edge nodes. When the WAN 275 ports are assigned private IPv4 addresses, this step can 276 register the type of NAT that translates these addresses 277 into public ones. 279 - Controller-facilitated IPsec SA management and NAT information 280 distribution 281 o The Overlay controller facilitates and manages the IPsec 282 configuration and peer authentication for all IPsec 283 tunnels terminated at the edge nodes. 285 - Establishing and Managing the topology and reachability for 286 services attached to the client ports of overlay edge nodes. 288 o This is for the overlay layer's route distribution, so 289 that a C-PE can populate its overlay routing table with 290 entries that identify the next hop for reaching a specific 291 route/service attached to remote nodes. [SECURE-EVPN] 292 describes EVPN and other options. 294 5.2. Using BGP UPDATE Messages 296 5.2.1. Lacking SD-WAN Segments Identifier 298 There could be multiple SD-WAN networks with their edge nodes 299 exchanging BGP UPDATE messages with the BGP RR. The multiple SD-WAN 300 networks could have common underlay networks. Therefore, it is very 301 important to have an identifier to differentiate BGP UPDATE messages 302 belonging to different SD-WAN networks (or sometimes called SD-WAN 303 Segmentations). Today's BGP doesn't have this feature yet, unless 304 there are multiple BGP instances and their corresponding RRs. 306 5.2.2. Missing attributes in Tunnel-Encap 308 [Tunnel-Encap] describes the BGP UPDATE Tunnel Path Attribute that 309 advertises endpoints' tunnel encapsulation capabilities for the 310 respective attached client routes encoded in the MP-NLRI Path 311 Attribute. The receivers of the BGP UPDATE can use any of the 312 supported encapsulations encoded in the Tunnel Path Attribute for 313 the routes encoded in the MP-NLRI Path Attribute. 315 Here are some of the issues raised by the use of [Tunnel-Encap] to 316 distribute Edge WAN port properties: 318 - [Tunnel-Encap] doesn't have the encoding to describe the NAT 319 information for WAN ports that are assigned private IPv4 addresses 320 yet. The NAT information needs to be propagated to the trusted 321 peers such as the virtual C-PEs instantiated in public Cloud DCs 322 via Controllers. 323 - The mechanism defined in [Tunnel-Encap] does not facilitate the 324 exchange of IPsec SA-specific parameters independently from 325 advertising the attached clients' routes, even after adding a new 326 IPsec tunnel type. 327 [Tunnel-Encap] requires all tunnels updates to be associated with 328 routes. There can be many client routes associated with an IPsec 329 tunnel established between two C-PEs' WAN ports; the corresponding 330 destination prefixes (as announced by the aforementioned routes) 331 may also be reached through the VPN underlay without any 332 encryption. 334 The establishment of an IPsec tunnel can fail, e.g., because the 335 two endpoints support different encryption algorithms. Multiple 336 negotiation messages that carry the IPsec SA parameters between 337 two end-points may be exchanged. This is why it is cleaner to 338 separate the establishment of an IPsec SA association between two 339 end-points from the policies enforced to map routes to a specific 340 IPSec SA. 342 If C-PEs need to establish a WAN Port-based IPsec SA, the 343 information encoded in the Tunnel Path Attribute should only apply 344 to the WAN ports and should be independent from the clients' 345 routes. 347 In addition, the Overlay IPsec SA Tunnel is very likely to be 348 established before clients' routes are attached. 350 - When an overlay network spans across large geographic regions 351 (such as countries or continents), one C-PE in one region may not 352 even be aware of remote CPEs in other regions that it needs to 353 communicate. Therefore, the distribution of the Overlay Edge WAN 354 ports information need to be restricted to the authorized peers. 356 5.3. SECURE-L3VPN/EVPN 358 [SECURE-L3VPN] describes a method to enrich BGP/MPLS VPN [RFC4364] 359 capabilities to allow some PEs to connect to other PEs via public 360 networks. [SECURE-L3VPN] introduces the concept of Red Interface & 361 Black Interface used by PEs, where the RED interfaces are used to 362 forward traffic into the VPN, and the Black Interfaces are used 363 between WAN ports through which only IPsec-formatted packets are 364 forwarded to the Internet or to any other backbone network, thereby 365 eliminating the need for MPLS transport in the backbone. 367 [SECURE-L3VPN] assumes PEs use MPLS over IPsec when sending traffic 368 through the Black Interfaces. 370 [SECURE-EVPN] describes a solution where point-to-multipoint BGP 371 signaling is used in the control plane for the Scenario #1 described 372 in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to 373 facilitate the key and policy exchange among PE devices to create 374 private pair-wise IPsec Security Associations without IKEv2 point- 375 to-point signaling or any other direct peer-to-peer session 376 establishment messages. 378 Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both 379 miss the aspects of aggregating VPN and Internet underlays. In 380 summary: 382 - Both documents assume that an IPsec tunnel is associated with 383 client traffic. Regardless of which WAN ports the traffic egress 384 from the edge, the client traffic associated with IPsec is always 385 encrypted. Within the context of an overlay architecture that 386 relies upon minimizing resource used for encryption, traffic sent 387 from an edge node can be encrypted once and forwarded through a 388 WAN port towards an untrusted network, but can also remain 389 unencrypted and be forwarded at different times through a WAN port 390 to the BGP/MPLS VPN. 392 - The [SECURE-L3VPN] assumes that a CPE "registers" with the RR. 393 However, it does not say how. It assumes that the remote CPEs are 394 pre-configured with the IPsec SA manually. For overlay networks to 395 connect Hybrid Cloud DCs, Zero Touch Provisioning is expected. 396 Manual configuration is not an option. 398 - The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an 399 IPsec tunnel. For management channel, TLS/DTLS is more economical 400 than IPsec. The following assumption made by [SECURE-L3VPN] can be 401 difficult to meet in the environment where zero touch provisioning 402 is expected: 403 A CPE must also be provisioned with whatever additional 404 information is needed in order to set up an IPsec SA with 405 each of the red RRs 407 - IPsec requires periodic refreshment of the keys. The draft does 408 not provide any information about how to synchronize the 409 refreshment among multiple nodes. 410 - IPsec usually sends configuration parameters to two endpoints only 411 and lets these endpoints negotiate the key. The [SECURE-L3VPN] 412 assumes that the RR is responsible for creating/managing the key 413 for all endpoints. When one endpoint is compromised, all other 414 connections may be impacted. 416 5.4. Preventing attacks from Internet-facing ports 418 When C-PEs have Internet-facing ports, additional security risks are 419 raised. 421 To mitigate security risks, in addition to requiring Anti-DDoS 422 features on C-PEs, it is necessary for C-PEs to support means to 423 determine whether traffic sent by remote peers is legitimate to 424 prevent spoofing attacks, in particular. 426 6. C-PEs not directly connected to VPN PEs 428 Because of the ephemeral property of the selected Cloud DCs for 429 specific workloads/Apps, an enterprise or its network service 430 provider may not have direct physical connections to the Cloud DCs 431 that are optimal for hosting the enterprise's specific 432 workloads/Apps. Under those circumstances, an overlay network design 433 can be an option to interconnect the enterprise's on-premises data 434 centers & branch offices to its desired Cloud DCs. 436 However, overlay paths established over the public Internet can have 437 unpredictable performance, especially over long distances and across 438 operators' domains. Therefore, it is highly desirable to minimize 439 the distance or the number of segments that traffic had to be 440 forwarded over the public Internet. 442 The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud] 443 also describes a use case of network operators using Overlay paths 444 over a LTE network or the public Internet for the last mile access 445 where the VPN service providers cannot always provide the required 446 physical infrastructure. 448 In these scenarios, some overlay edge nodes may not be directly 449 attached to the PEs that participate to the delivery and the 450 operation of the enterprise's VPN. 452 When using an overlay network to connect the enterprise's sites to 453 the workloads hosted in Cloud DCs, the corresponding C-PEs have to 454 be upgraded to connect to the said overlay network. If the 455 workloads hosted in Cloud DCs need to be connected to many sites, 456 the upgrade process can be very expensive. 458 [Net2Cloud-Problem] describes a hybrid network approach that extends 459 the existing MPLS-based VPNs to the Cloud DC Workloads over the 460 access paths that are not under the VPN provider's control. To make 461 it work properly, a small number of the PEs of the BGP/MPLS VPN can 462 be designated to connect to the remote workloads via secure IPsec 463 tunnels. Those designated PEs are shown as fPE (floating PE or 464 smart PE) in Figure 3. Once the secure IPsec tunnels are 465 established, the workloads hosted in Cloud DCs can be reached by the 466 enterprise's VPN without upgrading all of the enterprise's CPEs. The 467 only CPE that needs to connect to the overlay network would be a 468 virtualized CPE instantiated within the cloud DC. 470 +--------+ +--------+ 471 | Host-a +--+ +----| Host-b | 472 | | | (') | | 473 +--------+ | +-----------+ ( ) +--------+ 474 | +-+--+ ++-+ ++-+ +--+-+ (_) 475 | | CPE|--|PE| |PE+--+ CPE| | 476 +--| | | | | | | |---+ 477 +-+--+ ++-+ ++-+ +----+ 478 / | | 479 / | MPLS +-+---+ +--+-++--------+ 480 +------+-+ | Network |fPE-1| |CPE || Host | 481 | Host | | | |- --| || d | 482 | c | +-----+ +-+---+ +--+-++--------+ 483 +--------+ |fPE-2|-----+ 484 +---+-+ (|) 485 (|) (|) Overlay 486 (|) (|) over any access 487 +=\======+=========+ 488 // \ | Cloud DC \\ 489 // \ ++-----+ \\ 490 +Remote| 491 | CPE | 492 +-+----+ 493 ----+-------+-------+----- 494 | | 495 +---+----+ +---+----+ 496 | Remote | | Remote | 497 | App-1 | | App-2 | 498 +--------+ +--------+ 500 Figure 3: VPN Extension to Cloud DC 502 In Figure 3, the optimal Cloud DC to host the workloads (as a 503 function of the proximity, capacity, pricing, or any other criteria 504 chosen by the enterprises) does not have a direct connection to the 505 PEs of the NGP/MPLS VPN that interconnects the enterprise's sites. 507 6.1. Floating PEs to connect to Remote CPEs 509 To extend BGP/MPLS VPNs to remote CPEs, it is necessary to establish 510 secure tunnels (such as IPsec tunnels) between the Floating PEs and 511 the remote CPEs. 513 Even though a set of PEs can be manually selected to act as the 514 floating PEs for a specific cloud data center, there are no standard 515 protocols for those PEs to interact with the remote CPEs (most 516 likely virtualized) instantiated in the third party cloud data 517 centers (e.g., to exchange performance or route information). 519 When there is more than one fPE available for use (as there should 520 be for resiliency purposes or because of the need to support 521 multiple cloud DCs geographically scattered), it is not 522 straightforward to designate an egress fPE to remote CPEs based on 523 applications. There is too much applications' traffic traversing 524 PEs, and it is not feasible for PEs to recognize applications from 525 the payload of packets. 527 6.2. NAT Traversal 529 Cloud DCs that only assign private IPv4 addresses to the 530 instantiated workloads assume that traffic to/from the workload 531 usually needs to traverse NATs. 532 An overlay edge node can solicit a STUN (Session Traversal of UDP 533 Through Network Address Translation, [RFC3489]) Server to get the 534 information about the NAT property, the public IP addresses and port 535 numbers so that such information can be communicated to the relevant 536 peers. 538 6.3. Complexity of using BGP between PEs and remote CPEs via Internet 540 Even though an EBGP (external BGP) Multi-Hop design can be used to 541 connect peers that are not directly connected to each other, there 542 are still some issues about extending BGP from MPLS VPN PEs to 543 remote CPEs via any access path (e.g., Internet). 545 The path between the remote CPEs and VPN PEs that maintain VPN 546 routes may very well traverse untrusted nodes. 548 EBGP Multi-hop design requires configuration on both peers, either 549 manually or via NETCONF from a controller. To use EBGP between a PE 550 and remote CPEs, the PE has to be manually configured with the 551 "next-hop" set to the IP address of the CPEs. When remote CPEs, 552 especially remote virtualized CPEs are dynamically instantiated or 553 removed, the configuration of Multi-Hop EBGP on the PE has to be 554 changed accordingly. 556 Egress peering engineering (EPE) is not sufficient. Running BGP on 557 virtualized CPEs in Cloud DCs requires GRE tunnels to be 558 established first, which requires the remote CPEs to support 559 address and key management capabilities. RFC 7024 (Virtual Hub & 560 Spoke) and Hierarchical VPN do not support the required 561 properties. 563 Also, there is a need for a mechanism to automatically trigger 564 configuration changes on PEs when remote CPEs' are instantiated or 565 moved (leading to an IP address change) or deleted. 567 EBGP Multi-hop design does not include a security mechanism by 568 default. The PE and remote CPEs need secure communication channels 569 when connecting via the public Internet. 571 Remote CPEs, if instantiated in Cloud DCs might have to traverse 572 NATs to reach PEs. It is not clear how BGP can be used between 573 devices located beyond the NAT and the devices located behind the 574 NAT. It is not clear how to configure the Next Hop on the PEs to 575 reach private IPv4 addresses. 577 6.4. Designated Forwarder to the remote edges 579 Among the multiple floating PEs that are reachable from a remote 580 CPE, multicast traffic sent by the remote CPE towards the MPLS VPN 581 can be forwarded back to the remote CPE due to the PE receiving the 582 multicast packets forwarding the multicast/broadcast frame to other 583 PEs that in turn send to all attached CPEs. This process may cause 584 traffic loops. 586 This problem can be solved by selecting one floating PE as the CPE's 587 Designated Forwarder, similar to TRILL's Appointed Forwarders 588 [RFC6325]. 590 BGP/MPLS VPNs do not have features like TRILL's Appointed 591 Forwarders. 593 6.5. Traffic Path Management 595 When there are multiple floating PEs that have established IPsec 596 tunnels with a remote CPE, the latter can forward outbound traffic 597 to the Designated Forwarder PE, which in turn forwards traffic to 598 egress PEs and then to the final destinations. However, it is not 599 straightforward for the egress PE to send back the return traffic to 600 the Designated Forwarder PE. 602 As Figure 3: 604 - fPE-1 is DF for communication between App-1 <-> Host-a due to 605 latency, pricing or other criteria. 606 - fPE-2 is DF for communication between App-1 <-> Host-b. 608 7. Manageability Considerations 610 Zero touch provisioning of overlay networks to interconnect Hybrid 611 Clouds is highly desired. It is necessary for a newly powered up 612 edge node to establish a secure connection (by means of TLS, DTLS, 613 etc.) with its controller. 615 8. Security Considerations 617 Cloud Services are built upon shared infrastructures, therefore 618 not secure by nature. 620 Secure user identity management, authentication, and access 621 control mechanisms are important. Developing appropriate security 622 measurements can enhance the confidence needed by enterprises to 623 fully take advantage of Cloud Services. 625 9. IANA Considerations 627 This document requires no IANA actions. RFC Editor: Please remove 628 this section before publication. 630 10. References 632 10.1. Normative References 634 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 635 Requirement Levels", BCP 14, RFC 2119, March 1997. 637 10.2. Informative References 639 [RFC8192] S. Hares, et al, "Interface to Network Security Functions 640 (I2NSF) Problem Statement and Use Cases", July 2017 642 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 643 Address Family Identifier (SAFI) and the BGP Tunnel 644 Encapsulation Attribute", April 2009. 646 [BGP-SDWAN-PORT]L. Dunbar, et al, "Subsequent Address Family 647 Indicator for SDWAN Ports", draft-dunbar-idr-sdwan-port- 648 safi-00, Work-in-progress, March 2019. 650 [BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for 651 SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework- 652 00, work-in-progress, Feb 2019. 654 [Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation 655 Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018. 657 [SECURE-EVPN A. Sajassi, et al, draft-sajassi-bess-secure-evpn-01, 658 work in progress, March 2019. 660 [SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public 661 Infrastructure", draft-rosen-bess-secure-l3vpn-00, work- 662 in-progress, July 2018 664 [DMVPN] Dynamic Multi-point VPN: 665 https://www.cisco.com/c/en/us/products/security/dynamic- 666 multipoint-vpn-dmvpn/index.html 668 [DSVPN] Dynamic Smart VPN: 669 http://forum.huawei.com/enterprise/en/thread-390771-1- 670 1.html 672 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 673 storage, distribution and enforcement of policies for 674 network security", Nov 2007. 676 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect 677 Underlay to Cloud Overlay Problem Statement", draft-dm- 678 net2cloud-problem-statement-02, June 2018 680 11. Acknowledgments 682 Acknowledgements to John Drake for his review and contributions. 683 Many thanks to John Scudder for stimulating the clarification 684 discussion on the Tunnel-Encap draft so that our gap analysis can be 685 more accurate. 687 This document was prepared using 2-Word-v2.0.template.dot. 689 Authors' Addresses 691 Linda Dunbar 692 Futurewei 693 Email: ldunbar@futurewei.com 695 Andrew G. Malis 696 Independent 697 Email: agmalis@gmail.com 699 Christian Jacquenet 700 Orange 701 Rennes, 35000 702 France 703 Email: Christian.jacquenet@orange.com