idnits 2.17.1 draft-ietf-rtgwg-net2cloud-gap-analysis-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 519 has weird spacing: '...t carry the I...' -- The document date (May 1, 2020) is 1455 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'MEF-Cloud' is mentioned on line 145, but not defined == Missing Reference: 'RFC3489' is mentioned on line 262, but not defined ** Obsolete undefined reference: RFC 3489 (Obsoleted by RFC 5389) == Missing Reference: 'RFC6325' is mentioned on line 317, but not defined == Missing Reference: 'RFC2332' is mentioned on line 357, but not defined == Missing Reference: 'BGP-SDWAN-PORT' is mentioned on line 391, but not defined == Missing Reference: 'SECURE-EVPN' is mentioned on line 561, but not defined == Missing Reference: 'Tunnel-Encap' is mentioned on line 510, but not defined == Missing Reference: 'RFC4364' is mentioned on line 541, but not defined == Unused Reference: 'RFC2119' is defined on line 677, but no explicit reference was found in the text == Unused Reference: 'RFC8192' is defined on line 682, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 685, but no explicit reference was found in the text == Unused Reference: 'ITU-T-X1036' is defined on line 715, but no explicit reference was found in the text -- No information found for draft-dunbar-idr-sdwan-framework - is the name correct? == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 == Outdated reference: A later version (-07) exists of draft-dm-net2cloud-problem-statement-02 Summary: 1 error (**), 0 flaws (~~), 16 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft Futurewei 3 Intended status: Informational A. Malis 4 Expires: November 1, 2020 Independent 6 C. Jacquenet 7 Orange 8 May 1, 2020 10 Networks Connecting to Hybrid Cloud DCs: Gap Analysis 11 draft-ietf-rtgwg-net2cloud-gap-analysis-06 13 Abstract 15 This document analyzes the technical gaps that may affect the 16 dynamic connection to workloads and applications hosted in hybrid 17 Cloud Data Centers from enterprise premises. 19 Status of this Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF), its areas, and its working groups. Note that 26 other groups may also distribute working documents as Internet- 27 Drafts. 29 Internet-Drafts are draft documents valid for a maximum of six 30 months and may be updated, replaced, or obsoleted by other documents 31 at any time. It is inappropriate to use Internet-Drafts as 32 reference material or to cite them other than as "work in progress." 34 The list of current Internet-Drafts can be accessed at 35 http://www.ietf.org/ietf/1id-abstracts.txt 37 The list of Internet-Draft Shadow Directories can be accessed at 38 http://www.ietf.org/shadow.html 40 This Internet-Draft will expire on November 1, 2020. 42 Copyright Notice 44 Copyright (c) 2020 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with 52 respect to this document. Code Components extracted from this 53 document must include Simplified BSD License text as described in 54 Section 4.e of the Trust Legal Provisions and are provided without 55 warranty as described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction...................................................3 60 2. Conventions used in this document..............................3 61 3. Gap Analysis for Accessing Cloud Resources.....................4 62 3.1. Multiple PEs connecting to virtual CPEs in Cloud DCs......7 63 3.2. Access Control for workloads in the Cloud DCs.............7 64 3.3. NAT Traversal.............................................8 65 3.4. BGP between PEs and remote CPEs via Internet..............8 66 3.5. Multicast traffic from/to the remote edges................9 67 4. Gap Analysis of Overlay Edge Node's WAN Port Management........9 68 5. Aggregating VPN paths and Internet paths......................11 69 5.1. Control Plane for Overlay over Heterogeneous Networks....12 70 5.2. Using BGP UPDATE Messages................................13 71 5.2.1. Lacking SD-WAN Segments Identifier..................13 72 5.2.2. Missing attributes in Tunnel-Encap..................13 73 5.3. SECURE-L3VPN/EVPN........................................15 74 5.4. Preventing attacks from Internet-facing ports............16 75 6. Gap Summary...................................................16 76 7. Manageability Considerations..................................17 77 8. Security Considerations.......................................17 78 9. IANA Considerations...........................................18 79 10. References...................................................18 80 10.1. Normative References....................................18 81 10.2. Informative References..................................18 82 11. Acknowledgments..............................................19 84 1. Introduction 86 [Net2Cloud-Problem] describes the problems enterprises face today 87 when interconnecting their branch offices with dynamic workloads 88 hosted in third party data centers (a.k.a. Cloud DCs). In 89 particular, this document analyzes the routing protocols to identify 90 whether there are any gaps that may impede such interconnection 91 which may for example justify additional specification effort to 92 define proper protocol extensions. 94 For the sake of readability, an edge, an endpoint, C-PE, or CPE are 95 used interchangeably throughout this document. More precisely: 97 . Edge: may include multiple devices (virtual or physical); 98 . endpoint: refers to a WAN port of device located in the edge; 99 . C-PE: provider-owned edge, e.g. for SECURE-EVPN's PE-based 100 BGP/MPLS VPN, where PE is the edge node; 101 . CPE: device located in enterprise premises. 103 2. Conventions used in this document 105 Cloud DC: Third party Data Centers that usually host applications 106 and workload owned by different organizations or 107 tenants. 109 Controller: Used interchangeably with Overlay controller to manage 110 overlay path creation/deletion and monitor the path 111 conditions between sites. 113 CPE-Based VPN: Virtual Private Network designed and deployed from 114 CPEs. This is to differentiate from most commonly used 115 PE-based VPNs a la RFC 4364. 117 OnPrem: On Premises data centers and branch offices 119 SDWAN: Software Defined Wide Area Network, "SDWAN" refers to 120 the solutions of pooling WAN bandwidth from multiple 121 underlay networks to get better WAN bandwidth 122 management, visibility & control. When the underlay is a 123 private network, traffic may be forwarded without any 124 additional encryption; when the underlay networks are 125 public, such as the Internet, some traffic needs to be 126 encrypted when passing through (depending on user- 127 provided policies). 129 3. Gap Analysis for Accessing Cloud Resources 131 Because of the ephemeral property of the selected Cloud DCs for 132 specific workloads/Apps, an enterprise or its network service 133 provider may not have direct physical connections to the Cloud DCs 134 that are optimal for hosting the enterprise's specific 135 workloads/Apps. Under those circumstances, an overlay network design 136 can be an option to interconnect the enterprise's on-premises data 137 centers & branch offices to its desired Cloud DCs. 139 However, overlay paths established over the public Internet can have 140 unpredictable performance, especially over long distances and across 141 operators' domains. Therefore, it is highly desirable to minimize 142 the distance or the number of segments that traffic had to be 143 forwarded over the public Internet. 145 The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud] 146 also describes a use case of network operators using Overlay paths 147 over an LTE network or the public Internet for the last mile access 148 where the VPN service providers cannot always provide the required 149 physical infrastructure. 151 In these scenarios, some overlay edge nodes may not be directly 152 attached to the PEs that participate to the delivery and the 153 operation of the enterprise's VPN. 155 When using an overlay network to connect the enterprise's sites to 156 the workloads hosted in Cloud DCs, the existing C-PEs at 157 enterprise's sites have to be upgraded to connect to the said 158 overlay network. If the workloads hosted in Cloud DCs need to be 159 connected to many sites, the upgrade process can be very expensive. 161 [Net2Cloud-Problem] describes a hybrid network approach that extends 162 the existing MPLS-based VPNs to the Cloud DC Workloads over the 163 access paths that are not under the VPN provider's control. To make 164 it work properly, a small number of the PEs of the BGP/MPLS VPN can 165 be designated to connect to the remote workloads via secure IPsec 166 tunnels. Those designated PEs are shown as fPE (floating PE or 167 smart PE) in Figure 3. Once the secure IPsec tunnels are 168 established, the workloads hosted in Cloud DCs can be reached by the 169 enterprise's VPN without upgrading all of the enterprise's CPEs. The 170 only CPE that needs to connect to the overlay network would be a 171 virtualized CPE instantiated within the cloud DC. 173 +--------+ +--------+ 174 | Host-a +--+ +----| Host-b | 175 | | | (') | | 176 +--------+ | +-----------+ ( ) +--------+ 177 | +-+--+ ++-+ ++-+ +--+-+ (_) 178 | | CPE|--|PE| |PE+--+ CPE| | 179 +--| | | | | | | |---+ 180 +-+--+ ++-+ ++-+ +----+ 181 / | | 182 / | MPLS +-+---+ +--+-++--------+ 183 +------+-+ | Network |fPE-1| |CPE || Host | 184 | Host | | | |- --| || d | 185 | c | +-----+ +-+---+ +--+-++--------+ 186 +--------+ |fPE-2|-----+ 187 +---+-+ (|) 188 (|) (|) Overlay 189 (|) (|) over any access 190 +=\======+=========+ 191 // \ | Cloud DC \\ 192 // \ ++-----+ \\ 193 + | 194 | vCPE | 195 +-+----+ 196 ----+-------+-------+----- 197 | | 198 +---+----+ +---+----+ 199 | Remote | | Remote | 200 | App-1 | | App-2 | 201 +--------+ +--------+ 203 Figure 1: VPN Extension to Cloud DC 205 In Figure 1, the optimal Cloud DC to host the workloads (as a 206 function of the proximity, capacity, pricing, or any other criteria 207 chosen by the enterprises) does not have a direct connection to the 208 PEs of the NGP/MPLS VPN that interconnects the enterprise's sites. 210 3.1. Multiple PEs connecting to virtual CPEs in Cloud DCs 212 To extend BGP/MPLS VPNs to virtual CPEs in Cloud DCs, it is 213 necessary to establish secure tunnels (such as IPsec tunnels) 214 between the PEs and the vCPEs. 216 Even though a set of PEs can be manually selected for a specific 217 cloud data center, there are no standard protocols for those PEs to 218 interact with the vCPEs instantiated in the third party cloud data 219 centers over unsecure networks. The interaction includes exchanging 220 performance, route information, etc.. 222 When there is more than one PE available for use (as there should be 223 for resiliency purposes or because of the need to support multiple 224 cloud DCs geographically scattered), it is not straightforward to 225 designate an egress PE to remote vCPEs based on applications. It 226 might not be possible for PEs to recognize all applications because 227 too much traffic traversing the PEs. 229 When there are multiple floating PEs that have established IPsec 230 tunnels with a remote CPE, the remove CPE can forward outbound 231 traffic to the optimal PE, which in turn forwards traffic to egress 232 PEs to reach the final destinations. However, it is not 233 straightforward for the ingress PE to select which egress PEs to 234 send traffic. For example, in Figure 1: 236 - fPE-1 is the optimal PE for communication between App-1 <-> 237 Host-a due to latency, pricing or other criteria. 239 - fPE-2 is the optimal PE for communication between App-1 <-> 240 Host-b. 242 3.2. Access Control for workloads in the Cloud DCs 244 There is widespread diffusion of access policy for Cloud Resource, 245 some of which is not easy for verification and validation. Because 246 there are multiple parties involved in accessing Cloud Resources, 247 policy enforcement points are not easily visible for policy 248 refinement, monitoring, and testing. 250 The current state of the art for specifying access policies for 251 Cloud Resources could be improved by having automated and reliable 252 tools to map the user-friendly (natural language) rules into machine 253 readable policies and to provide interfaces for enterprises to self- 254 manage policy enforcement points for their own workloads. 256 3.3. NAT Traversal 258 Cloud DCs that only assign private IPv4 addresses to the 259 instantiated workloads assume that traffic to/from the workload 260 usually needs to traverse NATs. 261 An overlay edge node can solicit a STUN (Session Traversal of UDP 262 Through Network Address Translation, [RFC3489]) Server to get the 263 information about the NAT property, the public IP addresses and port 264 numbers so that such information can be communicated to the relevant 265 peers. 267 3.4. BGP between PEs and remote CPEs via Internet 269 Even though an EBGP (external BGP) Multi-Hop design can be used to 270 connect peers that are not directly connected to each other, there 271 are still some issues about extending BGP from MPLS VPN PEs to 272 remote CPEs via any access path (e.g., Internet). 274 The path between the remote CPEs and VPN PEs that maintain VPN 275 routes can traverse untrusted segments. 277 EBGP Multi-hop design requires configuration on both peers, either 278 manually or via NETCONF from a controller. To use EBGP between a PE 279 and remote CPEs, the PE has to be manually configured with the 280 "next-hop" set to the IP address of the CPEs. When remote CPEs, 281 especially remote virtualized CPEs are dynamically instantiated or 282 removed, the configuration of Multi-Hop EBGP on the PE has to be 283 changed accordingly. 285 Egress peering engineering (EPE) is not sufficient. Running BGP on 286 virtualized CPEs in Cloud DCs requires GRE tunnels to be 287 established first, which requires the remote CPEs to support 288 address and key management capabilities. RFC 7024 (Virtual Hub & 289 Spoke) and Hierarchical VPN do not support the required 290 properties. 292 Also, there is a need for a mechanism to automatically trigger 293 configuration changes on PEs when remote CPEs' are instantiated or 294 moved (leading to an IP address change) or deleted. 296 EBGP Multi-hop design does not include a security mechanism by 297 default. The PE and remote CPEs need secure communication channels 298 when connecting via the public Internet. 300 Remote CPEs, if instantiated in Cloud DCs might have to traverse 301 NATs to reach PEs. It is not clear how BGP can be used between 302 devices located beyond the NAT and the devices located behind the 303 NAT. It is not clear how to configure the Next Hop on the PEs to 304 reach private IPv4 addresses. 306 3.5. Multicast traffic from/to the remote edges 308 Among the multiple floating PEs that are reachable from a remote 309 CPE, multicast traffic sent by the remote CPE towards the MPLS VPN 310 can be forwarded back to the remote CPE due to the PE receiving the 311 multicast packets forwarding the multicast/broadcast frame to other 312 PEs that in turn send to all attached CPEs. This process may cause 313 traffic loops. 315 This problem can be solved by selecting one floating PE as the CPE's 316 Designated Forwarder, similar to TRILL's Appointed Forwarders 317 [RFC6325]. 319 BGP/MPLS VPNs do not have features like TRILL's Appointed 320 Forwarders. 322 4. Gap Analysis of Overlay Edge Node's WAN Port Management 324 Very often the Hybrid Cloud DCs are interconnected by overlay 325 networks that arch over many different types of networks, such as 326 VPN, public Internet, wireless and wired infrastructures, etc. 327 Sometimes the enterprises' VPN providers do not have direct access 328 to the Cloud DCs that host some specific applications or workloads 329 operated by the enterprise. 331 Under those circumstances, the overlay network's edge nodes can have 332 WAN ports facing networks provided by different ISPs, some of these 333 networks may not be trustable, some others can be trusted like VPNs 334 (to some extent), etc. 336 If all WAN ports of an edge node are facing an untrusted network, 337 then all sensitive data to/from this edge node have to be encrypted, 338 usually by means of IPsec tunnels which can be terminated at the WAN 339 port address, at the edge node's loopback address if the loopback 340 address is routable in the wide area network, or even at the ingress 341 ports of the edge node. 343 If an edge node has some WAN ports facing trusted networks and 344 others facing untrusted networks, sensitive data can be forwarded 345 through ports facing the trusted networks natively (i.e., without 346 encryption) and forwarded through ports facing untrusted networks 347 assuming encryption. To achieve this flexibility of sending traffic 348 either encrypted or not encrypted depending on egress WAN ports, it 349 is necessary to have the IPsec tunnels terminated at the WAN ports 350 facing the untrusted networks. 352 In order to establish peer-wise secure encrypted communications 353 among those WAN ports of two edge nodes, it is necessary for the 354 edge nodes (peers) to be informed of the WAN port properties. 356 Some of those overlay networks (such as some deployed SDWAN 357 networks) use the modified NHRP protocol [RFC2332] to register WAN 358 ports of the edge nodes with their Controller (or NHRP server), 359 which then maps a private VPN address to a public IP address of the 360 destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to 361 establish tunnels between WAN ports of SDWAN edge nodes. 363 NHRP was originally intended for ATM address resolution, and as a 364 result, it misses many attributes that are necessary for dynamic 365 endpoint C-PE registration to the controller, such as: 367 - Interworking with the MPLS VPN control plane. An overlay edge can 368 have some ports facing the MPLS VPN network over which packets can 369 be forwarded without any encryption and some ports facing the 370 public Internet over which sensitive traffic needs to be 371 encrypted. 373 - Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of edge 374 nodes. When a network has more than 100 nodes, these protocols do 375 not scale well. 376 - NHRP does not have the IPsec attributes, which are needed for 377 peers to build Security Associations over the public Internet. 378 - NHRP messages do not have any field to encode the C-PE supported 379 encapsulation types, such as IPsec-GRE or IPsec-VxLAN. 380 - NHRP messages do not have any field to encode C-PE Location 381 identifiers, such as Site Identifier, System ID, and/or Port ID. 382 - NHRP messages do not have any field to describe the gateway(s) to 383 which the C-PE is attached. When a C-PE is instantiated in a Cloud 384 DC, it is desirable for the C-PE's owner to be informed about how 385 and where the C-PE is attached. 386 - NHRP messages do not have any field to describe C-PE's NAT 387 properties if the C-PE is using private IPv4 addresses, such as 388 the NAT type, Private address, Public address, Private port, 389 Public port, etc. 391 [BGP-SDWAN-PORT] describes how to use BGP to distribute SDWAN edge 392 properties to peers. SDWAN is an overlay network with specific 393 properties, such as application-based forwarding, augmented 394 transport, and user specified policies. There is a need to extend 395 the protocol to register WAN port properties of an edge node to the 396 overlay controller, which then propagates the information to other 397 overlay edge nodes that are authenticated and authorized to 398 communicate with them. 400 5. Aggregating VPN paths and Internet paths 402 Most likely, enterprises (especially the largest ones) already have 403 their C-PEs interconnected by VPN service providers, based upon VPN 404 techniques like EVPN, L2VPN, or L3VPN, and which can be lead to PE- 405 based or CPE-based VPN service designs. The commonly used PE-based 406 BGP/MPLS VPNs have C-PEs directly attached to PEs, the communication 407 between C-PEs and PEs is considered as secure as they are connected 408 by direct physical links albeit there could be routes leaking or 409 unauthorized routes being injected. MP-BGP can be used to learn & 410 distribute routes among C-PEs, but sometimes routes among C-PEs are 411 statically configured on the C-PEs. 413 For enterprises already interconnected by VPNs, if there are short 414 term high traffic volume that can't justify increasing the VPNs 415 capacity, it is desirable for the CPE to aggregate the bandwidth 416 that pertains to VPN paths and Internet paths by adding ports that 417 connect the CPE to the public Internet. Under this scenario, which 418 is referred to as the Overlay scenario throughout this document, it 419 is necessary for the C-PEs to manage and communicate with the 420 controller on how traffic is distributed among multiple 421 heterogeneous underlay networks, and also to manage secure tunnels 422 over untrusted networks. 424 When using NHRP for WAN port registration purposes, C-PEs need to 425 run two separate control planes: EVPN&BGP for CPE-based VPNs, and 426 NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate 427 control planes not only add complexity to C-PEs, but also increase 428 operational costs. 429 +---+ 430 +--------------|RR |----------+ 431 / Untrusted +-+-+ \ 432 / \ 433 / \ 434 +----+ +---------+ packets encrypted over +------+ +----+ 435 | TN3|--| A1-----+ Untrusted +------ B1 |--| TN1| 436 +----+ | C-PE A2-\ | C-PE | +----+ 437 +----+ | A A3--+--+ +---+---B2 B | +----+ 438 | TN2|--| | |PE+--------------+PE |---B3 |--| TN3| 439 +----+ +---------+ +--+ trusted +---+ +------+ +----+ 440 | WAN | 441 +----+ +---------+ +--+ packets +---+ +------+ +----+ 442 | TN1|--| C1--|PE| go natively |PE |-- D1 |--| TN1| 443 +----+ | C-PE C2--+--+ without encry+---+ | C-PE | +----+ 444 | C | +--------------+ | D | 445 | | | | 446 +----+ | C3--| without encrypt over | | +----+ 447 | TN2|--| C4--+---- Untrusted --+------D2 |--| TN2| 448 +----+ +---------+ +------+ +----+ 449 Figure 2: CPEs interconnected by VPN paths and Internet Paths 451 5.1. Control Plane for Overlay over Heterogeneous Networks 453 As described in [BGP-SDWAN-Usage], the Control Plane for Overlay 454 network over heterogeneous networks has three distinct properties: 456 - WAN Port Property registration to the Overlay Controller. 457 o To inform the Overlay controller and authorized peers of 458 the WAN port properties of the Edge nodes. When the WAN 459 ports are assigned private IPv4 addresses, this step can 460 register the type of NAT that translates these addresses 461 into public ones. 463 - Controller-facilitated IPsec SA management and NAT information 464 distribution 465 o The Overlay controller facilitates and manages the IPsec 466 configuration and peer authentication for all IPsec 467 tunnels terminated at the edge nodes. 469 - Establishing and Managing the topology and reachability for 470 services attached to the client ports of overlay edge nodes. 471 o This is for the overlay layer's route distribution, so 472 that a C-PE can populate its overlay routing table with 473 entries that identify the next hop for reaching a specific 474 route/service attached to remote nodes. [SECURE-EVPN] 475 describes EVPN and other options. 477 5.2. Using BGP UPDATE Messages 479 5.2.1. Lacking SD-WAN Segments Identifier 481 There could be multiple SD-WAN networks with their edge nodes 482 exchanging BGP UPDATE messages with the BGP RR. The multiple SD-WAN 483 networks could have common underlay networks. Therefore, it is very 484 important to have an identifier to differentiate BGP UPDATE messages 485 belonging to different SD-WAN networks (or sometimes called SD-WAN 486 Segmentations). Today's BGP doesn't have this feature yet, unless 487 there are multiple BGP instances and their corresponding RRs. 489 5.2.2. Missing attributes in Tunnel-Encap 491 [Tunnel-Encap] describes the BGP UPDATE Tunnel Path Attribute that 492 advertises endpoints' tunnel encapsulation capabilities for the 493 respective attached client routes encoded in the MP-NLRI Path 494 Attribute. The receivers of the BGP UPDATE can use any of the 495 supported encapsulations encoded in the Tunnel Path Attribute for 496 the routes encoded in the MP-NLRI Path Attribute. 498 Here are some of the issues raised by the use of [Tunnel-Encap] to 499 distribute Edge WAN port properties: 501 - [Tunnel-Encap] doesn't have the encoding to describe the NAT 502 information for WAN ports that are assigned private IPv4 addresses 503 yet. The NAT information needs to be propagated to the trusted 504 peers such as the virtual C-PEs instantiated in public Cloud DCs 505 via Controllers. 506 - The mechanism defined in [Tunnel-Encap] does not facilitate the 507 exchange of IPsec SA-specific parameters independently from 508 advertising the attached clients' routes, even after adding a new 509 IPsec tunnel type. 510 [Tunnel-Encap] requires all tunnels updates to be associated with 511 routes. There can be many client routes associated with an IPsec 512 tunnel established between two C-PEs' WAN ports; the corresponding 513 destination prefixes (as announced by the aforementioned routes) 514 may also be reached through the VPN underlay without any 515 encryption. 517 The establishment of an IPsec tunnel can fail, e.g., because the 518 two endpoints support different encryption algorithms. Multiple 519 negotiation messages that carry the IPsec SA parameters between 520 two end-points may be exchanged. This is why it is cleaner to 521 separate the establishment of an IPsec SA association between two 522 end-points from the policies enforced to map routes to a specific 523 IPSec SA. 525 If C-PEs need to establish a WAN Port-based IPsec SA, the 526 information encoded in the Tunnel Path Attribute should only apply 527 to the WAN ports and should be independent from the clients' 528 routes. 530 In addition, the Overlay IPsec SA Tunnel is very likely to be 531 established before clients' routes are attached. 533 - When an overlay network spans across large geographic regions 534 (such as countries or continents), one C-PE in one region may not 535 even be aware of remote CPEs in other regions that it needs to 536 communicate. Therefore, the distribution of the Overlay Edge WAN 537 ports information need to be restricted to the authorized peers. 539 5.3. SECURE-L3VPN/EVPN 541 [SECURE-L3VPN] describes a method to enrich BGP/MPLS VPN [RFC4364] 542 capabilities to allow some PEs to connect to other PEs via public 543 networks. [SECURE-L3VPN] introduces the concept of Red Interface & 544 Black Interface used by PEs, where the RED interfaces are used to 545 forward traffic into the VPN, and the Black Interfaces are used 546 between WAN ports through which only IPsec-formatted packets are 547 forwarded to the Internet or to any other backbone network, thereby 548 eliminating the need for MPLS transport in the backbone. 550 [SECURE-L3VPN] assumes PEs use MPLS over IPsec when sending traffic 551 through the Black Interfaces. 553 [SECURE-EVPN] describes a solution where point-to-multipoint BGP 554 signaling is used in the control plane for the Scenario #1 described 555 in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to 556 facilitate the key and policy exchange among PE devices to create 557 private pair-wise IPsec Security Associations without IKEv2 point- 558 to-point signaling or any other direct peer-to-peer session 559 establishment messages. 561 Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both 562 miss the aspects of aggregating VPN and Internet underlays. In 563 summary: 565 - Both documents assume that an IPsec tunnel is associated with 566 client traffic. Regardless of which WAN ports the traffic egress 567 from the edge, the client traffic associated with IPsec is always 568 encrypted. Within the context of an overlay architecture that 569 relies upon minimizing resource used for encryption, traffic sent 570 from an edge node can be encrypted once and forwarded through a 571 WAN port towards an untrusted network, but can also remain 572 unencrypted and be forwarded at different times through a WAN port 573 to the BGP/MPLS VPN. 575 - The [SECURE-L3VPN] assumes that a CPE "registers" with the RR. 576 However, it does not say how. It assumes that the remote CPEs are 577 pre-configured with the IPsec SA manually. For overlay networks to 578 connect Hybrid Cloud DCs, Zero Touch Provisioning is expected. 579 Manual configuration is not an option. 581 - The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an 582 IPsec tunnel. For management channel, TLS/DTLS is more economical 583 than IPsec. The following assumption made by [SECURE-L3VPN] can be 584 difficult to meet in the environment where zero touch provisioning 585 is expected: 586 A CPE must also be provisioned with whatever additional 587 information is needed in order to set up an IPsec SA with 588 each of the red RRs 590 - IPsec requires periodic refreshment of the keys. The draft does 591 not provide any information about how to synchronize the 592 refreshment among multiple nodes. 593 - IPsec usually sends configuration parameters to two endpoints only 594 and lets these endpoints negotiate the key. The [SECURE-L3VPN] 595 assumes that the RR is responsible for creating/managing the key 596 for all endpoints. When one endpoint is compromised, all other 597 connections may be impacted. 599 5.4. Preventing attacks from Internet-facing ports 601 When C-PEs have Internet-facing ports, additional security risks are 602 raised. 604 To mitigate security risks, in addition to requiring Anti-DDoS 605 features on C-PEs, it is necessary for C-PEs to support means to 606 determine whether traffic sent by remote peers is legitimate to 607 prevent spoofing attacks, in particular. 609 6. Gap Summary 611 Here is the summary of the technical gaps discussed in this 612 document: 614 - For Accessing Cloud Resources 616 a) When a remote vCPE can be reached by multiple PEs of one 617 provider VPN network, it is not straightforward to designate 618 which egress PE to the remote vCPE based on applications 619 b) Need automated and reliable tools to map the user-friendly 620 (natural language) access rules into machine readable 621 policies and to provide interfaces for enterprises to self- 622 manage policy enforcement points for their own workloads. 623 c) NAT Traversal. An enterprise's network controller needs to be 624 informed of the NAT properties for its workloads in Cloud 625 DCs. If the workloads are attached to the enterprise's own 626 vCPEs instantiated in the Cloud DCs, the task can be 627 achieved. 628 d) The multicast traffic to/from remote vCPE needs a feature 629 like Appointed Forwarder specified by TRILL to prevent 630 multicast data frames from looping around. 631 e) BGP between PEs and remote CPEs via untrusted networks. 632 f) Traffic Path Management 634 - Overlay Edge Node's WAN Port Management: BGP UPDATE propagate 635 client's routes information, but don't distinguish network facing 636 ports. 638 - Aggregating VPN paths and Internet paths 640 a) Control Plane for Overlay over Heterogeneous Networks is not 641 clear. 642 b) BGP UPDATE Messages missing properties: 644 - Lacking SD-WAN Segments Identifier 645 - Missing attributes in Tunnel-Encap 647 c) SECURE-L3VPN/EVPN is not enough 648 d) Missing clear methods in preventing attacks from Internet- 649 facing ports 651 7. Manageability Considerations 653 Zero touch provisioning of overlay networks to interconnect Hybrid 654 Clouds is highly desired. It is necessary for a newly powered up 655 edge node to establish a secure connection (by means of TLS, DTLS, 656 etc.) with its controller. 658 8. Security Considerations 660 Cloud Services are built upon shared infrastructures, therefore 661 not secure by nature. 663 Secure user identity management, authentication, and access 664 control mechanisms are important. Developing appropriate security 665 measurements can enhance the confidence needed by enterprises to 666 fully take advantage of Cloud Services. 668 9. IANA Considerations 670 This document requires no IANA actions. RFC Editor: Please remove 671 this section before publication. 673 10. References 675 10.1. Normative References 677 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 678 Requirement Levels", BCP 14, RFC 2119, March 1997. 680 10.2. Informative References 682 [RFC8192] S. Hares, et al, "Interface to Network Security Functions 683 (I2NSF) Problem Statement and Use Cases", July 2017 685 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 686 Address Family Identifier (SAFI) and the BGP Tunnel 687 Encapsulation Attribute", April 2009. 689 [BGP-SDWAN-PORT]L. Dunbar, et al, "Subsequent Address Family 690 Indicator for SDWAN Ports", draft-dunbar-idr-sdwan-port- 691 safi-00, Work-in-progress, March 2019. 693 [BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for 694 SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework- 695 00, work-in-progress, Feb 2019. 697 [Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation 698 Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018. 700 [SECURE-EVPN A. Sajassi, et al, draft-sajassi-bess-secure-evpn-01, 701 work in progress, March 2019. 703 [SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public 704 Infrastructure", draft-rosen-bess-secure-l3vpn-00, work- 705 in-progress, July 2018 707 [DMVPN] Dynamic Multi-point VPN: 708 https://www.cisco.com/c/en/us/products/security/dynamic- 709 multipoint-vpn-dmvpn/index.html 711 [DSVPN] Dynamic Smart VPN: 712 http://forum.huawei.com/enterprise/en/thread-390771-1- 713 1.html 715 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 716 storage, distribution and enforcement of policies for 717 network security", Nov 2007. 719 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect 720 Underlay to Cloud Overlay Problem Statement", draft-dm- 721 net2cloud-problem-statement-02, June 2018 723 11. Acknowledgments 725 Acknowledgements to John Drake for his review and contributions. 726 Many thanks to John Scudder for stimulating the clarification 727 discussion on the Tunnel-Encap draft so that our gap analysis can be 728 more accurate. 730 This document was prepared using 2-Word-v2.0.template.dot. 732 Authors' Addresses 734 Linda Dunbar 735 Futurewei 736 Email: ldunbar@futurewei.com 738 Andrew G. Malis 739 Independent 740 Email: agmalis@gmail.com 742 Christian Jacquenet 743 Orange 744 Rennes, 35000 745 France 746 Email: Christian.jacquenet@orange.com