< draft-ietf-rtgwg-net2cloud-gap-analysis-04.txt   draft-ietf-rtgwg-net2cloud-gap-analysis-05.txt >
Network Working Group L. Dunbar Network Working Group L. Dunbar
Internet Draft Futurewei Internet Draft Futurewei
Intended status: Informational A. Malis Intended status: Informational A. Malis
Expires: September 8, 2020 Independent Expires: September 18, 2020 Independent
C. Jacquenet C. Jacquenet
Orange Orange
March 8, 2020 March 18, 2020
Gap Analysis of Dynamic Networks to Hybrid Cloud DCs Networks Connecting to Hybrid Cloud DCs: Gap Analysis
draft-ietf-rtgwg-net2cloud-gap-analysis-04 draft-ietf-rtgwg-net2cloud-gap-analysis-05
Abstract Abstract
This document analyzes the technological gaps, especially IETF This document analyzes the technical gaps that may affect the
protocols gaps, to achieve dynamically interconnecting workloads and dynamic connection to workloads and applications hosted in hybrid
applications hosted in Hybrid Cloud Data Centers. Cloud Data Centers from enterprise premises.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 1, line 41 skipping to change at page 2, line ?
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress." reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on September 8, 2020. This Internet-Draft will expire on September 18, 2020.
xxx, et al. Expires September 18, 2020 [Page
1]
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................3 1. Introduction...................................................3
2. Conventions used in this document..............................3 2. Conventions used in this document..............................3
3. Gap Analysis for Accessing Cloud Resources.....................4 3. Gap Analysis for Accessing Cloud Resources.....................4
4. Gap Analysis of Overlay Edge Node's WAN Ports Management.......4 4. Gap Analysis of Overlay Edge Node's WAN Port Management........4
5. Aggregating VPN paths and Internet paths.......................6 5. Aggregating VPN paths and Internet paths.......................6
5.1. Control Plane for Overlay over Heterogeneous Networks.....7 5.1. Control Plane for Overlay over Heterogeneous Networks.....7
5.2. Using BGP UPDATE Messages.................................8 5.2. Using BGP UPDATE Messages.................................8
5.2.1. Lacking SD-WAN Segments Identifier...................8
5.2.2. Missing attributes in Tunnel-Encap...................8
5.3. SECURE-L3VPN/EVPN.........................................9 5.3. SECURE-L3VPN/EVPN.........................................9
5.4. Preventing attacks from Internet-facing ports............10 5.4. Preventing attacks from Internet-facing ports............11
6. C-PEs not directly connected to VPN PEs.......................10 6. C-PEs not directly connected to VPN PEs.......................11
6.1. Floating PEs to connect to Remote CPEs...................13 6.1. Floating PEs to connect to Remote CPEs...................14
6.2. NAT Traversal............................................13 6.2. NAT Traversal............................................14
6.3. Complexity of using BGP between PEs and remote CPEs via 6.3. Complexity of using BGP between PEs and remote CPEs via
Internet......................................................13 Internet......................................................14
6.4. Designated Forwarder to the remote edges.................14 6.4. Designated Forwarder to the remote edges.................15
6.5. Traffic Path Management..................................15 6.5. Traffic Path Management..................................16
7. Manageability Considerations..................................15 7. Manageability Considerations..................................16
8. Security Considerations.......................................15 8. Security Considerations.......................................16
9. IANA Considerations...........................................16 9. IANA Considerations...........................................17
10. References...................................................16 10. References...................................................17
10.1. Normative References....................................16 10.1. Normative References....................................17
10.2. Informative References..................................16 10.2. Informative References..................................17
11. Acknowledgments..............................................17 11. Acknowledgments..............................................18
1. Introduction 1. Introduction
[Net2Cloud-Problem] describes the problems enterprises face today [Net2Cloud-Problem] describes the problems enterprises face today
when interconnecting their branch offices with dynamic workloads in when interconnecting their branch offices with dynamic workloads
third party data centers (a.k.a. Cloud DCs). This document analyzes hosted in third party data centers (a.k.a. Cloud DCs). In
the IETF routing protocols to identify if there are gaps or if particular, this document analyzes the routing protocols to identify
protocol extension might be needed. whether there are any gaps that may impede such interconnection
which may for example justify additional specification effort to
define proper protocol extensions.
For the sake of readability, an edge, an endpoint, C-PE, or CPE are For the sake of readability, an edge, an endpoint, C-PE, or CPE are
used interchangeably throughout this document. However, each term used interchangeably throughout this document. More precisely:
has some minor emphasis, especially when used in other related
documents:
. Edge: could include multiple devices (virtual or physical); . Edge: may include multiple devices (virtual or physical);
. endpoint: to refer to a WAN port of an Edge device; . endpoint: refers to a WAN port of device located in the edge;
. C-PE: more for provider owned edge, e.g. for SECURE-EVPN's PE . C-PE: provider-owned edge, e.g. for SECURE-EVPN's PE-based
based VPN, where PE is the edge node; BGP/MPLS VPN, where PE is the edge node;
. CPE: more for enterprise owned edge. . CPE: device located in enterprise premises.
2. Conventions used in this document 2. Conventions used in this document
Cloud DC: Third party Data Centers that usually host applications Cloud DC: Third party Data Centers that usually host applications
and workload owned by different organizations or and workload owned by different organizations or
tenants. tenants.
Controller: Used interchangeably with Overlay controller to manage Controller: Used interchangeably with Overlay controller to manage
overlay path creation/deletion and monitor the path overlay path creation/deletion and monitor the path
conditions between sites. conditions between sites.
skipping to change at page 4, line 14 skipping to change at page 4, line 14
management, visibility & control. When the underlay is a management, visibility & control. When the underlay is a
private network, traffic may be forwarded without any private network, traffic may be forwarded without any
additional encryption; when the underlay networks are additional encryption; when the underlay networks are
public, such as the Internet, some traffic needs to be public, such as the Internet, some traffic needs to be
encrypted when passing through (depending on user- encrypted when passing through (depending on user-
provided policies). provided policies).
3. Gap Analysis for Accessing Cloud Resources 3. Gap Analysis for Accessing Cloud Resources
Many problems described in the [Net2Cloud-Problem] are not in the Many problems described in the [Net2Cloud-Problem] are not in the
scope of IETF, let alone IETF Routing area. Therefore, this document scope of IETF, let alone IETF Routing area. This document primarily
will not cover the detailed protocol gaps analysis for security, focuses on the gap analysis for protocols in IETF Routing area.
identity management or DNS for Cloud Resources.
4. Gap Analysis of Overlay Edge Node's WAN Ports Management 4. Gap Analysis of Overlay Edge Node's WAN Port Management
Very often the Hybrid Cloud DCs are interconnected by overlay Very often the Hybrid Cloud DCs are interconnected by overlay
networks that arch over many different types of networks, such as networks that arch over many different types of networks, such as
VPN, public internet, wireless, etc. Sometimes the enterprises' VPN VPN, public Internet, wireless and wired infrastructures, etc.
providers do not have direct access to the Cloud DCs that are Sometimes the enterprises' VPN providers do not have direct access
optimal for some specific applications or workloads. to the Cloud DCs that host some specific applications or workloads
operated by the enterprise.
Under those circumstances, the overlay network' edges can have WAN Under those circumstances, the overlay network's edge nodes can have
ports facing networks provided by different ISPs, some can be WAN ports facing networks provided by different ISPs, some of these
untrusted public internet, some can be trusted provider VPN, some networks may not be trustable, some others can be trusted like VPNs
can be Cloud internal networks, and some can be others. (to some extent), etc.
If all WAN ports of an edge node are facing untrusted network, then If all WAN ports of an edge node are facing an untrusted network,
all sensitive data to/from this edge have to be encrypted, usually then all sensitive data to/from this edge node have to be encrypted,
by IPsec tunnels which can be terminated at the WAN port address, at usually by means of IPsec tunnels which can be terminated at the WAN
the edge node's loopback address if the loopback address is routable port address, at the edge node's loopback address if the loopback
in the wide area network, or even at the ingress ports of the edge address is routable in the wide area network, or even at the ingress
node. ports of the edge node.
If an edge node has some WAN ports facing trusted VPN and some If an edge node has some WAN ports facing trusted networks and
facing untrusted networks, sensitive data can be forwarded through others facing untrusted networks, sensitive data can be forwarded
ports facing VPN natively without encryption and forwarded through through ports facing the trusted networks natively (i.e., without
ports facing public network with encryption. To achieve this encryption) and forwarded through ports facing untrusted networks
flexibility, it is necessary to have the IPsec tunnels terminated at assuming encryption. To achieve this flexibility of sending traffic
the WAN ports facing the untrusted networks. either encrypted or not encrypted depending on egress WAN ports, it
is necessary to have the IPsec tunnels terminated at the WAN ports
facing the untrusted networks.
In order to establish pair-wise secure encrypted connection among In order to establish peer-wise secure encrypted communications
those WAN ports, it is necessary for peers to be informed of the WAN among those WAN ports of two edge nodes, it is necessary for the
port properties. edge nodes (peers) to be informed of the WAN port properties.
Some of those overlay networks (such as some deployed SDWAN Some of those overlay networks (such as some deployed SDWAN
networks) use the modified NHRP protocol [RFC2332] to register WAN networks) use the modified NHRP protocol [RFC2332] to register WAN
ports of the edges with their "Controller" (or NHRP server), which ports of the edge nodes with their Controller (or NHRP server),
then map a private VPN address to a public IP address of the which then maps a private VPN address to a public IP address of the
destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to
establish tunnels between WAN ports of SDWAN edge nodes. establish tunnels between WAN ports of SDWAN edge nodes.
NHRP was originally intended for ATM address resolution, and as a NHRP was originally intended for ATM address resolution, and as a
result, it misses many attributes that are necessary for dynamic result, it misses many attributes that are necessary for dynamic
endpoint C-PE registration to the controller, such as: endpoint C-PE registration to the controller, such as:
- Interworking with the MPLS VPN control plane. An overlay edge can - Interworking with the MPLS VPN control plane. An overlay edge can
have some ports facing the MPLS VPN network over which packets can have some ports facing the MPLS VPN network over which packets can
be forwarded without any encryption and some ports facing the be forwarded without any encryption and some ports facing the
public Internet over which sensitive traffic needs to be public Internet over which sensitive traffic needs to be
encrypted. encrypted.
- Scalability: NHRP/DSVPN/DMVPN works fine with small numbers of - Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of edge
edge nodes. When a network has more than 100 nodes, these nodes. When a network has more than 100 nodes, these protocols do
protocols do not scale well. not scale well.
- NHRP does not have the IPsec attributes, which are needed for - NHRP does not have the IPsec attributes, which are needed for
peers to build Security Associations over the public internet. peers to build Security Associations over the public Internet.
- NHRP messages do not have any field to encode the C-PE supported - NHRP messages do not have any field to encode the C-PE supported
encapsulation types, such as IPsec-GRE or IPsec-VxLAN. encapsulation types, such as IPsec-GRE or IPsec-VxLAN.
- NHRP messages do not have any field to encode C-PE Location - NHRP messages do not have any field to encode C-PE Location
identifiers, such as Site Identifier, System ID, and/or Port ID. identifiers, such as Site Identifier, System ID, and/or Port ID.
- NHRP messages do not have any field to describe the gateway(s) to - NHRP messages do not have any field to describe the gateway(s) to
which the C-PE is attached. When a C-PE is instantiated in a Cloud which the C-PE is attached. When a C-PE is instantiated in a Cloud
DC, it is desirable for C-PE's owner to be informed of how/where DC, it is desirable for the C-PE's owner to be informed about how
the C-PE is attached. and where the C-PE is attached.
- NHRP messages do not have any field to describe C-PE's NAT - NHRP messages do not have any field to describe C-PE's NAT
properties if the C-PE is using private addresses, such as the NAT properties if the C-PE is using private IPv4 addresses, such as
type, Private address, Public address, Private port, Public port, the NAT type, Private address, Public address, Private port,
etc. Public port, etc.
[BGP-SDWAN-PORT] describes how to use BGP to distribute SDWAN edge [BGP-SDWAN-PORT] describes how to use BGP to distribute SDWAN edge
properties to peers. There is need to extend the protocol to properties to peers. SDWAN is an overlay network with specific
register WAN ports properties to the overlay controller, which then properties, such as application-based forwarding, augmented
propagates the information to other overlay edge nodes that are transport, and user specified policies. There is a need to extend
authenticated and authorized to communicate with them. the protocol to register WAN port properties of an edge node to the
overlay controller, which then propagates the information to other
overlay edge nodes that are authenticated and authorized to
communicate with them.
5. Aggregating VPN paths and Internet paths 5. Aggregating VPN paths and Internet paths
Most likely, enterprises (especially the largest ones) already have Most likely, enterprises (especially the largest ones) already have
their C-PEs interconnected by providers VPNs, such as EVPN, L2VPN, their C-PEs interconnected by VPN service providers, based upon VPN
or L3VPN, which can be PE-based or CPE-based. The commonly used PE- techniques like EVPN, L2VPN, or L3VPN, and which can be lead to PE-
based VPNs have C-PE directly attached to PEs, therefore the based or CPE-based VPN service designs. The commonly used PE-based
communication between C-PEs and PEs is considered as secure. MP-BGP BGP/MPLS VPNs have C-PEs directly attached to PEs, the communication
is used to learn & distribute routes among C-PEs, even though between C-PEs and PEs is considered as secure as they are connected
sometimes routes among C-PEs are statically configured on the C-PEs. by direct physical links albeit there could be routes leaking or
unauthorized routes being injected. MP-BGP can be used to learn &
distribute routes among C-PEs, but sometimes routes among C-PEs are
statically configured on the C-PEs.
For enterprises already interconnected by VPNs, it is desirable to For enterprises already interconnected by VPNs, if there are short
aggregate the bandwidth among VPN paths and Internet paths by C-PEs term high traffic volume that can't justify increasing the VPNs
adding additional ports facing public internet. Under this scenario, capacity, it is desirable for the CPE to aggregate the bandwidth
which is referred to as Overlay throughout this document, it is that pertains to VPN paths and Internet paths by adding ports that
necessary for the C-PEs to manage and communicate with controller on connect the CPE to the public Internet. Under this scenario, which
how traffic are distributed among multiple heterogenous WAN underlay is referred to as the Overlay scenario throughout this document, it
networks, and manage secure tunnels over untrusted networks is necessary for the C-PEs to manage and communicate with the
independently from the attached clients routes. controller on how traffic is distributed among multiple
heterogeneous underlay networks, and also to manage secure tunnels
over untrusted networks.
When using NHRP for WAN ports registration purposes, C-PEs need to When using NHRP for WAN port registration purposes, C-PEs need to
run two separate control planes: EVPN&BGP for CPE-based VPNs, and run two separate control planes: EVPN&BGP for CPE-based VPNs, and
NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate
control planes not only add complexity to C-PEs, but also increase control planes not only add complexity to C-PEs, but also increase
operational cost. operational costs.
+---+ +---+
+--------------|RR |----------+ +--------------|RR |----------+
/ Untrusted +-+-+ \ / Untrusted +-+-+ \
/ \ / \
/ \ / \
+----+ +---------+ packets encrypted over +------+ +----+ +----+ +---------+ packets encrypted over +------+ +----+
| TN3|--| A1-----+ Untrusted +------ B1 |--| TN1| | TN3|--| A1-----+ Untrusted +------ B1 |--| TN1|
+----+ | C-PE A2-\ | C-PE | +----+ +----+ | C-PE A2-\ | C-PE | +----+
+----+ | A A3--+--+ +---+---B2 B | +----+ +----+ | A A3--+--+ +---+---B2 B | +----+
skipping to change at page 7, line 30 skipping to change at page 7, line 30
| C | +--------------+ | D | | C | +--------------+ | D |
| | | | | | | |
+----+ | C3--| without encrypt over | | +----+ +----+ | C3--| without encrypt over | | +----+
| TN2|--| C4--+---- Untrusted --+------D2 |--| TN2| | TN2|--| C4--+---- Untrusted --+------D2 |--| TN2|
+----+ +---------+ +------+ +----+ +----+ +---------+ +------+ +----+
Figure 1: CPEs interconnected by VPN paths and Internet Paths Figure 1: CPEs interconnected by VPN paths and Internet Paths
5.1. Control Plane for Overlay over Heterogeneous Networks 5.1. Control Plane for Overlay over Heterogeneous Networks
As described in [BGP-SDWAN-Usage], the Control Plane for Overlay As described in [BGP-SDWAN-Usage], the Control Plane for Overlay
network over heterogenous networks has three distinct properties: network over heterogeneous networks has three distinct properties:
- WAN Port Property registration to the Overlay Controller. - WAN Port Property registration to the Overlay Controller.
o To inform the Overlay controller and authorized peers of o To inform the Overlay controller and authorized peers of
the WAN port properties of the Edge nodes. When the WAN the WAN port properties of the Edge nodes. When the WAN
ports are assigned private addresses, this step can ports are assigned private IPv4 addresses, this step can
register the type of NAT that translates private addresses register the type of NAT that translates these addresses
into public ones. into public ones.
- Controller facilitated IPsec SA management and NAT information - Controller-facilitated IPsec SA management and NAT information
distribution distribution
o It is for Overlay controller to facilitate or manage the o The Overlay controller facilitates and manages the IPsec
IPsec configuration and peer authentication for all IPsec configuration and peer authentication for all IPsec
tunnels terminated at the edge nodes. tunnels terminated at the edge nodes.
- Establishing and Managing the topology and reachability for - Establishing and Managing the topology and reachability for
services attached to the client ports of overlay edge nodes. services attached to the client ports of overlay edge nodes.
o This is for the overlay layer's route distribution, so o This is for the overlay layer's route distribution, so
that a C-PE can populate its overlay routing table with that a C-PE can populate its overlay routing table with
entries that identify the next hop for reaching a specific entries that identify the next hop for reaching a specific
route/service attached to remote nodes. [SECURE-EVPN] route/service attached to remote nodes. [SECURE-EVPN]
describes EVPN and other options. describes EVPN and other options.
5.2. Using BGP UPDATE Messages 5.2. Using BGP UPDATE Messages
[Tunnel-Encap] describe the BGP UPDATE Tunnel Path Attribute that 5.2.1. Lacking SD-WAN Segments Identifier
advertise endpoints' tunnel encapsulation capability for the
There could be multiple SD-WAN networks with their edge nodes
exchanging BGP UPDATE messages with the BGP RR. The multiple SD-WAN
networks could have common underlay networks. Therefore, it is very
important to have an identifier to differentiate BGP UPDATE messages
belonging to different SD-WAN networks (or sometimes called SD-WAN
Segmentations). Today's BGP doesn't have this feature yet, unless
there are multiple BGP instances and their corresponding RRs.
5.2.2. Missing attributes in Tunnel-Encap
[Tunnel-Encap] describes the BGP UPDATE Tunnel Path Attribute that
advertises endpoints' tunnel encapsulation capabilities for the
respective attached client routes encoded in the MP-NLRI Path respective attached client routes encoded in the MP-NLRI Path
Attribute. The receivers of the BGP UPDATE can use any of the Attribute. The receivers of the BGP UPDATE can use any of the
supported encapsulations encoded in the Tunnel Path Attribute for supported encapsulations encoded in the Tunnel Path Attribute for
the routes encoded in the MP-NLRI Path Attribute. the routes encoded in the MP-NLRI Path Attribute.
Here are some of the gaps using [Tunnel-Encap] to distribute Edge Here are some of the issues raised by the use of [Tunnel-Encap] to
WAN port properties: distribute Edge WAN port properties:
- [Tunnel-Encap] doesn't yet have the encoding to describe the NAT - [Tunnel-Encap] doesn't have the encoding to describe the NAT
information for WAN ports that have private addresses. The NAT information for WAN ports that are assigned private IPv4 addresses
information needs to be propagated to the trusted peers via yet. The NAT information needs to be propagated to the trusted
Controllers, such as the virtual C-PEs instantiated in public peers such as the virtual C-PEs instantiated in public Cloud DCs
Cloud DCs. via Controllers.
- It is not easy using the current mechanism in [Tunnel-Encap] to - The mechanism defined in [Tunnel-Encap] does not facilitate the
exchange IPsec SA specific parameters independently from exchange of IPsec SA-specific parameters independently from
advertising the attached clients' routes, even after adding a new advertising the attached clients' routes, even after adding a new
IPsec tunnel type. IPsec tunnel type.
[Tunnel-Encap] requires all tunnels updates are associated with [Tunnel-Encap] requires all tunnels updates to be associated with
routes. There can be many client routes associated with the IPsec routes. There can be many client routes associated with an IPsec
tunnel between two C-PEs' WAN ports; the corresponding destination tunnel established between two C-PEs' WAN ports; the corresponding
prefixes (as announced by the aforementioned routes) may also be destination prefixes (as announced by the aforementioned routes)
reached through the VPN underlay without any encryption. may also be reached through the VPN underlay without any
encryption.
The establishment of an IPsec tunnel can fail, such as due to the The establishment of an IPsec tunnel can fail, e.g., because the
two endpoints supporting different encryption algorithms or other two endpoints support different encryption algorithms. Multiple
reasons. There can be multiple negotiations messages for the IPsec negotiation messages that carry the IPsec SA parameters between
SA parameters between two end points. That is why IPsec SA two end-points may be exchanged. This is why it is cleaner to
association establishment between end points is independent from separate the establishment of an IPsec SA association between two
the policies on mapping routes to specific IPSec SA. end-points from the policies enforced to map routes to a specific
IPSec SA.
If C-PEs need to establish WAN Port based IPsec SA, the If C-PEs need to establish a WAN Port-based IPsec SA, the
information encoded in Tunnel Path Attribute should only apply to information encoded in the Tunnel Path Attribute should only apply
the WAN ports and should be independent of the clients' routes. to the WAN ports and should be independent from the clients'
routes.
In addition, the Overlay IPsec SA Tunnel may need to be In addition, the Overlay IPsec SA Tunnel is very likely to be
established before clients' routes are attached. established before clients' routes are attached.
- C-PEs tend to communicate with a subset of the other C-PEs, not - When an overlay network spans across large geographic regions
all the C-PEs need to be connected through a mesh topology. (such as countries or continents), one C-PE in one region may not
Therefore, the distribution of the Overlay Edge WAN ports even be aware of remote CPEs in other regions that it needs to
information need be be scoped to the authorized peers. communicate. Therefore, the distribution of the Overlay Edge WAN
ports information need to be restricted to the authorized peers.
5.3. SECURE-L3VPN/EVPN 5.3. SECURE-L3VPN/EVPN
[SECURE-L3VPN] describes how to extend the BGP/MPLS VPN [RFC4364] [SECURE-L3VPN] describes a method to enrich BGP/MPLS VPN [RFC4364]
capabilities to allow some PEs to connect to other PEs via public capabilities to allow some PEs to connect to other PEs via public
networks. [SECURE-L3VPN] introduces the concept of Red Interface & networks. [SECURE-L3VPN] introduces the concept of Red Interface &
Black Interface used by PEs, where the RED interfaces are used to Black Interface used by PEs, where the RED interfaces are used to
forward traffic into the VPN, and the Black Interfaces are used forward traffic into the VPN, and the Black Interfaces are used
between WAN ports through which only IPsec-protected packets are between WAN ports through which only IPsec-formatted packets are
forwarded to the Internet or to other backbone network thereby forwarded to the Internet or to any other backbone network, thereby
eliminating the need for MPLS transport in the backbone. eliminating the need for MPLS transport in the backbone.
[SECURE-L3VPN] assumes PEs using MPLS over IPsec when sending [SECURE-L3VPN] assumes PEs use MPLS over IPsec when sending traffic
traffic through the Black Interfaces. through the Black Interfaces.
[SECURE-EVPN] describes a solution where point-to-multipoint BGP [SECURE-EVPN] describes a solution where point-to-multipoint BGP
signaling is used in the control plane for the Scenario #1 described signaling is used in the control plane for the Scenario #1 described
in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to
facilitate the key and policy exchange among PE devices to create facilitate the key and policy exchange among PE devices to create
private pair-wise IPsec Security Associations without IKEv2 point- private pair-wise IPsec Security Associations without IKEv2 point-
to-point signaling or any other direct peer-to-peer session to-point signaling or any other direct peer-to-peer session
establishment messages. establishment messages.
Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both
miss the aspects of aggregating VPN and Internet underlays. In miss the aspects of aggregating VPN and Internet underlays. In
summary: summary:
- Both documents assume a client traffic is either forwarded all - Both documents assume that an IPsec tunnel is associated with
encrypted through an IPsec tunnel, or not encrypted at all through client traffic. Regardless of which WAN ports the traffic egress
a different tunnel regardless which WAN ports the traffic egress from the edge, the client traffic associated with IPsec is always
the PEs towards WAN. For Overlay arch over trusted VPN and encrypted. Within the context of an overlay architecture that
untrusted Internet, one client traffic can be forwarded encrypted relies upon minimizing resource used for encryption, traffic sent
at one time through a WAN port towards untrusted network and be from an edge node can be encrypted once and forwarded through a
forwarded unencrypted at different time through a WAN port to MPLS WAN port towards an untrusted network, but can also remain
VPN. unencrypted and be forwarded at different times through a WAN port
to the BGP/MPLS VPN.
- The [SECURE-L3VPN] assumes that a CPE "registers" with the RR. - The [SECURE-L3VPN] assumes that a CPE "registers" with the RR.
However, it does not say how. It assumes that the remote CPEs are However, it does not say how. It assumes that the remote CPEs are
pre-configured with the IPsec SA manually. In Overlay network to pre-configured with the IPsec SA manually. For overlay networks to
connect Hybrid Cloud DCs, Zero Touch Provisioning is expected. connect Hybrid Cloud DCs, Zero Touch Provisioning is expected.
Manual configuration is not an option, especially for the edge Manual configuration is not an option.
devices that are deployed in faraway places.
- The [SECURE-L3VPN] assumes that C-PEs and RR are connected via an - The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an
IPsec tunnel. Missing TLS/DTLS. The following assumption by IPsec tunnel. For management channel, TLS/DTLS is more economical
[SECURE-L3VPN] becomes invalid for the Overlay network to connect than IPsec. The following assumption made by [SECURE-L3VPN] can be
Hybrid Cloud DCs where automatic synchronization of IPsec SA difficult to meet in the environment where zero touch provisioning
between C-PEs and RR is needed: is expected:
A CPE must also be provisioned with whatever additional A CPE must also be provisioned with whatever additional
information is needed in order to set up an IPsec SA with information is needed in order to set up an IPsec SA with
each of the red RRs each of the red RRs
- IPsec requires periodic refreshment of the keys. The draft does - IPsec requires periodic refreshment of the keys. The draft does
not provide any information about how to synchronize the not provide any information about how to synchronize the
refreshment among multiple nodes. refreshment among multiple nodes.
- IPsec usually sends configuration parameters to two endpoints only - IPsec usually sends configuration parameters to two endpoints only
and lets these endpoints negotiate the key. The [SECURE-L3VPN] and lets these endpoints negotiate the key. The [SECURE-L3VPN]
assumes that the RR is responsible for creating/managing the key assumes that the RR is responsible for creating/managing the key
for all endpoints. When one endpoint is compromised, all other for all endpoints. When one endpoint is compromised, all other
connections will be impacted. connections may be impacted.
5.4. Preventing attacks from Internet-facing ports 5.4. Preventing attacks from Internet-facing ports
When C-PEs have Internet-facing ports, additional security risks are When C-PEs have Internet-facing ports, additional security risks are
raised. raised.
To mitigate security risks, in addition to requiring Anti-DDoS To mitigate security risks, in addition to requiring Anti-DDoS
features on C-PEs, it is necessary for C-PEs to support means to features on C-PEs, it is necessary for C-PEs to support means to
determine whether traffic sent by remote peers is legitimate to determine whether traffic sent by remote peers is legitimate to
prevent spoofing attacks. prevent spoofing attacks, in particular.
6. C-PEs not directly connected to VPN PEs 6. C-PEs not directly connected to VPN PEs
Because of the ephemeral property of the selected Cloud DCs for Because of the ephemeral property of the selected Cloud DCs for
specific workloads/Apps, an enterprise or its network service specific workloads/Apps, an enterprise or its network service
provider may not have direct physical connections to the Cloud DCs provider may not have direct physical connections to the Cloud DCs
that are optimal for hosting the enterprise's specific that are optimal for hosting the enterprise's specific
workloads/Apps. Under those circumstances, Overlay is a very workloads/Apps. Under those circumstances, an overlay network design
flexible choice to interconnect the enterprise on-premises data can be an option to interconnect the enterprise's on-premises data
centers & branch offices to its desired Cloud DCs. centers & branch offices to its desired Cloud DCs.
However, Overlay paths established over the public Internet can have However, overlay paths established over the public Internet can have
unpredictable performance, especially over long distances and across unpredictable performance, especially over long distances and across
operators' domains. Therefore, it is highly desirable to steer as operators' domains. Therefore, it is highly desirable to minimize
much as possible the portion of Overlay paths over the enterprise's the distance or the number of segments that traffic had to be
existing VPN that has guaranteed SLA to minimize the distance or the forwarded over the public Internet.
number of segments over the public Internet.
MEF Cloud Service Architecture [MEF-Cloud] also describes a use case The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud]
of network operators using Overlay path over LTE or the public also describes a use case of network operators using Overlay paths
Internet for last mile access where the VPN service providers cannot over a LTE network or the public Internet for the last mile access
necessarily provide the required physical infrastructure. where the VPN service providers cannot always provide the required
physical infrastructure.
Under those scenarios, one or two of the Overlay endpoints may not In these scenarios, some overlay edge nodes may not be directly
be directly attached to the PEs of a VPN Domain. attached to the PEs that participate to the delivery and the
operation of the enterprise's VPN.
When using Overlay to connect the enterprise's existing sites to the When using an overlay network to connect the enterprise's sites to
workloads hosted in Cloud DCs, the corresponding C-PEs have to be the workloads hosted in Cloud DCs, the corresponding C-PEs have to
upgraded to support the desired Overlay. If the workloads hosted in be upgraded to connect to the said overlay network. If the
Cloud DCs need to be connected to many sites, the upgrade process workloads hosted in Cloud DCs need to be connected to many sites,
can be very expensive. the upgrade process can be very expensive.
[Net2Cloud-Problem] describes a hybrid network approach that extend [Net2Cloud-Problem] describes a hybrid network approach that extends
the existing MPLS-based VPNs to the Cloud DC Workloads over the the existing MPLS-based VPNs to the Cloud DC Workloads over the
access paths that are not under the VPN provider's control. To make access paths that are not under the VPN provider's control. To make
it work properly, a small number of the PEs of the MPLS VPN can be it work properly, a small number of the PEs of the BGP/MPLS VPN can
designated to connect to the remote workloads via secure IPsec be designated to connect to the remote workloads via secure IPsec
tunnels. Those designated PEs are shown as fPE (floating PE or tunnels. Those designated PEs are shown as fPE (floating PE or
smart PE) in Figure 3. Once the secure IPsec tunnels are smart PE) in Figure 3. Once the secure IPsec tunnels are
established, the workloads hosted in Cloud DCs can be reached by the established, the workloads hosted in Cloud DCs can be reached by the
enterprise's VPN without upgrading all of the enterprise's existing enterprise's VPN without upgrading all of the enterprise's CPEs. The
CPEs. The only CPE that needs to support the Overlay would be a only CPE that needs to connect to the overlay network would be a
virtualized CPE instantiated within the cloud DC. virtualized CPE instantiated within the cloud DC.
+--------+ +--------+ +--------+ +--------+
| Host-a +--+ +----| Host-b | | Host-a +--+ +----| Host-b |
| | | (') | | | | | (') | |
+--------+ | +-----------+ ( ) +--------+ +--------+ | +-----------+ ( ) +--------+
| +-+--+ ++-+ ++-+ +--+-+ (_) | +-+--+ ++-+ ++-+ +--+-+ (_)
| | CPE|--|PE| |PE+--+ CPE| | | | CPE|--|PE| |PE+--+ CPE| |
+--| | | | | | | |---+ +--| | | | | | | |---+
+-+--+ ++-+ ++-+ +----+ +-+--+ ++-+ ++-+ +----+
skipping to change at page 12, line 38 skipping to change at page 13, line 38
----+-------+-------+----- ----+-------+-------+-----
| | | |
+---+----+ +---+----+ +---+----+ +---+----+
| Remote | | Remote | | Remote | | Remote |
| App-1 | | App-2 | | App-1 | | App-2 |
+--------+ +--------+ +--------+ +--------+
Figure 3: VPN Extension to Cloud DC Figure 3: VPN Extension to Cloud DC
In Figure 3, the optimal Cloud DC to host the workloads (as a In Figure 3, the optimal Cloud DC to host the workloads (as a
function of the proximity, capacity, pricing, or other criteria function of the proximity, capacity, pricing, or any other criteria
chosen by the enterprises) does not have a direct connection to the chosen by the enterprises) does not have a direct connection to the
PEs of the MPLS VPN that interconnects the enterprise's existing PEs of the NGP/MPLS VPN that interconnects the enterprise's sites.
sites.
6.1. Floating PEs to connect to Remote CPEs 6.1. Floating PEs to connect to Remote CPEs
To extend MPLS VPNs to remote CPEs, it is necessary to establish To extend BGP/MPLS VPNs to remote CPEs, it is necessary to establish
secure tunnels (such as IPsec tunnels) between the Floating PEs and secure tunnels (such as IPsec tunnels) between the Floating PEs and
the remote CPEs. the remote CPEs.
Even though a set of PEs can be manually selected to act as the Even though a set of PEs can be manually selected to act as the
floating PEs for a specific cloud data center, there are no standard floating PEs for a specific cloud data center, there are no standard
protocols for those PEs to interact with the remote CPEs (most protocols for those PEs to interact with the remote CPEs (most
likely virtualized) instantiated in the third party cloud data likely virtualized) instantiated in the third party cloud data
centers (such as exchanging performance or route information). centers (e.g., to exchange performance or route information).
When there is more than one fPE available for use (as there should When there is more than one fPE available for use (as there should
be for resiliency purposes or the ability to support multiple cloud be for resiliency purposes or because of the need to support
DCs geographically scattered), it is not straightforward to multiple cloud DCs geographically scattered), it is not
designate an egress fPE to remote CPEs based on applications. There straightforward to designate an egress fPE to remote CPEs based on
is too much applications' traffic traversing PEs, and it is not applications. There is too much applications' traffic traversing
feasible for PEs to recognize applications from the payload of PEs, and it is not feasible for PEs to recognize applications from
packets. the payload of packets.
6.2. NAT Traversal 6.2. NAT Traversal
Cloud DCs that only assign private IPv4 addresses to the Cloud DCs that only assign private IPv4 addresses to the
instantiated workloads assume that traffic to/from the workload instantiated workloads assume that traffic to/from the workload
usually needs to traverse NATs. usually needs to traverse NATs.
An overlay edge node can solicit a STUN (Session Traversal of UDP An overlay edge node can solicit a STUN (Session Traversal of UDP
Through Network Address Translation RFC 3489) Server to get the NAT Through Network Address Translation, [RFC3489]) Server to get the
property, the public IP address and the Public Port number so that information about the NAT property, the public IP addresses and port
such information can be communicated to the relevant peers. numbers so that such information can be communicated to the relevant
peers.
6.3. Complexity of using BGP between PEs and remote CPEs via Internet 6.3. Complexity of using BGP between PEs and remote CPEs via Internet
Even though an EBGP (external BGP) Multi-hop design can be used to Even though an EBGP (external BGP) Multi-Hop design can be used to
connect peers that are not directly connected to each other, there connect peers that are not directly connected to each other, there
are still some complications in extending BGP from MPLS VPN PEs to are still some issues about extending BGP from MPLS VPN PEs to
remote CPEs via any access path (e.g., Internet). remote CPEs via any access path (e.g., Internet).
The path between the remote CPEs and VPN PEs that maintain VPN The path between the remote CPEs and VPN PEs that maintain VPN
routes may very well traverse untrusted nodes. routes may very well traverse untrusted nodes.
EBGP Multi-hop design requires static configuration on both peers. EBGP Multi-hop design requires configuration on both peers, either
To use EBGP between a PE and remote CPEs, the PE has to be manually manually or via NETCONF from a controller. To use EBGP between a PE
configured with the "next-hop" set to the IP address of the CPEs. and remote CPEs, the PE has to be manually configured with the
When remote CPEs, especially remote virtualized CPEs are dynamically "next-hop" set to the IP address of the CPEs. When remote CPEs,
instantiated or removed, the configuration of Multi-Hop EBGP on the especially remote virtualized CPEs are dynamically instantiated or
PE has to be changed accordingly. removed, the configuration of Multi-Hop EBGP on the PE has to be
changed accordingly.
Egress peering engineering (EPE) is not sufficient. Running BGP on Egress peering engineering (EPE) is not sufficient. Running BGP on
virtualized CPEs in Cloud DCs requires GRE tunnels to be virtualized CPEs in Cloud DCs requires GRE tunnels to be
established first, which requires the remote CPEs to support established first, which requires the remote CPEs to support
address and key management capabilities. RFC 7024 (Virtual Hub & address and key management capabilities. RFC 7024 (Virtual Hub &
Spoke) and Hierarchical VPN do not support the required Spoke) and Hierarchical VPN do not support the required
properties. properties.
Also, there is a need for a mechanism to automatically trigger Also, there is a need for a mechanism to automatically trigger
configuration changes on PEs when remote CPEs' are instantiated or configuration changes on PEs when remote CPEs' are instantiated or
moved (leading to an IP address change) or deleted. moved (leading to an IP address change) or deleted.
EBGP Multi-hop design does not include a security mechanism by EBGP Multi-hop design does not include a security mechanism by
default. The PE and remote CPEs need secure communication channels default. The PE and remote CPEs need secure communication channels
when connecting via the public Internet. when connecting via the public Internet.
Remote CPEs, if instantiated in Cloud DCs, might have to traverse Remote CPEs, if instantiated in Cloud DCs might have to traverse
NATs to reach PEs. It is not clear how BGP can be used between NATs to reach PEs. It is not clear how BGP can be used between
devices located beyond the NAT and the devices located behind the devices located beyond the NAT and the devices located behind the
NAT. It is not clear how to configure the Next Hop on the PEs to NAT. It is not clear how to configure the Next Hop on the PEs to
reach private IPv4 addresses. reach private IPv4 addresses.
6.4. Designated Forwarder to the remote edges 6.4. Designated Forwarder to the remote edges
Among the multiple floating PEs that are reachable from a remote Among the multiple floating PEs that are reachable from a remote
CPE, multicast traffic sent by the remote CPE towards the MPLS VPN CPE, multicast traffic sent by the remote CPE towards the MPLS VPN
can be forwarded back to the remote CPE due to the PE receiving the can be forwarded back to the remote CPE due to the PE receiving the
multicast packets forwarding the multicast/broadcast frame to other multicast packets forwarding the multicast/broadcast frame to other
PEs that in turn send to all attached CPEs. This process may cause PEs that in turn send to all attached CPEs. This process may cause
traffic loops. traffic loops.
Therefore, it is necessary to designate one floating PE as the CPE's This problem can be solved by selecting one floating PE as the CPE's
Designated Forwarder, similar to TRILL's Appointed Forwarders Designated Forwarder, similar to TRILL's Appointed Forwarders
[RFC6325]. [RFC6325].
MPLS VPNs do not have features like TRILL's Appointed Forwarders. BGP/MPLS VPNs do not have features like TRILL's Appointed
Forwarders.
6.5. Traffic Path Management 6.5. Traffic Path Management
When there are multiple floating PEs that have established IPsec When there are multiple floating PEs that have established IPsec
tunnels with the remote CPE, the remote CPE can forward outbound tunnels with a remote CPE, the latter can forward outbound traffic
traffic to the Designated Forwarder PE, which in turn forwards to the Designated Forwarder PE, which in turn forwards traffic to
traffic to egress PEs and then to the final destinations. However, egress PEs and then to the final destinations. However, it is not
it is not straightforward for the egress PE to send back the return straightforward for the egress PE to send back the return traffic to
traffic to the Designated Forwarder PE. the Designated Forwarder PE.
Example of Return Path management using Figure 3 above. As Figure 3:
- fPE-1 is DF for communication between App-1 <-> Host-a due to - fPE-1 is DF for communication between App-1 <-> Host-a due to
latency, pricing or other criteria. latency, pricing or other criteria.
- fPE-2 is DF for communication between App-1 <-> Host-b. - fPE-2 is DF for communication between App-1 <-> Host-b.
7. Manageability Considerations 7. Manageability Considerations
Zero touch provisioning of Overlay networks to interconnect Hybrid Zero touch provisioning of overlay networks to interconnect Hybrid
Clouds is highly desired. It is necessary for a newly powered up Clouds is highly desired. It is necessary for a newly powered up
edge node to establish a secure connection (by means of TLS, DTLS, edge node to establish a secure connection (by means of TLS, DTLS,
etc.) with its controller. etc.) with its controller.
8. Security Considerations 8. Security Considerations
Cloud Services is built upon shared infrastructure, therefore not Cloud Services are built upon shared infrastructures, therefore
secure by nature. not secure by nature.
Secure user identity management, authentication, and access Secure user identity management, authentication, and access
control mechanisms are important. Developing appropriate security control mechanisms are important. Developing appropriate security
measurements can enhance the confidence needed by enterprises to measurements can enhance the confidence needed by enterprises to
fully take advantage of Cloud Services. fully take advantage of Cloud Services.
9. IANA Considerations 9. IANA Considerations
This document requires no IANA actions. RFC Editor: Please remove This document requires no IANA actions. RFC Editor: Please remove
this section before publication. this section before publication.
 End of changes. 76 change blocks. 
208 lines changed or deleted 241 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/