< draft-ietf-rtgwg-net2cloud-gap-analysis-00.txt   draft-ietf-rtgwg-net2cloud-gap-analysis-01.txt >
Network Working Group L. Dunbar Network Working Group L. Dunbar
Internet Draft A. Malis Internet Draft A. Malis
Intended status: Informational Huawei Intended status: Informational Huawei
Expires: August 6, 2019 C. Jacquenet Expires: September 25, 2019 C.
Jacquenet
Orange Orange
February 12, 2019 March 25, 2019
Gap Analysis of Interconnecting Underlay with Cloud Overlay Gap Analysis of Interconnecting Underlay with Cloud Overlay
draft-ietf-rtgwg-net2cloud-gap-analysis-00 draft-ietf-rtgwg-net2cloud-gap-analysis-01
Abstract Abstract
This document analyzes the technological gaps when using SD-WAN to This document analyzes the technological gaps when using SD-WAN to
interconnect workloads & apps hosted in various locations, interconnect workloads & apps hosted in various locations,
especially cloud data centers when the network service providers do especially cloud data centers when the network service providers do
not have or have limited physical infrastructure to reach the not have or have limited physical infrastructure to reach the
locations [Net2Cloud-problem]. locations [Net2Cloud-problem].
Status of this Memo Status of this Memo
skipping to change at page 1, line 41 skipping to change at page 1, line 42
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress." reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on July 6, 2019. This Internet-Draft will expire on September 25, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................2 1. Introduction...................................................2
2. Conventions used in this document..............................3 2. Conventions used in this document..............................3
3. Gap Analysis of C-PEs Registration Protocol....................4 3. Gap Analysis of C-PEs WAN Ports Registration...................4
4. Gap Analysis in aggregating VPN paths and Internet paths.......5 4. Gap Analysis in aggregating VPN paths and Internet paths.......5
4.1. Gap analysis of Using BGP to cover SD-WAN paths...........6 4.1. Gap analysis of Using BGP for SD-WAN......................6
4.2. Gaps in preventing attacks from Internet facing ports.....9 4.2. Gaps in preventing attacks from Internet-facing ports.....9
5. Gap analysis of CPEs not directly connected to VPN PEs........10 5. Gap analysis of CPEs not directly connected to VPN PEs........10
5.1. Gap Analysis of Floating PEs to connect to Remote CPEs...11 5.1. Gap Analysis of Floating PEs to connect to Remote CPEs...12
5.2. NAT Traversal............................................12 5.2. NAT Traversal............................................12
5.3. Complication of using BGP between PEs and remote CPEs via 5.3. Complication of using BGP between PEs and remote CPEs via
Internet......................................................12 Internet......................................................12
5.4. Designated Forwarder to the remote edges.................13 5.4. Designated Forwarder to the remote edges.................13
5.5. Traffic Path Management..................................13 5.5. Traffic Path Management..................................14
6. Manageability Considerations..................................14 6. Manageability Considerations..................................14
7. Security Considerations.......................................14 7. Security Considerations.......................................14
8. IANA Considerations...........................................14 8. IANA Considerations...........................................15
9. References....................................................14 9. References....................................................15
9.1. Normative References.....................................15 9.1. Normative References.....................................15
9.2. Informative References...................................15 9.2. Informative References...................................15
10. Acknowledgments..............................................16 10. Acknowledgments..............................................16
1. Introduction 1. Introduction
[Net2Cloud-Problem] describes the problems that enterprises face [Net2Cloud-Problem] describes the problems that enterprises face
today in transitioning their IT infrastructure to support digital today in transitioning their IT infrastructure to support digital
economy, such as connecting enterprises' branch offices to dynamic economy, such as connecting enterprises' branch offices to dynamic
workloads in different Cloud DCs. workloads in different Cloud DCs.
This document analyzes the technological gaps to interconnect This document analyzes the technological gaps to interconnect
dynamic workloads & apps hosted in various locations and in Cloud dynamic workloads & apps hosted in various locations and in Cloud
DCs that the enterprise existing VPN service provider might not have DCs that the enterprise's VPN service provider may not own/operate
or have limited the physical infrastructure to reach. When or may be unable to provide the required connectivity to access
enterprise' VPN service providers do not have or have insufficient these locations. When enterprise' VPN service providers have
bandwidth to reach a location, SD-WAN is emerged as way to aggregate insufficient bandwidth to reach a location, SD-WAN techniques can be
bandwidth of multiple networks, such as MPLS VPN, Public Internet, used to aggregate bandwidth of multiple networks, such as MPLS VPNs,
etc. This document primarily focuses on the technological gaps of the Public Internet, to achieve better performance and visibility.
SD-WAN. This document primarily focuses on the technological gaps of SD-WAN.
For ease of description, SD-WAN edge, SD-WAN end points, C-PE, or For ease of description, a SD-WAN edge, a SD-WAN end-point, C-PE, or
CPE are used interchangeably throughout this document. CPE are used interchangeably throughout this document.
2. Conventions used in this document 2. Conventions used in this document
Cloud DC: Third party Data Centers that usually host applications Cloud DC: Third party Data Centers that usually host applications
and workload owned by different organizations or and workload owned by different organizations or
tenants. tenants.
Controller: Used interchangeably with SD-WAN controller to manage Controller: Used interchangeably with SD-WAN controller to manage
SD-WAN overlay path creation/deletion and monitor the SD-WAN overlay path creation/deletion and monitor the
path conditions between sites. path conditions between sites.
CPE-Based VPN: Virtual Private Secure network formed among CPEs. CPE-Based VPN: Virtual Private Network designed and deployed from
This is to differentiate from most commonly used PE- CPEs. This is to differentiate from most commonly used
based VPNs a la RFC 4364. PE-based VPNs a la RFC 4364.
OnPrem: On Premises data centers and branch offices OnPrem: On Premises data centers and branch offices
SD-WAN: Software Defined Wide Area Network. In this document, SD-WAN: Software Defined Wide Area Network, "SDWAN" refers to
"SD-WAN" refers to the solutions specified by ONUG (Open the solutions of pooling WAN bandwidth from multiple
Network User Group), https://www.onug.net/software- underlay networks to get better WAN bandwidth
defined-wide-area-network-sd-wan/, which is about management, visibility & control. When the underlay is
pooling WAN bandwidth from multiple underlay networks to private network, traffic can traverse without additional
get better WAN bandwidth management, visibility &
control. When the underlay networks are private
networks, traffic can traverse without additional
encryption; when the underlay networks are public, such encryption; when the underlay networks are public, such
as Internet, some traffic needs to be encrypted when as the Internet, some traffic needs to be encrypted when
traversing through (depending on user provided traversing through (depending on user-provided
policies). policies).
3. Gap Analysis of C-PEs Registration Protocol 3. Gap Analysis of C-PEs WAN Ports Registration
SD-WAN, conceived in ONUG (Open Network User Group) a few years ago The SD-WAN WG stemmed out from ONUG (Open Network User Group) in
as a means to aggregate multiple connections between any two points, 2014 and was the placeholder to define SD-WAN as a means to
has emerged as an on-demand technology to securely interconnect the aggregate multiple underlay networks between any two points. SD-WAN
OnPrem branches with the workloads instantiated in Cloud DCs that do technology has emerged as an on-demand technology to securely
not connect to BGP/MPLS VPN PEs or have very limited bandwidth. interconnect the OnPrem branches with the workloads instantiated in
Cloud DCs that do not connect to BGP/MPLS VPN PEs or have very
limited bandwidth.
Some SD-WAN networks use the NHRP protocol [RFC2332] to register SD- Some SD-WAN networks use the NHRP protocol [RFC2332] to register WAN
WAN edges with a "Controller" (or NHRP server), which then has the ports of SD-WAN edges with a "Controller" (or NHRP server), which
ability to map a private VPN address to a public IP address of the then has the ability to map a private VPN address to a public IP
destination node. DSVPN [DSVPN] or DMVPN [DMVPN] are used to address of the destination node. DSVPN [DSVPN] or DMVPN [DMVPN] are
establish tunnels among SD-WAN edge nodes. used to establish tunnels between WAN ports of SD-WAN edge nodes.
NHRP was originally intended for ATM address resolution, and as a NHRP was originally intended for ATM address resolution, and as a
result, it misses many attributes that are necessary for dynamic result, it misses many attributes that are necessary for dynamic
endpoint C-PE registration to controller, such as: endpoint C-PE registration to controller, such as:
- Interworking with MPLS VPN control plane. A SD-WAN edge can have - Interworking with MPLS VPN control plane. A SD-WAN edge can have
some ports facing MPLS VPN network and some ports facing public some ports facing MPLS VPN network over which packets can be sent
Internet that requires encryption for some sensitive data to natively without encryption and some ports facing the public
traverse. Internet over which sensitive traffic needs to be encrypted before
- Scalability. NHRP/DSVPN/DMVPN works fine with small number of edge being sent.
nodes. When a network has more than 100 nodes, the protocol does - Scalability. NHRP/DSVPN/DMVPN works fine with small numbers of
not work well. edge nodes. When a network has more than 100 nodes, the protocol
- NHRP does not have the IPsec attributes, which are needed for peers to does not work well.
build Security Associations over public internet. - NHRP does not have the IPsec attributes, which are needed for
- NHRP does not have field to indicate C-PE supported encapsulation types, peers to build Security Associations over public internet.
such as IPsec-GRE, IPsec-VxLAN, or others. - NHRP messages do not have any field to encode the C-PE supported
- NHRP does not have field to indicate C-PE Location identifier, such as encapsulation types, such as IPsec-GRE or IPsec-VxLAN,.
Site Identifier, System ID, and/or Port ID. - NHRP messages do not have any field to encode C-PE Location
- NHRP does not have field to describe the gateway to which the C-PE identifiers, such as Site Identifier, System ID, and/or Port ID.
is attached. When a C-PE is instantiated in a Cloud DC, to - NHRP messages do not have any field to describe the gateway(s) to
establish connection to the C-PE, it is necessary to know the which the C-PE is attached. When a C-PE is instantiated in a Cloud
Cloud DC operator's Gateway to which the CPE is attached. DC, to establish connection to the C-PE, it is necessary to know
- NHRP does not have field to describe C-PE's NAT properties if the the Cloud DC operator's Gateway to which the CPE is attached.
C-PE is using private addresses, such as the NAT type, Private - NHRP messages do not have any field to describe C-PE's NAT
address, Public address, Private port, Public port, etc. properties if the C-PE is using private addresses, such as the NAT
type, Private address, Public address, Private port, Public port,
etc.
[BGP-SDWAN-EXT] describes how to use BGP for SD-WAN edge nodes to [BGP-SDWAN-PORT] describes how to use BGP for SD-WAN edge nodes to
register its properties to SD-WAN controller, which then register their WAN ports properties to the SD-WAN controller, which
disseminates the information to other SD-WAN edge nodes that are then disseminates the information to other SD-WAN edge nodes that
authenticated to communicate. are authenticated before the SD-WAN controller and the other SD-WAN
edge nodes can communicate with them.
4. Gap Analysis in aggregating VPN paths and Internet paths 4. Gap Analysis in aggregating VPN paths and Internet paths
Most likely, enterprises especially large ones already have their Most likely, enterprises (especially the largest ones) already have
CPEs interconnected by providers' VPNs, such as EVPN, L2VPN, or their CPEs interconnected by providers' VPNs, based upon VPN
L3VPN. The VPN can be PE based or CPE based as shown in the techniques such as EVPN, L2VPN, or L3VPN. The VPN can be PE-based or
following diagram. The commonly used CPE-based VPNs have CPE CPE-based. The commonly used PE-based VPNs have CPE directly
directly attached to PEs, therefore the communication is considered attached to PEs, therefore the communication between CPEs & PEs is
as secure. BGP are used to distribute routes among CPEs, even though considered as secure. MP-BGP is used to learn & distribute routes
sometimes routes among CPEs are statically configured. among CPEs, even though sometimes routes among CPEs are statically
+---+ EVPN MAC/IP BGP updates configured.
+======+RR +===========+
// +---+ \\
// \\
// <-----EVPN-VxLAN------> \\
+-+--+ ++-+ ++-+ +--+-+
| CPE|--|PE| |PE+----+ CPE|
| 1 | |1 | |x | | x |
+-+--+ ++-+ ++-+ +----+
| |
| VPN +-+---+ +----+
| Network | PE3 | |CPE |
| | |- --| 3 |
+--------+ +--+--+ +-+---+ +----+
| CPE +----+ PE4 |--------+
| 4 | +---+-+
+--------+
=== or \\ indicates control plane communications
Figure 1: L2 or L3 VPNs over IP WAN
To use SD-WAN to aggregate Internet routes with the VPN routes, the To aggregate paths over the Internet and paths over the VPN, the C-
C-PEs need to have some ports connected to PEs and other ports PEs need to have some WAN ports connected to the PEs of the VPNs and
connected to the Internet. It is necessary to have a registration other WAN ports connected to the Internet. It is necessary for the
protocol for C-PEs to register with their SD-WAN Controllers to CPEs to use a protocol so that they can register the WAN port
establish secure tunnels among relevant C-PEs. properties with their SD-WAN Controller(s): this information
conditions the establishment and the maintenance of IPsec SA
associations among relevant C-PEs.
If using NHRP for registration, C-PEs need to participate in two If using NHRP for registration purposes, C-PEs need to participate
separate control planes: EVPN&BGP for CPE-based VPNs via links in two separate control planes: EVPN&BGP for CPE-based VPNs and NHRP
directly attached to PEs and NHRP & DSVPN/DMVPN for ports connected & DSVPN/DMVPN for ports connected to the Internet. Two separate
to internet. Two separate control planes not only add complexity to control planes not only add complexity to C-PEs, but also increase
C-PEs, but also increase operational cost. operational cost.
+---------Internet paths--------------+ +---+
| | +--------------|RR |----------+
| +---+ | / Untrusted +-+-+ \
| |RR | | / \
| +======+---+===========+ | / \
| // \\ | +----+ +---------+ packets encrypted over +------+ +----+
| // <-----EVPN-VxLAN----> \\ | | TN3|--| A1-----+ Untrusted +------ B1 |--| TN1|
| +-+--+ ++-+ ++-+ +--+-+ (|) +----+ | C-PE A2-\ | C-PE | +----+
| | CPE|--|PE| |PE+--+ CPE| (|) +----+ | A A3--+--+ +---+---B2 B | +----+
+--| 1 | |1 | |x | | c |---+ | TN2|--| | |PE+--------------+PE |---B3 |--| TN3|
+-+--+ ++-+ ++-+ +----+ +----+ +---------+ +--+ trusted +---+ +------+ +----+
| | | WAN |
| VPN +-+---+ +----+ +----+ +---------+ +--+ packets +---+ +------+ +----+
+--------+ | Network | PE3 | |CPE | | TN1|--| C1--|PE| go natively |PE |-- D1 |--| TN1|
| CPE | | | |- --| 3 | +----+ | C-PE C2--+--+ without encry+---+ | C-PE | +----+
| c | +-----+ +-+---+ +----+ | C | +--------------+ | D |
+------+-+-------+ PE4 |-----+ | | | |
+---+-+ +----+ | C3--| without encrypt over | | +----+
Figure 2: CPEs interconnected by VPN paths and Internet Paths | TN2|--| C4--+---- Untrusted --+------D2 |--| TN2|
+----+ +---------+ +------+ +----+
Figure 1: CPEs interconnected by VPN paths and Internet Paths
4.1. Gap analysis of Using BGP to cover SD-WAN paths 4.1. Gap analysis of Using BGP for SD-WAN
Since C-PE already supports BGP, it is desirable to consider using
BGP to control the SD-WAN instead of two separate control planes.
This section analyzes the gaps of using BGP to control SD-WAN. This section analyzes the gaps of using BGP to control SD-WAN.
As described [BGP-SDWAN-Usage], SD-WAN Overlay Control Plane has As described in [BGP-SDWAN-Usage], SD-WAN Overlay Control Plane has
three distinct functional tiers: three distinct aspects:
- SD-WAN node's private address and WAN Ports/Addresses
registration to the SD-WAN Controller.
o It is for informing the SD-WAN controller and potential - SD-WAN node's WAN Ports Property registration to the SD-WAN
peers of the underlay networks to which the C-PE is Controller.
connected. o It is to inform the SD-WAN controller and potential peers
of the WAN ports property of the C-PE [SDWAN-Port]. When
the WAN ports are using private addresses, this step can
register the type of NAT that translate private addresses
into public ones.
- Controller Facilitated IPsec SA management and NAT information - Controller Facilitated IPsec SA management and NAT information
distribution distribution
o It is for SD-WAN controller to facilitate or manage the o It is for SD-WAN controller to facilitate or manage the
IPsec configurations and peer authentications for all IPsec configuration and peer authentication for all IPsec
IPsec tunnels terminated at the SDWAN nodes. For some tunnels terminated at the SDWAN nodes.
scenarios where the WAN ports are private addresses, this
step is for informing the type of NAT translating the
private addresses to public ones.
- Establishing and Managing the topology and reachability for - Establishing and Managing the topology and reachability for
services attached to the client ports of SD-WAN nodes. services attached to the client ports of SD-WAN nodes.
o This is for the overlay layer's routes distribution, so o This is for the overlay layer's route distribution, so
that a C-PE can establish the overlay routing table that that a C-PE can populate its overlay routing table with
identifies the next hop for reaching a specific entries that identify the next hop for reaching a specific
route/service attached to remote nodes. [SECURE-EVPN] route/service attached to remote nodes. [SECURE-EVPN]
describes EVPN and other options. describes EVPN and other options.
RFC5512 and [Tunnel-Encap] describe methods for endpoints to RFC5512 and [Tunnel-Encap] describe methods for endpoints to
advertise tunnel information and to trigger tunnel establishment. advertise tunnel information and trigger tunnel establishment.
RFC5512 & [Tunnel-Encap] have the Endpoint Address to indicate IPv4 RFC5512 & [Tunnel-Encap] use the Endpoint Address that indicates an
or IPv6 address format, the Tunnel Encapsulation attribute to IPv4 or an IPv6 address, and the Tunnel Encapsulation attribute to
indicate different encapsulation formats, such as L2TPv3, GRE, indicate different encapsulation formats, such as L2TPv3, GRE,
VxLAN, IP-in-IP, etc. There are sub-TLVs to describe the detailed VxLAN, IP-in-IP, etc. There are sub-TLVs to describe the detailed
tunnel information for each of the encapsulations. tunnel information for each of the encapsulation types.
[Tunnel-Encap] removed SAFI =7 (which was specified by RFC5512) for [Tunnel-Encap] removed SAFI =7 (which was specified by RFC5512) for
distributing encapsulation tunnel information. [Tunnel-Encap] distributing encapsulation tunnel information. [Tunnel-Encap]
require Tunnels being associated with routes. requires that tunnels need to be associated with routes.
There is also the Color sub-TLV to describe customer-specified There is also the Color sub-TLV to describe customer-specified
information about the tunnels (which can be creatively used for SD- information about the tunnels (which can be creatively used for SD-
WAN) WAN).
Here are some of the gaps using [Tunnel-Encap] to control SD-WAN: Here are some of the gaps using [Tunnel-Encap] to control SD-WAN:
- Lacking C-PE Registration functionality - Lacking C-PE WAN Port Property Registration functionality
- Lacking IPsec Tunnel type - Lacking IPsec Tunnel type
- [Tunnel-Encap] has Remote Address SubTLV, but does not have any - [Tunnel-Encap] has Remote Address SubTLV, but does not have any
field to indicate the Tunnel originating interface, which was in field to indicate the Tunnel originating interface, as defined in
RFC5512. RFC5512.
- The mechanisms described by [Tunnel-Encap] cannot be effectively - The mechanisms described by [Tunnel-Encap] cannot be effectively
used for SD-WAN overlay network because a SD-WAN Tunnel can be used for SD-WAN overlay network because a SD-WAN Tunnel can be
between Internet facing WAN ports of two C-PEs and needs to be established between Internet-facing WAN ports of two C-Pes. This
established before data arrival because the tunnel establishment tunnel needs to be established before data arrival because the
can fail, e.g. two end points supporting different encryption tunnel establishment can fail, e.g., in case the two end-points
algorithms. support different encryption algorithms.
- Client traffic (e.g. an EVPN route) can have option of going - Client traffic can either be forwarded through the MPLS network
through MPLS network natively without encryption, or going through natively without any encryption for better performance, or through
the IPsec tunnels between the internet facing WAN ports of two C- the Internet-facing ports with IPsec encryption.
PEs. - There can be many client routes associated with the SD-WAN IPsec
- There is no routes to be associated with the SD-WAN Tunnel between tunnel between two C-PE's Internet-facing WAN ports, but the
two C-PE's internet facing WAN ports, unless consider using the corresponding destination prefixes (as announced by the
interface facing WAN Port addresses assigned by ISP (Internet aforementioned routes) can also be reached over the VPN underlay
Service Providers) as the route for the Tunnel. natively without encryption. A more realistic approach to separate
IPsec SA management from client routes association with IPsec.
There is a suggestion on using a "Fake Route" for a SD-WAN node to There is a suggestion on using a "Fake Route" for a SD-WAN node to
use [Tunnel-Encap] to advertise its SD-WAN tunnel end-points use [Tunnel-Encap] to advertise its SD-WAN tunnel end-points
properties. However, using "Fake Route" can create deployment properties. However, using "Fake Route" can raise some design
complexity for large SD-WAN networks with many tunnels. For complexity for large SD-WAN networks with many tunnels. For
example, for a SD-WAN network with hundreds of nodes, with each example, for a SD-WAN network with hundreds of nodes, with each
node having many ports & many end-points to establish SD-WAN node having many ports & many end-points to establish SD-WAN
tunnels to their corresponding peers, the node would need many tunnels with their corresponding peers, the node would need as
"fake addresses". For large SD-WAN networks (such as has more than many "fake addresses". For large SD-WAN networks (such as those
10000 nodes), each node might need 10's thousands of "fake comprised of more than 10000 nodes), each node might need 10's
addresses", which is very difficult to manage and needs lots of thousands of "fake addresses", which is very difficult to manage
configuration to get the nodes provisioned. and needs lots of configuration to get the nodes provisioned.
- Does not have fields to carry detailed information of the remote - Does not have any field to carry detailed information about the
C-PE: such as Site-ID, System-ID, Port-ID remote C-PE, such as Site-ID, System-ID, Port-ID
- Does not have the proper field to express IPsec attributes among - Does not have any field to express IPsec attributes for the SD-WAN
the SD-WAN edge nodes to establish proper IPsec Security edge nodes to establish IPsec Security Associations with others.
Associations. - Does not have any proper way for two peer CPEs to negotiate IPsec
- Does not have proper way for two peer CPEs to negotiate IPSec
keys, based on the configuration sent by the Controller. keys, based on the configuration sent by the Controller.
- Does not have field to indicate the UDP NAT private address <-> - Does not have any field to indicate the UDP NAT private address
public address mapping <-> public address mapping
- C-PEs tend to communicate with a few other CPEs, not all the C-PEs - C-PEs tend to communicate with a subset of the other C-PEs, not
need to form mesh connections. Without any BGP extension, many all the C-PEs need to form mesh connections. Without any BGP
nodes can get dumped with too much information of other nodes that extension, many nodes can get dumped with too much information
they never need to communicate with. coming from other nodes that they never need to communicate with.
[VPN-over-Internet] describes a way to securely interconnect C-PEs [SECURE-L3VPN] describes how to extend the RFC4364 VPN to allow some
via IPsec using BGP. This method is useful, however, it still misses PEs to connect to other PEs via public networks. [SECURE-L3VPN]
some aspects to aggregate CPE-based VPN routes with internet routes introduces the concept of Red Interface & Black Interface on those
that interconnect the CPEs. In addition: PEs, where the RED interfaces are used to forward traffic into the
VPN, and the Black Interfaces are used between WAN ports over which
only IPsec-protected packets to the Internet or other backbone
network are sent thereby eliminating the need for MPLS transport in
the backbone.
- The draft does not have options of C-PE having both MPLS ports and [SECURE-L3VPN] assumes PEs terminate MPLS packets, and use MPLS over
Internet ports. IPsec when sending traffic through the Black Interfaces.
- The draft assumes that CPE "registers" with the RR. However, it does not
say how. It assumes that the remote CPEs are pre-configured with the
IPsec SA manually. In SD-WAN, Zero Touch Provisioning is expected. It is
not acceptable to require manual configuration.
- For RR communication with CPE, this draft only mentioned IPSec. Missing
TLS/DTLS.
- The draft assumes that CPEs and RR are connected with an IPsec tunnel.
With zero touch provisioning, we need an automatic way to synchronize
the IPsec SA between CPE and RR. The draft assumes:
A CPE must also be provisioned with whatever additional information
is needed in order to set up an IPsec SA with each of the red RRs
- IPsec requires periodic refreshment of the keys. The draft hasn't [SECURE-EVPN] describes a solution where point-to-multipoint BGP
addressed how to synchronize the refreshment among multiple nodes. signaling is used in the control plane for SDWAN Scenario #1. It
- IPsec usually only sends configuration parameters to two endpoints relies upon a BGP cluster design to facilitate the key and policy
and let the two endpoints negotiate the KEY. Now we assume that RR exchange among PE devices to create private pair-wise IPsec Security
is responsible for creating the KEY for all endpoints. When one Associations without IKEv2 point-to-point signaling or any other
endpoint is compromised, all other connections are impacted. direct peer-to-peer session establishment messages.
4.2. Gaps in preventing attacks from Internet facing ports Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both
miss the aspects of aggregating VPN and Internet underlays. In
summary:
When C-PEs have ports facing Internet, it brings in the security - These documents do not address the scenario of C-PE having some
risks of potential DDoS attacks to the C-PEs from the ports facing ports facing VPN PEs and other ports facing the Internet.
internet. -
- The [SECURE-L3VPN] assumes that CPE "registers" with the RR.
However, it does not say how. It assumes that the remote CPEs are
pre-configured with the IPsec SA manually. In SD-WAN, Zero Touch
Provisioning is expected. Manual configuration is not an option,
given the dimensioning figures but also the purpose of SD-WAN to
automate configuration tasks.
- For RR communication with CPEs, this draft only mentions IPsec.
Missing TLS/DTLS.
- The draft assumes that CPEs and RR are connected with an IPsec
tunnel. With zero touch provisioning, we need an automatic way to
synchronize the IPsec SAs between CPE and RR. The draft assumes:
A CPE must also be provisioned with whatever additional
information is needed in order to set up an IPsec SA with
each of the red RRs
To mitigate security risks, in addition to requring Anti-DDoS - IPsec requires periodic refreshment of the keys. The draft does
features on C-PEs to prevent major DDoS attacks, it is necessary to not provide any information about how to synchronize the
have ways for C-PEs to validate traffic from remote peers to prevent refreshment among multiple nodes.
spoofed traffic. - IPsec usually sends configuration parameters to two endpoints only
and lets these endpoints negotiate the key. Let us assume that the
RR is responsible for creating the key for all endpoints: When one
endpoint is compromised, all other connections will be impacted.
4.2. Gaps in preventing attacks from Internet-facing ports
When C-PEs have Internet-facing ports, additional security risks are
raised.
To mitigate security risks, in addition to requiring Anti-DDoS
features on C-PEs, it is necessary for CPEs to support means to
determine whether traffic sent by remote peers is legitimate to
prevent spoofing attacks.
5. Gap analysis of CPEs not directly connected to VPN PEs 5. Gap analysis of CPEs not directly connected to VPN PEs
Because of the ephemeral property of the selected Cloud DCs, an Because of the ephemeral property of the selected Cloud DCs, an
enterprise or its network service provider may not have the direct enterprise or its network service provider may not have direct
links to the Cloud DCs that are optimal for hosting the enterprise's connections to the Cloud DCs that are used for hosting the
specific workloads/Apps. Under those circumstances, SD-WAN is a very enterprise's specific workloads/Apps. Under those circumstances, SD-
flexible choice to interconnect the enterprise on-premises data WAN is a very flexible choice to interconnect the enterprise on-
centers & branch offices to its desired Cloud DCs. premises data centers & branch offices to its desired Cloud DCs.
However, SD-WAN paths over public Internet can have unpredictable However, SD-WAN paths over public Internet can have unpredictable
performance, especially over long distances and across domains. performance, especially over long distances and across domains.
Therefore, it is highly desirable to place as much as possible the Therefore, it is highly desirable to place as much as possible the
portion of SD-WAN paths over service provider VPN (e.g., portion of SD-WAN paths over service provider VPN (e.g.,
enterprise's existing VPN) that have guaranteed SLA to minimize the enterprise's existing VPN) that have guaranteed SLA to minimize the
distance/segments over public Internet. distance/segments over public Internet.
MEF Cloud Service Architecture [MEF-Cloud] also describes a use case MEF Cloud Service Architecture [MEF-Cloud] also describes a use case
of network operators needing to use SD-WAN over LTE or public of network operators that use SD-WAN over LTE or the public
Internet for last mile accesses where they are not present. Internet for last mile access where the VPN providers cannot
necessarily provide the required physical infrastructure.
Under those scenarios, one or both of the SD-WAN endpoints may not Under those scenarios, one or two of the SD-WAN endpoints may not be
directly be attached to the PEs of a VPN Domain. directly attached to the PEs of a VPN Domain.
Using SD-WAN to connect the enterprise existing sites with the When using SD-WAN to connect the enterprise's existing sites with
workloads in Cloud DC, the enterprise existing sites' CPEs have to the workloads in Cloud DC, the corresponding CPEs have to be
be upgraded to support SD-WAN. If the workloads in Cloud DC need to upgraded to support SD-WAN. If the workloads in Cloud DCs need to
be connected to many sites, the upgrade process can be very be connected to many sites, the upgrade process can be very
expensive. expensive.
[Net2Cloud-Problem] describes a hybrid network approach that [Net2Cloud-Problem] describes a hybrid network approach that
integrates SD-WAN with traditional MPLS-based VPNs, to extend the integrates SD-WAN with traditional MPLS-based VPNs, to extend the
existing MPLS-based VPNs to the Cloud DC Workloads over the access existing MPLS-based VPNs to the Cloud DC Workloads over the access
paths that are not under the VPN provider control. To make it work paths that are not under the VPN provider's control. To make it work
properly, a small number of the PEs of the MPLS VPN can be properly, a small number of the PEs of the MPLS VPN can be
designated to connect to the remote workloads via SD-WAN secure designated to connect to the remote workloads via SD-WAN secure
IPsec tunnels. Those designated PEs are shown as fPE (floating PE IPsec tunnels. Those designated PEs are shown as fPE (floating PE
or smart PE) in Figure 3. Once the secure IPsec tunnels are or smart PE) in Figure 3. Once the secure IPsec tunnels are
established, the workloads in Cloud DC can be reached by the established, the workloads in Cloud DC can be reached by the
enterprise's VPN without upgrading all of the enterprise's existing enterprise's VPN without upgrading all of the enterprise's existing
CPEs. The only CPE that needs to support SD-WAN would be a CPEs. The only CPE that needs to support SD-WAN would be a
virtualized CPE instantiated within the cloud DC. virtualized CPE instantiated within the cloud DC.
+--------+ +--------+ +--------+ +--------+
skipping to change at page 11, line 39 skipping to change at page 12, line 7
| | | |
+---+----+ +---+----+ +---+----+ +---+----+
| Remote | | Remote | | Remote | | Remote |
| App-1 | | App-2 | | App-1 | | App-2 |
+--------+ +--------+ +--------+ +--------+
Figure 3: VPN Extension to Cloud DC Figure 3: VPN Extension to Cloud DC
In Figure 3, the optimal Cloud DC to host the workloads (due to In Figure 3, the optimal Cloud DC to host the workloads (due to
proximity, capacity, pricing, or other criteria chosen by the proximity, capacity, pricing, or other criteria chosen by the
enterprises) does not happen to have a direct connection to the PEs enterprises) does not have a direct connection to the PEs of the
of the MPLS VPN that interconnects the enterprise's existing sites. MPLS VPN that interconnects the enterprise's existing sites.
5.1. Gap Analysis of Floating PEs to connect to Remote CPEs 5.1. Gap Analysis of Floating PEs to connect to Remote CPEs
To extend MPLS VPNs to remote CPEs, it is necessary to establish To extend MPLS VPNs to remote CPEs, it is necessary to establish
secure tunnels (such as IPsec tunnels) between the Floating PEs and secure tunnels (such as IPsec tunnels) between the Floating PEs and
the remote CPEs. the remote CPEs.
Gap: Gap:
Even though a set of PEs can be manually selected to act as the Even though a set of PEs can be manually selected to act as the
floating PEs for a specific cloud data center, there are no standard floating PEs for a specific cloud data center, there are no standard
protocols for those PEs to interact with the remote CPEs (most protocols for those PEs to interact with the remote CPEs (most
likely virtualized) instantiated in the third party cloud data likely virtualized) instantiated in the third party cloud data
centers (such as exchanging performance or route information). centers (such as exchanging performance or route information).
When there is more than one fPE available for use (as there should When there is more than one fPE available for use (as there should
be for resiliency or the ability to support multiple cloud DCs be for resiliency or the ability to support multiple cloud DCs
scattered geographically), it is not straightforward to designate an geographically scattered), it is not straightforward to designate an
egress fPE to remote CPEs based on applications. There is too much egress fPE to remote CPEs based on applications. There is too much
applications' traffic traversing PEs, and it is not feasible for PEs applications' traffic traversing PEs, and it is not feasible for PEs
to recognize applications from the payload of packets. to recognize applications from the payload of packets.
5.2. NAT Traversal 5.2. NAT Traversal
Most cloud DCs only assign private addresses to the workloads Most cloud DCs only assign private addresses to the instantiated
instantiated. Therefore, traffic to/from the workload usually need workloads. Therefore, traffic to/from the workload usually needs to
to traverse NAT. traverse NATs.
A SD-WAN edge node can inquire STUN (Session Traversal of UDP A SD-WAN edge node can solicit a STUN (Session Traversal of UDP
Through Network Address Translation RFC 3489) Server to get the NAT Through Network Address Translation RFC 3489) Server to get the NAT
property, the public IP address and the Public Port number to pass property, the public IP address and the Public Port number to pass
to peers. to peers.
5.3. Complication of using BGP between PEs and remote CPEs via Internet 5.3. Complication of using BGP between PEs and remote CPEs via Internet
Even though an EBGP (external BGP) Multi-hop design can be used to Even though an EBGP (external BGP) Multi-hop design can be used to
connect peers that are not directly connected to each other, there connect peers that are not directly connected to each other, there
are still some complications/gaps in extending BGP from MPLS VPN PEs are still some complications/gaps in extending BGP from MPLS VPN PEs
to remote CPEs via any access paths (e.g., Internet). to remote CPEs via any access paths (e.g., Internet).
The path between the remote CPEs and VPN PE can traverse untrusted The path between the remote CPEs and VPN PE can traverse untrusted
nodes. nodes.
EBGP Multi-hop scheme requires static configuration on both peers. EBGP Multi-hop design requires static configuration on both peers.
To use EBGP between a PE and remote CPEs, the PE has to be manually To use EBGP between a PE and remote CPEs, the PE has to be manually
configured with the "next-hop" set to the IP addresses of the CPEs. configured with the "next-hop" set to the IP address of the CPEs.
When remote CPEs, especially remote virtualized CPEs are dynamically When remote CPEs, especially remote virtualized CPEs are dynamically
instantiated or removed, the configuration on the PE Multi-Hop EBGP instantiated or removed, the configuration of Multi-Hop EBGP on the
has to be changed accordingly. PE has to be changed accordingly.
Gap: Gap:
Egress peering engineering (EPE) is not enough. Running BGP on Egress peering engineering (EPE) is not enough. Running BGP on
virtualized CPEs in Cloud DC requires GRE tunnels being virtualized CPEs in Cloud DC requires GRE tunnels being
established first, which in turn requires address and key established first, which in turn requires address and key
management for the remote CPEs. RFC 7024 (Virtual Hub & Spoke) and management for the remote CPEs. RFC 7024 (Virtual Hub & Spoke) and
Hierarchical VPN is not enough. Hierarchical VPN is not enough.
Also there is a need for a method to automatically trigger Also there is a need for a mechanism to automatically trigger
configuration changes on PE when remote CPEs' are instantiated or configuration changes on PEs when remote CPEs' are instantiated or
moved (leading to an IP address change) or deleted. moved (leading to an IP address change) or deleted.
EBGP Multi-hop scheme does not have an embedded security EBGP Multi-hop design does not include a security mechanism by
mechanism. The PE and remote CPEs need secure communication default. The PE and remote CPEs need secure communication channels
channels when connecting via the public Internet. when connecting via the public Internet.
Remote CPEs, if instantiated in Cloud DC, might have to traverse NAT Remote CPEs, if instantiated in Cloud DCs, might have to traverse
to reach PE. It is not clear how BGP can be used between devices NATs to reach PE. It is not clear how BGP can be used between
outside the NAT and the entities behind the NAT. It is not clear how devices outside the NAT and the entities behind the NAT. It is not
to configure the Next Hop on the PEs to reach private IPv4 clear how to configure the Next Hop on the PEs to reach private IPv4
addresses. addresses.
5.4. Designated Forwarder to the remote edges 5.4. Designated Forwarder to the remote edges
Among multiple floating PEs available for a remote CPE, multicast Among the multiple floating PEs that are available for a remote CPE,
traffic from the remote CPE towards the MPLS VPN can be forwarded multicast traffic sent by the remote CPE towards the MPLS VPN can be
back to the remote CPE due to the PE receiving the multicast data forwarded back to the remote CPE due to the PE receiving the
frame forwarding the multicast/broadcast frame to other PEs that in multicast data frame forwarding the multicast/broadcast frame to
turn send to all attached CPEs. This process may cause traffic loop. other PEs that in turn send to all attached CPEs. This process may
cause traffic loops.
Therefore, it is necessary to designate one floating PE as the CPE's Therefore, it is necessary to designate one floating PE as the CPE's
Designated Forwarder, similar to TRILL's Appointed Forwarders Designated Forwarder, similar to TRILL's Appointed Forwarders
[RFC6325]. [RFC6325].
Gap: the MPLS VPN does not have features like TRILL's Appointed Gap: the MPLS VPN does not have features like TRILL's Appointed
Forwarders. Forwarders.
5.5. Traffic Path Management 5.5. Traffic Path Management
When there are multiple floating PEs that have established IPsec When there are multiple floating PEs that have established IPsec
tunnels to the remote CPE, the remote CPE can forward the outbound tunnels with the remote CPE, the remote CPE can forward outbound
traffic to the Designated Forwarder PE, which in turn forwards the traffic to the Designated Forwarder PE, which in turn forwards
traffic to egress PEs to the destinations. However, it is not traffic to egress PEs and then to the final destinations. However,
straightforward for the egress PE to send back the return traffic to it is not straightforward for the egress PE to send back the return
the Designated Forwarder PE. traffic to the Designated Forwarder PE.
Example of Return Path management using Figure 3 above. Example of Return Path management using Figure 3 above.
- fPE-1 is DF for communication between App-1 <-> Host-a due to - fPE-1 is DF for communication between App-1 <-> Host-a due to
latency, pricing or other criteria. latency, pricing or other criteria.
- fPE-2 is DF for communication between App-1 <-> Host-b. - fPE-2 is DF for communication between App-1 <-> Host-b.
6. Manageability Considerations 6. Manageability Considerations
Zero touch provisioning of SD-WAN edge nodes is expected in SD- Zero touch provisioning of SD-WAN edge nodes is expected in SD-WAN
WAN deployment. It is necessary for a newly powered up SD-WAN deployment. It is necessary for a newly powered up SD-WAN edge
edges to establish a secure connection (such as TLS, DTLS, etc.) node to establish a secure connection (by means of TLS, DTLS,
to its controller. etc.) with its controller.
7. Security Considerations 7. Security Considerations
The intention of this draft is to identify the gaps in current and The intention of this draft is to identify the gaps in current and
proposed SD-WAN approaches that can address requirements proposed SD-WAN approaches that can address requirements
identified in [Net2Cloud-problem]. identified in [Net2Cloud-problem].
Several of these approaches have gaps in meeting enterprise Several of these approaches have gaps in meeting enterprise
security requirements when tunneling their traffic over the security requirements when tunneling their traffic over the
Internet, as is the general intention of SD-WAN. See the Internet, since this is the purpose of SD-WAN. See the individual
individual sections above for further discussion of these security sections above for further discussion of these security gaps.
gaps.
8. IANA Considerations 8. IANA Considerations
This document requires no IANA actions. RFC Editor: Please remove This document requires no IANA actions. RFC Editor: Please remove
this section before publication. this section before publication.
9. References 9. References
9.1. Normative References 9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
9.2. Informative References 9.2. Informative References
[RFC8192] S. Hares, et al, "Interface to Network Security Functions [RFC8192] S. Hares, et al, "Interface to Network Security Functions
(I2NSF) Problem Statement and Use Cases", July 2017 (I2NSF) Problem Statement and Use Cases", July 2017
[RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent
Address Family Identifier (SAFI) and the BGP Tunnel Address Family Identifier (SAFI) and the BGP Tunnel
Encapsulation Attribute", April 2009. Encapsulation Attribute", April 2009.
[BGP-SDWAN-EXT]L. Dunbar, "BGP Extension for SDWAN Overlay [BGP-SDWAN-PORT]L. Dunbar, et al, "Subsequent Address Family
Networks", draft-dunbar-idr-bgp-sdwan-overlay-ext-00, Oct Indicator for SDWAN Ports", draft-dunbar-idr-sdwan-port-
2018. safi-00, Work-in-progress, March 2019.
[BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for [BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for
SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework- SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework-
00, work-in-progress, Feb 2019. 00, work-in-progress, Feb 2019.
[Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation [Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation
Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018. Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018.
[VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over [SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public
Public Infrastructure", draft-rosen-bess-secure-l3vpn-00, Infrastructure", draft-rosen-bess-secure-l3vpn-00, work-
work-in-progress, July 2018 in-progress, July 2018
[DMVPN] Dynamic Multi-point VPN: [DMVPN] Dynamic Multi-point VPN:
https://www.cisco.com/c/en/us/products/security/dynamic- https://www.cisco.com/c/en/us/products/security/dynamic-
multipoint-vpn-dmvpn/index.html multipoint-vpn-dmvpn/index.html
[DSVPN] Dynamic Smart VPN: [DSVPN] Dynamic Smart VPN:
http://forum.huawei.com/enterprise/en/thread-390771-1- http://forum.huawei.com/enterprise/en/thread-390771-1-
1.html 1.html
[ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation,
 End of changes. 66 change blocks. 
267 lines changed or deleted 274 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/