< draft-dunbar-bess-bgp-sdwan-usage-07.txt   draft-dunbar-bess-bgp-sdwan-usage-08.txt >
Network Working Group L. Dunbar Network Working Group L. Dunbar
Internet Draft J. Guichard Internet Draft J. Guichard
Intended status: Informational Futurewei Intended status: Informational Futurewei
Expires: October 29, 2020 Ali Sajassi Expires: January 13, 2021 Ali Sajassi
Cisco Cisco
J. Drake J. Drake
Juniper Juniper
B. Najem B. Najem
Bell Canada Bell Canada
Ayan Barnerjee Ayan Barnerjee
D. Carrel D. Carrel
Cisco Cisco
April 29, 2020 July 13, 2020
BGP Usage for SDWAN Overlay Networks BGP Usage for SDWAN Overlay Networks
draft-dunbar-bess-bgp-sdwan-usage-07 draft-dunbar-bess-bgp-sdwan-usage-08
Abstract Abstract
The document describes three distinct SDWAN scenarios and discusses The document describes three distinct SDWAN scenarios and discusses
the applicability of BGP for each of those scenarios. The goal of the applicability of BGP for each of those scenarios. The goal of
the document is to demonstrate how BGP-based control plane is used the document is to demonstrate how BGP-based control plane is used
for large scale SDWAN overlay networks with little manual for large scale SDWAN overlay networks with little manual
intervention. intervention.
SDWAN edge nodes are commonly interconnected by multiple underlay SDWAN edge nodes are commonly interconnected by multiple underlay
skipping to change at page 2, line 18 skipping to change at page 2, line 18
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress." reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on October 29, 2020. This Internet-Draft will expire on January 13, 2009.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................3 1. Introduction...................................................3
2. Conventions used in this document..............................4 2. Conventions used in this document..............................4
3. Use Case Scenario Description and Requirements.................5 3. Use Case Scenario Description and Requirements.................6
3.1. Requirements..............................................6 3.1. Requirements..............................................6
3.1.1. Supporting Multiple SDWAN Segmentations..............6 3.1.1. Supporting Multiple SDWAN Segmentations..............6
3.1.2. Client Service Requirement...........................6 3.1.2. Client Service Requirement...........................6
3.1.3. Application Flow Based Segmentation..................7 3.1.3. Application Flow Based Segmentation..................7
3.1.4. Zero Touch Provisioning..............................8 3.1.4. Zero Touch Provisioning..............................8
3.1.5. Constrained Propagation of SDWAN Edge Properties.....9 3.1.5. Constrained Propagation of SDWAN Edge Properties.....9
3.2. Scenarios #1: Homogeneous WAN............................10 3.2. Scenarios #1: Homogeneous WAN............................10
3.3. Scenario #2: SDWAN WAN ports to VPN's PEs and to Internet11 3.3. Scenario #2: CPE based SDWAN over Hybrid WAN Underlay....11
3.4. Scenario #3: SDWAN WAN ports to MPLS VPN and the Internet14 3.4. Scenario #3: Private VPN PE based SDWAN..................14
4. BGP Walk Through..............................................15 4. BGP Walk Through..............................................15
4.1. BGP Walk Through for Homogeneous SDWAN...................15 4.1. BGP Walk Through for Homogeneous SDWAN...................15
4.2. BGP Walk Through for Application Flow Based Segmentation.18 4.2. BGP Walk Through for Application Flow Based Segmentation.18
4.3. Client Service Provisioning Model........................19 4.3. Client Service Provisioning Model........................19
4.4. WAN Ports Provisioning Model.............................20 4.4. WAN Ports Provisioning Model.............................20
4.5. Why BGP as Control Plane for SDWAN?......................20 4.5. Why BGP as Control Plane for SDWAN?......................20
5. SDWAN Traffic Forwarding Walk Through.........................21 5. SDWAN Traffic Forwarding Walk Through.........................21
5.1. SDWAN Network Startup Procedures.........................21 5.1. SDWAN Network Startup Procedures.........................21
5.2. Packet Walk-Through for Scenario #1......................22 5.2. Packet Walk-Through for Scenario #1......................22
5.3. Packet Walk-Through for Scenario #2......................22 5.3. Packet Walk-Through for Scenario #2......................22
5.3.1. SDWAN node WAN Ports Properties Registration........24 5.4. Packet Walk-Through for Scenario #3......................24
5.3.2. Controller Facilitated IPsec SA & NAT management....24 6. Manageability Considerations..................................24
5.4. Packet Walk-Through for Scenario #3......................26 7. Security Considerations.......................................24
6. Manageability Considerations..................................26 8. IANA Considerations...........................................25
7. Security Considerations.......................................26 9. References....................................................25
8. IANA Considerations...........................................26 9.1. Normative References.....................................25
9. References....................................................27 9.2. Informative References...................................25
9.1. Normative References.....................................27 10. Acknowledgments..............................................27
9.2. Informative References...................................27
10. Acknowledgments..............................................28
1. Introduction 1. Introduction
There are three key characteristics of "SDWAN" networks: Here are some key characteristics of "SDWAN" networks:
- Augment of transport, which refers to utilizing overlay paths - Augment of transport, which refers to utilizing overlay paths
over different underlay networks. Very often there are multiple over different underlay networks. Very often there are multiple
parallel overlay paths between any two SDWAN edges, some of parallel overlay paths between any two SDWAN edges, some of
which are private networks over which traffic can traverse with which are private networks over which traffic can traverse with
or without encryption, others require encryption, e.g. over or without encryption, others require encryption, e.g. over
untrusted public networks. untrusted public networks.
- Enable direct Internet access from remote sites, instead hauling - Enable direct Internet access from remote sites, instead hauling
all traffic to Corporate HQ for centralized policy control. all traffic to Corporate HQ for centralized policy control.
- Some traffic are routed based on application IDs instead of - Some traffic are routed based on application IDs instead of
based on destination IP addresses. based on destination IP addresses.
- The Application Routing can also be based on specific
performance criteria (e.g. packets delay, packet loos, jitter)
to provide better application performance by choosing the right
underlay that meets or exceeds the specified criteria.
[Net2Cloud-Problem] describes the network related problems that [Net2Cloud-Problem] describes the network related problems that
enterprises face to connect enterprises' branch offices to dynamic enterprises face to connect enterprises' branch offices to dynamic
workloads in different Cloud DCs, including using SDWAN to aggregate workloads in different Cloud DCs, including using SDWAN to aggregate
multiple paths provided by different service providers to achieve multiple paths provided by different service providers to achieve
better performance and to accomplish application ID based better performance and to accomplish application ID based
forwarding. forwarding.
Even though SDWAN has been positioned as a flexible way to reach Even though SDWAN has been positioned as a flexible way to reach
dynamic workloads in third party Cloud data centers over different dynamic workloads in third party Cloud data centers over different
skipping to change at page 5, line 9 skipping to change at page 5, line 10
NSP: Network Service Provider. NSP usually provides more NSP: Network Service Provider. NSP usually provides more
advanced network services, such as MPLS VPN, private advanced network services, such as MPLS VPN, private
leased lines, or managed Secure WAN connections, many leased lines, or managed Secure WAN connections, many
times within a private trusted domain, whereas an ISP times within a private trusted domain, whereas an ISP
usually provides plain internet services over public usually provides plain internet services over public
untrusted domains. untrusted domains.
PE: Provider Edge PE: Provider Edge
SDWAN End-point: a port (logical or physical) of a SDWAN edge node. SDWAN Edge Node: an edge node, which can be physical or virtual,
maps the attached clients' traffic to the wide area
network (WAN) overlay tunnels.
SDWAN: Software Defined Wide Area Network. In this document, SDWAN: Software Defined Wide Area Network. In this document,
"SDWAN" refers to the solutions of pooling WAN bandwidth "SDWAN" refers to the solutions of pooling WAN bandwidth
from multiple underlay networks to get better WAN from multiple underlay networks to get better WAN
bandwidth management, visibility & control. When the bandwidth management, visibility & control. When the
underlay networks are private, traffic can traverse underlay networks are private, traffic can traverse
without additional encryption; when the underlay without additional encryption; when the underlay
networks are public, such as the Internet, some traffic networks are public, such as the Internet, some traffic
may need to be encrypted when traversing through may need to be encrypted when traversing through
(depending on user provided policies). (depending on user provided policies).
skipping to change at page 6, line 20 skipping to change at page 6, line 25
3.1.1. Supporting Multiple SDWAN Segmentations 3.1.1. Supporting Multiple SDWAN Segmentations
The term "network segmentation", a.k.a. SDWAN instances, is The term "network segmentation", a.k.a. SDWAN instances, is
referring to the process of dividing the network into logical sub- referring to the process of dividing the network into logical sub-
networks using isolation techniques on a forwarding device such as a networks using isolation techniques on a forwarding device such as a
switch, router, or firewall. For a homogeneous network, such as MPLS switch, router, or firewall. For a homogeneous network, such as MPLS
VPN or Layer 2 network, VRF or VLAN are used to achieve the network VPN or Layer 2 network, VRF or VLAN are used to achieve the network
segmentation. segmentation.
As SDWAN is an overlay network arching over multiple types of As SDWAN is an overlay network arching over multiple types of
networks, VRF or VLAN can't be used directly to differentiate SDWAN networks, MPLS L2VPN/L3VPN or pure L2 underlay can continue using
network segmentations. the VRF, VN-ID or VLAN to differentiate SDWAN network segmentations.
For public internet, the IPsec inner encapsulation header can carry
However, BGP already has the capability to differentiate SDWAN the SDWAN Instance Identifier to differentiate the packets belonging
segmentations: to different SDWAN instances.
- Create a SDWAN Target ID in the BGP Extended Community to BGP already has the capability to differentiate control packets for
represent different SDWAN Segmentations different network instances. When using BGP for SDWAN, the SDWAN
- Same as Route Target, just use a different name to segmentations can be differentiated by the SDWAN Target ID in the
differentiate from VPN if a CPE supports traditional VPN with BGP Extended Community in the same way as VPN instances being
multiple VRFs and supports multiple SDWAN Segmentations represented by the Route Target. Same as Route Target, need to use a
(instances). different name to differentiate from VPN if a CPE supports
- When the SDWAN Target ID is used, traditional VPN with multiple VRFs and supports multiple SDWAN
- Use the similar approach as VPN Label carried by NLRI Path Segmentations (instances). The actual SDWAN Target ID encoding is
Attribute [RFC8277] to identify routes belonging to different proposed by [SDWAN-EDGE-Discovery].
SDWAN Segmentations.
- The MPLS VPN SAFI 128 & Route Distinguisher can be used for
routes belonging to different SDWAN instances.
3.1.2. Client Service Requirement 3.1.2. Client Service Requirement
Client interface of SDWAN nodes can be IP or Ethernet based. Client interface of SDWAN nodes can be IP or Ethernet based.
For Ethernet based client interfaces, SDWAN edge should support For Ethernet based client interfaces, SDWAN edge should support
VLAN-based service interfaces (EVI100), VLAN bundle service VLAN-based service interfaces (EVI100), VLAN bundle service
interfaces (EVI200), or VLAN-Aware bundling service interfaces. EVPN interfaces (EVI200), or VLAN-Aware bundling service interfaces. EVPN
service requirements are applicable to the Client traffic, as service requirements are applicable to the Client traffic, as
described in the Section 3.1 of RFC8388. described in the Section 3.1 of RFC8388.
skipping to change at page 9, line 41 skipping to change at page 9, line 41
+======+====+=+ +======+====+=====+ +======+====+=+ +======+====+=====+
/ / | +---+ | \ \ / / | +---+ | \ \
/ / | | \ \ / / | | \ \
+-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+
|C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE|
| 1 | | 2 | | 3 | |4 | | 5 | | 6 | | 1 | | 2 | | 3 | |4 | | 5 | | 6 |
+----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+ +----+
Tenant 1 Tenant 2 Tenant 1 Tenant 2
Figure 1: Peer Groups managed by RR Figure 1: Peer Groups managed by RR
Tenant separation is achieved by the RR creating different Tenant Tenant separation is achieved by the SDWAN instance identification
based Peer Groups. represented in control plane and data plane, respectively.
3.2. Scenarios #1: Homogeneous WAN 3.2. Scenarios #1: Homogeneous WAN
This is referring to a type of SDWAN network with edge nodes This is referring to a type of SDWAN network with edge nodes
encrypting all traffic over WAN to other edge nodes, regardless of encrypting all traffic over WAN to other edge nodes, regardless of
whether the underlay is private or public. For lack of better whether the underlay is private or public. For lack of better
terminology, we call this Homogeneous SDWAN throughout this terminology, we call this Homogeneous SDWAN throughout this
document. document.
Some typical scenarios for the use of a Homogeneous SDWAN network Some typical scenarios for the use of a Homogeneous SDWAN network
skipping to change at page 11, line 12 skipping to change at page 11, line 12
Figure 2: Homogeneous SDWAN Figure 2: Homogeneous SDWAN
One of the key properties of homogeneous SDWAN is that the SDWAN One of the key properties of homogeneous SDWAN is that the SDWAN
Local Network Controller (RR)is connected to C-PEs via untrusted Local Network Controller (RR)is connected to C-PEs via untrusted
public network, therefore, requiring secure connection between RR public network, therefore, requiring secure connection between RR
and C-PEs (TLS, DTLS, etc.). and C-PEs (TLS, DTLS, etc.).
Homogeneous SDWAN has some similarity to commonly deployed IPsec Homogeneous SDWAN has some similarity to commonly deployed IPsec
VPN, albeit the IPsec VPN is usually point-to-point among a small VPN, albeit the IPsec VPN is usually point-to-point among a small
number of endpoints and with heavy manual configuration for IPsec number of nodes and with heavy manual configuration for IPsec
between end-points, whereas an SDWAN network can have a large number between nodes, whereas an SDWAN network can have a large number of
of end-points with an SDWAN controller to manage requiring zero edge nodes with an SDWAN controller to manage requiring zero touch
touch provisioning upon powering up. provisioning upon powering up.
Existing Private VPNs (e.g. MPLS based) can use homogeneous SDWAN to Existing Private VPNs (e.g. MPLS based) can use homogeneous SDWAN to
extend over public network to remote sites to which the VPN operator extend over public network to remote sites to which the VPN operator
does not own or lease infrastructural connectivity, as described in does not own or lease infrastructural connectivity, as described in
[SECURE-EVPN] and [SECURE-L3VPN] [SECURE-EVPN] and [SECURE-L3VPN]
3.3. Scenario #2: SDWAN WAN ports to VPN's PEs and to Internet 3.3. Scenario #2: CPE based SDWAN over Hybrid WAN Underlay
In this scenario, SDWAN edge nodes (a.k.a. C-PEs) have some WAN In this scenario, SDWAN edge nodes (a.k.a. C-PEs) have some WAN
ports connected to PEs of Private VPNs over which packets can be ports connected to PEs of Private VPNs over which packets can be
forwarded natively without encryption, and some WAN ports connected forwarded natively without encryption, and some WAN ports connected
to the Internet over which sensitive traffic have to be encrypted to the public Internet over which sensitive traffic have to be
(usually by IPsec SA). encrypted (usually by IPsec SA).
In this scenario, the SDWAN edge nodes' egress WAN ports are all In this scenario, the SDWAN edge nodes' egress WAN ports are all
IP/Ethernet based, either egress to PEs of the VPNs or to the IP/Ethernet based, either egress to PEs of the VPNs or egress to the
Internet. Even if the VPN is a MPLS network, the VPN's PEs have public Internet. Even if the VPN is a MPLS network, the VPN's PEs
IP/Ethernet connections to the SDWAN edge (C-PEs). Throughout this have IP/Ethernet links to the SDWAN edge (C-PEs). Throughout this
document, this scenario is also called CPE based SDWAN over Hybrid document, this scenario is also called CPE based SDWAN over Hybrid
Networks. Networks.
Even though IPsec SA can secure the packets traversing the Internet, Even though IPsec SA can secure the packets traversing the Internet,
it does not offer the premium SLA commonly offered by Private VPNs, it does not offer the premium SLA commonly offered by Private VPNs,
especially over long distance. Clients need to have policies to especially over long distance. Clients need to have policies to
specify criteria for flows only traversing private VPNs or specify criteria for flows only traversing private VPNs or
traversing either as long as encrypted when over the Internet. For traversing either as long as encrypted when over the Internet. For
example, client can have those polices for the flows: example, client can have those polices for the flows:
1. A policy or criteria for sending the flows over a private 1. A policy or criteria for sending the flows over a private
network without encryption (for better performance), network without encryption (for better performance),
2. A policy or criteria for sending the flows over any networks 2. A policy or criteria for sending the flows over any networks
as long as the packets of the flows are encrypted when as long as the packets of the flows are encrypted when
traversing untrusted networks, or traversing untrusted networks, or
3. A policy of not needing encryption at all. 3. A policy of not needing encryption at all.
If a flow traversing multiple segments, such as A<->B<->C<->D, has If a flow traversing multiple segments, such as A<->B<->C<->D, has
either Policy 2 or 3 above, the flow can traverse different either Policy 2 or 3 above, the flow can traverse different
underlays in different segments, such as over Private network underlays in different network segments, such as over Private
underlay between A<->B without encryption, or over the public network underlay between A<->B without encryption, or over the
internet between B<->C in an IPsec SA. public internet between B<->C in an IPsec SA.
As shown in the figure below, C-PE-1 has two different types of As shown in the figure below, C-PE-1 has two different types of
interfaces (A1 to Internet and A2 & A3 to VPN). The C-PEs' loopback interfaces (A1 to Internet and A2 & A3 to VPN). The C-PEs' loopback
addresses and addresses attached to C-PEs may or may not be visible addresses and addresses attached to C-PEs may or may not be visible
to the ISPs/NSPs. The addresses for the WAN ports can have addresses to the ISPs/NSPs. The addresses for the WAN ports can have addresses
allocated by service providers or dynamically assigned (e.g. by allocated by service providers or dynamically assigned (e.g. by
DHCP). One WAN port shown in the figure below (e.g. A1, A2, A3 etc.) DHCP). One WAN port shown in the figure below (e.g. A1, A2, A3 etc.)
is a logical representation of potential multiple physical ports on is a logical representation of potential multiple physical ports on
the C-PEs. the C-PEs.
skipping to change at page 14, line 7 skipping to change at page 14, line 7
- Different services, routes, or VLANs attached to SDWAN nodes can - Different services, routes, or VLANs attached to SDWAN nodes can
be aggregated over one underlay path; same service/routes/VLAN can be aggregated over one underlay path; same service/routes/VLAN can
spread over multiple SDWAN underlays at different times depending spread over multiple SDWAN underlays at different times depending
on the policies specified for the service. For example, one on the policies specified for the service. For example, one
tenant's packets to HQ need to be encrypted when sent over the tenant's packets to HQ need to be encrypted when sent over the
Internet or have to be sent over private networks, while the same Internet or have to be sent over private networks, while the same
tenant's packets to Facebook can be sent over the Internet without tenant's packets to Facebook can be sent over the Internet without
encryption. encryption.
3.4. Scenario #3: SDWAN WAN ports to MPLS VPN and the Internet 3.4. Scenario #3: Private VPN PE based SDWAN
This scenario refers to existing VPN (e.g. MPLS based VPN, such as This scenario refers to existing VPN (e.g. MPLS based VPN, such as
EVPN or IPVPN) adding extra ports facing untrusted public networks EVPN or IPVPN) adding extra ports facing untrusted public networks
allowing PEs to offload some low priority traffic to ports facing allowing PEs to offload some low priority traffic to ports facing
public networks when the VPN MPLS paths are congested. Throughout public networks when the VPN MPLS paths are congested. Throughout
this document, this scenario is also called Internet Offload for this document, this scenario is also called Internet Offload for
Private VPN, or PE based SDWAN. Private VPN, or PE based SDWAN.
In this scenario, the packets offloaded to untrusted public network In this scenario, the packets offloaded to untrusted public network
must be encrypted. must be encrypted.
skipping to change at page 16, line 24 skipping to change at page 16, line 24
--|/ | | | |-12.1.1.x/24 --|/ | | | |-12.1.1.x/24
+---------+ | +------+ +---------+ | +------+
| |
C-PE3 | C-PE3 |
+---------+ | +---------+ |
--|---+---------------------------+ --|---+---------------------------+
| / | | / |
| / | | / |
--|/ | --|/ |
+---------+ +---------+
Figure 5: (see *.pdf for more accurate figure) Figure 5: Homogeneous SDWAN
The BGP UPDATE Message from C-PE2 to RR should have the client The BGP UPDATE Message from C-PE2 to RR should have the client
routes encoded in the MP-NLRI Path Attribute and the IPsec Tunnel routes encoded in the MP-NLRI Path Attribute and the IPsec Tunnel
associated information encoded in the Tunnel-Encap Path Attributes associated information encoded in the Tunnel-Encap Path Attributes
as described in the [SECURE-EVPN]: as described in the [SECURE-EVPN]:
- MP-NLRI Path Attribute: to indicate multiple routes attached to - MP-NLRI Path Attribute: to indicate multiple routes attached to
the C-PE2: the C-PE2:
10.1.x.x/16 10.1.x.x/16
VLAN #15 VLAN #15
12.1.1.x/24 12.1.1.x/24
- Tunnel-Encap Path Attribute: to describe the IPsec attributes - Tunnel-Encap Path Attribute: to describe the IPsec attributes
for routes encoded in the NLRI Path Attribute: for routes encoded in the NLRI Path Attribute:
IPsec attributes for remote nodes to establish the IPsec IPsec attributes for remote nodes to establish the IPsec
tunnel to C-PE2. tunnel to C-PE2.
If different client routes attached to C-PE2 needs to be reached by If different client routes attached to C-PE2 needs to be reached by
separate IPsec tunnels, then multiple BGP UPDATE messages need to be separate IPsec tunnels, then multiple BGP UPDATE messages need to be
sent to the remote nodes. If C-PE2 doesn't have the policy on sent to the remote nodes via RR. If C-PE2 doesn't have the policy on
mapping of clients' routes to IPsec tunnels, RR needs to check the authorized peers for the specific client routes, RR needs to check
client routes policies to send separate BGP UPDATE messages to the the client routes policies to propagate the BGP UPDATE messages to
remote edge nodes. the remote authorized edge nodes.
There could be policies governing the topologies of a client's There could be policies governing the topologies of a client's
different routes attached to an edge node. For example, VLAN #25 and different routes attached to an edge node. For example, VLAN #25 and
route 22.1.1.x/24 could be the Payment Applications described in the route 22.1.1.x/24 could be the Payment Applications described in the
Section 3.1.2 that can only communicate with Payment Gateway Section 3.1.2 that can only communicate with Payment Gateway
attached to C-PE3. If C-PEs don't have the policy to govern the attached to C-PE3. If C-PEs don't have the policy to govern the
communication peers, RR can take over the responsibility of only communication peers, RR can take over the responsibility of only
send BGP UPDATE for the Blue routes to C-PE1 and send the RED routes send BGP UPDATE to the authorized peers.
to C-PE3.
+---+ +---+
+-----------|RR |----------+ +-----------|RR |----------+
/ Untrusted +---+ \ / Untrusted +---+ \
/ \ / \
/ \ / \
+---------+ +------+ +---------+ +------+
--+ --------------------------------------> |-10.1.x.x/16 --+ --------------------------------------> |-10.1.x.x/16
| C-PE1 | |C-PE2 |- VLAN = 15 | C-PE1 | |C-PE2 |- VLAN = 15
| | +-----------> |- 12.1.1.x/24 | | +-----------> |- 12.1.1.x/24
skipping to change at page 18, line 6 skipping to change at page 18, line 4
- Tunnel-Encap Path Attribute: - Tunnel-Encap Path Attribute:
IPsec SA attributes for IPsec tunnels to C-PE2 from any node IPsec SA attributes for IPsec tunnels to C-PE2 from any node
for reaching 10.1.x.x/16, VLAN #15, and 12.1.1.x/24. for reaching 10.1.x.x/16, VLAN #15, and 12.1.1.x/24.
UPDATE 2 (only sent to C-PE3) UPDATE 2 (only sent to C-PE3)
- MP-NLRI Path Attribute: - MP-NLRI Path Attribute:
VLAN #25 VLAN #25
22.1.1.x/24 22.1.1.x/24
- Tunnel-Encap: - Tunnel-Encap:
IPsec SA attributes for IPsec tunnels to C-PE2 from C-PE3 for IPsec SA attributes for IPsec tunnels to C-PE2 from C-PE3 for
reaching VLAN #25 and subnet 22.1.1./24. reaching VLAN #25 and subnet 22.1.1./24.
4.2. BGP Walk Through for Application Flow Based Segmentation 4.2. BGP Walk Through for Application Flow Based Segmentation
If the applications are assigned with unique IP addresses, the If the applications are assigned with unique IP addresses, the
Application Flow based Segmentation described in Section 3.1.2 can Application Flow based Segmentation described in Section 3.1.2 can
be achieved by advertising different BGP UPDATE messages to be achieved by advertising different BGP UPDATE messages to
different nodes. In the Figure below, the following BGP Updates can different nodes. In the Figure below, the following BGP Updates can
be advertised to ensure that Payment Application only communicates be advertised to ensure that Payment Application only communicates
with the Payment Gateway: with the Payment Gateway:
BGP UPDATE #1 from C-PE2 to RR for the RED P2P topology (only BGP UPDATE #1 from C-PE2 to RR for the P2P topology that is only
propagated to Payment GW node: propagated to Payment GW node:
- MP-NLRI Path Attribute: - MP-NLRI Path Attribute:
- 30.1.1.x/24 - 30.1.1.x/24
- Tunnel Encap Path Attribute - Tunnel Encap Path Attribute
- IPsec Attributes for PaymentGW ->C-PE2 - IPsec Attributes for PaymentGW ->C-PE2
BGP UPDATE #2 from C-PE2 to RR for the routes to be reached by BGP UPDATE #2 from C-PE2 to RR for the routes to be reached by C-PE1
Purple: and C-PE2:
- MP-NLRI Path Attribute: - MP-NLRI Path Attribute:
- 10.1.x.x - 10.1.x.x
- 12.4.x.x - 12.4.x.x
- TunnelEncap Path Attribute: - Tunnel-Encap Path Attribute:
- Any node to C-PE2 - Any node to C-PE2
+-------+ +-------+
|Payment| |Payment|
+-------->| GW |<-------+ +-------->| GW |<-------+
/Hub-spoke +-------+ \ /Hub-spoke +-------+ \
/for Payment Ppp \ /for Payment App \
C-PE1 / \ C-PE2 C-PE1 / \ C-PE2
+------/--+ +----\-+ +------/--+ +----\-+
--|-----/ | | -|- 30.1.1.x/24 --|-----/ | | -|- 30.1.1.x/24
+ ---------------------------------------> |-10.1.x.x/16 + ---------------------------------------> |-10.1.x.x/16
| | | |- | | | |-
| | +------------> |- 12.1.1.x/24 | | +------------> |- 12.1.1.x/24
--|---------------------------+ | | --|---------------------------+ | |
+---------+ +------> |- VLAN=25; +---------+ +------> |- VLAN=25;
/ +------+ 22.1.1.x/24 / +------+ 22.1.1.x/24
+---------+ / +---------+ /
--| -----------------------------+ --| -----------------------------+
| C-PE3 | / | C-PE3 | /
| | / | | /
--| --------------------------+ --| --------------------------+
+---------+ +---------+
Figure 7: (see *.pdf for more accurate figure) Figure 7: Application Based SDWAN Segmentation
4.3. Client Service Provisioning Model 4.3. Client Service Provisioning Model
The provisioning tasks described in Section 4 of RFC8388 are the The provisioning tasks described in Section 4 of RFC8388 are the
same for the SDWAN client traffic. When client traffic is multi- same for the SDWAN client traffic. When client traffic is multi-
homed to two (or more) C-PEs, the Non-Service-Specific parameters homed to two (or more) C-PEs, the Non-Service-Specific parameters
need to be provisioned per the Section 4.1.1 of RFC8388. need to be provisioned per the Section 4.1.1 of RFC8388.
Since most SDWAN nodes are ephemeral and have small number of IP Since some SDWAN nodes are ephemeral and have small number of IP
subnets or VLANs attached to their client ports, it is recommended subnets or VLANs attached to their client ports, it is recommended
to have default and simplified Service-specific parameters for each to have default and simplified Service-specific parameters for each
client port, remotely managed by the SDWAN Network Controller via client port, remotely managed by the SDWAN Network Controller via
the secure channel (TLS/DTLS) between the controller and the C-PEs. the secure channel (TLS/DTLS) between the controller and the C-PEs.
4.4. WAN Ports Provisioning Model 4.4. WAN Ports Provisioning Model
Since the deployment of PEs to MPLS VPN are for relatively long Since the deployment of PEs to MPLS VPN are for relatively long
term, the common provisioning procedure for PE's WAN ports is via term, the common provisioning procedure for PE's WAN ports is via
CLI. CLI.
skipping to change at page 22, line 17 skipping to change at page 22, line 17
removed from) existing VPN PEs. removed from) existing VPN PEs.
5.2. Packet Walk-Through for Scenario #1 5.2. Packet Walk-Through for Scenario #1
Upon power up, a SDWAN node can learn client routes from the Client Upon power up, a SDWAN node can learn client routes from the Client
facing ports, in the same way as EVPN described in RFC8388. facing ports, in the same way as EVPN described in RFC8388.
Controller facilitates the IPsec SA establishment and rekey Controller facilitates the IPsec SA establishment and rekey
management as described in [SECURE-EVPN]. Controller manages how management as described in [SECURE-EVPN]. Controller manages how
client's routes are associated with individual IPSec SA. client's routes are associated with individual IPSec SA.
[SECURE-L3VPN] describes how to extend the RFC4364 VPN to allow some [SECURE-EVPN] describes a solution for SDWAN Scenario #1. It
PEs being connected to other PEs via public networks. [SECURE-L3VPN]
introduces the concept of Red Interface & Black Interface on those
PEs. RED interfaces face the VPN over which packets can be forwarded
natively without encryption. Black Interfaces face public network
over which only IPsec-protected packets are forwarded.
[SECURE-L3VPN] assumes PEs terminate MPLS packets, and use MPLS over
IPsec when sending over the Black Interfaces.
[SECURE-EVPN] describes a solution for SDWAN Scenario #1. It
utilizes the BGP RR to facilitate the key and policy exchange among utilizes the BGP RR to facilitate the key and policy exchange among
PE devices to create private pair-wise IPsec Security Associations PE devices to create private pair-wise IPsec Security Associations
without IKEv2 point-to-point signaling or any other direct peer-to- without IKEv2 point-to-point signaling or any other direct peer-to-
peer session establishment messages. peer session establishment messages.
When C-PEs do not support MPLS, the approaches described by RFC8365 When C-PEs do not support MPLS, the approaches described by RFC8365
can be used, with addition of IPsec encrypting the IP packets when can be used, with addition of IPsec encrypting the IP packets when
sending packets over the Black Interfaces. sending packets over the Black Interfaces.
5.3. Packet Walk-Through for Scenario #2 5.3. Packet Walk-Through for Scenario #2
skipping to change at page 23, line 12 skipping to change at page 22, line 46
WAN ports facing the trusted VPN without encryption, which can WAN ports facing the trusted VPN without encryption, which can
egress the WAN ports facing the public Internet with encryption, or egress the WAN ports facing the public Internet with encryption, or
which can egress WAN ports facing the public Internet without which can egress WAN ports facing the public Internet without
encryption. encryption.
The internet facing WAN ports can face potential DDoS attacks, The internet facing WAN ports can face potential DDoS attacks,
additional anti-DDoS mechanism has to be enabled on those WAN ports additional anti-DDoS mechanism has to be enabled on those WAN ports
and the Control Plane should not learn routes from the Public and the Control Plane should not learn routes from the Public
Network facing WAN ports. Network facing WAN ports.
The Scenario #2 SDWAN Control Plane should include those three For the Scenario #2, if a client route can be reached by MPLS VPN
distinct functional components: and IPsec Tunnel via public network, the BGP UPDATE for the client
route should indicate all available tunnels in the Tunnel Path
Attribute of the BGP NLRI.
+---+ +---+
+--------------|RR |----------+ +--------------|RR |----------+
/ Untrusted +-+-+ \ / Untrusted +-+-+ \
/ Network \ / Network \
/ \ / \
+----+ +---------+ packets encrypted over +--------+ +----+ +----+ +---------+ packets encrypted over +--------+ +----+
| CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1|
+----+ | C-PE-a A2-----+ +-------B2 C-PE-b| +----+ +----+ | C-PE-a A2-----+ +-------B2 C-PE-b| +----+
|10.1.1.1 | |10.1.2.1| |10.1.1.1 | |10.1.2.1|
+----+ | | +--+ +---+ | | +----+ +----+ | | +--+ +---+ | | +----+
| CN2|--| A3 |PE+--------------+PE |---B3 |--| CN3| | CN2|--| A3 |PE+--------------+PE |---B3 |--| CN3|
+----+ +---------+ +--+ trusted +---+ +--------+ +----+ +----+ +---------+ +--+ trusted +---+ +--------+ +----+
| VPN | | VPN |
+--------------+ +--------------+
Figure 8: SDWAN Scenario #2 Figure 8: SDWAN Scenario #2
- SDWAN node's WAN ports property registration to the SDWAN For example, if the CN1 route can be reached by both VPN and Public
Network Controller. internet, the CN1's BGP route UPDATE should include the following:
o See 5.3.1. for detail.
- Controller Facilitated IPsec SA management and NAT information
distribution
o See 5.3.2. for details.
- Attached routes distribution via BGP, which can be EVPN, IPVPN
or others.
o This is for the clients' route distribution, so that a C-
PE can establish the overlay routing table that identifies
the next hop for reaching a specific route/service
attached to remote nodes. [SECURE-EVPN] describes EVPN and
other options.
5.3.1. SDWAN node WAN Ports Properties Registration
In Figure 6, A1/A2/A3/B1/B2/B3 WAN ports can be from different
network providers.
+---+
10.1.1.1 via +--------------|RR |----------+ 10.1.2.1 via
A1/A2/A3 / Untrusted +-+-+ \ B1/B2/B3
/ \
/ \
+----+ +---------+ packets encrypted over +--------+ +----+
| CN3|--| A1-----+ Untrusted +------ B1 |--| CN1|
+----+ | C-PE1 A2-----+ +-------B2 C-PE2 | +----+
|10.1.1.1 | |10.1.2.1|
+----+ | | +--+ +---+ | | +----+
| CN2|--| A3 |PE+--------------+PE |---B3 |--| CN3|
+----+ +---------+ +--+ trusted +---+ +--------+ +----+
| VPN |
+--------------+
Figure 9: SDWAN Scenario #2 WAN Ports Registration
Each SDWAN edge(C-PE) needs to register its WAN ports properties
along with its Loopback addresses to the SDWAN Network Controller.
The policies that govern the communications among peers are managed
and controlled by the SDWAN Controller. Individual SDWAN edge relies
on its SDWAN Controller to determine which peers can establish
connections. The SDWAN controller is responsible for propagating the
mapping information to the authorized peers. If C-PE-1 is not
authorized to communicate with C-PE-n, C-PE-1's WAN port<->Loopback
address mapping will not be propagated to C-PE-n.
A C-PE's Loopback addresses & attached routes may not be visible to
some ISPs/NSPs to which the CPE's WAN port is connected.
Section 4 describes how C-PEs use the BGP UPDATE messages to - MP-NLRI Path Attribute:
propagate their local information to their corresponding RR.
5.3.2. Controller Facilitated IPsec SA & NAT management CN1
Setting up and managing one IPsec SA between two points is - Tunnel-Encap Path Attribute:
straightforward and simple. But managing multi-point IPsec SAs among
many points can be overwhelming. For a 1,000-node network, each node
may need to manage 999 IPSec SA keys to all their peers, which could
potentially result in 1,000,000 key exchanges to authenticate among
all nodes. In addition, when an edge node has multiple tenants
attached, the edge node may need to establish multiple tunnels to
isolate traffic from different tenants. When the SDWAN IPsec SAs are
fine-grained, such as per client address, per client's VLAN, the
number of IPsec SAs & Keys to be managed can go much higher, leading
to more IPsec management complexity.
All the IPsec keys have to be refreshed periodically. Therefore, Tunnel 1: MPLS-in-GRE encapsulation
simplification facilitated by an SDWAN controller is necessary for With the MPLS-in-GRE Sub-TLV specified by Tunnel-Encap;
large-scale SDWAN deployment.
SDWAN edge nodes can rely on the SDWAN controller to facilitate the Tunnel 2: IPsec-GRE encapsulation
pair-wise IPsec key establishment and refreshment [RFC7296] and With the IPsec Sub-TLVs specified by the [SECURE-EVPN] and
maintain the Security Policy Database (SPD) [RFC4301]. [BGP-EDGE-DISCOVERY]
- For the SDWAN Scenario #2 in described in Section 3.3. , if C-PE1 There could be multiple IPsec SA tunnels terminated at the edge node
& C-PE2 loopback addresses are not visible to the public network's loopback address or terminated at WAN ports. For the Scenario #2,
ISP(s). C-PE1 & C-PE2 can use their provider assigned IP addresses there can be policies to determine which IPsec SA tunnels that the
for WAN ports A1/A2/B1/B2 to establish WAN Port based IPsec SA client route can be carried. When a client route can be carried by
through the public network. Under this circumstance, there need to multiple IPsec SA tunnels terminated by two different WAN ports,
be minimum four IPsec SAs between C-PE1 & C-PE2 internet facing multiple Tunnel Path Attributes with different Tunnel-end-point Sub-
WAN ports. TLVs need to be included in the NLRI of the BGP UPDATE for the
- When C-PEs loopback addresses are visible to ISPs/NSPs, i.e. the client route.
C-PEs' private source and destination IPs are part of a prefix
exported to the ISP(s) in each site, it is possible to have one
IPsec SA between C-PE1 & C-PE2.
The IP addresses of SDWAN WAN port can be dynamic (e.g. assigned by 5.4. Packet Walk-Through for Scenario #3
DHCP) or private IP. Some SDWAN nodes are identified by "System-ID"
or Loopback addresses that are only locally significant. In some
SDWAN environments, "System-ID + PortID" are used to uniquely
identify a SDWAN WAN port. Sometimes, a SDWAN tunnel end-point can
be associated with "private IP" + "public IP" (if NAT is used.)
When CPE WAN ports are private addresses, an additional sub-TLV has The behavior described in [SECURE-L3VPN] applies to this scenario.
to be added to the [Tunnel-Encap] to describe the additional
information about the NAT property of SDWAN nodes' WAN ports. A
SDWAN node can inquire STUN (Session Traversal of UDP through
Network Address Translation [RFC 3489]) Server to get the NAT
property, the public IP address and the Public Port number to pass
to the authorized peers via the SDWAN Controller.
5.4. Packet Walk-Through for Scenario #3 [SECURE-L3VPN] describes how to extend the RFC4364 VPN to allow some
PEs being connected to other PEs via public networks. In this
scenario, the PEs is the SDWAN Edge nodes. [SECURE-L3VPN] introduces
the concept of RED Interface & Black Interface on those PEs. RED
interfaces face the VPN over which packets can be forwarded natively
without encryption. Black Interfaces face public network over which
only IPsec-protected packets are forwarded. [SECURE-L3VPN] assumes
PEs terminate MPLS packets, and use MPLS over IPsec when sending
over the Black Interfaces.
The behavior described in [SECURE-L3VPN] applies to this scenario, The C-PEs not only have RED interfaces facing clients but also have
except C-PEs not only have RED interfaces facing clients but also RED interface facing MPLS backbone, with additional BLACK interfaces
have RED interface facing MPLS backbone, with additional BLACK facing the untrusted public networks for the WAN side. The C-PEs
interfaces facing the untrusted public networks for the WAN side. cannot mix the routes learned from the Black Interfaces with the
The C-PEs cannot mix the routes learned from the Black Interfaces Routes from RED Interfaces. The routes learned from core-facing RED
with the Routes from RED Interfaces. The routes learned from core- interfaces are for underlay and cannot be mixed with the routes
facing RED interfaces are for underlay and cannot be mixed with the learned over access-facing RED interfaces that are for overlay.
routes learned over access-facing RED interfaces that are for Furthermore, the routes learned over core-facing interfaces (both
overlay. Furthermore, the routes learned over core-facing interfaces RED and BLACK) can be shared in the same GLOBAL route table.
(both RED and BLACK) can be shared in the same GLOBAL route table.
There may be some added risks of the packets from the ports facing There may be some added risks of the packets from the ports facing
the Internet. Therefore, special consideration has to be given to the Internet. Therefore, special consideration has to be given to
the routes from WAN ports facing the Internet. RFC4364 describes the routes from WAN ports facing the Internet. RFC4364 describes
using an RD to create different routes for reaching same system. A using an RD to create different routes for reaching same system. A
similar approach can be considered to force packets received from similar approach can be considered to force packets received from
the Internet facing ports to go through special security functions the Internet facing ports to go through special security functions
before being sent over to the VPN backbone WAN ports. before being sent over to the VPN backbone WAN ports.
6. Manageability Considerations 6. Manageability Considerations
SDWAN overlay networks utilize the SDWAN controller to facilitate SDWAN overlay networks utilize the SDWAN controller to facilitate
route distribution, central configurations, and others. To route distribution, central configurations, and others. SDWAN Edge
minimize the burden on SDWAN edge nodes, SDWAN Edge nodes might nodes need to advertise the attached routes to their controller
not need to learn the routes from clients. (i.e. RR in BGP case).
7. Security Considerations 7. Security Considerations
Having WAN ports facing the public Internet introduces the following Having WAN ports facing the public Internet introduces the following
security risks: security risks:
1) Potential DDoS attack to the C-PEs with ports facing internet. 1) Potential DDoS attack to the C-PEs with ports facing internet.
2) Potential risk of provider VPN network being injected with 2) Potential risk of provider VPN network being injected with
illegal traffic coming from the public Internet WAN ports on the C- illegal traffic coming from the public Internet WAN ports on the C-
skipping to change at page 27, line 41 skipping to change at page 26, line 9
Encapsulation Attribute", April 2009. Encapsulation Attribute", April 2009.
[BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension for [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension for
SDWAN Overlay Networks", draft-dunbar-idr-bgp-sdwan- SDWAN Overlay Networks", draft-dunbar-idr-bgp-sdwan-
overlay-ext-03, work-in-progress, Nov 2018. overlay-ext-03, work-in-progress, Nov 2018.
[Net2Cloud-Gap] L. Dunbar, A. Malis, C. Jacquenet, "Gap Analysis of [Net2Cloud-Gap] L. Dunbar, A. Malis, C. Jacquenet, "Gap Analysis of
Interconnecting Underlay with Cloud Overlay", draft-dm- Interconnecting Underlay with Cloud Overlay", draft-dm-
net2cloud-gap-analysis-02, work in progress, Oct. 2018. net2cloud-gap-analysis-02, work in progress, Oct. 2018.
[SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. Majumdar,
"BGP UPDATE for SDWAN Edge Discovery", draft-dunbar-idr-
sdwan-edge-discovery-00, work-in-progress, July 2020.
[VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over [VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over
Public Infrastructure", draft-rosen-bess-secure-l3vpn-00, Public Infrastructure", draft-rosen-bess-secure-l3vpn-00,
work-in-progress, July 2018 work-in-progress, July 2018
[DMVPN] Dynamic Multi-point VPN: [DMVPN] Dynamic Multi-point VPN:
https://www.cisco.com/c/en/us/products/security/dynamic- https://www.cisco.com/c/en/us/products/security/dynamic-
multipoint-vpn-dmvpn/index.html multipoint-vpn-dmvpn/index.html
[DSVPN] Dynamic Smart VPN: [DSVPN] Dynamic Smart VPN:
http://forum.huawei.com/enterprise/en/thread-390771-1- http://forum.huawei.com/enterprise/en/thread-390771-1-
 End of changes. 44 change blocks. 
186 lines changed or deleted 117 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/