< draft-bryant-rtgwg-enhanced-vpn-01.txt   draft-bryant-rtgwg-enhanced-vpn-02.txt >
Routing Area Working Group S. Bryant Routing Area Working Group S. Bryant
Internet-Draft J. Dong Internet-Draft J. Dong
Intended status: Informational Huawei Intended status: Informational Huawei
Expires: May 3, 2018 October 30, 2017 Expires: September 6, 2018 Z. Li
China Mobile
T. Miyasaka
KDDI Corporation
March 05, 2018
Enhanced Virtual Private Networks (VPN+) Enhanced Virtual Private Networks (VPN+)
draft-bryant-rtgwg-enhanced-vpn-01 draft-bryant-rtgwg-enhanced-vpn-02
Abstract Abstract
This draft describes a number of enhancements that need to be made to This draft describes a number of enhancements that need to be made to
virtual private networks (VPNs) to support the needs of new virtual private networks (VPNs) to support the needs of new
applications, particularly applications that are associated with 5G applications, particularly applications that are associated with 5G
services. A network enhanced with these properties may form the services. A network enhanced with these properties may form the
underpin of network slicing, but will also be of use in its own underpin of network slicing, but will also be of use in its own
right. right.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 3, 2018. This Internet-Draft will expire on September 6, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Requirements Language . . . . . . . . . . . . . . . . . . . . 3 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4
3. Overview of the Requirements . . . . . . . . . . . . . . . . 3 3. Overview of the Requirements . . . . . . . . . . . . . . . . 4
3.1. Isolation between VPNs . . . . . . . . . . . . . . . . . 4 3.1. Isolation between Virtual Networks . . . . . . . . . . . 4
3.2. Guaranteed Performance . . . . . . . . . . . . . . . . . 5 3.2. Diverse Performance Guarantees . . . . . . . . . . . . . 6
3.3. Integration . . . . . . . . . . . . . . . . . . . . . . . 6 3.3. A Pragmatic Approach to Isolation . . . . . . . . . . . . 7
3.4. Dynamic Configuration . . . . . . . . . . . . . . . . . . 6 3.4. Integration . . . . . . . . . . . . . . . . . . . . . . . 8
3.5. Customized Control Plane . . . . . . . . . . . . . . . . 7 3.5. Dynamic Configuration . . . . . . . . . . . . . . . . . . 8
4. Architecture and Components of VPN+ . . . . . . . . . . . . . 7 3.6. Customized Control Plane . . . . . . . . . . . . . . . . 9
4.1. Data-Plane . . . . . . . . . . . . . . . . . . . . . . . 7 4. Architecture and Components of VPN+ . . . . . . . . . . . . . 9
4.1.1. Data-plane Layering . . . . . . . . . . . . . . . . . 8 4.1. Communications Layering . . . . . . . . . . . . . . . . . 9
4.1.2. Use of Segment Routing Constructs . . . . . . . . . . 8 4.2. Multi-Point to Multi-point . . . . . . . . . . . . . . . 10
4.1.3. Segment Routing and Isolation . . . . . . . . . . . . 8 4.3. Candidate Underlay Technologies . . . . . . . . . . . . . 10
4.2. Stateful and Stateless Virtual Networks . . . . . . . . . 9 4.3.1. FlexE . . . . . . . . . . . . . . . . . . . . . . . . 11
4.3. Latency Support . . . . . . . . . . . . . . . . . . . . . 10 4.3.2. Dedicated Queues . . . . . . . . . . . . . . . . . . 12
4.4. Integrating Service Function Chains and VPNs . . . . . . 11 4.3.3. Time Sensitive Networking . . . . . . . . . . . . . . 12
4.5. Application Specific Network Types . . . . . . . . . . . 11 4.3.4. Deterministic Networking . . . . . . . . . . . . . . 12
4.6. Control Plane Considerations . . . . . . . . . . . . . . 12 4.3.5. MPLS Traffic Engineering (MPLS-TE) . . . . . . . . . 13
5. Scalability Considerations . . . . . . . . . . . . . . . . . 12 4.3.6. Segment Routing . . . . . . . . . . . . . . . . . . . 13
5.1. Maximum Stack Depth . . . . . . . . . . . . . . . . . . . 13 4.4. Control Plane Considerations . . . . . . . . . . . . . . 16
5.2. RSVP scalability . . . . . . . . . . . . . . . . . . . . 13 4.5. Application Specific Network Types . . . . . . . . . . . 17
6. OAM and Instrumentation . . . . . . . . . . . . . . . . . . . 14 4.6. Integration with Service Functions . . . . . . . . . . . 17
7. Service Disruption During Change . . . . . . . . . . . . . . 14 5. Scalability Considerations . . . . . . . . . . . . . . . . . 17
8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 5.1. Maximum Stack Depth . . . . . . . . . . . . . . . . . . . 18
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 5.2. RSVP scalability . . . . . . . . . . . . . . . . . . . . 18
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 15 6. OAM and Instrumentation . . . . . . . . . . . . . . . . . . . 19
10.1. Normative References . . . . . . . . . . . . . . . . . . 15 7. Enhanced Resiliency . . . . . . . . . . . . . . . . . . . . . 19
10.2. Informative References . . . . . . . . . . . . . . . . . 15 8. Security Considerations . . . . . . . . . . . . . . . . . . . 20
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.1. Normative References . . . . . . . . . . . . . . . . . . 21
10.2. Informative References . . . . . . . . . . . . . . . . . 21
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction 1. Introduction
Virtual networks, often referred to as virtual private networks Virtual networks, often referred to as virtual private networks
(VPNs) have served the industry well as a means of providing (VPNs) have served the industry well as a means of providing
different groups of users with logically isolated access to a common different groups of users with logically isolated access to a common
network. The common or base network that is used to provide the VPNs network. The common or base network that is used to provide the VPNs
is often referred to as the underlay, and the VPN is often called an is often referred to as the underlay, and the VPN is often called an
overlay. overlay.
skipping to change at page 3, line 16 skipping to change at page 3, line 23
The tenant of such a network can require a degree of isolation and The tenant of such a network can require a degree of isolation and
performance that previously could only be satisfied by dedicated performance that previously could only be satisfied by dedicated
networks. Additionally the tenant may ask for some level of control networks. Additionally the tenant may ask for some level of control
of their virtual network e.g. to customize the service paths in the of their virtual network e.g. to customize the service paths in the
network slice. network slice.
These properties cannot be met with pure overlay networks, as they These properties cannot be met with pure overlay networks, as they
require tighter coordination and integration between the underlay and require tighter coordination and integration between the underlay and
the overlay network. This document introduces a new network service the overlay network. This document introduces a new network service
called enhanced VPN (VPN+). VPN+ refers to a virtual network which called enhanced VPN (VPN+). VPN+ refers to a virtual network which
has dedicated network resources allocated from the underlay network, has dedicated network resources allocated from the underlay network.
and can achieve a greater isolation and lower latency than Unlike traditional VPN, an enhanced VPN can achieve greater isolation
traditional VPN. and guaranteed performance.
These new network layer properties, which have general applicability, These new network layer properties, which have general applicability,
may also be of interest as part of a network slicing solution may also be of interest as part of a network slicing solution.
[I-D.geng-netslices-architecture]
In this draft we identify the new and modified components that need This document specifies a framework for using the existing, modified
to be provided in the network layer and their associated control and and potential new networking technologies as components to provide an
monitoring of an enhanced VPN (VPN+). Specifically we are concerned enhanced VPN (VPN+) service. Specifically we are concerned with:
with the technology needed to be provided by the enhanced VPN
underlay, the enhanced VPN data-plane and the necessary protocols in o The design of the enhanced VPN data-plane
both the underlay and the overlay of enhanced VPN. One use for
enhanced VPNs is to create network slices with different isolation o The necessary protocols in both, underlay and the overlay of
requirements. Such slices may be used to provide different tenants enhanced VPN, and
of vertical industrial markets with their own virtual network with
the explicit characteristics required. These slices may be "hard" o The mechanisms to achieve integration between overlay and underlay
slices providing a high degree of confidence that the VPN+
o The necessary method of monitoring an enhanced VPN
o The methods of instrumenting an enhanced VPN to ensure that the
required tenant Service Level Agreement (SLA) is maintained
The required layer structure necessary to achieve this is shown in
Section 4.1.
One use for enhanced VPNs is to create network slices with different
isolation requirements. Such slices may be used to provide different
tenants of vertical industrial markets with their own virtual network
with the explicit characteristics required. These slices may be
"hard" slices providing a high degree of confidence that the VPN+
characteristics will be maintained over the slice life cycle, of they characteristics will be maintained over the slice life cycle, of they
may be "soft" slices in which case some degree of interaction may be may be "soft" slices in which case some degree of interaction may be
experienced. experienced.
2. Requirements Language 2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in "OPTIONAL" in this document are to be interpreted as described in
[RFC2119]. [RFC2119].
3. Overview of the Requirements 3. Overview of the Requirements
In this section we provide an overview of the requirements of an In this section we provide an overview of the requirements of an
enhanced VPN. enhanced VPN.
3.1. Isolation between VPNs 3.1. Isolation between Virtual Networks
The requirement is to provide both hard and soft isolation between The requirement is to provide both hard and soft isolation between
the tenants/applications using one enhanced VPN and the tenants/ the tenants/applications using one enhanced VPN and the tenants/
applications using another enhanced VPN. Hard isolation is needed so applications using another enhanced VPN. Hard isolation is needed so
that applications with exacting requirements can function correctly that applications with exacting requirements can function correctly
despite a flash demand being created on another VPN competing for the despite a flash demand being created on another VPN competing for the
underlying resources. An example might be a network supporting both underlying resources. An example might be a network supporting both
emergency services and public broadband multi-media services. emergency services and public broadband multi-media services.
During a major incident the VPNs supporting these services would both During a major incident the VPNs supporting these services would both
skipping to change at page 4, line 30 skipping to change at page 4, line 47
We introduce the terms hard (static) and soft (dynamic) isolation to We introduce the terms hard (static) and soft (dynamic) isolation to
cover cases such as the above. A VPN has soft isolation if the cover cases such as the above. A VPN has soft isolation if the
traffic of one VPN cannot be inspected by the traffic of another. traffic of one VPN cannot be inspected by the traffic of another.
Both IP and MPLS VPNs are examples of soft isolated VPNs because the Both IP and MPLS VPNs are examples of soft isolated VPNs because the
network delivers the traffic only to the required VPN endpoints. network delivers the traffic only to the required VPN endpoints.
However the traffic from one or more VPNs and regular network traffic However the traffic from one or more VPNs and regular network traffic
may congest the network resulting in delays for other VPNs operating may congest the network resulting in delays for other VPNs operating
normally. The ability for a VPN to be sheltered from this effect is normally. The ability for a VPN to be sheltered from this effect is
called hard isolation, and this property is required by some critical called hard isolation, and this property is required by some critical
applications. applications. Although these isolation requirements are triggered by
the needs of 5G networks, they have general utility. In the
Although these isolation requirements are triggered by the needs of remainder of this section we explore how isolation may be achieved in
5G networks, they have general utility. packet networks.
It is of course possible to achieve high degrees of isolation in the It is of course possible to achieve high degrees of isolation in the
optical layer. However this is done at the cost of allocating optical layer. However this is done at the cost of allocating
resources on a long term basis and end-to-end basis. Such an resources on a long term basis and end-to-end basis. Such an
arrangement means that the full cost of the resources must be borne arrangement means that the full cost of the resources must be borne
by the service that is allocated the resources. On the other hand, by the service that is allocated the resources. On the other hand,
isolation at the packet layer allows the resources to be shared isolation at the packet layer allows the resources to be shared
among-st many services and only dedicated to a service on a temporary amongst many services and only dedicated to a service on a temporary
basis. This allows greater statistical multiplexing of network basis. This allows greater statistical multiplexing of network
resources and amortizes the cost over many services, leading to resources and amortizes the cost over many services, leading to
better economy. However, the degree of isolation required by network better economy. However, the degree of isolation required by network
slicing cannot easily be met with MPLS-TE packet LSPs. slicing cannot easily be met with MPLS-TE packet LSPs as they
guarantee long-term bandwidth, but not latency.
Thus some trade-off between the two approaches needs to be considered Thus some trade-off between the two approaches needs to be considered
to provide the required isolation between virtual networks while to provide the required isolation between virtual networks while
still allows reasonable sharing inside each VPN. The work of the still allows reasonable sharing inside each VPN.
IEEE project on Time Sensitive Networking is introducing the concept
of packet scheduling where a high priority packet stream may be given The work of the IEEE project on Time Sensitive Networking is
a scheduled time slot thereby guaranteeing that it experiences no introducing the concept of packet scheduling where a high priority
queuing delay and hence a reduced latency. However where no packet stream may be given a scheduled time slot thereby guaranteeing
scheduled packet arrives its reserved time-slot is handed over to that it experiences no queuing delay and hence a reduced latency.
best effort traffic, thereby improving the economics of the network. However where no scheduled packet arrives its reserved time-slot is
handed over to best effort traffic, thereby improving the economics
of the network. Such a scheduling mechanism may be usable directly,
or with extension to achieve isolation between multiple VPNs.
One of the key areas in which isolation needs to be provided is at One of the key areas in which isolation needs to be provided is at
the interfaces. If nothing is done the system falls back to the the interfaces. If nothing is done the system falls back to the
router queuing system in which the ingress places it on a selected router queuing system in which the ingress places it on a selected
output queue. Modern routers have quite sophisticated output queuing output queue. Modern routers have quite sophisticated output queuing
systems, but normally these do not provide the type of scheduling systems, traditionally these have not provided the type of scheduling
system needed to support the levels of isolation needed for the system needed to support the levels of isolation needed for the
applications that are the target of VPN+ networks. One alternative applications that are the target of VPN+ networks. However some of
approach is to employ a true time domain multiplexing system with the more modern approaches to queuing allow the construction of
fixed time elements allocated to a number of sub-interfaces. This is logical virtual channelized sub-interfaces (VCSI). With VCSIs there
the approach that FlexE uses. As noted elsewhere this produces hard is only one physical interface, and routing sees a single adjacency,
isolation but at the cost of making the reclamation of unused but the queuing system is used to provide virtual interfaces at
bandwidth harder. various priorities. Sophisticated queuing systems of this type may
be used to provide end-to-end virtual isolation between tenant's
Another approach, pursued in the Time Sensitive Networking (TSN) traffic in an otherwise homogeneous network.
space, is to time schedule the transmission of one, or a small number
of packets from a queue dedicated to a time-slot. Such an approach
appears to offer greater flexibility for the reclamation of unused
bandwidth, thereby improving the economics of the system.
These approaches can usefully be used in tandem. It is possible to [FLEXE] provides the ability to multiplex multiple channels over an
use FlexE to provide tenant isolation, and then to use the TSN Ethernet link in a way that provides hard isolation. However it is a
approach over FlexE to provide service performance guarantee inside only a link technology. When packets are received by the downstream
the a slice/tenant VPN. node they need to be processed in a way that preserves that
isolation. This in turn requires a queuing and forwarding
implementation that preserves the isolation, such as a sliced
hardware system, or an LVI system of the type described above.
3.2. Guaranteed Performance 3.2. Diverse Performance Guarantees
There are several aspects to guaranteed performance, guaranteed There are several aspects to guaranteed performance, guaranteed
maximum packet loss, guaranteed maximum delay and guaranteed delay maximum packet loss, guaranteed maximum delay and guaranteed delay
variation. variation.
Guaranteed maximum packet loss is a common parameter, and is usually Guaranteed maximum packet loss is a common parameter, and is usually
addressed by setting the packet priorities, queue size and discard addressed by setting the packet priorities, queue size and discard
policy. However this becomes more difficult when the requirement is policy. However this becomes more difficult when the requirement is
combine with the latency requirement. The limiting case is zero combine with the latency requirement. The limiting case is zero
congestion loss, and than is the goal of the Deterministic Networking congestion loss, and than is the goal of the Deterministic Networking
skipping to change at page 6, line 17 skipping to change at page 6, line 39
Guaranteed maximum delay variation is a service that may also be Guaranteed maximum delay variation is a service that may also be
needed. Time transfer is one example of a service that needs this, needed. Time transfer is one example of a service that needs this,
although the fungible nature of time means that it might be delivered although the fungible nature of time means that it might be delivered
by the underlay as a shared service and not provided through by the underlay as a shared service and not provided through
different virtual networks. Alternatively a dedicated virtual different virtual networks. Alternatively a dedicated virtual
network may be used to provide this as a shared service. The need network may be used to provide this as a shared service. The need
for guaranteed maximum delay variation as a general requirement is for guaranteed maximum delay variation as a general requirement is
for further study. for further study.
This leads to the concept that there is a spectrum of grades of
service guarantee that need to be considered when deploying and
enhanced VPN. As a guide to understanding the design requirements we
can consider four types:
o Guaranteed latency,
o Enhanced delivery
o Assured bandwidth,
o Best effort
In Section 3.1 we considered the work of the IEEE Time Sensitive
Networking (TSN) project and the work of the IETF DetNet Working
group in the context of isolation. However this work is of greater
relevance in assuring end-to-end packet latency. It is also of
importance in considering enhanced delivery.
A service that is guaranteed latency has a latency upper bound
provided by the network. It is important to note that assuring the
upper bound is more important than achieving the minimum latency.
A service that is offered enhanced delivery is one in which the
network (at layer 3) attempts to deliver the packet through multiple
paths in the hope of avoiding transient congestion
[I-D.ietf-detnet-dp-sol].
A useful mechanism to provide these guarantees is to use Flex A useful mechanism to provide these guarantees is to use Flex
Ethernet [FLEXE] as the underlay. This is a method of bonding Ethernet [FLEXE] as the underlay. This is a method of bonding
Ethernets together and of providing time-slot based channelization Ethernets together and of providing time-slot based channelization
over an Ethernet bearer. Such channels are fully isolated from other over an Ethernet bearer. Such channels are fully isolated from other
channels running over the same Ethernet bearer. channels running over the same Ethernet bearer. As noted elsewhere
this produces hard isolation but at the cost of making the
reclamation of unused bandwidth harder.
3.3. Integration These approaches can usefully be used in tandem. It is possible to
use FlexE to provide tenant isolation, and then to use the TSN
approach over FlexE to provide service performance guarantee inside
the a slice/tenant VPN.
3.3. A Pragmatic Approach to Isolation
A key question to consider is whether whether it is possible to
achieve hard isolation in packet networks? Packet networks were
never designed to support hard isolation, just the opposite, they
were designed to provide a high degree of statistical multiplexing
and hence a significant economic advantage when compared to a
dedicated, or a Time Division Multiplexing (TDM) network. However
the key thing to bear in mind is that the concept of hard isolation
needs to be viewed from the perspective of the application, and there
is no need to provide any harder isolation than is required by the
application. From a historical perspective it is good to think about
pseudowires [RFC3985] which emulate services that in many would have
had hard isolation in their native form. However experience has
shown that in most cases an approximation to this requirement is
sufficient for most uses.
Thus, for example, using FlexE or channelized sub-interface,together
with packet scheduling as interface slicing, and optionally, also
together with the slicing of node resources (Network Processor Unit
(NPU), etc.), it may be possible to provide a type of hard isolation
that is adequate for many applications. Other applications may be
satisfied with a classical VPN and reserved bandwidth, but yet others
may require dedicated point to point fiber. The requirement is thus
to qualify the needs of each application and provide an economic
solution that satisfies those needs without over-engineering.
3.4. Integration
A solution to the enhanced VPN problem will need to provide seamless A solution to the enhanced VPN problem will need to provide seamless
integration of both physical and virtual network. Given the integration of both Overlay VPN and the underlay network resources.
targeting of both this technology and service function chaining at This needs be done in a flexible and scalable way so that it can be
mobile networks and in particular 5G the co-integration of service widely deployed in operator networks. Given the targeting of both
functions is a likely requirement. this technology and service function chaining at mobile networks and
in particular 5G the co-integration of service functions is a likely
requirement.
3.4. Dynamic Configuration 3.5. Dynamic Configuration
It is necessary that new enhanced VPNs can be introduced to the It is necessary that new enhanced VPNs can be introduced to the
network, modified, and removed from the network. In doing so due network, modified, and removed from the network according to service
regard must be given to the impact of other enhanced VPNs that are demand. In doing so due regard must be given to the impact of other
operational. An enhanced VPN that requires hard isolation must not enhanced VPNs that are operational. An enhanced VPN that requires
be disrupted by the installation or modification of another enhanced hard isolation must not be disrupted by the installation or
VPN. modification of another enhanced VPN.
Whether modification of an enhanced VPN can be disruptive to that Whether modification of an enhanced VPN can be disruptive to that
VPN, and in particular the traffic in flight is to be determined, but VPN, and in particular the traffic in flight is to be determined, but
is likely to be a difficult problem to address. is likely to be a difficult problem to address.
The data-plane aspect of this are discussed further in Section 7. The data-plane aspect of this are discussed further in Section 4.3.
The control-plane aspects of this, particularly the garbage The control-plane and management-plane aspects of this, particularly
collection are likely to be challenging and are for further study. the garbage collection are likely to be challenging and are for
further study.
3.5. Customized Control Plane As well as managing dynamic changes to the VPN in a seamless way,
dynamic changes to the underlay and its transport network need to be
managed in order to avoid disruption to sensitive services.
In addition to non-disruptively managing the network as a result of
gross change such as the inclusion of a new VPN endpoint or a change
to a link, consideration has to be given to the need to move VPN
traffic as a result of traffic volume changes.
3.6. Customized Control Plane
In some cases it is desirable that an enhanced VPN has a custom In some cases it is desirable that an enhanced VPN has a custom
control plane, so that the tenant of the enhanced VPN can have some control-plane, so that the tenant of the enhanced VPN can have some
control to the resources and functions partitioned for this VPN. control to the resources and functions partitioned for this VPN.
Each enhanced VPN may have its own dedicated controller, or be Each enhanced VPN may have its own dedicated controller, it may be
provided with an interface to the control plane of the underlay. provided with an interface to a control-plane that is shared with a
set of other tenants, or it may be provided with an interface to the
control-plane of the underlay provided by the underlay network
operator.
Further detail on this requirement will be provided in a future Further detail on this requirement will be provided in a future
version of the draft. version of the draft.
4. Architecture and Components of VPN+ 4. Architecture and Components of VPN+
VPN+ runs over a substrate or underlay that it draws on to provide Normally a number of enhanced VPN services will be provided by a
the resources and features needed to provide enhanced VPN services to common network infrastructure. Each enhanced VPN consists of both
its tenants. The assumption is that a number of such enhanced VPNs the overlay and a specific set of dedicated network resources and
are layered on this underlay, each drawing the resources that they functions allocated in the underlay to satisfy the needs of the VPN
need to satisfy the needs of their tenants. Thus each enhanced VPN tenant. The integration between overlay and underlay ensures the
is bound to a specific set of resources allocated from the underlay, isolation and between different enhanced VPNs, and facilitates the
with different subsets of the underlay resources dedicated to guaranteed performance for different services.
different enhanced VPNs. The consequence of this is that any VPN+
solution needs tighter coupling to underlay that is the case with
classical VPNs.
An enhanced VPN needs to be designed with consideration given to: An enhanced VPN needs to be designed with consideration given to:
o The layering of the VPN+ data-plane onto the substrate. o Isolation of enhanced VPN data plane.
o The amount of static and dynamic state. o A scalable control plane to match the data plane isolation.
o The ammount of state in the packet vs the am-mount of state in the o The amount of state in the packet vs the amount of state in the
control plane. control plane.
o How sufficient isolation is achieved between VPN+ instances and o Mechanism for diverse performance guarantee within an enhanced VPN
between VPN+ instances and the best effort traffic.
o How the required latency demands are achieved.
o Support of the required integration between network functions and o Support of the required integration between network functions and
service functions. service functions.
o The design of the control plane. 4.1. Communications Layering
4.1. Data-Plane The communications layering model use to build an enhanced VPN is
shown in Figure 1.
The data-plane is required to provide each VPN+ with paths through Tenant Tenant connection Tenant
the specific resources allocated to it. This requires a finer CE1 ----------------------------------------CE2
granularity of packet steering than is normally provided in networks. \ /
AC \ OP Provider VPN OP /AC
+- PE1------------------------------PE1 -+
Enhanced Path
==============================
Underlay
++++++++++++++++++++++++++++++
One of the candidate approaches is to use segment routing. This is Figure 1: Communication Layering
discussed Section 4.1.2.
4.1.1. Data-plane Layering The network operator is required to provide a tenant connection
between the tenant's Customer Equipment (CE) (CE1 and CE2). These
CEs attach to the Operator's Provider Edge Equipments (PE) (PE1 and
PE2 respectively). The attachment circuits (AC) are outside the
scope of this document other than to note that they obviously need to
provide a connection of sufficient quality in terms of isolation,
latency etc so as to satisfy the needs of the user. The subtlety to
be aware of is that the ACs are often provided by a network rather
than a fixed point to point connection and thus the considerations in
this document may apply to the network that provides the AC.
An enhanced VPN needs to run on a substrate or underlay that can draw A provider VPN is constructed between PE1 and PE2 to carry tenant
on to provide the resources and features it needs. In order to meet traffic. This is a normal VPN, and provides one stage of isolation
the isolation requirement, network resources in the underlay need to between tenants.
split into different subsets, which are dedicated to different VPNs.
For VPNs which require hard isolation, at least the underlay tunnels
cannot be shared. In a scalable solution we need to layer a number
of VPNs on the underlay, and for each enhanced VPN to draws on the
resources provided by the underlay to deliver a service with the
required properties.
Different subsets of the underlay resources are dedicated to An enhanced path is constructed to carry the provider VPN using
different VPNs. The VPN+ solution needs tighter coupling with dedicated resources drawn from the underlay.
underlay. We cannot for example share the tunnel between enhanced
VPNs which require hard isolation.
4.1.2. Use of Segment Routing Constructs 4.2. Multi-Point to Multi-point
Clearly we can use traditional constructs to create a VPN, but there At a VPN level connections are frequently multi-point-to-multi-point
are advantages to the use of other constructs such as Segment Routing (MP2MP). As far as such services are concerned the underlay is also
(SR) in the creation of virtual networks with enhanced properties. an abstract MP2MP medium. However when service guarantees are
provided, such as with an enhanced VPN, each point to point path
through the underlay needs to be specifically engineered to meet the
required performance guarantees.
4.3. Candidate Underlay Technologies
A VPN is a network created by applying a multiplexing technique to
the underlying network (the underlay) in order to distinguish the
traffic of one VPN from that of another. A VPN path that travels by
other than the shortest path through the underlay normally requires
state in the underlay to specify that path. State is normally
applied to the underlay through the use of the RSVP Signaling
protocol, or directly through the use of an SDN controller, although
other techniques may emerge as this problem is studied. This state
gets harder to manage as the number of VPN paths increases.
Furthermore, as we increase the coupling between the underlay and the
overlay to support the VPN which requires enhanced VPN service, this
state will increase further.
In an enhanced VPN different subsets of the underlay resources are
dedicated to different VPNs. Any enhanced VPN solution thus needs
tighter coupling with underlay than is the case with classical VPNs.
We cannot for example share the tunnel between enhanced VPNs which
require hard isolation.
In the following sections we consider a number of candidate underlay
solutions for proving the required VPN separation.
o FlexE
o Time Sensitive Networking
o Deterministic Networking
o Dedicated Queues
We then consider the problem of slice differentiation and resource
representation. Candidate technologies are:
o MPLS
o MPLS-SR
o Segment Routing over IPv6 (SRv6)
4.3.1. FlexE
FlexE [FLEXE] is a method of creating a point-to-point Ethernet with
a specific fixed bandwidth. FlexE supports the bonding of multiple
links, which supports creating larger links out of multiple slower
links in a more efficient way that traditional link aggregation.
FlexE also supports the sub-rating of links, which allows an operator
to only use a portion of a link. FlexE also supports the
channelization of links, which allows one link to carry several
lower-speed or sub-rated links from different sources.
If different FlexE channels are used for different services, then no
sharing is possible between the services. This in turn means that it
is not possible to dynamically re-distribute unused bandwidth to
lower priority services increasing the cost of operation of the
network. FlexE can on the other hand be used to provide hard
isolation between different tenants by providing hard isolation on an
interface. The tenant can then use other methods to manage the
relative priority of their own traffic.
Methods of dynamically re-sizing FlexE channels and the implication
for enhanced VPN are under study.
4.3.2. Dedicated Queues
In an enhanced VPN providing multiple isolated virtual networks the
conventional Diff-Serv based queuing system is insufficient for our
purposes due to the limited number of queues which cannot
differentiate between traffic of different VPNs and the range of
service classes that each need to provide their tenants. This
problem is particularly acute with an MPLS underlay due to the small
number of traffic class services available. In order to address this
problem and thus reduce the interference between VPNs, it is likely
to be necessary to steer traffic of VPNs to dedicated input and
output queues.
4.3.3. Time Sensitive Networking
Time Sensitive Networking (TSN) is an IEEE project that is designing
a method of carrying time sensitive information over Ethernet. As
Ethernet this can obviously be tunneled over a Layer 3 network in a
pseudowire. However the TSN payload would be opaque to the underlay
and thus not treated specifically as time sensitive data. The
preferred method of carrying TSN over a layer 3 network is through
the use of deterministic networking as explained in the following
section of this document.
The machanisms defined in TSN can be used to meet the requirements of
time sensitive services of an enhanced VPN.
4.3.4. Deterministic Networking
Deterministic Networking (DetNet) [I-D.ietf-detnet-architecture] is a
technique being developed in the IETF to enhance the ability of layer
3 networks to deliver packets more reliably and with greater control
over the delay. The design cannot use classical re-transmission
techniques such as TCP since can add delay that is above the maximum
tolerated by the applications. Even the delay improvements that are
achieved with SCTP-PR are outside the bounds set by application
demands. The approach is to pre-emptively send copies of the packet
over various paths in the expectation that this minimizes the chance
of all packets being lost, but to trim duplicate packets to prevent
excessive flooding of the network and to prevent multiple packets
being delivered to the destination. It also seeks to set an upper
bound on latency. Note that it is not the goal to minimize latency,
and the optimum upper bound paths may not be the minimum latency
paths.
DetNet is based on flows. It currently makes no comment on the
underlay, and so at this stage must be assumed to use the base
topology. To be of use in this application DetNet there needs to be
a description of how to deal with the concept of flows within an
enhanced VPN.
How we use DetNet in a multi-tenant (VPN) network, and how to improve
the scalability of DetNet in a multi-tenant (VPN) network is for
further study.
4.3.5. MPLS Traffic Engineering (MPLS-TE)
Normal MPLS runs on the base topology and has the concepts of
reserving end to end bandwidth for an LSP, and of creating VPNs. VPN
traffic can be run over RSVP-TE tunnels to provide reserved bandwidth
for a specific VPN connection. This is rarely deployed in practice
due to scaling and management overhead concerns.
4.3.6. Segment Routing
Segment Routing [I-D.ietf-spring-segment-routing] is a method that Segment Routing [I-D.ietf-spring-segment-routing] is a method that
prepends instructions to packets at entry and sometimes at various prepends instructions to packets at entry and sometimes at various
points as it passes though the network. These instructions allow points as it passes though the network. These instructions allow
packets to be routed on paths other than the shortest path for packets to be routed on paths other than the shortest path for
various traffic engineering reasons. These paths can be strict or various traffic engineering reasons. These paths can be strict or
loose paths, depending on the compactness required of the instruction loose paths, depending on the compactness required of the instruction
list and the degree of autonomy granted to the network (for example list and the degree of autonomy granted to the network (for example
to support ECMP). to support ECMP).
With SR, a path needs to be dynamically created through a set of With SR, a path needs to be dynamically created through a set of
resources by simply specifying the Segment IDs (SIDs), i.e. resources by simply specifying the Segment IDs (SIDs), i.e.
instructions rooted at a particular point in the network. Thus if a instructions rooted at a particular point in the network. Thus if a
path is to be provisioned from some ingress point A to some egress path is to be provisioned from some ingress point A to some egress
point B in the underlay, A is provided with the A..B SID list and point B in the underlay, A is provided with the A..B SID list and
instructions on how to identify the packets to which the SID list is instructions on how to identify the packets to which the SID list is
to be prepended. to be prepended.
4.1.3. Segment Routing and Isolation
With current segment routing, the instructions are used to specify
the nodes and links to be traversed. However, in order to achieve
the required isolation between different services, new instructions
can be created which can be prepended to a packet to steer it through
specific dedicated network resources and functions, e.g. links,
queues, processors, services etc.
4.2. Stateful and Stateless Virtual Networks
A VPN is a network created by applying a multiplexing technique to
the underlying network (the underlay) in order to distinguish the
traffic of one VPN from that of another. A VPN path that travels by
other than the shortest path through the underlay normally requires
state in the underlay to specify that path. State is normally
applied to the underlay through the use of the RSVP Signaling
protocol, or directly through the use of an SDN controller, although
other techniques may emerge as this problem is studied. This state
gets harder to manage as the number of VPN paths increases.
Furthermore, as we increase the coupling between the underlay and the
overlay to support the VPN which requires enhanced VPN service, this
state will increase further.
By encoding the state in the packet, as is done in Segment Routing, By encoding the state in the packet, as is done in Segment Routing,
state is transitioned out of the network. state is transitioned out of the network.
A-------B-----E A-------B-----E
| | | | | |
| | | | | |
C-------D-----+ C-------D-----+
Figure 1: An SR Network Fragment Figure 2: An SR Network Fragment
Consider the network fragment shown in Figure 1. To send a packet Consider the network fragment shown in Figure 2. To send a packet
from A to E via B, D & E: Node A prepends the ordered list of SIDs:D, from A to E via B, D & E: Node A prepends the ordered list of SIDs:D,
E to the packet and pushes the packet to B. SID list {B, D, E} can E to the packet and pushes the packet to B. SID list {B, D, E} can
be used as a VPN path. Thus, to create a VPN, a set of SID Lists is be used as a VPN path. Thus, to create a VPN, a set of SID Lists is
created and provided to each ingress node of the VPN together with created and provided to each ingress node of the VPN together with
packet selection criteria. In this way it is possible to create a packet selection criteria. In this way it is possible to create a
VPN with no state in the core. However this is at the expense of VPN with no state in the core. However this is at the expense of
creating a larger packet with possible MTU and hardware restriction creating a larger packet with possible MTU and hardware restriction
limits that need to be overcome. limits that need to be overcome.
Note in the above if A and E support multiple VPN an additional VPN Note in the above if A and E support multiple VPN an additional VPN
identifier will need to be added to the packet, but this is omitted identifier will need to be added to the packet, but this is omitted
from this text for simplicity. from this text for simplicity.
A---P---B---S---E A---P---B---S---E
| | | | | |
| Q | | Q |
| | | | | |
C---R---D-------+ C---R---D-------+
Figure 2: Another SR Network Fragment Figure 3: Another SR Network Fragment
Consider a further network fragment shown in Figure 2, and further Consider a further network fragment shown in Figure 3, and further
consider VPN A+D+E. consider VPN A+D+E.
A has lists: {P, B, Q, D}, {P, B, S, E} A has lists: {P, B, Q, D}, {P, B, S, E}
D has lists: {Q, B, P, A}, {E} D has lists: {Q, B, P, A}, {E}
E has lists: {S, B, P, A}, {D} E has lists: {S, B, P, A}, {D}
To create a new VPN C+D+B the following list are introduced: To create a new VPN C+D+B the following list are introduced:
C lists: {R, D}, {A, P, B} C lists: {R, D}, {A, P, B}
D lists: {R, C}, {Q, B} D lists: {R, C}, {Q, B}
B lists: {Q, D}, {P, A, C} B lists: {Q, D}, {P, A, C}
Thus VPN C+D+B was created without touching the settings of the core Thus VPN C+D+B was created without touching the settings of the core
routers, indeed it is possible to add endpoints to the VPNs, and move routers, indeed it is possible to add endpoints to the VPNs, and move
the paths around simply by providing new lists to the affected the paths around simply by providing new lists to the affected
endpoints. endpoints.
When SR is extended to support isolation finer granularity state There are a number of limitations in SR as it is currently defined
needs to be added to the core in anticipation of its use. We that limit its applicability to enhanced VPNs:
therefore need to evaluate the balance between this additional state
and the performance delivered by the network.
4.3. Latency Support o Segments are shared between different VPNs,
The IETF has ongoing work on support for a latency ceiling o There is no reservation of bandwidth,
[I-D.ietf-detnet-architecture]. The provision of a latency ceiling
is a requirement of the application seeking the use of enhanced o There is limited differentiation in the data plane.
virtual networks. The current design of DetNet assumes the design of
the underlay network is unchanged. In this document we look at some Thus some extensions to SR are needed to provide isolation between
changes that could be used to assist in achieving low latency ceiling different enhanced VPNs. This can be achieved by including a finer
across the wide area. granularity of state in the core in anticipation of its future use by
authorized services. We therefore need to evaluate the balance
between this additional state and the performance delivered by the
network.
Both MPLS Segment Routing and SRv6 Segment Routing are candidate
technologies for enhanced VPN.
With current segment routing, the instructions are used to specify
the nodes and links to be traversed. However, in order to achieve
the required isolation between different services, new instructions
can be created which can be prepended to a packet to steer it through
specific dedicated network resources and functions, e.g. links,
queues, processors, services etc.
Clearly we can use traditional constructs to create a VPN, but there
are advantages to the use of other constructs such as Segment Routing
(SR) in the creation of virtual networks with enhanced properties.
Traditionally a traffic engineered path operates with a granularity Traditionally a traffic engineered path operates with a granularity
of a link with hints about priority provided through the use of the of a link with hints about priority provided through the use of the
traffic class field in the header. However to achieve the latency traffic class field in the header. However to achieve the latency
and isolation characteristics that are sought by VPN+ users, steering and isolation characteristics that are sought by VPN+ users, steering
packets through specific queues resources will likely be required. packets through specific queues resources will likely be required.
The extent to which these needs can be satisfied through existing QoS The extent to which these needs can be satisfied through existing QoS
mechanisms is to be determined. What is clear is that a fine control mechanisms is to be determined. What is clear is that a fine control
of which services wait for which, with a fine granularity of queue of which services wait for which, with a fine granularity of queue
management policy is needed. Note that the concept of a queue is a management policy is needed. Note that the concept of a queue is a
skipping to change at page 11, line 35 skipping to change at page 16, line 24
dequeuing. dequeuing.
This concept can be further generalized, since as well as queuing to This concept can be further generalized, since as well as queuing to
the output port of a router, it is possible to queue to any resource, the output port of a router, it is possible to queue to any resource,
for example: for example:
o A network processor unit (NPU) o A network processor unit (NPU)
o A Central Processing Unit (CPU) Core o A Central Processing Unit (CPU) Core
o A Look-up engine such as TCAM o A Look-up engine such as TCAMs
4.4. Integrating Service Function Chains and VPNs
There is a significant overlap between the problem of routing a
packet though a set of network resources and the problem of routing a
packet through a set of compute resources. Service Function Chain
technology is designed to forward a packet through a set of compute
resources.
A future version of this document will discuss this further.
4.5. Application Specific Network Types
Although the transport service that underpins the extended VPN is
likely MPLS/IP based, it needs to be able to carry application
specific non-MPLS/IP traffic. This can be accommodated through the
use of pseudowires (PWs).
4.6. Control Plane Considerations 4.4. Control Plane Considerations
It is expected that VPN+ would be based on a hybrid control It is expected that VPN+ would be based on a hybrid control
mechanism, which takes advantage of the logically centralized mechanism, which takes advantage of the logically centralized
controller for on-demand provisioning and global optimization, whilst controller for on-demand provisioning and global optimization, whilst
still relies on distributed control plane to provide scalability, still relies on distributed control plane to provide scalability,
high reliability, fast reaction, automatic failure recovery etc. high reliability, fast reaction, automatic failure recovery etc.
Extension and optimization to the distributed control plane is needed Extension and optimization to the distributed control plane is needed
to support the enhanced properties of VPN+. to support the enhanced properties of VPN+.
Where SR is used as a the data-plane construct it needs to be noted Where SR is used as a the data-plane construct it needs to be noted
that it does not have the capability of reserving resources along the that it does not have the capability of reserving resources along the
path nor do its currently specified distributed control plane (the path nor do its currently specified distributed control plane (the
link state routing protocols). An SDN controller can clearly do link state routing protocols). An SDN controller can clearly do
this, from the controllers point of view, and no resource reservation this, from the controllers point of view, and no resource reservation
is done on the device. Thus if a distributed control plane is needed is done on the device. Thus if a distributed control plane is needed
either in place of an SDN controller or as an assistant to it there either in place of an SDN controller or as an assistant to it, the
is a risk of resource conflict. This needs further study. design of the control system needs to ensure that resources are
uniquely allocated to the correct service, and no allocated to
multiple services casing unintended resource conflict. This needs
further study.
On the other hand an advantage of using an SR approach is that it On the other hand an advantage of using an SR approach is that it
provides a way of efficiently binding the network underlay and the provides a way of efficiently binding the network underlay and the
enhanced VPN overlay. With a technology such as RSVP-TE LSPs, each enhanced VPN overlay. With a technology such as RSVP-TE LSPs, each
virtual path in the VPN is bound to the underlay with a dedicated TE- virtual path in the VPN is bound to the underlay with a dedicated TE-
LSP. LSP.
RSVP-TE could be enhanced to bind the VPN to specific resources RSVP-TE could be enhanced to bind the VPN to specific resources
within the underlay, but as noted elsewhere in this document there within the underlay, but as noted elsewhere in this document there
are concerns as to the scalability of this approach. With an SR- are concerns as to the scalability of this approach. With an SR-
based approach to resource reservation (per-slice reservation), it is based approach to resource reservation (per-slice reservation), it is
straightforward to create dedicated SR network slices, and the VPN straightforward to create dedicated SR network slices, and the VPN
can be bound to a particular SR network slice. can be bound to a particular SR network slice.
4.5. Application Specific Network Types
Although a lot of the traffic that will be carried over the enhanced
VPN will likely be IPv4 or IPv6, the design has to be capable of
carrying other traffic types. In particular the design SHOULD be
capable of carrying Ethernet traffic. This is easily accomplished
through the various pseudowire (PW) techniques [RFC3985]. Where the
underlay is MPLS Ethernet can be carried over the enhanced VPN
encapsulated according to the method specified in [RFC4448]. Where
the underlay is IP Layer Two Tunneling Protocol - Version 3 (L2TPv3)
[RFC3931] can be used with Ethernet traffic carried according to
[RFC4719]. Encapsulations have been defined for most of the common
layer two type for both PW over MPLS and for L2TPv3.
4.6. Integration with Service Functions
There is a significant overlap between the problem of routing a
packet though a set of network resources and the problem of routing a
packet through a set of compute resources. Service Function Chain
technology is designed to forward a packet through a set of compute
resources.
A future version of this document will discuss this further.
5. Scalability Considerations 5. Scalability Considerations
For a packet to transit a network, other than on a best effort, For a packet to transit a network, other than on a best effort,
shortest path basis, it is necessary to introduce additional state, shortest path basis, it is necessary to introduce additional state,
either in the packet, or in the network of some combination of both. either in the packet, or in the network of some combination of both.
There are at least three ways of doing this: There are at least three ways of doing this:
o Introduce the complete state into the packet. That is how SR does o Introduce the complete state into the packet. That is how SR does
this, and this allows the controller to specify the precise series this, and this allows the controller to specify the precise series
skipping to change at page 14, line 4 skipping to change at page 18, line 48
adding state to the network and minimizing stack depth, or minimizing adding state to the network and minimizing stack depth, or minimizing
state and increasing the stack depth. state and increasing the stack depth.
5.2. RSVP scalability 5.2. RSVP scalability
The traditional method of creating a resource allocated path through The traditional method of creating a resource allocated path through
an MPLS network is to use the RSVP protocol. However there have been an MPLS network is to use the RSVP protocol. However there have been
concerns that this requires significant continuous state maintenance concerns that this requires significant continuous state maintenance
in the network. There are ongoing works to improve the scalability in the network. There are ongoing works to improve the scalability
of RSVP-TE LSPs in the control plane of RSVP-TE LSPs in the control plane
[I-D.ietf-teas-rsvp-te-scaling-rec]. This will be considered further [I-D.ietf-teas-rsvp-te-scaling-rec]. This will be considered further
in a future version of this document. in a future version of this document.
There is also concern at the scalability of the forwarder footprint There is also concern at the scalability of the forwarder footprint
of RSVP as the number of paths through an LSR grows of RSVP as the number of paths through an LSR grows
[I-D.sitaraman-mpls-rsvp-shared-labels] proposes to address this by [I-D.sitaraman-mpls-rsvp-shared-labels] proposes to address this by
employing SR within a tunnel established by RSVP-TE. This work will employing SR within a tunnel established by RSVP-TE. This work will
be considered in a future version of this document. be considered in a future version of this document.
6. OAM and Instrumentation 6. OAM and Instrumentation
This will be discussed in a future version of this draft. A A study of OAM in SR networks has been documented in
discussion should include [I-D.ietf-spring-oam-usecase].
o Instrumentation of the underlay. The enhanced VPN OAM design needs to consider the following
requirements:
o Instrumentation of the overlay by both customer and provider. o Instrumentation of the underlay so that the network operator can
be sure that the resources committed to a tenant are operating
correctly and delivering the required performance.
o Instrumentation of the overlay by the tenant. This is likely to
be transparent to the network operator and to use existing
methods. Particular consideration needs to be given to the need
to verify the isolation and the various committed performance
characteristics.
o Instrumentation of the overlay by the network provider to
proactively demonstrate that the committed performance is being
delivered. This needs to be done in a non-intrusive manner,
particularly when the tenant is deploying a performance sensitive
application
o Verification of the conformity of the path to the service o Verification of the conformity of the path to the service
requirement. requirement. This may need to be done as part of a commissioning
test.
7. Service Disruption During Change These issues will be discussed in a future version of this document.
7. Enhanced Resiliency
Each enhanced VPN, of necessity, has a life-cycle, and needs Each enhanced VPN, of necessity, has a life-cycle, and needs
modification during deployment as the needs of its user change. modification during deployment as the needs of its user change.
Additionally as the network as a whole evolves there will need to be Additionally as the network as a whole evolves there will need to be
garbage collection performed to consolidate resources into usable garbage collection performed to consolidate resources into usable
quanta. quanta.
Systems in which the path is imposed such as SR, or some form of Systems in which the path is imposed such as SR, or some form of
explicit routing tend to do well in these applications because it is explicit routing tend to do well in these applications because it is
possible to perform an atomic transition from one path to another. possible to perform an atomic transition from one path to another.
skipping to change at page 15, line 41 skipping to change at page 21, line 11
9. IANA Considerations 9. IANA Considerations
There are no requested IANA actions. There are no requested IANA actions.
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, <https://www.rfc- DOI 10.17487/RFC2119, March 1997,
editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
10.2. Informative References 10.2. Informative References
[FLEXE] "Flex Ethernet Implementation Agreement", March 2016, [FLEXE] "Flex Ethernet Implementation Agreement", March 2016,
<http://www.oiforum.com/wp-content/uploads/ <http://www.oiforum.com/wp-content/uploads/
OIF-FLEXE-01.0.pdf>. OIF-FLEXE-01.0.pdf>.
[I-D.geng-netslices-architecture]
67, 4., Dong, J., Bryant, S., kiran.makhijani@huawei.com,
k., Galis, A., Foy, X., and S. Kuklinski, "Network Slicing
Architecture", draft-geng-netslices-architecture-02 (work
in progress), July 2017.
[I-D.ietf-detnet-architecture] [I-D.ietf-detnet-architecture]
Finn, N., Thubert, P., Varga, B., and J. Farkas, Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", draft-ietf- "Deterministic Networking Architecture", draft-ietf-
detnet-architecture-03 (work in progress), August 2017. detnet-architecture-04 (work in progress), October 2017.
[I-D.ietf-detnet-dp-sol]
Korhonen, J., Andersson, L., Jiang, Y., Finn, N., Varga,
B., Farkas, J., Bernardos, C., Mizrahi, T., and L. Berger,
"DetNet Data Plane Encapsulation", draft-ietf-detnet-dp-
sol-01 (work in progress), January 2018.
[I-D.ietf-spring-oam-usecase]
Geib, R., Filsfils, C., Pignataro, C., and N. Kumar, "A
Scalable and Topology-Aware MPLS Dataplane Monitoring
System", draft-ietf-spring-oam-usecase-10 (work in
progress), December 2017.
[I-D.ietf-spring-segment-routing] [I-D.ietf-spring-segment-routing]
Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B.,
Litkowski, S., and R. Shakir, "Segment Routing Litkowski, S., and R. Shakir, "Segment Routing
Architecture", draft-ietf-spring-segment-routing-13 (work Architecture", draft-ietf-spring-segment-routing-15 (work
in progress), October 2017. in progress), January 2018.
[I-D.ietf-teas-rsvp-te-scaling-rec] [I-D.ietf-teas-rsvp-te-scaling-rec]
Beeram, V., Minei, I., Shakir, R., Pacella, D., and T. Beeram, V., Minei, I., Shakir, R., Pacella, D., and T.
Saad, "Techniques to Improve the Scalability of RSVP Saad, "Techniques to Improve the Scalability of RSVP
Traffic Engineering Deployments", draft-ietf-teas-rsvp-te- Traffic Engineering Deployments", draft-ietf-teas-rsvp-te-
scaling-rec-07 (work in progress), September 2017. scaling-rec-09 (work in progress), February 2018.
[I-D.sitaraman-mpls-rsvp-shared-labels] [I-D.sitaraman-mpls-rsvp-shared-labels]
Sitaraman, H., Beeram, V., Parikh, T., and T. Saad, Sitaraman, H., Beeram, V., Parikh, T., and T. Saad,
"Signaling RSVP-TE tunnels on a shared MPLS forwarding "Signaling RSVP-TE tunnels on a shared MPLS forwarding
plane", draft-sitaraman-mpls-rsvp-shared-labels-02 (work plane", draft-sitaraman-mpls-rsvp-shared-labels-03 (work
in progress), September 2017. in progress), December 2017.
[NETCALC] "Applicability of Network Calculus to DetNet", November
2017, <https://datatracker.ietf.org/meeting/100/materials/
slides-100-detnet-applicability-of-network-calculus-to-
detnet>.
[RFC3931] Lau, J., Ed., Townsley, M., Ed., and I. Goyret, Ed.,
"Layer Two Tunneling Protocol - Version 3 (L2TPv3)",
RFC 3931, DOI 10.17487/RFC3931, March 2005,
<https://www.rfc-editor.org/info/rfc3931>.
[RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation
Edge-to-Edge (PWE3) Architecture", RFC 3985,
DOI 10.17487/RFC3985, March 2005,
<https://www.rfc-editor.org/info/rfc3985>.
[RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron,
"Encapsulation Methods for Transport of Ethernet over MPLS
Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006,
<https://www.rfc-editor.org/info/rfc4448>.
[RFC4719] Aggarwal, R., Ed., Townsley, M., Ed., and M. Dos Santos,
Ed., "Transport of Ethernet Frames over Layer 2 Tunneling
Protocol Version 3 (L2TPv3)", RFC 4719,
DOI 10.17487/RFC4719, November 2006,
<https://www.rfc-editor.org/info/rfc4719>.
Authors' Addresses Authors' Addresses
Stewart Bryant Stewart Bryant
Huawei Huawei
Email: stewart.bryant@gmail.com Email: stewart.bryant@gmail.com
Jie Dong Jie Dong
Huawei Huawei
Email: jie.dong@huawei.com Email: jie.dong@huawei.com
Zhenqiang Li
China Mobile
Email: lizhenqiang@chinamobile.com
Takuya Miyasaka
KDDI Corporation
Email: ta-miyasaka@kddi.com
 End of changes. 71 change blocks. 
225 lines changed or deleted 510 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/