< draft-ietf-nvo3-overlay-problem-statement-02.txt   draft-ietf-nvo3-overlay-problem-statement-03.txt >
Internet Engineering Task Force T. Narten, Ed. Internet Engineering Task Force T. Narten, Ed.
Internet-Draft IBM Internet-Draft IBM
Intended status: Informational E. Gray, Ed. Intended status: Informational E. Gray, Ed.
Expires: August 11, 2013 Ericsson Expires: November 11, 2013 Ericsson
D. Black D. Black
EMC EMC
D. Dutt
Cumulus Networks
L. Fang L. Fang
Cisco Systems
L. Kreeger L. Kreeger
Cisco Cisco
M. Napierala M. Napierala
AT&T AT&T
M. Sridharan May 10, 2013
Microsoft
February 7, 2013
Problem Statement: Overlays for Network Virtualization Problem Statement: Overlays for Network Virtualization
draft-ietf-nvo3-overlay-problem-statement-02 draft-ietf-nvo3-overlay-problem-statement-03
Abstract Abstract
This document describes issues associated with providing multi- This document describes issues associated with providing multi-
tenancy in large data center networks and how these issues may be tenancy in large data center networks and how these issues may be
addressed using an overlay-based network virtualization approach. A addressed using an overlay-based network virtualization approach. A
key multi-tenancy requirement is traffic isolation, so that one key multi-tenancy requirement is traffic isolation, so that one
tenant's traffic is not visible to any other tenant. Another tenant's traffic is not visible to any other tenant. Another
requirement is address space isolation, so that different tenants can requirement is address space isolation, so that different tenants can
use the same address space within different virtual networks. use the same address space within different virtual networks.
skipping to change at page 2, line 13 skipping to change at page 2, line 10
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 11, 2013. This Internet-Draft will expire on November 11, 2013.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 11 skipping to change at page 3, line 11
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6
3. Problem Areas . . . . . . . . . . . . . . . . . . . . . . . . 6 3. Problem Areas . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. Need For Dynamic Provisioning . . . . . . . . . . . . . . 6 3.1. Need For Dynamic Provisioning . . . . . . . . . . . . . . 6
3.2. Virtual Machine Mobility Limitations . . . . . . . . . . . 6 3.2. Virtual Machine Mobility Limitations . . . . . . . . . . . 7
3.3. Inadequate Forwarding Table Sizes . . . . . . . . . . . . 7 3.3. Inadequate Forwarding Table Sizes . . . . . . . . . . . . 7
3.4. Need to Decouple Logical and Physical Configuration . . . 7 3.4. Need to Decouple Logical and Physical Configuration . . . 7
3.5. Need For Address Separation Between Virtual Networks . . . 8 3.5. Need For Address Separation Between Virtual Networks . . . 8
3.6. Need For Address Separation Between Virtual Networks 3.6. Need For Address Separation Between Virtual Networks
and Infrastructure . . . . . . . . . . . . . . . . . . . . 8 and Infrastructure . . . . . . . . . . . . . . . . . . . . 8
3.7. Optimal Forwarding . . . . . . . . . . . . . . . . . . . . 8 3.7. Optimal Forwarding . . . . . . . . . . . . . . . . . . . . 9
4. Using Network Overlays to Provide Virtual Networks . . . . . . 9 4. Using Network Overlays to Provide Virtual Networks . . . . . . 10
4.1. Overview of Network Overlays . . . . . . . . . . . . . . . 10 4.1. Overview of Network Overlays . . . . . . . . . . . . . . . 10
4.2. Communication Between Virtual and Non-virtualized 4.2. Communication Between Virtual and Non-virtualized
Networks . . . . . . . . . . . . . . . . . . . . . . . . . 11 Networks . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.3. Communication Between Virtual Networks . . . . . . . . . . 12 4.3. Communication Between Virtual Networks . . . . . . . . . . 12
4.4. Overlay Design Characteristics . . . . . . . . . . . . . . 12 4.4. Overlay Design Characteristics . . . . . . . . . . . . . . 13
4.5. Control Plane Overlay Networking Work Areas . . . . . . . 13 4.5. Control Plane Overlay Networking Work Areas . . . . . . . 14
4.6. Data Plane Work Areas . . . . . . . . . . . . . . . . . . 14 4.6. Data Plane Work Areas . . . . . . . . . . . . . . . . . . 15
5. Related IETF and IEEE Work . . . . . . . . . . . . . . . . . . 15 5. Related IETF and IEEE Work . . . . . . . . . . . . . . . . . . 15
5.1. BGP/MPLS IP VPNs . . . . . . . . . . . . . . . . . . . . . 15 5.1. BGP/MPLS IP VPNs . . . . . . . . . . . . . . . . . . . . . 15
5.2. BGP/MPLS Ethernet VPNs . . . . . . . . . . . . . . . . . . 15 5.2. BGP/MPLS Ethernet VPNs . . . . . . . . . . . . . . . . . . 16
5.3. 802.1 VLANs . . . . . . . . . . . . . . . . . . . . . . . 16 5.3. 802.1 VLANs . . . . . . . . . . . . . . . . . . . . . . . 16
5.4. IEEE 802.1aq - Shortest Path Bridging . . . . . . . . . . 16 5.4. IEEE 802.1aq - Shortest Path Bridging . . . . . . . . . . 17
5.5. ARMD . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.5. VDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.6. TRILL . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.6. ARMD . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.7. L2VPNs . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.7. TRILL . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.8. Proxy Mobile IP . . . . . . . . . . . . . . . . . . . . . 18 5.8. L2VPNs . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.9. LISP . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.9. Proxy Mobile IP . . . . . . . . . . . . . . . . . . . . . 18
5.10. VDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.10. LISP . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6. Further Work . . . . . . . . . . . . . . . . . . . . . . . . . 18 6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 19
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 19 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 19
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
10. Security Considerations . . . . . . . . . . . . . . . . . . . 19 10. Security Considerations . . . . . . . . . . . . . . . . . . . 20
11. Informative References . . . . . . . . . . . . . . . . . . . . 20 11. Informative References . . . . . . . . . . . . . . . . . . . . 20
Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 21 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 22
A.1. Changes From -01 to -02 . . . . . . . . . . . . . . . . . 21 A.1. Changes From -02 to -03 . . . . . . . . . . . . . . . . . 22
A.2. Changes From -00 to -01 . . . . . . . . . . . . . . . . . 21 A.2. Changes From -01 to -02 . . . . . . . . . . . . . . . . . 22
A.3. Changes from A.3. Changes From -00 to -01 . . . . . . . . . . . . . . . . . 22
draft-narten-nvo3-overlay-problem-statement-04.txt . . . . 22 A.4. Changes from
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22 draft-narten-nvo3-overlay-problem-statement-04.txt . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23
1. Introduction 1. Introduction
Data Centers are increasingly being consolidated and outsourced in an Data Centers are increasingly being consolidated and outsourced in an
effort to improve the deployment time of applications and reduce effort to improve the deployment time of applications and reduce
operational costs. This coincides with an increasing demand for operational costs. This coincides with an increasing demand for
compute, storage, and network resources from applications. In order compute, storage, and network resources from applications. In order
to scale compute, storage, and network resources, physical resources to scale compute, storage, and network resources, physical resources
are being abstracted from their logical representation, in what is are being abstracted from their logical representation, in what is
referred to as server, storage, and network virtualization. referred to as server, storage, and network virtualization.
skipping to change at page 6, line 7 skipping to change at page 6, line 7
solutions). Some deployments require an L2 service, others an L3 solutions). Some deployments require an L2 service, others an L3
service, and some may require both. service, and some may require both.
Section 2 gives terminology. Section 3 describes the problem space Section 2 gives terminology. Section 3 describes the problem space
details. Section 4 describes overlay networks in more detail. details. Section 4 describes overlay networks in more detail.
Sections 5 and 6 review related and further work, while Section 7 Sections 5 and 6 review related and further work, while Section 7
closes with a summary. closes with a summary.
2. Terminology 2. Terminology
This document uses the same terminology as This document uses the same terminology as [I-D.ietf-nvo3-framework].
[I-D.lasserre-nvo3-framework]. In addition, this document use the In addition, this document use the following terms.
following terms.
In-Band Virtual Network: A Virtual Network that separates tenant In-Band Virtual Network: A Virtual Network that separates tenant
traffic without hiding tenant forwarding information from the traffic without hiding tenant forwarding information from the
physical infrastructure. The Tenant System may also retain physical infrastructure. The Tenant System may also retain
visibility of a tenant within the underlying physical visibility of a tenant within the underlying physical
infrastructure. IEEE 802.1 networks using C-VIDs are an example infrastructure. IEEE 802.1Q-1998 networks only using C-VIDs are
of an in-band Virtual Network. an example of an in-band Virtual Network.
Overlay Virtual Network: A Virtual Network in which the separation Overlay Virtual Network: A Virtual Network in which the separation
of tenants is hidden from the underlying physical infrastructure. of tenants is hidden from the underlying physical infrastructure.
That is, the underlying transport network does not need to know That is, the underlying transport network does not need to know
about tenancy separation to correctly forward traffic. about tenancy separation to correctly forward traffic. IEEE 802.1
Provider Backbone Bridging (PBB) [IEEE-802.1Q] is an example of an
L2 Overlay Network. PBB uses MAC-in-MAC encapsulation and the
underlying transport network forwards traffic using only the B-MAC
and B-VID in the outer header. The underlay transport network is
unaware of the tenancy separation provided by, for example, a 24-
bit I-SID.
VLANs: An informal term referring to IEEE 802.1 networks using C-VLAN: This document refers to C-VLANs as implemented by many
C-VIDs. routers, i.e., an L2 virtual network identified by a C-VID. An
end station (e.g., a VM) in this context that is part of an L2
virtual network will effectively belong to a C-VLAN. Within an
IEEE 802.1Q-2011 network, other tags may be used as well, but such
usage is generally not visible to the end station. Section 5.3
provides more details on VLANs defined by [802.1Q].
3. Problem Areas 3. Problem Areas
The following subsections describe aspects of multi-tenant data The following subsections describe aspects of multi-tenant data
center networking that pose problems for network infrastructure. center networking that pose problems for network infrastructure.
Different problem aspects may arise based on the network architecture Different problem aspects may arise based on the network architecture
and scale. and scale.
3.1. Need For Dynamic Provisioning 3.1. Need For Dynamic Provisioning
skipping to change at page 7, line 13 skipping to change at page 7, line 23
including its IP and MAC address(es). Preservation of MAC addresses including its IP and MAC address(es). Preservation of MAC addresses
may be necessary, for example, when software licenses are bound to may be necessary, for example, when software licenses are bound to
MAC addresses. More generally, any change in the VM's MAC addresses MAC addresses. More generally, any change in the VM's MAC addresses
resulting from a move would be visible to the VM and thus potentially resulting from a move would be visible to the VM and thus potentially
result in unexpected disruptions. Retaining IP addresses after a result in unexpected disruptions. Retaining IP addresses after a
move is necessary to prevent existing transport connections (e.g., move is necessary to prevent existing transport connections (e.g.,
TCP) from breaking and needing to be restarted. TCP) from breaking and needing to be restarted.
In data center networks, servers are typically assigned IP addresses In data center networks, servers are typically assigned IP addresses
based on their physical location, for example based on the Top of based on their physical location, for example based on the Top of
Rack (ToR) switch for the server rack or the VLAN configured to the Rack (ToR) switch for the server rack or the C-VLAN configured to the
server. Servers can only move to other locations within the same IP server. Servers can only move to other locations within the same IP
subnet. This constraint is not problematic for physical servers, subnet. This constraint is not problematic for physical servers,
which move infrequently, but it restricts the placement and movement which move infrequently, but it restricts the placement and movement
of VMs within the data center. Any solution for a scalable multi- of VMs within the data center. Any solution for a scalable multi-
tenant data center must allow a VM to be placed (or moved) anywhere tenant data center must allow a VM to be placed (or moved) anywhere
within the data center, without being constrained by the subnet within the data center, without being constrained by the subnet
boundary concerns of the host servers. boundary concerns of the host servers.
3.3. Inadequate Forwarding Table Sizes 3.3. Inadequate Forwarding Table Sizes
Today's virtualized environments place additional demands on the Today's virtualized environments place additional demands on the
forwarding tables of forwarding nodes in the physical infrastructure. forwarding tables of forwarding nodes in the physical infrastructure.
The core problem is that location independence results in specific The core problem is that location independence results in specific
end state information being propagated into the forwarding system end state information being propagated into the forwarding system
(e.g., /32 host routes in L3 networks, or MAC addresses in L2 (e.g., /32 host routes in L3 networks, or MAC addresses in L2
networks). In L2 networks, for instance, instead of just one link- networks). In L2 networks, for instance, instead of just one address
layer address per server, the switching infrastructure may have to per server, the network infrastructure may have to learn addresses of
learn addresses of the individual VMs (which could range in the 100s the individual VMs (which could range in the 100s per server). This
per server). This increases the demand on a forwarding node's table increases the demand on a forwarding node's table capacity compared
capacity compared to non-virtualized environments. to non-virtualized environments.
3.4. Need to Decouple Logical and Physical Configuration 3.4. Need to Decouple Logical and Physical Configuration
Data center operators must be able to achieve high utilization of Data center operators must be able to achieve high utilization of
server and network capacity. For efficient and flexible allocation, server and network capacity. For efficient and flexible allocation,
operators should be able to spread a virtual network instance across operators should be able to spread a virtual network instance across
servers in any rack in the data center. It should also be possible servers in any rack in the data center. It should also be possible
to migrate compute workloads to any server anywhere in the network to migrate compute workloads to any server anywhere in the network
while retaining the workload's addresses. In networks using VLANs, while retaining the workload's addresses.
In networks of many types (e.g., IP subnets, MPLS VPNs, VLANs, etc.)
moving servers elsewhere in the network may require expanding the moving servers elsewhere in the network may require expanding the
scope of the VLAN beyond its original boundaries. While this can be scope of a portion of the network (e.g., subnet, VPN, VLAN, etc.)
done, it requires potentially complex network configuration changes beyond its original boundaries. While this can be done, it requires
and can conflict with the desire to bound the size of broadcast potentially complex network configuration changes and may (in some
domains, especially in larger data centers. In addition, when VMs cases - e.g., a VLAN or L2VPN) conflict with the desire to bound the
migrate, the physical network (e.g., access lists) may need to be size of broadcast domains. In addition, when VMs migrate, the
reconfigured which can be time consuming and error prone. physical network (e.g., access lists) may need to be reconfigured
which can be time consuming and error prone.
An important use case is cross-pod expansion. A pod typically An important use case is cross-pod expansion. A pod typically
consists of one or more racks of servers with associated network and consists of one or more racks of servers with associated network and
storage connectivity. A tenant's virtual network may start off on a storage connectivity. A tenant's virtual network may start off on a
pod and, due to expansion, require servers/VMs on other pods, pod and, due to expansion, require servers/VMs on other pods,
especially the case when other pods are not fully utilizing all their especially the case when other pods are not fully utilizing all their
resources. This use case requires that virtual networks span resources. This use case requires that virtual networks span
multiple pods in order to provide connectivity to all of its tenant's multiple pods in order to provide connectivity to all of its tenant's
servers/VMs. Such expansion can be difficult to achieve when tenant servers/VMs. Such expansion can be difficult to achieve when tenant
addressing is tied to the addressing used by the underlay network or addressing is tied to the addressing used by the underlay network or
when the expansion requires that the scope of the underlying L2 VLAN when the expansion requires that the scope of the underlying C-VLAN
expand beyond its original pod boundary. expand beyond its original pod boundary.
3.5. Need For Address Separation Between Virtual Networks 3.5. Need For Address Separation Between Virtual Networks
Individual tenants need control over the addresses they use within a Individual tenants need control over the addresses they use within a
virtual network. But it can be problematic when different tenants virtual network. But it can be problematic when different tenants
want to use the same addresses, or even if the same tenant wants to want to use the same addresses, or even if the same tenant wants to
reuse the same addresses in different virtual networks. reuse the same addresses in different virtual networks.
Consequently, virtual networks must allow tenants to use whatever Consequently, virtual networks must allow tenants to use whatever
addresses they want without concern for what addresses are being used addresses they want without concern for what addresses are being used
skipping to change at page 8, line 38 skipping to change at page 9, line 7
As in the previous case, a tenant needs to be able to use whatever As in the previous case, a tenant needs to be able to use whatever
addresses it wants in a virtual network independent of what addresses addresses it wants in a virtual network independent of what addresses
the underlying data center network is using. Tenants (and the the underlying data center network is using. Tenants (and the
underlay infrastructure provider) should be able use whatever underlay infrastructure provider) should be able use whatever
addresses make sense for them, without having to worry about address addresses make sense for them, without having to worry about address
collisions between addresses used by tenants and those used by the collisions between addresses used by tenants and those used by the
underlay data center network. underlay data center network.
3.7. Optimal Forwarding 3.7. Optimal Forwarding
Another problem area relates to the routing of traffic into and out Another problem area relates to the optimal forwarding of traffic
of a virtual network. A virtual network may have two routers for between peers that are not connected to the same virtual network.
traffic to/from other VNs or external to all VNs, and the optimal Such forwarding happens when a host on a virtual network communicates
choice of router may depend on where the VM is located. The two with a host not on any virtual network (e.g., an Internet host) as
routers may not be equally "close" to a given VM. The issue appears well as when a host on a virtual network communicates with a host on
both when a VM is initially instantiated on a virtual network or when a different virtual network. A virtual network may have two (or
a VM migrates or is moved to a different location. After a more) gateways for forwarding traffic onto and off of the virtual
migration, the VM's closest router for such traffic may change, i.e., network and the optimal choice of which gateway to use may depend on
the VM may get better service by switching to the "closer" router, the set of available paths between the communicating peers. The set
and this may improve the utilization of network resources. of available gateways may not be equally "close" to a given
destination. The issue appears both when a VM is initially
instantiated on a virtual network or when a VM migrates or is moved
to a different location. After a migration, for instance, a VM's
best-choice gateway for such traffic may change, i.e., the VM may get
better service by switching to the "closer" gateway, and this may
improve the utilization of network resources.
IP implementations in network endpoints typically do not distinguish IP implementations in network endpoints typically do not distinguish
between multiple routers on the same subnet - there may only be a between multiple routers on the same subnet - there may only be a
single default gateway in use, and any use of multiple routers single default gateway in use, and any use of multiple routers
usually considers all of them to be one-hop away. Routing protocol usually considers all of them to be one-hop away. Routing protocol
functionality is constrained by the requirement to cope with these functionality is constrained by the requirement to cope with these
endpoint limitations - for example VRRP has one router serve as the endpoint limitations - for example VRRP has one router serve as the
master to handle all outbound traffic. This problem can be master to handle all outbound traffic. This problem can be
particularly acute when the virtual network spans multiple data particularly acute when the virtual network spans multiple data
centers, as a VM is likely to receive significantly better service centers, as a VM is likely to receive significantly better service
skipping to change at page 11, line 23 skipping to change at page 11, line 46
packet. packet.
A key aspect of overlays is the decoupling of the "virtual" MAC A key aspect of overlays is the decoupling of the "virtual" MAC
and/or IP addresses used by VMs from the physical network and/or IP addresses used by VMs from the physical network
infrastructure and the infrastructure IP addresses used by the data infrastructure and the infrastructure IP addresses used by the data
center. If a VM changes location, the overlay edge devices simply center. If a VM changes location, the overlay edge devices simply
update their mapping tables to reflect the new location of the VM update their mapping tables to reflect the new location of the VM
within the data center's infrastructure space. Because an overlay within the data center's infrastructure space. Because an overlay
network is used, a VM can now be located anywhere in the data center network is used, a VM can now be located anywhere in the data center
that the overlay reaches without regards to traditional constraints that the overlay reaches without regards to traditional constraints
imposed by the underlay network such as the L2 VLAN scope, or the IP imposed by the underlay network such as the C-VLAN scope, or the IP
subnet scope. subnet scope.
Multi-tenancy is supported by isolating the traffic of one virtual Multi-tenancy is supported by isolating the traffic of one virtual
network instance from traffic of another. Traffic from one virtual network instance from traffic of another. Traffic from one virtual
network instance cannot be delivered to another instance without network instance cannot be delivered to another instance without
(conceptually) exiting the instance and entering the other instance (conceptually) exiting the instance and entering the other instance
via an entity (e.g., a gateway) that has connectivity to both virtual via an entity (e.g., a gateway) that has connectivity to both virtual
network instances. Without the existence of a gateway entity, tenant network instances. Without the existence of a gateway entity, tenant
traffic remains isolated within each individual virtual network traffic remains isolated within each individual virtual network
instance. instance.
skipping to change at page 12, line 38 skipping to change at page 13, line 13
network functionality (e.g., load balancing, firewall support) that network functionality (e.g., load balancing, firewall support) that
is applied to traffic forwarded between virtual networks. is applied to traffic forwarded between virtual networks.
4.4. Overlay Design Characteristics 4.4. Overlay Design Characteristics
Below are some of the characteristics of environments that must be Below are some of the characteristics of environments that must be
taken into account by the overlay technology. taken into account by the overlay technology.
1. Highly distributed systems: The overlay should work in an 1. Highly distributed systems: The overlay should work in an
environment where there could be many thousands of access environment where there could be many thousands of access
switches (e.g. residing within the hypervisors) and many more switches (e.g., residing within the hypervisors) and many more
Tenant Systems (e.g. VMs) connected to them. This leads to a Tenant Systems (e.g., VMs) connected to them. This leads to a
distributed mapping system that puts a low overhead on the distributed mapping system that puts a low overhead on the
overlay tunnel endpoints. overlay tunnel endpoints.
2. Many highly distributed virtual networks with sparse membership: 2. Many highly distributed virtual networks with sparse membership:
Each virtual network could be highly dispersed inside the data Each virtual network could be highly dispersed inside the data
center. Also, along with expectation of many virtual networks, center. Also, along with expectation of many virtual networks,
the number of end systems connected to any one virtual network is the number of end systems connected to any one virtual network is
expected to be relatively low; Therefore, the percentage of NVEs expected to be relatively low; Therefore, the percentage of NVEs
participating in any given virtual network would also be expected participating in any given virtual network would also be expected
to be low. For this reason, efficient delivery of multi- to be low. For this reason, efficient delivery of multi-
skipping to change at page 13, line 48 skipping to change at page 14, line 25
build mapping tables entirely via learning (as is done in 802.1 build mapping tables entirely via learning (as is done in 802.1
networks). Another approach is to use a specialized control plane networks). Another approach is to use a specialized control plane
protocol. While there are some advantages to using or leveraging an protocol. While there are some advantages to using or leveraging an
existing protocol for maintaining mapping tables, the fact that large existing protocol for maintaining mapping tables, the fact that large
numbers of NVE's will likely reside in hypervisors places constraints numbers of NVE's will likely reside in hypervisors places constraints
on the resources (cpu and memory) that can be dedicated to such on the resources (cpu and memory) that can be dedicated to such
functions. functions.
From an architectural perspective, one can view the address mapping From an architectural perspective, one can view the address mapping
dissemination problem as having two distinct and separable dissemination problem as having two distinct and separable
components. The first component consists of a back-end "oracle" that components. The first component consists of a back-end Network
is responsible for distributing and maintaining the mapping Virtualization Authority (NVA) that is responsible for distributing
information for the entire overlay system. For this document, we use and maintaining the mapping information for the entire overlay
the term "oracle" in its generic sense, referring to an entity that system. For this document, we use the term NVA to refer to an entity
supplies answers, without regard to how it knows the answers it is that supplies answers, without regard to how it knows the answers it
providing. The second component consists of the on-the-wire is providing. The second component consists of the on-the-wire
protocols an NVE uses when interacting with the oracle. protocols an NVE uses when interacting with the NVA.
The back-end oracle could provide high performance, high resiliency, The back-end NVA could provide high performance, high resiliency,
failover, etc. and could be implemented in significantly different failover, etc. and could be implemented in significantly different
ways. For example, one model uses a traditional, centralized ways. For example, one model uses a traditional, centralized
"directory-based" database, using replicated instances for "directory-based" database, using replicated instances for
reliability and failover. A second model involves using and possibly reliability and failover. A second model involves using and possibly
extending an existing routing protocol (e.g., BGP, IS-IS, etc.). To extending an existing routing protocol (e.g., BGP, IS-IS, etc.). To
support different architectural models, it is useful to have one support different architectural models, it is useful to have one
standard protocol for the NVE-oracle interaction while allowing standard protocol for the NVE-NVA interaction while allowing
different protocols and architectural approaches for the oracle different protocols and architectural approaches for the NVA itself.
itself. Separating the two allows NVEs to transparently interact Separating the two allows NVEs to transparently interact with
with different types of oracles, i.e., either of the two different types of NVAs, i.e., either of the two architectural models
architectural models described above. Having separate protocols described above. Having separate protocols could also allow for a
could also allow for a simplified NVE that only interacts with the simplified NVE that only interacts with the NVA for the mapping table
oracle for the mapping table entries it needs and allows the oracle entries it needs and allows the NVA (and its associated protocols) to
(and its associated protocols) to evolve independently over time with evolve independently over time with minimal impact to the NVEs.
minimal impact to the NVEs.
A third work area considers the attachment and detachment of VMs (or A third work area considers the attachment and detachment of VMs (or
Tenant Systems [I-D.lasserre-nvo3-framework] more generally) from a Tenant Systems [I-D.ietf-nvo3-framework] more generally) from a
specific virtual network instance. When a VM attaches, the NVE specific virtual network instance. When a VM attaches, the NVE
associates the VM with a specific overlay for the purposes of associates the VM with a specific overlay for the purposes of
tunneling traffic sourced from or destined to the VM. When a VM tunneling traffic sourced from or destined to the VM. When a VM
disconnects, the NVE should notify the oracle that the Tenant System disconnects, the NVE should notify the NVA that the Tenant System to
to NVE address mapping is no longer valid. In addition, if this VM NVE address mapping is no longer valid. In addition, if this VM was
was the last remaining member of the virtual network, then the NVE the last remaining member of the virtual network, then the NVE can
can also terminate any tunnels used to deliver tenant multi- also terminate any tunnels used to deliver tenant multi-destination
destination packets within the VN to the NVE. In the case where an packets within the VN to the NVE. In the case where an NVE and
NVE and hypervisor are on separate physical devices separated by an hypervisor are on separate physical devices separated by an access
access network, a standardized protocol may be needed. network, a standardized protocol may be needed.
In summary, there are three areas of potential work. The first area In summary, there are three areas of potential work. The first area
concerns the implementation of the oracle function itself and any concerns the implementation of the NVA function itself and any
protocols it needs (e.g., if implemented in a distributed fashion). protocols it needs (e.g., if implemented in a distributed fashion).
A second area concerns the interaction between the oracle and NVEs. A second area concerns the interaction between the NVA and NVEs. The
The third work area concerns protocols associated with attaching and third work area concerns protocols associated with attaching and
detaching a VM from a particular virtual network instance. All three detaching a VM from a particular virtual network instance. All three
work areas are important to the development of scalable, work areas are important to the development of scalable,
interoperable solutions. interoperable solutions.
4.6. Data Plane Work Areas 4.6. Data Plane Work Areas
The data plane carries encapsulated packets for Tenant Systems. The The data plane carries encapsulated packets for Tenant Systems. The
data plane encapsulation header carries a VN Context identifier data plane encapsulation header carries a VN Context identifier
[I-D.lasserre-nvo3-framework] for the virtual network to which the [I-D.ietf-nvo3-framework] for the virtual network to which the data
data packet belongs. Numerous encapsulation or tunneling protocols packet belongs. Numerous encapsulation or tunneling protocols
already exist that can be leveraged. In the absence of strong and already exist that can be leveraged. In the absence of strong and
compelling justification, it would not seem necessary or helpful to compelling justification, it would not seem necessary or helpful to
develop yet another encapsulation format just for NVO3. develop yet another encapsulation format just for NVO3.
5. Related IETF and IEEE Work 5. Related IETF and IEEE Work
The following subsections discuss related IETF and IEEE work. The The following subsections discuss related IETF and IEEE work. The
items are not meant to provide complete coverage of all IETF and IEEE items are not meant to provide complete coverage of all IETF and IEEE
data center related work, nor should the descriptions be considered data center related work, nor should the descriptions be considered
comprehensive. Each area aims to address particular limitations of comprehensive. Each area aims to address particular limitations of
skipping to change at page 15, line 38 skipping to change at page 16, line 15
network) and tenant IP addresses. Deployment of enterprise L3 VPNs network) and tenant IP addresses. Deployment of enterprise L3 VPNs
has been shown to scale to thousands of VPNs and millions of VPN has been shown to scale to thousands of VPNs and millions of VPN
prefixes. BGP/MPLS IP VPNs are currently deployed in some large prefixes. BGP/MPLS IP VPNs are currently deployed in some large
enterprise data centers. The potential limitation for deploying BGP/ enterprise data centers. The potential limitation for deploying BGP/
MPLS IP VPNs in data center environments is the practicality of using MPLS IP VPNs in data center environments is the practicality of using
BGP in the data center, especially reaching into the servers or BGP in the data center, especially reaching into the servers or
hypervisors. There may be computing work force skill set issues, hypervisors. There may be computing work force skill set issues,
equipment support issues, and potential new scaling challenges. A equipment support issues, and potential new scaling challenges. A
combination of BGP and lighter weight IP signaling protocols, e.g., combination of BGP and lighter weight IP signaling protocols, e.g.,
XMPP, have been proposed to extend the solutions into DC environment XMPP, have been proposed to extend the solutions into DC environment
[I-D.marques-l3vpn-end-system], while taking advantage of built-in [I-D.ietf-l3vpn-end-system], while taking advantage of built-in VPN
VPN features with its rich policy support; it is especially useful features with its rich policy support; it is especially useful for
for inter-tenant connectivity. inter-tenant connectivity.
5.2. BGP/MPLS Ethernet VPNs 5.2. BGP/MPLS Ethernet VPNs
Ethernet Virtual Private Networks (E-VPNs) [I-D.ietf-l2vpn-evpn] Ethernet Virtual Private Networks (E-VPNs) [I-D.ietf-l2vpn-evpn]
provide an emulated L2 service in which each tenant has its own provide an emulated L2 service in which each tenant has its own
Ethernet network over a common IP or MPLS infrastructure. A BGP/MPLS Ethernet network over a common IP or MPLS infrastructure. A BGP/MPLS
control plane is used to distribute the tenant MAC addresses and the control plane is used to distribute the tenant MAC addresses and the
MPLS labels that identify the tenants and tenant MAC addresses. MPLS labels that identify the tenants and tenant MAC addresses.
Within the BGP/MPLS control plane a thirty two bit Ethernet Tag is Within the BGP/MPLS control plane a 32-bit Ethernet Tag is used to
used to identify the broadcast domains (VLANs) associated with a identify the broadcast domains (VLANs) associated with a given L2
given L2 VLAN service instance and these Ethernet tags are mapped to VLAN service instance and these Ethernet tags are mapped to VLAN IDs
VLAN IDs understood by the tenant at the service edges. This means understood by the tenant at the service edges. This means that any
that the limit of 4096 VLANs is associated with an individual tenant customer site VLAN based limitation is associated with an individual
service edge, enabling a much higher level of scalability. tenant service edge, enabling a much higher level of scalability.
Interconnection between tenants is also allowed in a controlled Interconnection between tenants is also allowed in a controlled
fashion. fashion.
VM Mobility [I-D.raggarwa-data-center-mobility] introduces the VM Mobility [I-D.raggarwa-data-center-mobility] introduces the
concept of a combined L2/L3 VPN service in order to support the concept of a combined L2/L3 VPN service in order to support the
mobility of individual Virtual Machines (VMs) between Data Centers mobility of individual Virtual Machines (VMs) between Data Centers
connected over a common IP or MPLS infrastructure. connected over a common IP or MPLS infrastructure.
5.3. 802.1 VLANs 5.3. 802.1 VLANs
skipping to change at page 17, line 7 skipping to change at page 17, line 32
SPBM extends IS-IS in order to perform link-state routing among core SPBM extends IS-IS in order to perform link-state routing among core
SPBM nodes, obviating the need for learning for communication among SPBM nodes, obviating the need for learning for communication among
core SPBM nodes. Learning is still used to build and maintain the core SPBM nodes. Learning is still used to build and maintain the
mapping tables of edge nodes to encapsulate Tenant System traffic for mapping tables of edge nodes to encapsulate Tenant System traffic for
transport across the SPBM core. transport across the SPBM core.
SPB is compatible with all other 802.1 standards thus allows SPB is compatible with all other 802.1 standards thus allows
leveraging of other features, e.g., VSI Discovery Protocol (VDP), OAM leveraging of other features, e.g., VSI Discovery Protocol (VDP), OAM
or scalability solutions. or scalability solutions.
5.5. ARMD 5.5. VDP
VDP is the Virtual Station Interface (VSI) Discovery and
Configuration Protocol specified by IEEE P802.1Qbg [IEEE-802.1Qbg].
VDP is a protocol that supports the association of a VSI with a port.
VDP is run between the end system (e.g., a hypervisor) and its
adjacent switch, i.e., the device on the edge of the network. VDP is
used for example to communicate to the switch that a Virtual Machine
(Virtual Station) is moving, i.e., designed for VM migration.
5.6. ARMD
The ARMD WG examined data center scaling issues with a focus on The ARMD WG examined data center scaling issues with a focus on
address resolution and developed a problem statement document address resolution and developed a problem statement document
[RFC6820]. While an overlay-based approach may address some of the [RFC6820]. While an overlay-based approach may address some of the
"pain points" that were raised in ARMD (e.g., better support for "pain points" that were raised in ARMD (e.g., better support for
multi-tenancy), an overlay approach may also push some of the L2 multi-tenancy). Analysis will be needed to understand the scaling
scaling concerns (e.g., excessive flooding) to the IP level (flooding
via IP multicast). Analysis will be needed to understand the scaling
tradeoffs of an overlay based approach compared with existing tradeoffs of an overlay based approach compared with existing
approaches. On the other hand, existing IP-based approaches such as approaches. On the other hand, existing IP-based approaches such as
proxy ARP may help mitigate some concerns. proxy ARP may help mitigate some concerns.
5.6. TRILL 5.7. TRILL
TRILL is a network protocol that provides an Ethernet L2 service to TRILL is a network protocol that provides an Ethernet L2 service to
end systems and is designed to operate over any L2 link type. TRILL end systems and is designed to operate over any L2 link type. TRILL
establishes forwarding paths using IS-IS routing and encapsulates establishes forwarding paths using IS-IS routing and encapsulates
traffic within its own TRILL header. TRILL as defined today, traffic within its own TRILL header. TRILL as defined today,
supports only the standard (and limited) 12-bit C-VID identifier. supports only the standard (and limited) 12-bit C-VID identifier.
Approaches to extend TRILL to support more than 4094 VLANs are Approaches to extend TRILL to support more than 4094 VLANs are
currently under investigation [I-D.ietf-trill-fine-labeling] currently under investigation [I-D.ietf-trill-fine-labeling]
5.7. L2VPNs 5.8. L2VPNs
The IETF has specified a number of approaches for connecting L2 The IETF has specified a number of approaches for connecting L2
domains together as part of the L2VPN Working Group. That group, domains together as part of the L2VPN Working Group. That group,
however has historically been focused on Provider-provisioned L2 however has historically been focused on Provider-provisioned L2
VPNs, where the service provider participates in management and VPNs, where the service provider participates in management and
provisioning of the VPN. In addition, much of the target environment provisioning of the VPN. In addition, much of the target environment
for such deployments involves carrying L2 traffic over WANs. Overlay for such deployments involves carrying L2 traffic over WANs. Overlay
approaches as discussed in this document are intended be used within approaches as discussed in this document are intended be used within
data centers where the overlay network is managed by the data center data centers where the overlay network is managed by the data center
operator, rather than by an outside party. While overlays can run operator, rather than by an outside party. While overlays can run
skipping to change at page 18, line 6 skipping to change at page 18, line 39
Other L2VPN approaches, such as L2TP [RFC3931] require significant Other L2VPN approaches, such as L2TP [RFC3931] require significant
tunnel state at the encapsulating and decapsulating end points. tunnel state at the encapsulating and decapsulating end points.
Overlays require less tunnel state than other approaches, which is Overlays require less tunnel state than other approaches, which is
important to allow overlays to scale to hundreds of thousands of end important to allow overlays to scale to hundreds of thousands of end
points. It is assumed that smaller switches (i.e., virtual switches points. It is assumed that smaller switches (i.e., virtual switches
in hypervisors or the adjacent devices to which VMs connect) will be in hypervisors or the adjacent devices to which VMs connect) will be
part of the overlay network and be responsible for encapsulating and part of the overlay network and be responsible for encapsulating and
decapsulating packets. decapsulating packets.
5.8. Proxy Mobile IP 5.9. Proxy Mobile IP
Proxy Mobile IP [RFC5213] [RFC5844] makes use of the GRE Key Field Proxy Mobile IP [RFC5213] [RFC5844] makes use of the GRE Key Field
[RFC5845] [RFC6245], but not in a way that supports multi-tenancy. [RFC5845] [RFC6245], but not in a way that supports multi-tenancy.
5.9. LISP 5.10. LISP
LISP[RFC6830] essentially provides an IP over IP overlay where the LISP[RFC6830] essentially provides an IP over IP overlay where the
internal addresses are end station Identifiers and the outer IP internal addresses are end station Identifiers and the outer IP
addresses represent the location of the end station within the core addresses represent the location of the end station within the core
IP network topology. The LISP overlay header uses a 24-bit Instance IP network topology. The LISP overlay header uses a 24-bit Instance
ID used to support overlapping inner IP addresses. ID used to support overlapping inner IP addresses.
5.10. VDP 6. Summary
VDP is the Virtual Station Interface (VSI) Discovery and
Configuration Protocol specified by IEEE P802.1Qbg [Qbg]. VDP is a
protocol that supports the association of a VSI with a port. VDP is
run between the end system (e.g., a hypervisor) and its adjacent
switch, i.e., the device on the edge of the network. VDP is used for
example to communicate to the switch that a Virtual Machine (Virtual
Station) is moving, i.e. designed for VM migration.
6. Further Work
It is believed that overlay-based approaches may be able to reduce
the overall amount of flooding and other multicast and broadcast
related traffic (e.g, ARP and ND) currently experienced within
current data centers with a large flat L2 network. Further analysis
is needed to characterize expected improvements.
There are a number of VPN approaches that provide some if not all of
the desired semantics of virtual networks. A gap analysis will be
needed to assess how well existing approaches satisfy the
requirements.
7. Summary
This document has argued that network virtualization using overlays This document has argued that network virtualization using overlays
addresses a number of issues being faced as data centers scale in addresses a number of issues being faced as data centers scale in
size. In addition, careful study of current data center problems is size. In addition, careful study of current data center problems is
needed for development of proper requirements and standard solutions. needed for development of proper requirements and standard solutions.
This document identified three potential control protocol work areas. This document identified three potential control protocol work areas.
The first involves a backend Network Virtualization Authority and how
The first involves a backend "oracle" and how it learns and it learns and distributes the mapping information NVEs use when
distributes the mapping information NVEs use when processing tenant processing tenant traffic. A second involves the protocol an NVE
traffic. A second involves the protocol an NVE would use to would use to communicate with the backend NVA to obtain the mapping
communicate with the backend oracle to obtain the mapping
information. The third potential work concerns the interactions that information. The third potential work concerns the interactions that
take place when a VM attaches or detaches from an specific virtual take place when a VM attaches or detaches from an specific virtual
network instance. network instance.
8. Acknowledgments There are a number of approaches that provide some if not all of the
desired semantics of virtual networks. Each approach needs to be
analyzed in detail to assess how well it satisfies the requirements.
7. Acknowledgments
Helpful comments and improvements to this document have come from Lou Helpful comments and improvements to this document have come from Lou
Berger, John Drake, Janos Farkas, Ilango Ganga, Ariel Hendel, Vinit Berger, John Drake, Ilango Ganga, Ariel Hendel, Vinit Jain, Petr
Jain, Petr Lapukhov, Thomas Morin, Benson Schliesser, Xiaohu Xu, Lucy Lapukhov, Thomas Morin, Benson Schliesser, Qin Wu, Xiaohu Xu, Lucy
Yong and many others on the NVO3 mailing list. Yong and many others on the NVO3 mailing list.
Special thanks to Janos Farkas for his persistence and numerous
detailed comments related to the lack of precision in the text
relating to IEEE 802.1 technologies.
8. Contributors
Dinesh Dutt and Murari Sridharin were original co-authors of the
Internet-Draft that led to the BoF that formed the NVO3 WG. That
original draft eventually became the basis for the WG Problem
Statement document.
9. IANA Considerations 9. IANA Considerations
This memo includes no request to IANA. This memo includes no request to IANA.
10. Security Considerations 10. Security Considerations
Because this document describes the problem space associated with the Because this document describes the problem space associated with the
need for virtualization of networks in complex, large-scale, data- need for virtualization of networks in complex, large-scale, data-
center networks, it does not itself introduce any security risks. center networks, it does not itself introduce any security risks.
However, it is clear that security concerns need to be a However, it is clear that security concerns need to be a
skipping to change at page 20, line 10 skipping to change at page 20, line 33
plane. The prevention of the installation of improper control plane. The prevention of the installation of improper control
information, and other forms of denial of service are also concerns. information, and other forms of denial of service are also concerns.
Hereto, some environments may also be concerned about confidentiality Hereto, some environments may also be concerned about confidentiality
of the control plane. of the control plane.
11. Informative References 11. Informative References
[I-D.ietf-l2vpn-evpn] [I-D.ietf-l2vpn-evpn]
Sajassi, A., Aggarwal, R., Henderickx, W., Balus, F., Sajassi, A., Aggarwal, R., Henderickx, W., Balus, F.,
Isaac, A., and J. Uttaro, "BGP MPLS Based Ethernet VPN", Isaac, A., and J. Uttaro, "BGP MPLS Based Ethernet VPN",
draft-ietf-l2vpn-evpn-02 (work in progress), October 2012. draft-ietf-l2vpn-evpn-03 (work in progress),
February 2013.
[I-D.ietf-trill-fine-labeling] [I-D.ietf-l3vpn-end-system]
Eastlake, D., Zhang, M., Agarwal, P., Perlman, R., and D. Marques, P., Fang, L., Pan, P., Shukla, A., Napierala, M.,
Dutt, "TRILL: Fine-Grained Labeling", and N. Bitar, "BGP-signaled end-system IP/VPNs.",
draft-ietf-trill-fine-labeling-04 (work in progress), draft-ietf-l3vpn-end-system-01 (work in progress),
December 2012. April 2013.
[I-D.lasserre-nvo3-framework] [I-D.ietf-nvo3-framework]
Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y.
Rekhter, "Framework for DC Network Virtualization", Rekhter, "Framework for DC Network Virtualization",
draft-lasserre-nvo3-framework-03 (work in progress), draft-ietf-nvo3-framework-02 (work in progress),
July 2012. February 2013.
[I-D.marques-l3vpn-end-system] [I-D.ietf-trill-fine-labeling]
Marques, P., Fang, L., Pan, P., Shukla, A., Napierala, M., Eastlake, D., Zhang, M., Agarwal, P., Perlman, R., and D.
and N. Bitar, "BGP-signaled end-system IP/VPNs.", Dutt, "TRILL (Transparent Interconnection of Lots of
draft-marques-l3vpn-end-system-07 (work in progress), Links): Fine-Grained Labeling",
August 2012. draft-ietf-trill-fine-labeling-06 (work in progress),
March 2013.
[I-D.raggarwa-data-center-mobility] [I-D.raggarwa-data-center-mobility]
Aggarwal, R., Rekhter, Y., Henderickx, W., Shekhar, R., Aggarwal, R., Rekhter, Y., Henderickx, W., Shekhar, R.,
Fang, L., and A. Sajassi, "Data Center Mobility based on Fang, L., and A. Sajassi, "Data Center Mobility based on
E-VPN, BGP/MPLS IP VPN, IP Routing and NHRP", E-VPN, BGP/MPLS IP VPN, IP Routing and NHRP",
draft-raggarwa-data-center-mobility-04 (work in progress), draft-raggarwa-data-center-mobility-04 (work in progress),
December 2012. December 2012.
[Qbg] "IEEE P802.1Qbg Edge Virtual Bridging", February 2012. [IEEE-802.1Q]
IEEE 802.1Q-2011, "IEEE standard for local and
metropolitan area networks: Media access control (MAC)
bridges and virtual bridged local area networks,",
August 2011.
[IEEE-802.1Qbg]
IEEE 802.1Qbg-2012, "IEEE standard for local and
metropolitan area networks: Media access control (MAC)
bridges and virtual bridged local area networks --
Amendment 21: Edge virtual bridging,", July 2012.
[RFC3931] Lau, J., Townsley, M., and I. Goyret, "Layer Two Tunneling [RFC3931] Lau, J., Townsley, M., and I. Goyret, "Layer Two Tunneling
Protocol - Version 3 (L2TPv3)", RFC 3931, March 2005. Protocol - Version 3 (L2TPv3)", RFC 3931, March 2005.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, February 2006. Networks (VPNs)", RFC 4364, February 2006.
[RFC5213] Gundavelli, S., Leung, K., Devarapalli, V., Chowdhury, K., [RFC5213] Gundavelli, S., Leung, K., Devarapalli, V., Chowdhury, K.,
and B. Patil, "Proxy Mobile IPv6", RFC 5213, August 2008. and B. Patil, "Proxy Mobile IPv6", RFC 5213, August 2008.
skipping to change at page 21, line 22 skipping to change at page 22, line 10
Specification", RFC 6325, July 2011. Specification", RFC 6325, July 2011.
[RFC6820] Narten, T., Karir, M., and I. Foo, "Address Resolution [RFC6820] Narten, T., Karir, M., and I. Foo, "Address Resolution
Problems in Large Data Center Networks", RFC 6820, Problems in Large Data Center Networks", RFC 6820,
January 2013. January 2013.
[RFC6830] Farinacci, D., Fuller, V., Meyer, D., and D. Lewis, "The [RFC6830] Farinacci, D., Fuller, V., Meyer, D., and D. Lewis, "The
Locator/ID Separation Protocol (LISP)", RFC 6830, Locator/ID Separation Protocol (LISP)", RFC 6830,
January 2013. January 2013.
[SPB] "IEEE P802.1aq/D4.5 Draft Standard for Local and [SPB] IEEE 802.1aq, "IEEE standard for local and metropolitan
Metropolitan Area Networks -- Media Access Control (MAC) area networks: Media access control (MAC) bridges and
Bridges and Virtual Bridged Local Area Networks, virtual bridged local area networks -- Amendment 20:
Amendment 8: Shortest Path Bridging", February 2012. Shortest path bridging,", June 2012.
Appendix A. Change Log Appendix A. Change Log
A.1. Changes From -01 to -02 A.1. Changes From -02 to -03
1. Comments from Janos Farkas, including:
* Defined C-VLAN and changed VLAN -> C-VLAN where appropriate.
* Improved references to IEEE work.
* Removed Section "Further Work".
2. Improved first paragraph in "Optimal Forwarding" Section (per Qin
Wu).
3. Replaced "oracle" term with Network Virtualization Authority, to
match terminology discussion on list.
4. Reduced number of authors to 6. Still above the usual guideline
of 5, but chairs will ask for exception in this case.
A.2. Changes From -01 to -02
1. Security Considerations changes (Lou Berger) 1. Security Considerations changes (Lou Berger)
2. Changes to section on Optimal Forwarding (Xuxiaohu) 2. Changes to section on Optimal Forwarding (Xuxiaohu)
3. More wording improvements in L2 details (Janos Farkas) 3. More wording improvements in L2 details (Janos Farkas)
4. Referennces to ARMD and LISP documets are now RFCs. 4. References to ARMD and LISP documents are now RFCs.
A.2. Changes From -00 to -01 A.3. Changes From -00 to -01
1. Numerous editorial and clarity improvements. 1. Numerous editorial and clarity improvements.
2. Picked up updated terminology from the framework document (e.g., 2. Picked up updated terminology from the framework document (e.g.,
Tenant System). Tenant System).
3. Significant changes regarding IEEE 802.1 Ethernets and VLANs. 3. Significant changes regarding IEEE 802.1 Ethernets and VLANs.
All text moved to the Related Work section, where the technology All text moved to the Related Work section, where the technology
is summarized. is summarized.
skipping to change at page 22, line 21 skipping to change at page 23, line 28
that IP is assumed as the underlay transport in the data center. that IP is assumed as the underlay transport in the data center.
6. Added new section (2.6) on Optimal Forwarding. 6. Added new section (2.6) on Optimal Forwarding.
7. Added a section on Data Plane issues. 7. Added a section on Data Plane issues.
8. Significant improvement to Section describing SPBM. 8. Significant improvement to Section describing SPBM.
9. Added sub-section on VDP in "Related Work" 9. Added sub-section on VDP in "Related Work"
A.3. Changes from draft-narten-nvo3-overlay-problem-statement-04.txt A.4. Changes from draft-narten-nvo3-overlay-problem-statement-04.txt
1. This document has only one substantive change relative to 1. This document has only one substantive change relative to
draft-narten-nvo3-overlay-problem-statement-04.txt. Two draft-narten-nvo3-overlay-problem-statement-04.txt. Two
sentences were removed per the discussion that led to WG adoption sentences were removed per the discussion that led to WG adoption
of this document. of this document.
Authors' Addresses Authors' Addresses
Thomas Narten (editor) Thomas Narten (editor)
IBM IBM
skipping to change at page 22, line 39 skipping to change at page 24, line 4
Thomas Narten (editor) Thomas Narten (editor)
IBM IBM
Email: narten@us.ibm.com Email: narten@us.ibm.com
Eric Gray (editor) Eric Gray (editor)
Ericsson Ericsson
Email: eric.gray@ericsson.com Email: eric.gray@ericsson.com
David Black David Black
EMC EMC
Email: david.black@emc.com Email: david.black@emc.com
Dinesh Dutt
Cumulus Networks
Email: ddutt.ietf@hobbesdutt.com
Luyuan Fang Luyuan Fang
Cisco Systems
111 Wood Avenue South 111 Wood Avenue South
Iselin, NJ 08830 Iselin, NJ 08830
USA USA
Email: lufang@cisco.com Email: lufang@cisco.com
Lawrence Kreeger Lawrence Kreeger
Cisco Cisco
Email: kreeger@cisco.com Email: kreeger@cisco.com
Maria Napierala Maria Napierala
AT&T AT&T
200 Laurel Avenue 200 Laurel Avenue
Middletown, NJ 07748 Middletown, NJ 07748
USA USA
Email: mnapierala@att.com Email: mnapierala@att.com
Murari Sridharan
Microsoft
Email: muraris@microsoft.com
 End of changes. 62 change blocks. 
177 lines changed or deleted 216 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/