INTERNET-DRAFT                                               Luyuan Fang
Intended Status: Informational                              Rex Fernando                                David Ward
Expires: April 15, 22, 2013                                     Rex Fernando
                                                                   Cisco
                                                         Maria Napierala
                                                                    AT&T
                                                             Nabil Bitar
                                                                 Verizon
                                                          Dhananjaya Rao
                                                                   Cisco

                                                        October 15, 22, 2012

                    BGP L3VPN Virtual PE Framework
                draft-fang-l3vpn-virtual-pe-framework-00
                draft-fang-l3vpn-virtual-pe-framework-01

Abstract

   This document describes a framework for BGP/MPLS L3VPN with virtual
   PE solutions. It provides functional description of the information on control plane, plane
   and data plane of the virtual PE solutions, solutions. It also describes
   interactions among the vPE solutions and its interaction with other network elements. The solution supports
   virtual PE solutions support further control plane and forwarding
   plane separation from when compared with traditional L3VPN PE solutions.
   It allows the L3VPN functions to be extended to application end systems
   devices for large scale and efficient IP application support.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

INTERNET DRAFT              <Document Title>                <Issue Date>

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

Copyright and License Notice
INTERNET DRAFT              <Document Title>                <Issue Date>

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . .  3  4
     1.1 Overview . .  Terminology . . . . . . . . . . . . . . . . . . . . . . . .  3  4
     1.2 Scope of the document Motivation . . . . . . . . . . . . . . . . . . .  3
     1.3  Terminology . . . . . .  5
     1.3 Scope of the document  . . . . . . . . . . . . . . . . . .  4 .  6
   2. Virtual PE Architecture and Reference Model . . . . . . . . . .  4  6
     2.1 Virtual PE . . . . . . . . . . . . . . . . . . . . . . . . .  4  7
     2.2 Architecture reference model . . . . . . . . . . . . . . . .  5  7
   3. Control Plane . . . . . . . . . . . . . . . . . . . . . . . . .  8 10
     3.1 Router server of vPE Control Plane  . . . . . . . . . . . . . . . . . . . .  9
     3.2 Control plane options . 10
     3.1 Route server of vPE  . . . . . . . . . . . . . . . .  9 . . . . 11
     3.3 Use of router reflector  . . . . . . . . . . . . . . . . . . 10 11
     3.4 Use of RT constraint . . . . . . . . . . . . . . . . . . . . 10 12
   4. Forwarding Plane  . . . . . . . . . . . . . . . . . . . . . . . 10 12
     4.1 Virtual Interface  . . . . . . . . . . . . . . . . . . . . . 10 12
     4.2 VPN forwarder  . . . . . . . . . . . . . . . . . . . . . . . 10 12
     4.3 Encapsulation  . . . . . . . . . . . . . . . . . . . . . . . 11 12
     4.4 Optimal forwarding . . . . . . . . . . . . . . . . . . . . . 11 13
   5. Addressing  . . . . . . . . . . . . . . . . . . . . . . . . . . 12 14
     5.1 IPv4 and IPv6 support  . . . . . . . . . . . . . . . . . . . 12 14
     5.2 Address space separation . . . . . . . . . . . . . . . . . . 12 14
     6.0 Inter-connection considerations  . . . . . . . . . . . . . . 14
   7.  Security Considerations  . . . . . . . . . . . . . . . . . . . 12 15
   8.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 12 15
   9.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 12 15
     9.1  Normative References  . . . . . . . . . . . . . . . . . . . 12 15

INTERNET DRAFT              <Document Title>                <Issue Date>

     9.2  Informative References  . . . . . . . . . . . . . . . . . . 13 16
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 13 16

INTERNET DRAFT              <Document Title>                <Issue Date>

1  Introduction

   Network virtualization is to provide multiple individual network
   services by sharing through shared common available network resources. Network
   virtualization is not a new concept, for concept. For example, BGP BGP/MPLS layer 3
   Virtual Private Networks (L3 VPNs) (L3VPNs) [RFC4364] have been widely deployed
   to provide network based, service provider provisioned VPNs. based virtual private network services. It
   provides routing isolation and forwarding separation for individual
   VPNs, allow IP address overlapping among different VPNs while
   forward
   forwarding traffic over common network infrastructure.

   The recent development of server virtulization, provided

   Network virtualization enables the support of multiple isolated
   individual networks over a common network infrastructure. Network
   virtualization is not a new
   opportunities for the virtual Provider Edge (vPE) solution on
   application end-system. The virtual PE solution can be powerful and
   attractive concept. For example, BGP/MPLS IP Virtual
   Private Network (IP VPNs) [RFC4364] have been widely deployed to
   provide network based, service providers and enterprises in the fast growing
   cloud computing provider provisioned IP VPNs for
   multiple customers with overlapping IP address spaces over a common
   service provider IP/MPLS network. BGP/MPLS IPVPNs provide routing
   isolation among customers and intelligent mobility environment.

1.1 Overview

   L3 VPN allow address overlapping among
   different VPNs by having per-customer Virtual Provider Edge may provide full or partial L3VPN PE
   functions or partial PE functions. The virtual PE has two components
   - control Routing and forwarding. The control component can be Forwarding
   Instance (VRF) at a software
   instance resides in service provided Edge (PE), while forwarding
   customer traffic over a physical device, most commonly seen in common IP/MPLS network infrastructure.

   With the end-
   system devices where multiple applications are supported, e.g.,
   mobile application server in a wireless call center, or an end-system
   in a SP virtual Private Cloud (vPC) data center, or a host in an
   Financial back-office.

   The architecture advent of compute capabilities and protocols defined in IETF for BGP/MPLS L3VPN
   [RFC4364] is the foundation for virtual PE solution. Certain protocol
   extensions or integration may be needed to support the virtual PE
   solutions.

   This document defines a framework proliferation of
   virtualization in end devices for using standard protocols to
   build BGP L3 VPN with virtual systems and applications, PE solutions. The goal
   functionality virtualization on such end device is to support
   large becoming feasible,
   and in some cases attractive for scale deployment and reduce operational complexity. The
   targeted efficiency. Scale and
   efficiency are crutial factors in the cloud computing environment can be virtualized wireless providers call
   centers, large scale service providers data centers, large enterprise
   central
   supporting various applications and branch facilities, services, and in traditional
   service provider managed
   services.

1.2 Scope of the document

   This focus of space.

   The virtual Provider Edge (vPE) solution described in this document
   is BGP L3VPN virtual PE solutions.

   It is assumed that to extend the readers are familiar with functionality of BGP/MPLS L3VPN
   technologies, the base technology and operation will not be covered
   in this document.

INTERNET DRAFT              <Document Title>                <Issue Date>

   The following network elements are in scope: BGP L3VPN vPE; to the
   interaction of vPE with other network elements, including BGP L3VPN
   physical PE, physical BGP Route Reflector (RR) and virtual BGP Route
   Reflector (vRR), and Autonomic System Border Router (ASBR), external
   controller, provisioning/orchestration systems. Definitions of
   protocol extensions is out of scope of this document.

1.3 application
   end device.

1.1  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

   Term              Definition
   -----------       --------------------------------------------------
   3GPP              3rd Generation Partnership Project (3GPP)
   AS                Autonomous Systems
   ASBR              Autonomous Systems Border Router

INTERNET DRAFT              <Document Title>                <Issue Date>

   BGP               Border Gateway Protocol
   End-Device        A device
   ED                End device: where Guest OS and OS, Host OS/Hypervisor OS/Hypervisor,
                     applications, VMs, and virtual router may reside, aka End-system reside
   Forwarder         L3VPN forwarding function
   GRE               Generic Routing Encapsulation
   IaaS              Infrastructure as a Service
   IRS               Interface to Routing System
   LTE               Long Term Evolution
   MP-BGP            Multi-Protocol Border Gateway Protocol
   PCEF              Policy Charging and Enforcement Function
   P                 Provider backbone router
   RR                Route Reflector
   RT                Route Target
   RTC               RT Constraint
   ToR               Top-of-Rack switch
   VM                Virtual Machine
   Hypervisor        Virtual Machine Manager
   VM                Virtual Machine
   SDN               Software Defined Network
   VI                Virtual Interface
   vCE               virtual Customer Router
   vPC               virtual Private Cloud
   vPE               virtual Provider Edge
   VPN               Virtual Private Network
   vRR               virtual Route Reflector

2. Virtual PE Architecture and Reference Model

2.1 Reflector1.2 Scope of the document
   WAN               Wide Area Network

   Virtual PE

   A virtual PE is a PE with control instance and a forwarding
   components reside resides in a shared physical an end device where multiple
   applications are supported. (e.g., a server) along
   with client/application VMs.

   Through out this document, the term virtual PE (vPE) is used to
   denote BGP/MPLS L3VPN virtual Provider Edge router.

1.2 Motivation

   The control recent rapid adoption of Cloud Services by enterprises and forwarding components the
   phenomenal growth of mobile IP applications accelerate the needs to
   extend the L3VPN capability to the end devices. For example,
   Enterprise customers requested Service Providers to extend and
   integrate their L3VPN services available in the WAN into the new
   Cloud services; large enterprise have existing L3VPN deployment are
   decoupled, they may reside
   extending them into their data centers; mobile providers are adopting
   L3VPN into their 3GPP Mobile infrastructure are looking to extend the
   L3VPNs to their end device of their call processing center.

   The virtual PE solution described in this document is aimed to meet
   the same or different physical devices. following key requirement [I-D.fang-l3vpn-end-system-req].

INTERNET DRAFT              <Document Title>                <Issue Date>

   A key motivation

   1) Support end device multi-tenancy, per tenant routing isolation and
   traffic separation.

   2) Support large scale L3VPNs in service network, upto tens of using virtual PE solution is to place
   thousands of end devices and Millions of VMs in the single service
   network, e.g., a data center.

   3) Support end-to-end L3VPN
   termination point as close as possible connectivity, e.g. L3VPN can start from a
   service network end device, connect to where a corresponding L3VPN in the
   services/applications reside;
   WAN, and to take the advantage of terminate in another service network end device.

   4) Decoupling control plane and forwarding decoupling for better scalability network virtualization
   and allow flexible abstraction.

   L3VPN is the proven technologies which is capable of providing
   routing policy control and fast provisioning. In many cases, forwarding separation, and it is proven with large scale
   deployment (e.g. supporting 7-8 million L3VPN routes in single
   Service Provider networks today).

   By extending L3VPN solution to the end device with vPE solution,
   application end-to-end (VM to VM, applications to the end user) L3VPN
   connectivity cab be achieved, and well as the true network
   virtualization and abstraction.

   The architecture and protocols defined in BGP/MPLS IP VPN [RFC4364]
   is the foundation for virtual PE extension. Certain protocol
   extensions or integration may be needed to support the virtual PE
   solutions.

1.3 Scope of the document

   It is placed assumed that the readers are familiar with BGP/MPLS IP VPN
   [RFC4364] terms and technologies, the base technology and its
   operation are not reviewed in details in this document.

   The following network elements are discussed in this document: the service end systems where
   concept of BGP L3VPN vPE; the interaction of vPE with other network
   elements, including BGP L3VPN physical PE, physical or virtual BGP
   Route Reflectors (RR, vRR), and Autonomous System Border Router
   (ASBR), Service Network gateway routers, external controllers,
   provisioning/orchestration systems, and the vPE inter-connections
   with other non L3VPN networks.

   The definitions of protocols extensions are out of the scope of this
   document.

2. Virtual
   Machines running various applications. PE Architecture and Reference Model
INTERNET DRAFT              <Document Title>                <Issue Date>

2.1 Virtual PE

   As defined in [RFC4364], a L3VPN is created by applying policies to
   form a subset of sites among all sites connected the backbone
   network. It is collection of "sites". A site can be considered as a
   set of IP systems maintain IP inter-connectivity without connecting
   through the backbone. The typical use of L3VPM has been to inter-
   connect different sites of an Enterprise networks through Service
   Provider's L3VPN L3VPNs in the WAN.

   A virtual PE (vPE) is a PE instance which resides in one or more
   physical devices, it is commonly placed in a network service (e.g. a
   Data Center) end device (e.g., a Server) where the client/application
   VMs are hosted. The recent rapid adoption of Cloud Services, control and the phenomenal
   growth forwarding components of mobile IP applications, accelerate the needs vPE are
   decoupled, they may reside in the same physical device or in
   different physical devices.

   In the case that a vPE is in a Data Center server along with
   client/application VMs, one can view the vPE to extend VM relationship as a
   typical PE-CE relationship. Unlike a regular physical PE, vPE allows
   L3VPN control plane and forwarding function residing on different
   physical devices. The full MP-BGP control plane may reside on the
   VPN capability end
   device, or may be external to the end-systems. Enterprise customers requests
   Service Providers end device, e.g., in a BGP L3VPN
   boarder router (ASBR)/DC gateway router, a Route Reflector (RR), or
   an external controller.

   Virtual PE solution allows the placement of L3VPN termination point
   right inside the end device (e.g., a server). In this case, the vPE
   to extend CE (VM) connection is internal to the device. If both control and
   forwarding elements are placed on the end device, L3VPN services routing and
   forwarding starts from the end device, the eliminate the needs for
   additional process in the WAN into next hop (e.g., layer2 and layer 3
   integration). This approach helps to simplify the
   new Cloud services supported by various Data Center technologies.
   Mobile providers who have already adopted L3VPN into operation and
   improve the Mobile
   infrastructure are looking routing and forwarding efficiency in large scale
   deployment.

   Another important benefit is that vPE solution allows full control
   and forwarding decoupling for scale and achieving true network
   virtualization to support allow network abstraction, flexible and dynamic
   policy control, quick service virtualization with
   L3VPN on the end-system of mobile applications. turn up time and VM mobility support.

2.2 Architecture reference model

   Figure 1 illustrate the topology that vPE is reside inside in the end-
   system end device
   where the applications are hosted.

INTERNET DRAFT              <Document Title>                <Issue Date>

                   +--------------------------------------------+
                   |                                            |
                   |                            +-----------+   |
                   |                            |vPE| End   |   |
                   |                            |---+ System| Device|   |
                   |              +---------+   +-----------+   |
      .------.     |              |Transport|   +-----------+   |
     (        )    | +-------+    | Device  |   |vPE| End   |   |
    (          )   | |Gateway|    +---------+   |---+ System| Device|   |
   :            :--+-|  PE   |                  +-----------+   |
   :            :  | +-------+    +---------+   +-----------+   |
   :   IP/MPLS  :  |              |Transport|   |vPE| End   |   |
   :    WAN     :  | +-------+    | Device  |   |---+ System| Device|   |
   :            :--+-|Gateway|    +---------+   +-----------+   |
    (          )   | |  PE   |                  +-----------+   |
     (        )    | +-------+    +---------+   |vPE| End   |   |
      '------'     |              |Transport|   |---+ System| Device|   |
                   |              | Device  |   +-----------+   |
                   |              +---------+   +-----------+   |
                   |                            |vPE| End   |   |
                   |                            |---+ System| Device|   |
                   |                            +-----------+   |
                   | Virtualized Service Network                |
                   +--------------------------------------------+

             Figure 1. Virtualized Service network with vPE

   The Virtualized Service Network in Figure 1 consists of WAN gateway
   PE devices, transport devices, and end-systems. end devices. In some networks, it
   is feasible the VPN Gateways may be implemented as vPEs as well.

   Examples of service network may be a network that supports cloud
   computing services, mobile call centers, and SP or enterprise private
   or public data
   centers.

   The

   Note that the transport devices in the service network in the diagram
   do not participate L3VPNs, they function similar as P routers in MPLS
   back bone, they do not maintain the L3VPN states, they and are not aware of
   the L3VPN in the network.
   aware.

INTERNET DRAFT              <Document Title>                <Issue Date>

             +----------------------------------------------------+
             | +---------+ +---------+    +---------+ +---------+ |
             | |  VM1    | |  VM2    |    |  VM47   | |  VM48   | |
             | |(VPN Red)| |(VPN Grn)|... |(VPN Grn)| |(VPN Blu)| |
             | +----+----+ +---+-----+    +----+----+ +----+----+ |
             |      |          |               |           |      |
             |      +---+      | +-------------+       +---+      |
             |          |      | |                     |          |
      to     |      +---+------+-+---------------------+---+      |
      Gateway|      |   |      | |                     |   |      |
      PE     |      | +-+-+   ++-++            +---+ +-+-+ |      |
             |      | |VRF|   |VRF|   .......  |VRF| |VRF| |      |
      <------+------+ |Red|   |Grn|            |Yel| |Blu| |      |
             |      | +---+   +---+            +---+ +---+ |      |
             |      |            L3 VPN virtual PE         |      |
             |      +--------------------------------------+      |
             |                                                    |
             |                     End-system                     End Device                     |
             +----------------------------------------------------+

          Figure 2. Virtual L3 VPN PE logical connections VM in the End-system end device to VRF in vPE mapping

   An end-system end device shown in Figure 2 is a virtualized server or system
   which hosting hosts multiple VMs, the a virtual PE resides in the end-system. end device.
   The vPE supports multiple VRFs, VRF Red, VRF Grn, VRF Yel, VRF Blu,
   etc. Each client or application VM is associated to a particular VRF
   as a member of the particular VPN. For example, VM1 is associated to
   VRF Red, VM2 and VM47 are associated to RFC Grn, etc. Routing isolations apply between VPNs.

   The vPE connectivity relationship
   isolation applies between vPE to VM is similar like
   the PE to CE in a regular BGP L3VPNs. VPNs for multi-tenancy support. For
   example, VM1 and VM2 can not connect to communicate with each other in a simple
   intranet VPN L3VPN topology as shown in the configuration.

   The L3VPN provides routing
   isolation among different VPNs. vPE connectivity relationship between vPE and the application VM
   is similar to the PE to CE relationship in a regular BGP L3VPNs.

INTERNET DRAFT              <Document Title>                <Issue Date>

               +----------------------+   +----------------------------+
               |                      |   |              +-----------+ |
   +------+--+ | +-----+      +-----+ |   | +-----+      |      +---+| |
   |VPN   +--+ | |     |      |     | |   | |+---+|      +---+  |VM1|| |
   |Red   |CE|-+-|+---+|      |+---+| |   | ||VRF||      |vPE|  +---+| |
   |Site A+--+ | ||VRF||      ||VRF|| |   | ||Red||      +---+  |VM2|| |
   +------+--+ | ||Red||      ||Red|| |   | |+---+|      |      +---+|      |Server+---+| |
               | |+---+|      |+---+|-+---+-|+---+|      +-----------+ |
               | |+---+|      |+---+| |   | ||VRF||      +-----------+ |
   +------+--+ | ||VRF||      ||VRF|| |   | ||Grn||      |      +---+| |
   |VPN   +--+-+-||Grn||      ||Grn|| |   | |+---+|      +---+  |VM1|| |
   |Grn   |CE| | |+---+|      |+---+| |   | |GWay |      |vPE|  +---+| |
   |Site A+--+ | | PE  |      | PE  | |   | | PE  |      +---+  |VM2|| |
   +------+--+ | +-----+      +-----+ |   | +-----+      |      +---+|      |Server+---+| |
               |                      |   |              +-----------+ |
               |      IP/MPLS WAN     |   |Virtualized Service Network Data Center     |
               +----------------------+   +----------------------------+

           |eBGP  |                 |        |           |
           |Static|      MP-BGP     | MP-BGP |(Options)  |
           |<---->|<--------------->|<------>|<--------->|
                                     Inter-AS
                                     Option B

             Figure 3. L3VPN Logical Connection protocols

   Options Connecting Enterprise CE to DC VM over WAN

   The example of connection from an Enterprise site to application VMs
   through vPE on the end device of a SP provisioned virtualized data
   center.

   There are multiple options for protocol VPN control plane signaling between
   the Gateway PE to vPE on the end-system:

   1. MP-BGP.

   2. XMPP server within the data center. It can
   use MP-BGP as in regular L3VPN, or use other extensible IP messaging
   protocols which are familiar by
   computing work force.

   3. Network Controller

   Options for protocols between IP/MPLS defined in IETF, or use controller direct signaling as a
   SDN approach.

   The inter-connection from DC Gateway PE to MPLS WAN network and Virtualized
   service network depending on may use one of
   the design. Inter-AS options if they are in different ASes. Option B may be
   more practical for the reasons it is more scalable than Option A, and
   more restricted than Option C. Consider route aggregation with Option
   B if both sides have large number of routes.

   The connection between backbone VPN to VPN CE on the left hand side
   is
   an example. regular L3VPN connection, e-BGP, or static, or other protocols can
   be used.

3. Control Plane

3.1 vPE Control Plane

   The vPE control plane can be distributed or centralized.

   1) Distributed control plane

INTERNET DRAFT              <Document Title>                <Issue Date>

   vPE participates in underlay routing through IGP protocols: ISIS or
   OSPF.

   vPE participates in overlay L3VPN control protocol is protocol: MP-BGP as defined in
   [RFC4364].

   When extending L3VPN into service network, supporting

   While MP-BGP on is the
   end-system may or may not de facto preferred choice between vPE and
   gateway-PE, using extensible signaling messaging protocols can be practical, alternatives
   alternative, such technologies have been proposed for this segment of
   signaling [I-D.ietf-l3vpn-end-system].

   2. Centralized routing controller

   This is a SDN approach. In the virtual PE implementation, not only
   the service network infrastructure and the VPN overlay networks are
   decoupled, but also

INTERNET DRAFT              <Document Title>                <Issue Date>

   considered here. the vPE control plane and data plane are
   physically decoupled. The control plane directing the data flow may
   reside elsewhere, such a centralized controller. This requires
   standard interface to routing system (IRS). The Interface to Routing
   System (IRS) is work in progress in IETF [I-D.ward-irs-framework],
   [I-D.rfernando-irs-fw-req].

3.1 Router Route server of vPE

   A virtual PE consist the control plane element and the forwarding
   plane element. Since the proposed solution decoupled the two element,
   they may or may not reside on the same physical device.

   The Route Server of L3VPN vPE is a software application that
   implements the BGP/MPLS L3VPN PE control plane functionality.

   In the case other control/signaling/messaging protocol are used (see
   options listed in the next sub-section), used, the
   route server is also the server of the particular protocol(s), it
   interacts with VPN forwarder
   (see the section on forwarding).

3.2 Control plane options of vPE

   a. MP-BGP

   MP-BGP can be used in service network where both the end system
   implementation and operation work force has the knowledge and skills
   to support it. In this case, the design and deployment is very
   similar to what applies to regular L3VPN deployment.

   b. Extensible messaging protocols

   It is redeemed as a good alternative for bridging the gap between
   Gateway PE and the vPE where operators could not support BGP.

   Although the modern end-systems may be able to support MP-BGP, the
   operational support of the computing and storage community often may
   not have the adequate BGP skills or willingness to step up to BGP
   operation. Since this is a common situation today, an light weight,
   extensible IP protocol is very welcome by the cloud service
   communities. One example of such protocol is to use XMPP to signal
   L3VPN connectivity between the Gateway PE the virtual PE on the end-
   systems. The technology solution is described in details in [I-
   D.ietf-l3vpn-end-system].

   c. Controller

   This is a SDN approach. In the virtual PE implementation, not only
   the service network infrastructure and the VPN overlay networks are
   decoupled, but also the vPE control plane and data plane are
   physically decoupled. The control plane directing the data flow may
   reside elsewhere, such a centralized controller. This requires
   standard interface to routing system (IRS). The IRS work is in

INTERNET DRAFT              <Document Title>                <Issue Date>

   progress in IETF, [ID.ward-irs-framework], [ID.rfernando-irs-
   framework-requirement]. forwarder.

3.3 Use of router reflector

   Modern service networks can be very large in scale, scale. For example, the
   number of VPNs routes in a very large data centers can easily pass the scale
   of those in SP backbone VPN networks.
   The end-systems and therefore the vPEs There are may be several thousand tens of
   thousands of end devices in a single service network.

   Use of Router Reflector (RR) is necessary in large scale L3VPN
   networks to avoid full iBGP mesh among all vPEs and PEs. The L3 VPN
   routes can be partitioned to a set of RRs, the partition techniques
   are detailed in [RFC4364].

   When RR is residing in a device physical device, e.g., a server, which is

INTERNET DRAFT              <Document Title>                <Issue Date>

   partitioned to support multi-
   functions multi-functions and application, client/applications VMs,
   the RR become becomes virtualized RR (vRR). Since RR's is performs control
   plane only, a physical or virtualized server with large scale of
   computing power and memory can be a perfect good candidate as host of vRRs.
   The vRR can also reside be in Gateway PE, or in an end-system. end device.
   Redundant RR design is even more important in when using vRR.

3.4 Use of RT constraint

   The Route Target Constraint (RT Constraint, RTC) [RFC4684] is a
   powerful tool for VPN selective L3VPN route filtering. distribution. With RT constraint,
   Constraint, only the BGP receiver (a PE,
   vPE, RR, vRR, ASBRs, (e.g, PE/vPE/RR/vRR/ASBRs, etc.)
   with the particular L3VPN routes L3VPNs will receive the route update. update for the
   corresponding VPNs. It is critical to use RT constraint
   particularly in to support
   large scale L3VPN development.

4. Forwarding Plane

4.1 Virtual Interface

   Virtual Interface (VI) is an interface in an end-system end device which is used
   for connecting the vPE to the application VMs or other applications in the end-
   system. end device. The later can
   latter cab be viewed treated as CEs in traditional L3VPN scenario. the regular L3VPN's view.

4.2 VPN forwarder

   VPN Forwarder is the forwarding component of a vPE solution. vPE.

   The VPN forwarder location options:

   1) within the end-system end device where the virtual interface is and application
   VMs are.

   2) in an external device of end-system which the end-system end device connect to, for
   example, a ToR.

INTERNET DRAFT              <Document Title>                <Issue Date> Top of the Rack (ToR) in a data center.

   Multiple factors should be considered for the location of the VPN
   forwarder, including device capability, overall economy, solution economics,
   QoS/firewall/NAT placement, optimal forwarding, latency and
   performance, operation impact, etc. There are design trade offs, it
   is worth the effort to study the traffic pattern and forwarding
   looking trend in your own unique service network as part of the
   exercise.

4.3 Encapsulation

   There are two existing standardized forwarding encapsulation/forwarding options
   for BGP/MPLS L3VPN.

INTERNET DRAFT              <Document Title>                <Issue Date>

       1. MPLS Encapsulation, [RFC3032].

       2. Encapsulating MPLS in IP or Generic Routing Encapsulation
   (GRE), [RFC4023].

   The most common BGP/MPLS L3VPNs deployment in SP networks are using
   MPLS forwarding. This requires MPLS MPLS, e.g., Label Switched Protocol
   (LDP) [RFC5036] to be deployed in the network. It is proven to scale scale,
   and provide it comes with various security mechanism mechanisms to protect network
   against attacks.

   The

   However, the service network environment, such as a data center, is
   different than Service Provider VPN networks or large enterprise
   backbones. MPLS deployment may or may not be feasible. Two major
   challenges for MPLS deployment in the this new environment: 1) the
   capabilities for of the
   end-system end devices or and the transport/forwarding devices;
   2) the workforce skill set.

   Encapsulating MPLS in IP or GRE tunnel [RFC4023] may often be more
   practical in many
   cases. But bare in mind, most data center, and computing environment. Note that
   when IP or GRE tunnel does not provide encapsulations are used, the same
   level of associated security mechanism as MPLS forwarding.

   There
   considerations must be analyzed carefully.

   In addition, there are new encapsulation proposals for service
   network/Data center currently as work in progress in IETF, including
   several UDP based encapsulations proposals and some TCP based proposals.
   proposal. These
   mechanism may overlay encapsulations can be considered as alternative to MPLS suitable alternatives
   for a vPE, considering the availability and leverage of support in
   virtual and IP/GRE encap. physical devices.

4.4 Optimal forwarding

   As reported by many large cloud service operators, the traffic
   pattern in their data centers are were dominated by East-West across
   subnet traffic (between the end-systems end device hosting different applications) applications
   in different subnets) than North-
   South North-South traffic (going in and out the
   DC to the WAN). WAN) or switched traffic within subnets. This is a key primary
   reason that many large scale new design has moved away from
   traditional L2 design to L3.

   When forwarding the traffic within the same VPN, the end-system

INTERNET DRAFT              <Document Title>                <Issue Date> vPE should be
   able to access directly between provide direct communication among the VMs/application
   sender/receivers
   senders/receivers without the need of going through gateway devices.
   If it is on the same end-system, end device, the traffic should not need to leave
   the same device. If it is on different end-system, end device, optimal routing
   should be applied.

   When multiple VPNs need to be accessed to accomplish the task the

INTERNET DRAFT              <Document Title>                <Issue Date>

   user requested (this is common too), the end-system end device virtual
   interfaces should be able to directly access multiple VPNs in via use of
   extranet VPN style techniques without the need of Gateway facilitation. Use
   BGP L3VPN policy control is the tool mechanisms to support this function.

5. Addressing

5.1 IPv4 and IPv6 support

   Both IPv4 and IPv6 should be supported in the virtual PE solution.

   This may present challenging to older devices, but may not be issues
   to newer forwarding devices and servers. A server is replaced much
   more frequently than a network router/switch in the infrastructure
   network. Newer
   network, newer equipment most likely support IPv6. should be capable of IPv6 support.

5.2 Address space separation

   The addresses used for L3VPNs in the service network should be in
   separate address blocks than the ones used the underlay
   infrastructure of the service network. This practice is to protect
   the service network infrastructure being attacked if the attacker
   gain access of the tenant VPNs.

   Similarity, the addresses used for the service network, e.g., a cloud
   service center of a SP, should be separated from the WAN backbone
   addresses space, for security reasons.

6.0 Inter-connection considerations

   There are also deployment scenarios that L3VPN may not be supported
   in every segment of the networks to provide end-to-end L3VPN
   connectivity, a L3VPN vPE may be reachable only via an intermediate
   inter-connecting network, interconnection may be needed in these
   cases.

   When multiple technologies are employed in the overall solution, a
   clear demarcation should be preserved at the inter-connecting points.
   The problems encountered in one domain should not impact the other
   domains.

   From L3VPN point of view: A L3VPN vPE that implements [RFC4364] is a
   component of L3VPN network only. A L3VPN VRF on physical PE or vPE
   contains IP routes only, including routes learnt over the locally
   attached network.

   As described earlier in this document, the L3VPN vPE should ideally
   be located as close to the "customer" edge devices. For cases, where

INTERNET DRAFT              <Document Title>                <Issue Date>

   this is not possible, simple existing "L3VPN CE connectivity"
   mechanisms should be used, such as static, or direct VM attachments
   such as described in the vCE option below.

   Consider the following scenarios when BGP MPLS VPN technology is
   considered as whole or partial deployment:

   Scenario 1: All VPN sites (CEs/VMs) support IP connectivity. The best
   suited BGP solution is to use L3 VPNs [RFC4364] for all sites with PE
   and/or vPE solutions. This is a straightforward case.

   Scenario 2: Legacy layer 2 connectivity must be supported in certain
   sites/CEs/VMs, and the rest sites/CEs/VMs need only 3 connectivity.

   One can consider to use combined vPE and vCE solution to solved the
   problem. Use L3VPN for all sites with IP connectivity, and use a
   physical or virtual CE (vCE, may reside on the end device) to
   aggregate the L2 sites which, for example, are in a single container
   in a data center. The CE/vCE can be considered as inter-connecting
   point, where the L2 network are terminated and the corresponding
   routes for connectivity of the L2 network are inserted into L3VPN
   VRF. The L2 aspect is transparent to the L3VPN in this case.

   Reducing operation complicity and maintaining the robustness of the
   solution are the primary reasons for the recommendations.

7.  Security Considerations

   To

   vPE solution presented a virtualized L3VPN PE model. There are
   potential implications to L3VPN control plane, forwarding plane, and
   management plane. Security considerations are currently under study,
   will be added. included in the future revisions.

8.  IANA Considerations

   None.

9.  References

9.1  Normative References
INTERNET DRAFT              <Document Title>                <Issue Date>

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3032]  Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y.,
              Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack
              Encoding", RFC 3032, January 2001.

INTERNET DRAFT              <Document Title>                <Issue Date>

   [RFC4023]  Worster, T., Rekhter, Y., and E. Rosen, Ed.,
              "Encapsulating MPLS in IP or Generic Routing Encapsulation
              (GRE)", RFC 4023, March 2005.

   [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
              Border Gateway Protocol 4 (BGP-4)", RFC 4271, January
              2006.

   [RFC4364]  Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
              Networks (VPNs)", RFC 4364, February 2006.

   [RFC4684]  Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk,
              R., Patel, K., and J. Guichard, "Constrained Route
              Distribution for Border Gateway Protocol/MultiProtocol
              Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual
              Private Networks (VPNs)", RFC 4684, November 2006.

   [RFC5036]  Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed.,
              "LDP Specification", RFC 5036, October 2007.

   [I-D.ietf-l3vpn-end-system] Marques, P., Fang, L., Pan, P., Shukla,
              A., Napierala, M., "BGP-signaled end-system IP/VPNs",
              draft-ietf-l3vpn-end-system-00, October 2012.

9.2  Informative References

   [ID.ward-irs-framework]

   [I-D.fang-l3vpn-end-system-req] Napierala, M., adn Fang, L.,
              "Requirements for Extending BGP/MPLS VPNs to End-Systems",
              draft-fang-l3vpn-end-system-requirements-00, Oct. 2012.

   [I-D.ward-irs-framework] Atlas, A., Nadeau, T., Ward. D., "Interface
              to the Routing System Framework", draft-ward-irs-
              framework-00, July 2012.

   [ID.rfernando-irs-framework-requirement]

   [I-D.rfernando-irs-fw-req] Fernando, R., Medved, J., Ward, D., Atlas,
              A., Rijsman, B., "IRS Framework Requirements", draft-rfernando-irs-framework-requirement-
              00, Oct., draft-
              rfernando-irs-framework-requirement-00, Oct. 2012.

Authors' Addresses

   Luyuan Fang
   Cisco
   111 Wood Ave. South
   Iselin, NJ 08830

INTERNET DRAFT              <Document Title>                <Issue Date>

   Iselin, NJ 08830

   Email: lufang@cisco.com

   David Ward
   Cisco
   170 W Tasman Dr
   San Jose, CA 95134
   Email: wardd@cisco.com

   Rex Fernando
   Cisco
   170 W Tasman Dr
   San Jose, CA
   Email: rex@cisco.com

   Maria Napierala
   AT&T
   200 Laurel Avenue
   Middletown, NJ 07748
   Email: mnapierala@att.com

   Nabil Bitar
   Verizon
   40 Sylvan Road
   Waltham, MA 02145
   Email: nabil.bitar@verizon.com

   Dhananjaya Rao
   Cisco
   170 W Tasman Dr
   San Jose, CA
   Email: dhrao@cisco.com