Network Working Group K. Kumaki Internet-Draft KDDI Corporation Intended status: Informational P. Mohapatra Expires: July 15, 2014 Cumulus Networks D. Meyer Brocade K. Varadhan Z. Ali Cisco Systems January 11, 2014 A Virtualized Network Element Framework draft-kumaki-vnef-02 Abstract This document outlines a new architectural paradigm of virtualizing the service provider networks and its associated engineering requirements. The framework is referred to as the virtual network element framework (VNEF) in the document. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on July 15, 2014. Copyright Notice Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents Kumaki, et al. Expires July 15, 2014 [Page 1] Internet-Draft A Virtualized Network Element Framework January 2014 carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Kumaki, et al. Expires July 15, 2014 [Page 2] Internet-Draft A Virtualized Network Element Framework January 2014 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. Use Case 1: Operational Independence . . . . . . . . . . . 4 1.2. Use Case 2: Network Redundancy . . . . . . . . . . . . . . 5 1.3. Requirements Language . . . . . . . . . . . . . . . . . . 5 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Problem Statement and Related Areas . . . . . . . . . . . . . 6 4. Reference Model . . . . . . . . . . . . . . . . . . . . . . . 7 5. Applications . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.1. Network Wholesale Services . . . . . . . . . . . . . . . . 8 5.2. On-demand Network Services . . . . . . . . . . . . . . . . 8 6. Requirement Details . . . . . . . . . . . . . . . . . . . . . 9 6.1. Dynamic binding - On-demand Service Creation . . . . . . . 9 6.2. Control Plane/Routing Layer Separation . . . . . . . . . . 9 6.3. Data Plane Separation . . . . . . . . . . . . . . . . . . 9 6.4. Separate Operation of Services . . . . . . . . . . . . . . 9 6.5. QoS/SLA . . . . . . . . . . . . . . . . . . . . . . . . . 10 6.6. Load Balancing . . . . . . . . . . . . . . . . . . . . . . 10 6.7. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 10 7. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 10 7.1. Element Virtualization . . . . . . . . . . . . . . . . . . 10 7.2. Network Virtualization . . . . . . . . . . . . . . . . . . 11 7.3. Service Identification . . . . . . . . . . . . . . . . . . 13 8. Comparison with Other Technologies . . . . . . . . . . . . . . 13 8.1. VRF . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 8.2. Network Virtualization Over L3 . . . . . . . . . . . . . . 13 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 11. Security Considerations . . . . . . . . . . . . . . . . . . . 14 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14 12.1. Normative References . . . . . . . . . . . . . . . . . . . 14 12.2. Informational References . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 15 Kumaki, et al. Expires July 15, 2014 [Page 3] Internet-Draft A Virtualized Network Element Framework January 2014 1. Introduction VNEF allows service providers to operate multiple services independently on a common backbone network. To see why this is required, let us walk through a few example use cases. 1.1. Use Case 1: Operational Independence Network consolidation is increasingly becoming the norm in service provider networks. This consolidation creates a common backbone infrastructure on which the service specific networks provide end-to- end service-level connectivity. The services include native access services and packet based services such as Internet and Layer 3 VPNs. The consolidation provides numerous benefits, including higher utilization and cost savings in Capital Expenditures (CAPEX) and Operational Expenditures (OPEX). The model is described in Figure 1. __________ __________ / Service1 \ / Service1 \ | Network | | Network | \----------/ \ / \----------/ \ __________________ / / Common backbone \ | Network | \ / / \----------------/ \ __________ / \ __________ / Service2 \ / Service2 \ | Network | | Network | \----------/ \----------/ Figure 1: Network Consolidation In many service providers, the migration of purpose-built service networks to the common core is a work-in-progress. Additionally, the common backbone opens up the potential for a new business model, referred to as "network tenancy". That is, the service provider instantiates a service dynamically on the common backbone and rents it out to the customer. As the number of such services carried on the common backbone continues to increase, it becomes desirable to create a level of network and operational isolation, much like what a dedicated network would have offered. That is, each "service" operator expects to have an isolated virtual network on the backbone that can be managed independently. This provides all the classic benefits of virtualization including efficiency, fast provisioning, and isolation Kumaki, et al. Expires July 15, 2014 [Page 4] Internet-Draft A Virtualized Network Element Framework January 2014 while maintaining the advantages of a common backbone network. 1.2. Use Case 2: Network Redundancy Service providers want the ability to create multiple redundant networks to protect against failures. Examples of such failures are severe protocol errors that melt down a majority of the network elements. Redundancy allows the operator to switch the traffic from one network to the other. It is worth noting that there have been attempts at building such redundancy through separate physical networks in the past, but that turns out to be expensive. As is clear from the use cases, there are two key aspects to VNEF: a. An isolated environment on the network elements that can be operated independently per service ("Element Virtualization") b. A virtualized network on the core physical network that can be operated independently per service ("Network Virtualization"). In the following sections, we delve into these aspects of the framework and draw a set of key properties for each. We recognize that there are multiple ways to "peel the onion". As such, any reference to specific terminology describing a solution should be construed as an example. 1.3. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Terminology Network Element: A routing and forwarding device in the service provider network. Other terms used interchangeably for this: router, platform. Service: A term used to denote an autonomously administered entity (an organization, a department, a setup, or an application) that needs to run an independent network and that has a configured presence on a network element. Other terms used interchangeably for this: router slice. Network slice: a virtual network implemented on the physical network and assigned to a particular service. How it is implemented is not really that important. For example, it can be Kumaki, et al. Expires July 15, 2014 [Page 5] Internet-Draft A Virtualized Network Element Framework January 2014 constructed using network overlays. What matter are the properties that it needs to provide, such as traffic isolation, quality of service. These are discussed in the later sections. Other terms used interchangeabily for this: virtual network. Virtual Network Element Framework (VNEF): the architectural framework discussed in this document. Service ID: A field used to denote each service running on a network element. One requirement of the service ID is to organize the forwarding tables per service and direct packets to the appropriate forwarding table for processing. VM: Virtual Machine 3. Problem Statement and Related Areas Service providers need to consider the following properties of a common backbone network: Smooth migration and simple network: Service providers want to consolidate many service networks into a new common backbone network without changing network topology and routing / signaling policy such as IGP, BGP and MPLS. Network scalability: Routing and signaling protocols need to scale in proportion to the increase in the number of nodes and protocol sessions. Operational policy: Service providers want to keep operational policy per each service network intact even if we consolidate many service networks into a new common backbone network. MTR [RFC4915], [RFC5120], [I-D.ietf-mpls-ldp-multi-topology] allows service providers to keep network topology and routing / signaling policy such as IGP, BGP and MPLS intact. However, this technology does not meet our requirements because 'service' network topologies may be completely different from the physical topology. Actually, it depends on the services. Moreover, MTR follows an intrusive design. That is, it does not provide a true separation of routing and signaling layers between different services (or a 'service' and the parent). We refer to this design as the "vertical approach". Control plane protocols scale higher through hierarchical routing techniques - multi-area and multi-domain routing. We refer to this design as the "horizontal approach". These techniques alone do not meet our requirements because 'service' network topologies might be Kumaki, et al. Expires July 15, 2014 [Page 6] Internet-Draft A Virtualized Network Element Framework January 2014 changed from the physical network topology. Furthermore, in this approach, we can deploy carrier's carrier [RFC4364] to consolidate the service networks. However, it's realistically difficult to enable MPLS-capable interfaces on the edge routers in the pure-IP ISP network. Actually, we need to tranfer MPLS knowledge to operators and replace or upgrade the exsiting routers to MPLS-capable ones in the production networks. 4. Reference Model VNEF's model of operation is depicted in Figure 2. ____________ / \ __/ Virtualized \___ / \ Network 1 / \ +---------------+ / \____________/ \ +---------------+ | +-----------+ | / / \__________/\ \ | +-----------+ | | | Service 1 |---/ / \ \---| Service 1 | | | +-----------+ | | Backbone | | +-----------+ | | | | Network | | | | +-----------+ | \ / | +-----------+ | | | Service 2 |---\ \ __________ / /----| Service 2 | | | +-----------+ | \ \/__________\/ / | +-----------+ | | | \ / \ / | | +---------------+ \__/ Virtualized \__/ +---------------+ Platform \ Network 2 / \____________/ Figure 2: Virtualized Networks A router slice is realized by the allocation of resources on the network element. Similarly the network slice is realized through a network overlay-like mechanism across all the elements in the backbone network. By virtue of separating the resources, the virtualized network maintains independent control plane, data plane, and management plane. For example, a route modification by a network failure on the virtualized network of service 1 in Figure 2 does not affect the virtualized network of service 2. Therefore, it is easy to assure different SLAs for service 1 and service 2. After the implementation of virtualized network, there are many cases that the virtualized network runs out of allocated resources with the expansion of service scale. Even in such case, VNEF provides the adaptive operation to service providers by allocating additional router resources to the starved virtualized network. Kumaki, et al. Expires July 15, 2014 [Page 7] Internet-Draft A Virtualized Network Element Framework January 2014 5. Applications 5.1. Network Wholesale Services Service providers want to provide a network resource (i.e. a network slice) to an ISP as shown in Figure 3. In this case, the ISP1 have own network topology and AS. Also, the ISP1 want to control and manage their network independently as they used to. ____________ / \ __/ Virtualized \___ / \ Network 1 / \ +---------------+ / \____________/ \ +---------------+ | +-----------+ | / / \__________/\ \ | +-----------+ | | | ISP 1 |---/ / \ \---| ISP 1 | | +-----------+ | | Backbone | | +-----------+ | | | | Network | | | | +-----------+ | \ / | +-----------+ | | | Service 1 |---\ \ __________ / /----| Service 1 | | | +-----------+ | \ \/__________\/ / | +-----------+ | | | \ / \ / | | +---------------+ \__/ Virtualized \__/ +---------------+ Platform \ Network 2 / \____________/ Figure 3: Service provider network 1 This service provider have already provided the cloud service (service1),where they provide some applications to customers on the cloud network. The ISP1 also can associate with the cloud platform on the cloud network. 5.2. On-demand Network Services Some ISPs need a network resource (i.e. a network slice) during the specific time and period. For instance, this service provider as shown in Figure 4. provides a network resource to the ISP1 permanently and a network resource to the ISP2 on demand. Kumaki, et al. Expires July 15, 2014 [Page 8] Internet-Draft A Virtualized Network Element Framework January 2014 ____________ / \ __/ Virtualized \___ / \ Network 1 / \ +---------------+ / \____________/ \ +---------------+ | +-----------+ | / / \__________/\ \ | +-----------+ | | | ISP 1 |---/ / \ \---| ISP 1 | | +-----------+ | | Backbone | | +-----------+ | | | | Network | | | | +-----------+ | \ / | +-----------+ | | | ISP 2 |---\ \ __________ / /----| ISP 2 | | | +-----------+ | \ \/__________\/ / | +-----------+ | | | \ / \ / | | +---------------+ \__/ Virtualized \__/ +---------------+ Platform \ Network 2 / \____________/ Figure 4: Service provider network 2 6. Requirement Details 6.1. Dynamic binding - On-demand Service Creation The solution needs to provide the ability to create a new virtualized network on demand. The virtualized network should be built dynamically. 6.2. Control Plane/Routing Layer Separation The solution needs to support an independent control plane and routing layer per virtualized network. The conventional routing/ signaling protocols (such as OSPF, IS-IS, BGP, RSVP-TE) should be able to run on each control plane and routing layer in the virtualized network. 6.3. Data Plane Separation The solution needs to have independent forwarding table and data plane mechanisms per virtualized network. Each data plane on the virtualized network may be associated with physical interfaces or logical interfaces (such as VLAN or tunnel). 6.4. Separate Operation of Services The solution needs to support independent operation of a virtualized network and service. Administrators should be able to control routing/ signaling protocols depending on their policy, and manage Kumaki, et al. Expires July 15, 2014 [Page 9] Internet-Draft A Virtualized Network Element Framework January 2014 virtualized resources allocated to them (i.e. CPU, Memory, and other platform resources). Also, the virtualized networks should not affect each other in any way. 6.5. QoS/SLA The solution needs to provide an independent QoS/SLA per virtualized network depending on a service level. Each QoS on the virtualized network should support eight service levels at least. 6.6. Load Balancing Each virtualized network may create multiple paths towards a destination (depending on the control plane algorithms running on the virtual network). The network elements should support traffic load balancing, as they would do for a physical network. 6.7. Scale The solution should scale well with consideration to at least the following metrics. 1. The number of virtualized networks 2. The number of nodes 3. The number of routes and MPLS LSPs on virtualized networks 7. Architecture This section defines the VNEF architectural framework and the associated logical components. 7.1. Element Virtualization Element virtualization is the mechanism to create an isolated environment per service on each network element running on the common backbone. Virtual machine (VM) technology offers a way to implement element virtualization. Essentially, multiple VMs are created on the platform by configuration. Each VM represents a service, running its own operating system, routing protocols, management software, and other applications. It also satisfies the requirement of independent software upgradeability per service. Note that VMs are one way of providing element virtualization. The implementation may choose to use another container strategy as long as the requirements are satisfied. Kumaki, et al. Expires July 15, 2014 [Page 10] Internet-Draft A Virtualized Network Element Framework January 2014 +---------+ +---------+ |Service_1| |Service_n| | | ... | | +---------+ +---------+ +---------------------------+ | Service Manager | +---------------------------+ Figure 5: Element Virtualization Figure 5 describes the conceptual model for element virtualization. "Service Manager" operates as the "owner" software of all resources in the network element. Its principal role is to arbitrate and manage the resources of the network element so that multiple service environments can run in parallel. It responds to operator requests to create services, assign resources, and associate a set of physical or logical interfaces with each service. In one particular implementation choice, the service manager spawns VMs for each service that is created by the service provider. The creation of virtual networks per service is described in the next section. For example, for network overlay model of building virtual networks, the service manager software acts as the server conduit that listens to requests from the service's software. The "service manager" also guarantees performance isolation between services. If VMs are utilized to implement element virtualization, there are well-known resource allocation mechanisms to allocate CPU and memory per VM. This needs to be configuration- driven so that the operator can deploy customized policies for resource allocation (e.g. proportional sharing or priviledged allocation for certain services). 7.2. Network Virtualization Network virtualization is the mechanism to create independent virtual networks over the same physical network infrastructure. The primary characteristics of virtual networks are: o Each virtual network can be engineered and managed separately (by the operator of the corresponding service). As an example, all routing protocols should be able to run on each virtual network to create the desired logical topology. o Isolation of one service's traffic from other services. Virtual network constructs are not new (e.g. Kumaki, et al. Expires July 15, 2014 [Page 11] Internet-Draft A Virtualized Network Element Framework January 2014 [I-D.ietf-nvo3-overlay-problem-statement] describes a few of the constructs). We focus on network overlays to create virtual networks in this document. More specifically, we will discuss a specific technology of a network overlay design using MPLS LSPs: GMPLS UNI ([RFC4208]). The LSPs can either be created inline on the network element or as a separate layer ([RFC5440], [I-D.crabbe-pce-stateful-pce]). GMPLS UNI calls for a clean separation between the edge nodes and core nodes. We find that suitable in the context of the interface between a service and the core network. The reference model from [RFC4208] is reproduced here. Overlay Overlay Network +----------------------------------+ Network +---------+ | | +---------+ | +----+ | | +-----+ +-----+ +-----+ | | +----+ | | | | | | | | | | | | | | | | | | -+ EN +-+-----+--+ CN +----+ CN +----+ CN +---+-----+-+ EN +- | | | | | +--+--| | | | | | | | | | | | +----+ | | | +--+--+ +--+--+ +--+--+ | | +----+ | | | | | | | | | | | +---------+ | | | | | | +---------+ | | | | | | +---------+ | | | | | | +---------+ | | | | +--+--+ | +--+--+ | | | | +----+ | | | | | +-------+ | | | +----+ | | | +-+--+ | | CN +---------------+ CN | | | | | | | -+ EN +-+-----+--+ | | +---+-----+-+ EN +- | | | | | | +-----+ +-----+ | | | | | | +----+ | | | | +----+ | | | +----------------------------------+ | | +---------+ Core Network +---------+ Overlay Overlay Network Network Legend: EN - Edge or Service Node CN - Core Node Figure 6: Overlay Network The core network depicted as the large box in the center is the representation of the physical network. The nodes marked as 'EN' and described as "Overlay Network" represent a single virtual network for a particular service. In essence, the 'EN' nodes belong to a service (hosted by a virtualized network element as described in Kumaki, et al. Expires July 15, 2014 [Page 12] Internet-Draft A Virtualized Network Element Framework January 2014 Section 7.1). They request links to be established between them to the 'CN' nodes. The links come up as GMPLS LSPs using the overlay signaling mechanism. Collectively, the links between all the 'EN' nodes belonging to a service represent the topology of that service. The links show up as normal interfaces to the 'EN' nodes. The 'EN' nodes can, thus, run normal routing protocols on the top to engineer the desired traffic forwarding paths. 7.3. Service Identification When the network element receives a packet from the edge, how does it choose which service the packet belongs to? There are a couple of design choices: 1. The edge-facing interfaces are associated with particular services by configuration. This could be the entire physical interface or divided into multiple sub-interfaces (e.g. VLANs) per service. 2. A per-service policy is configured to look at the packet attributes and dynamically identify a service. Similarly, when a packet is received from the core, the service is identified based on the virtual network that carried the packet. For example, if GMPLS UNI LSPs are used as the virtual network construct, the top MPLS label identifies the LSP and hence the service. 8. Comparison with Other Technologies 8.1. VRF Most of the routers today support the concept of a virtual routing and forwarding (VRF) [RFC4364]. Though there are separate forwarding tables maintained per VRF, the technology fails to meet the requirements discussed in this document as follows: There is no operational isolation between different VRFs on a network element There is no network isolation (both for control plane and forwarding plane) on the core network between VRFs. 8.2. Network Virtualization Over L3 [I-D.ietf-nvo3-overlay-problem-statement] describes an architectural requirement for virtualizing the network. However, it focuses primarily on data center and multi- tenancy and the requirements do Kumaki, et al. Expires July 15, 2014 [Page 13] Internet-Draft A Virtualized Network Element Framework January 2014 not apply to service provider networks. 9. Acknowledgments The authors wish to thank Tony Li, Akash Deshpande, Rex Fernando, and John Bettink for all the discussions that led to the architectural model described in the document. 10. IANA Considerations 11. Security Considerations 12. References 12.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 12.2. Informational References [I-D.crabbe-pce-stateful-pce] Medved, J., Minei, I., Crabbe, E., and R. Varga, "PCEP Extensions for Stateful PCE", draft-crabbe-pce-stateful-pce-02 (work in progress), January 2012. [I-D.ietf-mpls-ldp-multi-topology] Zhao, Q., Fang, L., Zhou, C., Li, L., and K. Raza, "LDP Extensions for Multi Topology Routing", draft-ietf-mpls-ldp-multi-topology-09 (work in progress), October 2013. [I-D.ietf-nvo3-overlay-problem-statement] Narten, T., Gray, E., Black, D., Fang, L., Kreeger, L., and M. Napierala, "Problem Statement: Overlays for Network Virtualization", draft-ietf-nvo3-overlay-problem-statement-04 (work in progress), July 2013. [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, "Forwarding and Control Element Separation (ForCES) Framework", RFC 3746, April 2004. Kumaki, et al. Expires July 15, 2014 [Page 14] Internet-Draft A Virtualized Network Element Framework January 2014 [RFC4110] Callon, R. and M. Suzuki, "A Framework for Layer 3 Provider-Provisioned Virtual Private Networks (PPVPNs)", RFC 4110, July 2005. [RFC4208] Swallow, G., Drake, J., Ishimatsu, H., and Y. Rekhter, "Generalized Multiprotocol Label Switching (GMPLS) User- Network Interface (UNI): Resource ReserVation Protocol- Traffic Engineering (RSVP-TE) Support for the Overlay Model", RFC 4208, October 2005. [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, February 2006. [RFC4915] Psenak, P., Mirtorabi, S., Roy, A., Nguyen, L., and P. Pillay-Esnault, "Multi-Topology (MT) Routing in OSPF", RFC 4915, June 2007. [RFC5120] Przygienda, T., Shen, N., and N. Sheth, "M-ISIS: Multi Topology (MT) Routing in Intermediate System to Intermediate Systems (IS-ISs)", RFC 5120, February 2008. [RFC5440] Vasseur, JP. and JL. Le Roux, "Path Computation Element (PCE) Communication Protocol (PCEP)", RFC 5440, March 2009. Authors' Addresses Kenji Kumaki KDDI Corporation Garden Air Tower Iidabashi, Chiyoda-ku, Tokyo, 102-8460 Japan Email: ke-kumaki@kddi.com Pradosh Mohapatra Cumulus Networks Email: pmohapat@cumulusnetworks.com David Meyer Brocade Email: dmm@1-4-5.net Kumaki, et al. Expires July 15, 2014 [Page 15] Internet-Draft A Virtualized Network Element Framework January 2014 Kannan Varadhan Cisco Systems Email: kvaradha@cisco.com Zafar Ali Cisco Systems Email: zali@cisco.com Kumaki, et al. Expires July 15, 2014 [Page 16]