nfvrg R. Szabo, Ed. Internet-Draft Ericsson Intended status: Informational S. Lee, Ed. Expires: May 3, 2017 ETRI N. Figueira Brocade October 30, 2016 Policy-Based Resource Management draft-irtf-nfvrg-policy-based-resource-management-02 Abstract abstract to be defined Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 3, 2017. Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Szabo, et al. Expires May 3, 2017 [Page 1] Internet-Draft Policy-based RM October 2016 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4 5. Architecture Considerations . . . . . . . . . . . . . . . . . 4 5.1. MANO Architecture . . . . . . . . . . . . . . . . . . . . 5 5.2. Policies in the MANO Architecture . . . . . . . . . . . . 8 5.3. Global vs Local Policies . . . . . . . . . . . . . . . . 9 5.4. Hierarchical Policy Framework . . . . . . . . . . . . . . 10 5.4.1. Mapping to Hierarchical Resource Orchestration . . . 12 5.5. Policy Pub/Sub Bus . . . . . . . . . . . . . . . . . . . 13 5.5.1. Pub/sub bus in the hierarchical framework . . . . . . 15 5.6. Policy Intent Statement versus Subsystem Actions and Configurations . . . . . . . . . . . . . . . . . . . . . 17 5.7. Static vs Dynamic vs Autonomic Policies . . . . . . . . . 17 5.8. Policy Conflicts and Resolution . . . . . . . . . . . . . 17 5.9. Soft vs Hard Policy Constraints . . . . . . . . . . . . . 17 5.10. Operational Policies for Resource management . . . . . . 17 5.10.1. Operational Policies at NFVO . . . . . . . . . . . . 19 5.10.2. Operational Policies at VIM/WIM . . . . . . . . . . 19 6. Policy-Based Resource Management Examples . . . . . . . . . . 20 6.1. Policy-Based Multipoint Ethernet Service . . . . . . . . 20 6.2. Policy-Based NFV Placement . . . . . . . . . . . . . . . 20 6.3. Policy-Based VNF-FG Management . . . . . . . . . . . . . 20 6.4. Policy-Based Fault Management . . . . . . . . . . . . . . 22 7. Implementation Examples . . . . . . . . . . . . . . . . . . . 28 8. Gaps and Open Questions . . . . . . . . . . . . . . . . . . . 28 9. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 28 9.1. Relation to other IETF/IRTF activities . . . . . . . . . 28 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 28 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 28 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 13. Security Considerations . . . . . . . . . . . . . . . . . . . 29 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 29 14.1. Normative References . . . . . . . . . . . . . . . . . . 29 14.2. Informative References . . . . . . . . . . . . . . . . . 29 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32 1. Introduction NFV "Point of Presence" (PoP) will be likely constrained in compute and storage capacity. Since practically all NFV PoPs are foreseen to be distributed, inter-datacenter network capacity is also a constraint. Additionally, energy is also a constraint, both as a general concern for NFV operators, and in particular for specific- Szabo, et al. Expires May 3, 2017 [Page 2] Internet-Draft Policy-based RM October 2016 purpose NFV PoPs such as those in mobile base stations. This draft focuses on the optimized resource management and workload distribution based on policy to address such contraints. 1.1. Scope For the first version of the draft, only the research group currently adopted drafts (i.e., [I-D.norival-nfvrg-nfv-policy-arch], [I-D.irtf-nfvrg-resource-management-service-chain], and [I-D.unify-nfvrg-recursive-programming]) are considered as inputs to this document. The initial goal is to summarize these inputs and to assess gaps and open questions. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 3. Definitions This document uses the terms of [ETSI-NFV-TERM]: o MANO - Management and Orchestration: Describes the architecture framework to manage NFVI and orchestrate the allocation of resources needed by the NSs and VNFs. o NF - Network Functions: A functional building block within a network infrastructure, which has well-defined external interfaces and a well-defined functional behavior. o NFV Framework: The totality of all entities, reference points, information models and other constructs defined by the specifications published by the ETSI ISG NFV. o NFVI - NFV Infrastructure: The NFV-Infrastructure is the totality of all hardware and software components which build up the environment in which VNFs are deployed. o NFVI-PoP: A location or point of presence that hosts NFV infrastructure o NFVO - Network Function Virtualization Orchestrator: The NFV Orchestrator is in charge of the network wide orchestration and management of NFV (infrastructure and software) resources, and realizing NFV service topology on the NFVI. Szabo, et al. Expires May 3, 2017 [Page 3] Internet-Draft Policy-based RM October 2016 o NS - Network service: A composition of network functions and defined by its functional and behavioural specification. o VNF - Virtualized Network Function: An implementation of an NF that can be deployed on a Network Function Virtualization Infrastructure (NFVI). o VNF-FG - VNF Forwarding Graph: A NF forwarding graph where at least one node is a VNF. Additionally, we use the following terms: o NFP - Network Forwarding Path: The sequence of hardware/software switching ports and operations in the NFV network infrastructure as configured by management and orchestration that implements a logical VNF forwarding graph "link" connecting VNF "node" logical interfaces. o Virtual Link: A set of connection points along with the connectivity relationship between them and any associated target performance metrics (e.g. bandwidth, latency, QoS). The Virtual Link can interconnect two or more entities (VNF components, VNFs, or PNFs). o Scaling: Ability to dynamically extend/reduce resources granted to the Virtual Network function (VNF) as needed. o NFVIaaS: NFV infrastructure as a service to other SP customers. o SDN: Software Defined Networking. o BSS: Business Support Systems o OSS: Operation Support Systems o DC: Data Center o VM: Virtual machine 4. Requirements tbd 5. Architecture Considerations Szabo, et al. Expires May 3, 2017 [Page 4] Internet-Draft Policy-based RM October 2016 5.1. MANO Architecture According to the ETSI MANO framework [ETSI-NFV-MANO], an NFVO is split into two functions (see Figure 1): o The orchestration of NFVI resources across multiple VIMs, fulfilling the Resource Orchestration functions. The NFVO uses the Resource Orchestration functionality to provide services that support accessing NFVI resources in an abstracted manner independently of any VIMs, as well as governance of VNF instances sharing resources of the NFVI infrastructure o The lifecycle management of Network Services, fulfilling the network Service Orchestration functions. Similarly, a VIM is split into two functions (see Figure 1): o Orchestrating the allocation/upgrade/release/reclamation of NFVI resources (including the optimization of such resources usage), and o managing the association of the virtualised resources to the physical compute, storage, networking resources. Szabo, et al. Expires May 3, 2017 [Page 5] Internet-Draft Policy-based RM October 2016 +-------------------+ |NVFO | | +--------------+ | | |NFVO: | | | |Service | | | |Lifecycle | | | |Management | | | +------+-------+ | | | | | +------+-------+ | | |NFVO: | | | |Resource | | | |Orchestration | | | +--+---+----+--+ | +-----|---|----|----+ / | \ /---------/ | \------------\ / | \ +-------------|-----+ +--------|----------+ +------|------------+ |VIM | | |VIM | | |VIM | | | +----------+---+ | | +-----+--------+ | | +---+----------+ | | |VIM: | | | |VIM: | | | |VIM: | | | |Orchestration | | | |Orchestration | | | |Orchestration | | | |& | | | |& | | | |& | | | |Optimization | | | |Optimization | | | |Optimization | | | +------+-------+ | | +------+-------+ | | +------+-------+ | | | | | | | | | | | +------+-------+ | | +------+-------+ | | +------+-------+ | | |VIM: | | | |VIM: | | | |VIM: | | | |Virtualized 2 | | | |Virtualized 2 | | | |Virtualized 2 | | | |Pys mapping | | | |Pys mapping | | | |Pys mapping | | | +--------------+ | | +--------------+ | | +--------------+ | +-------------------+ +-------------------+ +-------------------+ Figure 1: Functional decomposition of the NFVO and the VIM according to the ETSI MANO In Figure 2 we show various policies mapped to the MANO architecture (see Section 5.2 for more dicussions on policies in the MANO architeture): o Tenant Policies: Tenant policies exist whenever a domain offers a virtualization service to more than one consumer. User tenants may exists at the northbound of the NFVO. Additionally, if a VIM exposes resource services to more than one NFVO, then each NFVO may appear as a tenant (virtualization consumer) at the northbound of the VIM. Szabo, et al. Expires May 3, 2017 [Page 6] Internet-Draft Policy-based RM October 2016 o Wherever virtualization services are produced or consumed corresponding export and import policies may exist. Export policies govern the details of resources, capabilities, costs, etc. exposed to consumers. In turn, consumers (tenants) apply import policies to filter, tweak, annotate resources and services received from their southbound domains. An entity may at the same time consume and produce virtualization services hence apply both import and export policies. o Operational policies support the business logic realized by the domain's ownership. They are often associated with Operations or Business Support Systems (OSS or BSS) and frequently determine operational objectives like energy optimization, utilization targets, offered services, charging models, etc. Operational policies may be split according to different control plane layers, for example, i) lifecycle and ii) resource management layers within the NFVO. Szabo, et al. Expires May 3, 2017 [Page 7] Internet-Draft Policy-based RM October 2016 T1 T2...Tn | | | +-----|--|---|------+ |NVFO | | | | Tenant | +--+--+---+----+ | <- Policies | |NFVO: | | Operational | |Service | | Policies-> | |Lifecycle | | | |Management | | | +------+-------+ | | | | | +------+-------+ | | |NFVO: | | Operational | |Resource | | Policies-> | |Orchestration | | ^ | +--+---+----+--+ | |Import to +-----|---|---------+ |Policies other NFVO / \ \ +-------+ \ \ / \ to NFVO ^ +------\------|-----+ \ / |Export |VIM \ | | \ / |Policies | +-----+----+---+ | +--------|----|-----+ | |VIM: | | |VIM | | | Tenant | |Orchestration | | | +-----+----+---+ | <- Policies | |& | | | |VIM: | | | |Optimization | | . | |Orchestration | | | +------+-------+ | . | |& | | <- Operational | | | | |Optimization | | Policies | +------+-------+ | | +------+-------+ | | |VIM: | | | | | | |Virtualized 2 | | | +------+-------+ | | |Pys mapping | | | |VIM: | | <- Operational | +--------------+ | | |Virtualized 2 | | Policies +-------------------+ | |Pys mapping | | | +--------------+ | +-------------------+ Figure 2: Policies within the MANO framework 5.2. Policies in the MANO Architecture The current industry work in the area of policy for NFV is mostly considered in the framework of general cloud services, and typically focused on individual subsystems and addressing very specific use cases or environments. For example, [ETSI-NFV-WHITE-PAPER] addresses network subsystem policy for network virtualization, [ODL-GB-POLICY] and [ODL-NIC-PROJECT] are open source projects in the area of network Szabo, et al. Expires May 3, 2017 [Page 8] Internet-Draft Policy-based RM October 2016 policy as part of the OpenDaylight [ODL-SDN-CONTROLLER] software defined networking (SDN) controller framework, [RFC3060] specifies an information model for network policy, [VM-HOSTING-NET-CLUSTER] focuses on placement and migration policies for distributed virtual computing, [OPENSTACK-CONGRESS] is an open source project proposal in OpenStack [OPENSTACK] to address policy for general cloud environments. A policy framework applicable to the MANO architure must consider NFV services from the perspective of overall orchestration requirements for services involving multiple subsystems (e.g., Figure 1 and Figure 2). While this document discusses policy atributes as applicable to the MANO architecture, the general topic of policy has already been intensively studied and documented on numerous publications over the past 10 to 15 years (see [RFC3060], [POLICY-FRAMEWORK-WG], [RFC3670], [RFC3198], and [CERI-DATALOG] to name a few). This document's purpose is to discuss and document a policy framework applicable to the MANO architecture using known policy concepts and theories to address the unique requirements of NFV services including multiple PoPs and networks forming hierarchical domain architectures [SDN-MULTI-DOMAIN]. With the above goals, this document analyses "global versus local policies" (Section 5.3), a "hierarchical policy framework" (Section 5.4) to address the demanding and growing requirements of NFV environments, a "policy pub/sub bus in the hierarchical framework" (Section 5.5), "policy intent versus subsystem actions" (Section 5.6), "static versus dynamic versus autonomic policies" (Section 5.7), "policy conflict detection and resolution" (Section 5.8), and "soft versus hard policy constraints" (Section 5.9), which can be relevant to resource management in service chains [RESOURCE-MGMT-SERVICE-CHAIN]. 5.3. Global vs Local Policies Some policies may be subsystem specific in scope, while others may have broader scope and interact with multiple subsystems. For example, a policy constraining certain customer types (or specific customers) to only use certain server types for VNF or Virtual Machine (VM) deployment would be within the scope of the compute subsystem. A policy dictating that a given customer type (or specific customers) must be given "platinum treatment" could have different implications on different subsystems. As shown in Figure 8, that "platinum treatment" could be translated to servers of a given performance specification in a compute subsystem and storage of a given performance specification in a storage subsystem. Szabo, et al. Expires May 3, 2017 [Page 9] Internet-Draft Policy-based RM October 2016 Policies with broader scope, or global policies, would be defined outside affected subsystems and enforced by a global policy engine (Figure 3), while subsystem-specific policies or local policies, would be defined and enforced at the local policy engines of the respective subsystems. Examples of sub-system policies can include thresholds for utilization of sub-system resources, affinity/anti-affinity constraints with regard to utilization or mapping of sub-system resources for specific tasks, network services, or workloads, or monitoring constraints regarding under-utilization or over- utilization of sub-system resources. +----------------------------------------------------------------+ | +----------------------------------------------+ | | | Global Policy Engine | | | +----------------------------------------------+ | | | | +----------------------------------------------+ | | | Global Policies | | | +----------------------------------------------+ | +----------------------------------------------------------------+ ^ ^ ^ ^ | | | | V V V V +-------------+ +-------------+ +-------------+ +-------------+ |Compute | |Network | |Storage | |Whatever | |Subsystem | |Subsystem | |Subsystem | |Subsystem | | | | | | | | | |Local Policy | |Local Policy | |Local Policy | |Local Policy | |Engine | |Engine | |Engine | |Engine | | | | | | | | | |Local | |Local | |Local | |Local | |Policies: | |Policies | |Policies | |Policies | | P0, P1, | | P0, P1, | | P0, P1, | | P0, P1, | | | | | | | | | +-------------+ +-------------+ +-------------+ +-------------+ Figure 3: Global versus Local Policy Engines 5.4. Hierarchical Policy Framework So far, we have referenced compute, network, and storage as subsystems examples. However, the following subsystems may also support policy engines and subsystem specific policies: o SDN Controllers, e.g., OpenDaylight [ODL-SDN-CONTROLLER]. Szabo, et al. Expires May 3, 2017 [Page 10] Internet-Draft Policy-based RM October 2016 o OpenStack [OPENSTACK] components such as, Neutron, Cinder, Nova, and etc. o Directories, e.g., LDAP, ActiveDirectory, and etc. o Applications in general, e.g., standalone or on top of OpenDaylight or OpenStack. o Physical and virtual network elements, e.g., routers, firewalls, application delivery controllers (ADCs), and etc. o Energy subsystems, e.g., OpenStack Neat [OPENSTACK-NEAT]. Therefore, a policy framework may involve a multitude of subsystems. Subsystems may include other lower level subsystems, e.g., Neutron [OPENSTACK-NEUTRON] would be a lower level subsystem in the OpenStack subsystem. In other words, the policy framework is hierarchical in nature, where the policy engine of a subsystem may be viewed as a higher level policy engine by lower level subsystems. In fact, the global policy engine in Figure 3 could be the policy engine of a Data Center subsystem and multiple Data Center subsystems could be grouped in a region containing a region global policy engine. In addition, one could define regions inside regions, hierarchically, as shown in Figure 4. Metro and wide-area network (WAN) used to interconnect data centers would also be independent subsystems with their own policy engines. Szabo, et al. Expires May 3, 2017 [Page 11] Internet-Draft Policy-based RM October 2016 To higher level domain ^ Region 1 | Domain V +-------------------+ +-------------------+ | +---------------+ | | +---------------+ | | |Region 1 Global| |<------>| |WAN 1 Global | | | |Policy Engine | | | |Policy Engine | | | +---------------+ | | +---------------+ | | | | | | +---------------+ | | +---------------+ | | |Whatever | | | |Whatever | | | |Subsystems | | | |Subsystems | | | | | | | | | | | |Local Policy | | | |Local Policy | | | |Engines | | | |Engines | | | +---------------+ | | +---------------+ | +-------------------+ +-------------------+ ^ ^ | | | +-------------------------+ | | DC 1 Domain V DC N Domain V +-------------------+ +-------------------+ | +---------------+ | | +---------------+ | | |DC 1 Global | | | |DC N Global | | | |Policy Engine | | | |Policy Engine | | | +---------------+ | | +---------------+ | | | | | | +---------------+ | | +---------------+ | | |Whatever | | | |Whatever | | | |Subsystems | | | |Subsystems | | | | | | | | | | | |Local Policy | | | |Local Policy | | | |Engines | | | |Engines | | | +---------------+ | | +---------------+ | +-------------------+ +-------------------+ Figure 4: A Hierarchical Policy Framework 5.4.1. Mapping to Hierarchical Resource Orchestration If the MANO framework is extended to multi layer hierarchies [I-D.unify-nfvrg-recursive-programming], then a potential mapping of the hierarchical policies to the MANO architecture is shown in Figure 5 Szabo, et al. Expires May 3, 2017 [Page 12] Internet-Draft Policy-based RM October 2016 T1 T2...Tn +--|--|----|---+ | | |Service | Domain 4 | Orchestrator| ************************************ +--+-----------+ T1 T2...Tn *******|****************** Tenant Policies -> +--|--|----|---+ | | NFVO: | | | Service | | <-Domain Policies | Lifecycle | | | Orchestrator | | +-------+------+ / Domain 3 | / ******************** +-+---------+--+ <-Tenant Policies * |NFVO: | ****************** * |Rersource | <-Domain Policies T1 T2...Tn * * |Orchestration | +--|--|----|---+ * * +--+---+-------+ | NFVO: | * ********|***|************************ | Service | * / | | Lifecycle | /---------/ | <-Domain Policies | Orchestrator | / * | +---------+----+ | * | | | * | +--+-------+---+* | <-Tenant Policies | |* | |NFVO: |* | <-Domain Policies |Rersource |* | |Orchestration |* | +------+-------+* | /|\ * *********|********** +------+-------+* * +------+-------+ * <-Tenant Policies +--|a/VIM: |* * |VIM: | * +--|b |Resource |* * |Resource | * <-Domain Policies |c | |Orchestrationn|* * |Orchestration | * | | +--------------+* * +--------------+ * Domain 1 * * Domain 2 * ************************ * * Figure 5: Policies in a Hierarchical Orchestration View 5.5. Policy Pub/Sub Bus In [I-D.irtf-nfvrg-nfv-policy-arch] the authors argued for the need of policy subsystems to subscribe to policy updates at a higher policy level. A policy publication/subscription (pub/sub) bus would be required as shown in Figure 6. Szabo, et al. Expires May 3, 2017 [Page 13] Internet-Draft Policy-based RM October 2016 +----------------------------------------------------------------+ | +----------------------------------------------+ | | | Global Policy Engine | | | +----------------------------------------------+ | | | | +----------------------------------------------+ | | | Global Policies | | | +----------------------------------------------+ | +----------------------------------------------------------------+ ^ | | Policy Pub/Sub Bus V -------------------------------------------------------------- ^ ^ ^ ^ | | | | | | | | V V V V +-------------+ +-------------+ +-------------+ +-------------+ |Compute | |Network | |Storage | |Whatever | |Subsystem | |Subsystem | |Subsystem | |Subsystem | | | | | | | | | |Local Policy | |Local Policy | |Local Policy | |Local Policy | |Engine | |Engine | |Engine | |Engine | | | | | | | | | |Local | |Local | |Local | |Local | |Policies: | |Policies | |Policies | |Policies | | P0, P1, | | P0, P1, | | P0, P1, | | P0, P1, | | | | | | | | | +-------------+ +-------------+ +-------------+ +-------------+ Figure 6: A Policy Pub/Sub Bus A higher tier policy engine would communicate policies to lower tier policy engines using a policy pub/sub bus. Conversely, lower tier policy engines would communicate their configured policies and services to the higher tier policy engine using the same policy pub/ sub bus. Such communications require each policy pub/sub bus to have a pre-defined/pre-configured policy "name space". For example, a pub/sub bus could define services using the name space "Platinum", "Gold", and "Silver". A policy could then be communicated over that pub/sub bus specifying a Silver service requirement. In a hierarchical policy framework, a policy engine may use more than one policy pub/sub bus, e.g., a policy pub/sub bus named "H" to communicate with a higher tier policy engine and a policy pub/sub bus named "L" to communicate with lower tier policy engines. As the name spaces of policy pub/sub buses H and L may be different, the policy Szabo, et al. Expires May 3, 2017 [Page 14] Internet-Draft Policy-based RM October 2016 engine would translate policies defined using the policy pub/sub bus H name space to policies defined using the policy pub/sub bus L name space, and vice-versa. 5.5.1. Pub/sub bus in the hierarchical framework Figure 7 shows the Pub/sub bus in the hierarchical MANO framework. Policy communications would employ a policy pub/sub bus between the subsystems' policy engines in the policy hierarchy (see Section 5.4). The global NFVO subsystem should have visibility into the policies defined locally at each PoP to be able to detect any potential global policy conflicts, e.g., a local PoP administrator could add a local policy that violates or conflicts with a global policy. In addition, the global NFVO subsystem would benefit from being able to import the currently configured services at each PoP. The global NFVO would use such information to monitor global policy conformance and also to facilitate detection of policy violations when new global policies are created, e.g., a global level administrator is about to add a new global policy that, if committed, would make certain already configured services a violation of the policy. The publication of subsystem service tables for consumption by a global policy engine is a concept used in the Congress [OPENSTACK-CONGRESS] OpenStack [OPENSTACK] project. Szabo, et al. Expires May 3, 2017 [Page 15] Internet-Draft Policy-based RM October 2016 Customers | V +--------------+ | Web Portal | +--------------+ ^ | V +-----------------+ +-----------------+ | OSS/BSS | | Global NFVO | | +-------------+ | | +-------------+ | | |OSS/BSS | | Policy | |NFVO | | | |Policy Engine|<---------->|Policy Engine| | | +-------------+ | | +-------------+ | | | | ^ | | ... | | | ... | +-----------------+ +--------|--------+ | Policy (Pub/Sub Bus) V ------------------------------------------- ^ ^ ^ | | | +-------|-------+ +-------|-------+ +-------|-------+ | PoP A | | | PoP B | | | PoP C | | | V | | V | | V | | +-----------+ | | +-----------+ | | +-----------+ | | |Local NFVO | | | |Local NFVO | | | |Local NFVO | | | | +-------+ | | | | +-------+ | | | | +-------+ | | | | |Policy | | | | | |Policy | | | | | |Policy | | | | | |Engine | | | | | |Engine | | | | | |Engine | | | | | +-------+ | | | | +-------+ | | | | +-------+ | | | +-----------+ | | +-----------+ | | +-----------+ | | ^ | | ^ | | ^ | | | | | | | | | | | V | | V | | V | | +-----------+ | | +-----------+ | | +-----------+ | | |VIM | | | |VIM | | | |VIM | | | | +-------+ | | | | +-------+ | | | | +-------+ | | | | |Policy | | | | | |Policy | | | | | |Policy | | | | | |Engine | | | | | |Engine | | | | | |Engine | | | | | +-------+ | | | | +-------+ | | | | +-------+ | | | +-----------+ | | +-----------+ | | +-----------+ | | ... | | ... | | ... | +---------------+ +---------------+ +---------------+ Figure 7: Pub/sub bus in the hierarchical MANO framework Szabo, et al. Expires May 3, 2017 [Page 16] Internet-Draft Policy-based RM October 2016 5.6. Policy Intent Statement versus Subsystem Actions and Configurations Content to be merged +----------------------------------------------------------------+ | Policy: "a given customer must be given Platinum treatment" | +----------------------------------------------------------------+ ^ ^ ^ ^ | | | | V V V V +-------------+ +-------------+ +-------------+ +-------------+ |Compute | |Network | |Storage | |Whatever | |Subsystem | |Subsystem | |Subsystem | |Subsystem | | | | | | | | | |Policy | |Policy | |Policy | |Policy | |translation: | |translation: | |translation: | |translation: | | | | | | | | | |Install | |Give customer| |Give customer| | ... | |customer VMs | |the best QoS,| |the fastest | | | |on servers | |which | |SSD storage. | | | |with 3GHz | |translates | | | | | |16-core Xeon | |here to set | | | | | |processors, | |DHCP to xx, | | | | | |and etc. | |and etc. | | | | | +-------------+ +-------------+ +-------------+ +-------------+ Figure 8: Example of Subsystem Translations of Policy Actions 5.7. Static vs Dynamic vs Autonomic Policies Content to be merged 5.8. Policy Conflicts and Resolution Content to be merged 5.9. Soft vs Hard Policy Constraints Content to be merged 5.10. Operational Policies for Resource management Szabo, et al. Expires May 3, 2017 [Page 17] Internet-Draft Policy-based RM October 2016 +-------------------+ |NFVO | <- AAA Policy | +--------------+ | | |NFVO: | | | |Service | | <- RD Policy | |Lifecycle | | | |Management | | | +------+-------+ | | | | | +------+-------+ | | |NFVO: | | | |Resource | | <- RS Policy | |Orchestration | | | +---+--+----+--+ | +------|--|----|----+ / \ \ / \ \ +-------+ / \ ---+ VNFM | / \ +---+---+ / \ | / \ | / \ | +---------+---------+ +-----+------+------+ |WIM | |VIM | | +--------------+ | | +--------------+ | | |WIM: | | | |VIM: | | | |Orchestration | | | |Orchestration | | <- RA Policy | |& | | | |& | | | |Optimization | | | |Optimization | | | +------+-------+ | | +------+-------+ | | | | | | | | +------+-------+ | | +------+-------+ | | |WIM: | | | |VIM: | | | |Virtualized 2 | | | |Virtualized 2 | | <- RE Policy | |Pys mapping | | | |Pys mapping | | | +--------------+ | | +--------------+ | +-------------------+ +-------------------+ Figure 9: Operational policies for resource management The use of NFVI resources for multiple network services can be optimized in various objectives as defined in the operational policies (as described in Section 5.2). The operational policies can be split to different layers of NFVO and VIM/WIM and they include 1) resource scheduling (RS) policy, resource adaptation (RD) policy and authentication, authorization, accounting (AAA) policy at NFVO, and 2) resource allocation (RA) policy and Szabo, et al. Expires May 3, 2017 [Page 18] Internet-Draft Policy-based RM October 2016 resource embedding (RE) policy at VIM/WIM. They can be mapped to the MANO architecture as shown in Figure 9. 5.10.1. Operational Policies at NFVO During NS/VNF lifecycles, states of NFVI/WAN resources or the performance of VNF and VL instances may vary in time (e.g., the performance degradation due to incorrect placement or incorrect forwarding action). Another concern for such dynamic changes is fail-over as a fundamental consideration, i.e., physical resources or virtualized resources in NFVI may fail during network services. These dynamic changes significantly could affect the overall performance for NS. Therefore, such dynamic changes triggered during NS/VNF lifecycles should be coped with for guaranteeing the NS performance and the optimized resource usage. Figure 9 shows that NFVO needs to enforce resource adaptation (RD) policy as an operational policy at NFVO. RD policy supports how NFVO adapts the allocated NFVI/WAN resources (e.g., VM migration, scaling) by dealing with triggered variations. RD policy engine can detect the changes from measurement and diagnosis from VNFM and/or VIM/WIM. Figure 9 also shows that NFVO needs to enforce resource scheduling (RS) policy. RS policy determines the locations of VNF and VL instances that constitute NS across multiple PoPs and WANs while optimally allocating NFVI and WAN resources to the instances. In particular, RD and RA policies would consider a business model from OSS/BSS which specifies operational (or business) objectives (e.g., overall energy consumption and NFVI resource utilization) within its domain and with taking account of (on-boarded) network service descriptor (NSD) as an NS policy including the virtualization aspects of application feature, QoS parameters, affinity, anti- affinity rules, and so on. On the one hand, for the user authorization, authentication, authorization, accounting (AAA) policy may be needed. Authentication policy provides a way of identifying a user while the authorization policy determines whether the user has the authority for virtualized resources (i.e., NFVI/WAN resources) to receive the network service or not. Accounting policy measures the resources the user consumes during the network service. This can include the amount of system time/data, and so on. 5.10.2. Operational Policies at VIM/WIM As shown in Figure 9, RA policy supports how each subsystem (e.g., compute, storage subsystem) in NFVI is allocated depending on the placement information from NFVO to further optimize the resource Szabo, et al. Expires May 3, 2017 [Page 19] Internet-Draft Policy-based RM October 2016 usage. Moreover, the assigned NFVI resources are embedded (or allocated) to physical resources in VIM/WIM depending on states and usage of resources by means of resource embedding (RE) policy as shown in Figure 9. In other words, RE policy determines and coordinates how the allocated virtual resources are mapped to physical resources. For example, RE policy may be updated when some physical resources are failed or a virtualization technique is changed. 6. Policy-Based Resource Management Examples 6.1. Policy-Based Multipoint Ethernet Service Content to be merged 6.2. Policy-Based NFV Placement Content to be merged 6.3. Policy-Based VNF-FG Management Szabo, et al. Expires May 3, 2017 [Page 20] Internet-Draft Policy-based RM October 2016 +-------------------+ | NFVO | | +---------------+ | | |NFVO | | | |RS Policy | | | |Engine | | | +---------------+ | | ^ | | | ... | +---------|---------+ | V Allocation information ----------------------------------------------- ^ ^ ^ | | | +-------|---------+ +-------|---------+ +-------|---------+ |vPoP A | | |WAN | | |vPoP B | | | V | | V | | V | | +-------------+ | | +-------------+ | | +-------------+ | | |VIM A | | | |WIM | | | |VIM B | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | | |RA Policy| | | | | |RA Policy| | | | | |RA Policy| | | | | |Engine | | | | | |Engine | | | | | |Engine | | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | +-------------+ | | +-------------+ | | +-------------+ | | ... | | ... | | ... | | | | | | | | +-----+ | | | | | | |VNF-A| | | | | | | +-----+ | | | | | | # +-----+ | | | | +-----+ | | #####|VNF-B|#############################|VNF-C| | | VL1 +-----+ | | VL2 | | +-----+ | | | | | | | +-----------------+ +-----------------+ +-----------------+ ### NFP Figure 10: Policy-based VNF-FG Management Another subsystem example for the policy framework is VNF-FG. When VNF-FGs of end-to-end network services are realized, NFVI resources across multiple NFVI-PoPs and WAN resources that connect among them should be allocated to the VNF-FGs. It depends on the target KPIs of individual VNF and VL instances that constitute VNF-FGs. In particular, in case of VNF-FG, chained performances and capabilities Szabo, et al. Expires May 3, 2017 [Page 21] Internet-Draft Policy-based RM October 2016 of VNF and VL instances need to be considered together with on VL instances the inter-connectivity between different NFVI-PoPs. For example, if one of the VNF instances or VL instances along the VNF-FG gets overloaded, the end-to-end network service may also get affected. Therefore, while features of such VNF-FG are carefully considered, proper operational policies for resource management (see Section 5.10) are required. As shown in Figure 10, consider a scenario where a user requests a VNF-FG composed of "VNF A-VL 1-VNF B-VL 2-VNF C". For the VNF-FG, an RA policy is enforced in which it is designed to avoid over- utilization of PoP A and to reduce latency on VL 1. Therefore, NFVO places VNF A, VNF B, and VL 1 on PoP A by consuming its computing and network resources to achieve low latency. On the other hand, VL 2 and VNF C is allocated to the resources of WAN and PoP B, respectively to avoid over-utilization of PoP A. On the one hand, dynamic changes such as a VNF failure significantly affect on the overall performance of VNF-FG since VNF-FG is a chain of VNF and VL instances. Thus, such dynamic changes should be coped with by RD policy for guaranteeing the VNF-FG performance and the optimized resource usage. A fault management for VNF-FG based on policy example is shown in Section 6.4. 6.4. Policy-Based Fault Management Szabo, et al. Expires May 3, 2017 [Page 22] Internet-Draft Policy-based RM October 2016 +-------------------+ | NFVO | | +---------------+ | | |NFVO | | | |RS Policy | | | |Engine | | | +---------------+ | | ^ | | | ... | +---------|---------+ V ---------------------------------------------- ^ ^ ^ | | | +-------|---------+ +-------|---------+ +-------|---------+ |vPoP A | | |vPoP B | | |vPoP C | | | V | | V | | V | | +-------------+ | | +-------------+ | | +-------------+ | | |VIM A | | | |VIM B | | | |VIM C | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | | |RA Policy| | | | | |RA Policy| | | | | |RA Policy| | | | | |Engine | | | | | |Engine | | | | | |Engine | | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | +-------------+ | | +-------------+ | | +-------------+ | | ... | | ... | | ... | | | | | | | | +-----+ | | +-----+ | | +-----+ | | |VNF-A|##############|VNF-B|#############|VNF-C| | | +-----+ VL-1 | | +-----+ VL-2| | +-----+ | | | | ^ | | | | | | | | | | | | | (failure) | | | | | | | | | +-----------------+ +-----------------+ +-----------------+ ### NFP Figure 11: Failure Scenario for VNF-FG Szabo, et al. Expires May 3, 2017 [Page 23] Internet-Draft Policy-based RM October 2016 +------------------------------------+ |NFVO | | +--------------------------------+ | | | RD policy engine | | | |Adapts resources to the failed | | | |elements while guaranteeing | | | |performance | | | +--------------------------------+ | +------------------+-----------------+ | | +-------------------+---------------------+ |VNFM/VIM/WIM | | +-------------------------------------+ | | | Diagnosis / Measurement | | | |A failure event | | | |Throughputs of VNF and VL instances | | | |Topological location... | | | +-------------------------------------+ | +-----------------------------------------+ Figure 12: Architecture for policy-based fault management As shown in Figure 11, consider a scenario that a VM related to VNF-B (i.e., a VNF-B instance) is failed in the given VNF-FG composed VNF- A, VNF-B, VNF-C in order. Note that the NFVI and WAN resources are already allocated to the instances by RS policy. For service continuity, failure of the VNF-B instance needs to be detected based on diagnosis function in VIM/VNFM and the failed one needs to be replaced with a new instance or to be assigned to the existing instance which is available. The diagnosis and measurement function may collect current performance measures and location for instances as well as such a failure event. Szabo, et al. Expires May 3, 2017 [Page 24] Internet-Draft Policy-based RM October 2016 +-------------------+ | NFVO | | +---------------+ | | |NFVO | | | |RD Policy | | | |Engine | | | +---------------+ | | ^ | | | ... | +---------|---------+ | V ---------------------------------------------- ^ ^ ^ | | | +-------|---------+ +-------|---------+ +-------|---------+ |vPoP A | | |vPoP B | | |vPoP C | | | V | | V | | V | | +-------------+ | | +-------------+ | | +-------------+ | | |VIM A | | | |VIM B | | | |VIM C | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | | |RA Policy| | | | | |RA Policy| | | | | |RA Policy| | | | | |Engine | | | | | |Engine | | | | | |Engine | | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | +-------------+ | | +-------------+ | | +-------------+ | | ... | | ... | | ... | | | | | | | | +-----+ | | +-----+ | | +-----+ | | |VNF-A| | | |VNF-B| | | |VNF-C| | | +-----+ | | +-----+ | | +-----+ | | # | | ^ | | # | | # New | | | | | # | | # VL-1 | | (failure) | # | | +-------+ | | | | # | | |New |#################################### | | |VNF-B | | | | | New VL-2 | | +-------+ | | | | | | | | | | | +-----------------+ +-----------------+ +-----------------+ ### NFP Figure 13: Re-instantiation for VNF-FG In the first case where a VNF instantiation is needed, a new VNF instantiation is determined by the RD policy engine in NFVO. For Szabo, et al. Expires May 3, 2017 [Page 25] Internet-Draft Policy-based RM October 2016 example, NFVO may avoid replacement of VNF B on NFVI-PoP B owing to high possibility of failure. Therefore, NFVO could instantiate VNF B on NFVI-PoP A or NFVI-PoP C with the setup of new connection points (CPs) while guaranteeing performance as shown in Figure 13. Szabo, et al. Expires May 3, 2017 [Page 26] Internet-Draft Policy-based RM October 2016 +-------------------+ | NFVO | | +---------------+ | | |NFVO | | | |RD Policy | | | |Engine | | | +---------------+ | | ^ | | | ... | +---------|---------+ | V ---------------------------------------------- ^ ^ ^ | | | +-------|---------+ +-------|---------+ +-------|---------+ |vPoP A | | |vPoP B | | |vPoP C | | | V | | V | | V | | +-------------+ | | +-------------+ | | +-------------+ | | |VIM A | | | |VIM B | | | |VIM C | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | | |RA Policy| | | | | |RA Policy| | | | | |RA Policy| | | | | |Engine | | | | | |Engine | | | | | |Engine | | | | | +---------+ | | | | +---------+ | | | | +---------+ | | | +-------------+ | | +-------------+ | | +-------------+ | | ... | | ... | | ... | | | | | | | | +-----+ | | +-----+ | | +-----+ | | |VNF-A| | | |VNF-B| | | |VNF-C| | | +-----+ | | +-----+ | | +-----+ | | # | | ^ | | # | | # | | | | | # | | # New | | (failure) | | # | | # VL-1 | | | | # New | | # | | +-------+ | | # VL-2 | | ##################| VNF-B1|############### | | | | +-------+ | | | +-----------------+ +-----------------+ +-----------------+ ### NFP Figure 14: No Re-instantiation for VNF-FG In the second case where no VNF instantiation is needed since a redundant VNF exists, the available VNF-B instance can used by the VNF-FG. For example, a redundant VNF B instance exists in NFVI-PoP Szabo, et al. Expires May 3, 2017 [Page 27] Internet-Draft Policy-based RM October 2016 B. Therefore, NFVO selects the instance and re-constructs two VLs as shown in Figure 14, and the corresponding NS can be continued without re-instantiation. 7. Implementation Examples tbd 8. Gaps and Open Questions tbd 9. Conclusions tbd 9.1. Relation to other IETF/IRTF activities tbd 10. Acknowledgements The research leading to some of the results described in this document has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 619609 - the UNIFY project. The views expressed here are those of the authors only. The European Commission is not liable for any use that may be made of the information in this document. 11. Contributors This document is the result of merging multiple drafts. This section acknowledges those who provided important ideas and text into this document. o Z. Qiang (Ericsson), M. Kind (DT-AG) from [I-D.unify-nfvrg-recursive-programming] o R. Krishnan (Dell), D. Lopez (Telefonica) and S. Wright (AT&T) from [I-D.irtf-nfvrg-nfv-policy-arch] o S. Lee (ETRI), S. Pack (KU), M-K. Shin (ETRI) and E. Paik (KT) from [I-D.irtf-nfvrg-resource-management-service-chain] Szabo, et al. Expires May 3, 2017 [Page 28] Internet-Draft Policy-based RM October 2016 12. IANA Considerations tbd 13. Security Considerations tbd 14. References 14.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3060] Moore, B., Ellesson, E., Strassner, J., and A. Westerinen, "Policy Core Information Model -- Version 1 Specification", RFC 3060, DOI 10.17487/RFC3060, February 2001, . [RFC3198] Westerinen, A., Schnizlein, J., Strassner, J., Scherling, M., Quinn, B., Herzog, S., Huynh, A., Carlson, M., Perry, J., and S. Waldbusser, "Terminology for Policy-Based Management", RFC 3198, DOI 10.17487/RFC3198, November 2001, . [RFC3670] Moore, B., Durham, D., Strassner, J., Westerinen, A., and W. Weiss, "Information Model for Describing Network Device QoS Datapath Mechanisms", RFC 3670, DOI 10.17487/RFC3670, January 2004, . 14.2. Informative References [CERI-DATALOG] Ceri, S. and others, "What you always wanted to know about Datalog (and never dared to ask", IEEE Transactions on Knowledge and Data Engineering, (Volume: 1, Issue: 1), August 2002. [ETSI-NFV-MANO] ETSI, "Network Function Virtualization (NFV) Management and Orchestration V0.6.3", October 2014. [ETSI-NFV-PER001] ETSI, "Network Function Virtualization: Performance and Portability Best Practices v1.1.1", June 2014. Szabo, et al. Expires May 3, 2017 [Page 29] Internet-Draft Policy-based RM October 2016 [ETSI-NFV-TERM] ETSI, "NFV Terminology for Main Concepts in NFV", October 2013. [ETSI-NFV-WHITE-PAPER] ETSI NFV White Paper, "Network Functions Virtualisation, An Introduction, Benefits, Enablers, Challenges, & Call for Action", . [I-D.ietf-bmwg-virtual-net] Morton, A., "Considerations for Benchmarking Virtual Network Functions and Their Infrastructure", draft-ietf- bmwg-virtual-net-04 (work in progress), August 2016. [I-D.irtf-nfvrg-nfv-policy-arch] Figueira, N., Krishnan, R., Lopez, D., Wright, S., and D. Krishnaswamy, "Policy Architecture and Framework for NFV Infrastructures", draft-irtf-nfvrg-nfv-policy-arch-04 (work in progress), September 2016. [I-D.irtf-nfvrg-resource-management-service-chain] Lee, S., Pack, S., Shin, M., Paik, E., and R. Browne, "Resource Management in Service Chaining", draft-irtf- nfvrg-resource-management-service-chain-03 (work in progress), March 2016. [I-D.liu-bmwg-virtual-network-benchmark] Liu, V., Liu, D., Mandeville, B., Hickman, B., and G. Zhang, "Benchmarking Methodology for Virtualization Network Performance", draft-liu-bmwg-virtual-network- benchmark-00 (work in progress), July 2014. [I-D.norival-nfvrg-nfv-policy-arch] Figueira, N., Krishnan, R., Lopez, D., and S. Wright, "Policy Architecture and Framework for NFV Infrastructures", draft-norival-nfvrg-nfv-policy-arch-04 (work in progress), June 2015. [I-D.unify-nfvrg-recursive-programming] Szabo, R., Qiang, Z., and M. Kind, "Towards recursive virtualization and programming for network and cloud resources", draft-unify-nfvrg-recursive-programming-02 (work in progress), October 2015. Szabo, et al. Expires May 3, 2017 [Page 30] Internet-Draft Policy-based RM October 2016 [ODL-GB-POLICY] "OpenDaylight Group Based Policy", . [ODL-NIC-PROJECT] "OpenDaylight Network Intent Composition Project", . [ODL-SDN-CONTROLLER] "OpenDaylight SDN Controller", . [OPENSTACK] "OpenStack", . [OPENSTACK-CONGRESS] "OpenStack Congress", . [OPENSTACK-NEAT] "OpenStack Neat", . [OPENSTACK-NEUTRON] "OpenStack Neutron", . [POLICY-FRAMEWORK-WG] "Policy Framework Working Group (IETF)", . [RESOURCE-MGMT-SERVICE-CHAIN] Lee, S. and others, "Resource Management in Service Chaining", . [SDN-MULTI-DOMAIN] Figueira, N. and R. Krishnan, "SDN Multi-Domain Orchestration and Control: Challenges and Innovative Future Directions", IEEE International Conference on Computing (ICNC), February 2015. [VM-HOSTING-NET-CLUSTER] Grit, L. and others, "Virtual Machine Hosting for Networked Clusters: Building the Foundations for "Autonomic" Orchestration", Virtualization Technology in Distributed Computing (VTDC), 2006. Szabo, et al. Expires May 3, 2017 [Page 31] Internet-Draft Policy-based RM October 2016 Authors' Addresses Robert Szabo (editor) Ericsson Konyves Kaman krt. 11 Budapest, EMEA 1097 Hungary Phone: +36703135738 Email: robert.szabo@ericsson.com Seungik Lee (editor) ETRI 218 Gajeong-ro Yuseung-Gu Daejeon 305-700 Korea Phone: +82 42 860 1483 Email: seungiklee@etri.re.kr Norival Figueira Brocade Email: nfigueir@Brocade.com Szabo, et al. Expires May 3, 2017 [Page 32]