idnits 2.17.1 draft-felix-nfvrg-recursive-orchestration-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 6, 2015) is 3211 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Research Task Force NFVRG G. Carrozzo, Ed. 3 Internet-Draft Nextworks 4 Intended status: Informational K. Pentikousis, Ed. 5 Expires: January 7, 2016 EICT 6 July 6, 2015 8 Recursive orchestration of federated virtual network functions 9 draft-felix-nfvrg-recursive-orchestration-00 11 Abstract 13 This document introduces a policy-based resource management and 14 orchestration framework which aims at contributing towards the 15 current namesake NFVRG near-term work items. 17 It describes key points of the recursive resource orchestration 18 framework developed within the wider research area of federated 19 virtual network function orchestration. The document also relates 20 this effort with respect to other orchestration frameworks, thus 21 addressing both the NFV research and practitioner communities. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on January 7, 2016. 40 Copyright Notice 42 Copyright (c) 2015 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 59 3. Recursive orchestration in federated virtual environments . . 5 60 3.1. Problem Statement . . . . . . . . . . . . . . . . . . . . 5 61 3.2. Resource Orchestrator . . . . . . . . . . . . . . . . . . 6 62 4. Policy-Based Resource Management . . . . . . . . . . . . . . 9 63 4.1. Certificate-based authN/authZ (C-BAS) . . . . . . . . . . 10 64 4.2. Resource Managers . . . . . . . . . . . . . . . . . . . . 11 65 5. Positioning w.r.t. existing Orchestration Frameworks . . . . 14 66 5.1. Openstack orchestration . . . . . . . . . . . . . . . . . 14 67 5.2. OpenMANO . . . . . . . . . . . . . . . . . . . . . . . . 15 68 5.3. Other orchestration approaches: federated SDN 69 infrastructures for research experimentation . . . . . . 15 70 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 71 7. Security Considerations . . . . . . . . . . . . . . . . . . . 17 72 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 73 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 17 74 10. Informative References . . . . . . . . . . . . . . . . . . . 17 75 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 77 1. Introduction 79 Today's Internet is a concatenation of IP networks interconnected by 80 many distributed functions integrated into a plethora of highly 81 specialized middleboxes. These elements implement complex network 82 functions like firewalls, NATs, DPI, traffic scrubbing, etc. The 83 product is a quite complex and rigid internetworking system in which 84 network administrators and users cannot easily determine what is 85 happening to traffic flows as they go toward destinations. In the 86 last decade networks, servers, storage technologies, and applications 87 have all undergone significant changes with the introduction of 88 virtualization, network overlays, and orchestration. Such 89 technologies have allowed network operators and service providers to 90 easily introduce a variety of (proprietary) hardware-based appliances 91 in order to improve their network manageability as well as rapidly 92 launch new services, keeping up with the pace of their users demand. 93 Therefore, the current Internet looks like a concatenation of 94 networks with many distributed functions, implemented via a plethora 95 of highly specialized middleboxes which implement firewalls, DPI, 96 NAT, traffic scrubbing, etc. [middlebox]. 98 Software Define Networking and programmable virtualized network 99 functions for flow processing are rapidly changing the current 100 scenario, extending the support of network functions by virtualized 101 and chained appliances beyond the virtual L2 switching over IP 102 networks (e.g. VXLAN, GRENV, STT) and the basic LAN based flow 103 pinpointing. 105 Virtual Functions of this kind, for network and non network (e.g. 106 computing) tasks, are generally available in heterogeneous pools 107 under different administrative domains, being them related to the 108 hosting infrastructures in which they originate. It is emerging a 109 need to interconnect, federate and implement policy control on these 110 pools of virtual resources, in order to abstract different 111 infrastructures, resources and functions, as well as procedures by 112 physical operators and infrastructure owners. This can allow 113 defining larger virtual overlays where different resources and 114 functions are deployed, combined and handled in the form of virtual 115 instances irrespective of the administrative domain and specific 116 technology from which they originate. Examples of application 117 contexts in which this federation of virtual function pools may occur 118 are: 120 o large scale experimentation over programmable networks, which 121 allows to reserve slices of network and non-network resources from 122 different federated providers to run experiments on network 123 control, protocols and algorithms at large scale (e.g. [FELIX]); 125 o virtual infrastructure operators, who intend to implement their 126 network service offer over a completely virtual infrastructure in 127 the form of virtual network nodes and functions, virtual servers 128 and storage, etc., all procured as a service from physical 129 providers. 131 This document discusses key points of the recursive resource 132 orchestration framework developed within the wider research area of 133 federated virtual network function orchestration. The proposed 134 architecture allows federation and integration of different network 135 and computing resources residing in a multi-domain heterogeneous 136 environment across different providers. To achieve this, the 137 architecture uses a combination of recursive and hierarchical 138 configurations for orchestration, request delegation and inter-domain 139 dependency management, with resource orchestrating entities (Resource 140 Orchestrators, RO) responsible for synchronization of resources 141 available in particular administrative domains. 143 This document is being discussed on the nfvrg@irtf.org mailing list. 145 2. Terminology 147 The following terminology is used in this document. 149 Virtual Network Function(VNF). One or more virtual machines running 150 different software and processes, on top of industry standard high 151 volume servers, switches and storage, or even cloud computing 152 infrastructure, and capable of implementing network functions 153 traditionally implemented via custom hardware appliances and 154 middleboxes (e.g. router, NAT, Firewall, load-balancer, etc.) 156 VNF Island. A set of virtualized network functions and related 157 network and IT resources under the same administrative ownership/ 158 control. A VNF island could consist of multiple zones, each 159 characterized by a specific set of control tools & interfaces. 161 VNF Zone. A set of virtual network functions grouped for homogeneity 162 of technologies and/or control tools and/or interfaces (e.g. L2 163 switching zone, optical switching zone, OF protocol controlled 164 zone, other transit domain zone with a control interface). The 165 major goal of defining SDN zones is to implement appropriate 166 policies for increasing availability, scalability and control of 167 the different resources of the island. Examples of zone 168 definitions are available in popular Cloud Management Systems 169 (CMS) like Cloudstack (e.g. refer to the Cloudstack Infrastructure 170 partitioning into regions, zones, pods, etc., [cloudstack]) and 171 OpenStack (e.g. refer to the infrastructure partitioning in 172 availability zones and host aggregates [openstack]). 174 Transit network domains. The network domains use a Bandwidth on 175 Demand Interface to expose automatically and on-demand control of 176 connectivity services and, optionally, inter-domain topology 177 exchange. In order to federate resources belonging to distant 178 facilities (i.e. islands/zones) it must be ensured that 179 interconnectivity is provided on-demand and with a specific 180 granularity. 182 Slice. A user-defined subset of virtual networking and IT resources, 183 created from the physical resources available in federated VNF 184 Zones and VNF Islands. A VNF Slice has the basic property of 185 being isolated from other slices defined over the same physical 186 resources, and being dynamically extensible across multiple VNF 187 Islands. Each VNF Slice instantiates the specific set of control 188 tools of the specific zones it traverses. 190 Resource Orchestrator (RO). Entity responsible for orchestrating 191 end-to-end network service and resources reservation in terms of 192 compute, storage and network functions over the infrastructure, as 193 well as delegating end-to-end resource and service provisioning in 194 a technology-agnostic way. 196 Resource Managers (RMs). Entity responsible for controlling and 197 managing different types of resources and/or network functions. 199 3. Recursive orchestration in federated virtual environments 201 3.1. Problem Statement 203 The coordinates creation of a virtual environment with pools of 204 virtual network and non-network functions from heterogeneous, multi- 205 domain and geographically distributed facilities requires appropriate 206 tools for resource and virtual function management and control 207 capable of orchestration and policy control across VNF islands, zones 208 and domains defined above. 210 Elements that belong to this control and orchestration layer operate 211 in a hierarchical way (parent-child) for efficient multi-domain 212 information management and sharing. This is generally referred as 213 Inter-island Orchestration Space [FELIX-D2.1] [FELIX-D2.2]. 215 Once the set of virtual network and non-network functions is 216 determined, reserved and deployed across the different islands, the 217 resulting virtual network environment is ready for being used as a 218 User Space by any tool or application the user wants to deploy in it. 220 In the Inter-island Orchestration Space (see Figure 1), Resource 221 Orchestrators (RO) are responsible for orchestrating end-to-end 222 network services and resources reservations in the whole 223 infrastructure. Moreover, ROs should be able to delegate end-to-end 224 resource and service provisioning in technology-agnostic way. 226 ROs are connected to different types of Resource Managers (RMs), 227 which are in turn used to control and manage different kinds of 228 technological resources. For example, the VNF RM WAN side provides 229 connectivity between L1/L2 network domains at the two ends. This 230 management can be achieved using frame, packet or circuit switching 231 technologies and should support different protocols. 233 On the other hand, the VNF RM (LAN side) manages the network 234 infrastructure composed of SDN-enabled devices, e.g. OpenFlow 235 switches or routers. In short, it can control the user traffic 236 environment by updating flow tables in physical devices. 238 In addition, the Virtual Function pool RM for computing resources is 239 responsible for setting up and configuring computing resources, i.e. 241 creating new virtual machine instances, powering on/off instances, 242 network interface card configuration, etc. 244 Authentication and Authorization Infrastructure (AAI) for 245 authenticating and authorizing users, is a cross layer function in 246 the Inter-island Orchestration Space, because it serving as a 'trust 247 anchor' to facilitate authN/authZ procedures in federated facilities. 249 Similarly, Monitoring allows to retrieve, correlate and abstract 250 statistics from the different components of the physical and virtual 251 resource pools and from the user's slice. 253 Figure 1 shows a parent RO coordinating orchestration actions at NFV 254 island level under the responsibility of two child ROs, each 255 orchestrating different types of RMs for the different kinds of 256 virtual network and non-network function pools. 258 +---+ +-------------------------------------------+ +---+ 259 | | | RESOURCE ORCHESTRATOR - RO | | | 260 | |-----------| (parent) |----| | 261 | | +-------------------------------------------+ | | 262 | | | | | | 263 | | | | | | 264 | | +-------------------------------------+ +---------+ | | 265 | M | | RO | | RO | | | 266 | O |--| (child) | | (child) |---| | 267 | N | +-------------------------------------+ +---------+ | A | 268 | I | | | | | | A | 269 | T | | | | | | A | 270 | O | +-----------+ +----------+ +----------+ +-----------+ | | 271 | R | | VF POOL | | VNF POOL | | VNF POOL | | Virtual | | | 272 | I |--| MANAGER | | MANAGER | | MANAGER | | pool |--| | 273 | N | |(computing)| |(LAN side)| |(WAN side)| | manager(s)| | | 274 | G | +-----------+ +----------+ +----------+ +-----------+ | | 275 | | | | | | | | 276 | | +----------+ +----------+ ----------+ +----------+ | | 277 | |--| VF | | VNF | | VNF | | VNF |--| | 278 +---+ +----------+ +----------+ +----------+ +----------+ +---+ 280 Figure 1: Recursive Orchestration architecture model 282 3.2. Resource Orchestrator 284 RO is the entity that orchestrates the different resources in the 285 Inter-island Orchestration Space. 287 There are two different modes in which RO may operate: 289 o Parent 291 o Child 293 For an inter-island federation, RO operates in parent mode and 294 attaches to child ROs, whilst in child mode ROs communicate with RMs. 296 One of RO's main objectives is to forward requests within the 297 infrastructure, either by: 299 o Passing user requests to the appropriate resource management 300 systems (RMs) in the layer below, as with any hierarchical mode. 302 o Proxying requests between other ROs in a recursive mode, depending 303 on the federation policy that is configured for the domain where 304 the RO is located. For this to work it is necessary to ensure 305 similar interfaces for each orchestrator. 307 Key functions of the RO can be summarized as follows. RO manages the 308 different VNF islands and users in terms of their resource and data 309 access policies. It mediates between the user and the technology 310 specific to a Resource Manager (RM) by means of delegation. It is 311 expected that different RMs will handle, for example, technology- 312 dependent aspects in SDN domains (VNF RM LAN side) and transit 313 network domains (VNF RM WAN side), as well non-network resource 314 pools. As part of this mediation, the RO will engage in the creation 315 (provisioning), maintenance, monitoring, and deletion (release) of 316 the used slices. 318 RO also maintains a high-level cross-island topological view, which 319 summarizes the different resources pools available along with their 320 inter-connections. This topology view is initialized and updated by 321 the underlying RMs, thus implementing a distributed hierarchical 322 resource discovery function. It determines which domains and which 323 inter-domain resources should be used to instantiate a given end-to- 324 end service for a particular experimenter's slice. 326 For example, based on a user request for a given type of service to 327 be instantiated in two remote islands, parent RO determines which 328 specific resource domains should be involved. 330 Finally, RO coordinates and ensures that the correct sequence of 331 actions takes place with respect to the operation of the technology- 332 specific RMs. This includes the provisioning of the slice resources 333 as per user's requirements. 335 RO also collects and correlates status alarms and warnings on 336 resources, either generated by the resources themselves or the 337 services managing them. This is done on a per-slice basis and 338 proceeds with reporting/notifying the corresponding users. 340 Different deployment models are possible for a Resource Orchestration 341 entity: 343 o hierarchical centralized (see Figure 2.A) 345 o distributed in chain (see Figure 2.B 347 o distributed full-mesh (see Figure 2.C) 349 o hierarchical hybrid (see Figure 3) 351 The hierarchical hybrid model is deemed to guarantee the optimal 352 trade-off between effectiveness of control, federation, trust 353 adjacencies and scalability. 355 +-----------+ 356 | RO | 357 +-----------+ 358 / | \ +------+ +------+ +------+ 359 / | \ | RO |---| RO |---| RO | 360 / | \ +------+ +------+ +------+ 361 +------+ +------+ +------+ 362 | RO | | RO | | RO | 363 +------+ +------+ +------+ 364 (A) (B) 366 +--------------------+ 367 | | 368 +------+ +------+ +------+ +------+ 369 | RO |---| RO |---| RO |---| RO | 370 +------+ +------+ +------+ +------+ 371 | | | | 372 | +--------------------+ | 373 +---------------------------------+ 374 (C) 376 Figure 2: RO deployment models 378 +-----------+ +-----------+ 379 | RO +----------------------+ RO | 380 +-----------+ +-----------+ 381 / | \ / | \ 382 / | \ / | \ 383 / | \ / | \ 384 +------+ +------+ +------+ +------+ +------+ +------+ 385 | RO | | RO | | RO | | RO | | RO | | RO | 386 +------+ +------+ +------+ +------+ +------+ +------+ 388 Figure 3: Hybrid RO deployment model 390 4. Policy-Based Resource Management 392 Aside from the main functions described above, each Resource Manager 393 is also part of the Authentication and Authorization Infrastructure 394 (AAI). 396 AAI provides the necessary mechanisms to authenticate and authorize 397 users, as well as provide accountability. In order to realize these 398 functions, our architecture suggests the implementation of a 399 ClearingHouse (CH) , which establishes the root of a trust chain. 400 This chain can then be used to verify the identity and privileges of 401 all actors in this architecture. 403 By using a certificate-based approach, the architecture has 404 flexibility to easily federate different VNF islands. By installing 405 ClearingHouse certificates, actors can be verified against different 406 ClearingHouses, and thus can utilize a multitude of resources. 408 A ClearingHouse (CH) comprises a set of related services supporting 409 AAA operations. CH serves as a central location to lookup 410 information about members, slices and other available services in the 411 VNF island. 413 There are three groups of CH services: 415 o Registration and management services to lookup for available 416 services in the facility as well as register new members, projects 417 and slice objects. 419 o Authentication and Authorization services to manage the 420 credentials of all entities and enforce predefined policies. 422 o Accountability services to facilitate tracking of all 423 transactions. 425 These services are offered by CH with the help of the following 426 functions and authorities. 428 o The Member Authority (MA) is responsible for managing and 429 asserting user attributes. It generates member certificates for 430 identification purposes and credentials to specify the attributes 431 and roles associated with each member. The MA maintains a 432 database of registered members and their associated information 433 including, but not limited to, certificates and credentials, SSH 434 (Secure Shell) and SSL (Secure Sockets Layer) keys as well as the 435 human readable identity information like real name, institute, 436 contact details. 438 o In addition, a Certificate Revocation List (CRL) can also be 439 accessed from MA for use in certificate verification process. 441 o The Slice Authority (SA) creates and manages slice objects and the 442 associated member credentials (called slice credentials). Slice 443 credentials map member roles and privileges on a slice object, 444 i.e., slice credentials authorize user actions at aggregates 445 within a slice context. SA also enables related operations on 446 slice objects like look up, modify, renew, etc. 448 o The Project Service (PS) hosted at SA maintains a list of existing 449 projects and asserts the member roles. 451 o The Service Registry (SR) serves as the primary network contact 452 point as it keeps a record of all available registered services 453 such as SA and MA and offers their URIs. 455 o The Logging Service (LS) realizes accountability by storing the 456 transaction details between user-agents and aggregate managers. 458 The user-agents and ROs can communicate with the CH through XMLRPC 459 calls over a secured connection (SSL). 461 AAI is ultimately responsible for granting access to the resources, 462 and can be further extended through policies, which are a set of 463 rules defined by the administrators to implement an upper-level 464 control on the resource usage (e.g. defining a maximum virtual memory 465 value for a VM resource or a maximum number of flow spaces). 467 4.1. Certificate-based authN/authZ (C-BAS) 469 Since VNF pools are finite, access to virtual functions and resources 470 should be policed according to set authorization levels throughout 471 the life-cycle of each experiment. 473 Access control is also required to ensure that infrastructures remain 474 operational. 476 Although a number of solutions for authentication (authN) and 477 authorization(authZ), such as Kerberos and LDAP, already exist, they 478 have several shortcomings: tight-coupling of authN/authZ mechanisms 479 with the implementation of the architecture; little or no regard for 480 re-usability (i.e., one authN/authZ architecture cannot be reused by 481 a different infrastructure); and no support for a standard access 482 interface between networks and the authN/authZ architecture. 484 C-BAS, certificate-based authN/authZ solution, is designed to serve 485 all these requirements and include i) multiple authoritative source 486 of trust, ii) flexible system of authorization, and iii) 487 synchronization of authN/authZ entities to realize federations. 489 For example, the Registry Service of C-BAS may be exploited to 490 implement load balancing and failover features. In addition, the 491 evolved architecture of C-BAS makes it robust against disruptions and 492 interference from attackers and enables support for various member 493 roles and permissions. 495 C-BAS employs X.509 certificates and SFA styled credentials to 496 realize AAA services. 498 The implementation of C-BAS is publicly available (www.eict.de/c-bas) 499 and is based on eiSoil (github.com/EICT/eiSoil/wiki) thus exploiting 500 its plugin capabilities that enable importing the functionality from 501 one plugin module to another. 503 4.2. Resource Managers 505 4.2.1. VNF Pool manager functionality 507 A VNF pool manager is a functional entity in charge of controlling a 508 specific type of VNFs, being the equivalent of the SFA Aggregate 509 Manager (see [SFA]). As such, a VNF pool manager is a Resource 510 Manager within the federation, capable of discovering resources, 511 capabilities ans functions from physical infrastructure, abstracting 512 them before publishing to the supervising RO and eventually capable 513 of managing specific technology-specific configurations and 514 provisioning towards the actual resource layer. 516 Whilst the northbound interface of the Resource Manager is abstract 517 and unified across different technology domains (e.g. based on REST 518 or XMLRPC), the southbound interface is based on the specific 519 interfaces exposed by the different types of resources (e.g. 520 OpenFlow, NETCONF, SNMP, CLI, OVSDB, etc.) 522 4.2.2. OpenFlow-based VNF pool manager 524 The VNF pool manager LAN side could be OpenFlow-based and provide the 525 mechanisms to control the network infrastructure inside a domain with 526 SDN-enabled hardware (typically, OpenFlow-enabled switches and 527 routers). The Inter-island Orchestration Space architecture is 528 agnostic of the physical network resources. For a SDN domain based 529 on OpenFlow users can control network behaviour by actively updating 530 the flow tables of the network elements. This update is usually done 531 by a SDN controller, which is configured according the user's 532 requirements. Typically, the controller will respond to an event 533 generated by a network element, such as a flow establishment request, 534 and update the flow tables appropriately. 536 For the network manager, the VNF pool manager LAN side will provide 537 management functionalities for the overall network resources and 538 virtual network functions in the LAN or data center. This element 539 acts as a proxy between the resources and the SDN controller, and 540 grants or denies the forwarding of control messages. The VNF pool 541 manager LAN side provides functions to build a unique flow space for 542 every experimenter so that traffic is isolated and distinguished from 543 that of other slices (e.g. like FlowVisor or OVX do). 545 These functionalities can prevent issues arising when several users 546 wish to use the same physical resources. In detail, a flow space can 547 contain a range of differentiators: source or destination IPs or MAC 548 addresses, TCP or UDP ports, for example. 550 One way to separate the traffic is assigning a VLAN tag to each 551 packet. In this case, the special purpose controller inspects the 552 incoming packet, identifies the VLAN tag and sends it to the 553 corresponding SDN controller. 555 4.2.3. Stitching Entity VNF pool manager 557 The Stitching Entity VNF pool Manager is among the pool managers WAN 558 side, and is in charge to control the Stitching Entity (SE), a 559 network element providing necessary translation mechanisms for a 560 slice setup on top of the L2 protocol stack performed in order to 561 hide from a user the real complexity of the multi-domain WAN 562 transport network. 564 The main responsibility of the SE is to provide at least one of the 565 following network functions: 567 o QinQ, to encapsulate slice traffic into a transport network 568 Ethernet frames, 570 o VLAN translation mechanism to hide from a user the actual VLAN 571 tagging, used by carrier networks while interconnecting two or 572 more VNF islands. 574 The SE RM communicates with an RO, a parent control entity, to i) 575 advertise an internal topology and capabilities of the SE under its 576 control, ii) receive requests, and iii) notify the RO about success 577 and failure events. 579 A single SE RM must relate only to a single RO and must be 580 implemented in each VNF island. A single SE RM is responsible for a 581 single SE or a group of SEs, which belong to a network domain and act 582 as an entry point to the island infrastructure. 584 4.2.4. Transit network VNF pool manager 586 The main responsibility of the Transit VNF pool manager (TN RM) is to 587 support the FELIX architecture with network connectivity mechanisms 588 within particular domains and between them. 590 In order to deliver the network services, it must be integrated with 591 its southbound interfaces within a particular network domain. Such a 592 domain can use different L1/L2 technologies and may be controlled by 593 a Network Management System (NMS) or by specific interfaces or 594 protocols that are technology-dependent, and unique in each case. 596 The Transit network VNF pool manager must communicate with an RO in 597 order to i) advertise resources under its control, ii) receive 598 requests, and iii) notify the RO about success and failure events. 600 A single TN RM must relate only to a single RO. A single TN RM is 601 responsible for a group of particular network resources, which belong 602 to a network domain and are usually managed by a single entity, i.e. 603 a network administrator or NMS. 605 TN RM usually manages L1/L2 transport networks, which are composed of 606 physical devices using frames/packets or circuit switching 607 technologies and support different protocols, e.g. MPLS/GMPLS. In 608 order to support inter-island connectivity between existing VNF 609 islands, the TN RM also supports the management of VPN set up and 610 tear down procedures. 612 The TN RM southbound interface can be based on Bandwidth on Demand 613 interfaces, like GMPLS UNI or similar approaches. 615 4.2.5. RM for virtual computing 617 The function of the Computing Resource pool Manager (C-RM) is to 618 provide a method to assign, set up and configure computing resources. 620 C-RM manages physical computing resources, and also the configuration 621 of its own slicing mechanisms (e.g. common hypervisors or other 622 virtualization stacks) and computer resources as presented to the 623 user (OS images, network interface configuration, and so on). 625 The management of physical computing resources should provide a 626 method for rebooting machines, remote control (of a machine's 627 console), or hard power on/off of a machine experiencing problems, 628 for example using a networked PDU (power distribution unit). 630 Management is typically performed only when problems occur, and when 631 a slice is created, destroyed or modified. Migration of computing 632 resources to other islands may also require reconfiguration. This 633 includes the configuration of network interfaces of the computing 634 resource, and setting the underlying resources (e.g. hypervisor, 635 physical machine), such that those interfaces are bridged onto the 636 correct physical interfaces. In particular, it may be necessary to 637 configure a slicing mechanism in this bridging, in the case where 638 multiple computing resources have to share a single physical 639 interface. This would typically be achieved using a (software-based) 640 SDN solution inside the virtualization platform. Once the SDN 641 solution has been properly set up, it becomes an SDN resource, which 642 is managed by the VNF pool manager LAN side. 644 5. Positioning w.r.t. existing Orchestration Frameworks 646 5.1. Openstack orchestration 648 Among cloud orchestration solution, OpenStack is the facto common 649 reference through its Heat module [os-heat]. 651 Openstack Heat implements an orchestration engine to launch multiple 652 composite cloud applications based on templates in the form of text 653 files that can be treated like code. 655 Many existing CloudFormation templates can be launched on OpenStack. 656 Heat provides both an OpenStack-native ReST API and a CloudFormation- 657 compatible Query API. 659 A Heat template describes the infrastructure for a cloud application 660 in a text file. Infrastructure resources that can be described 661 include: servers, floating ips, volumes, security groups, users, etc. 663 Templates can also specify the relationships between resources (e.g. 664 this volume is connected to this server). 666 Heat also provides an autoscaling service. 668 Heat primarily manages cloud infrastructure, does not support 669 federation and AAI is bundled in the OpenStack framework. 671 5.2. OpenMANO 673 OpenMANO is an open source project which implements the reference 674 architecture for Management & Orchestration under standardization at 675 ETSI's NFV ISG (NFV MANO) [openmano]. 677 OpenMANO consists of two main SW components: 679 o NFV VIM (Virtualised Infrastructure Manager) to provide computing 680 and networking capabilities and to deploy virtual machines. 682 o A reference implementation of an NFV-O (Network Functions 683 Virtualisation Orchestrator), which allows creation and deletion 684 of VNF templates, VNF instances, network service templates and 685 network service instances. 687 OpenMANO does not support federation and AAI as of today. 689 5.3. Other orchestration approaches: federated SDN infrastructures for 690 research experimentation 692 The FELIX project [FELIX] is part of an international research 693 experimentation infrastructure strategy (in Europe under the Future 694 Internet Research Experimentation - FIRE - framework), with a special 695 focus on SDN and Network Service Interface (NSI) developed by the 696 Open Grid Forum. FELIX is implementing federation and integration of 697 different network and computing resources controlled via SDN and NSI 698 in a multi-domain heterogeneous environment across, initially 699 spanning Europe and Japan. FELIX consortium has designed and is 700 implementing an architecture that extends and advances assets 701 previously developed in other Future Internet projects (e.g. 702 OFELIA), by realizing the federation concepts defined in SFA [SFA] 703 with a combination of recursive and hierarchical orchestration, 704 request delegation and inter-domain dependency management. Other 705 research testbeds have been working over the past year on federation 706 of SDN resources. Three of them are particularly relevant on the SDN 707 area: OFELIA, FIBRE and GridARS. 709 The OFELIA project [OFELIA] established a pan-European experimental 710 network facility which enables researchers to experiment with real 711 OpenFlow-enabled network equipment and to control and extend the 712 network itself in a precise and dynamic mode. The OFELIA facility 713 uses the OpenFlow protocol (and related tools) to support network 714 virtualization and control of the network environment through secure 715 and standardised interfaces. OFELIA consists of two layers. The 716 physical layer is comprised of the computing resources (servers, 717 processors) and network resources (routers, switches, links, wireless 718 devices and optical components). Resources are managed by the OFELIA 719 Control Framework (OCF). Furthermore, the control framework layer 720 contains components which manage and monitor the applications and 721 devices in the physical layer. Aggregate Managers and Resource 722 Managers are components of this layer, which can be seen as the 723 combination of three components: Expedient is the GUI and allows the 724 connection and federation with different Aggregate Managers via its 725 plugins; Aggregate Managers (AMs) enable experimenters to create both 726 compute and network resources via the VT AM and OF AM respectively; 727 Resource Managers directly interact with the physical layer, 728 provisioning compute resources (OFELIA Xen Agent) or flow rules to 729 establish the topology (FlowVisor). 731 The FIBRE project [FIBRE] federates SDN testbeds distributed across 732 Europe and Brazil. The FIBRE-EU system builds on top of the OFELIA 733 OCF and incorporates several wireless nodes based on commercial Wi-Fi 734 cards and Linux open source drivers. Unlike OFELIA, the FIBRE 735 infrastructure is managed by different types of control and 736 monitoring frameworks (CMFs). FIBRE deployed two top-domain 737 authorities, one in Brazil and one in Europe, to manage and own 738 resources in the respective continents. These inter-connected 739 authorities interoperate to allow the federation of BR and EU 740 testbeds. 742 In Japan, GridARS [GRIDARS] provides a reference implementation of 743 the Open Grid Forum (OGF) Network Services Interfaces Connection 744 Service (NSI-CS) protocol standard. GridARS can coordinate multiple 745 resources (services), such as a network connection, virtual machines 746 and storage spaces, via the NSICS protocol. It provides 747 experimenters a virtual infrastructure, which spans several cloud 748 resources, realised by multiple management domains including 749 commercial solutions. GridARS consists of three main components. 751 6. IANA Considerations 753 No IANA considerations are applicable. 755 7. Security Considerations 757 This document proposes a new architecture for resource and VNF 758 orchestration for the design of which security features are of utmost 759 importance to proceed to operational deployments. Frameworks for 760 Security in SDN are applicable to this document and are discussed in 761 literature, for example, in [SDNSecurity], [SDNSecServ] and 762 [SDNSecOF]. Security considerations regarding specific protocol 763 interfaces are TBD. 765 8. Acknowledgements 767 This work has been partially supported and funded by the European 768 Commission through the FP7 ICT FELIX project (Federated Testbeds for 769 Large Scale Infrastructure Experiments, grant agreement no. 608638) 770 and the National Institute of Information and Communications 771 Technology (NICT) in Japan. The views expressed here are those of 772 the author only. The European Commission and NICT are not liable for 773 any use that may be made of the information in this document. 775 9. Contributors 777 Authors would like to acknowledge (in alphabetical order) the 778 following contributors who have provided text, pointers, and ideas 779 for this document: 781 o B. Belter (PSNC, Poland) 783 o C. Bermudo (i2CAT, Spain) 785 o T. Kudoh (Univ. Tokyo/AIST, Japan) 787 o J. Tanaka (KDDI, Japan) 789 o B. Vermeulen(iMinds, Belgium) 791 10. Informative References 793 [cloudstack] 794 "Apache CloudStack documentation. Cloud Infrastructure 795 Concepts", Available online at 796 http://cloudstack.apache.org/docs/en- 797 US/Apache_CloudStack/4.1.0/html/Admin_Guide/ 798 cloud-infrastructure-concepts.html. 800 [FELIX] "FELIX Project website", http://www.ict-felix.eu. 802 [FELIX-D2.1] 803 R. Krzywania, W. Bogacki, B. Belter, K. Pentikousis, T. 804 Rothe, G. Carrozzo, N. Ciulli, C. Bermudo, T. Kudoh, A. 805 Takefusa, J. Tanaka and B. Puype, , "FELIX Deliverable 806 D2.1 - Experiment Use Cases and Requirements", Available 807 online at http://www.ict-felix.eu., September 2013. 809 [FELIX-D2.2] 810 R. Krzywania, W. Bogacki, B. Belter, K. Pentikousis, T. 811 Rothe, M. Broadbent, G. Carrozzo, N. Ciulli, R. Monno, C. 812 Bermudo, A. Vico, C. Fernandez, T. Kudoh, A. Takefusa, J. 813 Tanaka and B. Puype, , "FELIX Deliverable D2.2 - General 814 Architecture and Functional Blocks", Available online at 815 http://www.ict-felix.eu., February 2014. 817 [FIBRE] T. Salmito, L. Ciuffo, I. Machado, M. Salvador, et al, , 818 "FIBRE - An International Testbed for Future Internet 819 Experimentation", 32th Simposio Brasileiro de Redes de 820 Computadores e Sistemas Distribuidos (SBRC'14) , 2014. 822 [GRIDARS] [15] A. Takefusa, H. Nakada, T. Kudoh, Y. Tanaka and S. 823 Sekiguchi, , "GridARS: An Advance Reservation-based Grid 824 Co-allocation Framework for Distributed Computing and 825 Network Resources", Lecture Notes, Computer Science of the 826 Job Scheduling Strategies for Parallel Processing (JSSPP) 827 vol.4942, pp. 152-168, April 2008. 829 [middlebox] 830 A. Greenhalgh, F. Huici, M. Hoerdt, P. Papadimitriou, M. 831 Handley, and L. Mathy, , "Flow Processing and the Rise of 832 Commodity Network Hardware", ACM SIGCOMM Computer 833 Communication Review Volume 39 issue 2, April 2009. 835 [OFELIA] M. Sune, L. Bergesio, H. Woesner, T. Rothe, A. Kopsel, et 836 al., , "Design and implementation of the OFELIA FP7 837 facility: The European OpenFlow testbed", The 838 International Journal of Computer and Telecommunications 839 Networking , December 2013. 841 [openmano] 842 "OpenMANO", Available online at 843 https://github.com/nfvlabs/openmano/wiki. 845 [openstack] 846 "Scaling Openstack", Available online at 847 http://docs.openstack.org/openstack-ops/content/ 848 scaling.html. 850 [os-heat] "OpenStack Orchestration - Heat", Available online at 851 https://wiki.openstack.org/wiki/Heat. 853 [SDNSecOF] 854 Kloti, R., Kotronis, V., and P. Smith, , "OpenFlow: A 855 Security Analysis", 21st IEEE International Conference on 856 Network Protocols (ICNP) pp. 1-6, October 2013. 858 [SDNSecServ] 859 Scott-Hayward, S., O'Callaghan, G., and S. Sezer, , "SDN 860 Security: A Survey", IEEE SDN for Future Networks and 861 Services (SDN4FNS) pp. 1-7, 2013. 863 [SDNSecurity] 864 Kreutz, D., Ramos, F., and P. Verissimo, , "Towards Secure 865 and Dependable Software-Defined Networks", Proceedings of 866 the second ACM SIGCOMM workshop on Hot Topics in Software 867 Defined Networking pp. 55-60, 2013. 869 [SFA] L. Peterson, R. Ricci, A. Falk and J. Chase, , "Slice- 870 based Federation Architecture (SFA) v2.0", , July 2010. 872 Authors' Addresses 874 Gino Carrozzo (Ed.) (editor) 875 Nextworks 876 via Livornese 1027 877 Pisa 56122 878 Italy 880 Email: g.carrozzo@nextworks.it 882 Kostas Pentikousis (Ed.) (editor) 883 EICT 884 Torgauer Strasse 12-15 885 Berlin 10829 886 Germany 888 Email: k.pentikousis@eict.de