idnits 2.17.1 draft-unify-nfvrg-challenges-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([I-D.caszpe-nfvrg-orchestration-challenges]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 13, 2016) is 3026 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: '---SP1---' is mentioned on line 315, but not defined == Missing Reference: '--------SP2-------' is mentioned on line 315, but not defined == Missing Reference: '---SP3----' is mentioned on line 460, but not defined == Missing Reference: '-------------------SP0-------------------' is mentioned on line 316, but not defined == Missing Reference: '----SP1---' is mentioned on line 460, but not defined == Missing Reference: '---------SP2--------' is mentioned on line 460, but not defined == Missing Reference: '---------------------SP0--------------------' is mentioned on line 363, but not defined == Outdated reference: A later version (-06) exists of draft-unify-nfvrg-devops-03 Summary: 1 error (**), 0 flaws (~~), 9 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFVRG R. Szabo 3 Internet-Draft A. Csaszar 4 Intended status: Informational Ericsson 5 Expires: July 16, 2016 K. Pentikousis 6 EICT 7 M. Kind 8 Deutsche Telekom AG 9 D. Daino 10 Telecom Italia 11 Z. Qiang 12 Ericsson 13 H. Woesner 14 BISDN 15 January 13, 2016 17 Unifying Carrier and Cloud Networks: Problem Statement and Challenges 18 draft-unify-nfvrg-challenges-03 20 Abstract 22 The introduction of network and service functionality virtualization 23 in carrier-grade networks promises improved operations in terms of 24 flexibility, efficiency, and manageability. In current practice, 25 virtualization is controlled through orchestrator entities that 26 expose programmable interfaces according to the underlying resource 27 types. Typically this means the adoption of, on the one hand, 28 established data center compute/storage and, on the other, network 29 control APIs which were originally developed in isolation. Arguably, 30 the possibility for innovation highly depends on the capabilities and 31 openness of the aforementioned interfaces. This document introduces 32 in simple terms the problems arising when one follows this approach 33 and motivates the need for a high level of programmability beyond 34 policy and service descriptions. This document also summarizes the 35 challenges related to orchestration programming in this unified cloud 36 and carrier network production environment. A subsequent problem is 37 the resource orchestration. This is handled separately in 38 [I-D.caszpe-nfvrg-orchestration-challenges] and will be merged in the 39 next iteration of this document. 41 Status of This Memo 43 This Internet-Draft is submitted in full conformance with the 44 provisions of BCP 78 and BCP 79. 46 Internet-Drafts are working documents of the Internet Engineering 47 Task Force (IETF). Note that other groups may also distribute 48 working documents as Internet-Drafts. The list of current Internet- 49 Drafts is at http://datatracker.ietf.org/drafts/current/. 51 Internet-Drafts are draft documents valid for a maximum of six months 52 and may be updated, replaced, or obsoleted by other documents at any 53 time. It is inappropriate to use Internet-Drafts as reference 54 material or to cite them other than as "work in progress." 56 This Internet-Draft will expire on July 16, 2016. 58 Copyright Notice 60 Copyright (c) 2016 IETF Trust and the persons identified as the 61 document authors. All rights reserved. 63 This document is subject to BCP 78 and the IETF Trust's Legal 64 Provisions Relating to IETF Documents 65 (http://trustee.ietf.org/license-info) in effect on the date of 66 publication of this document. Please review these documents 67 carefully, as they describe your rights and restrictions with respect 68 to this document. Code Components extracted from this document must 69 include Simplified BSD License text as described in Section 4.e of 70 the Trust Legal Provisions and are provided without warranty as 71 described in the Simplified BSD License. 73 Table of Contents 75 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 76 2. Terms and Definitions . . . . . . . . . . . . . . . . . . . . 3 77 3. Motivations . . . . . . . . . . . . . . . . . . . . . . . . . 4 78 4. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 12 79 5. Challenges . . . . . . . . . . . . . . . . . . . . . . . . . 13 80 5.1. Orchestration . . . . . . . . . . . . . . . . . . . . . . . 13 81 5.2. Resource description . . . . . . . . . . . . . . . . . . . 13 82 5.3. Dependencies (de-composition) . . . . . . . . . . . . . . . 14 83 5.4. Elastic VNF . . . . . . . . . . . . . . . . . . . . . . . . 14 84 5.5. Measurement and analytics . . . . . . . . . . . . . . . . . 15 85 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 86 7. Security Considerations . . . . . . . . . . . . . . . . . . . 16 87 8. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 16 88 9. Informative References . . . . . . . . . . . . . . . . . . . 16 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 91 1. Introduction 93 To a large degree there is agreement in the network research, 94 practitioner, and standardization communities that rigid network 95 control limits the flexibility and manageability of speedy service 96 creation, as discussed in [NSC] and the references therein. For 97 instance, it is not unusual that today an average service creation 98 time cycle exceeds 90 hours, whereas given the recent advances in 99 virtualization and cloudification one would be interested in service 100 creation times in the order of minutes [EU-5GPPP-Contract] if not 101 seconds. 103 Flexible service definition and creation start by formalizing the 104 service into the concept of network function forwarding graphs, such 105 as the ETSI VNF Forwarding Graph [ETSI-NFV-Arch] or the ongoing work 106 in IETF [I-D.ietf-sfc-problem-statement]. These graphs represent the 107 way in which service end-points (e.g., customer access) are 108 interconnected with a set of selected network functionalities such as 109 firewalls, load balancers, and so on, to deliver a network service. 110 Service graph representations form the input for the management and 111 orchestration to instantiate and configure the requested service. 112 For example, ETSI defined a Management and Orchestration (MANO) 113 framework in [ETSI-NFV-MANO]. We note that throughout such a 114 management and orchestration framework different abstractions may 115 appear for separation of concerns, roles or functionality, or for 116 information hiding. 118 Compute virtualization is central to the concept of Network Function 119 Virtualization (NFV). However, carrier-grade services demand that 120 all components of the data path, such as Network Functions (NFs), 121 virtual NFs (VNFs) and virtual links, meet key performance 122 requirements. In this context, the inclusion of Data Center (DC) 123 platforms, such as OpenStack [OpenStack], into the SDN infrastructure 124 is far from trivial. 126 In this document we examine the problems arising as one combines 127 these two formerly isolated environments in an effort to create a 128 unified production environment and discuss the associated emerging 129 challenges. Our goal is the definition of a production environment 130 that allows multi-vendor and multi-domain operation based on open and 131 interoperable implementations of the key entities described in the 132 remainder of this document. 134 2. Terms and Definitions 136 We use the term compute and "compute and storage" interchangeably 137 throughout the document. Moreover, we use the following definitions, 138 as established in [ETSI-NFV-Arch]: 140 NFV: Network Function Virtualization - The principle of separating 141 network functions from the hardware they run on by using virtual 142 hardware abstraction. 144 NFVI PoP: NFV Infrastructure Point of Presence - Any combination of 145 virtualized compute, storage and network resources. 147 NFVI: NFV Infrastructure - Collection of NFVI PoPs under one 148 orchestrator. 150 VNF: Virtualized Network Function - a software-based network 151 function. 153 VNF FG: Virtualized Network Function Forwarding Graph - an ordered 154 list of VNFs creating a service chain. 156 MANO: Management and Orchestration - In the ETSI NFV framework 157 [ETSI-NFV-MANO], this is the global entity responsible for 158 management and orchestration of NFV lifecycle. 160 Further, we make use of the following terms: 162 NF: a network function, either software-based (VNF) or appliance- 163 based. 165 SW: a (routing/switching) network element with a programmable 166 control plane interface. 168 DC: a data center network element which in addition to a 169 programmable control plane interface offers a DC control interface 171 LSI: Logical Switch Instance - a software switch instance. 173 CN: an element equipped with compute and/or storage resources. 175 UN: Universal Node - an innovative element that integrates and 176 manages in a unified platform both compute and networking 177 components. 179 3. Motivations 181 Figure 1 illustrates a simple service graph comprising three network 182 functions (NFs). For the sake of simplicity, we will assume only two 183 types of infrastructure resources, namely SWs and DCs as per the 184 terminology introduced above, and ignore appliance-based NFs for the 185 time being. The goal is to implement the given service based on the 186 available infrastructure resources. 188 fr2 +---+ fr3 189 +->o-|NF2|-o-+ 190 | 4 +---+ 5 | 191 +---+ | V +---+ 192 1-->o-|NF1|-o----------->o-|NF3|-o-->8 193 2 +---+ 3 fr1 6 +---+ 7 195 Figure 1: Service graph 197 The service graph definition contains NF types (NF1, NF2, NF3) along 198 with the 200 o corresponding ports (NF1:{2,3}; NF2:{4,5}; NF3:{6,7}) 202 o service access points {1,8} corresponding to infrastructure 203 resources, 205 o definition of forwarding behavior (fr1, fr2, fr3) 207 The forwarding behavior contains classifications for matching of 208 traffic flows and corresponding outbound forwarding actions. 210 Assume now that we would like to use the infrastructure (topology, 211 network and software resources) depicted in Figure 2 and Figure 3 to 212 implement the aforementioned service graph. That is, we have three 213 SWs and two Points of Presence (PoPs) with DC software resources at 214 our disposal. 216 +---+ 217 +--|SW3|--+ 218 | +---+ | 219 +---+ | | +---+ 220 1 |PoP| +---+ +---+ |PoP| 8 221 o--|DC1|----|SW2|------|SW4 |---|DC2|--o 222 +---+ +---+ +---+ +---+ 224 [---SP1---][--------SP2-------][---SP3----] 226 Figure 2: Infrastructure resources 227 +----------+ 228 | +----+ |PoP DC (== NFVI PoP) 229 | | CN | | 230 | +----+ | 231 | | | | 232 | +----+ | 233 o-+--| SW |--+-o 234 | +----+ | 235 +----------+ 237 Figure 3: A virtualized Point of Presence (PoP) with software 238 resources (Compute Node - CN) 240 +----------+ 241 | +----+ | UN 242 | | CN | | 243 o-+--+----+--+-o 244 | | SW | | 245 | +----+ | 246 +----------+ 248 Figure 4: Universal Node - an innovative element that integrates on 249 the same platform both compute and networking components 251 In the simplest case, all resources would be part of the same service 252 provider (SP) domain. We need to ensure that each entity in Figure 2 253 can be procured from a different vendor and therefore 254 interoperability is key for multi-vendor NFVI deployment. Without 255 such interoperability different technologies for data center and 256 network operation result in distinct technology domains within a 257 single carrier. Multi-technology barriers start to emerge hindering 258 the full programmability of the NFVI and limiting the potential for 259 rapid service deployment. 261 We are also interested in a multi-operation environment, where the 262 roles and responsibilities are distributed according to some 263 organizational structure within the organization. Finally, we are 264 interested in multi-provider environment, where different 265 infrastructure resources are available from different service 266 providers (SPs). Figure 2 indicates a multi-provider environment in 267 the lower part of the figure as an example. We expect that this type 268 of deployments will become more common in the future as they are well 269 suited with the elasticity and flexibility requirements [NSC]. 271 Figure 2 also shows the service access points corresponding to the 272 overarching domain view, i.e., {1,8}. In order to deploy the service 273 graph of Figure 1 on the infrastructure resources of Figure 2, we 274 will need an appropriate mapping which can be implemented in 275 practice. 277 Figure 3 shows the structure of a PoP DC that presents compute and 278 network resources while Figure 4 shows the structure of the Universal 279 Node (UN), an innovative element that integrates on the same platform 280 both compute and networking components and that could be used in the 281 infrastructure as an alternative to elements depicted in Figure 2 for 282 what concerns network and/or compute resources. 284 In Figure 5 we illustrate a resource orchestrator (RO) as a 285 functional entity whose task is to map the service graph to the 286 infrastructure resources under some service constraints and taking 287 into account the NF resource descriptions. 289 fr2 +---+ fr3 290 +->o-|NF2|-o-+ 291 | 4 +---+ 5 | 292 +---+ | V +---+ 293 1-->o-|NF1|-o----------->o-|NF3|-o-->8 294 2 +---+ 3 fr1 6 +---+ 7 296 || 297 || 298 +--------+ \/ SP0 299 | NF | +---------------------+ 300 |Resource|==>|Resource Orchestrator|==> MAPPING 301 | Descr. | | (RO) | 302 +--------+ +---------------------+ 303 /\ 304 || 305 || 307 +---+ 308 +--|SW3|--+ 309 | +---+ | 310 +---+ | | +---+ 311 1 |PoP| +---+ +---+ |PoP| 8 312 o--|DC1|-----|SW2|-----|SW4|----|DC2|--o 313 +---+ +---+ +---+ +---+ 315 [---SP1---][--------SP2-------][---SP3----] 316 [-------------------SP0-------------------] 318 Figure 5: Resource Orchestrator: information base, inputs and output 319 NF resource descriptions are assumed to contain information necessary 320 to map NF types to a choice of instantiable VNF flavor or a selection 321 of an already deployed NF appliance and networking demands for 322 different operational policies. For example, if energy efficiency is 323 to be considered during the decision process then information related 324 to energy consumption of different NF flavors under different 325 conditions (e.g., network load) should be included in the resource 326 description. 328 Note that we also introduce a new service provider (SP0) which 329 effectively operates on top of the virtualized infrastructure offered 330 by SP1, SP2 and SP3. 332 In order for the RO to execute the resource mapping (which in general 333 is a hard problem) it needs to operate on the combined control plane 334 illustrated in Figure 6. In this figure we mark clearly that the 335 interfaces to the compute (DC) control plane and the SDN (SW) control 336 plane are distinct and implemented through different interfaces/APIs. 337 For example, Ic1 could be the Apache CloudStack API, while Ic2 could 338 be a control plane protocol such as ForCES or OpenFlow [RFC7426]. In 339 this case, the orchestrator at SP0 (top part of the figure) needs to 340 maintain a tight coordination across this range of interfaces. 342 +---------+ 343 |Orchestr.| 344 | SP0 | 345 _____+---------+_____ 346 / | \ 347 / V Ic2 \ 348 | +---------+ | 349 Ic1 V |SDN Ctrl | V Ic3 350 +---------+ | SP2 | +---------+ 351 |Comp Ctrl| +---------+ |Comp Ctrl| 352 | SP1 | / | \ | SP3 | 353 +---------+ +--- V ----+ +---------+ 354 | | +----+ | | 355 | | |SW3 | | | 356 V | +----+ | V 357 +----+ V / \ V +----+ 358 1 |PoP | +----+ +----+ |PoP | 8 359 o--|DC1 |----|SW2 |------|SW4 |----|DC2 |--o 360 +----+ +----+ +----+ +----+ 362 [----SP1---][---------SP2--------][---SP3----] 363 [---------------------SP0--------------------] 365 Figure 6: The RO Control Plane view. Control plane interfaces are 366 indicated with (line) arrows. Data plane connections are indicated 367 with simple lines. 369 In the real-world, however, orchestration operations do not stop, for 370 example, at the DC1 level as depicted in Figure 6. If we (so-to- 371 speak) "zoom into" DC1 we will see a similar pattern and the need to 372 coordinate SW and DC resources within DC1 as illustrated in Figure 7. 373 As depicted, this edge PoP includes compute nodes (CNs) and SWs which 374 in most of the cases will also contain an internal topology. 376 In Figure 7, IcA is an interface similar to Ic2 in Figure 6, while 377 IcB could be, for example, OpenStack Nova or similar. The Northbound 378 Interface (NBI) to the Compute Controller can use Ic1 or Ic3 as shown 379 in Figure 6. 381 NBI 382 | 383 +---------+ 384 |Comp Ctrl| 385 +---------+ 386 +----+ | 387 IcA V | IcB:to CNs 388 +---------+ V 389 |SDN Ctrl | | | ext port 390 +---------+ +---+ +---+ 391 to|SW |SW | |SW | 392 +-> ,+--++.._ _+-+-+ 393 V ,-" _|,,`.""-..+ 394 _,,,--"" | `. |""-.._ 395 +---+ +--++ `+-+-+ ""+---+ 396 |SW | |SW | |SW | |SW | 397 +---+ ,'+---+ ,'+---+ ,'+---+ 398 | | ,-" | | ,-" | | ,-" | | 399 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ 400 |CN| |CN| |CN| |CN| |CN| |CN| |CN| |CN| 401 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ 403 Figure 7: PoP DC Network with Compute Nodes (CN) 405 In turn, each single Compute Node (CN) may also have internal 406 switching resources (see Figure 8). In a carrier environment, in 407 order to meet data path requirements, allocation of compute node 408 internal distributed resources (blades, CPU cores, etc.) may become 409 equivalently important. 411 +-+ +-+ +-+ +-+ 412 |V| |V| |V| |V| 413 |N| |N| |N| |N| 414 |F| |F| |F| |F| 415 +-+ +-+ +-+ +-+ 416 | / / | 417 +---+ +---+ +---+ 418 |LSI| |LSI| |LSI| 419 +---+ +---+ +---+ 420 | / | 421 +---+ +---+ 422 |NIC| |NIC| 423 +---+ +---+ 424 | | 426 Figure 8: Compute Node with internal switching resource 428 Based on the recursion principles shown above and the complexity 429 implied by separate interfaces for compute and network resources, one 430 could imagine a recursive programmatic interface for joint compute, 431 storage and network provisioning as depicted in Figure 9. 433 +---------+ 434 |Service | 435 |Orchestr.| 436 +---------+ 437 | 438 | 439 V U 440 +-------------------+ 441 | Unified Recurrent | 442 | Control (URC) | 443 +-------------------+ 444 / | \ 445 / V U \ 446 | +---------+ | 447 U V | URC | V U 448 +---------+ | | +---------+ 449 | URC | +---------+ | URC | 450 | | / | \ | | 451 +---------+ +--- V ----+ +---------+ 452 | | +----+ | | 453 | | |SW3 | | | 454 V | +----+ | V 455 +----+ V / \ V +----+ 456 1 |PoP | +----+ +----+ |PoP | 8 457 o--|DC1 |----|SW2 |------|SW4 |----|DC2 |--o 458 +----+ +----+ +----+ +----+ 460 [----SP1---][---------SP2--------][---SP3----] 462 Figure 9: The RO Control Plane view considering a recursive 463 programmatic interface for joint compute, storage and network 464 provisioning 466 In Figure 9, Ic1, Ic2 and Ic3 of Figure 6 have been substituted by 467 the recursive programmatic interface U to use for both compute and 468 network resources and we find also the Unified Recurrent Control 469 (URC), an element that performs both compute and network control and 470 that can be used in a hierarchy structure. 472 Considering the use of the recursive programmatic interface U and the 473 Unified Recurrent Control, the PoP DC Network structure with Compute 474 Nodes view changes as reported in Figure 10. 476 NBI 477 | 478 +---------+ 479 | URC | 480 +---------+ 481 +----+ | 482 U V | U:to CNs 483 +---------+ V 484 | URC | | | ext port 485 +---------+ +---+ +---+ 486 to|SW |SW | |SW | 487 +-> ,+--++.._ _+-+-+ 488 V ,-" _|,,`.""-..+ 489 _,,,--"" | `. |""-.._ 490 +---+ +--++ `+-+-+ ""+---+ 491 |SW | |SW | |SW | |SW | 492 +---+ ,'+---+ ,'+---+ ,'+---+ 493 | | ,-" | | ,-" | | ,-" | | 494 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ 495 |CN| |CN| |CN| |CN| |CN| |CN| |CN| |CN| 496 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ 498 Figure 10: PoP DC Network with Compute Nodes (CN) considering the U 499 interface and the URC element 501 4. Problem Statement 503 The motivational examples of Section 3 illustrate that almost always 504 compute virtualization and network virtualization are tightly 505 connected. In particular Figure, 3 shows that in a PoP DC there are 506 not only compute resources (CNs) but also network resources (SWs), 507 and so it illustrates that compute virtualization implicitly involves 508 network virtualization unless we consider the unlikely scenario where 509 dedicated network elements are used to interconnect the different 510 virtual network functions implemented on the compute nodes (e.g.: to 511 implement Flexible Service Chaining). On the other hand, considering 512 a network scenario made not only of just pure SDN network elements 513 (SWs) but also of compute resources (CNs) or SDN network nodes that 514 are equipped also with compute resources (UNs), it is very likely 515 that virtualized network resources, if offered to clients, imply 516 virtualization of compute resources, unless we consider the unlikely 517 scenario where dedicated compute resources are available for every 518 virtualized network. 520 Furthermore, virtualization often leads to scenarios of recursions 521 with clients redefining and reselling resources and services at 522 different levels. 524 We argue that given the multi-level virtualization of compute, 525 storage and network domains, automation of the corresponding resource 526 provisioning could be more easily implemented by a recursive 527 programmatic interface. Existing separated compute and network 528 programming interfaces cannot easily provide such recursions and 529 cannot always satisfy key requirement for multi-vendor, multi- 530 technology and multi-provider interoperability environments. 531 Therefore we foresee the necessity of a recursive programmatic 532 interface for joint compute, storage and network provisioning. 534 5. Challenges 536 We summarize in this section the key questions and challenges, which 537 we hope will initiate further discussions in the NFVRG community. 539 5.1. Orchestration 541 Firstly, as motivated in Section 3, orchestrating networking 542 resources appears to have a recursive nature at different levels of 543 the hierarchy. Would a programmatic interface at the combined 544 compute and network abstraction better support this recursive and 545 constraint-based resource allocation? 547 Secondly, can such a joint compute, storage and network programmatic 548 interface allow an automated resource orchestration similar to the 549 recursive SDN architecture [ONF-SDN-ARCH]? 551 5.2. Resource description 553 Prerequisite for joint placement decisions of compute, storage and 554 network is the adequate description of available resources. This 555 means that the interfaces (IcA, IcB etc. in Figure 6 and Figure 7) 556 are of bidirectional nature, exposing resources as well as reserving. 557 There have been manifold attempts to create frameworks for resource 558 description, most prominently RDF of W3C, NDL, the GENI RPC and its 559 concept of Aggregate Managers, ONF's TTP and many more. 561 Quite naturally, all attempts to standardize "arbitrary" resource 562 descriptions lead to creating ontologies, complex graphs describing 563 relations of terms to each other. 565 Practical descriptions of compute resources are currently focusing on 566 number of logical CPU cores, available RAM and storage, allowing, 567 e.g., the OpenStack Nova scheduler to meet placement decisions. In 568 heterogeneous network and compute environments, hardware may have 569 different acceleration capabilities (e.g., AES-NI or hardware random 570 number generators), so the notion of logical compute cores is not 571 expressive enough. In addition, the network interfaces (and link 572 load) provide important information on how fast a certain VNF can be 573 executed in one node. 575 This may lead to a description of resources as VNF-FGs themselves. 576 Networking resource (SW) may expose the capability to forward and 577 process frames in, e.g., OpenFlow TableFeatures reply. Compute nodes 578 in the VNF-FG would expose lists of capabilities like the presence of 579 AES hardware acceleration, Intel DPDK support, or complex functions 580 like a running web server. An essential part of the compute node's 581 capability would be the ability to run a certain VNF of type X within 582 a certain QoS spec. As the QoS is service specific, it can only be 583 exposed by a control function within the instantiated VNF-FG. 585 5.3. Dependencies (de-composition) 587 Salt [SALT], Puppet [PUPPET], Chef [CHEF] and Ansible [ANSIBLE] are 588 tools to manage large scale installations of virtual machines in DC 589 environments. Essentially, the decomposition of a complex function 590 into its dependencies is encoded in "recipes" (Chef). 592 OASIS TOSCA [TOSCA] specification aims at describing application 593 layer services to automate interoperable deployment in alternative 594 cloud environments. The TOSCA specification "provides a language to 595 describe service components and their relationships using a service 596 topology". 598 Is there a dependency (decomposition) abstraction suitable to drive 599 resource orchestration between application layer descriptions (like 600 TOSCA) and cloud specific installations (like Chef recipes)? 602 5.4. Elastic VNF 604 In many use cases, a VNF may not be designed for scaling up/down, as 605 scaling up/down may require a restart of the VNF which the state data 606 may be lost. Normally a VNF may be capable for scaling in/out only. 607 Such VNF is designed running on top of a small VM and grouped as a 608 pool of one VNF function. VNF scaling may crossing multiple NFVI 609 PoPs (or data center)s in order to avoid limitation of the NVFI 610 capability. At cross DC scaling, the result is that the new VNF 611 instance may be placed at a remote cloud location. At VNF scaling, 612 it is a must requirement to provide the same level of Service Level 613 Agreement (SLA) including performance, reliability and security. 615 In general, a VNF is part of a VNF Forwarding Graph (VNF FG), meaning 616 the data traffic may traverse multiple stateful and stateless VNF 617 functions in sequence. When some VNF instances of a given service 618 function chain are placed / scaled out in a distant cloud execution, 619 the service traffic may have to traverse multiple VNF instances which 620 are located in multiple physical locations. In the worst case, the 621 data traffic may ping-pong between multiple physical locations. 622 Therefore it is important to take the whole service function chain's 623 performance into consideration when placing and scaling one of its 624 VNF instance. Network and cloud resources need mutual 625 considerations, see [I-D.zu-nfvrg-elasticity-vnf]. 627 5.5. Measurement and analytics 629 Programmable, dynamic, and elastic VNF deployment requires that the 630 Resource Orchestrator (RO) entities obtain timely information about 631 the actual operational conditions between different locations where 632 VNFs can be placed. Scaling VNFs in/out/up/down, VNF execution 633 migration and VNF mobility, as well as right-sizing the VNFI resource 634 allocations is a research area that is expected to grow in the coming 635 years as mechanisms, heuristics, and measurement and analytics 636 frameworks are developed. 638 For example, Veitch et al. [IAF] point out that NFV deployment will 639 "present network operators with significant implementation 640 challenges". They look into the problems arising from the lack of 641 proper tools for testing and diagnostics and explore the use of 642 embedded instrumentation. They find that in certain scenarios fine- 643 tuning resource allocation based on instrumentation can lead to at 644 least 50% reduction in compute provisioning. In this context, three 645 categories emerge where more research is needed. 647 First, in the compute domain, performance analysis will need to 648 evolve significantly from the current "safety factor" mentality which 649 has served well carriers in the dedicated, hardware-based appliances 650 era. In the emerging softwarized deployments, VNFI will require new 651 tools for planning, testing, and reliability assurance. Meirosu et 652 al. [I-D.unify-nfvrg-devops] describe in detail the challenges in 653 this area with respect to verification, testing, troubleshooting and 654 observability. 656 Second, in the network domain, performance measurement and analysis 657 will play a key role in determining the scope and range of VNF 658 distribution across the resources available. For example, IETF has 659 worked on the standardization of IP performance metrics for years. 660 The Two-Way Active Measurement Protocol (TWAMP) could be employed, 661 for instance, to capture the actual operational state of the network 662 prior to making RO decisions. TWAMP management, however, still lacks 663 a standardized and programmable management and configuration data 664 model [I-D.cmzrjp-ippm-twamp-yang]. We expect that as VNFI 665 programmability gathers interest from network carriers several IETF 666 protocols will be revisited in order to bring them up to date with 667 respect to the current operational requirements. To this end, NFVRG 668 can play an active role in identifying future IETF standardization 669 directions. 671 Third, non-technical considerations which relate to business aspects 672 or priorities need to be modeled and codified so that ROs can take 673 intelligent decisions. Meirosu et al. [I-D.unify-nfvrg-devops] 674 identify two aspects of this problem, namely a) how high-level 675 network goals are translated into low-level configuration commands; 676 and b) monitoring functions that go beyond measuring simple metrics 677 such as delay or packet loss. Energy efficiency and cost, for 678 example, can steer NFV placement. In NFVI deployments operational 679 practices such as follow-the-sun will be considered as earlier 680 research in the data center context implies. 682 6. IANA Considerations 684 This memo includes no request to IANA. 686 7. Security Considerations 688 TBD 690 8. Acknowledgement 692 The authors would like to thank the UNIFY team for inspiring 693 discussions and in particular Fritz-Joachim Westphal and Catalin 694 Meirosu for their comments and suggestions on how to refine this 695 draft. 697 This work is supported by FP7 UNIFY, a research project partially 698 funded by the European Community under the Seventh Framework Program 699 (grant agreement no. 619609). The views expressed here are those of 700 the authors only. The European Commission is not liable for any use 701 that may be made of the information in this document. 703 9. Informative References 705 [ANSIBLE] Ansible Inc., "Ansible Documentation", 2015, 706 . 708 [CHEF] Chef Software Inc., "An Overview of Chef", 2015, 709 . 711 [ETSI-NFV-Arch] 712 ETSI, "Architectural Framework v1.1.1", Oct 2013, 713 . 716 [ETSI-NFV-MANO] 717 ETSI, "Network Function Virtualization (NFV) Management 718 and Orchestration V0.6.1 (draft)", Jul. 2014, 719 . 722 [EU-5GPPP-Contract] 723 5G-PPP Association, "Contractual Arrangement: Setting up a 724 Public- Private Partnership in the Area of Advance 5G 725 Network Infrastructure for the Future Internet between the 726 European Union and the 5G Infrastructure Association", Dec 727 2013, . 729 [I-D.caszpe-nfvrg-orchestration-challenges] 730 Carrozzo, G., Szabo, R., and K. Pentikousis, "Network 731 Function Virtualization: Resource Orchestration 732 Challenges", draft-caszpe-nfvrg-orchestration- 733 challenges-00 (work in progress), November 2015. 735 [I-D.cmzrjp-ippm-twamp-yang] 736 Civil, R., Morton, A., Zheng, L., Rahman, R., 737 Jethanandani, M., and K. Pentikousis, "Two-Way Active 738 Measurement Protocol (TWAMP) Data Model", draft-cmzrjp- 739 ippm-twamp-yang-02 (work in progress), October 2015. 741 [I-D.ietf-sfc-problem-statement] 742 Quinn, P. and T. Nadeau, "Service Function Chaining 743 Problem Statement", draft-ietf-sfc-problem-statement-13 744 (work in progress), February 2015. 746 [I-D.unify-nfvrg-devops] 747 Meirosu, C., Manzalini, A., Steinert, R., Marchetto, G., 748 Papafili, I., Pentikousis, K., and S. Wright, "DevOps for 749 Software-Defined Telecom Infrastructures", draft-unify- 750 nfvrg-devops-03 (work in progress), October 2015. 752 [I-D.zu-nfvrg-elasticity-vnf] 753 Qiang, Z. and R. Szabo, "Elasticity VNF", draft-zu-nfvrg- 754 elasticity-vnf-01 (work in progress), March 2015. 756 [IAF] Veitch, P., McGrath, M. J., and Bayon, V., "An 757 Instrumentation and Analytics Framework for Optimal and 758 Robust NFV Deployment", Communications Magazine, vol. 53, 759 no. 2 IEEE, February 2015. 761 [NSC] John, W., Pentikousis, K., et al., "Research directions in 762 network service chaining", Proc. SDN for Future Networks 763 and Services (SDN4FNS), Trento, Italy IEEE, November 2013. 765 [ONF-SDN-ARCH] 766 ONF, "SDN architecture", Jun. 2014, 767 . 771 [OpenStack] 772 The OpenStack project, "Openstack cloud software", 2014, 773 . 775 [PUPPET] Puppet Labs., "Puppet 3.7 Reference Manual", 2015, 776 . 778 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S., 779 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software- 780 Defined Networking (SDN): Layers and Architecture 781 Terminology", RFC 7426, DOI 10.17487/RFC7426, January 782 2015, . 784 [SALT] SaltStack, "Salt (Documentation)", 2015, 785 . 787 [TOSCA] OASIS Standard, "Topology and Orchestration Specification 788 for Cloud Applications Version 1.0", November 2013, 789 . 792 Authors' Addresses 794 Robert Szabo 795 Ericsson Research, Hungary 796 Irinyi Jozsef u. 4-20 797 Budapest 1117 798 Hungary 800 Email: robert.szabo@ericsson.com 801 URI: http://www.ericsson.com/ 803 Andras Csaszar 804 Ericsson Research, Hungary 805 Irinyi Jozsef u. 4-20 806 Budapest 1117 807 Hungary 809 Email: andras.csaszar@ericsson.com 810 URI: http://www.ericsson.com/ 811 Kostas Pentikousis 812 EICT GmbH 813 EUREF-Campus Haus 13 814 Torgauer Strasse 12-15 815 10829 Berlin 816 Germany 818 Email: k.pentikousis@eict.de 820 Mario Kind 821 Deutsche Telekom AG 822 Winterfeldtstr. 21 823 10781 Berlin 824 Germany 826 Email: mario.kind@telekom.de 828 Diego Daino 829 Telecom Italia 830 Via Guglielmo Reiss Romoli 274 831 10148 Turin 832 Italy 834 Email: diego.daino@telecomitalia.ite 836 Zu Qiang 837 Ericsson 838 8400, boul. Decarie 839 Ville Mont-Royal, QC 8400 840 Canada 842 Email: zu.qiang@ericsson.com 843 URI: http://www.ericsson.com/ 845 Hagen Woesner 846 BISDN 847 Koernerstr. 7-10 848 Berlin 10785 849 Germany 851 Email: hagen.woesner@bisdn.de 852 URI: http://www.bisdn.de