idnits 2.17.1 draft-irtf-nfvrg-gaps-network-virtualization-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 31, 2016) is 2727 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-05) exists of draft-bernardos-nfvrg-multidomain-00 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFVRG CJ. Bernardos 3 Internet-Draft UC3M 4 Intended status: Informational A. Rahman 5 Expires: May 4, 2017 InterDigital 6 JC. Zuniga 7 SIGFOX 8 LM. Contreras 9 P. Aranda 10 TID 11 October 31, 2016 13 Network Virtualization Research Challenges 14 draft-irtf-nfvrg-gaps-network-virtualization-03 16 Abstract 18 This document describes open research challenges for network 19 virtualization. Network virtualization is following a similar path 20 as previously taken by cloud computing. Specifically, Cloud 21 computing popularized migration of computing functions (e.g., 22 applications) and storage from local, dedicated, physical resources 23 to remote virtual functions accessible through the Internet. In a 24 similar manner, network virtualization is encouraging migration of 25 networking functions from dedicated physical hardware nodes to a 26 virtualized pool of resources. However, network virtualization can 27 be considered to be a more complex problem than cloud computing as it 28 not only involves virtualization of computing and storage functions 29 but also involves abstraction of the network itself. This document 30 describes current research challenges in network virtualization 31 including guaranteeing quality-of-service, performance improvement, 32 supporting multiple domains, network slicing, service composition, 33 device virtualization, privacy and security. In addition, some 34 proposals are made for new activities in IETF/IRTF that could address 35 some of these challenges. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at http://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on May 4, 2017. 54 Copyright Notice 56 Copyright (c) 2016 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (http://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 72 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . 5 74 3.1. Network Function Virtualization . . . . . . . . . . . . . 5 75 3.2. Software Defined Networking . . . . . . . . . . . . . . . 8 76 3.3. Mobile Edge Computing . . . . . . . . . . . . . . . . . . 11 77 3.4. IEEE 802.1CF (OmniRAN) . . . . . . . . . . . . . . . . . 12 78 3.5. Distributed Management Task Force . . . . . . . . . . . . 12 79 3.6. Open Source initiatives . . . . . . . . . . . . . . . . . 12 80 3.7. Internet of Things (IoT) . . . . . . . . . . . . . . . . 14 81 4. Network Virtualization Challenges . . . . . . . . . . . . . . 14 82 4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . 14 83 4.2. Guaranteeing quality-of-service . . . . . . . . . . . . . 15 84 4.2.1. Virtualization Technologies . . . . . . . . . . . . . 15 85 4.2.2. Metrics for NFV characterization . . . . . . . . . . 15 86 4.2.3. Predictive analysis . . . . . . . . . . . . . . . . . 16 87 4.2.4. Portability . . . . . . . . . . . . . . . . . . . . . 16 88 4.3. Performance improvement . . . . . . . . . . . . . . . . . 16 89 4.3.1. Energy Efficiency . . . . . . . . . . . . . . . . . . 16 90 4.3.2. Improved link usage . . . . . . . . . . . . . . . . . 16 91 4.4. Multiple Domains . . . . . . . . . . . . . . . . . . . . 17 92 4.5. Network Slicing . . . . . . . . . . . . . . . . . . . . . 17 93 4.6. Service Composition . . . . . . . . . . . . . . . . . . . 18 94 4.7. End-user device virtualization . . . . . . . . . . . . . 19 95 4.8. Security and Privacy . . . . . . . . . . . . . . . . . . 19 96 4.9. Separation of control concerns . . . . . . . . . . . . . 21 98 5. Technology Gaps and Potential IETF Efforts . . . . . . . . . 21 99 6. Mapping to NFVRG Near-Term work items . . . . . . . . . . . . 21 100 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 101 8. Security Considerations . . . . . . . . . . . . . . . . . . . 22 102 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22 103 10. Informative References . . . . . . . . . . . . . . . . . . . 23 104 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 106 1. Introduction 108 The telecommunications sector is experiencing a major revolution that 109 will shape the way networks and services are designed and deployed 110 for the next decade. We are witnessing an explosion in the number of 111 applications and services demanded by users, which are now really 112 capable of accessing them on the move. In order to cope with such a 113 demand, some network operators are looking at the cloud computing 114 paradigm, which enables a potential reduction of the overall costs by 115 outsourcing communication services from specific hardware in the 116 operator's core to server farms scattered in datacenters. These 117 services have different characteristics if compared with conventional 118 IT services that have to be taken into account in this cloudification 119 process. Also the transport network is affected in that it is 120 evolving to a more sophisticated form of IP architecture with trends 121 like separation of control and data plane traffic, and more fine- 122 grained forwarding of packets (beyond looking at the destination IP 123 address) in the network to fulfill new business and service goals. 125 Virtualization of functions also provides operators with tools to 126 deploy new services much faster, as compared to the traditional use 127 of monolithic and tightly integrated dedicated machinery. As a 128 natural next step, mobile network operators need to re-think how to 129 evolve their existing network infrastructures and how to deploy new 130 ones to address the challenges posed by the increasing customers' 131 demands, as well as by the huge competition among operators. All 132 these changes are triggering the need for a modification in the way 133 operators and infrastructure providers operate their networks, as 134 they need to significantly reduce the costs incurred in deploying a 135 new service and operating it. Some of the mechanisms that are being 136 considered and already adopted by operators include: sharing of 137 network infrastructure to reduce costs, virtualization of core 138 servers running in data centers as a way of supporting their load- 139 aware elastic dimensioning, and dynamic energy policies to reduce the 140 monthly electricity bill. However, this has proved to be tough to 141 put in practice, and not enough. Indeed, it is not easy to deploy 142 new mechanisms in a running operational network due to the high 143 dependency on proprietary (and sometime obscure) protocols and 144 interfaces, which are complex to manage and often require configuring 145 multiple devices in a decentralized way. 147 Network Function Virtualization (NFV) and Software Defined Networking 148 (SDN) are changing the way the telecommunications sector will deploy, 149 extend and operate their networks. This document describes current 150 research challenges in network virtualization and correlates them to 151 activities currently occurring in the key standards forums and open 152 source efforts. Based on this analysis, we also go a step farther, 153 identifying which are the potential work areas where IETF/IRTF can 154 work on to complement the complex network virtualization map of 155 technologies being standardized today. 157 2. Terminology 159 The following terms used in this document are defined by the ETSI NVF 160 ISG, the ONF and the IETF: 162 Application Plane - The collection of applications and services 163 that program network behavior. 165 Control Plane (CP) - The collection of functions responsible for 166 controlling one or more network devices. CP instructs network 167 devices with respect to how to process and forward packets. The 168 control plane interacts primarily with the forwarding plane and, 169 to a lesser extent, with the operational plane. 171 Forwarding Plane (FP) - The collection of resources across all 172 network devices responsible for forwarding traffic. 174 Management Plane (MP) - The collection of functions responsible 175 for monitoring, configuring, and maintaining one or more network 176 devices or parts of network devices. The management plane is 177 mostly related to the operational plane (it is related less to the 178 forwarding plane). 180 NFV Infrastructure (NFVI): totality of all hardware and software 181 components which build up the environment in which VNFs are 182 deployed 184 NFV Management and Orchestration (NFV-MANO): functions 185 collectively provided by NFVO, VNFM, and VIM. 187 NFV Orchestrator (NFVO): functional block that manages the Network 188 Service (NS) lifecycle and coordinates the management of NS 189 lifecycle, VNF lifecycle (supported by the VNFM) and NFVI 190 resources (supported by the VIM) to ensure an optimized allocation 191 of the necessary resources and connectivity. 193 OpenFlow protocol (OFP): allowing vendor independent programming 194 of control functions in network nodes. 196 Operational Plane (OP) - The collection of resources responsible 197 for managing the overall operation of individual network devices. 199 Physical Network Function (PNF): Physical implementation of a 200 Network Function in a monolithic realization. 202 Service Function Chain (SFC): for a given service, the abstracted 203 view of the required service functions and the order in which they 204 are to be applied. This is somehow equivalent to the Network 205 Function Forwarding Graph (NF-FG) at ETSI. 207 Service Function Path (SFP): the selection of specific service 208 function instances on specific network nodes to form a service 209 graph through which an SFC is instantiated. 211 virtual EPC (vEPC): control plane of 3GPPs EPC operated on NFV 212 framework (as defined by [I-D.matsushima-stateless-uplane-vepc]). 214 Virtualized Infrastructure Manager (VIM): functional block that is 215 responsible for controlling and managing the NFVI compute, storage 216 and network resources, usually within one operator's 217 Infrastructure Domain. 219 Virtualized Network Function (VNF): implementation of a Network 220 Function that can be deployed on a Network Function Virtualization 221 Infrastructure (NFVI). 223 Virtualized Network Function Manager (VNFM): functional block that 224 is responsible for the lifecycle management of VNF. 226 3. Background 228 3.1. Network Function Virtualization 230 The ETSI ISG NFV is a working group which, since 2012, aims to evolve 231 quasi-standard IT virtualization technology to consolidate many 232 network equipment types into industry standard high volume servers, 233 switches, and storage. It enables implementing network functions in 234 software that can run on a range of industry standard server hardware 235 and can be moved to, or loaded in, various locations in the network 236 as required, without the need to install new equipment. To date, 237 ETSI NFV is by far the most accepted NFV reference framework and 238 architectural footprint [etsi_nvf_whitepaper]. The ETSI NFV 239 framework architecture framework is composed of three domains 240 (Figure 1): 242 o Virtualized Network Function, running over the NFVI. 244 o NFV Infrastructure (NFVI), including the diversity of physical 245 resources and how these can be virtualized. NFVI supports the 246 execution of the VNFs. 248 o NFV Management and Orchestration, which covers the orchestration 249 and life-cycle management of physical and/or software resources 250 that support the infrastructure virtualization, and the life-cycle 251 management of VNFs. NFV Management and Orchestration focuses on 252 all virtualization specific management tasks necessary in the NFV 253 framework. 255 +-------------------------------------------+ +---------------+ 256 | Virtualized Network Functions (VNFs) | | | 257 | ------- ------- ------- ------- | | | 258 | | | | | | | | | | | | 259 | | VNF | | VNF | | VNF | | VNF | | | | 260 | | | | | | | | | | | | 261 | ------- ------- ------- ------- | | | 262 +-------------------------------------------+ | | 263 | | 264 +-------------------------------------------+ | | 265 | NFV Infrastructure (NFVI) | | NFV | 266 | ----------- ----------- ----------- | | Management | 267 | | Virtual | | Virtual | | Virtual | | | and | 268 | | Compute | | Storage | | Network | | | Orchestration | 269 | ----------- ----------- ----------- | | | 270 | +---------------------------------------+ | | | 271 | | Virtualization Layer | | | | 272 | +---------------------------------------+ | | | 273 | +---------------------------------------+ | | | 274 | | ----------- ----------- ----------- | | | | 275 | | | Compute | | Storage | | Network | | | | | 276 | | ----------- ----------- ----------- | | | | 277 | | Hardware resources | | | | 278 | +---------------------------------------+ | | | 279 +-------------------------------------------+ +---------------+ 281 Figure 1: ETSI NFV framework 283 The NFV architectural framework identifies functional blocks and the 284 main reference points between such blocks. Some of these are already 285 present in current deployments, whilst others might be necessary 286 additions in order to support the virtualization process and 287 consequent operation. The functional blocks are (Figure 2): 289 o Virtualized Network Function (VNF). 291 o Element Management (EM). 293 o NFV Infrastructure, including: Hardware and virtualized resources, 294 and Virtualization Layer. 296 o Virtualized Infrastructure Manager(s) (VIM). 298 o NFV Orchestrator. 300 o VNF Manager(s). 302 o Service, VNF and Infrastructure Description. 304 o Operations and Business Support Systems (OSS/BSS). 306 +--------------------+ 307 +-------------------------------------------+ | ---------------- | 308 | OSS/BSS | | | NFV | | 309 +-------------------------------------------+ | | Orchestrator +-- | 310 | ---+------------ | | 311 +-------------------------------------------+ | | | | 312 | --------- --------- --------- | | | | | 313 | | EM 1 | | EM 2 | | EM 3 | | | | | | 314 | ----+---- ----+---- ----+---- | | ---+---------- | | 315 | | | | |--|-| VNF | | | 316 | ----+---- ----+---- ----+---- | | | manager(s) | | | 317 | | VNF 1 | | VNF 2 | | VNF 3 | | | ---+---------- | | 318 | ----+---- ----+---- ----+---- | | | | | 319 +------|-------------|-------------|--------+ | | | | 320 | | | | | | | 321 +------+-------------+-------------+--------+ | | | | 322 | NFV Infrastructure (NFVI) | | | | | 323 | ----------- ----------- ----------- | | | | | 324 | | Virtual | | Virtual | | Virtual | | | | | | 325 | | Compute | | Storage | | Network | | | | | | 326 | ----------- ----------- ----------- | | ---+------ | | 327 | +---------------------------------------+ | | | | | | 328 | | Virtualization Layer | |--|-| VIM(s) +-------- | 329 | +---------------------------------------+ | | | | | 330 | +---------------------------------------+ | | ---------- | 331 | | ----------- ----------- ----------- | | | | 332 | | | Compute | | Storage | | Network | | | | | 333 | | | hardware| | hardware| | hardware| | | | | 334 | | ----------- ----------- ----------- | | | | 335 | | Hardware resources | | | NFV Management | 336 | +---------------------------------------+ | | and Orchestration | 337 +-------------------------------------------+ +--------------------+ 339 Figure 2: ETSI NFV reference architecture 341 3.2. Software Defined Networking 343 The Software Defined Networking (SDN) paradigm pushes the 344 intelligence currently residing in the network elements to a central 345 controller implementing the network functionality through software. 346 In contrast to traditional approaches, in which the network's control 347 plane is distributed throughout all network devices, with SDN the 348 control plane is logically centralized. In this way, the deployment 349 of new characteristics in the network no longer requires of complex 350 and costly changes in equipment or firmware updates, but only a 351 change in the software running in the controller. The main advantage 352 of this approach is the flexibility it provides operators with to 353 manage their network, i.e., an operator can easily change its 354 policies on how traffic is distributed throughout the network. 356 The most visible of the SDN protocol stacks is the OpenFlow protocol 357 (OFP), which is maintained and extended by the Open Network 358 Foundation (ONF: https://www.opennetworking.org/). Originally this 359 protocol was developed specifically for IEEE 802.1 switches 360 conforming to the ONF OpenFlow Switch specification. As the benefits 361 of the SDN paradigm have reached a wider audience, its application 362 has been extended to more complex scenarios such as Wireless and 363 Mobile networks. Within this area of work, the ONF is actively 364 developing new OFP extensions addressing three key scenarios: (i) 365 Wireless backhaul, (ii) Cellular Evolved Packet Core (EPC), and (iii) 366 Unified access and management across enterprise wireless and fixed 367 networks. 369 +----------+ 370 | ------- | 371 | |Oper.| | O 372 | |Mgmt.| |<........> -+- Network Operator 373 | |Iface| | ^ 374 | ------- | +----------------------------------------+ 375 | | | +------------------------------------+ | 376 | | | | --------- --------- --------- | | 377 |--------- | | | | App 1 | | App 2 | ... | App n | | | 378 ||Plugins| |<....>| | --------- --------- --------- | | 379 |--------- | | | Plugins | | 380 | | | +------------------------------------+ | 381 | | | Application Plane | 382 | | +----------------------------------------+ 383 | | A 384 | | | 385 | | V 386 | | +----------------------------------------+ 387 | | | +------------------------------------+ | 388 |--------- | | | ------------ ------------ | | 389 || Netw. | | | | | Module 1 | | Module 2 | | | 390 ||Engine | |<....>| | ------------ ------------ | | 391 |--------- | | | Network Engine | | 392 | | | +------------------------------------+ | 393 | | | Controller Plane | 394 | | +----------------------------------------+ 395 | | A 396 | | | 397 | | V 398 | | +----------------------------------------+ 399 | | | +--------------+ +--------------+ | 400 | | | | ------------ | | ------------ | | 401 |----------| | | | OpenFlow | | | | OpenFlow | | | 402 ||OpenFlow||<....>| | ------------ | | ------------ | | 403 |----------| | | NE | | NE | | 404 | | | +--------------+ +--------------+ | 405 | | | Data Plane | 406 |Management| +----------------------------------------+ 407 +----------+ 409 Figure 3: High level SDN ONF architecture 411 Figure 3 shows the blocks and the functional interfaces of the ONF 412 architecture, which comprises three planes: Data, Controller, and 413 Application. The Data plane comprehends several Network Entities 414 (NE), which expose their capabilities toward the Controller plane via 415 a Southbound API. The Controller plane includes several cooperating 416 modules devoted to the creation and maintenance of an abstracted 417 resource model of the underneath network. Such model is exposed to 418 the applications via a Northbound API where the Application plane 419 comprises several applications/services, each of which has exclusive 420 control of a set of exposed resources. 422 The Management plane spans its functionality across all planes 423 performing the initial configuration of the network elements in the 424 Data plane, the assignment of the SDN controller and the resources 425 under its responsibility. In the Controller plane, the Management 426 needs to configure the policies defining the scope of the control 427 given to the SDN applications, to monitor the performance of the 428 system, and to configure the parameters required by the SDN 429 controller modules. In the Application plane, Management configures 430 the parameters of the applications and the service level agreements. 431 In addition to the these interactions, the Management plane exposes 432 several functions to network operators which can easily and quickly 433 configure and tune the network at each layer. 435 The SDNRG has documented a reference layer model in RFC7426 436 [RFC7426], which is reproduced in Figure 4. This model structures 437 SDN in planes and layers which are glued together by different 438 abstraction layers. This architecture differentiates between the 439 control and the management planes and provides for differentiated 440 southbound interfaces (SBIs). 442 o--------------------------------o 443 | | 444 | +-------------+ +----------+ | 445 | | Application | | Service | | 446 | +-------------+ +----------+ | 447 | Application Plane | 448 o---------------Y----------------o 449 | 450 *-----------------------------Y---------------------------------* 451 | Network Services Abstraction Layer (NSAL) | 452 *------Y------------------------------------------------Y-------* 453 | | 454 | Service Interface | 455 | | 456 o------Y------------------o o---------------------Y------o 457 | | Control Plane | | Management Plane | | 458 | +----Y----+ +-----+ | | +-----+ +----Y----+ | 459 | | Service | | App | | | | App | | Service | | 460 | +----Y----+ +--Y--+ | | +--Y--+ +----Y----+ | 461 | | | | | | | | 462 | *----Y-----------Y----* | | *---Y---------------Y----* | 463 | | Control Abstraction | | | | Management Abstraction | | 464 | | Layer (CAL) | | | | Layer (MAL) | | 465 | *----------Y----------* | | *----------Y-------------* | 466 | | | | | | 467 o------------|------------o o------------|---------------o 468 | | 469 | CP | MP 470 | Southbound | Southbound 471 | Interface | Interface 472 | | 473 *------------Y---------------------------------Y----------------* 474 | Device and resource Abstraction Layer (DAL) | 475 *------------Y---------------------------------Y----------------* 476 | | | | 477 | o-------Y----------o +-----+ o--------Y----------o | 478 | | Forwarding Plane | | App | | Operational Plane | | 479 | o------------------o +-----+ o-------------------o | 480 | Network Device | 481 +---------------------------------------------------------------+ 483 Figure 4: SDN Layer Architecture 485 3.3. Mobile Edge Computing 487 Mobile Edge Computing capabilities deployed in the edge of the mobile 488 network can facilitate the efficient and dynamic provision of 489 services to mobile users. The ETSI ISG MEC working group, operative 490 from end of 2014, intends to specify an open environment for 491 integrating MEC capabilities with service providers networks, 492 including also applications from 3rd parties. These distributed 493 computing capabilities will make available IT infrastructure as in a 494 cloud environment for the deployment of functions in mobile access 495 networks. It can be seen then as a complement to both NFV and SDN. 497 3.4. IEEE 802.1CF (OmniRAN) 499 The IEEE 802.1CF Recommended Practice specifies an access network, 500 which connects terminals to their access routers, utilizing 501 technologies based on the family of IEEE 802 Standards (e.g., 802.3 502 Ethernet, 802.11 Wi-Fi, etc.). The specification defines an access 503 network reference model, including entities and reference points 504 along with behavioral and functional descriptions of communications 505 among those entities. 507 The goal of this project is to help unifying the support of different 508 interfaces, enabling shared network control and use of software 509 defined network (SDN) principles, thereby lowering the barriers to 510 new network technologies, to new network operators, and to new 511 service providers. 513 3.5. Distributed Management Task Force 515 The DMTF is an industry standards organization working to simplify 516 the manageability of network-accessible technologies through open and 517 collaborative efforts by some technology companies. The DMTF is 518 involved in the creation and adoption of interoperable management 519 standards, supporting implementations that enable the management of 520 diverse traditional and emerging technologies including cloud, 521 virtualization, network and infrastructure. 523 There are several DMTF initiatives that are relevant to the network 524 virtualization area, such as the Open Virtualization Format (OVF), 525 for VNF packaging; the Cloud Infrastructure Management Interface 526 (CIM), for cloud infrastructure management; the Network Management 527 (NETMAN), for VNF management; and, the Virtualization Management 528 (VMAN), for virtualization infrastructure management. 530 3.6. Open Source initiatives 532 The Open Source community is especially active in the area of network 533 virtualization. We next summarize some of the active efforts: 535 o OpenStack. OpenStack is a free and open-source cloud-computing 536 software platform. OpenStack software controls large pools of 537 compute, storage, and networking resources throughout a 538 datacenter, managed through a dashboard or via the OpenStack API. 540 o OpenDayLight. OpenDaylight (ODL) is a highly available, modular, 541 extensible, scalable and multi-protocol controller infrastructure 542 built for SDN deployments on modern heterogeneous multi-vendor 543 networks. It provides a model-driven service abstraction platform 544 that allows users to write apps that easily work across a wide 545 variety of hardware and southbound protocols. 547 o ONOS. The ONOS (Open Network Operating System) project is an open 548 source community hosted by The Linux Foundation. The goal of the 549 project is to create a software-defined networking (SDN) operating 550 system for communications service providers that is designed for 551 scalability, high performance and high availability. 553 o OpenContrail. OpenContrail is an Apache 2.0-licensed project that 554 is built using standards-based protocols and provides all the 555 necessary components for network virtualization-SDN controller, 556 virtual router, analytics engine, and published northbound APIs. 557 It has an extensive REST API to configure and gather operational 558 and analytics data from the system. 560 o OPNFV. OPNFV is a carrier-grade, integrated, open source platform 561 to accelerate the introduction of new NFV products and services. 562 By integrating components from upstream projects, the OPNFV 563 community aims at conducting performance and use case-based 564 testing to ensure the platform's suitability for NFV use cases. 565 The scope of OPNFV's initial release is focused on building NFV 566 Infrastructure (NFVI) and Virtualized Infrastructure Management 567 (VIM) by integrating components from upstream projects such as 568 OpenDaylight, OpenStack, Ceph Storage, KVM, Open vSwitch, and 569 Linux. These components, along with application programmable 570 interfaces (APIs) to other NFV elements form the basic 571 infrastructure required for Virtualized Network Functions (VNF) 572 and Management and Network Orchestration (MANO) components. 573 OPNFV's goal is to increase performance and power efficiency; 574 improve reliability, availability, and serviceability; and deliver 575 comprehensive platform instrumentation. 577 o OSM. Open Source Mano (OSM) is an ETSI-hosted project to develop 578 an Open Source NFV Management and Orchestration (MANO) software 579 stack aligned with ETSI NFV. OSM is based on components from 580 previous projects, such Telefonica's OpenMANO or Canonical's Juju, 581 among others. 583 o OpenBaton. OpenBaton is a ETSI NFV compliant Network Function 584 Virtualization Orchestrator (NFVO). OpenBaton was part of the 585 OpenSDNCore project started with the objective of providing a 586 compliant implementation of the ETSI NFV specification. 588 Among the main areas that are being developed by the former open 589 source activities that related to network virtualization research, we 590 can highlight: policy-based resource management, analytics for 591 visibility and orchestration, service verification with regards to 592 security and resiliency. 594 3.7. Internet of Things (IoT) 596 The Internet of Things (IoT) refers to the vision of connecting a 597 multitude of automated devices (e.g. lights, environmental sensors, 598 traffic lights, parking meters, health and security systems, etc.) to 599 the Internet for purposes of reporting, and remote command and 600 control of the device. This vision is being realized by a multi- 601 pronged approach of standardization in various forums and 602 complementary open source activities. For example, in IETF, support 603 of IoT web services has been defined by an HTTP-like protocol adapted 604 for IoT called CoAP [RFC7252], and lately a group has been studying 605 the need to develop a new network layer to support IP applications 606 over Low Power Wide Area Networks (LPWAN). 608 Elsewhere, for 5G cellular evolution there is much discussion on the 609 need for supporting virtual "network slices" for the expected massive 610 numbers of IoT devices. A separate virtual network slice is 611 considered necessary for different 5G IoT use cases because devices 612 will have very different characteristics than typical cellular 613 devices like smart phones [ngmn_5G_whitepaper], and the number of IoT 614 devices is expected to be at least one or two orders of magnitude 615 higher than other 5G devices. 617 4. Network Virtualization Challenges 619 4.1. Introduction 621 Network Virtualization is changing the way the telecommunications 622 sector will deploy, extend and operate their networks. These new 623 technologies aim at reducing the overall costs by outsourcing 624 communication services from specific hardware in the operators' core 625 to server farms scattered in datacenters (i.e. compute and storage 626 virtualization). In addition, the connecting networks are 627 fundamentally affected in the way they route, process and control 628 traffic (i.e. network virtualization). 630 4.2. Guaranteeing quality-of-service 632 Guaranteeing a given quality-of-service in an NFV environment is not 633 an easy task. For example, ensuring a guaranteed and stable 634 forwarding data rate has proven not to be straightforward when the 635 forwarding function is virtualized and runs on top of COTS server 636 hardware. We next identify some of the challenges that this poses. 638 4.2.1. Virtualization Technologies 640 The issue of guaranteeing a network quality-of-service is less of an 641 issue for "traditional cloud computing". NFV poses very strict 642 requirements posed in terms of performance, stability and 643 consistency. Although there are some tools and mechanisms to improve 644 this, such as Enhanced Performance Awareness (EPA), SR-IOV, NUMA, 645 DPDK, etc, these are still unsolved challenges. One open research 646 issue is finding out technologies that are different from VM and more 647 suitable for dealing with network functionalities. 649 Lately, a number of light-weight virtualization technologies 650 including containers, unikernels (specialized VMs) and minimalistic 651 distributions of general-purpose OSes have appeared as virtualization 652 approaches that can be used when constructing an NFV platform. 653 [I-D.natarajan-nfvrg-containers-for-nfv] describes the challenges in 654 building such a platform and discusses to what extent these 655 technologies, as well as traditional VMs, are able to address them. 657 4.2.2. Metrics for NFV characterization 659 Another relevant aspect is the need for tools for diagnostics and 660 measurement suited for NFV. There is a pressing need to define 661 metrics and associated protocols to measure the performance of NFV. 662 Specifically, since NFV is based on the concept of taking centralized 663 functions and evolving it to highly distributed SW functions, there 664 is a commensurate need to fully understand and measure the baseline 665 performance of such systems. 667 The IP Performance Metrics (IPPM) WG defines metrics that can be used 668 to measure the quality and performance of Internet services and 669 applications running over transport layer protocols (e.g., TCP, UPD) 670 over IP. It also develops and maintains protocols for the 671 measurement of these metrics. While the IPPM WG is a long running WG 672 that started in 1997 it does not have a charter item or active drafts 673 related to the topic of network virtualization. In addition to using 674 IPPM metrics to evaluate the QoS, there is a need for specific 675 metrics for assessing the performance of network virtualization 676 techniques. 678 4.2.3. Predictive analysis 680 On top of diagnostic tools that enable an assessment of the QoS, 681 predictive analyses are required to react before anomalies occur. 682 Due to the SW characteristics of VNFs, a reliable diagnosis framework 683 could potentially enable the prevention of issues by a proper 684 diagnosis and then a reaction in terms of acting on the potentially 685 impacted service (e.g., migration to a different compute node, 686 scaling in/out, up/down, etc). 688 4.2.4. Portability 690 Portability is also a key feature that, if fully enabled, would 691 contribute to making the NFV environment achieve a better reliability 692 than a traditional system. The fact of running functionality in SW 693 over "commodity" infrastructure should make much easier to port/move 694 functions from one place to another. However this is not yet as 695 ideal as it sounds and there are aspects not fully tackled. The 696 existence of different hypervisors, specific hardware dependencies 697 (e.g., EPA related) or state synchronization aspects are just some 698 examples of trouble-makers for portability purposes. 700 4.3. Performance improvement 702 4.3.1. Energy Efficiency 704 Virtualization is typically seen as a direct enabler of energy 705 savings. Some of the enablers for this that are often mentioned are: 706 (i) the multiplexing gains achieved by centralizing functions in data 707 centers reduce overall the energy consumed, (ii) the flexibility 708 brought by network programmability enables to switch off 709 infrastructure as needed in a much easier way. However there is 710 still a lot of room for improvement in terms of virtualization 711 techniques to reduce the power consumption, such as enhanced 712 hypervisor technologies. 714 4.3.2. Improved link usage 716 The use of NFV and SDN technologies can help improving link usage. 717 SDN has shown already that it can greatly increase average link usage 718 (e.g., Google example). NFV adds more complexity (e.g., due to 719 service function chaining / VNF forwarding drafts) which need to be 720 considered. Aspects like the ones described in 721 [I-D.bagnulo-nfvrg-topology] on NFV data center topology design have 722 to be carefully looked as well. 724 4.4. Multiple Domains 726 Market fragmentation has resulted in a multitude of network operators 727 each focused on different countries and regions. This makes it 728 difficult to create infrastructure services spanning multiple 729 countries, such as virtual connectivity or compute resources, as no 730 single operator has a footprint everywhere. Cross-domain 731 orchestration of services over multiple administrations or over 732 multi-domain single administrations will allow end-to-end network and 733 service elements to mix in multi-vendor, heterogeneous technology and 734 resource environments. 736 For the specific use case of 'Network as a Service', it becomes even 737 more important to ensure, that Cross Domain Orchestration also takes 738 care of hierarchy of networks and their association, with respect to 739 provisioning tunnels and overlays. 741 Multi-domain orchestration is currently an active research topic, 742 which is being tackled, among others, by ETSI NFV ISG and the 5GEx 743 project (https://www.5gex.eu/) [I-D.bernardos-nfvrg-multidomain]. 745 4.5. Network Slicing 747 From the beginning of all 5G discussions in the research and industry 748 fora, it has been agreed that 5G will have to address much more use 749 cases than the preceding wireless generations, which first focused on 750 voice services, and then on voice and high speed packet data 751 services. In this case, 5G should be able to handle not only the 752 same (or enhanced) voice and packet data services, but also new 753 emerging services like tactile Internet and IoT. These use cases 754 take the requirements to opposite extremes, as some of them require 755 ultra-low latency and higher-speed, whereas some others require 756 ultra-low power consumption and high delay tolerance. 758 Because of these very extreme 5G use cases, it is envisioned that 759 different radio access networks are needed to better address the 760 specific requirements of each one of the use cases. However, on the 761 core network side, virtualization techniques can allow tailoring the 762 network resources on separate slices, specifically for each radio 763 access network and use case, in an efficient manner. 765 Network slicing techniques can also allow dedicating resources for 766 even more specific use cases within the major 5G categories. For 767 example, within the major IoT category, which is perhaps the most 768 disrupting one, some autonomous IoT devices will have very low 769 throughput, will have much longer sleep cycles (and therefore high 770 latency), and a battery life thousands of times longer compared to 771 smart phones or some other connected IoT devices that will have 772 almost continuous control and data communications. Hence, it is 773 envisioned that a single virtual core network could be used by 774 slicing separate resources to dedicated radio access networks (RANs) 775 that are better suited for specific use cases. 777 Network slicing is also a key for introducing new actors in existing 778 market at low cost -- by letting new players rent "blocks" of 779 capacity, if this new market provides performance that are adequate 780 with the application needs (e.g., broadcasting updates to many 781 sensors with satellite broadcasting capabilities). However, more 782 work needs to be done to define how network slicing will impact 783 existing architectures like ETSI NFV, and to define the impacts of 784 network slicing to guaranteeing quality-of-service as described in 785 Section 4.2. 787 4.6. Service Composition 789 Current network services deployed by operators often involve the 790 composition of several individual functions (such as packet 791 filtering, deep packet inspection, load balancing). These services 792 are typically implemented by the ordered combination of a number of 793 service functions that are deployed at different points within a 794 network, not necessary on the direct data path. This requires 795 traffic to be steered through the required service functions, 796 wherever they are deployed. 798 For a given service, the abstracted view of the required service 799 functions and the order in which they are to be applied is called a 800 Service Function Chain (SFC), which is called Network Function 801 Forwarding Graph (NF-FG) in ETSI. An SFC is instantiated through 802 selection of specific service function instances on specific network 803 nodes to form a service graph: this is called a Service Function Path 804 (SFP). The service functions may be applied at any layer within the 805 network protocol stack (network layer, transport layer, application 806 layer, etc.). 808 Service composition is a powerful tool which can provide significant 809 benefits when applied in a softwarized network environment. There 810 are however many research challenges in this area, as for example the 811 ones related to composition mechanisms and algorithms to enable load 812 balancing and improve reliability. The service composition should 813 also act as an enabler to gather information across all hierarchies 814 (underlays and overlays) of network deployments which may span across 815 multiple operators, for faster serviceability thus facilitating in 816 accomplishing aforementioned goals of "load balancing and improve 817 reliability". 819 The SFC working group is working on an architecture for service 820 function chaining that includes the necessary protocols or protocol 821 extensions to convey the Service Function Chain and Service Function 822 Path information to nodes that are involved in the implementation of 823 service functions and Service Function Chains, as well as mechanisms 824 for steering traffic through service functions. 826 In terms of actual work items, the SFC WG is has not yet considered 827 working on the management and configuration of SFC components related 828 to the support of Service Function Chaining. This part is of special 829 interest for operators and would be required in order to actually put 830 SFC mechanisms into operation. Similarly, redundancy and reliability 831 mechanisms are currently not dealt with by any WG in the IETF. While 832 this was the main goal of the VNFpool BoF efforts, it still remains 833 unaddressed. 835 4.7. End-user device virtualization 837 So far, most of the network softwarization efforts have focused on 838 virtualizing functions of network elements. While virtualization of 839 network elements started with the core, mobile networks architectures 840 are now heavily switching to also virtualize radio access network 841 (RAN) functions. The next natural step is to get virtualization down 842 at the level of the end-user device (i.e., virtualizing a 843 smartphone). The cloning of a device in the cloud (central or local) 844 bears attractive benefits to both the device and network operations 845 alike (e.g., power saving at the device by offloading computational- 846 heaving functions to the cloud, optimized networking -- both device- 847 to-device and device-to-infrastructure) for service delivery through 848 tighter integration of the device (via its clone in the networking 849 infrastructure). This is being explored for example by the European 850 H2020 ICIRRUS project (www.icirrus-5gnet.eu). 852 4.8. Security and Privacy 854 Similar to any other situation where resources are shared, security 855 and privacy are two important aspects that need to be taken into 856 account. 858 In the case of security, there are situations where multiple vendors 859 will need to coexist in a virtual or hybrid physical/virtual 860 environment. This requires attestation procedures amongst different 861 virtual/physical functions and resources, as well as ongoing external 862 monitoring. Similarly, different network slices operating on the 863 same infrastructure can present security problems, for instance if 864 one slice running critical applications (e.g. support for a safety 865 system) is affected by another slice running a less critical 866 application. In general, the minimum common denominator for security 867 measures on a shared system should be equal or higher than the one 868 required by the most critical application. Multiple and continuous 869 threat model analysis, as well as DevOps model are required to 870 maintain certain level of security in an NFV system. 872 On the other hand, privacy in its strictest interpretation, refers to 873 concerns about exposing users of the system to individual threats 874 such as surveillance, identification, stored data compromise, 875 secondary use, intrusion, etc. In this case, the storage, 876 transmission, collection, and potential correlation of information in 877 the NFV system, for purposes not originally intended or not known by 878 the user, should be avoided. This is particularly challenging, as 879 future intentions and threats cannot be easily predicted, and still 880 can be applied for instance on data collected in the past. 881 Therefore, well-known techniques such as data minimization, using 882 privacy features as default, and allowing users to opt in/out should 883 be used to prevent potential privacy issues. 885 Compared to traditional networks, NFV will result in networks that 886 are much more dynamic (in function distribution and topology) and 887 elastic (in size and boundaries). NFV will thus require network 888 operators to evolve their operational and administrative security 889 solutions to work in this new environment. For example, in NFV the 890 network orchestrator will become a key node to provide security 891 policy orchestration across the different physical and virtual 892 components of the virtualized network. For highly confidential data, 893 for example, the network orchestrator should take into account if 894 certain physical HW of the network is considered more secure (e.g., 895 because it is located in secure premises) than other HW. 897 Traditional telecom networks typically run under a single 898 administrative domain controlled by an operator. With NFV, it is 899 expected that in many cases, the telecom operator will now become a 900 tenant (running the VNFs), and the infrastructure (NFVI) may be run 901 by a different operator and/or cloud service provider (see also 902 Section 4.4). Thus, there will be multiple administrative domains 903 which will make coordination of security policy more complex. For 904 example, who will be in charge of provisioning and maintaining 905 security credentials such as public and private keys? Also, should 906 private keys be allowed to be replicated across the NFV for 907 redundancy reasons? 909 On a positive note, NFV will allow better defense against Denial of 910 Service (DoS) attacks because of the distributed nature of the 911 network (i.e. no single point of failure) and the ability to steer 912 (undesirable) traffic quickly. Also, NFVs which have physical HW 913 which is distributed across multiple data centers will also provide 914 better fault isolation environments. Especially, if each data center 915 is protected separately via fire walls, DMZs and other network 916 protection techniques. 918 4.9. Separation of control concerns 920 NFV environments offer two possible levels of SDN control. One level 921 is the need for controlling the NFVI to provide connectivity end-to- 922 end among VNFs or among VNFs and PNFs (Physical Network Functions). 923 A second level is the control and configuration of the VNFs 924 themselves (in other words, the configuration of the network service 925 implemented by those VNFs), taking profit of the programmability 926 brought by SDN. Both control concerns are separated in nature. 927 However, interaction between both could be expected in order to 928 optimize, scale or influence each other. 930 Clear mechanisms for such interaction are needed in order to avoid 931 mal-functioning or interference among concerns. These ideas are 932 considered in [etsi_sdn_in_nvf_eve005] and 933 [I-D.irtf-sdnrg-layered-sdn] 935 5. Technology Gaps and Potential IETF Efforts 937 Table 1 correlates the open network virtualization research areas 938 identified in this document to potential IETF groups that could 939 address some aspects of them. An example of a specific gap that the 940 group could potentially address is identified in parenthetical beside 941 the group name. 943 +----------------------------------+--------------------------------+ 944 | Open Research Area | Potential IETF/IRTF Group | 945 +----------------------------------+--------------------------------+ 946 | 1-Guaranteeing QoS | IPPM WG (Measurements of NFVI) | 947 | 2-Performance improvement | VNFPOOL BoF (NFV resilience) | 948 | 3-Multiple Domains | NFVRG | 949 | 4-Network Slicing | NVO3 WG (5G Traffic isolation) | 950 | 5-Service Composition | SFC WG (SFC Mgmt and Config) | 951 | 6-End-user device virtualization | N/A | 952 | 7-Security | N/A | 953 | 8-Separation of control concerns | SDNRG | 954 +----------------------------------+--------------------------------+ 956 Table 1: Mapping of Open Research Areas to Potential IETF Groups 958 6. Mapping to NFVRG Near-Term work items 960 Table 2 correlates the currently identified NFVRG near-work items to 961 the open network virtualization research areas enumerated in this 962 document. This can help the NFVRG in identifying and prioritizing 963 research topics. 965 +--------------------------------------+-------------------------+ 966 | NFVRG Near-Term work item | Open Research Area | 967 +--------------------------------------+-------------------------+ 968 | 1-Policy-based resource management | - Performance improvem. | 969 | | - Network Slicing | 970 | 2-Analytics for visibility & orches. | - Guaranteeing QoS | 971 | 3-Security and service verification | - Security | 972 | 4-Reliability and fault detection | - Guaranteeing QoS | 973 | 5-Service orchestration & lifecycle | - Multiple Domains | 974 | | - Network Slicing | 975 | | - Service Composition | 976 | 6-Real-time properties | - Guaranteeing QoS | 977 | | | 978 | (other) | - End-user device virt. | 979 | | - Separation of control | 980 +--------------------------------------+-------------------------+ 982 Table 2: Mapping of NFVRG Near-Term work items to Open Research Areas 984 7. IANA Considerations 986 N/A. 988 8. Security Considerations 990 This is an informational document, which therefore does not introduce 991 any security threat. Research challenges and gaps related to 992 security and privacy have been included in Section 4.8. 994 9. Acknowledgments 996 The authors want to thank Dirk von Hugo, Rafa Marin, Diego Lopez, 997 Ramki Krishnan, Kostas Pentikousis, Rana Pratap Sircar, Alfred 998 Morton, Nicolas Kuhn and Saumya Dikshit for their very useful reviews 999 and comments to the document. 1001 The work of Carlos J. Bernardos and Luis M. Contreras is partially 1002 supported by the H2020-ICT-2014 project 5GEx (Grant Agreement no. 1003 671636). 1005 The work of Pedro Aranda is supported by the European FP7 Project 1006 Trilogy2 under grant agreement 317756. 1008 10. Informative References 1010 [etsi_nvf_whitepaper] 1011 "Network Functions Virtualisation (NFV). White Paper 2", 1012 October 2014. 1014 [etsi_sdn_in_nvf_eve005] 1015 "Report on SDN Usage in NFV Architectural Framework", 1016 Dicember 2015. 1018 [I-D.bagnulo-nfvrg-topology] 1019 Bagnulo, M. and D. Dolson, "NFVI PoP Network Topology: 1020 Problem Statement", draft-bagnulo-nfvrg-topology-01 (work 1021 in progress), March 2016. 1023 [I-D.bernardos-nfvrg-multidomain] 1024 Bernardos, C. and L. Contreras, "Multi-domain Network 1025 Virtualization", draft-bernardos-nfvrg-multidomain-00 1026 (work in progress), March 2016. 1028 [I-D.irtf-sdnrg-layered-sdn] 1029 Contreras, L., Bernardos, C., Lopez, D., Boucadair, M., 1030 and P. Iovanna, "Cooperating Layered Architecture for 1031 SDN", draft-irtf-sdnrg-layered-sdn-01 (work in progress), 1032 October 2016. 1034 [I-D.matsushima-stateless-uplane-vepc] 1035 Matsushima, S. and R. Wakikawa, "Stateless user-plane 1036 architecture for virtualized EPC (vEPC)", draft- 1037 matsushima-stateless-uplane-vepc-06 (work in progress), 1038 March 2016. 1040 [I-D.natarajan-nfvrg-containers-for-nfv] 1041 natarajan.sriram@gmail.com, n., Krishnan, R., Ghanwani, 1042 A., Krishnaswamy, D., Willis, P., Chaudhary, A., and F. 1043 Huici, "An Analysis of Lightweight Virtualization 1044 Technologies for NFV", draft-natarajan-nfvrg-containers- 1045 for-nfv-03 (work in progress), July 2016. 1047 [ngmn_5G_whitepaper] 1048 "NGMN 5G. White Paper", February 2015. 1050 [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained 1051 Application Protocol (CoAP)", RFC 7252, 1052 DOI 10.17487/RFC7252, June 2014, 1053 . 1055 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S., 1056 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software- 1057 Defined Networking (SDN): Layers and Architecture 1058 Terminology", RFC 7426, DOI 10.17487/RFC7426, January 1059 2015, . 1061 Authors' Addresses 1063 Carlos J. Bernardos 1064 Universidad Carlos III de Madrid 1065 Av. Universidad, 30 1066 Leganes, Madrid 28911 1067 Spain 1069 Phone: +34 91624 6236 1070 Email: cjbc@it.uc3m.es 1071 URI: http://www.it.uc3m.es/cjbc/ 1073 Akbar Rahman 1074 InterDigital Communications, LLC 1075 1000 Sherbrooke Street West, 10th floor 1076 Montreal, Quebec H3A 3G4 1077 Canada 1079 Email: Akbar.Rahman@InterDigital.com 1080 URI: http://www.InterDigital.com/ 1082 Juan Carlos Zuniga 1083 SIGFOX 1084 425 rue Jean Rostand 1085 Labege 31670 1086 France 1088 Email: j.c.zuniga@ieee.org 1089 URI: http://www.sigfox.com/ 1091 Luis M. Contreras 1092 Telefonica I+D 1093 Ronda de la Comunicacion, S/N 1094 Madrid 28050 1095 Spain 1097 Email: luismiguel.contrerasmurillo@telefonica.com 1098 Pedro Aranda 1099 Telefonica I+D 1100 Ronda de la Comunicacion, S/N 1101 Madrid 28050 1102 Spain 1104 Email: pedroa.aranda@telefonica.com