idnits 2.17.1 draft-bernardos-nfvrg-multidomain-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 13, 2017) is 2593 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFV RG CJ. Bernardos 3 Internet-Draft UC3M 4 Intended status: Informational LM. Contreras 5 Expires: September 14, 2017 TID 6 I. Vaishnavi 7 Huawei 8 R. Szabo 9 Ericsson 10 March 13, 2017 12 Multi-domain Network Virtualization 13 draft-bernardos-nfvrg-multidomain-02 15 Abstract 17 This draft analyzes the problem of multi-provider multi-domain 18 orchestration, by first scoping the problem, then looking into 19 potential architectural approaches, and finally describing the 20 solutions being developed by the European 5GEx project. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on September 14, 2017. 39 Copyright Notice 41 Copyright (c) 2017 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 3. Background: the ETSI NFV 59 architecture . . . . . . . . . . . . . . . . . . . . . . . . 4 60 4. Multidomain problem statement . . . . . . . . . . . . . . . . 6 61 5. Multi-domain architectural approaches . . . . . . . . . . . . 7 62 5.1. ETSI NFV approaches . . . . . . . . . . . . . . . . . . . 8 63 5.2. Hierarchical . . . . . . . . . . . . . . . . . . . . . . 11 64 5.3. Cascading . . . . . . . . . . . . . . . . . . . . . . . . 13 65 6. Virtualization and Control for 66 Multi-Provider Multi-Domain . . . . . . . . . . . . . . . . . 13 67 6.1. Interworking interfaces . . . . . . . . . . . . . . . . . 15 68 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 69 8. Security Considerations . . . . . . . . . . . . . . . . . . . 16 70 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 16 71 10. Informative References . . . . . . . . . . . . . . . . . . . 16 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 74 1. Introduction 76 The telecommunications sector is experiencing a major revolution that 77 will shape the way networks and services are designed and deployed 78 for the next decade. We are witnessing an explosion in the number of 79 applications and services demanded by users, which are now really 80 capable of accessing them on the move. In order to cope with such a 81 demand, some network operators are looking at the cloud computing 82 paradigm, which enables a potential reduction of the overall costs by 83 outsourcing communication services from specific hardware in the 84 operator's core to server farms scattered in datacenters. These 85 services have different characteristics if compared with conventional 86 IT services that have to be taken into account in this cloudification 87 process. Also the transport network is affected in that it is 88 evolving to a more sophisticated form of IP architecture with trends 89 like separation of control and data plane traffic, and more fine- 90 grained forwarding of packets (beyond looking at the destination IP 91 address) in the network to fulfill new business and service goals. 93 Virtualization of functions also provides operators with tools to 94 deploy new services much faster, as compared to the traditional use 95 of monolithic and tightly integrated dedicated machinery. As a 96 natural next step, mobile network operators need to re-think how to 97 evolve their existing network infrastructures and how to deploy new 98 ones to address the challenges posed by the increasing customers' 99 demands, as well as by the huge competition among operators. All 100 these changes are triggering the need for a modification in the way 101 operators and infrastructure providers operate their networks, as 102 they need to significantly reduce the costs incurred in deploying a 103 new service and operating it. Some of the mechanisms that are being 104 considered and already adopted by operators include: sharing of 105 network infrastructure to reduce costs, virtualization of core 106 servers running in data centers as a way of supporting their load- 107 aware elastic dimensioning, and dynamic energy policies to reduce the 108 monthly electricity bill. However, this has proved to be tough to 109 put in practice, and not enough. Indeed, it is not easy to deploy 110 new mechanisms in a running operational network due to the high 111 dependency on proprietary (and sometime obscure) protocols and 112 interfaces, which are complex to manage and often require configuring 113 multiple devices in a decentralized way. 115 Network Function Virtualization (NFV) and Software Defined Networking 116 (SDN) are changing the way the telecommunications sector will deploy, 117 extend and operate their networks. TBD: add multi-domain. 119 2. Terminology 121 The following terms used in this document are defined by the ETSI NVF 122 ISG, and the ONF and the IETF: 124 NFV Infrastructure (NFVI): totality of all hardware and software 125 components which build up the environment in which VNFs are 126 deployed 128 NFV Management and Orchestration (NFV-MANO): functions 129 collectively provided by NFVO, VNFM, and VIM. 131 NFV Orchestrator (NFVO): functional block that manages the Network 132 Service (NS) lifecycle and coordinates the management of NS 133 lifecycle, VNF lifecycle (supported by the VNFM) and NFVI 134 resources (supported by the VIM) to ensure an optimized allocation 135 of the necessary resources and connectivity. 137 Network Service Orchestration (NSO): function responsible for the 138 Network Service lifecycle management, including operations such 139 as: On-board Network Service, Instantiate Network Service, Scale 140 Network Service, Update Network Service, etc. 142 OpenFlow protocol (OFP): allowing vendor independent programming 143 of control functions in network nodes. 145 Resource Orchestration (RO): subset of NFV Orchestrator functions 146 that are responsible for global resource management governance. 148 Service Function Chain (SFC): for a given service, the abstracted 149 view of the required service functions and the order in which they 150 are to be applied. This is somehow equivalent to the Network 151 Function Forwarding Graph (NF-FG) at ETSI. 153 Service Function Path (SFP): the selection of specific service 154 function instances on specific network nodes to form a service 155 graph through which an SFC is instantiated. 157 Virtualized Infrastructure Manager (VIM): functional block that is 158 responsible for controlling and managing the NFVI compute, storage 159 and network resources, usually within one operator's 160 Infrastructure Domain. 162 Virtualized Network Function (VNF): implementation of a Network 163 Function that can be deployed on a Network Function Virtualization 164 Infrastructure (NFVI). 166 Virtualized Network Function Manager (VNFM): functional block that 167 is responsible for the lifecycle management of VNF. 169 3. Background: the ETSI NFV architecture 171 The ETSI ISG NFV is a working group which, since 2012, aims to evolve 172 quasi-standard IT virtualization technology to consolidate many 173 network equipment types into industry standard high volume servers, 174 switches, and storage. It enables implementing network functions in 175 software that can run on a range of industry standard server hardware 176 and can be moved to, or loaded in, various locations in the network 177 as required, without the need to install new equipment. To date, 178 ETSI NFV is by far the most accepted NFV reference framework and 179 architectural footprint [etsi_nvf_whitepaper]. The ETSI NFV 180 framework architecture framework is composed of three domains 181 (Figure 1): 183 o Virtualized Network Function, running over the NFVI. 185 o NFV Infrastructure (NFVI), including the diversity of physical 186 resources and how these can be virtualized. NFVI supports the 187 execution of the VNFs. 189 o NFV Management and Orchestration, which covers the orchestration 190 and life-cycle management of physical and/or software resources 191 that support the infrastructure virtualization, and the life-cycle 192 management of VNFs. NFV Management and Orchestration focuses on 193 all virtualization specific management tasks necessary in the NFV 194 framework. 196 +-------------------------------------------+ +---------------+ 197 | Virtualized Network Functions (VNFs) | | | 198 | ------- ------- ------- ------- | | | 199 | | | | | | | | | | | | 200 | | VNF | | VNF | | VNF | | VNF | | | | 201 | | | | | | | | | | | | 202 | ------- ------- ------- ------- | | | 203 +-------------------------------------------+ | | 204 | | 205 +-------------------------------------------+ | | 206 | NFV Infrastructure (NFVI) | | NFV | 207 | ----------- ----------- ----------- | | Management | 208 | | Virtual | | Virtual | | Virtual | | | and | 209 | | Compute | | Storage | | Network | | | Orchestration | 210 | ----------- ----------- ----------- | | | 211 | +---------------------------------------+ | | | 212 | | Virtualization Layer | | | | 213 | +---------------------------------------+ | | | 214 | +---------------------------------------+ | | | 215 | | ----------- ----------- ----------- | | | | 216 | | | Compute | | Storage | | Network | | | | | 217 | | ----------- ----------- ----------- | | | | 218 | | Hardware resources | | | | 219 | +---------------------------------------+ | | | 220 +-------------------------------------------+ +---------------+ 222 Figure 1: ETSI NFV framework 224 The NFV architectural framework identifies functional blocks and the 225 main reference points between such blocks. Some of these are already 226 present in current deployments, whilst others might be necessary 227 additions in order to support the virtualization process and 228 consequent operation. The functional blocks are (Figure 2): 230 o Virtualized Network Function (VNF). 232 o Element Management (EM). 234 o NFV Infrastructure, including: Hardware and virtualized resources, 235 and Virtualization Layer. 237 o Virtualized Infrastructure Manager(s) (VIM). 239 o NFV Orchestrator. 241 o VNF Manager(s). 243 o Service, VNF and Infrastructure Description. 245 o Operations and Business Support Systems (OSS/BSS). 247 +--------------------+ 248 +-------------------------------------------+ | ---------------- | 249 | OSS/BSS | | | NFV | | 250 +-------------------------------------------+ | | Orchestrator +-- | 251 | ---+------------ | | 252 +-------------------------------------------+ | | | | 253 | --------- --------- --------- | | | | | 254 | | EM 1 | | EM 2 | | EM 3 | | | | | | 255 | ----+---- ----+---- ----+---- | | ---+---------- | | 256 | | | | |--|-| VNF | | | 257 | ----+---- ----+---- ----+---- | | | manager(s) | | | 258 | | VNF 1 | | VNF 2 | | VNF 3 | | | ---+---------- | | 259 | ----+---- ----+---- ----+---- | | | | | 260 +------|-------------|-------------|--------+ | | | | 261 | | | | | | | 262 +------+-------------+-------------+--------+ | | | | 263 | NFV Infrastructure (NFVI) | | | | | 264 | ----------- ----------- ----------- | | | | | 265 | | Virtual | | Virtual | | Virtual | | | | | | 266 | | Compute | | Storage | | Network | | | | | | 267 | ----------- ----------- ----------- | | ---+------ | | 268 | +---------------------------------------+ | | | | | | 269 | | Virtualization Layer | |--|-| VIM(s) +-------- | 270 | +---------------------------------------+ | | | | | 271 | +---------------------------------------+ | | ---------- | 272 | | ----------- ----------- ----------- | | | | 273 | | | Compute | | Storage | | Network | | | | | 274 | | | hardware| | hardware| | hardware| | | | | 275 | | ----------- ----------- ----------- | | | | 276 | | Hardware resources | | | NFV Management | 277 | +---------------------------------------+ | | and Orchestration | 278 +-------------------------------------------+ +--------------------+ 280 Figure 2: ETSI NFV reference architecture 282 4. Multidomain problem statement 284 Market fragmentation results from having a multitude of 285 telecommunications network and cloud operators each with a footprint 286 focused to a specific region. This makes it difficult to deploy cost 287 effective infrastructure services, such as virtual connectivity or 288 compute resources, spanning multiple countries as no single operator 289 has a big enough footprint. Even if operators largely aim to provide 290 the same infrastructure services (VPN connectivity, compute resources 291 based on virtual machines and block storage), inter-operator 292 collaboration tools for providing a service spanning several 293 administrative boundaries are very limited and cumbersome. This 294 makes service development and provisioning very time consuming. For 295 example, having a VPN with end-points in several countries, in order 296 to connect multiple sites of a business (such as a hotel chain), 297 requires contacting several network operators. Such an approach is 298 possible only with significant effort and integration work from the 299 side of the business. This is not only slow, but also inefficient 300 and expensive, since the business also needs to employ networking 301 specialists to do the integration instead of focusing on its core 302 business 304 Technology fragmentation also represents a major bottleneck 305 internally for an operator. Different networks and different parts 306 of a network may be built as different domains using separate 307 technologies, such as optical or packet switched (with different 308 packet switching paradigms included); having equipment from different 309 vendors; having different control paradigms, etc. Managing and 310 integrating these separate technology domains requires substantial 311 amount of effort, expertise, and time. The associated costs are paid 312 by both network operators and vendors alike, who need to design 313 equipment and develop complex integration features. In addition to 314 technology domains, there are other reasons for having multiple 315 domains within an operator, such as, different geographies, different 316 performance characteristics, scalability, policy or simply historic 317 (e.g., result of a merge or an acquisition). Multiple domains in a 318 network are a necessary and permanent feature however, these should 319 not be a roadblock towards service development and provisioning, 320 which should be fast and efficient. 322 A solution is needed to deal with both the multi-operator 323 collaboration issue, and address the multi-domain problem within a 324 single network operator. While these two problems are quite 325 different, they also share a lot of common aspects and can benefit 326 from having a number of common tools to solve them. 328 5. Multi-domain architectural approaches 330 This section summarizes different architectural options that can be 331 considered to tackle the multi-domain orchestration problem. 333 5.1. ETSI NFV approaches 335 Recently, the ETSI NFV ISG has started to look into viable 336 architectural options supporting the placement of functions in 337 different administrative domains. In the document [etsi_nvf_ifa009], 338 different approaches are considered, which we summarize next. 340 The first option (shown in Figure 3) is based on a split of the NFVO 341 into Network Service Orchestrator (NSO) and Resource Orchestrator 342 (RO). A use case that this separation could enable is the following: 343 a network operator offering its infrastructure to different 344 departments within the same operator, as well as to a different 345 network operator like in cases of network sharing agreements. In 346 this scenario, an administrative domain can be defined as one or more 347 data centers and VIMs, providing an abstracted view of the resources 348 hosted in it. 350 A service is orchestrated out of VNFs that can run on infrastructure 351 provided and managed by another Service Provider. The NSO manages 352 the lifecycle of network services, while the RO provides an overall 353 view of the resources present in the administrative domain to which 354 it provides access and hides the interfaces of the VIMs present below 355 it. 357 ------- 358 | NSO | 359 /-------\ 360 / \ 361 -------- / -------- \ -------- 362 | VNFM | | | VNFM | | | VNFM | 363 -------- / -------- \ -------- 364 / ____/ / \ \____ \ 365 / / _________/ \_________ \ \ 366 / / / \ \ \ 367 +-----------/-/-/---------+ +----------\-\-\----------+ 368 | --------- | | --------- | 369 | | RO | | | | RO | | 370 | --------- | | --------- | 371 | / | \ | | / | \ | 372 | / | \ | | / | \ | 373 | / | \ | | / | \ | 374 | ------- ------- ------- | | ------- ------- ------- | 375 | |VIM 1| |VIM 2| |VIM 3| | | |VIM 1| |VIM 2| |VIM 3| | 376 | ------- ------- ------- | | ------- ------- ------- | 377 | Administrative domain A | | Administrative domain B | 378 +-------------------------+ +-------------------------+ 380 Figure 3: Infrastructure provided using multiple administrative 381 domains (from ETSI GS NFV-IFA 009 V1.1.1) 383 The second option (shown in Figure 4) is based on having an umbrella 384 NFVO. A use case enabled by this is the following: a Network 385 Operator offers Network Services to different departments within the 386 same operator, as well as to a different network operator like in 387 cases of network sharing agreements. In this scenario, an 388 administrative domain is compose of one or more Datacentres, VIMs, 389 VNFMs (together with their related VNFs) and NFVO, allowing distinct 390 specific sets of network services to be hosted and offered on each. 392 A top Network Service can include another Network Service. A Network 393 Service containing other Network Services might also contain VNFs. 394 The NFVO in each admin domain provides visibility of the Network 395 Services specific to this admin domain. The umbrella NFVO is 396 providing the lifecycle management of umbrella network services 397 defined in this NFVO. In each admin domain, the NFVO is providing 398 standard NFVO functionalities, with a scope limited to the network 399 services, VNFs and resources that are part of its admin domain. 401 ------------ 402 | Umbrella | 403 | NFVO | 404 ------------ 405 / | \ 406 / | \ 407 / -------- \ 408 / | VNFM | \ 409 / -------- \ 410 / | \ 411 / ------- \ 412 / |VIM 1| \ 413 / ------- \ 414 --------------/------------ -------------\------------- 415 | -------- | | -------- | 416 | | NFVO | | | | NFVO | | 417 | -------- | | -------- | 418 | | | | | | | | | | 419 | -------- | | | -------- | | -------- | | | -------- | 420 | | VNFM | | | | | VNFM | | | | VNFM | | | | | VNFM | | 421 | -------- | | | -------- | | -------- | | | -------- | 422 | | \__/__|__\_/_ | | | | \__/__|__\_/_ | | 423 | | __/___|___/\ \ | | | | __/___|___/\ \ | | 424 | | / / | \ \ | | | | / / | \ \ | | 425 | ------- ------- ------- | | ------- ------- ------- | 426 | |VIM 1| |VIM 2| |VIM 3| | | |VIM 1| |VIM 2| |VIM 3| | 427 | ------- ------- ------- | | ------- ------- ------- | 428 | Administrative domain A | | Administrative domain B | 429 +-------------------------+ +-------------------------+ 431 Figure 4: Network services provided using multiple administrative 432 domains (from ETSI GS NFV-IFA 009 V1.1.1) 434 More recently, ETSI NFV has released a new whitepaper, titled 435 "Network Operator Perspectives on NFV priorities for 5G" 436 [etsi_nvf_whitepaper_5g], which provides network operator 437 perspectives on NFV priorities for 5G and identifies common technical 438 features in terms of NFV. This whitepaper identifies multi-site/ 439 multi-tenant orchestration as one key priority. ETSI highlights the 440 support of Infrastructure as a Service (IaaS), NFV as a Service 441 (NFVaaS) and Network Service (NS) composition in different 442 administrative domains (for example roaming scenarios in wireless 443 networks) as critical for the 5G work. 445 Related to this, a new Work Item, IFA028, and titled as "Report on 446 architecture options to support multiple administrative domains" has 447 been approved. 449 5.2. Hierarchical 451 Considering the potential split of the NFVO into a Network Service 452 Orchestrator (NSO) and a Resource Orchestrator (RO), multi-provider 453 hierarchical interfaces may exist at their northbound APIs. Figure 5 454 illustrates the various interconnection options, namely: 456 E/NSO (External NSO): an evolved NFVO northbound API based on 457 Network Service (NS). 459 E/RO (External RO): VNF-FG oriented resource embedding service. A 460 received VNF-FG that is mapped to the northbound resource view is 461 embedded into the distributed resources collected from southbound, 462 i.e., VNF-FG_in = VNF-FG_out_1 + VNF-FG_out_2 + ... + VNF- 463 FG_out_N, where VNF-FG_out_j corresponds to a spatial embedding to 464 subordinate domain "j". For example, Provider 3's MP-NFVO/RO 465 creates VNF-FG corresponding to its E/RO and E/VIM sub-domains. 467 E/VIM (External VIM): a generic VIM interface offered to an 468 external consumer. In this case the NFVI-PoP may be shared for 469 multiple consumers, each seeing a dedicated NFVI-PoP. This 470 corresponds to IaaS interface. 472 I/NSO (Internal NSO): if a Multi-provider NSO (MP-NSO) is 473 separated from the provider's operational NSO, e.g., due to 474 different operational policies, the MP-NSO may need this interface 475 to realize its northbound E/NSO requests. Provider 1 illustrates 476 a scenario the MP-NSO and the NSO are logically separated. 477 Observe that Provider 1's tenants connect to the NSO and MP-NSO 478 corresponds to "wholesale" services. 480 I/RO (Internal RO): VNF-FG oriented resource embedding service. A 481 received VNF-FG that is mapped to the northbound resource view is 482 embedded into the distributed resources collected from southbound, 483 i.e., VNF-FG_in = VNF-FG_out_1 + VNF-FG_out_2 + ... + VNF- 484 FG_out_N, where VNF-FG_out_j corresponds to a spatial embedding to 485 subordinate domain "j". For example, Provider 1's MP-NFVO/RO 486 creates VNF-FG corresponding to its I/RO and I/VIM sub-domains. 488 I/VIM (Internal VIM): a generic VIM interface at an NFVI-PoP. 490 Nfvo-Vim: a generic VIM interface between a (monolithic) NFVO and 491 a VIM. 493 We would like to explore use-cases and potential benefits for the 494 above multi-provider interfaces as well as to learn how much they may 495 differ from their existing counterparts. For example, are (E/RO, I/ 496 RO), (E/NSO, I/NSO), (E/VIM, I/VIM) pairs different? 497 Tenants 498 * Provider | 499 * * Domain 4 +--+-----------+ 500 * * |MP-NFVO/NSO: | 501 * * |Network Serv. | 502 * Provider * |Orchestrator | 503 * Domain 3 * +--+-----------+ 504 * Tenants * |E/RO 505 * | ************|************* 506 * ++-------------+ | 507 * |MP-NFVO/NSO: | | 508 Provider * |Network Serv. | | 509 Domain 1 * |Orchestrator | | 510 * +-+-----+------+ | 511 * E/NSO| | I/RO / 512 *.---------' +-+---------+--+ 513 /* |MP-NFVO/RO: | 514 / * |Rersource | 515 Tenants / * |Orchestrator | 516 | | * +--+---+-------+ 517 | +-----------+--+ *************|***|******************** 518 | |MP-NFVO/NSO: | | * \ Provider 519 | |Network Serv. | E/RO / * \ E/VIM Domain 2 520 | |Orchestrator | .-----------' * `-------. 521 | +-+------+-----+ | * | 522 | |I/NSO |I/RO | * | 523 | | +--+--------+--+ * | 524 | | |MP-NFVO/RO: | * | 525 | | |Rersource | * | 526 \ | |Orchestrator | * +------+-------+ 527 \ | +----+---- --+-+ * |VIM: | 528 +--+-----+ |I/RO |I/VIM * |Virtualized | 529 |NFVO/NSO| | | * |Pys mapping | 530 +------+-+ | | * +--------------+ 531 I/RO| | | * 532 +------+----+---+ | * 533 | NFVO/RO | | * 534 ++-------------++ | * 535 |Nfvo-Vim | | * 536 ++-------+ ++----+--+ * 537 |WIM|VIM || |VIM|WIM | * 538 +--------+| +--------+ * 539 +--------+ * 541 Figure 5: NSO-RO Split: possible multi-provider APIs - an 542 illustration 544 5.3. Cascading 546 Cascading is an alternative way of relationship among providers, from 547 the network service point of view. In this case, service 548 decomposition is implemented in a paired basis. This can be extended 549 in a recursive manner, then allowing for a concatenation of cascaded 550 relations between providers. 552 As a complement to this, from a service perspective, the cascading of 553 two remote providers (i.e., providers not directly interconnected) 554 could require the participation of a third provider (or more) 555 facilitating the necessary communication among the other two. In 556 that sense, the final service involves two providers while the 557 connectivity imposes the participation of more parties at resource 558 level. 560 6. Virtualization and Control for Multi-Provider Multi-Domain 562 Orchestration operation in multi-domain is somewhat different from 563 that in a single domain as the assumption in single domain single 564 provider orchestration is that the orchestrator is aware of the 565 entire topology and resource availability within its domain as well 566 as has complete control over those resources. This assumption of 567 technical control cannot be made in a multi domain scenario, 568 furthermore the assumption of the knowledge of the resources and 569 topologies cannot be made across providers. In such a scenario 570 solutions are required that enable the exchange of relevant 571 information across these orchestrators. This exchange needs to be 572 standardized as shown in Figure 6. 574 | | 575 + IF1 + 576 _____|____ ____|_____ 577 | Multi | IF2 | Multi | 578 | Provider |<--------+---------->| Provider | 579 |___Orch___| |___Orch___| 580 /\ /\ 581 / \ / \ 582 / \ IF3 / \ 583 _______/__ _\_________ ________/_ _\________ 584 | Domain | | Domain | | Domain | | Domain | 585 |___Orch___| |___Orch___| |___Orch___| |___Orch___| 587 Figure 6: Multi Domain Multi Provider reference architecture 589 The figure shows the Multi Provider orchestrator exposing an 590 interface 1 (IF1) to the tenant, interface 2 (IF2) to other Multi 591 Provider Orchestrator (MPO) and an interface 3 (IF3) to individual 592 domain orchestratrators. Each one of these interfaces could be a 593 possible standardization candidate. Interface 1 is exposed to the 594 tennnt who could request his specific services and/or slices to be 595 deployed. Interface 2 is between the orchestrator and is a key 596 interface to enable multi-provider operation. Interface 3 focuses on 597 abstracting the technology or vendor dependent implementation details 598 to support orchestration. 600 The proposed operation of the MPO follows three main technical steps. 601 First, over interface 2 various functions such as abstracted topology 602 discovery, pricing and service details are detected. Second, once a 603 request for deploying a service is received over interface 1 the 604 Multi Provider Orchestrator evaluates the best orchestrators to 605 implement parts of this request. The request to deploy these parts 606 are sent to the different domain orchestrators over IF2 and IF3 and 607 the acknowledgement that these are deployed in different domain are 608 received back over those interfaces. Third, on receipt of the 609 acknowledgement the slice specific assurance management is started 610 within the MPO. This assurance function collects the appropriate 611 information over IF2 and IF3 and reports the performance back to the 612 tenant over IF1. The assurance is also responsible for detecting any 613 failures in the service and violations in the SLA and recomending to 614 the orchestration engine the reconfiguration of the service or slice 615 which again needs to performed over IF2 and IF3. 617 Each of the three steps is assigned to a specific block in our high 618 level architecture shown in Figure 7. 620 | | 621 + IF1 + 622 ______________|______________ ____|_____ 623 | Multi Provider Orch | | Multi | 624 | ______ ________ _______ |<------+------->| Provider | 625 ||Assur-| | | | Catal-|| IF2 |___Orch___| 626 ||-ance | | NFVO | | logue || 627 || Mgmt.| | | | Topo. || 628 ||______| |________| |_Mgmt._|| 629 |_____________________________| 630 /\ 631 / \ IF3 633 Figure 7: Detailed MPO reference architecture 635 The catalogue and topology management system is responsible for step 636 1. It discovers the service as well as the resources exposed by the 637 other domains both on IF2 and IF3. The combination of these services 638 with coverage over the detected topology is provided to the user over 639 IF1. In turn the catalogue and topology management system is also 640 responsible for exposing the topology and service deployment 641 capabilities to the other domain. The exposure over interface 2 to 642 other MPO maybe abstracted and the mapping of this abstracted view to 643 the real view when requested byt he NFVO. 645 The NFVO (Network Function Virtualization Orchestrator) is 646 responsible for the second step. It deploys the service or slice as 647 is received from the tenant over IF2 and IF3. It then hands over the 648 deployment decisions to the Assurance managmeent subsystem which use 649 this information to collect the periodic monitoring tickets in step 650 3. On the other end it is responsible for receiving the request over 651 IF2 to deploy a part of the service, consult with the catalogue and 652 topology management system on the translation of the abstraction to 653 the received request and then for the actual deployment over the 654 domains using IF3. The result of this deployment and the management 655 and control handles to access the deployed slice or service is then 656 returned to the requesting MPO. 658 The assurance management component periodically studies the collected 659 results to report the overall service performance to the tenant or 660 the requesting MPO as well as to ensure that the service is 661 functioning within the specified parameters. In case of failures or 662 violations the Assurance management system recomends reconfigurations 663 to the NFVO. 665 6.1. Interworking interfaces 667 In this section we provide more details on the interworking 668 interfaces of the MPO reference architecture. Each interface IF1, 669 IF2 and IF3 is broken down into several sub-interfaces. Each of them 670 has a clear scope and functionality. [Ed. note: more details will 671 be added in future releases of this document] 673 For multi provider Network Service orchestration, the Multi-domain 674 Orchestrator (MdO) offers Network Services by exposing an OSS/BSS - 675 NFVO interface to other MPOs belonging to other providers. For 676 multi-provider resource orchestration, the MPO presents a VIM-like 677 view and exposes an extended NFVO - VIM interface to other MPOs. The 678 MPO exposes a northbound sub-interface (IF1-S) through which an MPO 679 customer sends the initial request for services. It handles command 680 and control functions to instantiate network services. Such 681 functions include requesting the instantiation and interconnection of 682 Network Functions (NFs). A sub-interface IF2-S is defined to perform 683 similar operations between MPOs of different administrative domains. 684 A set of sub-interfaces -- IF3-R and IF2-R -- are used to keep an 685 updated global view of the underlying infrastructure topology exposed 686 by domain orchestrators. The service catalogue exposes available 687 services to customers on a sub-interface IF1-C and to other MPO 688 service operators on sub-interface IF2-C. Resource orchestration 689 related interfaces are broken up to IF2-RC, IF2-RT, IF2-RMon to 690 reflect resource control, resource topology and resource monitoring 691 respectively. Furthermore, the sub-interfaces introduced before are 692 generalised and also used for interfaces IF3 and IF1. 694 7. IANA Considerations 696 N/A. 698 8. Security Considerations 700 TBD. 702 9. Acknowledgments 704 This work is supported by 5G-PPP 5GEx, an innovation action project 705 partially funded by the European Community under the H2020 Program 706 (grant agreement no. 671636). The views expressed here are those of 707 the authors only. The European Commission is not liable for any use 708 that may be made of the information in this presentation. 710 10. Informative References 712 [etsi_nvf_ifa009] 713 "Report on Architectural Options, ETSI GS NFV-IFA 009 714 V1.1.1", July 2016. 716 [etsi_nvf_whitepaper] 717 "Network Functions Virtualisation (NFV). White Paper 2", 718 October 2014. 720 [etsi_nvf_whitepaper_5g] 721 "Network Functions Virtualisation (NFV). White Paper on 722 "Network Operator Perspectives on NFV priorities for 5G"", 723 February 2017. 725 Authors' Addresses 727 Carlos J. Bernardos 728 Universidad Carlos III de Madrid 729 Av. Universidad, 30 730 Leganes, Madrid 28911 731 Spain 733 Phone: +34 91624 6236 734 Email: cjbc@it.uc3m.es 735 URI: http://www.it.uc3m.es/cjbc/ 736 Luis M. Contreras 737 Telefonica I+D 738 Ronda de la Comunicacion, S/N 739 Madrid 28050 740 Spain 742 Email: luismiguel.conterasmurillo@telefonica.com 744 Ishan Vaishnavi 745 Huawei Technologies Dusseldorf GmBH 746 Riesstrasse 25, 747 Munich 80992 748 Germany 750 Email: Ishan.vaishnavi@huawei.com 752 Robert Szabo 753 Ericsson 754 Konyves Kaman krt. 11 755 Budapest, EMEA 1097 756 Hungary 758 Phone: +36703135738 759 Email: robert.szabo@ericsson.com