idnits 2.17.1 draft-irtf-nfvrg-unify-recursive-programming-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 21 instances of too long lines in the document, the longest one being 148 characters in excess of 72. ** The abstract seems to contain references ([I-D.unify-nfvrg-challenges]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 567 has weird spacing: '...storage str...' == Line 619 has weird spacing: '...storage str...' == Line 672 has weird spacing: '...storage str...' -- The document date (March 21, 2016) is 2959 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-06) exists of draft-ietf-sfc-dc-use-cases-04 == Outdated reference: A later version (-04) exists of draft-unify-nfvrg-challenges-03 Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NFVRG R. Szabo, Ed. 3 Internet-Draft Z. Qiang 4 Intended status: Informational Ericsson 5 Expires: September 22, 2016 M. Kind 6 Deutsche Telekom AG 7 March 21, 2016 9 Recursive virtualization and programming for network and cloud resources 10 draft-irtf-nfvrg-unify-recursive-programming-00 12 Abstract 14 The introduction of Network Function Virtualization (NFV) in carrier- 15 grade networks promises improved operations in terms of flexibility, 16 efficiency, and manageability. NFV is an approach to combine network 17 and compute virtualizations together. However, network and compute 18 resource domains expose different virtualizations and programmable 19 interfaces. In [I-D.unify-nfvrg-challenges] we argued for a joint 20 compute and network virtualization by looking into different compute 21 abstractions. 23 In this document we analyze different approaches to orchestrate a 24 service graph with transparent network functions relying on a public 25 telecommunication network and ending in a commodity data center. We 26 show that a recursive compute and network joint virtualization and 27 programming has clear advantages compared to other approaches with 28 separated control between compute and network resources. In 29 addition, the joint virtualization will have cost and performance 30 advantages by removing additional virtualization overhead. The 31 discussion of the problems and the proposed solution is generic for 32 any data center use case; however, we use NFV as an example. 34 Status of This Memo 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. 39 Internet-Drafts are working documents of the Internet Engineering 40 Task Force (IETF). Note that other groups may also distribute 41 working documents as Internet-Drafts. The list of current Internet- 42 Drafts is at http://datatracker.ietf.org/drafts/current/. 44 Internet-Drafts are draft documents valid for a maximum of six months 45 and may be updated, replaced, or obsoleted by other documents at any 46 time. It is inappropriate to use Internet-Drafts as reference 47 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on September 22, 2016. 50 Copyright Notice 52 Copyright (c) 2016 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 68 2. Terms and Definitions . . . . . . . . . . . . . . . . . . . . 3 69 3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 3.1. Black Box DC . . . . . . . . . . . . . . . . . . . . . . . 4 71 3.1.1. Black Box DC with L3 tunnels . . . . . . . . . . . . . . 5 72 3.1.2. Black Box DC with external steering . . . . . . . . . . . 6 73 3.2. White Box DC . . . . . . . . . . . . . . . . . . . . . . . 8 74 3.3. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 9 75 4. Recursive approach . . . . . . . . . . . . . . . . . . . . . 10 76 4.1. Virtualization . . . . . . . . . . . . . . . . . . . . . . 11 77 4.1.1. The virtualizer's data model . . . . . . . . . . . . . . 13 78 5. Relation to ETSI NFV . . . . . . . . . . . . . . . . . . . . 24 79 5.1. Policy based resource management . . . . . . . . . . . . . 27 80 6. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 29 81 6.1. Infrastructure reports . . . . . . . . . . . . . . . . . . 29 82 6.2. Simple requests . . . . . . . . . . . . . . . . . . . . . . 35 83 7. Experimentations . . . . . . . . . . . . . . . . . . . . . . 37 84 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38 85 9. Security Considerations . . . . . . . . . . . . . . . . . . . 38 86 10. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 38 87 11. Informative References . . . . . . . . . . . . . . . . . . . 38 88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 39 90 1. Introduction 92 To a large degree there is agreement in the research community that 93 rigid network control limits the flexibility of service creation. In 94 [I-D.unify-nfvrg-challenges] 95 o we analyzed different compute domain abstractions to argue that 96 joint compute and network virtualization and programming is needed 97 for efficient combination of these resource domains; 99 o we described challenges associated with the combined handling of 100 compute and network resources for a unified production 101 environment. 103 Our goal here is to analyze different approaches to instantiate a 104 service graph with transparent network functions into a commodity 105 Data Center (DC). More specifically, we analyze 107 o two black box DC set-ups, where the intra-DC network control is 108 limited to some generic compute only control programming 109 interface; 111 o a white box DC set-up, where the intra-DC network control is 112 exposed directly to for a DC external control to coordinate 113 forwarding configurations; 115 o a recursive approach, which illustrates potential benefits of a 116 joint compute and network virtualization and control. 118 The discussion of the problems and the proposed solution is generic 119 for any data center use case; however, we use NFV as an example. 121 2. Terms and Definitions 123 We use the terms compute and "compute and storage" interchangeably 124 throughout the document. Moreover, we use the following definitions, 125 as established in [ETSI-NFV-Arch]: 127 NFV: Network Function Virtualization - The principle of separating 128 network functions from the hardware they run on by using virtual 129 hardware abstraction. 131 NFVI: NFV Infrastructure - Any combination of virtualized compute, 132 storage and network resources. 134 VNF: Virtualized Network Function - a software-based network 135 function. 137 MANO: Management and Orchestration - In the ETSI NFV framework 138 [ETSI-NFV-MANO], this is the global entity responsible for 139 management and orchestration of NFV lifecycle. 141 Further, we make use of the following terms: 143 NF: a network function, either software-based (VNF) or appliance- 144 based. 146 SW: a (routing/switching) network element with a programmable 147 control plane interface. 149 DC: a data center is an interconnection of Compute Nodes (see below) 150 with a data center controller, which offers programmatic resource 151 control interface to its clients. 153 CN: a server, which is controlled by a DC control plane and provides 154 execution environment for virtual machine (VM) images such as 155 VNFs. 157 3. Use Cases 159 Service Function Chaining (SFC) looks into the problem how to deliver 160 end-to-end services through the chain of network functions (NFs). 161 Many of such NFs are envisioned to be transparent to the client, 162 i.e., they intercept the client connection for adding value to the 163 services without the knowledge of the client. However, deploying 164 network function chains in DCs with Virtualized Network Functions 165 (VNFs) are far from trivial [I-D.ietf-sfc-dc-use-cases]. For 166 example, different exposures of the internals of the DC will imply 167 different dynamisms in operations, different orchestration 168 complexities and may yield for different business cases with regards 169 to infrastructure sharing. 171 We investigate different scenarios with a simple NF forwarding graph 172 of three VNFs (o->VNF1->VNF2->VNF3->o), where all VNFs are deployed 173 within the same DC. We assume that the DC is a multi-tier leaf and 174 spine (CLOS) and that all VNFs of the forwarding graph are bump-in- 175 the-wire NFs, i.e., the client cannot explicitly access them. 177 3.1. Black Box DC 179 In Black Bock DC set-ups, we assume that the compute domain is an 180 autonomous domain with legacy (e.g., OpenStack) orchestration APIs. 181 Due to the lack of direct forwarding control within the DC, no native 182 L2 forwarding can be used to insert VNFs running in the DC into the 183 forwarding graph. Instead, explicit tunnels (e.g., VxLAN) must be 184 used, which need termination support within the deployed VNFs. 185 Therefore, VNFs must be aware of the previous and the next hops of 186 the forwarding graph to receive and forward packets accordingly. 188 3.1.1. Black Box DC with L3 tunnels 190 Figure 1 illustrates a set-up where an external VxLAN termination 191 point in the SDN domain is used to forward packets to the first NF 192 (VNF1) of the chain within the DC. VNF1, in turn, is configured to 193 forward packets to the next SF (VNF2) in the chain and so forth with 194 VNF2 and VNF3. 196 In this set-up VNFs must be capable of handling L3 tunnels (e.g., 197 VxLAN) and must act as forwarders themselves. Additionally, an 198 operational L3 underlay must be present so that VNFs can address each 199 other. 201 Furthermore, VNFs holding chain forwarding information could be 202 untrusted user plane functions from 3rd party developers. 203 Enforcement of proper forwarding is problematic. 205 Additionally, compute only orchestration might result in sub-optimal 206 allocation of the VNFs with regards to the forwarding overlay, for 207 example, see back-forth use of a core switch in Figure 1. 209 In [I-D.unify-nfvrg-challenges] we also pointed out that within a 210 single Compute Node (CN) similar VNF placement and overlay 211 optimization problem may reappear in the context of network interface 212 cards and CPU cores. 214 | A A 215 +---+ | S | 216 |SW1| | D | 217 +---+ | N | P 218 / \ V | H 219 / \ | Y 220 | | A | S 221 +---+ +-+-+ | | I 222 |SW | |SW | | | C 223 ,+--++.._ _+-+-+ | | A 224 ,-" _|,,`.""-..+ | C | L 225 _,,,--"" | `. |""-.._ | L | 226 +---+ +--++ `+-+-+ ""+---+ | O | 227 |SW | |SW | |SW | |SW | | U | 228 +---+ ,'+---+ ,'+---+ ,'+---+ | D | 229 | | ,-" | | ,-" | | ,-" | | | | 230 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ | | 231 |CN| |CN| |CN| |CN| |CN| |CN| |CN| |CN| | | 232 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ V V 233 | | | 234 +-+ +-+ +-+ A 235 |V| |V| |V| | L 236 |N| |N| |N| | O 237 |F| |F| |F| | G 238 |1| |3| |2| | I 239 +-+ +-+ +-+ | C 240 +---+ --1>-+ | | +--<3---------------<3---+ | | A 241 |SW1| +-2>-----------------------------2>---+ | L 242 +---+ <4--------------+ V 244 <<=============================================>> 245 IP tunnels, e.g., VxLAN 247 Figure 1: Black Box Data Center with VNF Overlay 249 3.1.2. Black Box DC with external steering 251 Figure 2 illustrates a set-up where an external VxLAN termination 252 point in the SDN domain is used to forward packets among all the SFs 253 (VNF1-VNF3) of the chain within the DC. VNFs in the DC need to be 254 configured to receive and send packets between only the SDN endpoint, 255 hence are not aware of the next hop VNF address. Shall any VNFs need 256 to be relocated, e.g., due to scale in/out as described in 257 [I-D.zu-nfvrg-elasticity-vnf], the forwarding overlay can be 258 transparently re-configured at the SDN domain. 260 Note however, that traffic between the DC internal SFs (VNF1, VNF2, 261 VNF3) need to exit and re-enter the DC through the external SDN 262 switch. This, certainly, is sub-optimal an results in ping-pong 263 traffic similar to the local and remote DC case discussed in 264 [I-D.zu-nfvrg-elasticity-vnf]. 266 | A A 267 +---+ | S | 268 |SW1| | D | 269 +---+ | N | P 270 / \ V | H 271 / \ | Y 272 | | ext port A | S 273 +---+ +-+-+ | | I 274 |SW | |SW | | | C 275 ,+--++.._ _+-+-+ | | A 276 ,-" _|,,`.""-..+ | C | L 277 _,,,--"" | `. |""-.._ | L | 278 +---+ +--++ `+-+-+ ""+---+ | O | 279 |SW | |SW | |SW | |SW | | U | 280 +---+ ,'+---+ ,'+---+ ,'+---+ | D | 281 | | ,-" | | ,-" | | ,-" | | | | 282 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ | | 283 |CN| |CN| |CN| |CN| |CN| |CN| |CN| |CN| | | 284 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ V V 285 | | | 286 +-+ +-+ +-+ A 287 |V| |V| |V| | L 288 |N| |N| |N| | O 289 |F| |F| |F| | G 290 |1| |3| |2| | I 291 +-+ +-+ +-+ | C 292 +---+ --1>-+ | | | | | | A 293 |SW1| <2-----+ | | | | | L 294 | | --3>---------------------------------------+ | | 295 | | <4-------------------------------------------+ | 296 | | --5>------------+ | | 297 +---+ <6----------------+ V 299 <<=============================================>> 300 IP tunnels, e.g., VxLAN 302 Figure 2: Black Box Data Center with ext Overlay 304 3.2. White Box DC 306 Figure 3 illustrates a set-up where the internal network of the DC is 307 exposed in full details through an SDN Controller for steering 308 control. We assume that native L2 forwarding can be applied all 309 through the DC until the VNFs' port, hence IP tunneling and tunnel 310 termination at the VNFs are not needed. Therefore, VNFs need not be 311 forwarding graph aware but transparently receive and forward packets. 312 However, the implications are that the network control of the DC must 313 be handed over to an external forwarding controller (see that the SDN 314 domain and the DC domain overlaps in Figure 3). This most probably 315 prohibits clear operational separation or separate ownerships of the 316 two domains. 318 | A A 319 +---+ | S | 320 |SW1| | D | 321 +---+ | N | P 322 / \ | | H 323 / \ | | Y 324 | | ext port | A | S 325 +---+ +-+-+ | | | I 326 |SW | |SW | | | | C 327 ,+--++.._ _+-+-+ | | | A 328 ,-" _|,,`.""-..+ | | C | L 329 _,,,--"" | `. |""-.._ | | L | 330 +---+ +--++ `+-+-+ ""+---+ | | O | 331 |SW | |SW | |SW | |SW | | | U | 332 +---+ ,'+---+ ,'+---+ ,'+---+ V | D | 333 | | ,-" | | ,-" | | ,-" | | | | 334 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ | | 335 |CN| |CN| |CN| |CN| |CN| |CN| |CN| |CN| | | 336 +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+ V V 337 | | | 338 +-+ +-+ +-+ A 339 |V| |V| |V| | L 340 |N| |N| |N| | O 341 |F| |F| |F| | G 342 |1| |3| |2| | I 343 +-+ +-+ +-+ | C 344 +---+ --1>-+ | | +--<3---------------<3---+ | | A 345 |SW1| +-2>-----------------------------2>---+ | L 346 +---+ <4--------------+ V 348 <<=============================================>> 349 L2 overlay 351 Figure 3: White Box Data Center with L2 Overlay 353 3.3. Conclusions 355 We have shown that the different solutions imply different operation 356 and management actions. From network operations point of view, it is 357 not desirable to run and manage similar functions several times (L3 358 blackbox DC case) - especially if the networking overlay can be 359 easily managed upfront by using a programmatic interface, like with 360 the external steering in black and whitebox DC scenarios. 362 4. Recursive approach 364 We argued in [I-D.unify-nfvrg-challenges] and 365 [I-D.caszpe-nfvrg-orchestration-challenges] for a joint software and 366 network programming interface. Consider that such joint software and 367 network abstraction (virtualization) exists around the DC with a 368 corresponding resource programmatic interface. A software and 369 network programming interface could include VNF requests and the 370 definition of the corresponding network overlay. However, such 371 programming interface is similar to the top level services 372 definition, for example, by the means of a VNF Forwarding Graph. 374 Figure 4 illustrates a joint domain virtualization and programming 375 setup. In Figure 4 "[x]" denotes ports of the virtualized data plane 376 while "x" denotes port created dynamically as part of the VNF 377 deployment request. Over the joint software and network 378 virtualization VNF placement and the corresponding traffic steering 379 could be defined in an atomic, which is orchestrated, split and 380 handled to the next levels (see Figure 5) in the hierarchy for 381 further orchestration. Such setup allows clear operational 382 separation, arbitrary domain virtualization (e.g., topology details 383 could be omitted) and constraint based optimization of domain wide 384 resources. 386 | 387 +-----------------------[x]--------------------+ A 388 |Domain 0 | | |O 389 | +--------[x]----------+ | |V 390 | | / \ | | |E 391 |Big Switch | -<--- --->-- | | |R 392 |with | / BiS-BiS \ | | |A 393 |Big Software | | +-->-+ +-->-+ | | | |R 394 |(BiS-BiS) | | | | | | | | | |C 395 | +--x-x----x-x----x-x--+ | |H 396 | | | | | | | | |I 397 | +-+ +-+ +-+ | |N 398 | |V| |V| |V| | |G V 399 | |N| |N| |N| | | N 400 | |F| |F| |F| | | F 401 | |1| |2| |3| | | 402 | +-+ +-+ +-+ | | F 403 | | | G 404 +----------------------------------------------+ V 406 Figure 4: Recursive Domain Virtualization and Joint VNF FG 407 programming: Overarching View 409 +-------------------------|-----------------------------+ A 410 | +----------------------[x]---------------------+ AV | | 411 | | Domain 1 / \ | |N | | 412 | | | A | |F | | 413 | | Big Switch (BS) | | | | | |O 414 | | V | | |F | |V 415 | | / \ | |G | |E 416 | +-----------------[x]--------[x]---------------+ V1 | |R 417 | | | | |A 418 | +------------------|----------|----------------+ A | |R 419 | |Domain 2 | A | | | |C 420 | | V | | | | |H 421 | | +---[x]--------[x]----+ | |V | |I 422 | |Big Switch | / BiS-BiS \ | | |N | |N 423 | |with | / \ | | |F | |G 424 | |Big Software | | +-->-+ +-->-+ | | | | | | 425 | |(BiS-BiS) | | | | | | | | | |F | |V 426 | | +--x-x----x-x----x-x--+ | |G | |N 427 | | | | | | | | | |2 | |F 428 | | +-+ +-+ +-+ | | | | 429 | | |V| |V| |V| | | | |F 430 | | |N| |N| |N| | | | |G 431 | | |F| |F| |F| | | | | 432 | | |1| |2| |3| | | | | 433 | | +-+ +-+ +-+ | | | | 434 | +----------------------------------------------+ V | | 435 +-------------------------------------------------------+ V 437 Figure 5: Recursive Domain Virtualization and Joint VNF FG 438 programming: Domain Views 440 4.1. Virtualization 442 Let us first define the joint software and network abstraction 443 (virtualization) as a Big Switch with Big Software (BiS-BiS). A BiS- 444 BiS is a node abstraction, which incorporates both software and 445 networking resources with an associated joint software and network 446 control API (see Figure 6). 448 API o __ 449 | \ 450 Software Ctrler \ 451 API O-------------+ \ \ 452 | \ \ 453 Compute Ctrler \ | 454 | \ | 455 | +---------------------+ | 456 | | | | Joint Software & 457 | | {vCPU | | Network Ctrl API 458 | | memory | | o 459 | | storage} | | | 460 | | | | +---------------------+ 461 | | | | | {{vCPU | 462 | |Compute Node | \ [1 memory 3] 463 | | | ==> | storage} | 464 | +----------x----------+ / [2 {port rate 4] 465 \ | | | switching delay}} | 466 +----------x----------+ | +---------------------+ 467 | | | Big Switch & 468 [1 {port rate 3] | Big Software (BiS-BiS) 469 | switching delay} | | with joint 470 [2 4] / Software & Network Ctrler 471 | Network Element | / 472 +---------------------+ / 473 __/ 475 Figure 6: Big Switch with Big Software definition 477 The configuration over a BiS-BiS allows the atomic definition of NF 478 placements and the corresponding forwarding overlay as a Network 479 Function - Forwarding Graph (NF-FG). The embedment of NFs into a 480 BiS-BiS allows the inclusion of NF ports into the forwarding overlay 481 definition (see ports a, b, ...,f in Figure 7). Ports 1,2, ..., 4 482 are seen as infrastructure ports while NF ports are created and 483 destroyed with NF placements. 485 Step 1: Placement of NFs 486 Step 2: Interconnect NFs __ Step 1: Placement of NFs 487 \ with the forwarding 488 Compute Node \ overlay definition 489 +---------------------+ \ 490 | +-+ +-+ +-+ | \ +-+ +-+ +-+ 491 | |V| |V| |V| | | |V| |V| |V| 492 | |N| |N| |N| | | |N| |N| |N| 493 | |F| |F| |F| | | |F| |F| |F| 494 | |1| |2| |3| | | |1| |2| |3| 495 | +-+ +-+ +-+ | | +-+ +-+ +-+ 496 | | +---.| |.---+ | | \ | | | | | | 497 | +------\ /------+ | ==> +--a-b----c-d----e-f--+ 498 +----------x----------+ / | | | | | | | | 499 | | [1->+ +-->-+ +-->-+ | 3] 500 +----------x----------+ | | | | 501 | / \ | | [2 +->4] 502 [1->----->- -->---+ 3] | | | 503 | | | | +---------------------+ 504 [2 +->4] / Big Switch with 505 | Network Element | / Big Software (BiS-BiS) 506 +---------------------+ / 507 __/ 509 Figure 7: Big Switch with Big Software definition with a Network 510 Function - Forwarding Graph (NF-FG) 512 4.1.1. The virtualizer's data model 514 4.1.1.1. Tree view 516 module: virtualizer 517 +--rw virtualizer 518 +--rw id string 519 +--rw name? string 520 +--rw nodes 521 | +--rw node* [id] 522 | +--rw id string 523 | +--rw name? string 524 | +--rw type string 525 | +--rw ports 526 | | +--rw port* [id] 527 | | +--rw id string 528 | | +--rw name? string 529 | | +--rw port_type? string 530 | | +--rw capability? string 531 | | +--rw sap? string 532 | | +--rw sap_data 533 | | | +--rw technology? string 534 | | | +--rw resources 535 | | | +--rw delay? string 536 | | | +--rw bandwidth? string 537 | | | +--rw cost? string 538 | | +--rw control 539 | | | +--rw controller? string 540 | | | +--rw orchestrator? string 541 | | +--rw addresses 542 | | | +--rw l2? string 543 | | | +--rw l3* [id] 544 | | | | +--rw id string 545 | | | | +--rw name? string 546 | | | | +--rw configure? string 547 | | | | +--rw client? string 548 | | | | +--rw requested? string 549 | | | | +--rw provided? string 550 | | | +--rw l4? string 551 | | +--rw metadata* [key] 552 | | +--rw key string 553 | | +--rw value? string 554 | +--rw links 555 | | +--rw link* [id] 556 | | +--rw id string 557 | | +--rw name? string 558 | | +--rw src? -> 559 | | +--rw dst? -> 560 | | +--rw resources 561 | | +--rw delay? string 562 | | +--rw bandwidth? string 563 | | +--rw cost? string 564 | +--rw resources 565 | | +--rw cpu string 566 | | +--rw mem string 567 | | +--rw storage string 568 | | +--rw cost? string 569 | +--rw metadata* [key] 570 | | +--rw key string 571 | | +--rw value? string 572 | +--rw NF_instances 573 | | +--rw node* [id] 574 | | +--rw id string 575 | | +--rw name? string 576 | | +--rw type? string 577 | | +--rw ports 578 | | | +--rw port* [id] 579 | | | +--rw id string 580 | | | +--rw name? string 581 | | | +--rw port_type? string 582 | | | +--rw capability? string 583 | | | +--rw sap? string 584 | | | +--rw sap_data 585 | | | | +--rw technology? string 586 | | | | +--rw resources 587 | | | | +--rw delay? string 588 | | | | +--rw bandwidth? string 589 | | | | +--rw cost? string 590 | | | +--rw control 591 | | | | +--rw controller? string 592 | | | | +--rw orchestrator? string 593 | | | +--rw addresses 594 | | | | +--rw l2? string 595 | | | | +--rw l3* [id] 596 | | | | | +--rw id string 597 | | | | | +--rw name? string 598 | | | | | +--rw configure? string 599 | | | | | +--rw client? string 600 | | | | | +--rw requested? string 601 | | | | | +--rw provided? string 602 | | | | +--rw l4? string 603 | | | +--rw metadata* [key] 604 | | | +--rw key string 605 | | | +--rw value? string 606 | | +--rw links 607 | | | +--rw link* [id] 608 | | | +--rw id string 609 | | | +--rw name? string 610 | | | +--rw src? -> 611 | | | +--rw dst? -> 612 | | | +--rw resources 613 | | | +--rw delay? string 614 | | | +--rw bandwidth? string 615 | | | +--rw cost? string 616 | | +--rw resources 617 | | | +--rw cpu string 618 | | | +--rw mem string 619 | | | +--rw storage string 620 | | | +--rw cost? string 621 | | +--rw metadata* [key] 622 | | +--rw key string 623 | | +--rw value? string 624 | +--rw capabilities 625 | | +--rw supported_NFs 626 | | +--rw node* [id] 627 | | +--rw id string 628 | | +--rw name? string 629 | | +--rw type? string 630 | | +--rw ports 631 | | | +--rw port* [id] 632 | | | +--rw id string 633 | | | +--rw name? string 634 | | | +--rw port_type? string 635 | | | +--rw capability? string 636 | | | +--rw sap? string 637 | | | +--rw sap_data 638 | | | | +--rw technology? string 639 | | | | +--rw resources 640 | | | | +--rw delay? string 641 | | | | +--rw bandwidth? string 642 | | | | +--rw cost? string 643 | | | +--rw control 644 | | | | +--rw controller? string 645 | | | | +--rw orchestrator? string 646 | | | +--rw addresses 647 | | | | +--rw l2? string 648 | | | | +--rw l3* [id] 649 | | | | | +--rw id string 650 | | | | | +--rw name? string 651 | | | | | +--rw configure? string 652 | | | | | +--rw client? string 653 | | | | | +--rw requested? string 654 | | | | | +--rw provided? string 655 | | | | +--rw l4? string 656 | | | +--rw metadata* [key] 657 | | | +--rw key string 658 | | | +--rw value? string 659 | | +--rw links 660 | | | +--rw link* [id] 661 | | | +--rw id string 662 | | | +--rw name? string 663 | | | +--rw src? -> 664 | | | +--rw dst? -> 665 | | | +--rw resources 666 | | | +--rw delay? string 667 | | | +--rw bandwidth? string 668 | | | +--rw cost? string 669 | | +--rw resources 670 | | | +--rw cpu string 671 | | | +--rw mem string 672 | | | +--rw storage string 673 | | | +--rw cost? string 674 | | +--rw metadata* [key] 675 | | +--rw key string 676 | | +--rw value? string 677 | +--rw flowtable 678 | +--rw flowentry* [id] 679 | +--rw id string 680 | +--rw name? string 681 | +--rw priority? string 682 | +--rw port -> 683 | +--rw match string 684 | +--rw action string 685 | +--rw out? -> 686 | +--rw resources 687 | +--rw delay? string 688 | +--rw bandwidth? string 689 | +--rw cost? string 690 +--rw links 691 | +--rw link* [id] 692 | +--rw id string 693 | +--rw name? string 694 | +--rw src? -> 695 | +--rw dst? -> 696 | +--rw resources 697 | +--rw delay? string 698 | +--rw bandwidth? string 699 | +--rw cost? string 700 +--rw metadata* [key] 701 | +--rw key string 702 | +--rw value? string 703 +--rw version? string 705 Figure 8: Virtualizer's YANG data model: tree view 707 4.1.1.2. YANG Module 709 file "virtualizer.yang" 710 module virtualizer { 711 namespace "urn:unify:virtualizer"; 712 prefix "virtualizer"; 713 organization "ETH"; 714 contact "Robert Szabo "; 716 revision "2016-02-24" { 717 description "V5.0: Common port configuration were added to the yang model from the metadata fields"; 718 } 720 revision "2016-02-19" { 721 description "Added port/control (for Cf-Or interface); port/resources; link-resources/cost and sofware-resource/cost for administrative metric; clarifications for port/capability"; 722 } 724 revision "2016-01-28" { 725 description "Metadata added to infra_node and virtualizer level; Virtualizer's revised data model based on virtualizer3; changes: link key is set to id"; 726 } 728 //======================== REUSABLE GROUPS ================================ 730 grouping id-name { 731 leaf id { type string; } 732 leaf name { type string;} 733 } 735 grouping id-name-type { 736 uses id-name; 737 leaf type { 738 type string; 739 // for infrastructue view: mandatory true; --> refined in infrastrucutre view 740 mandatory false; 741 } 742 } 744 grouping metadata { 745 list metadata { 746 min-elements 0; 747 key key; 748 leaf key{ 749 type string; 750 mandatory true; 751 } 752 leaf value{ 753 type string; 754 mandatory false; 755 } 756 } 757 } 759 grouping link-resource { 760 leaf delay { 761 type string; 762 mandatory false; 763 } 764 leaf bandwidth { 765 type string; 766 mandatory false; 767 } 768 leaf cost { 769 description "Administrative metric."; 770 type string; 771 mandatory false; 772 } 774 } 776 grouping l3-address { 777 uses id-name; 778 leaf configure { 779 description "True: this is a configuration request; False: this is fyi"; 780 type string; 781 } 782 leaf client { 783 description "Configuration service support at the client: {'dhcp-client', 'pre-configured'}; if not present it is left to the infrastructure to deal with it."; 784 type string; 785 } 786 leaf requested { 787 description "To request port configuration, options: {'public', 'ip/mask'}, where public means the request of public IP address and private ip/mask a given address/mask configuration"; 788 type string; 789 } 790 leaf provided { 791 description "The provided L3 configuration in response to the requested field."; 792 type string; 793 } 794 } 795 // ------------ PORTS ------- 796 grouping port { 797 uses id-name; 798 leaf port_type { 799 description "{port-abstract, port-sap} port-sap is to represent UNIFY domain boundary; port-abstract is to represent UNIFY native port. Technology specific attributes of a SAP is in the metadata."; 800 type string; 801 } 802 leaf capability { 803 description "To describe match and action capabilities associated with the port, e.g., match=port,tag,ip,tcp,udp,mpls,of1.0, where port: based forwarding; tag: unify abstract tagging; ip: ip address matching etc."; 804 type string; 805 } 806 leaf sap { 807 type string; 808 } 809 container sap_data { 810 leaf technology { 811 description "e.g., ('IEEE802.1q': '0x00c', 'MPLS': 70, 'IEEE802.1q')"; 812 type string; 813 } 814 container resources{ 815 description "Only used for domain boundary ports (port-sap type), where this is used to derive interconnection link characteristics."; 816 uses link-resource; 817 } 818 } 819 container control { 820 description "Used to connect this port to a UNIFY orchestrator's Cf-Or reference point. Support controller - orchestrator or orchestrator - controller connection establishment."; 821 leaf controller{ 822 description "URI of the local controller service at this NF, e.g., http://*:8080/cf-or/"; 823 type string; 824 } 825 leaf orchestrator{ 826 description "URI of the scoped orchestration service offered to this NF specifically, e.g., http://192.168.1.100:8080/cf-or/"; 827 type string; 828 } 829 } 830 container addresses { 831 leaf l2 { 832 description "Requested or provided"; 833 type string; 834 } 835 list l3 { 836 key "id"; 837 uses l3-address; 838 } 839 leaf l4 { 840 description "e.g., request: {tcp/22, tcp/8080}; response {tcp/22: (192.168.1.100, 1001)"; 841 type string; 842 } 843 } 844 uses metadata; 845 } 847 // ------------ FLOW CONTROLS ------- 849 grouping flowentry { 850 description "The flowentry syntax will follow ovs-ofctrl string format. The UNIFY general tagging mechanism will be use like 'mpls'-> 'tag', i.e., push_tag:tag; pop_tag:tag..."; 851 uses id-name; 852 leaf priority { 853 type string; 854 } 855 leaf port { 856 type leafref { 857 path ""; 858 } 859 mandatory true; 860 } 861 leaf match { 862 description "The match syntax will follow ovs-ofctrl string format with 'mpls'->'tag', e.g.,: in_port=port, dl_tag=A, where port is the leafref above"; 863 type string; 864 mandatory true; 865 } 866 leaf action { 867 description "The action syntax will follow ovs-ofctrl string format with 'mpls'->'tag', e.g.,: push_tag:A, set_tag_label:A, output:out, where out is the leafref below"; 868 type string; 869 mandatory true; 871 } 872 leaf out { 873 type leafref { 874 path ""; 875 } 876 } 877 container resources{ 878 uses link-resource; 879 } 881 } 883 grouping flowtable { 884 container flowtable { 885 list flowentry { 886 key "id"; 887 uses flowentry; 888 } 889 } 890 } 892 // ------------ LINKS ------- 894 grouping link { 895 uses id-name; 896 leaf src { 897 type leafref { 898 path ""; 899 } 900 } 901 leaf dst { 902 type leafref { 903 path ""; 904 } 905 } 906 container resources{ 907 uses link-resource; 908 } 909 } 911 grouping links { 912 container links { 913 list link { 914 key "id"; 915 uses link; 916 } 917 } 919 } 921 // ---------- NODE ------------------- 923 grouping software-resource { 924 leaf cpu { 925 type string; 926 mandatory true; 927 } 928 leaf mem { 929 type string; 930 mandatory true; 931 } 932 leaf storage { 933 type string; 934 mandatory true; 935 } 936 leaf cost { 937 description "Administrative metric."; 938 type string; 939 mandatory false; 940 } 941 } 943 grouping node { 944 description "Any node: infrastructure or NFs"; 945 uses id-name-type; 946 container ports { 947 list port{ 948 key "id"; 949 uses port; 950 } 951 } 952 uses links; 953 container resources{ 954 uses software-resource; 955 } 956 uses metadata; 957 } 959 grouping nodes { 960 list node{ 961 key "id"; 962 uses node; 963 } 964 } 966 grouping infra-node { // they can contain other nodes (as NFs) 967 uses node { 968 refine type { 969 mandatory true; 970 } 971 } 972 container NF_instances { 973 uses nodes; 974 } 975 container capabilities { 976 container supported_NFs { // if supported NFs are enumerated 977 uses nodes; 978 } 979 } 980 uses flowtable; 981 } 983 //======================== NF-FG: Virtualizer and the Mapped request ================================ 985 container virtualizer { 986 description "Container for a single virtualizer"; 987 uses id-name { 988 refine id { 989 mandatory true; 990 } 991 } 992 container nodes{ 993 list node{ // infra nodes 994 key "id"; 995 uses infra-node; 996 } 997 } 998 uses links; // infra links 999 uses metadata; 1000 leaf version { 1001 description "yang and virtualizer library version"; 1002 type string; 1003 } 1004 } 1005 } 1006 1008 Figure 9: Virtualizer's YANG data model 1010 5. Relation to ETSI NFV 1012 According to the ETSI MANO framework [ETSI-NFV-MANO], an NFVO is 1013 split into two functions: 1015 o The orchestration of NFVI resources across multiple VIMs, 1016 fulfilling the Resource Orchestration functions. The NFVO uses 1017 the Resource Orchestration functionality to provide services that 1018 support accessing NFVI resources in an abstracted manner 1019 independently of any VIMs, as well as governance of VNF instances 1020 sharing resources of the NFVI infrastructure 1022 o The lifecycle management of Network Services, fulfilling the 1023 network Service Orchestration functions. 1025 Similarly, a VIM is split into two functions: 1027 o Orchestrating the allocation/upgrade/release/reclamation of NFVI 1028 resources (including the optimization of such resources usage), 1029 and 1031 o managing the association of the virtualised resources to the 1032 physical compute, storage, networking resources. 1034 The functional split is shown in Figure 14. 1036 +-------------------+ 1037 |NVFO | 1038 | +--------------+ | 1039 | |NFVO: | | 1040 | |Service | | 1041 | |Lifecycle | | 1042 | |Management | | 1043 | +------+-------+ | 1044 | | | 1045 | +------+-------+ | 1046 | |NFVO: | | 1047 | |Resrouce | | 1048 | |Orchestration | | 1049 | +--+---+----+--+ | 1050 +-----|---|----|----+ 1051 / | \ 1052 /---------/ | \------------\ 1053 / | \ 1054 +-------------|-----+ +--------|----------+ +------|------------+ 1055 |VIM | | |VIM | | |VIM | | 1056 | +----------+---+ | | +-----+--------+ | | +---+----------+ | 1057 | |VIM: | | | |VIM: | | | |VIM: | | 1058 | |Orchestration | | | |Orchestration | | | |Orchestration | | 1059 | |& | | | |& | | | |& | | 1060 | |Optimization | | | |Optimization | | | |Optimization | | 1061 | +------+-------+ | | +------+-------+ | | +------+-------+ | 1062 | | | | | | | | | 1063 | +------+-------+ | | +------+-------+ | | +------+-------+ | 1064 | |VIM: | | | |VIM: | | | |VIM: | | 1065 | |Virtualized 2 | | | |Virtualized 2 | | | |Virtualized 2 | | 1066 | |Pys mapping | | | |Pys mapping | | | |Pys mapping | | 1067 | +--------------+ | | +--------------+ | | +--------------+ | 1068 +-------------------+ +-------------------+ +-------------------+ 1070 Figure 10: Functional decomposition of the NFVO and the VIM according 1071 to the ETSI MANO 1073 If the Joint Software and Network Control API (Joint API) could be 1074 used between all the functional components working on the same 1075 abstraction, i.e., from the north of the VIM Virtualized to physical 1076 mapping component to the south of the NFVO: Service Lifecycle 1077 Management as shown in Figure 11, then a more flexible virtualization 1078 programming architecture could be created as shown in Figure 12. 1080 +-------------------+ 1081 |NVFO | 1082 | +--------------+ | 1083 | |NFVO: | | 1084 | |Service | | 1085 | |Lifecycle | | 1086 | |Management | | 1087 | +------+-------+ | 1088 | | | <-- Joint API 1089 | +------+-------+ | 1090 | |NFVO: | | 1091 | |Resrouce | | 1092 | |Orchestration | | 1093 | +--+---+-------+ | 1094 +-----|---|---------+ 1095 / | 1096 /---------/ | <-- Joint API 1097 / | 1098 +-------------|-----+ +--------|----------+ 1099 |VIM | | |VIM | | 1100 | +----------+---+ | | +-----+--------+ | 1101 | |VIM: | | | |VIM: | | 1102 | |Orchestration | | | |Orchestration | | 1103 | |& | | | |& | | 1104 | |Optimization | | | |Optimization | | 1105 | +------+-------+ | | +------+-------+ | 1106 | | | | | | <-- Joint API 1107 | +------+-------+ | | +------+-------+ | 1108 | |VIM: | | | |VIM: | | 1109 | |Virtualized 2 | | | |Virtualized 2 | | 1110 | |Pys mapping | | | |Pys mapping | | 1111 | +--------------+ | | +--------------+ | 1112 +-------------------+ +-------------------+ 1114 Figure 11: Functional decomposition of the NFVO and the VIM with the 1115 Joint Software and Network control API 1116 +--------------+ 1117 |NFVO: | 1118 |Service | 1119 Domain 4 |Lifecycle M. | 1120 +--+-----------+ 1121 **********************|****************** 1122 * +--------------+ | 1123 * |NFVO: | | 1124 * |Service | | 1125 * |Lifecycle | | 1126 * |Management | | 1127 * +-------+------+ / 1128 * | / <-- Joint API 1129 * +-+---------+--+ 1130 * | Rersource | 1131 * |Orchestration | 1132 ******************** * | | 1133 +--------------+ * * +--+---+-------+ Domain 3 1134 |NFVO: | * ********|***|************************* 1135 |Service | * / | 1136 |Lifecycle | /---------/ | 1137 |Management | / * | 1138 +---------+----+ | * | 1139 | | * | <-- Joint API 1140 +--+-------+---+* | 1141 | |* | 1142 | Rersource |* | 1143 |Orchestration |* | 1144 | |* | 1145 +------+-------+* | 1146 /|\ * *********|********** <-- Joint API 1147 +------+-------+* * +------+-------+ * 1148 +--|d1 |* * |VIM: | * 1149 +--|d2|Resource |* * |Virtualized 2 | * 1150 |d3| | Orchestration|* * |Pys mapping | * 1151 | | +--------------+* * +--------------+ * 1152 Domain 1 * * Domain 2 * 1153 ************************* * * 1155 Figure 12: Joint Software and Network Control API: Recurring Flexible 1156 Architecture 1158 5.1. Policy based resource management 1160 In Figure 13 we show various policies mapped to the MANO 1161 architecture: 1163 o Tenant Policies: Tenant policies exist whenever a domain offers a 1164 virtualization service to more than one consumer. User tenants 1165 may exists at the northbound of the NFVO. Additionally, if a VIM 1166 expose resource services to more than one NFVO, then each NFVO may 1167 appear as a tenant (virtualization consumer) at the northbound of 1168 the VIM. 1170 o Wherever virtualization services are produced or consumed 1171 corresponding export and import policies may exist. Export 1172 policies govern the details of resources, capabilities, costs, 1173 etc. exposed to consumers. In turn, consumers (tenants) apply 1174 import policies to filter, tweak, annotate resources and services 1175 received from their southbound domains. An entity may at the same 1176 time consume and produce virtualization services hence apply both 1177 import and export policies. 1179 o Operational policies support the business logic realized by the 1180 domain's ownership. They are often associated with Operations or 1181 Business Support Systems (OSS or BSS) and frequently determine 1182 operational objectives like energy optimization, utilization 1183 targets, offered services, charing models, etc. Operational 1184 policies may be split according to different control plane layers, 1185 for example, i) lifecycle and ii) resource management layers 1186 within the NFVO. 1188 T1 T2...Tn 1189 | | | 1190 +-----|--|---|------+ 1191 |NVFO | | | | Tenant 1192 | +--+--+---+----+ | <- Policies 1193 | |NFVO: | | 1194 Operational | |Service | | 1195 Policies-> | |Lifecycle | | 1196 | |Management | | 1197 | +------+-------+ | 1198 | | | 1199 | +------+-------+ | 1200 | |NFVO: | | 1201 Operational | |Resource | | 1202 Policies-> | |Orchestration | | ^ 1203 | +--+---+----+--+ | |Import 1204 to +-----|---|---------+ |Policies 1205 other NFVO / \ 1206 \ +-------+ \ 1207 \ / \ to NFVO ^ 1208 +------\------|-----+ \ / |Export 1209 |VIM \ | | \ / |Policies 1210 | +-----+----+---+ | +--------|----|-----+ 1211 | |VIM: | | |VIM | | | Tenant 1212 | |Orchestration | | | +-----+----+---+ | <- Policies 1213 | |& | | | |VIM: | | 1214 | |Optimization | | . | |Orchestration | | 1215 | +------+-------+ | . | |& | | <- Operational 1216 | | | | |Optimization | | Policies 1217 | +------+-------+ | | +------+-------+ | 1218 | |VIM: | | | | | 1219 | |Virtualized 2 | | | +------+-------+ | 1220 | |Pys mapping | | | |VIM: | | <- Operational 1221 | +--------------+ | | |Virtualized 2 | | Policies 1222 +-------------------+ | |Pys mapping | | 1223 | +--------------+ | 1224 +-------------------+ 1226 Figure 13: Policies within the MANO framework 1228 6. Examples 1230 6.1. Infrastructure reports 1232 Figure 14 and Figure 15 show a single node infrastructure report. 1233 The example shows a BiS-BiS with two ports, out of which Port 0 is 1234 also a Service Access Point 0 (SAP0). 1236 20 CPU 1237 +-----------+ 64GB MEM 1238 SAP1--[0 BiS-BiS | 1TB STO 1239 | (UUID13) | 1240 +[2 1]+ 1241 | +-----------+ | 1242 | | 1243 | | 1244 +-----------+ | | +-----------+ 1245 SAP0--[0 BiS-BiS 1]+ +[0 BiS-BiS 1]--SAP1 1246 | (UUID11) | | (UUID12) | 1247 | 2]-----------------[2 | 1248 +-----------+ +-----------+ 1249 20 CPU 10 CPU 1250 64GB MEM 32GB MEM 1251 100TB STO 100TB STO 1253 Figure 14: Single node infrastructure report example: Virtualization 1254 view 1256 1257 UUID001 1258 Single node simple infrastructure report 1259 1260 1261 UUID11 1262 single Bis-Bis node 1263 BisBis 1264 1265 1266 0 1267 SAP0 port 1268 port-sap 1269 ... 1270 1271 1272 1 1273 North port 1274 port-abstract 1275 ... 1276 1277 1278 2 1279 East port 1280 port-abstract 1281 ... 1282 1283 1284 1285 20 1286 64 GB 1287 100 TB 1288 1289 1290 1291 1293 Figure 15: Single node infrastructure report example: xml view 1295 Figure 16 and Figure 17 show a 3-node infrastructure report with 3 1296 BiS-BiS nodes. Infrastructure links are inserted into the 1297 virtualization view between the ports of the BiS-BiS nodes. 1299 20 CPU 1300 +-----------+ 64GB MEM 1301 SAP1--[0 BiS-BiS | 1TB STO 1302 | (UUID13) | 1303 +[2 1]+ 1304 | +-----------+ | 1305 | | 1306 | | 1307 +-----------+ | | +-----------+ 1308 SAP0--[0 BiS-BiS 1]+ +[0 BiS-BiS 1]--SAP1 1309 | (UUID11) | | (UUID12) | 1310 | 2]-----------------[2 | 1311 +-----------+ +-----------+ 1312 20 CPU 10 CPU 1313 64GB MEM 32GB MEM 1314 100TB STO 100TB STO 1316 Figure 16: 3-node infrastructure report example: Virtualization view 1318 1319 UUID002 1320 3-node simple infrastructure report 1321 1322 1323 UUID11 1324 West Bis-Bis node 1325 BisBis 1326 1327 1328 0 1329 SAP0 port 1330 port-sap 1331 ... 1332 1333 1334 1 1335 North port 1336 port-abstract 1337 ... 1338 1339 1340 2 1341 East port 1342 port-abstract 1343 ... 1344 1345 1346 1347 20 1348 64 GB 1349 100 TB 1350 1351 1352 1353 UUID12 1354 East Bis-Bis node 1355 BisBis 1356 1357 1358 1 1359 SAP1 port 1360 port-sap 1361 ... 1362 1363 1364 0 1365 North port 1366 port-abstract 1367 ... 1368 1369 1370 2 1371 West port 1372 port-abstract 1373 ... 1374 1375 1376 1377 10 1378 32 GB 1379 100 TB 1380 1381 1382 1383 UUID13 1384 North Bis-Bis node 1385 BisBis 1386 1387 1388 0 1389 SAP2 port 1390 port-sap 1391 ... 1392 1393 1394 1 1395 East port 1396 port-abstract 1397 ... 1398 1399 1400 2 1401 West port 1402 port-abstract 1403 ... 1404 1405 1406 1407 20 1408 64 GB 1409 1 TB 1410 1411 1412 1413 1414 1415 0 1416 Horizontal link 1417 ../../nodes/node[id=UUID11]/ports/port[id=2] 1418 ../../nodes/node[id=UUID12]/ports/port[id=2] 1419 1420 2 ms 1421 10 Gb 1422 1423 1424 1425 1 1426 West link 1427 ../../nodes/node[id=UUID11]/ports/port[id=1] 1428 ../../nodes/node[id=UUID13]/ports/port[id=2] 1429 1430 5 ms 1431 10 Gb 1432 1433 1434 1435 2 1436 East link 1437 ../../nodes/node[id=UUID12]/ports/port[id=0] 1438 ../../nodes/node[id=UUID13]/ports/port[id=1] 1439 1440 2 ms 1441 5 Gb 1442 1444 1445 1446 1448 Figure 17: 3-node infrastructure report example: xml view 1450 6.2. Simple requests 1452 Figure 18 and Figure 19 show the allocation request for 3 NFs (NF1: 1453 Parental control B.4, NF2: Http Cache 1.2 and NF3: Stateful firewall 1454 C) as instrumented over a BiS-BiS node. It can be seen that the 1455 configuration request contains both the NF placement and the 1456 forwarding overlay definition as a joint request. 1458 +---+ +---+ +---+ 1459 |NF1| |NF2| |NF3| 1460 +---+ +---+ +---+ 1461 | | | | | | 1462 +-2-3----4-5----6-7--+ 1463 --[0-/ \____/ \----|- \ | 1464 | |___________| \-+1]-- 1465 | | 1466 | BiS-BiS (UUID11) | 1467 +--------------------+ 1469 Figure 18: Simple request of 3 NFs on a single BiS-BiS: 1470 Virtualization view 1472 1473 UUID001 1474 Single node simple request 1475 1476 1477 UUID11 1478 1479 1480 NF1 1481 first NF 1482 Parental control B.4 1483 1484 1485 2 1486 in 1487 port-abstract 1488 ... 1489 1490 1491 3 1492 out 1493 port-abstract 1494 ... 1495 1496 1497 1498 1499 NF2 1500 cache 1501 Http Cache 1.2 1502 1503 1504 4 1505 in 1506 port-abstract 1507 ... 1508 1509 1510 5 1511 out 1512 port-abstract 1513 ... 1514 1515 1516 1517 1518 NF3 1519 firewall 1520 Stateful firewall C 1521 1522 1523 6 1524 in 1525 port-abstract 1526 ... 1527 1528 1529 7 1530 out 1531 port-abstract 1532 ... 1533 1534 1535 1536 1537 1538 1539 ../../ports/port[id=0] 1540 * 1541 output:../../NF_instances/node[id=NF1] 1542 /ports/port[id=2] 1543 1544 1545 ../../NF_instances/node[id=NF1] 1546 /ports/port[id=3] 1547 fr-a 1548 output:../../NF_instances/node[id=NF2] 1549 /ports/port[id=4] 1550 rpcre 1551 1552 ../../NF_instances/node[id=NF1] 1553 /ports/port[id=3] 1554 fr-b 1555 output:../../NF_instances/node[id=NF3] 1556 /ports/port[id=6] 1557 1558 1559 ../../NF_instances/node[id=NF2] 1560 /ports/port[id=5] 1561 * 1562 output:../../ports/port[id=1] 1563 1564 1565 ../../NF_instances/node[id=NF3] 1566 /ports/port[id=7] 1567 * 1568 output:../../ports/port[id=1] 1569 1570 1571 1572 1573 1575 Figure 19: Simple request of 3 NFs on a single BiS-BiS: xml view 1577 7. Experimentations 1579 We have implemented the proposed recursive control plane architecture 1580 with joint software and network virtualization and control. We used 1581 a Python based open source implementation [virtualizer-library] of 1582 the virtualizer data structure for the orchestration API. We used 1583 the Extensible Service ChAin Prototyping Environment (ESCAPE) 1584 [ESCAPE] as the general orchestration platform with various 1585 technology specific domain adapters like OpenStack, Docker and Ryu 1586 SDN controller. A detailed service function chaining report is 1587 available at [I-D.unify-sfc-control-plane-exp]. 1589 8. IANA Considerations 1591 This memo includes no request to IANA. 1593 9. Security Considerations 1595 TBD 1597 10. Acknowledgement 1599 The research leading to these results has received funding from the 1600 European Union Seventh Framework Programme (FP7/2007-2013) under 1601 grant agreement no. 619609 - the UNIFY project. The views expressed 1602 here are those of the authors only. The European Commission is not 1603 liable for any use that may be made of the information in this 1604 document. 1606 We would like to thank in particular David Jocha and Janos Elek from 1607 Ericsson for the useful discussions. 1609 11. Informative References 1611 [ESCAPE] BME, "Extensible Service ChAin Prototyping Environment 1612 (open source)", Mar. 2016, 1613 . 1615 [ETSI-NFV-Arch] 1616 ETSI, "Architectural Framework v1.1.1", Oct 2013, 1617 . 1620 [ETSI-NFV-MANO] 1621 ETSI, "Network Function Virtualization (NFV) Management 1622 and Orchestration V0.6.1 (draft)", Jul. 2014, 1623 . 1626 [I-D.caszpe-nfvrg-orchestration-challenges] 1627 Carrozzo, G., Szabo, R., and K. Pentikousis, "Network 1628 Function Virtualization: Resource Orchestration 1629 Challenges", draft-caszpe-nfvrg-orchestration- 1630 challenges-00 (work in progress), November 2015. 1632 [I-D.ietf-sfc-dc-use-cases] 1633 Surendra, S., Tufail, M., Majee, S., Captari, C., and S. 1634 Homma, "Service Function Chaining Use Cases In Data 1635 Centers", draft-ietf-sfc-dc-use-cases-04 (work in 1636 progress), January 2016. 1638 [I-D.unify-nfvrg-challenges] 1639 Szabo, R., Csaszar, A., Pentikousis, K., Kind, M., Daino, 1640 D., Qiang, Z., and H. Woesner, "Unifying Carrier and Cloud 1641 Networks: Problem Statement and Challenges", draft-unify- 1642 nfvrg-challenges-03 (work in progress), January 2016. 1644 [I-D.unify-sfc-control-plane-exp] 1645 Szabo, R. and B. Sonkoly, "SFC Control Plane Experiment: 1646 UNIFYed Approach", March 2016, . 1649 [I-D.zu-nfvrg-elasticity-vnf] 1650 Qiang, Z. and R. Szabo, "Elasticity VNF", draft-zu-nfvrg- 1651 elasticity-vnf-01 (work in progress), March 2015. 1653 [virtualizer-library] 1654 Ericsson, "Python based virtualizer library for Netconf 1655 protocol (open source)", Mar. 2016, 1656 . 1658 Authors' Addresses 1660 Robert Szabo (editor) 1661 Ericsson Research, Hungary 1662 Irinyi Jozsef u. 4-20 1663 Budapest 1117 1664 Hungary 1666 Email: robert.szabo@ericsson.com 1667 URI: http://www.ericsson.com/ 1669 Zu Qiang 1670 Ericsson 1671 8400, boul. Decarie 1672 Ville Mont-Royal, QC 8400 1673 Canada 1675 Email: zu.qiang@ericsson.com 1676 URI: http://www.ericsson.com/ 1677 Mario Kind 1678 Deutsche Telekom AG 1679 Winterfeldtstr. 21 1680 10781 Berlin 1681 Germany 1683 Email: mario.kind@telekom.de