idnits 2.17.1 draft-lachosrothenberg-alto-brokermdo-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 2, 2018) is 2123 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '1' on line 774 -- Looks like a reference, but probably isn't: '2' on line 777 -- Looks like a reference, but probably isn't: '3' on line 780 -- Looks like a reference, but probably isn't: '4' on line 783 == Missing Reference: 'TBD' is mentioned on line 529, but not defined == Outdated reference: A later version (-08) exists of draft-li-sfc-hhsfc-04 == Outdated reference: A later version (-24) exists of draft-ietf-alto-unified-props-new-03 == Outdated reference: A later version (-25) exists of draft-ietf-alto-path-vector-03 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ALTO WG D. Lachos 3 Internet-Draft C. Rothenberg 4 Intended status: Informational Unicamp 5 Expires: January 3, 2019 July 2, 2018 7 ALTO-based Broker-assisted Multi-domain Orchestration 8 draft-lachosrothenberg-alto-brokermdo-01 10 Abstract 12 Evolving networking scenarios (e.g., 5G) demand new multiple 13 administrative domain (aka multi-domain) orchestration models. This 14 document proposes the use of Application-Layer Traffic Optimization 15 (ALTO) services to offer topology and resources addressing network 16 service discovery and provisioning by multi-domain orchestrators. 17 The ALTO services with the proposed protocol extension offer 18 aggregated views on various types of resources contributing to a more 19 simple and scalable solution for resource and service discovery in 20 multi-domain, multi-technology environments. 22 Requirements Language 24 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 25 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 26 document are to be interpreted as described in RFC 2119 [RFC2119]. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on January 3, 2019. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Changes Since Version -00 . . . . . . . . . . . . . . . . . . 3 64 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 65 4. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 5. Problem Statement and Challenges . . . . . . . . . . . . . . 5 67 6. Proposed Approach . . . . . . . . . . . . . . . . . . . . . . 6 68 6.1. Inter-domain Resource (IdR) Component . . . . . . . . . . 7 69 6.2. Inter-domain Topology (IdT) Component . . . . . . . . . . 8 70 6.3. ALTO Server Functionalities . . . . . . . . . . . . . . . 8 71 6.4. Filtered Cost Map Extension . . . . . . . . . . . . . . . 8 72 6.4.1. Accept Input Parameters . . . . . . . . . . . . . . . 9 73 6.4.2. Response . . . . . . . . . . . . . . . . . . . . . . 9 74 6.5. Examples of Message Exchange . . . . . . . . . . . . . . 10 75 6.5.1. Property Map Service . . . . . . . . . . . . . . . . 10 76 6.5.2. Filtered Cost Map Service . . . . . . . . . . . . . . 11 77 7. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 14 78 7.1. Benefits . . . . . . . . . . . . . . . . . . . . . . . . 14 79 7.2. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 15 80 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 81 9. Security Considerations . . . . . . . . . . . . . . . . . . . 16 82 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 16 83 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 84 11.1. Normative References . . . . . . . . . . . . . . . . . . 16 85 11.2. Informative References . . . . . . . . . . . . . . . . . 16 86 11.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 18 87 Appendix A. Proof of Concept Use Case Implementation . . . . . . 18 88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 90 1. Introduction 92 Envisioned 5G network architectures and related service models 93 consider broader cooperation between stakeholders in order to provide 94 flexible multi-operator multi-domain services. These multi-provider 95 orchestration operations will require the information exchange across 96 Multi-domain Orchestrators (MdOs). The key information to be 97 exchanged between MdOs includes the abstract network topology, 98 resource availability (e.g., CPUs, Memory, and Storage) and 99 capability (e.g., supported network functions). 101 This document presents a federation networking paradigm where a 102 broker-plane works on top of the management and orchestration plane 103 to assist and coordinate the creation of an End-to-End Network 104 Service (E2ENS) spanning over multi-operator multi-domain networks. 105 Our design resorts to the Application-Layer Traffic Optimization 106 (ALTO) protocol [RFC7285] to address the lack of abstractions to 107 discover and adequately represent in confidentiality-preserving 108 fashion the resource and topology information from different 109 administrative domains. Moreover, this draft introduces an extension 110 to the ALTO base protocol for inter-domain connectivity information 111 discovery. 113 2. Changes Since Version -00 115 o Many minor style and grammar edits. 117 o Updated Problem Statement and Challenges section. 119 o Removed Property Map Extension section. The current Property Map 120 draft [DRAFT-PM] already supports property values encoded as 121 JSONArray. 123 o Added section on benefits and open questions in our proposed 124 architecture. 126 3. Terminology 128 We use the following definitions, as established in [ETSI-NFV-DEF]: 130 Administrative Domain: Collection of systems and networks operated 131 by a single organization or administrative authority. 133 Network Function (NF): Functional block within a network 134 infrastructure that has well-defined external interfaces and well- 135 defined functional behaviour. 137 Network Functions Virtualisation (NFV): The principle of separating 138 network functions from the hardware they run on by using virtual 139 hardware abstraction. 141 NF Forwarding Graph: (NFFG): Graph of logical links connecting NF 142 nodes for the purpose of describing traffic flow between these 143 network functions. 145 Network Service Orchestration (NSO): Function responsible for 146 network service lifecycle management. 148 Resource Orchestration (RO): Function responsible for global 149 resource management governance. 151 Our proof of concept implementation follows the architectural 152 proposal of the 5GEx project [H2020.5GEX]. Some additional 5GEx 153 terms commonly used in this document are defined below: 155 Domain Orchestrator (DO): Performs Resource Orchestration and/or 156 Service Orchestration within the same administrative domain. 158 Multi-domain Orchestrator (MdO): Coordinates resource and/or service 159 orchestration at multi-domain level, where multi-domain may refer 160 to multiple DOs or multiple administrative domains. 162 Resource Topology (RT): Functional module that is responsible for 163 keeping an updated global view of the underlying infrastructure 164 topology exposed by DOs. 166 Service Graph (SG): A high-level data model for defining flexible 167 network services (including traffic steering primitives). 169 Service Access Point (SAP): A named/tagged port supporting stitching 170 (service to service, domain to domain, etc.) 172 4. Scope 174 Existing proposals for the network service orchestration are 175 intrinsically conceived for single administrative domain scenarios. 176 For example, in the standard service orchestration model described in 177 ETSI NFV MANO framework [ETSI-NFV-MANO], one orchestrator is supposed 178 to work within one administrative domain. The analysis of 179 orchestration and management of network Services over multiple 180 administrative domains have begun to be addressed by ETSI 181 in [ETSI-NFV-MANO-MDO]. 183 Envisioned 5G scenarios are expected to work not only with 184 heterogeneous technologies but also across different network 185 operators. Many ongoing initiatives and projects related are 186 addressing the multi-provider multi-domain orchestration challenges 187 under different approaches. For example, [H2020.5GEX] seeks to 188 integrate multiple administrations and technologies through the 189 collaboration between operations. Other studies are envisioned to 190 use a centralized approach, where each domain advertises its 191 capabilities to a federation layer which will act as a 192 broker [VITAL][T-NOVA]. The proposed architecture in [ICAF] allows 193 the creation of cloud services from different administrative domains, 194 however, it is not related to the provisioning of NFV-based cross- 195 domain network services. 197 All such proposals described above envision the potential 198 introduction of new business model approaches, including federation 199 models [PPP-5:2013] among administrative domains. In this context, 200 this document considers each network operator involved in the 201 community advertises its abstracted capabilities (e.g., software/ 202 hardware resources, physical/virtual network functions, etc.) to a 203 broker (i.e., 3rd party). This latter, in its turn, provides or 204 assists coordinate E2E network services spanning multi-domain 205 networks. 207 5. Problem Statement and Challenges 209 The provision of a complete E2E network service requires chaining 210 services provided by multiple network operator with multiple 211 technologies. In this multi-domain environment, the orchestration 212 process will require an advertise mechanism through which single 213 domains can describe their capabilities, resources, and VNFs in an 214 interoperable manner. Moreover, a discovery mechanism is also 215 necessary so that source domains can obtain candidate domains (with 216 the corresponding connectivity information) which can provide a part 217 of the service and/or slide in an E2ENS requirement. 219 In order to the advertising and discovery process works in a proper 220 way, a number of challenges can be identified: 222 Lack of Abstractions: Multiple vendors with heterogeneous 223 technologies need an information model to adequately represent 224 in confidentiality-preserving fashion the resource and topology 225 information. 227 Scalability: Involves the distribution of topology and resource 228 information in a peer-to-peer fashion (MdO-to-MdO). Multi- 229 operator multi-domain environments where the information 230 distribution is advertised in a peer-to-peer model scales 231 linearly. It means more MdO interconnections one has, the more 232 it "costs" to distribute. 234 Flexibility: Considers that a distributed approach does not allow 235 domains without physical infrastructure (e.g., without BGP or 236 BGP-LS) to advertise resource capabilities and networking 237 resources. Such procedures consist in deploying and configuring 238 physical peering points for these domains. 240 Complexity: Refers to the discovery mechanism to pre-select 241 candidate domains, accounting for resources and capabilities, 242 necessary for an E2E network service deployment. An intrinsic 243 complexity exists in the process of assembling, logically 244 organizing, and enabling abstraction views of different 245 resources and capabilities in multi-domain scenarios. 247 6. Proposed Approach 249 The primary design goal for ALTO-based Broker-assisted Multi-domain 250 Orchestration is to discover resource and topology information from 251 different administrative domains involved in the federation, while 252 also safeguarding the privacy and autonomy of every domain. 254 In the architectural proposal shown in Figure 1, a broker component 255 is conceived to be working as coordinator of a set of MdOs, whose key 256 components are: the Inter-domain Resource (IdR), the Inter-domain 257 Topology (IdT) and the ALTO Server. 259 BROKER COMPONENT 260 +--------------------------------------------+ 261 | | 262 | +-----------------+ | 263 | | | | 264 XXXXXXXXXXXXXXXXXXX ALTO SERVER(s) | | 265 X | | + | 266 X | +----------------+\ | 267 X | / \ | 268 X | / \ | 269 X | +------+/+-------+ +---------------+ | 270 X | | Inter-domain | | Inter-domain | | 271 X | | Topology (IdT) | | Resource (IdR)| | 272 X | +-^------^-------+ +---^-------^---+ | 273 X | | | | | | 274 X +--------------------------------------------+ 275 X | | | | 276 X | | | | 277 +--X--------+---+--------------------+ | 278 | | | | 279 | | +------------+------------+---+ 280 | MdO1 | | | 281 | +<------------->+ +---+ 282 +---------------+ | MdO2 | | 283 | | | 284 +-+--------------+ | 285 | | 286 Legend: | MdO(n) | 287 XXX ALTO Protocol +------------------+ 289 Figure 1: Broker-assisted Multi-operator Network Architecture 291 6.1. Inter-domain Resource (IdR) Component 293 It creates a hierarchical database that contains inter-domain 294 resource information such as resource availability (i.e., CPU, 295 memory, and storage), Virtual Network Functions (VNFs) and Physical 296 Network Functions (PNFs) supported and Service Access Points (SAPs) 297 to access those resources. UNIFY [UNIFY.NFFG], TOSCA [TOSCA], ETSI- 298 NFV [ETSI-NFV-MANO], among other data models can be used to create 299 the interface between IdR and MdOs. 301 6.2. Inter-domain Topology (IdT) Component 303 A hierarchical TED (Traffic Engineering Database) that contains 304 inter-domain network topology information including additional key 305 parameters (e.g., throughput and latency of links). This information 306 can be retrieved from each MdO through BGP-LS or REST interfaces. 308 6.3. ALTO Server Functionalities 310 The ALTO server component is the core of the broker layer. Multiple 311 logically centralized ALTO servers use the information collected from 312 IdR and IdT modules to create and provide abstract maps with a 313 simplified view, yet enough information about MdOs involved in the 314 federation. This information includes domain-level topology, storage 315 resources, computation resources, networking resources and PNF/VNF 316 capabilities. 318 As an ALTO client, each MdO sends ALTO service queries to the ALTO 319 server. This server provides aggregated inter-domain information 320 exposed as set ALTO base services defined in [RFC7285], e.g., Network 321 Map, Cost Map and ALTO extension services, e.g., Property 322 Map [DRAFT-PM], Multi-Cost Map [RFC8189], Path Vector [DRAFT-PV]. 324 For example, when a source MdO receives a customer service request, 325 it checks whether or not it can deliver the full service. If it is 326 unable to do so, the MdO consumes from the ALTO Server the Property 327 Map service to have a clear global view of the resource information 328 offered by other MdOs. This information allows discovering which 329 candidate MdOs may be contacted to deliver the remaining requirements 330 of a requested end-to-end service deployment. The connectivity 331 information among discovered MdOs can be retrieved by a Cost Map 332 service, responding, for instance, a path vector with the AS-level 333 topology distance between the source MdO and candidate MdOs. 335 6.4. Filtered Cost Map Extension 337 The ALTO server MUST provide connectivity information for every SG 338 link in the SG path for an E2E requirement. This information is the 339 AS-level topology distance in the form of path vector, and it 340 includes all possible ways for each (source node, destination node) 341 pair in the SG link. 343 In this section, we introduce a non-normative overview of the 344 Filtered Cost Map defined in Section 6.1 of [DRAFT-PV] [1]. 346 The specifications for the "Media Types", "HTTP method", 347 "Capabilities" and "Uses" (described in Section 6.1 of [DRAFT-PV] 348 [2]) are unchanged. 350 6.4.1. Accept Input Parameters 352 The ReqFilteredCostMap object in Section 6.1.2 of [DRAFT-PV] [3] is 353 extended as follow: 355 object { 356 [NFFG sg;] 357 } ReqFilteredCostMap; 359 object { 360 JSONString nfs<1..*>; 361 JSONString saps<1..*>; 362 NextHops sg_links<1..*>; 363 REQs reqs<1..*>; 364 } NFFG; 366 object { 367 JSONNumber id; 368 JSONString src-node; 369 JSONString dst-node; 370 } NextHops; 372 object { 373 JSONString id; 374 JSONString src-node; 375 JSONString dst-node; 376 JSONNumber sg-path<1..*>; 377 } REQs; 379 sg: If present, the ALTO Server MUST allow the request input to 380 include an SG with a formatted body as an NFFG object. An NFFG 381 object contains NFs, SAPs, SG links representing logical 382 connections between NFs, SAPs or both and E2E requirements as a 383 list of ids of SG links. 385 It is worth noting that further versions of this draft will define a 386 more elaborated NFFG object to support extended parameters such as 387 monitoring parameters, resource requirements, etc. 389 6.4.2. Response 391 If the ALTO client includes the path vector cost mode in the "cost- 392 type" or "multi-cost-types" field of the input parameter, the 393 response for each SG link in each E2E requirement MUST be encoded as 394 a JSONArray of JSONArrays of JSONStrings. Anyone of the sub-arrays 395 indicates a potential candidate path calculated as the per-domain 396 topological distance corresponding to the amount of traversing 397 domains. 399 Moreover, as defined in Section 6.3.6 of [DRAFT-PV] [4], If an ALTO 400 client sends a request of the media type "application/alto- 401 costmapfilter+json" and accepts "multipart/related", the ALTO server 402 MUST provide path vector information along with the associated 403 Property Map information (e.g., entry points of the corresponding 404 foreign domains), in the same body of the response. 406 Section 6.5.2 gives an example of the Filtered Cost Map query and the 407 corresponding responses. 409 6.5. Examples of Message Exchange 411 This section list a couple of examples of the Property Map and 412 Filtered Cost Map queries and the corresponding responses. These 413 responses are based on the information in Table 1 and Table 2 of a 414 use case implementation described in Appendix A. 416 6.5.1. Property Map Service 418 In this example, the ALTO client wants to retrieve the entire 419 Property Map for PID entities with the "entry-point", "cpu", "mem", 420 "storage", "port" and "nf" properties. 422 GET /propmap/full/inet-ucmspn HTTP/1.1 423 Host: alto.example.com 424 Accept: application/alto-propmap+json,application/alto-error+json 426 HTTP/1.1 200 OK 427 Content-Length: ### 428 Content-Type: application/alto-propmap+json 429 { 430 "property-map": { 431 "pid:AS1": { 432 "entry-point": [ "http://172.25.0.10:8888/escape" ], 433 "cpu": [ "50.0" ], 434 "mem": [ "60.0" ], 435 "storage": [ "70.0" ], 436 "port": [ "SAP1" ], 437 "nf": [ "NF1", "NF3" ] 438 }, 439 "pid:AS2": { 440 "entry-point": [ "http://172.26.0.10:8888/escape" ], 441 "cpu": [ "10.0" ], 442 "mem": [ "20.0" ], 443 "storage": [ "30.0" ], 444 "nf": [ "NF2" ] 445 }, 446 "pid:AS3": { 447 "entry-point": [ "http://172.27.0.10:8888/escape" ], 448 "cpu": [ "80.0" ], 449 "mem": [ "90.0" ], 450 "storage": [ "100.0" ], 451 "port": [ "SAP2" ], 452 "nf": [ "NF1", "NF3" ] 453 } 454 } 455 } 457 6.5.2. Filtered Cost Map Service 459 The following example uses the Filtered Cost Map service to request 460 the path vector for a given E2E requirement. The SG request 461 information in Table 2 is used to describe the service, and it is 462 composed of three NFs (NF1, NF2, and NF3) and two SAPs (SAP1 and 463 SAP2). Links connecting the NFs and SAPs ("sg_links" tag) are also 464 included, followed by an E2E requirement ("reqs" tag) with 465 information about the order in which NFs are traversed from SAP1 to 466 SAP2. 468 Note that the request accepts "multipart/related" media type. This 469 means the ALTO server will include associated property information in 470 the same response. 472 POST /costmap/pv HTTP/1.1 473 Host: alto.example.com 474 Accept: multipart/related, application/alto-costmap+json, 475 application/alto-propmap+json, application/alto-error+json 476 Content-Length: [TBD] 477 Content-Type: application/alto-costmapfilter+json 479 { 480 "cost-type": { 481 "cost-mode": "array", 482 "cost-metric": "ane-path" 483 }, 484 "sg": { 485 "nfs": [ "NF1", "NF2", "NF3" ], 486 "saps": [ "SAP1", "SAP2" ], 487 "sg_links":[ 488 { 489 "id": 2, 490 "src-node": "SAP1", 491 "dst-node": "NF1", 493 }, 494 { 495 "id": 2, 496 "src-node": "NF1", 497 "dst-node": "NF2", 498 }, 499 { 500 "id": 3, 501 "src-node": "NF2", 502 "dst-node": "NF3", 503 }, 504 { 505 "id": 4, 506 "src-node": "NF3", 507 "dst-node": "SAP2", 508 } 509 ], 510 "reqs": [ 511 { 512 "id": 1, 513 "src-node": "SAP1", 514 "dst-node": "SAP2", 515 "sg-path": [ 1, 2, 3, 4 ] 516 } 517 ] 518 } 519 } 521 The ALTO server returns connectivity information for the E2E 522 requirement provided by the ALTO Client request of the above example. 523 Also, the response includes Property Map information for each element 524 in the path vector. In this case, it is retrieved a Property Map 525 with the "entry-point" property, i.e., the URL of the MdO entry point 526 for the corresponding network. 528 HTTP/1.1 200 OK 529 Content-Length: [TBD] 530 Content-Type: multipart/related; boundary=example 532 --example 533 Content-Type: application/alto-endpointcost+json 535 { 536 "meta": { 537 "cost-type": { 538 "cost-mode": "array", 539 "cost-metric": "ane-path" 540 }, 541 }, 543 "cost-map": { 544 "SAP1": { 545 "SAP2": { 546 "SAP1": { 547 "NF1": [ 548 [ "AS1" ], [ "AS1", "AS2", "AS3" ] 549 ] 550 }, 551 "NF1": { 552 "NF2": [ 553 [ "AS1", "AS2" ], [ "AS3", "AS2" ] 554 ] 555 }, 556 "NF2": { 557 "NF3": [ 558 [ "AS2", "AS1" ], [ "AS2", "AS3" ] 559 ] 561 }, 562 "NF3": { 563 "SAP2": [ 564 [ "AS1", "AS2", "AS3" ], [ "AS3" ] 565 ] 566 } 567 } 568 } 569 } 570 } 572 --example 573 Content-Type: application/alto-propmap+json 575 { 576 "property-map": { 577 "pid:AS1": { "entry-point": "http://172.25.0.10:8888/escape" }, 578 "pid:AS2": { "entry-point": "http://172.26.0.10:8888/escape" }, 579 "pid:AS3": { "entry-point": "http://172.27.0.10:8888/escape" } 580 } 581 } 583 --example-- 585 7. Discussion 587 In this section we analyze the benefits and open issues in our 588 broker-assisted architecture. 590 7.1. Benefits 592 The broker-assisted orchestration has numerous benefits, such as: 594 o Avoid the distribution of topology and resource information in a 595 peer-to-peer fashion (MdO-to-MdO) 597 o The (abstracted) information and offered resources, services are 598 maintained in each local MdO. 600 o Allow domains without physical infrastructure (hence without BGP 601 or BGP-LS) to advertise their capabilities. 603 o An ALTO-based privacy-preserving information model to provide 604 computing, storage and networking resource info. 606 o An MdO discovery method to determine the underlying network graph 607 and a potential set of paths before bilateral negotiation between 608 MdOs is started. 610 7.2. Open Issues 612 Although the broker-assisted information exchange has several 613 advantages, it also raises some questions which we try to answer from 614 our lessons learned. 616 o What kind of organization will manage and support the operation of 617 a broker entity? If a broker is used to exchange information, 618 then how does one ensure that the data delivered amongst the 619 operators by this 3rd party has not been changed? 621 * The broker entity must be trusted by each operator since it 622 stores and handles sensitive information. For example, future 623 deployment of SDN at IXPs can be used as a trusted third-party 624 platform to support rich business models between different 625 operators [DRAFT-HHSFC]. 627 o In the case of peer-to-peer information exchange model, an MdO 628 failure concerns only the domain where the failure occurs, other 629 peers can perform the information exchange without any limitation. 630 However, If any error occurs in the broker entity the information 631 exchange among all involved ASes will be impacted. How avoid this 632 single point of failure? 634 * The broker entity maintains a centralized database. Local 635 restoration/replication options may be applied. 637 o The MdO information exchange depends on the policies. Operators 638 have a preference to share a different view about its compute and 639 network resources towards different operators. For example, a 640 detailed view for the operators that are belonging to same 641 operator group and a high-level information towards the other 642 operators. How is the fine-grained/coarse-grained information 643 exchange handled?. 645 * It requires much more complex database handling and information 646 exchange with the MdOs depending on the policies. 648 8. IANA Considerations 650 This document includes no request to IANA. 652 9. Security Considerations 654 TBD. 656 10. Acknowledgments 658 This work is supported by the Innovation Center of Ericsson S.A., 659 Brazil (grant agreement UNI.64). 661 Thank you to Robert Szabo (Ericsson Research, Hungary) for the 662 contribution and substantial feedback and suggestions in this 663 document. 665 Many thanks to Richard Yang, Dawn Chan, Jensen Zhang, Shawn Lin, Qiao 666 Xiang, Sabine Randriamasy for their feedback on this draft. 668 11. References 670 11.1. Normative References 672 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 673 Requirement Levels", BCP 14, RFC 2119, March 1997, 674 . 676 [RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S., 677 Previdi, S., Roome, W., Shalunov, S., and R. Woundy, 678 "Application-Layer Traffic Optimization (ALTO) Protocol", 679 RFC 7285, DOI 10.17487/RFC7285, September 2014, 680 . 682 [RFC8189] Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost 683 Application-Layer Traffic Optimization (ALTO)", RFC 8189, 684 DOI 10.17487/RFC8189, October 2017, 685 . 687 11.2. Informative References 689 [DRAFT-HHSFC] 690 Li, G., Li, G., Li, T., Xu, Q., and H. Zhou, "Hybrid 691 Hierarchical Multi-Domain Service Function chaining", 692 draft-li-sfc-hhsfc-04 (work in progress), April 2018. 694 [DRAFT-PM] 695 Roome, W., Chen, S., Randriamasy, S., Yang, Y., and J. 696 Zhang, "Unified Properties for the ALTO Protocol", draft- 697 ietf-alto-unified-props-new-03 (work in progress), March 698 2018. 700 [DRAFT-PV] 701 Bernstein, G., Chen, S., Gao, K., Lee, Y., Roome, W., 702 Scharf, M., Yang, Y., and J. Zhang, "ALTO Extension: Path 703 Vector Cost Type", draft-ietf-alto-path-vector-03 (work in 704 progress), March 2018. 706 [ETSI-NFV-DEF] 707 ETSI, "Network Functions Virtualisation (NFV); Terminology 708 for Main Concepts in NFV V1.3.1", Jan 2018, 709 . 713 [ETSI-NFV-MANO] 714 ETSI, "Network Functions Virtualisation (NFV) Management 715 and Orchestration V1.1.1", Dec 2014, 716 . 719 [ETSI-NFV-MANO-MDO] 720 ETSI, "Network Functions Virtualisation (NFV) Release 3; 721 Management and Orchestration; Report on architecture 722 options to support multiple administrative domains 723 V3.1.1", Jan 2018, . 727 [H2020.5GEX] 728 Bernardos, C., Dugeon, O., Galis, A., Morris, D., Simon, 729 C., and R. Szabo, "5G Exchange (5GEx)--Multi-domain 730 Orchestration for Software Defined Infrastructures", 731 focus vol. 4, no.5, p.2, 2015. 733 [H2020.5GEX.ESCAPE] 734 5GEx Project, "ESCAPE: Extensible Service ChAin 735 Prototyping Environment", 2015, 736 . 738 [ICAF] Demchenko, Y., Makkes, M., Strijkers, R., Ngo, C., and C. 739 Laat, "Intercloud Architecture Framework for Heterogeneous 740 Multi-Provider Cloud based Infrastructure Services 741 Provisioning", International Journal of Next-Generation 742 Computing vol. 4, no.2, 2013. 744 [PPP-5:2013] 745 5G-PPP, "Advanced 5G Network Infrastructure for the Future 746 Internet", 2013, . 750 [T-NOVA] FP7 project T-NOVA, "T-NOVA Project, Network Functions as 751 a Service over Virtualised Infrastructures", 2014, 752 . 754 [TELEFONICA.NET.TOPO] 755 Telefonica I+D, "Netphony-Topology", 2016, 756 . 758 [TOSCA] OASIS, "TOSCA: Topology and Orchestration Specification 759 for Cloud Applications V1.0", 2013, . 762 [UNIFY.NFFG] 763 UNIFY Deliverable D3.2a, "Network Function Forwarding 764 Graph specification", 2015, . 768 [VITAL] VITAL PROJECT H2020, "VITAL -- VIrtualized hybrid 769 satellite-TerrestriAl systems for resilient and fLexible 770 future networks", 2015, . 772 11.3. URIs 774 [1] https://tools.ietf.org/html/draft-ietf-alto-path-vector- 775 02#section-6.1 777 [2] https://tools.ietf.org/html/draft-ietf-alto-path-vector- 778 02#section-6.1 780 [3] https://tools.ietf.org/html/draft-ietf-alto-path-vector- 781 02#section-6.1.2 783 [4] https://tools.ietf.org/html/draft-ietf-alto-path-vector- 784 02#section-6.3.6 786 Appendix A. Proof of Concept Use Case Implementation 788 A strawman use case scenario has been implemented following the 789 architectural proposal of the 5GEx project [H2020.5GEX]. It refers 790 to an E2ENS orchestration involving three administrative domains. 792 As shown in Figure 2, each administrative domain has an MdO (MdO-AS1, 793 MdO-AS2, and MdO-AS3) to coordinate resource and/or service 794 orchestration at multi-operator level via interface I2 APIs. For the 795 orchestration within the same administrative domain, each MdO uses 796 emulated DOs with emulated I3 interfaces, since no data-plane is 797 present. DOs use static configuration files to load local 798 information about resources (I3-RC) and topology (I3-RT). The 799 different MdO components are based on existing open source tools such 800 as ESCAPE [H2020.5GEX.ESCAPE] (Service/Resource Orchestrator) and 801 Netphony-topology [TELEFONICA.NET.TOPO] (Resource Topology) and run 802 in Docker containers on a single computer. Besides, MdOs expose I1 803 interfaces to the tenants who request services and/or slices which 804 should follow a Network Function Forwarding Graph (NFFG) [UNIFY.NFFG] 805 format. 807 In case of the broker layer, the IdR and IdT components use a UNIFY 808 Virtualizer API [UNIFY.NFFG] (broker-based I2-RC API) and a REST API 809 (broker-based I2-RT API) respectively, in order to create the 810 hierarchical databases. Regarding the IdT, the administrative domain 811 2 is a transit provider so that the domain-level topology computed 812 is: AS1-AS2-AS3. From the inter-domain information are created the 813 two different ALTO Map Services: (i) Property Map and (ii) Cost Map. 815 +----------------------------------------+ 816 | +---------------+ BROKER LAYER| 817 XXXXXXXXXXXXXX ALTO Server | | 818 X | +--------+----+-+ | 819 X | / \ | 820 X | +-----------+/+--+ +-\-------------+ | 821 X | | Inter-domain | | Inter-domain | | 822 +-----------+ X | | Topology (IdT) | | Resource (IdR)| | 823 | Service | X | +----------------+ +---------------+ | 824 | Graph (SG)| X +---------^-^----------------^--^--------+ 825 | Request | X * * ............. . 826 +-----+-----+ X * * . .............. 827 | X * * . . MdO-AS3 828 I1| X * * . . +--------------+ 829 | X MdO-AS1 * * . . | | 830 +-----|---------X-----------+ * * . . | MdO-AS2 | 831 | | | * * . . +---------------+ | 832 | +---v-------------------+ | * * . . | +-----------+ | | 833 | | | | * * . . | | | | | 834 | | Network Service Orch.| | * * . . | | NSO | | | 835 | | (NSO) | | * * . . | | | | | 836 | +-----------------------+ | * * . . | +-----------+ | | 837 | | * * . . | | | 838 | +---------+ | * * . . | +---+ | | 839 | | Resource........................... . | | | | | 840 | | Topoloy | | * * .......RT | | | 841 | | (RT) | +-----------+ | * * | | | | | 842 | +---------+ |Resource | | * * | +---+ +---+ | | 843 | |Orch. | | * ********************** | | | 844 | |(RO) ****** | |RO | +-+ 845 | +-----------+ | | | | | 846 | |<------------->| +---+ | 847 +---------------------------+ I2 +-----+---------+ 848 / \ | 849 I3/ \ |I3 850 +-------+---+ +-----------+ +-----------+ 851 | Domain | | Domain | | Domain | 852 | Orch (DO) | | Orch (DO) | | Orch (DO) | 853 +-----------+ +-----------+ +-----------+ 854 Legend: 855 XXX ALTO Protocol 856 ... broker-based I2-RT API 857 *** broker-based I2-RC API 859 Figure 2: Broker-assisted 5GEx Info Exchange 861 The Property Map includes property values grouped by Autonomous 862 System (AS). Such values are SAPs, NFs and the 5GEx Entry Point 863 (e.g., the URL of the ESCAPE orchestrator). An example of the 864 Property Map in our prototype is: 866 +-----+------------+-------+--------------+-----+-----+-------+-----+ 867 | | Entry | Port | Capabilities | CPU | MEM | Stor | ... | 868 | | Point | SAP | | | | age | | 869 +-----+------------+-------+--------------+-----+-----+-------+-----+ 870 | AS1 | http://... | SAP1 | {NF1, NF3} | 50 | 60 | 70 | ... | 871 | AS2 | http://... | - | {NF2} | 10 | 20 | 30 | ... | 872 | AS3 | http://... | SAP2 | {NF1, NF3} | 80 | 90 | 100 | ... | 873 +-----+------------+-------+--------------+-----+-----+-------+-----+ 875 Table 1: ALTO Property Map 877 The Cost Map defines a path vector as an array of ASes, representing 878 the AS-level topological distance for a given E2ENS request. Path 879 vector constraints (as described in the Multi-Cost Map [RFC8189]) can 880 be applied to restricts the response to costs that satisfy a list of 881 simple predicates. 883 Table 2 below shows a brief example of an SG request and its path 884 vector response containing a list of potential providers to be 885 traversed to deliver such service. Every AS path is computed from 886 the inter-domain topology information in the IdT module. In our 887 scenario, MdO-AS2 is a transit provider, so that the domain-level 888 topology map is AS1<->AS2<->AS3. 890 +--------------------+----------------------------------------------+ 891 | Service Graph (SG) | Path(s) Vector | 892 | Request | | 893 +--------------------+----------------------------------------------+ 894 | SAP1->NF1->NF2->NF | 1:{AS1:SAP1->AS1:NF1->AS2:NF2->AS3:NF3->AS3: | 895 | 3->SAP2 | SAP2} | 896 | | 2:{AS1:SAP1->AS1:NF1->AS2:NF2->AS1:NF3->AS2- | 897 | | >AS3:SAP2} | 898 | | 3:{AS1:SAP1->AS2->AS3:NF1->AS2:NF2->AS3:NF3- | 899 | | AS3:SAP2} | 900 | | 4:{AS1:SAP1->AS2->AS3:NF1->AS2:NF2->AS1:NF3- | 901 | | >AS2->AS3:SAP2} | 902 +--------------------+----------------------------------------------+ 904 Table 2: ALTO Cost Map 906 Authors' Addresses 908 Danny Alex Lachos Perez 909 University of Campinas 910 Av. Albert Einstein 400 911 Campinas, Sao Paulo 13083-970 912 Brazil 914 Email: dlachosp@dca.fee.unicamp.br 915 URI: https://intrig.dca.fee.unicamp.br/danny-lachos/ 917 Christian Esteve Rothenberg 918 University of Campinas 919 Av. Albert Einstein 400 920 Campinas, Sao Paulo 13083-970 921 Brazil 923 Email: chesteve@dca.fee.unicamp.br 924 URI: https://intrig.dca.fee.unicamp.br/christian/