idnits 2.17.1 draft-li-teas-hierarchy-ip-controllers-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 13, 2020) is 1354 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-27) exists of draft-dhodylee-pce-pcep-ls-15 == Outdated reference: A later version (-05) exists of draft-ietf-bess-l3vpn-yang-04 == Outdated reference: A later version (-17) exists of draft-ietf-idr-bgp-model-09 == Outdated reference: A later version (-04) exists of draft-ietf-idr-rtc-hierarchical-rr-03 == Outdated reference: A later version (-18) exists of draft-ietf-opsawg-l3sm-l3nm-03 == Outdated reference: A later version (-14) exists of draft-ietf-pce-pcep-extension-for-pce-controller-06 == Outdated reference: A later version (-13) exists of draft-ietf-pce-pcep-flowspec-09 == Outdated reference: A later version (-11) exists of draft-ietf-teas-actn-yang-05 == Outdated reference: A later version (-15) exists of draft-ietf-teas-te-service-mapping-yang-03 == Outdated reference: A later version (-22) exists of draft-ietf-teas-yang-path-computation-10 == Outdated reference: A later version (-36) exists of draft-ietf-teas-yang-te-24 -- No information found for draft-wu-model-driven-management-virtualization - is the name correct? -- Obsolete informational reference (is this intentional?): RFC 5575 (Obsoleted by RFC 8955) -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 12 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TEAS Working Group Z. Li 3 Internet-Draft D. Dhody 4 Intended status: Informational Huawei Technologies 5 Expires: January 14, 2021 H. Chen 6 Futurewei Technologies 7 July 13, 2020 9 Hierarchy of IP Controllers (HIC) 10 draft-li-teas-hierarchy-ip-controllers-05 12 Abstract 14 This document describes the interactions between various IP 15 controllers in a hierarchical fashion to provide various IP services. 16 It describes how the Abstraction and Control of Traffic Engineered 17 Networks (ACTN) framework is applied to the Hierarchy of IP 18 controllers (HIC) as well as document the interactions with other 19 protocols like BGP, Path Computation Element Communication Protocol 20 (PCEP) to provide end to end dynamic services spanning multiple 21 domains and controllers (e.g. Layer 3 Virtual Private Network 22 (L3VPN), Seamless MPLS etc). 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on January 14, 2021. 41 Copyright Notice 43 Copyright (c) 2020 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 2. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4 60 2.1. Mapping to ACTN . . . . . . . . . . . . . . . . . . . . . 4 61 2.2. Interface between Super Controller and Domain Controller 62 in HIC . . . . . . . . . . . . . . . . . . . . . . . . . 5 63 3. Key Concepts . . . . . . . . . . . . . . . . . . . . . . . . 6 64 3.1. Topology . . . . . . . . . . . . . . . . . . . . . . . . 6 65 3.2. Path Computation/Path instantiation . . . . . . . . . . . 7 66 3.3. BGP considerations . . . . . . . . . . . . . . . . . . . 8 67 4. VPN Service . . . . . . . . . . . . . . . . . . . . . . . . . 8 68 4.1. Seamless MPLS . . . . . . . . . . . . . . . . . . . . . . 8 69 4.2. L3VPN . . . . . . . . . . . . . . . . . . . . . . . . . . 10 70 4.3. L2VPN and EVPN service . . . . . . . . . . . . . . . . . 11 71 5. Possible Features/Extensions . . . . . . . . . . . . . . . . 11 72 6. Other Considerations . . . . . . . . . . . . . . . . . . . . 12 73 6.1. Control Plane . . . . . . . . . . . . . . . . . . . . . . 12 74 6.1.1. PCE / PCEP . . . . . . . . . . . . . . . . . . . . . 12 75 6.1.2. BGP . . . . . . . . . . . . . . . . . . . . . . . . . 14 76 6.2. Management Plane . . . . . . . . . . . . . . . . . . . . 15 77 6.2.1. YANG Models . . . . . . . . . . . . . . . . . . . . . 15 78 6.2.2. Protocol Considerations . . . . . . . . . . . . . . . 16 79 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 80 8. Security Considerations . . . . . . . . . . . . . . . . . . . 17 81 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 17 82 10. Contributing Authors . . . . . . . . . . . . . . . . . . . . 17 83 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 84 11.1. Normative References . . . . . . . . . . . . . . . . . . 17 85 11.2. Informative References . . . . . . . . . . . . . . . . . 17 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 88 1. Introduction 90 Software-Defined Networking (SDN) refers to a separation between the 91 control elements and the forwarding components so that software 92 running in a centralized system called a controller, can act to 93 program the devices in the network to behave in specific ways. A 94 required element in an SDN architecture is a component that plans how 95 the network resources will be used and how the devices will be 96 programmed. It is possible to view this component as performing 97 specific computations to place flows within the network given 98 knowledge of the availability of network resources, how other 99 forwarding devices are programmed, and the way that other flows are 100 routed. The Application-Based Network Operation (ABNO) [RFC7491] 101 describes how various components and technologies fit together. 103 A domain [RFC4655] is any collection of network elements within a 104 common sphere of address management or path computation 105 responsibility. Specifically within this document we mean a part of 106 an operator's network that is under common management. Network 107 elements will often be grouped into domains based on technology 108 types, vendor profiles, and geographic proximity and under a domain 109 controller. 111 Multiple such domains in the network are interconnected, and a path 112 is established through a series of connected domains to form an end- 113 to-end path over which various services are offered. Each domain is 114 under the control of the domain controller (or lower-level 115 controller), and a "super controller" (or high-level controller) 116 takes responsibility for a high-level view of the network before 117 distributing tasks to domain controllers (or lower-level 118 controllers). It is possible for each of the domain to use a 119 different tunnelling mechanism (eg RSVP-TE, Segment Routing (SR) 120 etc). 122 [RFC8453] describes the framework for Abstraction and Control of 123 Traffic Engineered Networks (ACTN) as well as a set of management and 124 control functions used to operate multiple TE networks. This 125 documents would apply the ACTN principles to Hierarchy of IP 126 controllers (HIC) and focus on the applicability and interactions 127 with other protocol and technologies (specific to IP packet domains). 129 Sometimes, service (such as Layer 3 Virtual Private Network (L3VPN), 130 Layer 2 Virtual Private Network (L2VPN), Ethernet VPN (EVPN), 131 Seamless MPLS) require sites attached to different domains (under the 132 control of different domain controller) to be interconnected as part 133 of the VPN service. This require multi-domain coordination between 134 domain controllers to compute and set-up E2E path for the VPN 135 service. 137 This document describes the interactions between various IP 138 controllers in a hierarchical fashion to provide various IP services. 139 It describes how the Abstraction and Control of Traffic Engineered 140 Networks (ACTN) framework is applied to the Hierarchy of IP 141 controllers (HIC) as well as document the interactions with control 142 plane protocols (like BGP, Path Computation Element Communication 143 Protocol (PCEP)) and management plane aspects (Yang models) to 144 provide end to end dynamic services spanning multiple domains and 145 controllers (e.g. L3VPN, Seamless MPLS etc.). 147 2. Overview 149 Figure 1 show examples of multi-domain IP domains under hierarchy of 150 IP controllers. 152 | 153 +------------+ 154 | SuperCo | 155 +------------+ 156 | 157 ---------------------------------- 158 | | | 159 +------------+ +------------+ +------------+ 160 | DoCo#1 | | DoCo#2 | | DoCo#3 | 161 +------------+ +------------+ +------------+ 163 +--Domain#1--+ +--Domain#2--+ +--Domain#3--+ 164 | | | | | | 165 | B------+---+---D-----E--+---+------J | 166 | / | | \ / | | \ | 167 | / | | \ / | | \ | 168 | A | | H | | L | 169 | \ | | / \ | | / | 170 | \ | | / \ | | / | 171 | C------+---+---F-----G--+---+------K | 172 | | | | | | 173 +------------+ +------------+ +------------+ 175 Figure 1: Example: Hierarchy of IP controllers (HIC) 177 The IP "Super Controller" receives request from the network/service 178 orchestrator to set-up dynamic services spanning multiple domains. 179 The IP "Super Controller" breaks down and assigns tasks to the domain 180 controllers, responsible for communicating to network devices in the 181 domain. It further coordinates between the controller to provide a 182 unified view of the multi-domain network. 184 2.1. Mapping to ACTN 186 As per [RFC8453], ACTN has following main functions - 188 o Multi-domain coordination 189 o Virtualization/Abstraction 191 o Customer mapping/translation 193 o Virtual service coordination 195 These functions are part of Multi Domain Service Coordinator (MDSC) 196 and/or Provisioning Network Controller (PNC). Further these 197 functions are part of the controller / orchestrator. 199 The HIC is an instantiation of ACTN framework for IP packet network. 200 The IP domain (lower-level) controllers implements the PNC 201 functionalities for configuring, controlling and monitoring the IP 202 domain. The "super controller" (high-level controller) implements 203 the MDSC functionalities for coordination between multiple domains as 204 well as maintaining an abstracted view of multiple domains. It also 205 takes care of the service related functionalities of customer 206 mapping/translation and virtual service coordination. 208 The ACTN functions are part of the IP controllers and responsible for 209 the TE topology and E2E path computation/set-up. There are other 210 functions along with ACTN that are needed to manage multiple IP 211 domain networks. 213 2.2. Interface between Super Controller and Domain Controller in HIC 215 The interaction between super controller and domain controller in HIC 216 is a combination of Control Plane and Management Plane interface as 217 shown in Figure 2. BGP [RFC4271] and PCEP [RFC5440] are example of 218 the control plane interface; where as NETCONF [RFC6241] and RESTCONF 219 [RFC8040] are example of management plane interface. 221 +----------------------------------------------+ 222 | Super Controller | 223 | | 224 | | 225 +------------------*------#---------------------+ 226 * # 227 * # 228 ************************* 229 * # * 230 ######*############### * 231 # * # * 232 +---------#-----*--+ +--#--------*------+ 233 | Domain | | Domain | 234 | Controller | | Controller | 235 +--#------------*--+ +--#------------*--+ 236 # * # * 237 # * # * 239 * -> Control Plane Interface 240 # -> Management Plane Interface 242 Figure 2: Interface between Super Controller and Domain Controller 244 Note that ACTN's MDSC-PNC Interface (MPI) could be implemented via 245 management plane interface using Yang models 246 [I-D.ietf-teas-actn-yang] or via PCEP control plane interface 247 [RFC8637]. 249 3. Key Concepts 251 3.1. Topology 253 The Domain Controller is expected to be aware of the topology of the 254 network devices in its domain. The domain controller could 255 participate in the IGP ([RFC3630] and [RFC5305]) or use BGP-LS 256 [RFC7752] by which link-state and TE information is collected and 257 shared with domain controller using the BGP routing protocol. 259 An alternate approach would be to rely on the management plane 260 interface which uses the YANG model for network/TE Topology as per 261 [RFC8345] and [I-D.ietf-teas-yang-te-topo]. 263 The domain controller is expected to share the domain topology to the 264 Super Controller as aligned to ACTN (where PNC abstract the topology 265 towards MDSC). A level of abstraction is usually applied while 266 presenting the topology to a higher level controller. Topology 267 abstraction is described in [RFC7926] as well as [RFC8453]. BGP-LS, 268 PCEP-LS [I-D.dhodylee-pce-pcep-ls] or management plane interface 269 based on the abstracted network/TE Topology could be used to carry 270 the abstract topology to the super-controller. At minimum the border 271 nodes and inter-domain links are exposed to the super-controller. 273 Further [RFC8453] defines three types of topology abstraction - (1) 274 Native/White Topology; (2) Black Topology; and (3) Grey Topology. 275 Based on the local policy, the domain controller would share the 276 domain topology to the Super Controller based on the abstraction 277 type. Note that any of the control plane or management plane 278 mechanism could be used to carry abstracted domain topology. The 279 Super Controller's MDSC function is expected to manage a E2E topology 280 by coordinating the abstracted domain topology received from the 281 domain controllers. 283 3.2. Path Computation/Path instantiation 285 The Domain Controller is responsible for computing and setup of path 286 when the source and destination is in the same domain, otherwise the 287 Super Controller coordinates the multi-domain path computation and 288 setup with the help of the domain controller. This is aligned to 289 ACTN. 291 PCEP [RFC5440] provides mechanisms for Path Computation Elements 292 (PCEs) [RFC4655] to perform path computations in response to Path 293 Computation Clients (PCCs) requests. Since then, the role and 294 function of the PCE has grown to allow delegated control [RFC8231] 295 and PCE-initiated use of network resources [RFC8281]. 297 Further, [RFC6805] and [RFC8751] describes a hierarchy of PCE with 298 Parent PCE coordinating multi-domain path computation function 299 between Child PCE(s). This fits well with HIC as described in this 300 document. 302 Note that a management plane interface which uses the YANG model for 303 path computation/setup ([I-D.ietf-teas-yang-path-computation] and 304 [I-D.ietf-teas-yang-te]) could be used in place of PCEP. 306 In case there is a need to stitch per domain tunnels into an E2E 307 tunnel, mechanism are described in 308 [I-D.dugeon-pce-stateful-interdomain]. 310 3.3. BGP considerations 312 [RFC4456] describes the concept of route-reflection where a "route 313 reflector" (RR) reflects the routes to avoid full mesh connection 314 between Internal BGP (IBGP) peers. The IP domain controller can play 315 the role of RR in its domain. The super controller can further act 316 as RR to towards the domain controller. 318 BGP can provide routing policies for the traffic management, like 319 route preference, AS-path filter policy, IP-prefix filter policy and 320 route aggregation. The controller can distribute these BGP Policy 321 into the routers in a single IP domain. For the scenario of multiple 322 domains, the super controller can distribute per BGP Policy into each 323 IP domain controller. Then the IP domain controller trickles down 324 the BGP Policy to the network devices. 326 [RFC5575] describes the concept of BGP Flowspec that can be used to 327 distribute traffic flow specifications. A flow specification is an 328 n-tuple consisting of several matching criteria that can be applied 329 to IP traffic. The controller can originate the flow specifications 330 and disseminate to the routers. The flow action includes the 331 redirection to a specific TE tunnel. Also, the IP domain controller 332 could be responsible for collecting the flow sample in its domain and 333 the super controller can act as the Flow Analysis Server. 335 [RFC7854] describes the BGP Monitoring Protocol (BMP) to monitor BGP 336 sessions. BMP is used to obtain the route views with a flexible way. 337 In the fashion of hierarchical architecture, the IP domain controller 338 can be used as the domain Monitoring Station. Meanwhile, the super 339 controller is responsible for a high-level view of the global network 340 state. 342 4. VPN Service 344 4.1. Seamless MPLS 346 Seamless MPLS [I-D.ietf-mpls-seamless-mpls] describes an architecture 347 which can be used to extend MPLS networks to integrate access and 348 core/aggregation networks into a single MPLS domain.In the seamless 349 MPLS for mobile backhaul, since there are multiple domains including 350 the core network and multiple mobile backhaul networks, for each 351 domain there is a domain controller. In order to implement the end- 352 to-end network service provision, there should be coordination among 353 multiple domain controllers. 355 | 356 | 357 | 358 +----------+ 359 |--------------|Super |---------| 360 | |Controller| | 361 | +----------+ | 362 | | | 363 | | | 364 | | | 365 +------+ +------+ +------+ 366 |----|DoCo |----| |---|DoCo |--| |----|DoCo |---| 367 | |#X1 | | | |#Y | | | |#X2 | | 368 | +------+ | | +------+ | | +------+ | 369 | | | | | | 370 | | | | | | 371 | | | | | | 372 | +----+ +----+ | 373 | ....|ABR1|...........|ABR3|.... | 374 +----+ ..... +----+ +----+ ..... +----+ 375 | PE |... ...| PE | 376 +----+ ..... +----+ 377 ....+----+ +----+ ..... 378 |ABR2|...........|ABR4|.... 379 +----+ +----+ 381 | IGP-X1 | IGP-Y | IGP-X2 | 382 | (MBH) | (Core) | (MBH) | 383 | | | | 384 |-----BGP LSP-----|-----BGP LSP----|------BGP LSP-----| 385 | | | | 386 |---LDP/TE LSP----|----LDP/TE LSP--|-----LDP/TE LSP---| 387 | | | | 389 Figure 3: Seamless MPLS 391 Super Controller is responsible for setting the seamless MPLS 392 service. It should break down the service model to network 393 configuration model [RFC8309] and the domain controller further break 394 it to the device configuration model to the PE/ASBR to make the E2E 395 seamless MPLS service. The selection of appropriate ASBRs and 396 handling of intra-domain tunnels is coordinated by the Super 397 Controller in the similar fashion as shown in Section 4.2. 399 By enabling BGP sessions between Domain Controller and Super 400 Controller, BGP labeled routes can also be learned at Super 401 Controller. As Super Controller is aware of the (abstract) topology, 402 it could make intelligent decisions regarding E2E BGP LSP to optimize 403 based on the overall traffic information. 405 4.2. L3VPN 407 A Layer 3 IP VPN service is a collection of sites that are authorized 408 to exchange traffic between each other over a shared IP 409 infrastructure. [RFC4110] provides a framework for Layer 3 Provider- 410 Provisioned Virtual Private Networks (PPVPNs). [RFC8299] provides a 411 L3VPN service delivery YANG model for PE-based VPNs. The Super 412 controller is expected to implement the L3SM model and translate it 413 to network models towards the domain controller, which in turn 414 translate it to the device model. See [RFC8309] for more details. 416 | L3SM 417 V 418 +--------------------+ 419 | Super Controller | 420 +--------------------+ 421 | 422 +-------------------------------+ 423 | | 424 V V 425 +--------+ +--------+ 426 | DoCo#1 | | DoCo#2 | 427 | | | | 428 +--------+ +--------+ 430 CE CE 431 \ AS 100 AS 200 / 432 \ / 433 A----B----C----ASBR1------ASBR2----D----E----F 434 / / / / / / / / 435 / / / / / / / / 436 CE----G----H----I----ASBR3------ASBR4----J----K----L------CE 438 Figure 4: L3VPN 440 Based on the user data in L3SM model, the network configurations need 441 to be trickle down to the network device to setup the L3VPN. 443 [I-D.ietf-opsawg-l3sm-l3nm] describes the need for a YANG model for 444 use between the entity that interacts directly with the customer 445 (service orchestrator) and the entity in charge of network 446 orchestration and control which, according to [RFC8309], can be 447 referred as Service Delivery Model. The resulting model is called 448 the L3VPN Network Model (L3NM). 450 Based on the QoS or Policy requirement for the L3VPN service, the 451 Super Controller may - 453 o Set the tunnel selection policy at the PE/ASBR routers so that 454 they could select the existing tunnels 456 o Select an existing tunnels at the controller level and bind it to 457 the VPN service 459 o Initiate the process of creating a new tunnel based on the QoS 460 requirement and bind it the VPN service 462 o Initiate the process of creating a new tunnel based on the the 463 policy 465 Refer [I-D.ietf-teas-te-service-mapping-yang] for more details from 466 ACTN perspective. 468 Apart from the Management plane interface based on respective YANG 469 models, the control plane interface PCEP could be used for path 470 computation and setup. 472 4.3. L2VPN and EVPN service 474 There are two fundamentally different kinds of Layer 2 VPN service 475 that a service provider could offer to a customer: Virtual Private 476 Wire Service (VPWS) and Virtual Private LAN Service (VPLS) [RFC4664]. 477 A VPWS is a VPN service that supplies an L2 point-to-point service. 478 A VPLS is an L2 service that emulates LAN service across a Wide Area 479 Network (WAN). A BGP MPLS-based Ethernet VPN (EVPN) [RFC7432] 480 addresses some of the limitations when it comes to multihoming and 481 redundancy, multicast optimization, provisioning simplicity, flow- 482 based load balancing, and multipathing etc. 484 The handling of L2VPN/EVPN service is done in a similar fashion as 485 shown in Section 4.2. 487 5. Possible Features/Extensions 489 This sections list some of the possible features or protocol 490 extensions that could be worked on to deploy HIC in a multi-domain 491 packet network. 493 1. Simplify the initial configurations needed to set-up the 494 relationship between the super controller and the domain 495 controllers. Note that this could be done via exchanges during 496 initial session establishment, discovery via other protocols, 497 service discovery (such as DNS etc.). 499 2. The (higher-level controller, lower-level controller) 500 relationship or the the role of the controller. 502 3. The learning and handling of various capabilities of the Super 503 Controller and Domain Controller. 505 4. Handling of multiple instances of controller at each level for 506 high availability. 508 [Editor's Note - This list is expected to be updated in next version 509 with more details] 511 6. Other Considerations 513 6.1. Control Plane 515 6.1.1. PCE / PCEP 517 The Path Computation Element communication Protocol (PCEP) [RFC5440] 518 provides mechanisms for Path Computation Elements (PCEs) [RFC4655] to 519 perform path computations in response to Path Computation Clients 520 (PCCs) requests. 522 The ability to compute shortest constrained TE LSPs in Multiprotocol 523 Label Switching (MPLS) and Generalized MPLS (GMPLS) networks across 524 multiple domains has been identified as a key motivation for PCE 525 development. 527 A stateful PCE [RFC8231] is capable of considering, for the purposes 528 of path computation, not only the network state in terms of links and 529 nodes (referred to as the Traffic Engineering Database or TED) but 530 also the status of active services (previously computed paths, and 531 currently reserved resources, stored in the Label Switched Paths 532 Database (LSPDB). 534 [RFC8051] describes general considerations for a stateful PCE 535 deployment and examines its applicability and benefits, as well as 536 its challenges and limitations through a number of use cases. 538 [RFC8231] describes a set of extensions to PCEP to provide stateful 539 control. A stateful PCE has access to not only the information 540 carried by the network's Interior Gateway Protocol (IGP), but also 541 the set of active paths and their reserved resources for its 542 computations. The additional state allows the PCE to compute 543 constrained paths while considering individual LSPs and their 544 interactions. [RFC8281] describes the setup, maintenance and 545 teardown of PCE-initiated LSPs under the stateful PCE model. 547 [RFC8231] also describes the active stateful PCE. The active PCE 548 functionality allows a PCE to reroute an existing LSP or make changes 549 to the attributes of an existing LSP, or a PCC to delegate control of 550 specific LSPs to a new PCE. 552 Computing paths across large multi-domain environments require 553 special computational components and cooperation between entities in 554 different domains capable of complex path computation. The PCE 555 provides an architecture and a set of functional components to 556 address this problem space. A PCE may be used to compute end-to-end 557 paths across multi-domain environments using a per-domain path 558 computation technique [RFC5152]. The Backward recursive PCE based 559 path computation (BRPC) mechanism [RFC5441] defines a PCE-based path 560 computation procedure to compute inter-domain constrained MPLS and 561 GMPLS TE networks. However, both per-domain and BRPC techniques 562 assume that the sequence of domains to be crossed from source to 563 destination is known, either fixed by the network operator or 564 obtained by other means. 566 [RFC6805] describes a Hierarchical PCE (H-PCE) architecture which can 567 be used for computing end-to-end paths for inter-domain MPLS Traffic 568 Engineering (TE) and GMPLS Label Switched Paths (LSPs) when the 569 domain sequence is not known. Within the Hierarchical PCE (H-PCE) 570 architecture, the Parent PCE (P-PCE) is used to compute a multi- 571 domain path based on the domain connectivity information. A Child 572 PCE (C-PCE) may be responsible for a single domain or multiple 573 domains, it is used to compute the intra-domain path based on its 574 domain topology information. 576 [RFC8751] state the considerations for stateful PCE(s) in 577 hierarchical PCE architecture. In particular, the behaviour changes 578 and additions to the existing stateful PCE mechanisms (including PCE- 579 initiated LSP set-up and active PCE usage) in the context of networks 580 using the H-PCE architecture. 582 [RFC8637] examines the applicability of PCE/PCEP to the ACTN 583 framework in detail. 585 [RFC8283] introduces the architecture for PCE as a central controller 586 as an extension of the architecture described in [RFC4655] and 587 assumes the continued use of PCEP as the protocol used between PCE 588 and PCC. Some related extension to PCEP [I-D.ietf-pce-pcep-flowspec] 589 and [I-D.ietf-pce-pcep-extension-for-pce-controller] are also 590 applicable in HIC. 592 6.1.2. BGP 594 [RFC7752] describes a mechanism by which link-state and TE 595 information can be collected from networks and shared with external 596 components using the BGP routing protocol. This is achieved using a 597 new BGP Network Layer Reachability Information (NLRI) encoding format 598 and a new BGP path attribute (BGP-LS attribute) that carries link, 599 node, and prefix parameters and attributes. 601 BGP-LS is a new approach to collect the network topology information. 602 It is an extension to BGP for distribution the network's link-state 603 (LS) topology to external entities, such as the SDN controller. 604 Network's link-state topology consists of nodes and links and a set 605 of attributes. The link-state topology is distributed among the IGP 606 domain. The specific protocol used in an IGP domain could be OSPF 607 [RFC2238] or IS-IS [ISO10589]. Note that, the detailed link-state 608 models of these two protocols are not identical. Therefore, BGP-LS 609 can provide a more abstract topology model which can map the IGP 610 models. 612 The domain controller acts as a consumer to collect the domain's 613 link-state and TE information via BGP-LS. The domain controller 614 would usually abstract the domain information towards the super- 615 controller and further send it via BGP-LS. 617 BGP-Flowspec is a solution devised for preventing distributed Denial- 618 of-service (DDoS) attack. BGP-Flowspec distributes specification 619 rules into neighbours. [RFC5575] defines a new BGP NLRI encoding 620 format that can be used to distribute traffic flow specifications. 621 Additionally, it defines two applications of that encoding format: 622 one that can be used to automate inter-domain coordination of traffic 623 filtering, such as what is required in order to mitigate DDoS 624 attacks; and a second application to provide traffic filtering in the 625 context of BGP/MPLS VPN service. 627 The IP domain controller can act as the traffic sampling node. The 628 super controller can act as the traffic analysis server. When the 629 super controller finds the attack happened, the super controller 630 should distribute the flow rules to associated IP domain controllers. 631 And each IP domain controller should distribute the flow rules into 632 the ingress routers. Additionally one of the actions taken could be 633 "redirect" where flow could be redirected to the TE tunnels created 634 by the controller. 636 [I-D.luo-grow-bgp-controller-based-ts] describes the traffic steering 637 based on BGP controller. The traditional method for traffic steering 638 depends on static configuration which is time consuming and 639 inefficient. With the hierarchical IP controller, the IP domain 640 controller can have the domain network topology view and routing 641 information while the super controller can have the global network 642 topology view and routing information. The super controller can 643 compute the end-to-end paths to satisfy the differentiated service 644 requirement. The IP domain controller may be used to distribute the 645 routing policy into the routers. BGP policy varies in many aspects. 646 Its goal is to meet the customer application and connectivity 647 requirement, and specific service transport needs. So the super BGP 648 controller is responsible for the coordination of multiple domain BGP 649 Policy. And then distribute Policy to related IP domain controller. 650 The IP domain controller is responsible for distributing the policy 651 to its network nodes. 653 [I-D.ietf-idr-rtc-hierarchical-rr] describes the route target (RT) 654 constrain mechanism in the hierarchical route reflection (RR) 655 scenario. [RFC4684] describes the route target constrain mechanism 656 to build a route distribution graph in order to restrict the 657 propagation of Virtual Private Network (VPN) routes. 658 [I-D.ietf-idr-rtc-hierarchical-rr] proposes a solution to address the 659 RT constrain issue in the hierarchical RR scenarios. The super 660 controller corresponding to higher level RR can receive the RT- 661 constrain routes from the lower level RR, which is acted by the IP 662 domain controller. The higher level RR will select one of the 663 received routes as the best route. then it should advertise the best 664 route to all the lower level RR to build the route distribution 665 graph. This fits well with the HIC as described in this document. 667 6.2. Management Plane 669 6.2.1. YANG Models 671 This is an non-exhaustive list of possible yang models developed or 672 in-development that could be used for HIC. 674 Topology: [RFC8345] defines a generic YANG data model for network 675 topology. [I-D.ietf-teas-yang-te-topo] defines a YANG data model 676 for representing, retrieving and manipulating Traffic Engineering 677 (TE) Topologies. 679 Tunnel: [I-D.ietf-teas-yang-te] defines a YANG data model for the 680 configuration and management of Traffic Engineering (TE) 681 interfaces, tunnels and Label Switched Paths (LSPs). 683 L3VPN: The Layer 3 service model (L3SM) is defined in [RFC8299], 684 which is a YANG data model that can be used for communication 685 between customers and network operators and to deliver a Layer 3 686 provider-provisioned VPN service. [I-D.ietf-bess-l3vpn-yang] 687 defines a YANG data model that can be used to configure and manage 688 BGP Layer 3 VPNs at the device. Note that a network configuration 689 model at the Domain Controller level needs to be developed. 691 L2VPN/EVPN: [RFC8466] defines a YANG data model that can be used 692 to configure a Layer 2 Provider Provisioned VPN service. This 693 model is intended to be instantiated at management system to 694 deliver the overall service. [I-D.ietf-bess-l2vpn-yang] and 695 [I-D.ietf-bess-evpn-yang] defines a YANG data model to configure 696 and manage L2VPN and EVPN service respectively. Note that a 697 network configuration model at the Domain Controller level needs 698 to be developed. 700 OAM: TBD 702 BGP Policy: [I-D.ietf-idr-bgp-model] defines a YANG data model 703 that can be used to configure BGP Policy based on data centre, 704 carrier and content provider operational requirements. The model 705 is intended to be vendor-neutral, in order to allow operators to 706 manage BGP configuration in heterogeneous environments with 707 routers supplied by multiple vendors. Note that a network 708 configuration model at the Domain Controller level needs to be 709 developed. 711 BGP Flowspec: [I-D.wu-idr-flowspec-yang-cfg] defines a YANG data 712 model for Flow Specification implementations. The configuration 713 data is described as flow specification rules that can be 714 distributed as BGP NLRI to a network element. The rules can be 715 used to filter Distributed Denial of Service attacks (DDoS) 716 besides other use cases. Note that a network configuration model 717 at the Domain Controller level needs to be developed. 719 [I-D.wu-model-driven-management-virtualization] provides a framework 720 that describes and discusses an architecture for service and network 721 management automation that takes advantage of YANG modeling 722 technologies. This is quite apt for HIC and includes interactions 723 between multiple YANG models as described in 724 [I-D.wu-model-driven-management-virtualization]. 726 [Editor's Note - the above list should be extended.] 728 6.2.2. Protocol Considerations 730 The Network Configuration Protocol (NETCONF) [RFC6241] provides 731 mechanisms to install, manipulate, and delete the configuration of 732 network devices. The RESTCONF [RFC8040] describes an HTTP-based 733 protocol that provides a programmatic interface for accessing data 734 defined in YANG, using the data-store concepts defined in NETCONF. 736 Some other mechanism like gRPC/gNMI could also be used between 737 controllers using the same YANG data models. 739 7. IANA Considerations 741 There are no IANA concerns in this document. 743 8. Security Considerations 745 There are no new security concerns in this document. 747 9. Acknowledgments 749 10. Contributing Authors 751 Dailongfei (Larry) 752 Huawei Technologies, 753 Beijing, China 755 Email: larry.dai@huawei.com 757 11. References 759 11.1. Normative References 761 [RFC8453] Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for 762 Abstraction and Control of TE Networks (ACTN)", RFC 8453, 763 DOI 10.17487/RFC8453, August 2018, 764 . 766 11.2. Informative References 768 [I-D.dhodylee-pce-pcep-ls] 769 Dhody, D., Peng, S., Lee, Y., and D. Ceccarelli, "PCEP 770 extensions for Distribution of Link-State and TE 771 Information", draft-dhodylee-pce-pcep-ls-15 (work in 772 progress), March 2020. 774 [I-D.dugeon-pce-stateful-interdomain] 775 Dugeon, O., Meuric, J., Lee, Y., and D. Ceccarelli, "PCEP 776 Extension for Stateful Inter-Domain Tunnels", draft- 777 dugeon-pce-stateful-interdomain-04 (work in progress), 778 July 2020. 780 [I-D.ietf-bess-evpn-yang] 781 Brissette, P., Shah, H., Hussain, I., Tiruveedhula, K., 782 and J. Rabadan, "Yang Data Model for EVPN", draft-ietf- 783 bess-evpn-yang-07 (work in progress), March 2019. 785 [I-D.ietf-bess-l2vpn-yang] 786 Shah, H., Brissette, P., Chen, I., Hussain, I., Wen, B., 787 and K. Tiruveedhula, "YANG Data Model for MPLS-based 788 L2VPN", draft-ietf-bess-l2vpn-yang-10 (work in progress), 789 July 2019. 791 [I-D.ietf-bess-l3vpn-yang] 792 Jain, D., Patel, K., Brissette, P., Li, Z., Zhuang, S., 793 Liu, X., Haas, J., Esale, S., and B. Wen, "Yang Data Model 794 for BGP/MPLS L3 VPNs", draft-ietf-bess-l3vpn-yang-04 (work 795 in progress), October 2018. 797 [I-D.ietf-idr-bgp-model] 798 Jethanandani, M., Patel, K., Hares, S., and J. Haas, "BGP 799 YANG Model for Service Provider Networks", draft-ietf-idr- 800 bgp-model-09 (work in progress), June 2020. 802 [I-D.ietf-idr-rtc-hierarchical-rr] 803 Dong, J., Chen, M., and R. Raszuk, "Extensions to RT- 804 Constrain in Hierarchical Route Reflection Scenarios", 805 draft-ietf-idr-rtc-hierarchical-rr-03 (work in progress), 806 July 2017. 808 [I-D.ietf-mpls-seamless-mpls] 809 Leymann, N., Decraene, B., Filsfils, C., Konstantynowicz, 810 M., and D. Steinberg, "Seamless MPLS Architecture", draft- 811 ietf-mpls-seamless-mpls-07 (work in progress), June 2014. 813 [I-D.ietf-opsawg-l3sm-l3nm] 814 Barguil, S., Dios, O., Boucadair, M., Munoz, L., and A. 815 Aguado, "A Layer 3 VPN Network YANG Model", draft-ietf- 816 opsawg-l3sm-l3nm-03 (work in progress), April 2020. 818 [I-D.ietf-pce-pcep-extension-for-pce-controller] 819 Li, Z., Peng, S., Negi, M., Zhao, Q., and C. Zhou, "PCEP 820 Procedures and Protocol Extensions for Using PCE as a 821 Central Controller (PCECC) of LSPs", draft-ietf-pce-pcep- 822 extension-for-pce-controller-06 (work in progress), July 823 2020. 825 [I-D.ietf-pce-pcep-flowspec] 826 Dhody, D., Farrel, A., and Z. Li, "PCEP Extension for Flow 827 Specification", draft-ietf-pce-pcep-flowspec-09 (work in 828 progress), June 2020. 830 [I-D.ietf-teas-actn-yang] 831 Lee, Y., Zheng, H., Ceccarelli, D., Yoon, B., Dios, O., 832 Shin, J., and S. Belotti, "Applicability of YANG models 833 for Abstraction and Control of Traffic Engineered 834 Networks", draft-ietf-teas-actn-yang-05 (work in 835 progress), February 2020. 837 [I-D.ietf-teas-te-service-mapping-yang] 838 Lee, Y., Dhody, D., Fioccola, G., WU, Q., Ceccarelli, D., 839 and J. Tantsura, "Traffic Engineering (TE) and Service 840 Mapping Yang Model", draft-ietf-teas-te-service-mapping- 841 yang-03 (work in progress), March 2020. 843 [I-D.ietf-teas-yang-path-computation] 844 Busi, I., Belotti, S., Lopezalvarez, V., Sharma, A., and 845 Y. Shi, "Yang model for requesting Path Computation", 846 draft-ietf-teas-yang-path-computation-10 (work in 847 progress), July 2020. 849 [I-D.ietf-teas-yang-te] 850 Saad, T., Gandhi, R., Liu, X., Beeram, V., and I. Bryskin, 851 "A YANG Data Model for Traffic Engineering Tunnels, Label 852 Switched Paths and Interfaces", draft-ietf-teas-yang-te-24 853 (work in progress), July 2020. 855 [I-D.ietf-teas-yang-te-topo] 856 Liu, X., Bryskin, I., Beeram, V., Saad, T., Shah, H., and 857 O. Dios, "YANG Data Model for Traffic Engineering (TE) 858 Topologies", draft-ietf-teas-yang-te-topo-22 (work in 859 progress), June 2019. 861 [I-D.luo-grow-bgp-controller-based-ts] 862 Luo, Y., Ou, L., Huang, X., Zhuang, S., and Z. Li, 863 "Traffic Steering Based on BGP Controller", draft-luo- 864 grow-bgp-controller-based-ts-00 (work in progress), March 865 2018. 867 [I-D.wu-idr-flowspec-yang-cfg] 868 Wu, N., Zhuang, S., and A. Choudhary, "A YANG Data Model 869 for Flow Specification", draft-wu-idr-flowspec-yang-cfg-02 870 (work in progress), October 2015. 872 [I-D.wu-model-driven-management-virtualization] 873 WU, Q., Boucadair, M., Lopez, D., Xie, C., and L. Geng, "A 874 Framework for Automating Service and Network Management 875 with YANG", draft-wu-model-driven-management- 876 virtualization-07 (work in progress), October 2019. 878 [ISO10589] 879 ISO, "Intermediate system to Intermediate system routing 880 information exchange protocol for use in conjunction with 881 the Protocol for providing the Connectionless-mode Network 882 Service (ISO 8473)", ISO/IEC 10589:2002, 1992. 884 [RFC2238] Clouston, B., Ed. and B. Moore, Ed., "Definitions of 885 Managed Objects for HPR using SMIv2", RFC 2238, 886 DOI 10.17487/RFC2238, November 1997, 887 . 889 [RFC3630] Katz, D., Kompella, K., and D. Yeung, "Traffic Engineering 890 (TE) Extensions to OSPF Version 2", RFC 3630, 891 DOI 10.17487/RFC3630, September 2003, 892 . 894 [RFC4110] Callon, R. and M. Suzuki, "A Framework for Layer 3 895 Provider-Provisioned Virtual Private Networks (PPVPNs)", 896 RFC 4110, DOI 10.17487/RFC4110, July 2005, 897 . 899 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 900 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 901 DOI 10.17487/RFC4271, January 2006, 902 . 904 [RFC4456] Bates, T., Chen, E., and R. Chandra, "BGP Route 905 Reflection: An Alternative to Full Mesh Internal BGP 906 (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006, 907 . 909 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 910 Element (PCE)-Based Architecture", RFC 4655, 911 DOI 10.17487/RFC4655, August 2006, 912 . 914 [RFC4664] Andersson, L., Ed. and E. Rosen, Ed., "Framework for Layer 915 2 Virtual Private Networks (L2VPNs)", RFC 4664, 916 DOI 10.17487/RFC4664, September 2006, 917 . 919 [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 920 R., Patel, K., and J. Guichard, "Constrained Route 921 Distribution for Border Gateway Protocol/MultiProtocol 922 Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual 923 Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, 924 November 2006, . 926 [RFC5152] Vasseur, JP., Ed., Ayyangar, A., Ed., and R. Zhang, "A 927 Per-Domain Path Computation Method for Establishing Inter- 928 Domain Traffic Engineering (TE) Label Switched Paths 929 (LSPs)", RFC 5152, DOI 10.17487/RFC5152, February 2008, 930 . 932 [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic 933 Engineering", RFC 5305, DOI 10.17487/RFC5305, October 934 2008, . 936 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 937 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 938 DOI 10.17487/RFC5440, March 2009, 939 . 941 [RFC5441] Vasseur, JP., Ed., Zhang, R., Bitar, N., and JL. Le Roux, 942 "A Backward-Recursive PCE-Based Computation (BRPC) 943 Procedure to Compute Shortest Constrained Inter-Domain 944 Traffic Engineering Label Switched Paths", RFC 5441, 945 DOI 10.17487/RFC5441, April 2009, 946 . 948 [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., 949 and D. McPherson, "Dissemination of Flow Specification 950 Rules", RFC 5575, DOI 10.17487/RFC5575, August 2009, 951 . 953 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 954 and A. Bierman, Ed., "Network Configuration Protocol 955 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 956 . 958 [RFC6805] King, D., Ed. and A. Farrel, Ed., "The Application of the 959 Path Computation Element Architecture to the Determination 960 of a Sequence of Domains in MPLS and GMPLS", RFC 6805, 961 DOI 10.17487/RFC6805, November 2012, 962 . 964 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 965 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 966 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 967 2015, . 969 [RFC7491] King, D. and A. Farrel, "A PCE-Based Architecture for 970 Application-Based Network Operations", RFC 7491, 971 DOI 10.17487/RFC7491, March 2015, 972 . 974 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 975 S. Ray, "North-Bound Distribution of Link-State and 976 Traffic Engineering (TE) Information Using BGP", RFC 7752, 977 DOI 10.17487/RFC7752, March 2016, 978 . 980 [RFC7854] Scudder, J., Ed., Fernando, R., and S. Stuart, "BGP 981 Monitoring Protocol (BMP)", RFC 7854, 982 DOI 10.17487/RFC7854, June 2016, 983 . 985 [RFC7926] Farrel, A., Ed., Drake, J., Bitar, N., Swallow, G., 986 Ceccarelli, D., and X. Zhang, "Problem Statement and 987 Architecture for Information Exchange between 988 Interconnected Traffic-Engineered Networks", BCP 206, 989 RFC 7926, DOI 10.17487/RFC7926, July 2016, 990 . 992 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 993 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 994 . 996 [RFC8051] Zhang, X., Ed. and I. Minei, Ed., "Applicability of a 997 Stateful Path Computation Element (PCE)", RFC 8051, 998 DOI 10.17487/RFC8051, January 2017, 999 . 1001 [RFC8231] Crabbe, E., Minei, I., Medved, J., and R. Varga, "Path 1002 Computation Element Communication Protocol (PCEP) 1003 Extensions for Stateful PCE", RFC 8231, 1004 DOI 10.17487/RFC8231, September 2017, 1005 . 1007 [RFC8281] Crabbe, E., Minei, I., Sivabalan, S., and R. Varga, "Path 1008 Computation Element Communication Protocol (PCEP) 1009 Extensions for PCE-Initiated LSP Setup in a Stateful PCE 1010 Model", RFC 8281, DOI 10.17487/RFC8281, December 2017, 1011 . 1013 [RFC8283] Farrel, A., Ed., Zhao, Q., Ed., Li, Z., and C. Zhou, "An 1014 Architecture for Use of PCE and the PCE Communication 1015 Protocol (PCEP) in a Network with Central Control", 1016 RFC 8283, DOI 10.17487/RFC8283, December 2017, 1017 . 1019 [RFC8299] Wu, Q., Ed., Litkowski, S., Tomotaki, L., and K. Ogaki, 1020 "YANG Data Model for L3VPN Service Delivery", RFC 8299, 1021 DOI 10.17487/RFC8299, January 2018, 1022 . 1024 [RFC8309] Wu, Q., Liu, W., and A. Farrel, "Service Models 1025 Explained", RFC 8309, DOI 10.17487/RFC8309, January 2018, 1026 . 1028 [RFC8345] Clemm, A., Medved, J., Varga, R., Bahadur, N., 1029 Ananthakrishnan, H., and X. Liu, "A YANG Data Model for 1030 Network Topologies", RFC 8345, DOI 10.17487/RFC8345, March 1031 2018, . 1033 [RFC8466] Wen, B., Fioccola, G., Ed., Xie, C., and L. Jalil, "A YANG 1034 Data Model for Layer 2 Virtual Private Network (L2VPN) 1035 Service Delivery", RFC 8466, DOI 10.17487/RFC8466, October 1036 2018, . 1038 [RFC8637] Dhody, D., Lee, Y., and D. Ceccarelli, "Applicability of 1039 the Path Computation Element (PCE) to the Abstraction and 1040 Control of TE Networks (ACTN)", RFC 8637, 1041 DOI 10.17487/RFC8637, July 2019, 1042 . 1044 [RFC8751] Dhody, D., Lee, Y., Ceccarelli, D., Shin, J., and D. King, 1045 "Hierarchical Stateful Path Computation Element (PCE)", 1046 RFC 8751, DOI 10.17487/RFC8751, March 2020, 1047 . 1049 Authors' Addresses 1051 Zhenbin Li 1052 Huawei Technologies 1053 Huawei Bld., No.156 Beiqing Rd. 1054 Beijing 100095 1055 China 1057 EMail: lizhenbin@huawei.com 1059 Dhruv Dhody 1060 Huawei Technologies 1061 Divyashree Techno Park, Whitefield 1062 Bangalore, Karnataka 560066 1063 India 1065 EMail: dhruv.ietf@gmail.com 1066 Huaimo Chen 1067 Futurewei Technologies 1068 Boston, MA 1069 USA 1071 EMail: huaimo.chen@futurewei.com