idnits 2.17.1 draft-li-teas-hierarchy-ip-controllers-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (7 March 2022) is 780 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-27) exists of draft-dhodylee-pce-pcep-ls-23 == Outdated reference: A later version (-17) exists of draft-ietf-idr-bgp-model-13 == Outdated reference: A later version (-04) exists of draft-ietf-idr-rtc-hierarchical-rr-03 == Outdated reference: A later version (-04) exists of draft-ietf-pce-stateful-interdomain-03 == Outdated reference: A later version (-11) exists of draft-ietf-teas-actn-yang-08 == Outdated reference: A later version (-15) exists of draft-ietf-teas-te-service-mapping-yang-09 == Outdated reference: A later version (-22) exists of draft-ietf-teas-yang-path-computation-17 == Outdated reference: A later version (-36) exists of draft-ietf-teas-yang-te-29 -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 9 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TEAS Working Group Z. Li 3 Internet-Draft D. Dhody 4 Intended status: Informational Huawei Technologies 5 Expires: 8 September 2022 H. Chen 6 Futurewei Technologies 7 7 March 2022 9 Hierarchy of IP Controllers (HIC) 10 draft-li-teas-hierarchy-ip-controllers-09 12 Abstract 14 This document describes the interactions between various IP 15 controllers in a hierarchical fashion to provide various IP services. 16 It describes how the Abstraction and Control of Traffic Engineered 17 Networks (ACTN) framework is applied to the Hierarchy of IP 18 controllers (HIC) as well as document the interactions with other 19 protocols like BGP, Path Computation Element Communication Protocol 20 (PCEP) to provide end to end dynamic services spanning multiple 21 domains and controllers (e.g. Layer 3 Virtual Private Network 22 (L3VPN), Seamless MPLS etc). 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on 8 September 2022. 41 Copyright Notice 43 Copyright (c) 2022 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 48 license-info) in effect on the date of publication of this document. 49 Please review these documents carefully, as they describe your rights 50 and restrictions with respect to this document. Code Components 51 extracted from this document must include Revised BSD License text as 52 described in Section 4.e of the Trust Legal Provisions and are 53 provided without warranty as described in the Revised BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 2. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4 59 2.1. Mapping to ACTN . . . . . . . . . . . . . . . . . . . . . 5 60 2.2. Interface between Super Controller and Domain Controller in 61 HIC . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 62 3. Key Concepts . . . . . . . . . . . . . . . . . . . . . . . . 6 63 3.1. Topology . . . . . . . . . . . . . . . . . . . . . . . . 6 64 3.2. Path Computation/Path instantiation . . . . . . . . . . . 7 65 3.3. BGP considerations . . . . . . . . . . . . . . . . . . . 8 66 4. VPN Service . . . . . . . . . . . . . . . . . . . . . . . . . 8 67 4.1. Seamless MPLS . . . . . . . . . . . . . . . . . . . . . . 8 68 4.2. L3VPN . . . . . . . . . . . . . . . . . . . . . . . . . . 10 69 4.3. L2VPN and EVPN service . . . . . . . . . . . . . . . . . 11 70 5. Possible Features/Extensions . . . . . . . . . . . . . . . . 11 71 6. Other Considerations . . . . . . . . . . . . . . . . . . . . 12 72 6.1. Control Plane . . . . . . . . . . . . . . . . . . . . . . 12 73 6.1.1. PCE / PCEP . . . . . . . . . . . . . . . . . . . . . 12 74 6.1.2. BGP . . . . . . . . . . . . . . . . . . . . . . . . . 14 75 6.2. Management Plane . . . . . . . . . . . . . . . . . . . . 15 76 6.2.1. YANG Models . . . . . . . . . . . . . . . . . . . . . 15 77 6.2.2. Protocol Considerations . . . . . . . . . . . . . . . 17 78 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 79 8. Security Considerations . . . . . . . . . . . . . . . . . . . 17 80 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 17 81 10. Contributing Authors . . . . . . . . . . . . . . . . . . . . 17 82 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 83 11.1. Normative References . . . . . . . . . . . . . . . . . . 17 84 11.2. Informative References . . . . . . . . . . . . . . . . . 17 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 87 1. Introduction 89 Software-Defined Networking (SDN) refers to a separation between the 90 control elements and the forwarding components so that software 91 running in a centralized system called a controller, can act to 92 program the devices in the network to behave in specific ways. A 93 required element in an SDN architecture is a component that plans how 94 the network resources will be used and how the devices will be 95 programmed. It is possible to view this component as performing 96 specific computations to place flows within the network given 97 knowledge of the availability of network resources, how other 98 forwarding devices are programmed, and the way that other flows are 99 routed. The Application-Based Network Operation (ABNO) [RFC7491] 100 describes how various components and technologies fit together. 102 A domain [RFC4655] is any collection of network elements within a 103 common sphere of address management or path computation 104 responsibility. Specifically, within this document, it means a part 105 of an operator's network that is under common management. Network 106 elements will often be grouped into domains based on technology 107 types, vendor profiles, and geographic proximity and under a domain 108 controller. 110 Multiple such domains in the network are interconnected, and a path 111 is established through a series of connected domains to form an end- 112 to-end path over which various services are offered. Each domain is 113 under the control of the domain controller (or lower-level 114 controller), and a "super controller" (or high-level controller) 115 takes responsibility for a high-level view of the network before 116 distributing tasks to domain controllers (or lower-level 117 controllers). It is possible for each of the domain to use a 118 different tunnelling mechanism (eg RSVP-TE, Segment Routing (SR) 119 etc). 121 [RFC8453] describes the framework for Abstraction and Control of 122 Traffic Engineered Networks (ACTN) as well as a set of management and 123 control functions used to operate multiple TE networks. This 124 documents would apply the ACTN principles to the Hierarchy of IP 125 controllers (HIC) and focus on the applicability and interactions 126 with other protocols and technologies (specific to IP packet 127 domains). 129 Sometimes, service (such as Layer 3 Virtual Private Network (L3VPN), 130 Layer 2 Virtual Private Network (L2VPN), Ethernet VPN (EVPN), 131 Seamless MPLS) require sites attached to different domains (under the 132 control of different domain controller) to be interconnected as part 133 of the VPN service. This requires multi-domain coordination between 134 domain controllers to compute and set-up an E2E path for the VPN 135 service. 137 This document describes the interactions between various IP 138 controllers in a hierarchical fashion to provide various IP services. 139 It describes how the Abstraction and Control of Traffic Engineered 140 Networks (ACTN) framework is applied to the Hierarchy of IP 141 controllers (HIC) as well as document the interactions with control 142 plane protocols (like BGP, Path Computation Element Communication 143 Protocol (PCEP)) and management plane aspects (Yang models) to 144 provide end to end dynamic services spanning multiple domains and 145 controllers (e.g. L3VPN, Seamless MPLS, etc.). 147 2. Overview 149 Figure 1 show examples of multi-domain IP domains under the hierarchy 150 of IP controllers. 152 | 153 +------------+ 154 | SuperCo | 155 +------------+ 156 | 157 ---------------------------------- 158 | | | 159 +------------+ +------------+ +------------+ 160 | DoCo#1 | | DoCo#2 | | DoCo#3 | 161 +------------+ +------------+ +------------+ 163 +--Domain#1--+ +--Domain#2--+ +--Domain#3--+ 164 | | | | | | 165 | B------+---+---D-----E--+---+------J | 166 | / | | \ / | | \ | 167 | / | | \ / | | \ | 168 | A | | H | | L | 169 | \ | | / \ | | / | 170 | \ | | / \ | | / | 171 | C------+---+---F-----G--+---+------K | 172 | | | | | | 173 +------------+ +------------+ +------------+ 175 Figure 1: Example: Hierarchy of IP controllers (HIC) 177 The IP "Super Controller" receives a request from the network/service 178 orchestrator to set-up dynamic services spanning multiple domains. 179 The IP "Super Controller" breaks down and assigns tasks to the domain 180 controllers, responsible for communicating to network devices in the 181 domain. It further coordinates between the controller to provide a 182 unified view of the multi-domain network. 184 2.1. Mapping to ACTN 186 As per [RFC8453], ACTN has following the main functions - 188 * Multi-domain coordination 190 * Virtualization/Abstraction 192 * Customer mapping/translation 194 * Virtual service coordination 196 These functions are part of Multi-Domain Service Coordinator (MDSC) 197 and/or Provisioning Network Controller (PNC). Further, these 198 functions are part of the controller/orchestrator. 200 The HIC is an instantiation of the ACTN framework for the IP packet 201 network. The IP domain (lower-level) controllers implement the PNC 202 functionalities for configuring, controlling, and monitoring the IP 203 domain. The "super controller" (high-level controller) implements 204 the MDSC functionalities for coordination between multiple domains as 205 well as maintaining an abstracted view of multiple domains. It also 206 takes care of the service-related functionalities of the customer- 207 mapping/translation and virtual service coordination. 209 The ACTN functions are part of the IP controllers and responsible for 210 the TE topology and E2E path computation/set-up. There are other 211 functions along with ACTN that are needed to manage multiple IP 212 domain networks. 214 2.2. Interface between Super Controller and Domain Controller in HIC 216 The interaction between super controller and the domain controllers 217 in HIC is a combination of Control Plane and Management Plane 218 interface as shown in Figure 2. BGP [RFC4271] and PCEP [RFC5440] are 219 example of the control plane interface; whereas NETCONF [RFC6241] and 220 RESTCONF [RFC8040] are examples of the management plane interface. 222 +----------------------------------------------+ 223 | Super Controller | 224 | | 225 | | 226 +------------------*------#---------------------+ 227 * # 228 * # 229 ************************* 230 * # * 231 ######*############### * 232 # * # * 233 +---------#-----*--+ +--#--------*------+ 234 | Domain | | Domain | 235 | Controller | | Controller | 236 +--#------------*--+ +--#------------*--+ 237 # * # * 238 # * # * 240 * -> Control Plane Interface 241 # -> Management Plane Interface 243 Figure 2: Interface between Super Controller and Domain Controller 245 Note that ACTN's MDSC-PNC Interface (MPI) could be implemented via 246 management plane interface using Yang models 247 [I-D.ietf-teas-actn-yang] or via PCEP control plane interface 248 [RFC8637]. 250 3. Key Concepts 252 3.1. Topology 254 The Domain Controller is expected to be aware of the topology of the 255 network devices in its domain. The domain controller could 256 participate in the IGP ([RFC3630] and [RFC5305]) or use BGP-LS 257 [RFC7752] by which link-state and TE information is collected and 258 shared with the domain controller using the BGP routing protocol. 260 An alternate approach would be to rely on the management plane 261 interface which uses the YANG model for network/TE Topology as per 262 [RFC8345] and [RFC8795]. 264 The domain controller is expected to share the domain topology to the 265 Super Controller, as per ACTN (where PNC abstract the topology 266 towards MDSC). A level of abstraction is usually applied while 267 presenting the topology to a higher-level controller. Topology 268 abstraction is described in [RFC7926] as well as [RFC8453]. BGP-LS, 269 PCEP-LS [I-D.dhodylee-pce-pcep-ls] or management plane interface 270 based on the abstracted network/TE Topology could be used to carry 271 the abstract topology to the super-controller. At minimum, the 272 border nodes and inter-domain links are exposed to the super- 273 controller. 275 Further [RFC8453] defines three types of topology abstraction - (1) 276 Native/White Topology; (2) Black Topology; and (3) Grey Topology. 277 Based on the local policy, the domain controller would share the 278 domain topology to the Super Controller based on the abstraction 279 type. Note that any of the control plane or management plane 280 mechanism could be used to carry abstracted domain topology. The 281 Super Controller's MDSC function is expected to manage a E2E topology 282 by coordinating the abstracted domain topology received from the 283 domain controllers. 285 3.2. Path Computation/Path instantiation 287 The Domain Controller is responsible for computing and setup of path 288 when the source and destination are in the same domain, otherwise the 289 Super Controller coordinates the multi-domain path computation and 290 setup with the help of the domain controller. This is aligned to 291 ACTN. 293 PCEP [RFC5440] provides mechanisms for Path Computation Elements 294 (PCEs) [RFC4655] to perform path computations in response to Path 295 Computation Clients (PCCs) requests. Since then, the role and 296 function of the PCE has grown to allow delegated control [RFC8231] 297 and PCE-initiated use of network resources [RFC8281]. 299 Further, [RFC6805] and [RFC8751] describes a hierarchy of PCE with 300 Parent PCE coordinating multi-domain path computation function 301 between Child PCE(s). This fits well with HIC as described in this 302 document. 304 Note that a management plane interface which uses the YANG model for 305 path computation/setup ([I-D.ietf-teas-yang-path-computation] and 306 [I-D.ietf-teas-yang-te]) could be used in place of PCEP. 308 In case there is a need to stitch per domain tunnels into an E2E 309 tunnel, mechanism are described in 310 [I-D.ietf-pce-stateful-interdomain]. 312 3.3. BGP considerations 314 [RFC4456] describes the concept of route-reflection where a "route 315 reflector" (RR) reflects the routes to avoid full mesh connection 316 between Internal BGP (IBGP) peers. The IP domain controller can play 317 the role of RR in its domain. The super controller can further act 318 as RR to towards the domain controller. 320 BGP can provide routing policies for traffic management, like route 321 preference, AS-path filter policy, IP-prefix filter policy and route 322 aggregation. The controller can distribute these BGP policies into 323 the routers in a single IP domain. For the scenario of multiple 324 domains, the super controller can distribute per BGP Policy into each 325 IP domain controller. Then the IP domain controller trickles down 326 the BGP Policy to the network devices. 328 [RFC8955] describes the concept of BGP Flowspec that can be used to 329 distribute traffic flow specifications. A flow specification is an 330 n-tuple consisting of several matching criteria that can be applied 331 to IP traffic. The controller can originate the flow specifications 332 and disseminate it to the routers. The flow action includes the 333 redirection to a specific TE tunnel. Also, the IP domain controller 334 could be responsible for collecting the flow sample in its domain and 335 the super controller can act as the Flow Analysis Server. 337 [RFC7854] describes the BGP Monitoring Protocol (BMP) to monitor BGP 338 sessions. BMP is used to obtain route views with a flexible way. In 339 the fashion of hierarchical architecture, the IP domain controller 340 can be used as the domain Monitoring Station. Meanwhile, the super 341 controller is responsible for a high-level view of the global network 342 state. 344 4. VPN Service 346 4.1. Seamless MPLS 348 Seamless MPLS [I-D.ietf-mpls-seamless-mpls] describes an architecture 349 which can be used to extend MPLS networks to integrate access and 350 core/aggregation networks into a single MPLS domain.In the seamless 351 MPLS for mobile backhaul, since there are multiple domains including 352 the core network and multiple mobile backhaul networks, for each 353 domain there is a domain controller. In order to implement the end- 354 to-end network service provision, there should be coordination among 355 multiple domain controllers. 357 | 358 | 359 | 360 +----------+ 361 |--------------|Super |---------| 362 | |Controller| | 363 | +----------+ | 364 | | | 365 | | | 366 | | | 367 +------+ +------+ +------+ 368 |----|DoCo |----| |---|DoCo |--| |----|DoCo |---| 369 | |#X1 | | | |#Y | | | |#X2 | | 370 | +------+ | | +------+ | | +------+ | 371 | | | | | | 372 | | | | | | 373 | | | | | | 374 | +----+ +----+ | 375 | ....|ABR1|...........|ABR3|.... | 376 +----+ ..... +----+ +----+ ..... +----+ 377 | PE |... ...| PE | 378 +----+ ..... +----+ 379 ....+----+ +----+ ..... 380 |ABR2|...........|ABR4|.... 381 +----+ +----+ 383 | IGP-X1 | IGP-Y | IGP-X2 | 384 | (MBH) | (Core) | (MBH) | 385 | | | | 386 |-----BGP LSP-----|-----BGP LSP----|------BGP LSP-----| 387 | | | | 388 |---LDP/TE LSP----|----LDP/TE LSP--|-----LDP/TE LSP---| 389 | | | | 391 Figure 3: Seamless MPLS 393 Super Controller is responsible for setting the seamless MPLS 394 service. It should break down the service model to network 395 configuration model [RFC8309] and the domain controller further break 396 it to the device configuration model to the PE/ASBR to make the E2E 397 seamless MPLS service. The selection of appropriate ASBRs and 398 handling of intra-domain tunnels is coordinated by the Super 399 Controller in a similar fashion as shown in Section 4.2. 401 By enabling BGP sessions between Domain Controller and Super 402 Controller, BGP labeled routes can also be learned at Super 403 Controller. As Super Controller is aware of the (abstract) topology, 404 it could make intelligent decisions regarding E2E BGP LSP to optimize 405 based on the overall traffic information. 407 4.2. L3VPN 409 A Layer 3 IP VPN service is a collection of sites that are authorized 410 to exchange traffic between each other over a shared IP 411 infrastructure. [RFC4110] provides a framework for Layer 3 Provider- 412 Provisioned Virtual Private Networks (PPVPNs). [RFC8299] provides a 413 L3VPN service delivery YANG model for PE-based VPNs. The Super 414 controller is expected to implement the L3SM model and translate it 415 to network models towards the domain controller, which in turn 416 translate it to the device model. See [RFC8309] for more details. 418 | L3SM 419 V 420 +--------------------+ 421 | Super Controller | 422 +--------------------+ 423 | 424 +-------------------------------+ 425 | | 426 V V 427 +--------+ +--------+ 428 | DoCo#1 | | DoCo#2 | 429 | | | | 430 +--------+ +--------+ 432 CE CE 433 \ AS 100 AS 200 / 434 \ / 435 A----B----C----ASBR1------ASBR2----D----E----F 436 / / / / / / / / 437 / / / / / / / / 438 CE----G----H----I----ASBR3------ASBR4----J----K----L------CE 440 Figure 4: L3VPN 442 Based on the user data in the L3SM model, the network configurations 443 need to be trickle down to the network device to set up the L3VPN. 445 [RFC9182] describes the need for a YANG model for use between the 446 entity that interacts directly with the customer (service 447 orchestrator) and the entity in charge of network orchestration and 448 control which, according to [RFC8309], can be referred to as Service 449 Delivery Model. The resulting model is called the L3VPN Network 450 Model (L3NM). 452 Based on the QoS or Policy requirement for the L3VPN service, the 453 Super Controller may - 455 * Set the tunnel selection policy at the PE/ASBR routers so that 456 they could select the existing tunnels 458 * Select an existing tunnel at the controller level and bind it to 459 the VPN service 461 * Initiate the process of creating a new tunnel based on the QoS 462 requirement and bind it to the VPN service 464 * Initiate the process of creating a new tunnel based on the policy 466 Refer [I-D.ietf-teas-te-service-mapping-yang] for more details from 467 ACTN perspective. 469 Apart from the Management plane interface based on respective YANG 470 models, the control plane interface PCEP could be used for path 471 computation and setup. 473 4.3. L2VPN and EVPN service 475 There are two fundamentally different kinds of Layer 2 VPN service 476 that a service provider could offer to a customer: Virtual Private 477 Wire Service (VPWS) and Virtual Private LAN Service (VPLS) [RFC4664]. 478 A VPWS is a VPN service that supplies an L2 point-to-point service. 479 A VPLS is an L2 service that emulates LAN service across a Wide Area 480 Network (WAN). A BGP MPLS-based Ethernet VPN (EVPN) [RFC7432] 481 addresses some of the limitations when it comes to multihoming and 482 redundancy, multicast optimization, provisioning simplicity, flow- 483 based load balancing, and multipathing, etc. 485 The handling of L2VPN/EVPN service is done in a similar fashion as 486 shown in Section 4.2. 488 5. Possible Features/Extensions 490 This sections list some of the possible features or protocol 491 extensions that could be worked on to deploy HIC in a multi-domain 492 packet network. 494 1. Simplify the initial configurations needed to set-up the 495 relationship between the super controller and the domain 496 controllers. Note that this could be done via exchanges during 497 initial session establishment, discovery via other protocols, 498 service discovery (such as DNS etc.). 500 2. The (higher-level controller, lower-level controller) 501 relationship or the role of the controller. 503 3. The learning and handling of various capabilities of the Super 504 Controller and Domain Controller. 506 4. Handling of multiple instances of the controller at each level 507 for high availability. 509 [Editor's Note - This list is expected to be updated in the next 510 version with more details] 512 6. Other Considerations 514 6.1. Control Plane 516 6.1.1. PCE / PCEP 518 The Path Computation Element communication Protocol (PCEP) [RFC5440] 519 provides mechanisms for Path Computation Elements (PCEs) [RFC4655] to 520 perform path computations in response to Path Computation Clients 521 (PCCs) requests. 523 The ability to compute shortest constrained TE LSPs in Multiprotocol 524 Label Switching (MPLS) and Generalized MPLS (GMPLS) networks across 525 multiple domains have been identified as a key motivation for PCE 526 development. 528 A stateful PCE [RFC8231] is capable of considering, for the purposes 529 of path computation, not only the network state in terms of links and 530 nodes (referred to as the Traffic Engineering Database or TED) but 531 also the status of active services (previously computed paths, and 532 currently reserved resources, stored in the Label Switched Paths 533 Database (LSPDB). 535 [RFC8051] describes general considerations for a stateful PCE 536 deployment and examines its applicability and benefits, as well as 537 its challenges and limitations through a number of use cases. 539 [RFC8231] describes a set of extensions to PCEP to provide stateful 540 control. A stateful PCE has access to not only the information 541 carried by the network's Interior Gateway Protocol (IGP), but also 542 the set of active paths and their reserved resources for its 543 computations. The additional state allows the PCE to compute 544 constrained paths while considering individual LSPs and their 545 interactions. [RFC8281] describes the setup, maintenance and 546 teardown of PCE-initiated LSPs under the stateful PCE model. 548 [RFC8231] also describes the active stateful PCE. The active PCE 549 functionality allows a PCE to reroute an existing LSP or make changes 550 to the attributes of an existing LSP, or a PCC to delegate control of 551 specific LSPs to a new PCE. 553 Computing paths across large multi-domain environments require 554 special computational components and cooperation between entities in 555 different domains capable of complex path computation. The PCE 556 provides an architecture and a set of functional components to 557 address this problem space. A PCE may be used to compute end-to-end 558 paths across multi-domain environments using a per-domain path 559 computation technique [RFC5152]. The Backward recursive PCE based 560 path computation (BRPC) mechanism [RFC5441] defines a PCE-based path 561 computation procedure to compute inter-domain constrained MPLS and 562 GMPLS TE networks. However, both per-domain and BRPC techniques 563 assume that the sequence of domains to be crossed from source to 564 destination is known, either fixed by the network operator or 565 obtained by other means. 567 [RFC6805] describes a Hierarchical PCE (H-PCE) architecture which can 568 be used for computing end-to-end paths for inter-domain MPLS Traffic 569 Engineering (TE) and GMPLS Label Switched Paths (LSPs) when the 570 domain sequence is not known. Within the Hierarchical PCE (H-PCE) 571 architecture, the Parent PCE (P-PCE) is used to compute a multi- 572 domain path based on the domain connectivity information. A Child 573 PCE (C-PCE) may be responsible for a single domain or multiple 574 domains, it is used to compute the intra-domain path based on its 575 domain topology information. 577 [RFC8751] state the considerations for stateful PCE(s) in 578 hierarchical PCE architecture. In particular, the behaviour changes 579 and additions to the existing stateful PCE mechanisms (including PCE- 580 initiated LSP set-up and active PCE usage) in the context of networks 581 using the H-PCE architecture. 583 [RFC8637] examines the applicability of PCE/PCEP to the ACTN 584 framework in detail. 586 [RFC8283] introduces the architecture for PCE as a central controller 587 as an extension of the architecture described in [RFC4655] and 588 assumes the continued use of PCEP as the protocol used between PCE 589 and PCC. Some related extension to PCEP [RFC9168] and [RFC9050] are 590 also applicable in HIC. 592 6.1.2. BGP 594 [RFC7752] describes a mechanism by which link-state and TE 595 information can be collected from networks and shared with external 596 components using the BGP routing protocol. This is achieved using a 597 new BGP Network Layer Reachability Information (NLRI) encoding format 598 and a new BGP path attribute (BGP-LS attribute) that carries link, 599 node, and prefix parameters and attributes. 601 BGP-LS is another approach to collect network topology information. 602 It is an extension to BGP for distribution of the network's link- 603 state (LS) topology to external entities, such as the SDN controller. 604 Network's link-state topology consists of nodes and links and a set 605 of attributes. The link-state topology is distributed among the IGP 606 domain. The specific protocol used in an IGP domain could be OSPF 607 [RFC2238] or IS-IS [ISO10589]. Note that, the detailed link-state 608 models of these two protocols are not identical. Therefore, BGP-LS 609 can provide a more abstract topology model that can map the IGP 610 models. 612 The domain controller acts as a consumer to collect the domain's 613 link-state and TE information via BGP-LS. The domain controller 614 would usually abstract the domain information towards the super- 615 controller and further send it via BGP-LS. 617 BGP-Flowspec is a solution devised for preventing distributed Denial- 618 of-service (DDoS) attack. BGP-Flowspec distributes specification 619 rules into neighbours. [RFC8955] defines a new BGP NLRI encoding 620 format that can be used to distribute traffic flow specifications. 621 Additionally, it defines two applications of that encoding format: 622 one that can be used to automate inter-domain coordination of traffic 623 filtering, such as what is required in order to mitigate DDoS 624 attacks; and a second application to provide traffic filtering in the 625 context of BGP/MPLS VPN service. 627 The IP domain controller can act as the traffic sampling node. The 628 super controller can act as the traffic analysis server. When the 629 super controller finds the attack happened, the super controller 630 should distribute the flow rules to associated IP domain controllers. 631 And each IP domain controller should distribute the flow rules into 632 the ingress routers. Additionally one of the actions taken could be 633 "redirect" where flow could be redirected to the TE tunnels created 634 by the controller. 636 [I-D.luo-grow-bgp-controller-based-ts] describes the traffic steering 637 based on BGP controller. The traditional method for traffic steering 638 depends on the static configuration which is time-consuming and 639 inefficient. With the hierarchical IP controller, the IP domain 640 controller can have the domain network topology view and routing 641 information while the super controller can have the global network 642 topology view and routing information. The super controller can 643 compute the end-to-end paths to satisfy the differentiated service 644 requirement. The IP domain controller may be used to distribute the 645 routing policy into the routers. BGP policy varies in many aspects. 646 Its goal is to meet the customer application and connectivity 647 requirement, and specific service transport needs. So the super BGP 648 controller is responsible for the coordination of multiple domain BGP 649 Policy. And then distribute Policy to the related IP domain 650 controller. The IP domain controller is responsible for distributing 651 the policy to its network nodes. 653 [I-D.ietf-idr-rtc-hierarchical-rr] describes the route target (RT) 654 constrain mechanism in the hierarchical route reflection (RR) 655 scenario. [RFC4684] describes the route-target constrain mechanism 656 to build a route distribution graph in order to restrict the 657 propagation of Virtual Private Network (VPN) routes. 658 [I-D.ietf-idr-rtc-hierarchical-rr] proposes a solution to address the 659 RT constrain issue in the hierarchical RR scenarios. The super 660 controller corresponding to higher level RR can receive the RT- 661 constrain routes from the lower level RR, which is acted by the IP 662 domain controller. The higher level RR will select one of the 663 received routes as the best route. then it should advertise the best 664 route to all the lower level RR to build the route distribution 665 graph. This fits well with the HIC as described in this document. 667 6.2. Management Plane 669 6.2.1. YANG Models 671 This is a non-exhaustive list of possible yang models developed or 672 in-development that could be used for HIC. 674 Topology: [RFC8345] defines a generic YANG data model for network 675 topology. [RFC8795] defines a YANG data model for representing, 676 retrieving and manipulating Traffic Engineering (TE) Topologies. 678 Tunnel: [I-D.ietf-teas-yang-te] defines a YANG data model for the 679 configuration and management of Traffic Engineering (TE) 680 interfaces, tunnels and Label Switched Paths (LSPs). 682 L3VPN: The Layer 3 service model (L3SM) is defined in [RFC8299], 683 which is a YANG data model that can be used for communication 684 between customers and network operators and to deliver a Layer 3 685 provider-provisioned VPN service. [I-D.ietf-bess-l3vpn-yang] 686 defines a YANG data model that can be used to configure and manage 687 BGP Layer 3 VPNs at the device. Note that a network configuration 688 model at the Domain Controller level needs to be developed. 690 L2VPN/EVPN: [RFC8466] defines a YANG data model that can be used 691 to configure a Layer 2 Provider-Provisioned VPN service. This 692 model is intended to be instantiated at the management system to 693 deliver the overall service. [I-D.ietf-bess-l2vpn-yang] and 694 [I-D.ietf-bess-evpn-yang] defines a YANG data model to configure 695 and manage L2VPN and EVPN service respectively. Note that a 696 network configuration model at the Domain Controller level needs 697 to be developed. 699 OAM: TBD 701 BGP Policy: [I-D.ietf-idr-bgp-model] defines a YANG data model 702 that can be used to configure BGP Policy based on data centre, 703 carrier and content provider operational requirements. The model 704 is intended to be vendor-neutral, in order to allow operators to 705 manage BGP configuration in heterogeneous environments with 706 routers supplied by multiple vendors. Note that a network 707 configuration model at the Domain Controller level needs to be 708 developed. 710 BGP Flowspec: [I-D.wu-idr-flowspec-yang-cfg] defines a YANG data 711 model for Flow Specification implementations. The configuration 712 data is described as flow specification rules that can be 713 distributed as BGP NLRI to a network element. The rules can be 714 used to filter Distributed Denial of Service attacks (DDoS) 715 besides other use cases. Note that a network configuration model 716 at the Domain Controller level needs to be developed. 718 [RFC8969] provides a framework that describes and discusses an 719 architecture for service and network management automation that takes 720 advantage of YANG modeling technologies. This is quite apt for HIC 721 and includes interactions between multiple YANG models as described 722 in [RFC8969]. 724 [Editor's Note - the above list should be extended.] 726 6.2.2. Protocol Considerations 728 The Network Configuration Protocol (NETCONF) [RFC6241] provides 729 mechanisms to install, manipulate, and delete the configuration of 730 network devices. The RESTCONF [RFC8040] describes an HTTP-based 731 protocol that provides a programmatic interface for accessing data 732 defined in YANG, using the datastore concepts defined in NETCONF. 734 Some other mechanism like gRPC/gNMI could also be used between 735 controllers using the same YANG data models. 737 7. IANA Considerations 739 There are no IANA concerns in this document. 741 8. Security Considerations 743 There are no new security concerns in this document. 745 9. Acknowledgments 747 10. Contributing Authors 749 Dailongfei (Larry) 750 Huawei Technologies, 751 Beijing, China 753 Email: larry.dai@huawei.com 755 11. References 757 11.1. Normative References 759 [RFC8453] Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for 760 Abstraction and Control of TE Networks (ACTN)", RFC 8453, 761 DOI 10.17487/RFC8453, August 2018, 762 . 764 11.2. Informative References 766 [I-D.dhodylee-pce-pcep-ls] 767 Dhody, D., Peng, S., Lee, Y., Ceccarelli, D., Wang, A., 768 Mishra, G., and S. Sivabalan, "PCEP extensions for 769 Distribution of Link-State and TE Information", Work in 770 Progress, Internet-Draft, draft-dhodylee-pce-pcep-ls-23, 5 771 March 2022, . 774 [I-D.ietf-bess-evpn-yang] 775 Brissette, P., Shah, H., Hussain, I., Tiruveedhula, K., 776 and J. Rabadan, "Yang Data Model for EVPN", Work in 777 Progress, Internet-Draft, draft-ietf-bess-evpn-yang-07, 11 778 March 2019, . 781 [I-D.ietf-bess-l2vpn-yang] 782 Shah, H., Brissette, P., Chen, I., Hussain, I., Wen, B., 783 and K. Tiruveedhula, "YANG Data Model for MPLS-based 784 L2VPN", Work in Progress, Internet-Draft, draft-ietf-bess- 785 l2vpn-yang-10, 2 July 2019, 786 . 789 [I-D.ietf-bess-l3vpn-yang] 790 Jain, D., Patel, K., Brissette, P., Li, Z., Zhuang, S., 791 Liu, X., Haas, J., Esale, S., and B. Wen, "Yang Data Model 792 for BGP/MPLS L3 VPNs", Work in Progress, Internet-Draft, 793 draft-ietf-bess-l3vpn-yang-05, 13 April 2021, 794 . 797 [I-D.ietf-idr-bgp-model] 798 Jethanandani, M., Patel, K., Hares, S., and J. Haas, "BGP 799 YANG Model for Service Provider Networks", Work in 800 Progress, Internet-Draft, draft-ietf-idr-bgp-model-13, 6 801 March 2022, . 804 [I-D.ietf-idr-rtc-hierarchical-rr] 805 Dong, J., Chen, M., and R. Raszuk, "Extensions to RT- 806 Constrain in Hierarchical Route Reflection Scenarios", 807 Work in Progress, Internet-Draft, draft-ietf-idr-rtc- 808 hierarchical-rr-03, 3 July 2017, 809 . 812 [I-D.ietf-mpls-seamless-mpls] 813 Leymann, N., Decraene, B., Filsfils, C., Konstantynowicz, 814 M., and D. Steinberg, "Seamless MPLS Architecture", Work 815 in Progress, Internet-Draft, draft-ietf-mpls-seamless- 816 mpls-07, 28 June 2014, 817 . 820 [I-D.ietf-pce-stateful-interdomain] 821 Dugeon, O., Meuric, J., Lee, Y., and D. Ceccarelli, "PCEP 822 Extension for Stateful Inter-Domain Tunnels", Work in 823 Progress, Internet-Draft, draft-ietf-pce-stateful- 824 interdomain-03, 4 March 2022, 825 . 828 [I-D.ietf-teas-actn-yang] 829 Lee, Y., Zheng, H., Ceccarelli, D., Yoon, B. Y., and S. 830 Belotti, "Applicability of YANG models for Abstraction and 831 Control of Traffic Engineered Networks", Work in Progress, 832 Internet-Draft, draft-ietf-teas-actn-yang-08, 8 September 833 2021, . 836 [I-D.ietf-teas-te-service-mapping-yang] 837 Lee, Y., Dhody, D., Fioccola, G., Wu, Q., Ceccarelli, D., 838 and J. Tantsura, "Traffic Engineering (TE) and Service 839 Mapping YANG Model", Work in Progress, Internet-Draft, 840 draft-ietf-teas-te-service-mapping-yang-09, 24 October 841 2021, . 844 [I-D.ietf-teas-yang-path-computation] 845 Busi, I., Belotti, S., Dios, O. G. D., Sharma, A., Shi, 846 Y., and D. Ceccarelli, "A YANG Data Model for requesting 847 path computation", Work in Progress, Internet-Draft, 848 draft-ietf-teas-yang-path-computation-17, 7 March 2022, 849 . 852 [I-D.ietf-teas-yang-te] 853 Saad, T., Gandhi, R., Liu, X., Beeram, V. P., Bryskin, I., 854 and O. G. D. Dios, "A YANG Data Model for Traffic 855 Engineering Tunnels, Label Switched Paths and Interfaces", 856 Work in Progress, Internet-Draft, draft-ietf-teas-yang-te- 857 29, 7 February 2022, 858 . 861 [I-D.luo-grow-bgp-controller-based-ts] 862 Luo, Y., Ou, L., Huang, X., Zhuang, S., and Z. Li, 863 "Traffic Steering Based on BGP Controller", Work in 864 Progress, Internet-Draft, draft-luo-grow-bgp-controller- 865 based-ts-00, 5 March 2018, 866 . 869 [I-D.wu-idr-flowspec-yang-cfg] 870 Wu, N., Zhuang, S., and A. Choudhary, "A YANG Data Model 871 for Flow Specification", Work in Progress, Internet-Draft, 872 draft-wu-idr-flowspec-yang-cfg-02, 1 October 2015, 873 . 876 [ISO10589] ISO, "Intermediate system to Intermediate system routing 877 information exchange protocol for use in conjunction with 878 the Protocol for providing the Connectionless-mode Network 879 Service (ISO 8473)", ISO/IEC 10589:2002, 1992. 881 [RFC2238] Clouston, B., Ed. and B. Moore, Ed., "Definitions of 882 Managed Objects for HPR using SMIv2", RFC 2238, 883 DOI 10.17487/RFC2238, November 1997, 884 . 886 [RFC3630] Katz, D., Kompella, K., and D. Yeung, "Traffic Engineering 887 (TE) Extensions to OSPF Version 2", RFC 3630, 888 DOI 10.17487/RFC3630, September 2003, 889 . 891 [RFC4110] Callon, R. and M. Suzuki, "A Framework for Layer 3 892 Provider-Provisioned Virtual Private Networks (PPVPNs)", 893 RFC 4110, DOI 10.17487/RFC4110, July 2005, 894 . 896 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 897 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 898 DOI 10.17487/RFC4271, January 2006, 899 . 901 [RFC4456] Bates, T., Chen, E., and R. Chandra, "BGP Route 902 Reflection: An Alternative to Full Mesh Internal BGP 903 (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006, 904 . 906 [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path 907 Computation Element (PCE)-Based Architecture", RFC 4655, 908 DOI 10.17487/RFC4655, August 2006, 909 . 911 [RFC4664] Andersson, L., Ed. and E. Rosen, Ed., "Framework for Layer 912 2 Virtual Private Networks (L2VPNs)", RFC 4664, 913 DOI 10.17487/RFC4664, September 2006, 914 . 916 [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 917 R., Patel, K., and J. Guichard, "Constrained Route 918 Distribution for Border Gateway Protocol/MultiProtocol 919 Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual 920 Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, 921 November 2006, . 923 [RFC5152] Vasseur, JP., Ed., Ayyangar, A., Ed., and R. Zhang, "A 924 Per-Domain Path Computation Method for Establishing Inter- 925 Domain Traffic Engineering (TE) Label Switched Paths 926 (LSPs)", RFC 5152, DOI 10.17487/RFC5152, February 2008, 927 . 929 [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic 930 Engineering", RFC 5305, DOI 10.17487/RFC5305, October 931 2008, . 933 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 934 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 935 DOI 10.17487/RFC5440, March 2009, 936 . 938 [RFC5441] Vasseur, JP., Ed., Zhang, R., Bitar, N., and JL. Le Roux, 939 "A Backward-Recursive PCE-Based Computation (BRPC) 940 Procedure to Compute Shortest Constrained Inter-Domain 941 Traffic Engineering Label Switched Paths", RFC 5441, 942 DOI 10.17487/RFC5441, April 2009, 943 . 945 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 946 and A. Bierman, Ed., "Network Configuration Protocol 947 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 948 . 950 [RFC6805] King, D., Ed. and A. Farrel, Ed., "The Application of the 951 Path Computation Element Architecture to the Determination 952 of a Sequence of Domains in MPLS and GMPLS", RFC 6805, 953 DOI 10.17487/RFC6805, November 2012, 954 . 956 [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 957 Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based 958 Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 959 2015, . 961 [RFC7491] King, D. and A. Farrel, "A PCE-Based Architecture for 962 Application-Based Network Operations", RFC 7491, 963 DOI 10.17487/RFC7491, March 2015, 964 . 966 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 967 S. Ray, "North-Bound Distribution of Link-State and 968 Traffic Engineering (TE) Information Using BGP", RFC 7752, 969 DOI 10.17487/RFC7752, March 2016, 970 . 972 [RFC7854] Scudder, J., Ed., Fernando, R., and S. Stuart, "BGP 973 Monitoring Protocol (BMP)", RFC 7854, 974 DOI 10.17487/RFC7854, June 2016, 975 . 977 [RFC7926] Farrel, A., Ed., Drake, J., Bitar, N., Swallow, G., 978 Ceccarelli, D., and X. Zhang, "Problem Statement and 979 Architecture for Information Exchange between 980 Interconnected Traffic-Engineered Networks", BCP 206, 981 RFC 7926, DOI 10.17487/RFC7926, July 2016, 982 . 984 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 985 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 986 . 988 [RFC8051] Zhang, X., Ed. and I. Minei, Ed., "Applicability of a 989 Stateful Path Computation Element (PCE)", RFC 8051, 990 DOI 10.17487/RFC8051, January 2017, 991 . 993 [RFC8231] Crabbe, E., Minei, I., Medved, J., and R. Varga, "Path 994 Computation Element Communication Protocol (PCEP) 995 Extensions for Stateful PCE", RFC 8231, 996 DOI 10.17487/RFC8231, September 2017, 997 . 999 [RFC8281] Crabbe, E., Minei, I., Sivabalan, S., and R. Varga, "Path 1000 Computation Element Communication Protocol (PCEP) 1001 Extensions for PCE-Initiated LSP Setup in a Stateful PCE 1002 Model", RFC 8281, DOI 10.17487/RFC8281, December 2017, 1003 . 1005 [RFC8283] Farrel, A., Ed., Zhao, Q., Ed., Li, Z., and C. Zhou, "An 1006 Architecture for Use of PCE and the PCE Communication 1007 Protocol (PCEP) in a Network with Central Control", 1008 RFC 8283, DOI 10.17487/RFC8283, December 2017, 1009 . 1011 [RFC8299] Wu, Q., Ed., Litkowski, S., Tomotaki, L., and K. Ogaki, 1012 "YANG Data Model for L3VPN Service Delivery", RFC 8299, 1013 DOI 10.17487/RFC8299, January 2018, 1014 . 1016 [RFC8309] Wu, Q., Liu, W., and A. Farrel, "Service Models 1017 Explained", RFC 8309, DOI 10.17487/RFC8309, January 2018, 1018 . 1020 [RFC8345] Clemm, A., Medved, J., Varga, R., Bahadur, N., 1021 Ananthakrishnan, H., and X. Liu, "A YANG Data Model for 1022 Network Topologies", RFC 8345, DOI 10.17487/RFC8345, March 1023 2018, . 1025 [RFC8466] Wen, B., Fioccola, G., Ed., Xie, C., and L. Jalil, "A YANG 1026 Data Model for Layer 2 Virtual Private Network (L2VPN) 1027 Service Delivery", RFC 8466, DOI 10.17487/RFC8466, October 1028 2018, . 1030 [RFC8637] Dhody, D., Lee, Y., and D. Ceccarelli, "Applicability of 1031 the Path Computation Element (PCE) to the Abstraction and 1032 Control of TE Networks (ACTN)", RFC 8637, 1033 DOI 10.17487/RFC8637, July 2019, 1034 . 1036 [RFC8751] Dhody, D., Lee, Y., Ceccarelli, D., Shin, J., and D. King, 1037 "Hierarchical Stateful Path Computation Element (PCE)", 1038 RFC 8751, DOI 10.17487/RFC8751, March 2020, 1039 . 1041 [RFC8795] Liu, X., Bryskin, I., Beeram, V., Saad, T., Shah, H., and 1042 O. Gonzalez de Dios, "YANG Data Model for Traffic 1043 Engineering (TE) Topologies", RFC 8795, 1044 DOI 10.17487/RFC8795, August 2020, 1045 . 1047 [RFC8955] Loibl, C., Hares, S., Raszuk, R., McPherson, D., and M. 1048 Bacher, "Dissemination of Flow Specification Rules", 1049 RFC 8955, DOI 10.17487/RFC8955, December 2020, 1050 . 1052 [RFC8969] Wu, Q., Ed., Boucadair, M., Ed., Lopez, D., Xie, C., and 1053 L. Geng, "A Framework for Automating Service and Network 1054 Management with YANG", RFC 8969, DOI 10.17487/RFC8969, 1055 January 2021, . 1057 [RFC9050] Li, Z., Peng, S., Negi, M., Zhao, Q., and C. Zhou, "Path 1058 Computation Element Communication Protocol (PCEP) 1059 Procedures and Extensions for Using the PCE as a Central 1060 Controller (PCECC) of LSPs", RFC 9050, 1061 DOI 10.17487/RFC9050, July 2021, 1062 . 1064 [RFC9168] Dhody, D., Farrel, A., and Z. Li, "Path Computation 1065 Element Communication Protocol (PCEP) Extension for Flow 1066 Specification", RFC 9168, DOI 10.17487/RFC9168, January 1067 2022, . 1069 [RFC9182] Barguil, S., Gonzalez de Dios, O., Ed., Boucadair, M., 1070 Ed., Munoz, L., and A. Aguado, "A YANG Network Data Model 1071 for Layer 3 VPNs", RFC 9182, DOI 10.17487/RFC9182, 1072 February 2022, . 1074 Authors' Addresses 1076 Zhenbin Li 1077 Huawei Technologies 1078 Huawei Bld., No.156 Beiqing Rd. 1079 Beijing 1080 100095 1081 China 1082 Email: lizhenbin@huawei.com 1084 Dhruv Dhody 1085 Huawei Technologies 1086 Divyashree Techno Park, Whitefield 1087 Bangalore 560066 1088 Karnataka 1089 India 1090 Email: dhruv.ietf@gmail.com 1091 Huaimo Chen 1092 Futurewei Technologies 1093 Boston, MA 1094 United States of America 1095 Email: huaimo.chen@futurewei.com