idnits 2.17.1 draft-ietf-teas-scheduled-resources-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 16, 2018) is 2286 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'T0' is mentioned on line 403, but not defined == Missing Reference: 'B0' is mentioned on line 403, but not defined == Missing Reference: 'T1' is mentioned on line 403, but not defined == Missing Reference: 'B1' is mentioned on line 403, but not defined == Missing Reference: 'T2' is mentioned on line 403, but not defined == Missing Reference: 'B2' is mentioned on line 403, but not defined == Missing Reference: 'T3' is mentioned on line 403, but not defined == Missing Reference: 'B3' is mentioned on line 403, but not defined -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TEAS Working Group Y. Zhuang 3 Internet-Draft Q. Wu 4 Intended status: Informational H. Chen 5 Expires: July 20, 2018 Huawei 6 A. Farrel 7 Juniper Networks 8 January 16, 2018 10 Architecture for Scheduled Use of Resources 11 draft-ietf-teas-scheduled-resources-05 13 Abstract 15 Time-scheduled reservation of traffic engineering (TE) resources can 16 be used to provide resource booking for TE Label Switched Paths so as 17 to better guarantee services for customers and to improve the 18 efficiency of network resource usage into the future. This document 19 provides a framework that describes and discusses the architecture 20 for the scheduled reservation of TE resources. This document does 21 not describe specific protocols or protocol extensions needed to 22 realize this service. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on July 20, 2018. 41 Copyright Notice 43 Copyright (c) 2018 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 2. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 3 60 2.1. Provisioning TE-LSPs and TE Resources . . . . . . . . . . 3 61 2.2. Selecting the Path of an LSP . . . . . . . . . . . . . . 4 62 2.3. Planning Future LSPs . . . . . . . . . . . . . . . . . . 4 63 2.4. Looking at Future Demands on TE Resources . . . . . . . . 5 64 2.5. Requisite State Information . . . . . . . . . . . . . . . 5 65 3. Architectural Concepts . . . . . . . . . . . . . . . . . . . 6 66 3.1. Where is Scheduling State Held? . . . . . . . . . . . . . 6 67 3.2. What State is Held? . . . . . . . . . . . . . . . . . . . 8 68 4. Architecture Overview . . . . . . . . . . . . . . . . . . . . 10 69 4.1. Service Request . . . . . . . . . . . . . . . . . . . . . 10 70 4.1.1. Reoptimization After TED Updates . . . . . . . . . . 11 71 4.2. Initialization and Recovery . . . . . . . . . . . . . . . 12 72 4.3. Synchronization Between PCEs . . . . . . . . . . . . . . 12 73 5. Multi-Domain Considerations . . . . . . . . . . . . . . . . . 13 74 6. Security Considerations . . . . . . . . . . . . . . . . . . . 15 75 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 76 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 16 77 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 16 78 10. Informative References . . . . . . . . . . . . . . . . . . . 16 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 81 1. Introduction 83 Traffic Engineering Label Switched Paths (TE-LSPs) are connection 84 oriented tunnels in packet and non-packet networks [RFC3209], 85 [RFC3945]. TE-LSPs may reserve network resources for use by the 86 traffic they carry, thus providing some guarantees of service 87 delivery and allowing a network operator to plan the use of the 88 resources across the whole network. 90 In some technologies (such as wavelength switched optical networks) 91 the resource is synonymous with the label that is switched on the 92 path of the LSP so that it is not possible to establish an LSP that 93 can carry traffic without assigning a concrete resource to the LSP. 94 In other technologies (such as packet switched networks) the 95 resources assigned to an LSP are a measure of the capacity of a link 96 that is dedicated for use by the traffic on the LSP. 98 In all cases, network planning consists of selecting paths for LSPs 99 through the network so that there will be no contention for 100 resources. LSP establishment is the act of setting up an LSP and 101 reserving resources within the network. Network optimization or re- 102 optimization is the process of re-positioning LSPs in the network to 103 make the unreserved network resources more useful for potential 104 future LSPs while ensuring that the established LSPs continue to 105 fulfill their objectives. 107 It is often the case that it is known that an LSP will be needed at 108 some time in the future. While a path for that LSP could be computed 109 using knowledge of the currently established LSPs and the currently 110 available resources, this does not give any degree of certainty that 111 the necessary resources will be available when it is time to set up 112 the new LSP. Yet setting up the LSP ahead of the time when it is 113 needed (which would guarantee the availability of the resources) is 114 wasteful since the network resources could be used for some other 115 purpose in the meantime. 117 Similarly, it may be known that an LSP will no longer be needed after 118 some future time and that it will be torn down releasing the network 119 resources that were assigned to it. This information can be helpful 120 in planning how a future LSP is placed in the network. 122 Time-Scheduled (TS) reservation of TE resources can be used to 123 provide resource booking for TE-LSPs so as to better guarantee 124 services for customers and to improve the efficiency of network 125 resource usage into the future. This document provides a framework 126 that describes and discusses the architecture for the scheduled 127 reservation of TE resources. This document does not describe 128 specific protocols or protocol extensions needed to realize this 129 service. 131 2. Problem Statement 133 2.1. Provisioning TE-LSPs and TE Resources 135 TE-LSPs in existing networks are provisioned using RSVP-TE as a 136 signaling protocol [RFC3209] [RFC3473], by direct control of network 137 elements such as in the Software Defined Networking (SDN) paradigm, 138 and using the PCE Communication Protocol (PCEP) [RFC5440] as a 139 control protocol. 141 TE resources are reserved at the point of use. That is, the 142 resources (wavelengths, timeslots, bandwidth, etc.) are reserved for 143 use on a specific link and are tracked by the Label Switching Routers 144 (LSRs) at the end points of the link. Those LSRs learn which 145 resources to reserve during the LSP setup process. 147 The use of TE resources can be varied by changing the parameters of 148 the LSP that uses them, and the resources can be released by tearing 149 down the LSP. 151 2.2. Selecting the Path of an LSP 153 Although TE-LSPs can determine their paths hop-by-hop using the 154 shortest path toward the destination to route the signaling protocol 155 messages [RFC3209], in practice this option is not applied because it 156 does not look far enough ahead into the network to verify that the 157 desired resources are available. Instead, the full length of the 158 path of an LSP is computed ahead of time either by the head-end LSR 159 of a signaled LSP, or by Path Computation Element (PCE) functionality 160 in a dedicated server or built into network management software 161 [RFC4655]. 163 Such full-path computation is applied in order that an end-to-end 164 view of the available resources in the network can be used to 165 determine the best likelihood of establishing a viable LSP that meets 166 the service requirements. Even in this situation, however, it is 167 possible that two LSPs being set up at the same time will compete for 168 scarce network resources meaning that one or both of them will fail 169 to be established. This situation is avoided by using a centralized 170 PCE that is aware of the LSP setup requests that are in progress. 172 2.3. Planning Future LSPs 174 LSPs may be established "on demand" when the requester determines 175 that a new LSP is needed. In this case, the path of the LSP is 176 computed as described in Section 2.2. 178 However, in many situations, the requester knows in advance that an 179 LSP will be needed at a particular time in the future. For example, 180 the requester may be aware of a large traffic flow that will start at 181 a well-known time, perhaps for a database synchronization or for the 182 exchange of content between streaming sites. Furthermore, the 183 requester may also know for how long the LSP is required before it 184 can be torn down. 186 The set of requests for future LSPs could be collected and held in a 187 central database (such as at a Network Management System - NMS): when 188 the time comes for each LSP to be set up the NMS can ask the PCE to 189 compute a path and can then request the LSP to be provisioned. This 190 approach has a number of drawbacks because it is not possible to 191 determine in advance whether it will be possible to deliver the LSP 192 since the resources it needs might be used by other LSPs in the 193 network. Thus, at the time the requester asks for the future LSP, 194 the NMS can only make a best-effort guarantee that the LSP will be 195 set up at the desired time. 197 A better solution, therefore, is for the requests for future LSPs to 198 be serviced at once. The paths of the LSPs can be computed ahead of 199 time and converted into reservations of network resources during 200 specific windows in the future. 202 2.4. Looking at Future Demands on TE Resources 204 While path computation as described in Section 2.2 takes account of 205 the currently available network resources, and can act to place LSPs 206 in the network so that there is the best possibility of future LSPs 207 being accommodated, it cannot handle all eventualities. It is simple 208 to construct scenarios where LSPs that are placed one at a time lead 209 to future LSPs being blocked, but where foreknowledge of all of the 210 LSPs would have made it possible for them all to be set up. 212 If, therefore, we were able to know in advance what LSPs were going 213 to be requested we could plan for them and ensure resources were 214 available. Furthermore, such an approach enables a commitment to be 215 made to a service user that an LSP will be set up and available at a 216 specific time. 218 This service can be achieved by tracking the current use of network 219 resources and also a future view of the resource usage. We call this 220 Time-Scheduled TE (TS-TE) resource reservation. 222 2.5. Requisite State Information 224 In order to achieve the TS-TE resource reservation, the use of 225 resources on the path needs to be scheduled. Scheduling state is 226 used to indicate when resources are reserved and when they are 227 available for use. 229 A simple information model for one piece of scheduling state is as 230 follows: 232 { 233 link id; 234 resource id or reserved capacity; 235 reservation start time; 236 reservation end time 237 } 239 The resource that is scheduled can be link capacity, physical 240 resources on a link, CPU utilization, memory, buffers on an 241 interfaces, etc. The resource might also be the maximal unreserved 242 bandwidth of the link over a time interval. For any one resource 243 there could be multiple pieces of scheduling state, and for any one 244 link, the timing windows might overlap. 246 There are multiple ways to realize this information model and 247 different ways to store the data. The resource state could be 248 expressed as a start time and an end time as shown above, or could be 249 expressed as a start time and a duration. Multiple reservation 250 periods, possibly of different lengths, may be need to be recorded 251 for each resource. Furthermore, the current state of network 252 reservation could be kept separate from the scheduled usage, or 253 everything could be merged into a single TS database. 255 An application may make a reservation request for immediate resource 256 usage, or to book resources for future use so as to maximize the 257 chance of services being delivered and to avoid contention for 258 resources in the future. A single reservation request may book 259 resources for multiple periods and might request a reservation that 260 repeats on a regular cycle. 262 A computation engine (that is, a PCE) may use the scheduling state 263 information to help optimize the use of resources into the future and 264 reduce contention or blocking when the resources are actually needed. 266 Note that it is also necessary to store the information about future 267 LSPs as distinct from the specific resource scheduling. This 268 information is held to allow the LSPs to be instantiated when they 269 are due and using the paths/resources that have been computed for 270 them, but also to provide correlation with the TS-TE resource 271 reservations so that it is clear why resources were reserved allowing 272 pre-emption and handling release of reserved resources in the event 273 of cancellation of future LSPs. See Section 3.2 for further 274 discussion of the distinction between scheduled resource state and 275 scheduled LSP state. 277 3. Architectural Concepts 279 This section examines several important architectural concepts that 280 lead to design decisions that will influence how networks can achieve 281 TS-TE in a scalable and robust manner. 283 3.1. Where is Scheduling State Held? 285 The scheduling state information described in Section 2.5 has to be 286 held somewhere. There are two places where this makes sense: 288 o In the network nodes where the resources exist; 289 o In a central scheduling controller where decisions about resource 290 allocation are made. 292 The first of these makes policing of resource allocation easier. It 293 means that many points in the network can request immediate or 294 scheduled LSPs with the associated resource reservation and that all 295 such requests can be correlated at the point where the resources are 296 allocated. However, this approach has some scaling and technical 297 problems: 299 o The most obvious issue is that each network node must retain the 300 full time-based state for all of its resources. In a busy network 301 with a high arrival rate of new LSPs and a low hold time for each 302 LSP, this could be a lot of state. Yet network nodes are normally 303 implemented with minimal spare memory. 305 o In order that path computation can be performed, the computing 306 entity normally known as a Path Computation Element (PCE) 307 [RFC4655] needs access to a database of available links and nodes 308 in the network, and of the TE properties of the links. This 309 database is known as the Traffic Engineering Database (TED) and is 310 usually populated from information advertised in the IGP by each 311 of the network nodes or exported using BGP-LS [RFC7752]. To be 312 able to compute a path for a future LSP the PCE needs to populate 313 the TED with all of the future resource availability: if this 314 information is held on the network nodes it must also be 315 advertised in the IGP. This could be a significant scaling issue 316 for the IGP and the network nodes as all of the advertised 317 information is held at every network node and must be periodically 318 refreshed by the IGP. 320 o When a normal node restarts it can recover resource reservation 321 state from the forwarding hardware, from Non-Volatile Random- 322 Access Memory (NVRAM), or from adjacent nodes through the 323 signaling protocol [RFC5063]. If scheduling state is held at the 324 network nodes it must also be recovered after the restart of a 325 network node. This cannot be achieved from the forwarding 326 hardware because the reservation will not have been made, could 327 require additional expensive NVRAM, or might require that all 328 adjacent nodes also have the scheduling state in order to re- 329 install it on the restarting node. This is potentially complex 330 processing with scaling and cost implications. 332 Conversely, if the scheduling state is held centrally it is easily 333 available at the point of use. That is, the PCE can utilize the 334 state to plan future LSPs and can update that stored information with 335 the scheduled reservation of resources for those future LSPs. This 336 approach also has several issues: 338 o If there are multiple controllers then they must synchronize their 339 stored scheduling state as they each plan future LSPs, and must 340 have a mechanism to resolve resource contention. This is 341 relatively simple and is mitigated by the fact that there is ample 342 processing time to re-plan future LSPs in the case of resource 343 contention. 345 o If other sources of immediate LSPs are allowed (for example, other 346 controllers or autonomous action by head-end LSRs) then the 347 changes in resource availability caused by the setup or tear down 348 of these LSPs must be reflected in the TED (by use of the IGP as 349 currently) and may have an impact of planned future LSPs. This 350 impact can be mitigated by re-planning future LSPs or through LSP 351 preemption. 353 o If the scheduling state is held centrally at a PCE, the state must 354 be held and restored after a system restart. This is relatively 355 easy to achieve on a central server that can have access to non- 356 volatile storage. The PCE could also synchronize the scheduling 357 state with other PCEs after restart. See Section 4.2 for details. 359 o Of course, a centralized system must store information about all 360 of the resources in the network. In a busy network with a high 361 arrival rate of new LSPs and a low hold time for each LSP, this 362 could be a lot of state. This is multiplied by the size of the 363 network measured both by the number of links and nodes, and by the 364 number of trackable resources on each link or at each node. This 365 challenge may be mitigated by the centralized server being 366 dedicated hardware, but there remains the problem of collecting 367 the information from the network in a timely way when there is 368 potentially a very large amount of information to be collected and 369 when the rate of change of that information is high. This latter 370 challenge is only solved if the central server has full control of 371 the booking of resources and the establishment of new LSPs so that 372 the information from the network only serves to confirm what the 373 central server expected. 375 Thus the architectural conclusion is that scheduling state should be 376 held centrally at the point of use and not in the network devices. 378 3.2. What State is Held? 380 As already described, the PCE needs access to an enhanced, time-based 381 TED. It stores the traffic engineering (TE) information such as 382 bandwidth for every link for a series of time intervals. There are a 383 few ways to store the TE information in the TED. For example, 384 suppose that the amount of the unreserved bandwidth at a priority 385 level for a link is Bj in a time interval from time Tj to Tk (k = 386 j+1), where j = 0, 1, 2, .... 388 Bandwidth 389 ^ 390 | B3 391 | B1 ___________ 392 | __________ 393 |B0 B4 394 |__________ B2 _________ 395 | ________________ 396 | 397 -+-------------------------------------------------------> Time 398 |T0 T1 T2 T3 T4 400 Figure 1: A Plot of Bandwidth Usage against Time 402 The unreserved bandwidth for the link can be represented and stored 403 in the TED as [T0, B0], [T1, B1], [T2, B2], [T3, B3], ... as shown in 404 Figure 1. 406 But it must be noted that service requests for future LSPs are known 407 in terms of the LSPs whose paths are computed and for which resources 408 are scheduled. For example, if the requester of a future LSP decides 409 to cancel the request or to modify the request, the PCE must be able 410 to map this to the resources that were reserved. When the LSP or the 411 request for the LSP with a number of time intervals is cancelled, the 412 PCE must release the resources that were reserved on each of the 413 links along the path of the LSP in every time intervals from the TED. 414 If the bandwidth reserved on a link for the LSP is B from time T2 to 415 T3 and the unreserved bandwidth on the link is B2 from T2 to T3, B is 416 added to the link for the time interval from T2 to T3 and the 417 unreserved bandwidth on the link from T2 to T3 will be B2 + B. 419 This suggests that the PCE needs an LSP Database (LSP-DB) [RFC8231] 420 that contains information not only about LSPs that are active in the 421 network, but also those that are planned. The information for an LSP 422 stored in the LSP-DB includes for each time interval that applies to 423 the LSP: the time interval, the paths computed for the LSP satisfying 424 the constraints in the time interval, and the resources such as 425 bandwidth reserved for the LSP in the time interval. See also 426 Section 2.3 428 It is an implementation choice how the TED and LSP-DB are stored both 429 for dynamic use and for recovery after failure or restart, but it may 430 be noted that all of the information in the scheduled TED can be 431 recovered from the active network state and from the scheduled LSP- 432 DB. 434 4. Architecture Overview 436 The architectural considerations and conclusions described in the 437 previous section lead to the architecture described in this section 438 and illustrated in Figure 2. The interfaces and interactions shown 439 on the figure and labeled (a) through (f) are described in 440 Section 4.1. 442 ------------------- 443 | Service Requester | 444 ------------------- 445 ^ 446 a| 447 v 448 ------- b -------- 449 | |<--->| LSP-DB | 450 | | -------- 451 | PCE | 452 | | c ----- 453 | |<---->| TED | 454 ------- ----- 455 ^ ^ 456 | | 457 d| |e 458 | | 459 ------+-----+-------------------- 460 | | Network 461 | -------- 462 | | Router | 463 v -------- 464 ----- ----- 465 | LSR |<------>| LSR | 466 ----- f ----- 468 Figure 2: Reference Architecture for Scheduled Use of Resources 470 4.1. Service Request 472 As shown in Figure 2, some component in the network requests a 473 service. This may be an application, an NMS, an LSR, or any 474 component that qualifies as a Path Computation Client (PCC). We show 475 this on the figure as the "Service Requester" and it sends a request 476 to the PCE for an LSP to be set up at some time (either now or in the 477 future). The request, indicated on Figure 2 by the arrow (a), 478 includes all of the parameters of the LSP that the requester wishes 479 to supply such as bandwidth, start time, and end time. Note that the 480 requester in this case may be the LSR shown in the figure or may be a 481 distinct system. 483 The PCE enters the LSP request in its LSP-DB (b), and uses 484 information from its TED (c) to compute a path that satisfies the 485 constraints (such as bandwidth) for the LSP in the time interval from 486 the start time to the end time. It updates the future resource 487 availability in the TED so that further path computations can take 488 account of the scheduled resource usage. It stores the path for the 489 LSP into the LSP-DB (b). 491 When it is time (i.e., at the start time) for the LSP to be set up, 492 the PCE sends a PCEP Initiate request to the head end LSR (d) 493 providing the path to be signaled as well as other parameters such as 494 the bandwidth of the LSP. 496 As the LSP is signaled between LSRs (f) the use of resources in the 497 network is updated and distributed using the IGP. This information 498 is shared with the PCE either through the IGP or using BGP-LS (e), 499 and the PCE updates the information stored in its TED (c). 501 After the LSP is set up, the head end LSR sends a PCEP LSP State 502 Report (PCRpt message) to the PCE (d). The report contains the 503 resources such as bandwidth usage for the LSP. The PCE updates the 504 status of the LSP in the LSP-DB according to the report. 506 When an LSP is no longer required (either because the Service 507 Requester has cancelled the request, or because the LSP's scheduled 508 lifetime has expired) the PCE can remove it. If the LSP is currently 509 active, the PCE instructs the head-end LSR to tear it down (d), and 510 the network resource usage will be updated by the IGP and advertised 511 back to the PCE through the IGP or BGP-LS (e). Once the LSP is no 512 longer active, the PCE can remove it from the LSP-DB (b). 514 4.1.1. Reoptimization After TED Updates 516 When the TED is updated as indicated in Section 4.1, the PCE may 517 perform reoptimization of the LSPs for which it has computed paths. 518 These LSPs may be already provisioned in which case the PCE issues 519 PCEP Update request messages for the LSPs that should be adjusted. 520 Additionally, the LSPs being reoptimized may be scheduled LSPs that 521 have not yet been provisioned, in which case reoptimization involves 522 updating the store of scheduled LSPs and resources. 524 In all cases, the purpose of reoptimization is to take account of the 525 resource usage and availability in the network and to compute paths 526 for the current and future LSPs that best satisfy the objectives of 527 those LSPs while keeping the network as clear as possible to support 528 further LSPs. 530 4.2. Initialization and Recovery 532 When a PCE in the architecture shown in Figure 2 is initialized, it 533 must learn state from the network, from its stored databases, and 534 potentially from other PCEs in the network. 536 The first step is to get an accurate view of the topology and 537 resource availability in the network. This would normally involve 538 reading the state direct from the network via the IGP or BGP-LS (e), 539 but might include receiving a copy of the TED from another PCE. Note 540 that a TED stored from a previous instantiation of the PCE is 541 unlikely to be valid. 543 Next, the PCE must construct a time-based TED to show scheduled 544 resource usage. How it does this is implementation specific and this 545 document does not dictate any particular mechanism: it may recover a 546 time-based TED previously saved to non-volatile storage, or it may 547 reconstruct the time-based TED from information retrieved from the 548 LSP-DB previously saved to non-volatile storage. If there is more 549 than one PCE active in the network, the recovering PCE will need to 550 synchronize the LSP-DB and time-based TED with other PCEs (see 551 Section 4.3). 553 Note that the stored LSP-DB needs to include the intended state and 554 actual state of the LSPs so that when a PCE recovers it is able to 555 determine what actions are necessary. 557 4.3. Synchronization Between PCEs 559 If there is more than one PCE that supports scheduling active in the 560 network, it is important to achieve some consistency between the 561 scheduled TED and scheduled LSP-DB held by the PCEs. 563 [RFC7399] answers various questions around synchronization between 564 the PCEs. It should be noted that the time-based "scheduled" 565 information adds another dimension to the issue of synchronization 566 between PCEs. It should also be noted that a deployment may use a 567 primary PCE and the have other PCEs as backup, where a backup PCE can 568 take over only in the event of a failure of the primary PCE. 569 Alternatively, the PCEs may share the load at all times. The choice 570 of the synchronization technique is largely dependent on the 571 deployment of PCEs in the network. 573 One option for ensuring that multiple PCEs use the same scheduled 574 information is simply to have the PCEs driven from the same shared 575 database, but it is likely to be inefficient and interoperation 576 between multiple implementations will be harder. 578 Another option is for each PCE to be responsible for its own 579 scheduled database and to utilize some distributed database 580 synchronization mechanism to have consistent information. Depending 581 on the implementation, this could be efficient, but interoperation 582 between heterogeneous implementations is still hard. 584 A further approach is to utilize PCEP messages to synchronize the 585 scheduled state between PCEs. This approach would work well if the 586 number of PCEs which support scheduling is small, but as the number 587 increases considerable message exchange needs to happen to keep the 588 scheduled databases synchronized. Future solutions could also 589 utilize some synchronization optimization techniques for efficiency. 590 Another variation would be to request information from other PCEs for 591 a particular time slice, but this might have impact on the 592 optimization algorithm. 594 5. Multi-Domain Considerations 596 Multi-domain path computation usually requires some form of 597 cooperation between PCEs each of which has responsibility for 598 determining a segment of the end-to-end path in the domain for which 599 it has computational responsibility. When computing a scheduled 600 path, resources need to be booked in all of the domains that the path 601 will cross so that they are available when the LSP is finally 602 signalled. 604 Per-domain path computation [RFC5152] is not an appropriate mechanism 605 when a scheduled LSP is being computed because the computation 606 requests at downstream PCEs are only triggered by signaling. 607 However, a similar mechanism could be used where cooperating PCEs 608 exchange PCReq messages for a scheduled LSP as shown in Figure 3. In 609 this case the service requester asks for a scheduled LSP that will 610 span two domains (a). PCE1 computes a path across Domain 1 and 611 reserves the resources, and also asks PCE2 to compute and reserve in 612 Domain 2 (b). PCE2 may return a full path, or could return a path 613 key [RFC5520]. When it is time for LSP setup PCE1 triggers the head- 614 end LSR (c) and the LSP is signaled (d). If a path key is used, the 615 entry LSR in Domain 2 will consult PCE2 for the path expansion (e) 616 before completing signaling (f). 618 ------------------- 619 | Service Requester | 620 ------------------- 621 ^ 622 a| 623 v 624 ------ b ------ 625 | |<---------------->| | 626 | PCE1 | | PCE2 | 627 | | | | 628 ------ ------ 629 ^ ^ 630 | | 631 c| e| 632 | | 633 ----+----------------- ----+----------------- 634 | | Domain 1 | | | Domain 2 | 635 | v | | v | 636 | ----- d ----- | | ----- f ----- | 637 | | LSR |<--->| LSR |<-+--+->| LSR |<--->| LSR | | 638 | ----- ----- | | ----- ----- | 639 ---------------------- ---------------------- 641 Figure 3: Per-Domain Path Computation for Scheduled LSPs 643 Another mechanism for PCE cooperation in multi-domain LSP setup is 644 Backward- Recursive Path Computation (BRPC) [RFC5441]. This approach 645 relies on the downstream domain supply a variety of potential paths 646 to the upstream domain. Although BRPC can arrive at a more optimal 647 end-to-end path than per-domain path computation, it is not well 648 suited to LSP scheduling because the downstream PCE would need to 649 reserve resources on all of the potential paths and then release 650 those that the upstream PCE announced it did not plan to use. 652 Finally we should consider hierarchical PCE (H-PCE) [RFC6805]. This 653 mode of operation is similar to that shown in Figure 3, but a parent 654 PCE is used to coordinate the requests to the child PCEs resulting in 655 better visibility of the end-to-end path and better coordination of 656 the resource booking. The sequenced flow of control is shown in 657 Figure 4. 659 ------------------- 660 | Service Requester | 661 ------------------- 662 ^ 663 a| 664 v 665 -------- 666 | | 667 | Parent | 668 | PCE | 669 | | 670 -------- 671 ^ ^ b 672 b| |_______________________ 673 | | 674 v v 675 ------ ------ 676 | | | | 677 | PCE1 | | PCE2 | 678 | | | | 679 ------ ------ 680 ^ ^ 681 | | 682 c| e| 683 | | 684 ----+----------------- ----+----------------- 685 | | Domain 1 | | | Domain 2 | 686 | v | | v | 687 | ----- d ----- | | ----- f ----- | 688 | | LSR |<--->| LSR |<-+--+->| LSR |<--->| LSR | | 689 | ----- ----- | | ----- ----- | 690 ---------------------- ---------------------- 692 Figure 4: Hierarchical PCE for Path Computation for Scheduled LSPs 694 6. Security Considerations 696 The protocol implications of scheduled resources are unchanged from 697 "on-demand" LSP computation and setup. A discussion of securing PCEP 698 is found in [RFC5440] and work to extend that security is provided in 699 [RFC8253]. Furthermore, the path key mechanism described in 700 [RFC5520] can be used to enhance privacy and security. 702 Similarly, there is no change to the security implications for the 703 signaling of scheduled LSPs. A discussion of the security of the 704 signaling protocols that would be used is found in [RFC5920]. 706 However, the use of scheduled LSPs extends the attack surface for a 707 PCE-enabled TE system by providing a larger (logically infinite) 708 window during which an attack can be initiated or planned. That is, 709 if bogus scheduled LSPs can be requested, they can be entered into 710 the LSP-DB, then a large number of LSPs could be launched, or 711 significant network resources could be blocked. Of course, 712 additional authorization could be applied for access to LSP 713 scheduling, and diagnostic tools could inspect the LSP DB to spot 714 attacks. 716 7. IANA Considerations 718 This architecture document makes no request for IANA action. 720 8. Acknowledgements 722 This work has benefited from the discussions of resource scheduling 723 over the years. In particular the DRAGON project [DRAGON] and 724 [I-D.yong-ccamp-ason-gmpls-autobw-service] both of which provide 725 approaches to auto-bandwidth services in GMPLS networks. 727 Mehmet Toy, Lei Liu, and Khuzema Pithewan contributed the earlier 728 version of [I-D.chen-teas-frmwk-tts]. We would like to thank the 729 authors of that draft on Temporal Tunnel Services. 731 Thanks to Michael Scharf and Daniele Ceccarelli for useful comments 732 on this work. 734 Jonathan Hardwick provided a helpful Routing Directorate review. 736 9. Contributors 738 The following people contributed to discussions that led to the 739 development of this document: 741 Dhruv Dhody 742 Email: dhruv.dhody@huawei.com 744 10. Informative References 746 [DRAGON] National Science Foundation, "http://www.maxgigapop.net/ 747 wp-content/uploads/The-DRAGON-Project.pdf". 749 [I-D.chen-teas-frmwk-tts] 750 Chen, H., Toy, M., Liu, L., and K. Pithewan, "Framework 751 for Temporal Tunnel Services", draft-chen-teas-frmwk- 752 tts-01 (work in progress), March 2016. 754 [I-D.yong-ccamp-ason-gmpls-autobw-service] 755 Yong, L. and Y. Lee, "ASON/GMPLS Extension for Reservation 756 and Time Based Automatic Bandwidth Service", draft-yong- 757 ccamp-ason-gmpls-autobw-service-00 (work in progress), 758 October 2006. 760 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 761 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 762 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 763 . 765 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 766 Switching (GMPLS) Signaling Resource ReserVation Protocol- 767 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 768 DOI 10.17487/RFC3473, January 2003, 769 . 771 [RFC3945] Mannie, E., Ed., "Generalized Multi-Protocol Label 772 Switching (GMPLS) Architecture", RFC 3945, 773 DOI 10.17487/RFC3945, October 2004, 774 . 776 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 777 Element (PCE)-Based Architecture", RFC 4655, 778 DOI 10.17487/RFC4655, August 2006, 779 . 781 [RFC5063] Satyanarayana, A., Ed. and R. Rahman, Ed., "Extensions to 782 GMPLS Resource Reservation Protocol (RSVP) Graceful 783 Restart", RFC 5063, DOI 10.17487/RFC5063, October 2007, 784 . 786 [RFC5152] Vasseur, JP., Ed., Ayyangar, A., Ed., and R. Zhang, "A 787 Per-Domain Path Computation Method for Establishing Inter- 788 Domain Traffic Engineering (TE) Label Switched Paths 789 (LSPs)", RFC 5152, DOI 10.17487/RFC5152, February 2008, 790 . 792 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 793 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 794 DOI 10.17487/RFC5440, March 2009, 795 . 797 [RFC5441] Vasseur, JP., Ed., Zhang, R., Bitar, N., and JL. Le Roux, 798 "A Backward-Recursive PCE-Based Computation (BRPC) 799 Procedure to Compute Shortest Constrained Inter-Domain 800 Traffic Engineering Label Switched Paths", RFC 5441, 801 DOI 10.17487/RFC5441, April 2009, 802 . 804 [RFC5520] Bradford, R., Ed., Vasseur, JP., and A. Farrel, 805 "Preserving Topology Confidentiality in Inter-Domain Path 806 Computation Using a Path-Key-Based Mechanism", RFC 5520, 807 DOI 10.17487/RFC5520, April 2009, 808 . 810 [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS 811 Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, 812 . 814 [RFC6805] King, D., Ed. and A. Farrel, Ed., "The Application of the 815 Path Computation Element Architecture to the Determination 816 of a Sequence of Domains in MPLS and GMPLS", RFC 6805, 817 DOI 10.17487/RFC6805, November 2012, 818 . 820 [RFC7399] Farrel, A. and D. King, "Unanswered Questions in the Path 821 Computation Element Architecture", RFC 7399, 822 DOI 10.17487/RFC7399, October 2014, 823 . 825 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 826 S. Ray, "North-Bound Distribution of Link-State and 827 Traffic Engineering (TE) Information Using BGP", RFC 7752, 828 DOI 10.17487/RFC7752, March 2016, 829 . 831 [RFC8231] Crabbe, E., Minei, I., Medved, J., and R. Varga, "Path 832 Computation Element Communication Protocol (PCEP) 833 Extensions for Stateful PCE", RFC 8231, 834 DOI 10.17487/RFC8231, September 2017, 835 . 837 [RFC8253] Lopez, D., Gonzalez de Dios, O., Wu, Q., and D. Dhody, 838 "PCEPS: Usage of TLS to Provide a Secure Transport for the 839 Path Computation Element Communication Protocol (PCEP)", 840 RFC 8253, DOI 10.17487/RFC8253, October 2017, 841 . 843 Authors' Addresses 845 Yan Zhuang 846 Huawei 847 101 Software Avenue, Yuhua District 848 Nanjing, Jiangsu 210012 849 China 851 Email: zhuangyan.zhuang@huawei.com 853 Qin Wu 854 Huawei 855 101 Software Avenue, Yuhua District 856 Nanjing, Jiangsu 210012 857 China 859 Email: bill.wu@huawei.com 861 Huaimo Chen 862 Huawei 863 Boston, MA 864 US 866 Email: huaimo.chen@huawei.com 868 Adrian Farrel 869 Juniper Networks 871 Email: afarrel@juniper.net