idnits 2.17.1 draft-ietf-teas-scheduled-resources-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 20, 2018) is 2256 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'T0' is mentioned on line 456, but not defined == Missing Reference: 'B0' is mentioned on line 456, but not defined == Missing Reference: 'T1' is mentioned on line 456, but not defined == Missing Reference: 'B1' is mentioned on line 456, but not defined == Missing Reference: 'T2' is mentioned on line 456, but not defined == Missing Reference: 'B2' is mentioned on line 456, but not defined == Missing Reference: 'T3' is mentioned on line 456, but not defined == Missing Reference: 'B3' is mentioned on line 456, but not defined -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TEAS Working Group Y. Zhuang 3 Internet-Draft Q. Wu 4 Intended status: Informational H. Chen 5 Expires: August 24, 2018 Huawei 6 A. Farrel 7 Juniper Networks 8 February 20, 2018 10 Framework for Scheduled Use of Resources 11 draft-ietf-teas-scheduled-resources-06 13 Abstract 15 Time-scheduled reservation of traffic engineering (TE) resources can 16 be used to provide resource booking for TE Label Switched Paths so as 17 to better guarantee services for customers and to improve the 18 efficiency of network resource usage at any moment in time including 19 future planned network usage. This document provides a framework 20 that describes and discusses the architecture for supporting 21 scheduled reservation of TE resources. This document does not 22 describe specific protocols or protocol extensions needed to realize 23 this service. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on August 24, 2018. 42 Copyright Notice 44 Copyright (c) 2018 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 3 61 2.1. Provisioning TE-LSPs and TE Resources . . . . . . . . . . 3 62 2.2. Selecting the Path of an LSP . . . . . . . . . . . . . . 4 63 2.3. Planning Future LSPs . . . . . . . . . . . . . . . . . . 5 64 2.4. Looking at Future Demands on TE Resources . . . . . . . . 6 65 2.5. Requisite State Information . . . . . . . . . . . . . . . 6 66 3. Architectural Concepts . . . . . . . . . . . . . . . . . . . 7 67 3.1. Where is Scheduling State Held? . . . . . . . . . . . . . 8 68 3.2. What State is Held? . . . . . . . . . . . . . . . . . . . 10 69 3.3. Enforcement of Operator Policy . . . . . . . . . . . . . 11 70 4. Architecture Overview . . . . . . . . . . . . . . . . . . . . 12 71 4.1. Service Request . . . . . . . . . . . . . . . . . . . . . 13 72 4.1.1. Reoptimization After TED Updates . . . . . . . . . . 14 73 4.2. Initialization and Recovery . . . . . . . . . . . . . . . 15 74 4.3. Synchronization Between PCEs . . . . . . . . . . . . . . 15 75 5. Multi-Domain Considerations . . . . . . . . . . . . . . . . . 16 76 6. Security Considerations . . . . . . . . . . . . . . . . . . . 18 77 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 78 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 79 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 19 80 10. Informative References . . . . . . . . . . . . . . . . . . . 19 81 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 83 1. Introduction 85 Traffic Engineering Label Switched Paths (TE-LSPs) are connection 86 oriented tunnels in packet and non-packet networks [RFC3209], 87 [RFC3945]. TE-LSPs may reserve network resources for use by the 88 traffic they carry, thus providing some guarantees of service 89 delivery and allowing a network operator to plan the use of the 90 resources across the whole network. 92 In some technologies (such as wavelength switched optical networks) 93 the resource is synonymous with the label that is switched on the 94 path of the LSP so that it is not possible to establish an LSP that 95 can carry traffic without assigning a physical resource to the LSP. 96 In other technologies (such as packet switched networks) the 97 resources assigned to an LSP are a measure of the capacity of a link 98 that is dedicated for use by the traffic on the LSP. 100 In all cases, network planning consists of selecting paths for LSPs 101 through the network so that there will be no contention for 102 resources. LSP establishment is the act of setting up an LSP and 103 reserving resources within the network. Network optimization or re- 104 optimization is the process of re-positioning LSPs in the network to 105 make the unreserved network resources more useful for potential 106 future LSPs while ensuring that the established LSPs continue to 107 fulfill their objectives. 109 It is often the case that it is known that an LSP will be needed at 110 some specific time in the future. While a path for that LSP could be 111 computed using knowledge of the currently established LSPs and the 112 currently available resources, this does not give any degree of 113 certainty that the necessary resources will be available when it is 114 time to set up the new LSP. Yet setting up the LSP ahead of the time 115 when it is needed (which would guarantee the availability of the 116 resources) is wasteful since the network resources could be used for 117 some other purpose in the meantime. 119 Similarly, it may be known that an LSP will no longer be needed after 120 some future time and that it will be torn down releasing the network 121 resources that were assigned to it. This information can be helpful 122 in planning how a future LSP is placed in the network. 124 Time-Scheduled (TS) reservation of TE resources can be used to 125 provide resource booking for TE-LSPs so as to better guarantee 126 services for customers and to improve the efficiency of network 127 resource usage into the future. This document provides a framework 128 that describes the problem and discusses the architecture for the 129 scheduled reservation of TE resources. This document does not 130 describe specific protocols or protocol extensions needed to realize 131 this service. 133 2. Problem Statement 135 2.1. Provisioning TE-LSPs and TE Resources 137 TE-LSPs in existing networks are provisioned using a variety of 138 techniques. They may be set up using RSVP-TE as a signaling protocol 139 [RFC3209] [RFC3473]. Alternatively, they could be established by 140 direct control of network elements such as in the Software Defined 141 Networking (SDN) paradigm. They could also be provisioned using the 142 PCE Communication Protocol (PCEP) [RFC5440] as a control protocol to 143 communicate with the network elements. 145 TE resources are reserved at the point of use. That is, the 146 resources (wavelengths, timeslots, bandwidth, etc.) are reserved for 147 use on a specific link and are tracked by the Label Switching Routers 148 (LSRs) at the end points of the link. Those LSRs learn which 149 resources to reserve during the LSP setup process. 151 The use of TE resources can be varied by changing the parameters of 152 the LSP that uses them, and the resources can be released by tearing 153 down the LSP. 155 Resources that have been reserved in the network for use by one LSP 156 may be pre-empted for use by another LSP. If RSVP-TE signaling is in 157 use, a holding priority and a pre-emption priority are used to 158 determine which LSPs may pre-empted the resources in use for which 159 other LSPs. If direct (central) control is in use, the controller is 160 able to make pre-emption decisions. In either case, operator policy 161 forms a key part of pre-emption since there is a trade between 162 disrupting existing LSPs and enabling new LSPs. 164 2.2. Selecting the Path of an LSP 166 Although TE-LSPs can determine their paths hop-by-hop using the 167 shortest path toward the destination to route the signaling protocol 168 messages [RFC3209], in practice this option is not applied because it 169 does not look far enough ahead into the network to verify that the 170 desired resources are available. Instead, the full length of the 171 path of an LSP is usually computed ahead of time either by the head- 172 end LSR of a signaled LSP, or by Path Computation Element (PCE) 173 functionality in a dedicated server or built into network management 174 software [RFC4655]. 176 Such full-path computation is applied in order that an end-to-end 177 view of the available resources in the network can be used to 178 determine the best likelihood of establishing a viable LSP that meets 179 the service requirements. Even in this situation, however, it is 180 possible that two LSPs being set up at the same time will compete for 181 scarce network resources meaning that one or both of them will fail 182 to be established. This situation is avoided by using a centralized 183 PCE that is aware of the LSP setup requests that are in progress. 185 Path selection may make allowance for pre-emption as described in 186 Section 2.1. That is, when selecting a path, the decision may be 187 made to choose a path that will result in the pre-emption of an 188 existing LSP. The trade-off between selecting a less optimal path, 189 failing to select any path at all, and pre-empting an existing LSP 190 must be subject to operator policy. 192 Path computation is subject to "objective functions" that define what 193 criteria are to be met when the LSP is placed [RFC4655]. These can 194 be criteria that apply to the LSP itself (such as shortest path to 195 destination) or to the network state after the LSP is set up (such as 196 maximized residual link bandwidth). The objective functions may be 197 requested by the application requesting the LSP, and may be filtered 198 and enhanced by the computation engine according to operator policy. 200 2.3. Planning Future LSPs 202 LSPs may be established "on demand" when the requester determines 203 that a new LSP is needed. In this case, the path of the LSP is 204 computed as described in Section 2.2. 206 However, in many situations, the requester knows in advance that an 207 LSP will be needed at a particular time in the future. For example, 208 the requester may be aware of a large traffic flow that will start at 209 a well-known time, perhaps for a database synchronization or for the 210 exchange of content between streaming sites. Furthermore, the 211 requester may also know for how long the LSP is required before it 212 can be torn down. 214 The set of requests for future LSPs could be collected and held in a 215 central database (such as at a Network Management System - NMS): when 216 the time comes for each LSP to be set up the NMS can ask the PCE to 217 compute a path and can then request the LSP to be provisioned. This 218 approach has a number of drawbacks because it is not possible to 219 determine in advance whether it will be possible to deliver the LSP 220 since the resources it needs might be used by other LSPs in the 221 network. Thus, at the time the requester asks for the future LSP, 222 the NMS can only make a best-effort guarantee that the LSP will be 223 set up at the desired time. 225 A better solution, therefore, is for the requests for future LSPs to 226 be serviced at once. The paths of the LSPs can be computed ahead of 227 time and converted into reservations of network resources during 228 specific windows in the future. That is, while the path of the LSP 229 is computed and the network resources are reserved, the LSP is not 230 established in the network until the time for which it is scheduled. 232 There is a need to take into account items that need to be subject to 233 operator policy such as the amount of capacity available for 234 scheduling future reservations and the operator preference for the 235 measures which are used with respect to the use of scheduled 236 resources during rapid changes in traffic demand events or a complex 237 (multiple nodes/links) failure event so as to protect against network 238 destabilization. Operator policy is discussed further in 239 Section 3.3. 241 2.4. Looking at Future Demands on TE Resources 243 While path computation as described in Section 2.2 takes account of 244 the currently available network resources, and can act to place LSPs 245 in the network so that there is the best possibility of future LSPs 246 being accommodated, it cannot handle all eventualities. It is simple 247 to construct scenarios where LSPs that are placed one at a time lead 248 to future LSPs being blocked, but where foreknowledge of all of the 249 LSPs would have made it possible for them all to be set up. 251 If, therefore, we were able to know in advance what LSPs were going 252 to be requested, we could plan for them and ensure resources were 253 available. Furthermore, such an approach enables a commitment to be 254 made to a service user that an LSP will be set up and available at a 255 specific time. 257 A reservation service can be achieved by tracking the current use of 258 network resources and also having a future view of the resource 259 usage. We call this Time-Scheduled TE (TS-TE) resource reservation. 261 2.5. Requisite State Information 263 In order to achieve the TS-TE resource reservation, the use of 264 resources on the path needs to be scheduled. Scheduling state is 265 used to indicate when resources are reserved and when they are 266 available for use. 268 A simple information model for one piece of scheduling state is as 269 follows: 271 { 272 link id; 273 resource id or reserved capacity; 274 reservation start time; 275 reservation end time 276 } 278 The resource that is scheduled could be link capacity, physical 279 resources on a link, buffers on an interfaces, etc., and could 280 include advanced considerations such as CPU utilization and the 281 availability of memory at nodes within the network. The resource- 282 related information might also include the maximal unreserved 283 bandwidth of the link over a time interval. That is, the intention 284 is to book (reserve) a percentage of the residual (unreserved) 285 bandwidth of the link. This could be used, for example, to reserve 286 bandwidth for a particular class of traffic (such as IP) that doesn't 287 have a provisioned LSP. 289 For any one resource there could be multiple pieces of scheduling 290 state, and for any one link, the timing windows might overlap. 292 There are multiple ways to realize this information model and 293 different ways to store the data. The resource state could be 294 expressed as a start time and an end time as shown above, or could be 295 expressed as a start time and a duration. Multiple reservation 296 periods, possibly of different lengths, may need to be recorded for 297 each resource. Furthermore, the current state of network reservation 298 could be kept separate from the scheduled usage, or everything could 299 be merged into a single TS database. 301 An application may make a reservation request for immediate resource 302 usage, or to book resources for future use so as to maximize the 303 chance of services being delivered and to avoid contention for 304 resources in the future. A single reservation request may book 305 resources for multiple periods and might request a reservation that 306 repeats on a regular cycle. 308 A computation engine (that is, a PCE) may use the scheduling state 309 information to help optimize the use of resources into the future and 310 reduce contention or blocking when the resources are actually needed. 312 Note that it is also necessary to store the information about future 313 LSPs as distinct from the specific resource scheduling. This 314 information is held to allow the LSPs to be instantiated when they 315 are due and using the paths/resources that have been computed for 316 them, but also to provide correlation with the TS-TE resource 317 reservations so that it is clear why resources were reserved, 318 allowing pre-emption, and handling release of reserved resources in 319 the event of cancellation of future LSPs. See Section 3.2 for 320 further discussion of the distinction between scheduled resource 321 state and scheduled LSP state. 323 Network performance factors (such as maximum link utilization and the 324 residual capacity of the network) with respect to supporting 325 scheduled reservations need to be supported and are subject to 326 operator policy. 328 3. Architectural Concepts 330 This section examines several important architectural concepts to 331 understand the design decisions reached in this document to achieve 332 TS-TE in a scalable and robust manner. 334 3.1. Where is Scheduling State Held? 336 The scheduling state information described in Section 2.5 has to be 337 held somewhere. There are two places where this makes sense: 339 o In the network nodes where the resources exist; 341 o In a central scheduling controller where decisions about resource 342 allocation are made. 344 The first of these makes policing of resource allocation easier. It 345 means that many points in the network can request immediate or 346 scheduled LSPs with the associated resource reservation and that all 347 such requests can be correlated at the point where the resources are 348 allocated. However, this approach has some scaling and technical 349 problems: 351 o The most obvious issue is that each network node must retain the 352 full time-based state for all of its resources. In a busy network 353 with a high arrival rate of new LSPs and a low hold time for each 354 LSP, this could be a lot of state. Network nodes are normally 355 implemented with minimal spare memory. 357 o In order that path computation can be performed, the computing 358 entity normally known as a Path Computation Element (PCE) 359 [RFC4655] needs access to a database of available links and nodes 360 in the network, and of the TE properties of the links. This 361 database is known as the Traffic Engineering Database (TED) and is 362 usually populated from information advertised in the IGP by each 363 of the network nodes or exported using BGP-LS [RFC7752]. To be 364 able to compute a path for a future LSP the PCE needs to populate 365 the TED with all of the future resource availability: if this 366 information is held on the network nodes it must also be 367 advertised in the IGP. This could be a significant scaling issue 368 for the IGP and the network nodes as all of the advertised 369 information is held at every network node and must be periodically 370 refreshed by the IGP. 372 o When a normal node restarts it can recover resource reservation 373 state from the forwarding hardware, from Non-Volatile Random- 374 Access Memory (NVRAM), or from adjacent nodes through the 375 signaling protocol [RFC5063]. If scheduling state is held at the 376 network nodes it must also be recovered after the restart of a 377 network node. This cannot be achieved from the forwarding 378 hardware because the reservation will not have been made, could 379 require additional expensive NVRAM, or might require that all 380 adjacent nodes also have the scheduling state in order to re- 381 install it on the restarting node. This is potentially complex 382 processing with scaling and cost implications. 384 Conversely, if the scheduling state is held centrally it is easily 385 available at the point of use. That is, the PCE can utilize the 386 state to plan future LSPs and can update that stored information with 387 the scheduled reservation of resources for those future LSPs. This 388 approach also has several issues: 390 o If there are multiple controllers then they must synchronize their 391 stored scheduling state as they each plan future LSPs, and must 392 have a mechanism to resolve resource contention. This is 393 relatively simple and is mitigated by the fact that there is ample 394 processing time to re-plan future LSPs in the case of resource 395 contention. 397 o If other sources of immediate LSPs are allowed (for example, other 398 controllers or autonomous action by head-end LSRs) then the 399 changes in resource availability caused by the setup or tear down 400 of these LSPs must be reflected in the TED (by use of the IGP as 401 currently) and may have an impact of planned future LSPs. This 402 impact can be mitigated by re-planning future LSPs or through LSP 403 preemption. 405 o If the scheduling state is held centrally at a PCE, the state must 406 be held and restored after a system restart. This is relatively 407 easy to achieve on a central server that can have access to non- 408 volatile storage. The PCE could also synchronize the scheduling 409 state with other PCEs after restart. See Section 4.2 for details. 411 o Of course, a centralized system must store information about all 412 of the resources in the network. In a busy network with a high 413 arrival rate of new LSPs and a low hold time for each LSP, this 414 could be a lot of state. This is multiplied by the size of the 415 network measured both by the number of links and nodes, and by the 416 number of trackable resources on each link or at each node. This 417 challenge may be mitigated by the centralized server being 418 dedicated hardware, but there remains the problem of collecting 419 the information from the network in a timely way when there is 420 potentially a very large amount of information to be collected and 421 when the rate of change of that information is high. This latter 422 challenge is only solved if the central server has full control of 423 the booking of resources and the establishment of new LSPs so that 424 the information from the network only serves to confirm what the 425 central server expected. 427 Thus, considering these tradeoffs, the architectural conclusion is 428 that scheduling state should be held centrally at the point of use 429 and not in the network devices. 431 3.2. What State is Held? 433 As already described, the PCE needs access to an enhanced, time-based 434 TED. It stores the traffic engineering (TE) information such as 435 bandwidth for every link for a series of time intervals. There are a 436 few ways to store the TE information in the TED. For example, 437 suppose that the amount of the unreserved bandwidth at a priority 438 level for a link is Bj in a time interval from time Tj to Tk (k = 439 j+1), where j = 0, 1, 2, .... 441 Bandwidth 442 ^ 443 | B3 444 | B1 ___________ 445 | __________ 446 |B0 B4 447 |__________ B2 _________ 448 | ________________ 449 | 450 -+-------------------------------------------------------> Time 451 |T0 T1 T2 T3 T4 453 Figure 1: A Plot of Bandwidth Usage against Time 455 The unreserved bandwidth for the link can be represented and stored 456 in the TED as [T0, B0], [T1, B1], [T2, B2], [T3, B3], ... as shown in 457 Figure 1. 459 But it must be noted that service requests for future LSPs are known 460 in terms of the LSPs whose paths are computed and for which resources 461 are scheduled. For example, if the requester of a future LSP decides 462 to cancel the request or to modify the request, the PCE must be able 463 to map this to the resources that were reserved. When the LSP or the 464 request for the LSP with a number of time intervals is cancelled, the 465 PCE must release the resources that were reserved on each of the 466 links along the path of the LSP in every time intervals from the TED. 467 If the bandwidth reserved on a link for the LSP is B from time T2 to 468 T3 and the unreserved bandwidth on the link is B2 from T2 to T3, B is 469 added to the link for the time interval from T2 to T3 and the 470 unreserved bandwidth on the link from T2 to T3 will be B2 + B. 472 This suggests that the PCE needs an LSP Database (LSP-DB) [RFC8231] 473 that contains information not only about LSPs that are active in the 474 network, but also those that are planned. The information for an LSP 475 stored in the LSP-DB includes for each time interval that applies to 476 the LSP: the time interval, the paths computed for the LSP satisfying 477 the constraints in the time interval, and the resources such as 478 bandwidth reserved for the LSP in the time interval. See also 479 Section 2.3 481 It is an implementation choice how the TED and LSP-DB are stored both 482 for dynamic use and for recovery after failure or restart, but it may 483 be noted that all of the information in the scheduled TED can be 484 recovered from the active network state and from the scheduled LSP- 485 DB. 487 3.3. Enforcement of Operator Policy 489 Computation requests for LSPs are serviced according to operator 490 policy. For example, a PCE may refuse a computation request because 491 the application making the request does not have sufficient 492 permissions, or because servicing the request might take specific 493 resource usage over a given threshold. 495 Furthermore, the pre-emption and holding priorities of any particular 496 computation request may be subject to the operator's policies. The 497 request could be rejected if it does not conform to the operator's 498 policies, or (possibly more likely) the priorities could be set/ 499 overwritten according to the operator's policies. 501 Additionally, the objective functions (OFs) of computation request 502 (such as maximizing residual bandwidth) are also subject to operator 503 policies. It is highly likely that the choice of OFs is not 504 available to an application and is selected by the PCE or management 505 system subject to operator policies and knowledge of the application. 507 None of these statements is new to scheduled resources. They apply 508 to stateless, stateful, passive, and active PCEs, and they continue 509 to apply to scheduling of resources. 511 An operator may choose to configure special behavior for a PCE that 512 handles resource scheduling. For example, an operator might want 513 only a certain percentage of any resource to be bookable. And an 514 operator might want the pre-emption of booked resources to be an 515 inverse function of how far in the future the resources are needed 516 for the first time. 518 It is a general assumption about the architecture described in 519 Section 4 that a PCE is under the operational control of the operator 520 that owns the resources that the PCE manipulates. Thus the operator 521 may configure any amount of (potentially complex) policy at the PCE. 522 This configuration would also include policy points surrounding re- 523 optimization of existing and planned LSPs in the event of changes in 524 the current and future (planned) resource availability. 526 The granularity of the timing window offered to an application will 527 depend on an operator's policy as well as the implementation in the 528 PCE, and goes to define the operator' service offerings. Different 529 granularities and different lengths of pre-booking may be offered to 530 different applications. 532 4. Architecture Overview 534 The architectural considerations and conclusions described in the 535 previous section lead to the architecture described in this section 536 and illustrated in Figure 2. The interfaces and interactions shown 537 on the figure and labeled (a) through (f) are described in 538 Section 4.1. 540 ------------------- 541 | Service Requester | 542 ------------------- 543 ^ 544 a| 545 v 546 ------- b -------- 547 | |<--->| LSP-DB | 548 | | -------- 549 | PCE | 550 | | c ----- 551 | |<---->| TED | 552 ------- ----- 553 ^ ^ 554 | | 555 d| |e 556 | | 557 ------+-----+-------------------- 558 | | Network 559 | -------- 560 | | Router | 561 v -------- 562 ----- ----- 563 | LSR |<------>| LSR | 564 ----- f ----- 566 Figure 2: Reference Architecture for Scheduled Use of Resources 568 4.1. Service Request 570 As shown in Figure 2, some component in the network requests a 571 service. This may be an application, an NMS, an LSR, or any 572 component that qualifies as a Path Computation Client (PCC). We show 573 this on the figure as the "Service Requester" and it sends a request 574 to the PCE for an LSP to be set up at some time (either now or in the 575 future). The request, indicated on Figure 2 by the arrow (a), 576 includes all of the parameters of the LSP that the requester wishes 577 to supply such as priority, bandwidth, start time, and end time. 578 Note that the requester in this case may be the LSR shown in the 579 figure or may be a distinct system. 581 The PCE enters the LSP request in its LSP-DB (b), and uses 582 information from its TED (c) to compute a path that satisfies the 583 constraints (such as bandwidth) for the LSP in the time interval from 584 the start time to the end time. It updates the future resource 585 availability in the TED so that further path computations can take 586 account of the scheduled resource usage. It stores the path for the 587 LSP into the LSP-DB (b). 589 When it is time (i.e., at the start time) for the LSP to be set up, 590 the PCE sends a PCEP Initiate request to the head end LSR (d) 591 providing the path to be signaled as well as other parameters such as 592 the bandwidth of the LSP. 594 As the LSP is signaled between LSRs (f) the use of resources in the 595 network is updated and distributed using the IGP. This information 596 is shared with the PCE either through the IGP or using BGP-LS (e), 597 and the PCE updates the information stored in its TED (c). 599 After the LSP is set up, the head end LSR sends a PCEP LSP State 600 Report (PCRpt message) to the PCE (d). The report contains the 601 resources such as bandwidth usage for the LSP. The PCE updates the 602 status of the LSP in the LSP-DB according to the report. 604 When an LSP is no longer required (either because the Service 605 Requester has cancelled the request, or because the LSP's scheduled 606 lifetime has expired) the PCE can remove it. If the LSP is currently 607 active, the PCE instructs the head-end LSR to tear it down (d), and 608 the network resource usage will be updated by the IGP and advertised 609 back to the PCE through the IGP or BGP-LS (e). Once the LSP is no 610 longer active, the PCE can remove it from the LSP-DB (b). 612 4.1.1. Reoptimization After TED Updates 614 When the TED is updated as indicated in Section 4.1, the PCE may 615 perform reoptimization of the LSPs for which it has computed paths 616 depending on operator policy so as to minimize network perturbations. 617 These LSPs may be already provisioned in which case the PCE issues 618 PCEP Update request messages for the LSPs that should be adjusted. 619 Additionally, the LSPs being reoptimized may be scheduled LSPs that 620 have not yet been provisioned, in which case reoptimization involves 621 updating the store of scheduled LSPs and resources. 623 In all cases, the purpose of reoptimization is to take account of the 624 resource usage and availability in the network and to compute paths 625 for the current and future LSPs that best satisfy the objectives of 626 those LSPs while keeping the network as clear as possible to support 627 further LSPs. Since reoptimization may perturb established LSPs, it 628 is subject to operator oversight and policy. As the stability of the 629 network will be impacted by frequent changes, the extent and impact 630 of any reoptimization needs to be subject to operator policy. 632 Additionally, the status of the reserved resources (alarms) can 633 enhance the computation and planning for future LSPs, and may 634 influence repair and reoptimization. Control of recalculations based 635 on failures and notifications to the operator is also subject to 636 policy. 638 See Section 3.3 for further discussion of operator policy. 640 4.2. Initialization and Recovery 642 When a PCE in the architecture shown in Figure 2 is initialized, it 643 must learn state from the network, from its stored databases, and 644 potentially from other PCEs in the network. 646 The first step is to get an accurate view of the topology and 647 resource availability in the network. This would normally involve 648 reading the state direct from the network via the IGP or BGP-LS (e), 649 but might include receiving a copy of the TED from another PCE. Note 650 that a TED stored from a previous instantiation of the PCE is 651 unlikely to be valid. 653 Next, the PCE must construct a time-based TED to show scheduled 654 resource usage. How it does this is implementation specific and this 655 document does not dictate any particular mechanism: it may recover a 656 time-based TED previously saved to non-volatile storage, or it may 657 reconstruct the time-based TED from information retrieved from the 658 LSP-DB previously saved to non-volatile storage. If there is more 659 than one PCE active in the network, the recovering PCE will need to 660 synchronize the LSP-DB and time-based TED with other PCEs (see 661 Section 4.3). 663 Note that the stored LSP-DB needs to include the intended state and 664 actual state of the LSPs so that when a PCE recovers it is able to 665 determine what actions are necessary. 667 4.3. Synchronization Between PCEs 669 If there is more than one PCE that supports scheduling active in the 670 network, it is important to achieve some consistency between the 671 scheduled TED and scheduled LSP-DB held by the PCEs. 673 [RFC7399] answers various questions around synchronization between 674 the PCEs. It should be noted that the time-based "scheduled" 675 information adds another dimension to the issue of synchronization 676 between PCEs. It should also be noted that a deployment may use a 677 primary PCE and the have other PCEs as backup, where a backup PCE can 678 take over only in the event of a failure of the primary PCE. 679 Alternatively, the PCEs may share the load at all times. The choice 680 of the synchronization technique is largely dependent on the 681 deployment of PCEs in the network. 683 One option for ensuring that multiple PCEs use the same scheduled 684 information is simply to have the PCEs driven from the same shared 685 database, but it is likely to be inefficient and interoperation 686 between multiple implementations will be harder. 688 Another option is for each PCE to be responsible for its own 689 scheduled database and to utilize some distributed database 690 synchronization mechanism to have consistent information. Depending 691 on the implementation, this could be efficient, but interoperation 692 between heterogeneous implementations is still hard. 694 A further approach is to utilize PCEP messages to synchronize the 695 scheduled state between PCEs. This approach would work well if the 696 number of PCEs which support scheduling is small, but as the number 697 increases considerable message exchange needs to happen to keep the 698 scheduled databases synchronized. Future solutions could also 699 utilize some synchronization optimization techniques for efficiency. 700 Another variation would be to request information from other PCEs for 701 a particular time slice, but this might have impact on the 702 optimization algorithm. 704 5. Multi-Domain Considerations 706 Multi-domain path computation usually requires some form of 707 cooperation between PCEs each of which has responsibility for 708 determining a segment of the end-to-end path in the domain for which 709 it has computational responsibility. When computing a scheduled 710 path, resources need to be booked in all of the domains that the path 711 will cross so that they are available when the LSP is finally 712 signalled. 714 Per-domain path computation [RFC5152] is not an appropriate mechanism 715 when a scheduled LSP is being computed because the computation 716 requests at downstream PCEs are only triggered by signaling. 717 However, a similar mechanism could be used where cooperating PCEs 718 exchange PCReq messages for a scheduled LSP as shown in Figure 3. In 719 this case the service requester asks for a scheduled LSP that will 720 span two domains (a). PCE1 computes a path across Domain 1 and 721 reserves the resources, and also asks PCE2 to compute and reserve in 722 Domain 2 (b). PCE2 may return a full path, or could return a path 723 key [RFC5520]. When it is time for LSP setup PCE1 triggers the head- 724 end LSR (c) and the LSP is signaled (d). If a path key is used, the 725 entry LSR in Domain 2 will consult PCE2 for the path expansion (e) 726 before completing signaling (f). 728 ------------------- 729 | Service Requester | 730 ------------------- 731 ^ 732 a| 733 v 734 ------ b ------ 735 | |<---------------->| | 736 | PCE1 | | PCE2 | 737 | | | | 738 ------ ------ 739 ^ ^ 740 | | 741 c| e| 742 | | 743 ----+----------------- ----+----------------- 744 | | Domain 1 | | | Domain 2 | 745 | v | | v | 746 | ----- d ----- | | ----- f ----- | 747 | | LSR |<--->| LSR |<-+--+->| LSR |<--->| LSR | | 748 | ----- ----- | | ----- ----- | 749 ---------------------- ---------------------- 751 Figure 3: Per-Domain Path Computation for Scheduled LSPs 753 Another mechanism for PCE cooperation in multi-domain LSP setup is 754 Backward- Recursive Path Computation (BRPC) [RFC5441]. This approach 755 relies on the downstream domain supply a variety of potential paths 756 to the upstream domain. Although BRPC can arrive at a more optimal 757 end-to-end path than per-domain path computation, it is not well 758 suited to LSP scheduling because the downstream PCE would need to 759 reserve resources on all of the potential paths and then release 760 those that the upstream PCE announced it did not plan to use. 762 Finally we should consider hierarchical PCE (H-PCE) [RFC6805]. This 763 mode of operation is similar to that shown in Figure 3, but a parent 764 PCE is used to coordinate the requests to the child PCEs resulting in 765 better visibility of the end-to-end path and better coordination of 766 the resource booking. The sequenced flow of control is shown in 767 Figure 4. 769 ------------------- 770 | Service Requester | 771 ------------------- 772 ^ 773 a| 774 v 775 -------- 776 | | 777 | Parent | 778 | PCE | 779 | | 780 -------- 781 ^ ^ b 782 b| |_______________________ 783 | | 784 v v 785 ------ ------ 786 | | | | 787 | PCE1 | | PCE2 | 788 | | | | 789 ------ ------ 790 ^ ^ 791 | | 792 c| e| 793 | | 794 ----+----------------- ----+----------------- 795 | | Domain 1 | | | Domain 2 | 796 | v | | v | 797 | ----- d ----- | | ----- f ----- | 798 | | LSR |<--->| LSR |<-+--+->| LSR |<--->| LSR | | 799 | ----- ----- | | ----- ----- | 800 ---------------------- ---------------------- 802 Figure 4: Hierarchical PCE for Path Computation for Scheduled LSPs 804 6. Security Considerations 806 The protocol implications of scheduled resources are unchanged from 807 "on-demand" LSP computation and setup. A discussion of securing PCEP 808 is found in [RFC5440] and work to extend that security is provided in 809 [RFC8253]. Furthermore, the path key mechanism described in 810 [RFC5520] can be used to enhance privacy and security. 812 Similarly, there is no change to the security implications for the 813 signaling of scheduled LSPs. A discussion of the security of the 814 signaling protocols that would be used is found in [RFC5920]. 816 However, the use of scheduled LSPs extends the attack surface for a 817 PCE-enabled TE system by providing a larger (logically infinite) 818 window during which an attack can be initiated or planned. That is, 819 if bogus scheduled LSPs can be requested and entered into the LSP-DB, 820 then a large number of LSPs could be launched, and significant 821 network resources could be blocked. Control of scheduling requests 822 needs to be subject to operator policy and additional authorization 823 needs to be applied for access to LSP scheduling. Diagnostic tools 824 need to be provided to inspect the LSP DB to spot attacks. 826 7. IANA Considerations 828 This architecture document makes no request for IANA action. 830 8. Acknowledgements 832 This work has benefited from the discussions of resource scheduling 833 over the years. In particular the DRAGON project [DRAGON] and 834 [I-D.yong-ccamp-ason-gmpls-autobw-service] both of which provide 835 approaches to auto-bandwidth services in GMPLS networks. 837 Mehmet Toy, Lei Liu, and Khuzema Pithewan contributed to an earlier 838 version of [I-D.chen-teas-frmwk-tts]. We would like to thank the 839 authors of that draft on Temporal Tunnel Services for material that 840 assisted in thinking about this document. 842 Thanks to Michael Scharf and Daniele Ceccarelli for useful comments 843 on this work. 845 Jonathan Hardwick provided a helpful Routing Directorate review. 847 Deborah Brungard suggested many changes during her AD review. 849 9. Contributors 851 The following people contributed to discussions that led to the 852 development of this document: 854 Dhruv Dhody 855 Email: dhruv.dhody@huawei.com 857 10. Informative References 859 [DRAGON] National Science Foundation, "http://www.maxgigapop.net/ 860 wp-content/uploads/The-DRAGON-Project.pdf". 862 [I-D.chen-teas-frmwk-tts] 863 Chen, H., Toy, M., Liu, L., and K. Pithewan, "Framework 864 for Temporal Tunnel Services", draft-chen-teas-frmwk- 865 tts-01 (work in progress), March 2016. 867 [I-D.yong-ccamp-ason-gmpls-autobw-service] 868 Yong, L. and Y. Lee, "ASON/GMPLS Extension for Reservation 869 and Time Based Automatic Bandwidth Service", draft-yong- 870 ccamp-ason-gmpls-autobw-service-00 (work in progress), 871 October 2006. 873 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 874 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 875 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 876 . 878 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 879 Switching (GMPLS) Signaling Resource ReserVation Protocol- 880 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 881 DOI 10.17487/RFC3473, January 2003, 882 . 884 [RFC3945] Mannie, E., Ed., "Generalized Multi-Protocol Label 885 Switching (GMPLS) Architecture", RFC 3945, 886 DOI 10.17487/RFC3945, October 2004, 887 . 889 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 890 Element (PCE)-Based Architecture", RFC 4655, 891 DOI 10.17487/RFC4655, August 2006, 892 . 894 [RFC5063] Satyanarayana, A., Ed. and R. Rahman, Ed., "Extensions to 895 GMPLS Resource Reservation Protocol (RSVP) Graceful 896 Restart", RFC 5063, DOI 10.17487/RFC5063, October 2007, 897 . 899 [RFC5152] Vasseur, JP., Ed., Ayyangar, A., Ed., and R. Zhang, "A 900 Per-Domain Path Computation Method for Establishing Inter- 901 Domain Traffic Engineering (TE) Label Switched Paths 902 (LSPs)", RFC 5152, DOI 10.17487/RFC5152, February 2008, 903 . 905 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 906 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 907 DOI 10.17487/RFC5440, March 2009, 908 . 910 [RFC5441] Vasseur, JP., Ed., Zhang, R., Bitar, N., and JL. Le Roux, 911 "A Backward-Recursive PCE-Based Computation (BRPC) 912 Procedure to Compute Shortest Constrained Inter-Domain 913 Traffic Engineering Label Switched Paths", RFC 5441, 914 DOI 10.17487/RFC5441, April 2009, 915 . 917 [RFC5520] Bradford, R., Ed., Vasseur, JP., and A. Farrel, 918 "Preserving Topology Confidentiality in Inter-Domain Path 919 Computation Using a Path-Key-Based Mechanism", RFC 5520, 920 DOI 10.17487/RFC5520, April 2009, 921 . 923 [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS 924 Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, 925 . 927 [RFC6805] King, D., Ed. and A. Farrel, Ed., "The Application of the 928 Path Computation Element Architecture to the Determination 929 of a Sequence of Domains in MPLS and GMPLS", RFC 6805, 930 DOI 10.17487/RFC6805, November 2012, 931 . 933 [RFC7399] Farrel, A. and D. King, "Unanswered Questions in the Path 934 Computation Element Architecture", RFC 7399, 935 DOI 10.17487/RFC7399, October 2014, 936 . 938 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 939 S. Ray, "North-Bound Distribution of Link-State and 940 Traffic Engineering (TE) Information Using BGP", RFC 7752, 941 DOI 10.17487/RFC7752, March 2016, 942 . 944 [RFC8231] Crabbe, E., Minei, I., Medved, J., and R. Varga, "Path 945 Computation Element Communication Protocol (PCEP) 946 Extensions for Stateful PCE", RFC 8231, 947 DOI 10.17487/RFC8231, September 2017, 948 . 950 [RFC8253] Lopez, D., Gonzalez de Dios, O., Wu, Q., and D. Dhody, 951 "PCEPS: Usage of TLS to Provide a Secure Transport for the 952 Path Computation Element Communication Protocol (PCEP)", 953 RFC 8253, DOI 10.17487/RFC8253, October 2017, 954 . 956 Authors' Addresses 958 Yan Zhuang 959 Huawei 960 101 Software Avenue, Yuhua District 961 Nanjing, Jiangsu 210012 962 China 964 Email: zhuangyan.zhuang@huawei.com 966 Qin Wu 967 Huawei 968 101 Software Avenue, Yuhua District 969 Nanjing, Jiangsu 210012 970 China 972 Email: bill.wu@huawei.com 974 Huaimo Chen 975 Huawei 976 Boston, MA 977 US 979 Email: huaimo.chen@huawei.com 981 Adrian Farrel 982 Juniper Networks 984 Email: afarrel@juniper.net