idnits 2.17.1 draft-ietf-pce-inter-layer-frwk-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group E. Oki 3 Internet Draft University of Electro-Communications 4 Category: Informational Tomonori Takeda 5 Created: March, 2009 NTT 6 Expires: November, 2009 J-L Le Roux 7 France Telecom 8 A. Farrel 9 Old Dog Consulting 11 Framework for PCE-Based Inter-Layer MPLS and GMPLS Traffic Engineering 13 draft-ietf-pce-inter-layer-frwk-10.txt 15 Status of this Memo 17 This Internet-Draft is submitted to IETF in full conformance with 18 the provisions of BCP 78 and BCP 79. 20 Internet-Drafts are working documents of the Internet Engineering 21 Task Force (IETF), its areas, and its working groups. Note that 22 other groups may also distribute working documents as Internet- 23 Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six 26 months and may be updated, replaced, or obsoleted by other documents 27 at any time. It is inappropriate to use Internet-Drafts as reference 28 material or to cite them other than as "work in progress." 30 The list of current Internet-Drafts can be accessed at 31 http://www.ietf.org/ietf/1id-abstracts.txt. 33 The list of Internet-Draft Shadow Directories can be accessed at 34 http://www.ietf.org/shadow.html. 36 Abstract 38 A network may comprise multiple layers. It is important to globally 39 optimize network resource utilization, taking into account all 40 layers, rather than optimizing resource utilization at each layer 41 independently. This allows better network efficiency to be achieved 42 through a process that we call inter-layer traffic engineering. The 43 Path Computation Element (PCE) can be a powerful tool to achieve 44 inter-layer traffic engineering. 46 This document describes a framework for applying the PCE-based 47 architecture to inter-layer Multiprotocol Label Switching (MPLS) and 48 Generalized MPLS (GMPLS) traffic engineering. It provides 49 suggestions for the deployment of PCE in support of multi-layer 50 networks. This document also describes network models where PCE 51 performs inter-layer traffic engineering, and the relationship 52 between PCE and a functional component called the Virtual Network 53 Topology Manager (VNTM). 55 Table of Contents 57 1. Introduction...................................................3 58 1.1. Terminology..................................................3 59 2. Inter-Layer Path Computation...................................4 60 3. Inter-Layer Path Computation Models............................6 61 3.1. Single PCE Inter-Layer Path Computation......................7 62 3.2. Multiple PCE Inter-Layer Path Computation....................7 63 3.3. General Observations.........................................9 64 4. Inter-Layer Path Control......................................10 65 4.1. VNT Management..............................................10 66 4.2. Inter-Layer Path Control Models.............................10 67 4.2.1. PCE-VNTM Cooperation Model................................10 68 4.2.2. Higher-Layer Signaling Trigger Model......................12 69 4.2.3. NMS-VNTM Cooperation Model................................15 70 4.2.4. Possible Combinations of Inter-Layer Path Computation and 71 Inter-Layer Path Control Models..................................20 72 5. Choosing Between Inter-Layer Path Control Models..............21 73 5.1. VNTM Functions..............................................21 74 5.2. Border LSR Functions........................................22 75 5.3. Complete Inter-Layer LSP Setup Time.........................22 76 5.4. Network Complexity..........................................23 77 5.5. Separation of Layer Management..............................24 78 6. Stability Considerations......................................24 79 7. IANA Considerations...........................................25 80 8. Manageability Considerations..................................25 81 8.1. Control of Function and Policy..............................25 82 8.1.1. Control of Inter-Layer Computation Function...............25 83 8.1.2. Control of Per-Layer Policy...............................26 84 8.1.3. Control of Inter-Layer Policy.............................26 85 8.2. Information and Data Models.................................27 86 8.3. Liveness Detection and Monitoring...........................27 87 8.4. Verifying Correct Operation.................................28 88 8.5. Requirements on Other Protocols and Functional Components...28 89 8.6. Impact on Network Operation.................................28 90 9. Security Considerations.......................................29 91 10. Acknowledgments..............................................30 92 11. References...................................................30 93 11.1. Normative Reference........................................30 94 11.2. Informative Reference......................................31 95 12. Authors' Addresses...........................................32 96 13. Intellectual Property Statement..............................32 97 14. Full Copyright Statement.....................................33 99 1. Introduction 101 A network may comprise multiple layers. These layers may represent 102 separations of technologies (e.g., packet switch capable (PSC), time 103 division multiplex (TDM), or lambda switch capable (LSC)) [RFC3945], 104 separation of data plane switching granularity levels (e.g., PSC-1, 105 PSC-2, VC4, or VC12) [RFC5212], or a distinction between client and 106 server networking roles. In this multi-layer network, Label Switched 107 Paths (LSPs) in a lower layer are used to carry higher-layer LSPs 108 across the lower-layer network. The network topology formed by 109 lower-layer LSPs and advertised as traffic engineering links (TE 110 links) in the higher layer network is called the Virtual Network 111 Topology (VNT) [RFC5212]. 113 It may be effective to optimize network resource utilization 114 globally, i.e., taking into account all layers, rather than 115 optimizing resource utilization at each layer independently. This 116 allows better network efficiency to be achieved and is what we call 117 inter-layer traffic engineering. This includes mechanisms allowing 118 the computation of end-to-end paths across layers (known as inter- 119 layer path computation), and mechanisms for control and management 120 of the Virtual Network Topology (VNT) by setting up and releasing 121 LSPs in the lower layers [RFC5212]. 123 Inter-layer traffic engineering is included in the scope of the Path 124 Computation Element (PCE)-based architecture [RFC4655], and PCE can 125 provide a suitable mechanism for resolving inter-layer path 126 computation issues. 128 PCE Communication Protocol requirements for inter-layer traffic 129 engineering are set out in [PCE-INTER-LAYER-REQ]. 131 This document describes a framework for applying the PCE-based 132 architecture to inter-layer traffic engineering. It provides 133 suggestions for the deployment of PCE in support of multi-layer 134 networks. This document also describes network models where PCE 135 performs inter-layer traffic engineering, and the relationship 136 between PCE and a functional component in charge of the control and 137 management of the VNT, called the Virtual Network Topology Manager 138 (VNTM). 140 1.1. Terminology 142 This document uses terminology from the PCE-based path computation 143 architecture [RFC4655] and also common terminology from Multi 144 Protocol Label Switching (MPLS) [RFC3031], Generalized MPLS (GMPLS) 145 [RFC3945], and Multi-Layer Networks [RFC5212]. 147 2. Inter-Layer Path Computation 149 This section describes key topics of inter-layer path computation in 150 MPLS and GMPLS networks. 152 [RFC4206] defines a way to signal a higher-layer LSP, which has an 153 explicit route that includes hops traversed by LSPs in lower layers. 154 The computation of end-to-end paths across layers is called Inter- 155 Layer Path Computation. 157 A Label Switching Router (LSR) in the higher-layer might not have 158 information on the topology of the lower-layer, particularly in an 159 overlay or augmented model deployment, and hence may not be able to 160 compute an end-to-end path across layers. 162 PCE-based Inter-Layer Path Computation consists of using one or more 163 PCEs to compute an end-to-end path across layers. This could be 164 achieved by a single PCE path computation where the PCE has topology 165 information about multiple layers and can directly compute an end- 166 to-end path across layers considering the topology of all of the 167 layers. Alternatively, the inter-layer path computation could be 168 performed as a multiple PCE computation where each member of a set 169 of PCEs has information about the topology of one or more layers 170 (but not all layers), and the PCEs collaborate to compute an end-to- 171 end path. 173 ----- ----- ----- ----- 174 | LSR |--| LSR |................| LSR |--| LSR | 175 | H1 | | H2 | | H3 | | H4 | 176 ----- -----\ /----- ----- 177 \----- -----/ 178 | LSR |--| LSR | 179 | L1 | | L2 | 180 ----- ----- 182 Figure 1 - A Simple Example of a Multi-Layer Network. 184 Consider, for instance, the two-layer network shown in Figure 1, 185 where the higher-layer network is a packet-based IP/MPLS or GMPLS 186 network (LSRs H1, H2, H3, and H4), and the lower-layer network 187 (LSRs, H2, L1, L2, and H3) is a GMPLS optical network. An ingress 188 LSR in the higher-layer network (H1) tries to set up an LSP to an 189 egress LSR (H4) also in the higher-layer network across the lower- 190 layer network, and needs a path in the higher-layer network. However, 191 suppose that there is no TE link in the higher-layer network between 192 the border LSRs located on the boundary between the higher-layer and 193 lower-layer networks (H2 and H3). Suppose also that the ingress LSR 194 does not have topology visibility into the lower layer. If a single- 195 layer path computation is applied in the higher-layer, the path 196 computation fails because of the missing TE link. On the other hand, 197 inter-layer path computation is able to provide a route in the 198 higher-layer (H1-H2-H3-H4) and a suggestion that a lower-layer LSP 199 be set up between the border LSRs (H2-L1-L2-H3). 201 Lower-layer LSPs that are advertised as TE links into the higher- 202 layer network form a Virtual Network Topology (VNT) that can be used 203 for routing higher-layer LSPs. Inter-layer path computation for end- 204 to-end LSPs in the higher-layer network that span the lower-layer 205 network may utilize the VNT, and PCE is a candidate for computing 206 the paths of such higher-layer LSPs within the higher-layer network. 207 Alternatively, the PCE-based path computation model can: 209 - Perform a single computation on behalf of the ingress LSR using 210 information gathered from more than one layer. This mode is 211 referred to as Single PCE Computation in [RFC4655]. 213 - Compute a path on behalf of the ingress LSR through cooperation 214 with PCEs responsible for each layer. This mode is referred to as 215 Multiple PCE Computation with inter-PCE communication in [RFC4655]. 217 - Perform separate path computations on behalf of the TE-LSP head- 218 end and each transit border LSR that is the entry point to a new 219 layer. This mode is referred to as Multiple PCE Computation 220 (without inter-PCE communication) in [RFC4655]. This option 221 utilizes per-layer path computation performed independently by 222 successive PCEs. 224 Note that when a network consists of more than two layers (e.g., MPLS 225 over SONET over OTN), and a path traversing more than two layers 226 needs to be computed, it is possible to combine multiple PCE-based 227 path computation models. For example, the single PCE computation 228 model could be used for computing a path across the SONET layer and 229 the OTN layer, and the multiple PCE computation with inter-PCE 230 communication model could be used for computing a path across the 231 MPLS layer (computed by higher-layer PCE) and the SONET layer 232 (computed by lower-layer PCE). 234 The PCE invoked by the head-end LSR computes a path that the LSR can 235 use to signal an MPLS-TE or GMPLS LSP once the path information has 236 been converted to an Explicit Route Object (ERO) for use in RSVP-TE 237 signaling. There are two options. 239 - Option 1: Mono-layer path. 241 The PCE computes a "mono-layer" path, i.e., a path that includes 242 only TE links from the same layer. There are two cases for this 243 option. In the first case the PCE computes a path that includes 244 already established lower-layer LSPs or lower-layer LSPs to be 245 established on demand. That is, the resulting ERO includes sub- 246 object(s) corresponding to lower-layer hierarchical LSPs expressed 247 as the TE link identifiers of the hierarchical LSPs when advertised 248 as TE links in the higher-layer network. The TE link may be a 249 regular TE link that is actually established, or a virtual TE link 250 that is not established yet (see [RFC5212]). If it is a virtual TE 251 link, this triggers a setup attempt for a new lower-layer LSP when 252 signaling reaches the head-end of the lower-layer LSP. Note that 253 the path of a virtual TE link is not necessarily known in advance, 254 and this may require a further (lower-layer) path computation. 256 The second case is that the PCE computes a path that includes a 257 loose hop that spans the lower-layer network. The higher layer path 258 computation selects which lower layer network to use, and selects 259 the entry and exit points of that lower-layer network, but does not 260 select the path across the lower-layer network. A transit LSR that 261 is the entry point to the lower-layer network is expected to expand 262 the loose hop (either itself or relying on the services of a PCE). 263 The path expansion process on the border LSR may result either in 264 the selection of an existing lower-layer LSP, or in the computation 265 and setup of a new lower-layer LSP. 267 Note that even if a PCE computes a path with a loose hop expecting 268 that the loose hop will be expanded across the lower-layer network, 269 the LSR (that is an entry point to the lower-layer network) may 270 simply expand the loose hop in the same layer. If more strict 271 control of how the LSR establishes the path is required, mechanisms 272 such as Path Key [PATH-KEY] could be applied. 274 - Option 2: Multi-layer path. 276 The PCE computes a "multi-layer" path, i.e., a path that includes 277 TE links from distinct layers [RFC4206]. Such a path can include 278 the complete path of one or more lower-layer LSPs that already 279 exist or are not yet established. In the latter case, the signaling 280 of the higher-layer LSP will trigger the establishment of the 281 lower-layer LSPs. 283 3. Inter-Layer Path Computation Models 285 In Section 2, three models are defined to perform PCE-based inter- 286 layer path computation, namely Single PCE Computation, Multiple PCE 287 Computation with inter-PCE communication, and Multiple PCE 288 Computation without inter-PCE communication. Single PCE Computation 289 is discussed in Section 3.1 below, and Multiple PCE Computation (with 290 or without inter-PCE communication) is discussed in Section 3.2 291 below. 293 3.1. Single PCE Inter-Layer Path Computation 295 In this model inter-layer path computation is performed by a single 296 PCE that has topology visibility into all layers. Such a PCE is 297 called a multi-layer PCE. 299 In Figure 2, the network is comprised of two layers. LSRs H1, H2, H3, 300 and H4 belong to the higher layer, and LSRs H2, H3, L1, and L2 301 belong to the lower layer. The PCE is a multi-layer PCE that has 302 visibility into both layers. It can perform end-to-end path 303 computation across layers (single PCE path computation). For 304 instance, it can compute an optimal path H1-H2-L1-L2-H3-H4, for a 305 higher layer LSP from H1 to H4. This path includes the path of a 306 lower layer LSP from H2 to H3, already in existence or not yet 307 established. 309 ----- 310 | PCE | 311 ----- 312 ----- ----- ----- ----- 313 | LSR |--| LSR |................| LSR |--| LSR | 314 | H1 | | H2 | | H3 | | H4 | 315 ----- -----\ /----- ----- 316 \----- -----/ 317 | LSR |--| LSR | 318 | L1 | | L2 | 319 ----- ----- 321 Figure 2: Single PCE Inter-Layer Path Computation 323 3.2. Multiple PCE Inter-Layer Path Computation 325 In this model there is at least one PCE per layer, and each PCE has 326 topology visibility restricted to its own layer. Some providers may 327 want to keep the layer boundaries due to factors such as 328 organizational and/or service management issues. The choice for 329 multiple PCE computation instead of single PCE computation may also 330 be driven by scalability considerations, as in this mode a PCE only 331 needs to maintain topology information for one layer (resulting in a 332 size reduction for the Traffic Engineering Database (TED)). 334 These PCEs are called mono-layer PCEs. Mono-layer PCEs collaborate 335 to compute an end-to-end optimal path across layers. 337 Figure 3 shows multiple PCE inter-layer computation with inter-PCE 338 communication. There is one PCE in each layer. The PCEs from each 339 layer collaborate to compute an end-to-end path across layers. PCE 340 Hi is responsible for computations in the higher layer and may 341 "consult" with PCE Lo to compute paths across the lower layer. PCE 342 Lo is responsible for path computation in the lower layer. A simple 343 example of cooperation between the PCEs could be as follows: 345 - LSR H1 sends a request for a path H1-H4 to PCE Hi 346 - PCE Hi selects H2 as the entry point to the lower layer, and H3 as 347 the exit point. 348 - PCE Hi requests a path H2-H3 from PCE Lo. 349 - PCE Lo returns H2-L1-L2-H3 to PCE Hi. 350 - PEC Hi is now able to compute the full path (H1-H2-L1-L2-H3-H4) 351 and return it to H1. 353 Of course, more complex cooperation may be required if an optimal 354 end-to-end path is desired. 356 ----- 357 | PCE | 358 | Hi | 359 --+-- 360 | 361 ----- ----- | ----- ----- 362 | LSR |--| LSR |............|...........| LSR |--| LSR | 363 | H1 | | H2 | | | H3 | | H4 | 364 ----- -----\ --+-- /----- ----- 365 \ | PCE | / 366 \ | Lo | / 367 \ ----- / 368 \ / 369 \----- -----/ 370 | LSR |--| LSR | 371 | L1 | | L2 | 372 ----- ----- 374 Figure 3: Multiple PCE Inter-Layer Path Computation with Inter-PCE 375 Communication 377 Figure 4 shows multiple PCE inter-layer path computation without 378 inter-PCE communication. As described in Section 2, separate path 379 computations are performed on behalf of the TE-LSP head-end and each 380 transit border LSR that is the entry point to a new layer. 382 ----- 383 | PCE | 384 | Hi | 385 ----- 386 ----- ----- ----- ----- 387 | LSR |--| LSR |........................| LSR |--| LSR | 388 | H1 | | H2 | | H3 | | H4 | 389 ----- -----\ ----- /----- ----- 390 \ | PCE | / 391 \ | Lo | / 392 \ ----- / 393 \ / 394 \----- -----/ 395 | LSR |--| LSR | 396 | L1 | | L2 | 397 ----- ----- 399 Figure 4: Multiple PCE Inter-layer Path Computation Without Inter- 400 PCE Communication 402 3.3. General Observations 404 - Depending on implementation details, the time to perform inter- 405 layer path computation in the single PCE inter-layer path 406 computation model may be less than that of the multiple PCE model 407 with cooperating mono-layer PCEs, because there is no requirement 408 to exchange messages between cooperating PCEs. 410 - When TE topology for all layer networks is visible within one 411 routing domain, the single PCE inter-layer path computation model 412 may be adopted because a PCE is able to collect all layers' TE 413 topologies by participating in only one routing domain. 415 - As the single PCE inter-layer path computation model uses more TE 416 topology information in one computation than is used by PCEs in the 417 multiple PCE path computation model, it requires more computation 418 power and memory. 420 When there are multiple candidate layer border nodes (we may say 421 that the higher layer is multi-homed), optimal path computation 422 requires that all the possible paths transiting different layer 423 border nodes or links be examined. This is relatively simple in the 424 single PCE inter-layer path computation model because the PCE has 425 full visibility - the computation is similar to the computation 426 within a single domain of a single layer. In the multiple PCE inter- 427 layer path computation model, backward recursive techniques 428 described in [BRPC] could be used, by considering layers as separate 429 domains. 431 4. Inter-Layer Path Control 433 4.1. VNT Management 435 As a result of mono-layer path computation, a PCE may determine that 436 there is insufficient bandwidth available in the higher-layer 437 network to support this or future higher-layer LSPs. The problem 438 might be resolved if new LSPs were provisioned across the lower- 439 layer network. Furthermore, the modification, re-organization and 440 new provisioning of lower-layer LSPs may enable better utilization 441 of lower-layer network resources given the demands of the higher- 442 layer network. In other words, the VNT needs to be controlled or 443 managed in cooperation with inter-layer path computation. 445 A VNT Manager (VNTM) is defined as a functional element that manages 446 and controls the VNT. PCE and VNT Manager are distinct functional 447 elements that may or may not be co-located. 449 4.2. Inter-Layer Path Control Models 451 4.2.1. PCE-VNTM Cooperation Model 453 ----- ------ 454 | PCE |--->| VNTM | 455 ----- ------ 456 ^ : 457 : : 458 : : 459 v V 460 ----- ----- ----- ----- 461 | LSR |----| LSR |................| LSR |----| LSR | 462 | H1 | | H2 | | H3 | | H4 | 463 ----- -----\ /----- ----- 464 \----- -----/ 465 | LSR |--| LSR | 466 | L1 | | L2 | 467 ----- ----- 469 Figure 5: PCE-VNTM Cooperation Model 471 A multi-layer network consists of higher-layer and lower-layer 472 networks. LSRs H1, H2, H3, and H4 belong to the higher-layer network, 473 LSRs H2, L1, L2, and H3 belong to the lower-layer network, as shown 474 in Figure 5. The case of single PCE inter-layer path computation is 475 considered here to explain the cooperation model between PCE and 476 VNTM, but multiple PCE path computation with or without inter-PCE 477 communication can also be applied to this model. 479 Consider that H1 requests the PCE to compute an inter-layer path 480 between H1 and H4. There is no TE link in the higher-layer between 481 H2 and H3 before the path computation request, so the request fails. 482 But the PCE may provide information to the VNT Manager responsible 483 for the lower layer network that may help resolve the situation for 484 future higher-layer LSP setup. 486 The roles of PCE and VNTM are as follows. PCE performs inter-layer 487 path computation and is unable to supply a path because there is no 488 TE link between H2 and H3. The computation fails, but PCE suggests 489 to VNTM that a lower-layer LSP (H2-H3) could be established to 490 support future LSP requests. Messages from PCE to VNTM contain 491 information about the higher-layer demand (from H2 to H3), and may 492 include a suggested path in the lower layer (if the PCE has 493 visibility into the lower layer network). VNTM uses local policy and 494 possibly management/configuration input to determine how to process 495 the suggestion from PCE, and may request an ingress LSR (e.g. H2) to 496 establish a lower-layer LSP. VNTM or the ingress LSR (H2) may 497 themselves use a PCE with visibility into the lower layer to compute 498 the path of this new LSP. 500 When the higher-layer PCE fails to compute a path and notifies VNTM, 501 it may wait for the lower-layer LSP to be set up and advertised as a 502 TE link. PCE may have a timer. After TED is updated within a 503 specified duration, PCE will know a new TE link. It could then 504 compute the complete end-to-end path for the higher-layer LSP and 505 return the result to the PCC. In this case, the PCC may be kept 506 waiting for some time, and it is important that the PCC understands 507 this. It is also important that the PCE and VNTM have an agreement 508 that the lower-layer LSP will be set up in a timely manner, or that 509 the PCE will be notified by VNTM that no new LSP will become 510 available. In any case, if the PCE decides to wait, it must operate 511 a timeout. An example of such a cooperative procedure between PCE 512 and VNTM is as follows using the example network in Figure 4. 514 Step 1: H1 (PCC) requests PCE to compute a path between H1 and H4. 516 Step 2: The path computation fails because there is no TE link 517 across the lower-layer network. 519 Step 3: PCE suggests to VNTM that a new TE link connecting H2 and H3 520 would be useful. The PCE notifies VNTM that it will be waiting for 521 the TE link to be created. VNTM considers whether lower-layer LSPs 522 should be established if necessary and if acceptable within VNTM's 523 policy constraints. 525 Step 4: VNTM requests an ingress LSR in the lower-layer network 526 (e.g., H2) to establish a lower-layer LSP. The request message may 527 include a lower-layer LSP route obtained from the PCE responsible 528 for the lower-layer network. 530 Step 5: The ingress LSR signals to establish the lower-layer LSP. 532 Step 6: If the lower-layer LSP setup is successful, the ingress LSR 533 notifies VNTM that the LSP is complete and supplies the tunnel 534 information. 536 Step 7: The ingress LSR (H2) advertises the new LSP as a TE link in 537 the higher-layer network routing instance. 539 Step 8: PCE notices the new TE link advertisement and recomputes the 540 requested path. 542 Step 9: PCE replies to H1 (PCC) with a computed higher-layer LSP 543 route. The computed path is categorized as a mono-layer path that 544 includes the already-established lower layer-LSP as a single hop in 545 the higher layer. The higher-layer route is specified as H1-H2-H3-H4, 546 where all hops are strict. 548 Step 10: H1 initiates signaling with the computed path H2-H3-H4 to 549 establish the higher-layer LSP. 551 4.2.2. Higher-Layer Signaling Trigger Model 553 ----- 554 | PCE | 555 ----- 556 ^ 557 : 558 : 559 v 560 ----- ----- ----- ----- 561 | LSR |----| LSR |................| LSR |--| LSR | 562 | H1 | | H2 | | H3 | | H4 | 563 ----- -----\ /----- ----- 564 \----- -----/ 565 | LSR |--| LSR | 566 | L1 | | L2 | 567 ----- ----- 569 Figure 6: Higher-layer Signaling Trigger Model 571 Figure 6 shows the higher-layer signaling trigger model. The case of 572 single PCE path computation is considered to explain the higher- 573 layer signaling trigger model here, but multiple PCE path 574 computation with/without inter-PCE communication can also be applied 575 to this model. 577 As in the case described in Section 4.2.1, consider that H1 requests 578 PCE to compute a path between H1 and H4. There is no TE link in the 579 higher-layer between H2 and H3 before the path computation request. 581 PCE is unable to compute a mono-layer path, but may judge that the 582 establishment of a lower-layer LSP between H2 and H3 would provide 583 adequate connectivity. If the PCE has inter-layer visibility it may 584 return a path that includes hops in the lower layer (H1-H2-L1-L2-H3- 585 H4), but if it has no visibility into the lower layer, it may return 586 a path with a loose hop from H2 to H3 (H1-H2-H3(loose)-H4). The 587 former is a multi-layer path, and the latter a mono-layer path that 588 includes loose hops. 590 In the higher-layer signaling trigger model with a multi-layer path, 591 the LSP route supplied by the PCE includes the route of a lower- 592 layer LSP that is not yet established. A border LSR that is located 593 at the boundary between the higher-layer and lower-layer networks 594 (H2 in this example) receives a higher-layer signaling message, 595 notices that the next hop is in the lower-layer network, starts to 596 setup the lower-layer LSP as described in [RFC4206]. Note that these 597 actions depend on a policy being applied at the border LSR. An 598 example procedure of the signaling trigger model with a multi-layer 599 path is as follows. 601 Step 1: H1 (PCC) requests PCE to compute a path between H1 and H4. 602 The request indicates that inter-layer path computation is allowed. 604 Step 2: As a result of the inter-layer path computation, PCE judges 605 that a new lower-layer LSP needs to be established. 607 Step 3: PCE replies to H1 (PCC) with a computed multi-layer route 608 including higher-layer and lower-layer LSP routes. The route may be 609 specified as H1-H2-L1-L2-H3-H4, where all hops are strict. 611 Step 4: H1 initiates higher-layer signaling using the computed 612 explicit router of H2-L1-L2-H3-H4. 614 Step 5: The border LSR (H2) that receives the higher-layer signaling 615 message starts lower-layer signaling to establish a lower-layer LSP 616 along the specified lower-layer route of H2-L1-L2-H3. That is, the 617 border LSR recognizes the hops within the explicit route that apply 618 to the lower-layer network, verifies with local policy that a new 619 LSP is acceptable, and establishes the required lower-layer LSP. 620 Note that it is possible that a suitable lower-layer LSP has already 621 been established (or become available) between the time that the 622 computation was performed and the moment when the higher-layer 623 signaling message reached the border LSR. In this case, the border 624 LSR may select such a lower-layer LSP without the need to signal a 625 new LSP provided that the lower-layer LSP satisfies the explicit 626 route in the higher-layer signaling request. 628 Step 6: After the lower-layer LSP is established, the higher-layer 629 signaling continues along the specified higher-layer route of H2-H3- 630 H4 using hierarchical signaling [RFC4206]. 632 On the other hand, in the signaling trigger model with a mono-layer 633 path, a higher-layer LSP route includes a loose hop to traverse the 634 lower-layer network between the two border LSRs. A border LSR that 635 receives a higher-layer signaling message needs to determine a path 636 for a new lower-layer LSP. It applies local policy to verify that a 637 new LSP is acceptable and then either consults a PCE with 638 responsibility for the lower-layer network or computes the path by 639 itself, and initiates signaling to establish the lower-layer LSP. 640 Again, it is possible that a suitable lower-layer LSP has already 641 been established (or become available). In this case, the border LSR 642 may select such a lower-layer LSP without the need to signal a new 643 LSP provided that the existing lower-layer LSP satisfies the 644 explicit route in the higher-layer signaling request. Since the 645 higher-layer signaling request used a loose hop without specifying 646 any specifics of the path within the lower-layer network, the border 647 LSR has greater freedom to choose a lower-layer LSP than in the 648 previous example. 650 The difference between procedures of the signaling trigger model 651 with a multi-layer path and a mono-layer path is Step 5. Step 5 of 652 the signaling trigger model with a mono-layer path is as follows: 654 Step 5': The border LSR (H2) that receives the higher-layer 655 signaling message applies local policy to verify that a new LSP is 656 acceptable and then initiates establishment of a lower-layer LSP. It 657 either consults a PCE with responsibility for the lower-layer 658 network or computes the route by itself to expand the loose hop 659 route in the higher-layer path. 661 Finally, note that a virtual TE link may have been advertised into 662 the higher-layer network. This causes the PCE to return a path H1- 663 H2-H3-H4 where all the hops are strict. But when the higher-layer 664 signaling message reaches the layer border node H2 (that was 665 responsible for advertising the virtual TE link) it realizes that 666 the TE link does not exist yet, and signals the necessary LSP across 667 the lower-layer network using its own path determination (just as 668 for a loose hop in the higher layer) before continuing with the 669 higher-layer signaling. 671 PCE 672 ^ 673 : 674 : 675 V 676 H1--H2 H3--H4 677 \ / 678 L1==L2==L3--L4--L5 679 | 680 | 681 L6--L7 682 \ 683 H5--H6 685 Figure 7: Example of a Multi-Layer Network 687 Examples of multi-layer EROs are explained using Figure 7. It is 688 described how lower-layer LSP setup is performed in the higher-layer 689 signaling trigger model using an ERO that can include subobjects in 690 both the higher and lower layers. It gives rise to several options 691 for the ERO when it reaches the last LSR in the higher layer network 692 (H2). 693 1. The next subobject is a loose hop to H3 (mono layer ERO). 694 2. The next subobject is a strict hop to L1 followed by a loose hop 695 to H3. 696 3. The next subobjects are a series of hops (strict or loose) in the 697 lower-layer network followed by H3. For example, {L1(strict), 698 L3(loose), L5(loose), H3(strict)} 700 In the first example, the lower layer can utilize any LSP tunnel 701 that will deliver the end-to-end LSP to H3. In the third case, the 702 lower layer must select an LSP tunnel that traverses L3 and L5. 703 However, this does not mean that the lower layer can or should use 704 an LSP from L1 to L3 and another from L3 to L5. 706 4.2.3. NMS-VNTM Cooperation Model 708 In this model, NMS and VNTM cooperate to establish a lower-layer LSP. 709 There are two flavors in this model. One is where interaction between 710 layers in path computation is performed at the PCE level. This is 711 called "integrated flavor". The other is where interaction between 712 layers in path computation is achieved through NMS and VNTM 713 cooperation, which could be a point of application of administrative, 714 billing, and security policy. This is called "separated flavor". 716 o NMS-VNTM Cooperation Model (integrated flavor) 717 ------ ----- 718 | NMS |<-->| PCE | 719 | | ----- 720 | ---- | 721 ||VNTM|| 722 | ---- | 723 ------ 724 : : 725 : --------- 726 : : 727 V V 728 ----- ----- ----- ----- 729 | LSR |----| LSR |................| LSR |----| LSR | 730 | H1 | | H2 | | H3 | | H4 | 731 ----- -----\ /----- ----- 732 \----- -----/ 733 | LSR |--| LSR | 734 | L1 | | L2 | 735 ----- ----- 737 Figure 8: NMS-VNTM Cooperation Model (integrated flavor) 739 Figure 8 shows NMS-VNTM cooperation model (integrated flavor). The 740 case of single PCE path computation is considered to explain the NMS- 741 VNTM cooperation model (integrated flavor) here, but multiple PCE 742 path computation with inter-PCE communication can also be applied to 743 this model. Note that multiple PCE path computation without inter-PCE 744 communication does not fit in with this model. For this model to have 745 meaning, the VNTM and NMS are closely coupled. 747 The NMS sends the path computation request to the PCE. The PCE 748 returns inter-layer path computation result. When the NMS receives 749 the path computation result, the NMS works with the VNTM and sends 750 the request to LSR H2 to set up the lower-layer LSP. VNTM uses local 751 policy and possibly management/configuration input to determine how 752 to process the computation result from PCE. 754 An example procedure of the NMS-VNTM cooperation model (integrated 755 flavor) is as follows. 757 Step 1: NMS requests PCE to compute a path between H1 and H4. 758 The request indicates that inter-layer path computation is allowed. 760 Step 2: PCE computes a path. The result (H1-H2-L1-L2-H3-H4) is sent 761 back to the NMS. 763 Step 3: NMS discovers that a lower layer LSP is needed. NMS works 764 with VNTM to determine whether the new TE LSP H2-L1-L2-H3 is 765 permitted according to policy, etc. 767 Step 4: VNTM requests the ingress LSR in the lower-layer network (H2) 768 to establish a lower-layer LSP. The request message includes the 769 lower-layer LSP route obtained from PCE. 771 Step 5: H2 signals to establish the lower-layer LSP. 773 Step 6: If the lower-layer LSP setup is successful, H2 notifies VNTM 774 that the LSP is complete and supplies the tunnel information. 776 Step 7: H2 advertises the new LSP as a TE link in the higher-layer 777 network routing instance. 779 Step 8: VNTM notifies NMS that the underlying lower-layer LSP has 780 been set up, and NMS notices the new TE link advertisement. 782 Step 9: NMS requests H1 to set up a higher-layer LSP between H1 and 783 H4 with the path computed in Step 2. The lower layer links are 784 replaced by the corresponding higher layer TE link. Hence, the NMS 785 sends the path H1-H2-H3-H4 to H1. 787 Step 10: H1 initiates signaling with the path H2-H3-H4 to establish 788 the higher-layer LSP. 790 o NMS-VNTM Cooperation Model (separate flavor) 791 ----- 792 | NMS | 793 | | ----- 794 ----- | PCE | 795 ^ ^ | Hi | 796 : : ----- 797 : : ^ 798 : : : 799 : : : 800 : v v 801 : ------ ----- ----- ------ 802 : | LSR |--| LSR |........................| LSR |--| LSR | 803 : | H1 | | H2 | | H3 | | H4 | 804 : ------ -----\ /----- ------ 805 : ^ \ / 806 : : \ / 807 : -------- \ / 808 v : \ / 809 ------ ----- \----- -----/ 810 | VNTM |<-->| PCE | | LSR |--| LSR | 811 | | | Lo | | L1 | | L2 | 812 ------ ----- ----- ----- 814 Figure 9: NMS-VNTM Cooperation Model (separate flavor) 816 Figure 9 shows the NMS-VNTM cooperation model (separate flavor). The 817 NMS manages the higher layer. The case of multiple PCE computation 818 without inter-PCE communication is used to explain the NMS-VNTM 819 cooperation model here, but single PCE path computation could also be 820 applied to this model. Note that multiple PCE path computation with 821 inter-PCE communication does not fit in with this model. 823 The NMS requests a head-end LSR (H1 in this example) to set up a 824 higher-layer LSP between head-end and tail-end LSRs without 825 specifying any route. The head-end LSR, which is a PCC, requests the 826 higher-layer PCE to compute a path between head-end and tail-end 827 LSRs. There is no TE link in the higher-layer between border LSRs 828 (H2 and H3 in this example). When the PCE fails to compute a path, 829 it informs the PCC (i.e., head-end LSR) that notifies the NMS. The 830 notification may include information about the reason for failure 831 (such as that there is no TE link between the border LSRs or that 832 computation constraints cannot be met). 834 Note that it is equally valid for the higher-layer PCE to be 835 consulted by the NMS rather than by the head-end LSR. In this case, 836 the result is the same - the NMS discovers that an end-to-end LSP 837 cannot be provisioned owing to the lack of a TE link between H2 and 838 H3. 840 The NMS may now suggest (or request) to the VNTM that a lower-layer 841 LSP between the border LSRs could be established and could be 842 advertised as a TE link in the higher layer to support future 843 higher-layer LSP requests. The communication between the NMS and the 844 VNTM may be performed in an automatic manner or in a manual manner, 845 and is a key interaction between layers that may also be separate 846 administrative domains. Thus, this communication is potentially a 847 point of application of administrative, billing, and security policy. 848 The NMS may wait for the lower-layer LSP to be set up and advertised 849 as a TE link, or may reject the operator's request for the service 850 that requires the higher-layer LSP with a suggestion that the 851 operator tries again later. 853 The VNTM requests the lower-layer PCE to compute a path, and then 854 requests H2 to establish a lower-layer LSP. Alternatively, the VNTM 855 may make a direct request to H2 for the LSP, and H2 may consult the 856 lower-layer PCE. After the NMS is informed or notices that the 857 lower-layer LSP has been established, it can request the head-end 858 LSR (H1) to set up the higher-layer end-to-end LSP between H1 and H4. 860 Thus, cooperation between the high layer and lower layer is 861 performed though communication between NMS and VNTM. An example of 862 such a procedure of the NSM-VNTM cooperation model is as follows 863 using the example network in Figure 6. 865 Step 1: NMS requests a head-end LSR (H1) to set up a higher-layer 866 LSP between H1 and H4 without specifying any route. 868 Step 2: H1 (PCC) requests PCE to compute a path between H2 and H3. 870 Step 3: The path computation fails because there is no TE link 871 across the lower-layer network. 873 Step 4: H1 (PCC) notifies NMS. The notification may include an 874 indication that there is no TE link between H2 and H4. 876 Step 5: NMS suggests (or requests) to VNTM that a new TE link 877 connecting H2 and H3 would be useful. The NMS notifies VNTM that it 878 will be waiting for the TE link to be created. VNTM considers 879 whether lower-layer LSPs should be established if necessary and if 880 acceptable within VNTM's policy constraints. 882 Step 6: VNTM requests the lower-layer PCE for path computation. 884 Step 7: VNTM requests the ingress LSR in the lower-layer network 885 (H2) to establish a lower-layer LSP. The request message includes a 886 lower-layer LSP route obtained from the lower-layer PCE responsible 887 for the lower-layer network. 889 Step 8: H2 signals the lower-layer LSP. 891 Step 9: If the lower-layer LSP setup is successful, H2 notifies VNTM 892 that the LSP is complete and supplies the tunnel information. 894 Step 10: H2 advertises the new LSP as a TE link in the higher-layer 895 network routing instance. 897 Step 11: VNTM notifies NMS that the underlying lower-layer LSP has 898 been set up, and NMS notices the new TE link advertisement. 900 Step 12: NMS again requests H1 to set up a higher-layer LSP between 901 H1 and H4. 903 Step 13: H1 requests the higher-layer PCE to compute a path and 904 obtains a successful result that includes the higher-layer route 905 that is specified as H1-H2-H3-H4, where all hops are strict. 907 Step 14: H1 initiates signaling with the computed path H2-H3-H4 to 908 establish the higher-layer LSP. 910 4.2.4. Possible Combinations of Inter-Layer Path Computation and Inter- 911 Layer Path Control Models 913 Table 1 summarizes the possible combinations of inter-layer path 914 computation and inter-layer path control models. There are three 915 inter-layer path computation models: the single PCE path computation 916 model; the multiple PCE path computation with inter-PCE 917 communication model; and the multiple PCE path computation without 918 inter-PCE communication model. There are also four inter-layer path 919 control models: the PCE-VNTM cooperation model; the higher-layer 920 signaling trigger model; the NMS-VNTM cooperation model (integrated 921 flavor); the NMS-VNTM cooperation model (separate flavor). All the 922 combinations between inter-layer path computation and path control 923 models, except for the combination of the multiple PCE path 924 computation with inter-layer PCE communication model and the NMS- 925 VNTM cooperation model are possible. 927 Table 1: Possible Combinations of Inter-Layer Path Computation and 928 Inter-Layer Path Control Models. 930 ------------------------------------------------------ 931 | Path computation | Single | Multiple | Multiple | 932 | \ | PCE | PCE with | PCE w/o | 933 | Path control | | inter-PCE | inter-PCE | 934 |---------------------+--------------------------------| 935 | PCE-VNTM | Yes | Yes | Yes | 936 | cooperation | | | | 937 |---------------------+--------+-----------+-----------| 938 | Higher-layer | Yes | Yes | Yes | 939 | signaling trigger | | | | 940 |---------------------+--------+-----------+-----------| 941 | NMS-VNTM | Yes | Yes | No | 942 | cooperation | | | | 943 | (integrated flavor) | | | | 944 |---------------------+--------+-----------+-----------| 945 | NMS-VNTM | No* | No | Yes | 946 | cooperation | | | | 947 | (separate flavor) | | | | 948 ---------------------+--------+-----------+----------- 950 *Note that, in case of NSM-VNTM cooperation (separate flavor) and 951 single PCE inter-layer path computation, the PCE function used by NMS 952 and VNTM may be collocated, but it will operate on separate TEDs. 954 5. Choosing Between Inter-Layer Path Control Models 956 This section compares the cooperation model between PCE and VNTM, 957 the higher-layer signaling trigger model, and NMS-VNTM cooperation 958 model, in terms of VNTM functions, border LSR functions, higher-layer 959 signaling time, and complexity (in terms of number of states and 960 messages). An appropriate model may be chosen by a network operator 961 in different deployment scenarios taking all these considerations 962 into account. 964 5.1. VNTM Functions 966 VNTM functions are required in both the PCE-VNTM cooperation model 967 and the NMS-VNTM model. In the PCE-VNTM cooperation model, 968 communications are required between PCE and VNTM, and between VNTM 969 and a border LSR. Communications between a higher-layer PCE and the 970 VNTM are event notifications and may use SNMP notifications from the 971 PCE MIB modules [PCE-MIB]. Note that communications from the PCE to 972 the VNTM do not have any acknowledgements. VNTM-LSR communication can 973 use existing GMPLS-TE MIB modules [RFC4802]. 975 In the NMS-VNTM cooperation model, communications are required 976 between NMS and VNTM, between VNTM and a lower-layer PCE, and between 977 VNTM and a border LSR. NMS-VNTM communications, which are out of 978 scope of this document, may use proprietary or standard interfaces, 979 some of which, for example, are standardized in TM Forum. 980 Communications between VNTM and a lower-layer PCE use PCEP [RFC5440]. 981 VNTM-LSR communications are the same as in the PCE-VNTM cooperation 982 model. 984 In the higher-layer signaling trigger model, no VNTM functions are 985 required, and no such communications are required. 987 If VNTM functions are not supported in a multi-layer network, the 988 higher-layer signaling trigger model has to be chosen. 990 The inclusion of VNTM functionality allows better coordination of 991 cross-network LSP tunnels and application of network-wide policy 992 that is far harder to apply in the trigger model since it requires 993 the coordination of policy between multiple border LSRs. 995 Also, VNTM functions could be applied to establish LSPs (or 996 connections) in non-MPLS/GMPLS networks, which do not have signaling 997 capabilities, by configuring each node along the path from the VNTM. 999 5.2. Border LSR Functions 1001 In the higher-layer signaling trigger model, a border LSR must have 1002 some additional functions. It needs to trigger lower-layer signaling 1003 when a higher-layer path message suggests that lower-layer LSP setup 1004 is necessary. Note that, if virtual TE links are used, the border 1005 LSRs must be capable of triggered signaling. 1007 If the ERO in the higher-layer Path message uses a mono-layer path 1008 or specifies a loose hop, the border LSR receiving the Path message 1009 must obtain a lower-layer route either by consulting a PCE or by 1010 using its own computation engine. If the ERO in the higher-layer 1011 Path message uses a multi-layer path, the border LSR must judge 1012 whether lower-layer signaling is needed. 1014 In the PCE-VNTM cooperation model and the NMS-VNTM model, no 1015 additional function for triggered signaling is required in border 1016 LSRs except when virtual TE links are used. Therefore, if these 1017 additional functions are not supported in border LSRs, where a 1018 border LSR is controlled by VNTM to set up a lower-layer LSP, the 1019 cooperation model has to be chosen. 1021 5.3. Complete Inter-Layer LSP Setup Time 1023 The complete inter-layer LSP setup time includes inter-layer path 1024 computation, signaling, and the communication time between PCC and 1025 PCE, PCE and VNTM, NMS and VNTM, and VNTM and LSR. In the PCE-VNTM 1026 cooperation model and the NMS-VNTM model, the additional 1027 communication steps are required compared with the higher-layer 1028 signaling trigger model. On the other hand, the cooperation model 1029 provides better control at the cost of a longer service setup time. 1031 Note that, in terms of higher-layer signaling time, in the higher- 1032 layer signaling trigger model, the required time from when higher- 1033 layer signaling starts to when it is completed, is more than that of 1034 the cooperation model except when a virtual TE link is included. 1035 This is because the former model requires lower-layer signaling to 1036 take place during the higher-layer signaling. A higher-layer ingress 1037 LSR has to wait for more time until the higher-layer signaling is 1038 completed. A higher-layer ingress LSR is required to be tolerant of 1039 longer path setup times. 1041 5.4. Network Complexity 1043 If the higher and lower layer networks have multiple interconnects 1044 then optimal path computation for end-to-end LSPs that cross the 1045 layer boundaries is non-trivial. The higher layer LSP must be routed 1046 to the correct layer border nodes to achieve optimality in both 1047 layers. 1049 Where the lower layer LSPs are advertised into the higher layer 1050 network as TE links, the computation can be resolved in the higher 1051 layer network. Care needs to be taken in the allocation of TE 1052 metrics (i.e., costs) to the lower layer LSPs as they are advertised 1053 as TE links into the higher layer network, and this might be a 1054 function for a VNT Manager component. Similarly, attention should be 1055 given to the fact that the LSPs crossing the lower-layer network 1056 might share points of common failure (e.g., they might traverse the 1057 same link in the lower-layer network) and the shared risk link 1058 groups (SRLGs) for the TE links advertised in the higher-layer must 1059 be set accordingly. 1061 In the single PCE model an end-to-end path can be found in a single 1062 computation because there is full visibility into both layers and 1063 all possible paths through all layer interconnects can be considered. 1065 Where PCEs cooperate to determine a path, an iterative computation 1066 model such as [BRPC] can be used to select an optimal path across 1067 layers. 1069 When non-cooperating mono-layer PCEs, each of which is in a separate 1070 layer, are used with the triggered LSP model, it is not possible to 1071 determine the best border LSRs, and connectivity cannot even be 1072 guaranteed. In this case, signaling crankback techniques [RFC4920] 1073 can be used to eventually achieve connectivity, but optimality is 1074 far harder to achieve. In this model, a PCE that is requested by an 1075 ingress LSR to compute a path expects a border LSR to setup a lower- 1076 layer path triggered by high-layer signaling when there is no TE 1077 link between border LSRs. 1079 5.5. Separation of Layer Management 1081 Many network operators may want to provide a clear separation 1082 between the management of the different layer networks. In some 1083 cases, the lower layer network may come from a separate commercial 1084 arm of an organization or from a different corporate body entirely. 1085 In these cases, the policy applied to the establishment of LSPs in 1086 the lower-layer network and to the advertisement of these LSPs as TE 1087 links in the higher-layer network will reflect commercial agreements 1088 and security concerns (see Section 9). Since the capacity of the 1089 LSPs in the lower-layer network are likely to be significantly 1090 larger than those in the client higher-layer network (multiplex- 1091 server model), the administrator of the lower-layer network may want 1092 to exercise caution before allowing a single small demand in the 1093 higher layer to tie up valuable resources in the lower layer. 1095 The necessary policy points for this separation of administration 1096 and management are more easily achieved through the VNTM approach 1097 than by using triggered signaling. In effect, the VNTM is the 1098 coordination point for all lower layer LSPs and can be closely tied 1099 to a human operator as well as to policy and billing. Such a model 1100 can also be achieved using triggered signaling. 1102 6. Stability Considerations 1104 Inter-layer traffic engineering needs to be managed and operated 1105 correctly to avoid introducing instability problems. 1107 Lower-layer LSPs are likely, by the nature of the technologies used 1108 in layered networks, to be of considerably higher capacity than the 1109 higher-layer LSPs. This has the benefit of allowing multiple higher- 1110 layer LSPs to be carried across the lower-layer network in a single 1111 lower-layer LSP. However, when a new lower-layer LSP is set up to 1112 support a request for a higher-layer LSP because there is no 1113 suitable route in the higher-layer network, it may be the case that 1114 a very large LSP is established in support of a very small traffic 1115 demand. Further, if the higher-layer LSP is short-lived, the 1116 requirement for the lower-layer LSP will go away leaving it either 1117 in-place but unused, or requiring it to be torn down. This may cause 1118 excessive tie-up of unused lower-layer network resources, or may 1119 introduce instability into the lower-layer network. It is important 1120 that appropriate policy controls or configuration features are 1121 available so that demand-led establishment of lower-layer LSPs (the 1122 so-called "bandwidth on demand") is filtered according to the 1123 requirements of the lower-layer network. 1125 When a higher-layer LSP is requested to be set up, a new lower-layer 1126 LSP may be established if there is no route with the requested 1127 bandwidth for the higher-layer LSP. After the lower-layer LSP is 1128 established, existing high-layer LSPs could be re-routed to use the 1129 newly established lower-layer LSP if using the lower-layer LSP 1130 provides a better route than that taken by the existing LSPs. This 1131 re-routing may result in lower utilization of other lower-layer LSPs 1132 that used to carry the existing higher-layer LSPs. When the 1133 utilization of a lower-layer LSP drops below a threshold (or drops 1134 to zero), the LSP is deleted according to lower-layer network policy. 1136 But consider that some other new higher-layer LSP may be requested 1137 at once requiring the establishment or re-establishment of a lower- 1138 layer LSP. This, in turn, may cause higher-layer re-routing making 1139 other lower-layer LSPs under-utilized, in a cyclic manner. This 1140 behavior makes the higher-layer network unstable. 1142 Inter-layer traffic engineering needs to avoid network instability 1143 problems. To solve the problem, network operators may have some 1144 constraints achieved through configuration or policy, where inter- 1145 layer path control actions such as re-routing and deletion of lower- 1146 layer LSPs are not easily allowed. For example, threshold parameters 1147 for the actions are determined so that hysteresis control behavior 1148 can be performed. 1150 7. IANA Considerations 1152 This informational document makes no requests for IANA action. 1154 8. Manageability Considerations 1156 Inter-layer MPLS or GMPLS traffic engineering must be considered in 1157 the light of administrative and management boundaries that are 1158 likely to coincide with the technology layer boundaries. That is, 1159 each layer network may possibly be under separate management control 1160 with different policies applied to the networks, and specific policy 1161 rules applied at the boundaries between the layers. 1163 Management mechanisms are required to make sure that inter-layer 1164 traffic engineering can be applied without violating the policy and 1165 administrative operational procedures used by the network operators. 1167 8.1. Control of Function and Policy 1169 8.1.1. Control of Inter-Layer Computation Function 1170 PCE implementations that are capable of supporting inter-layer 1171 computations should provide a configuration switch to allow support 1172 of inter-layer path computations to be enabled or disabled. 1174 When a PCE is capable of, and configured for, inter-layer path 1175 computation, it should advertise this capability as described in 1176 [PCE-INTER-LAYER-REQ], but this advertisement may be suppressed 1177 through a secondary configuration option. 1179 8.1.2. Control of Per-Layer Policy 1181 Where each layer is operated as a separate network, the operators 1182 must have control over the policies applicable to each network, and 1183 that control should be independent of the control of policies for 1184 other networks. 1186 Where multiple layers are operated as part of the same network, the 1187 operator may have a single point of control for an integrated policy 1188 across all layers, or may have control of separate policies for each 1189 layer. 1191 8.1.3. Control of Inter-Layer Policy 1193 Probably the most important issue for inter-layer traffic 1194 engineering is inter-layer policy. This may cover issues such as 1195 under what circumstances a lower layer LSP may be established to 1196 provide connectivity in the higher layer network. Inter-layer policy 1197 may exist to protect the lower layer (high capacity) network from 1198 very dynamic changes in micro-demand in the higher layer network 1199 (see Section 6). It may also be used to ensure appropriate billing 1200 for the lower layer LSPs. 1202 Inter-layer policy should include the definition of the points of 1203 connectivity between the network layers, the inter-layer TE model to 1204 be applied (for example, the selection between the models described 1205 in this document), and the rules for path computation and LSP setup. 1206 Where inter-layer policy is defined, it must be used consistently 1207 throughout the network, and should be made available to the PCEs 1208 that perform inter-layer computation so that appropriate paths are 1209 computed. Mechanisms for providing policy information to PCEs are 1210 discussed in [RFC5394]. 1212 VNTM may provide a suitable functional component for the 1213 implementation of inter-layer policy. Use of VNTM allows the 1214 administrator of the lower layer network to apply inter-layer policy 1215 without making that policy public to the operator of the higher 1216 layer network. Similarly, a cooperative PCE model (with or without 1217 inter-PCE communication) allows separate application of policy 1218 during the selection of paths. 1220 8.2. Information and Data Models 1222 Any protocol extensions to support inter-layer computations must be 1223 accompanied by the definition of MIB objects for the control and 1224 monitoring of the protocol extensions. These MIB object definitions 1225 will conventionally be placed in a separate document from that which 1226 defines the protocol extensions. The MIB objects may be provided in 1227 the same MIB module as used for the management of the base protocol 1228 that is being extended. 1230 Note that inter-layer PCE functions should, themselves, be 1231 manageable through MIB modules. In general, this means that the MIB 1232 modules for managing PCEs should include objects that can be used to 1233 select and report on the inter-layer behavior of each PCE. It may 1234 also be appropriate to provide statistical information that reports 1235 on the inter-layer PCE interactions. 1237 Where there are communications between a PCE and VNTM, additional 1238 MIB modules may be necessary to manage and model these 1239 communications. On the other hand, if these communications are 1240 provided through MIB notifications, then those notifications must 1241 form part of a MIB module definition. 1243 Policy Information Base (PIB) modules may also be appropriate to 1244 meet the requirements as described in Section 6.1 and [RFC5394]. 1246 8.3. Liveness Detection and Monitoring 1248 Liveness detection and monitoring is required between PCEs and PCCs, 1249 and between cooperating PCEs as described in [RFC4657]. Inter-layer 1250 traffic engineering does not change this requirement. 1252 Where there are communications between a PCE and VNTM, additional 1253 liveness detection and monitoring may be required to allow the PCE 1254 to know whether the VNTM has received its information about failed 1255 path computations and desired TE links. 1257 When a lower layer LSP fails (perhaps because of the failure of a 1258 lower layer network resource) or is torn down as a result of lower 1259 layer network policy, the consequent change should be reported to 1260 the higher layer as a change in the VNT, although inter-layer policy 1261 may dictate that such a change is hidden from the higher layer. The 1262 higher layer network may additionally operate data plane failure 1263 techniques over the virtual TE links in the VNT in order to monitor 1264 the liveness of the connections, but it should be noted that if the 1265 virtual TE link is advertised but not yet established as an LSP in 1266 the lower layer, such higher layer OAM techniques will report a 1267 failure. 1269 8.4. Verifying Correct Operation 1271 The correct operation of the PCE computations and interactions are 1272 described in [RFC4657], [RFC5440], etc., and does not need further 1273 discussion here. 1275 The correct operation of inter-layer traffic engineering may be 1276 measured in several ways. First, the failure rate of higher layer 1277 path computations owing to an absence of connectivity across the 1278 lower layer may be observed as a measure of the effectiveness of the 1279 VNT and may be reported as part of the data model described in 1280 Section 6.2. Second, the rate of change of the VNT (i.e., the rate 1281 of establishment and removal of higher layer TE links based on lower 1282 layer LSPs) may be seen as a measure of the correct planning of the 1283 VNT and may also form part of the data model described in Section 1284 6.2. Third, network resource utilization in the lower layer (both in 1285 terms of resource congestion, and in consideration of under 1286 utilization of LSPs set up to support virtual TE links) can indicate 1287 whether effective inter-layer traffic engineering is being applied. 1289 Management tools in the higher layer network should provide a view 1290 of which TE links are provided using planned lower layer capacity 1291 (that is, physical connectivity or permanent connections) and which 1292 TE links are dynamic and achieved through inter-layer traffic 1293 engineering. Management tools in the lower layer should provide a 1294 view of the use to which lower layer LSPs are put including whether 1295 they have been set up to support TE links in a VNT, and if so for 1296 which client network. 1298 8.5. Requirements on Other Protocols and Functional Components 1300 There are no protocols or protocol extensions defined in this 1301 document and so it is not appropriate to consider specific 1302 interactions with other protocols. It should be noted, however, that 1303 the objective of this document is to enable inter-layer traffic 1304 engineering for MPLS-TE and GMPLS networks and so it is assumed that 1305 the necessary features for inter-layer operation of routing and 1306 signaling protocols are in existence or will be developed. 1308 This document introduces roles for various network components (PCE, 1309 LSR, NMS, and VNTM). Those components are all required to play their 1310 part in order that inter-layer TE can be effective. That is, an 1311 inter-layer TE model that assumes the presence and operation of any 1312 of these functional components obviously depends on those components 1313 to fulfill their roles as described in this document. 1315 8.6. Impact on Network Operation 1316 The use of a PCE to compute inter-layer paths is expected to have a 1317 significant and beneficial impact on network operations. Inter-layer 1318 traffic engineering of itself may provide additional flexibility to 1319 the higher layer network while allowing the lower layer network to 1320 support more and varied client networks in a more efficient way. 1321 Traffic engineering across network layers allows optimal use to be 1322 made of network resources in all layers. 1324 The use of PCE as described in this document may also have a 1325 beneficial effect on the loading of PCEs responsible for performing 1326 inter-layer path computation while facilitating a more independent 1327 operation model for the network layers. 1329 9. Security Considerations 1331 Inter-layer traffic engineering with PCE raises new security issues 1332 in all three inter-layer path control models. 1334 In the cooperation model between PCE and VNTM, when the PCE 1335 determines that a new lower-layer LSP is desirable, communications 1336 are needed between the PCE and VNTM and between VNTM and a border 1337 LSR. In this case, these communications should have security 1338 mechanisms to ensure authenticity, privacy and integrity of the 1339 information exchanged. In particular, it is important to protect 1340 against false triggers for LSP setup in the lower-layer network 1341 since such falsification could tie up lower-layer network resources 1342 (achieving a denial of service attack on the lower-layer network and 1343 on the higher layer network that is attempting to use it) and could 1344 result in incorrect billing for services provided by the lower-layer 1345 network. Where the PCE MIB modules are used to provide the 1346 notification exchanges between the higher-layer PCE and the VNTM, 1347 SNMP v3 should be used to ensure adequate security. Additionally, 1348 the VNTM should provide configurable or dynamic policy functions so 1349 that the VNTM behavior upon receiving notification from a higher- 1350 layer PCE can be controlled. 1352 The main security concern in the higher-layer signaling trigger 1353 model is related to confidentiality. The PCE may inform a higher- 1354 layer PCC about a multi-layer path that includes an ERO in the 1355 lower-layer network, but the PCC may not have TE topology visibility 1356 into the lower-layer network and might not be trusted with this 1357 information. A loose hop across the lower-layer network could be 1358 used, but this decreases the benefit of multi-layer traffic 1359 engineering. A better alternative may be to mask the lower-layer 1360 path using a path key [PATH-KEY] that can be expanded within the 1361 lower-layer network. Consideration must also be given to filtering 1362 the recorded path information from the lower-layer - see [RFC4208], 1363 for example. 1365 Additionally, in the higher-layer signaling trigger model, 1366 consideration must be given to the security of signaling at the 1367 inter-layer interface since the layers may belong to different 1368 administrative or trust domains. 1370 The NMS-VNTM cooperation model introduces communication between the 1371 NMS and the VNTM. Both of these components belong to the management 1372 plane and the communication is out of scope for this PCE document. 1373 Note that the NMS-VNTM cooperation model may be considered to 1374 address many security and policy concerns because the control and 1375 decision-making is placed within the sphere of influence of the 1376 operator in contrast to the more dynamic mechanisms of the other 1377 models. However, the security issues have simply moved, and will 1378 require authentication of operators and of policy. 1380 Security issues may also exist when a single PCE is granted full 1381 visibility of TE information that applies to multiple layers. Any 1382 access to the single PCE will immediately gain access to the 1383 topology information for all network layers - effectively, a single 1384 security breach can expose information that requires multiple 1386 breaches in other models. 1388 Note that, as described in Section 6, inter-layer TE can cause 1389 network stability issues, and this could be leveraged to attack 1390 either the higher or lower layer network. Precautionary measures, 1391 such as those described in Section 8.1.3, can be applied through 1392 policy or configuration to dampen any network oscillations. 1394 10. Acknowledgments 1396 We would like to thank Kohei Shiomoto, Ichiro Inoue, Julien Meuric, 1397 Jean-Francois Peltier, Young Lee, Ina Minei, and Jean-Philippe 1398 Vasseur, Franz Rambach for their useful comments. 1400 11. References 1402 11.1. Normative Reference 1404 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 1405 Label Switching Architecture", RFC 3031, January 2001. 1407 [RFC3945] Mannie, E., "Generalized Multi-Protocol Label Switching 1408 Architecture", RFC 3945, October 2004. 1410 [RFC4206] K. Kompella and Y. Rekhter, "Label Switched Paths (LSP) 1411 Hierarchy with Generalized Multi-Protocol Label Switching 1412 (GMPLS) Traffic Engineering (TE)", RFC 4206, October 2005. 1414 11.2. Informative Reference 1416 [RFC5212] K. Shiomoto et al., "Requirements for GMPLS-Based Multi- 1417 Region and Multi-Layer Networks (MRN/MLN)", RFC 5212, July 1418 2008. 1420 [PCE-INTER-LAYER-REQ] E. Oki et al., "PCC-PCE Communication 1421 Requirements for Inter-Layer Traffic Engineering", draft- 1422 ietf-pce-inter-layer-req work in progress. 1424 [BRPC] JP. Vasseur et al., "A Backward Recursive PCE-based 1425 Computation (BRPC) procedure to compute shortest inter- 1426 domain Traffic Engineering Label Switched Paths", draft- 1427 ietf-pce-brpc, work in progress. 1429 [RFC4920] A. Farrel et al., "Crankback Signaling Extensions for MPLS 1430 and GMPLS RSVP-TE", RFC 4920, July 2007. 1432 [PCE-MIB] E. Stephan, "Definitions of Textual Conventions for Path 1433 Computation Element", draft-ietf-pce-tc-mib.txt, work in 1434 progress. 1436 [RFC4802] A. Farrel and T. Nadeau, "Generalized Multiprotocol Label 1437 Switching (GMPLS) Traffic Engineering Management 1438 Information Base", RFC 4802, February 2007. 1440 [PATH-KEY] Bradford, R., Vasseur, JP., and Farrel, A., "Preserving 1441 Topology Confidentiality in Inter-Domain Path Computation 1442 Using a Key Based Mechanism", draft-ietf-pce-path-key, work 1443 in progress. 1445 [RFC4208] Swallow, G., Drake, J., Ishimatsu, H., and Rekhter, Y., 1446 "Generalized Multiprotocol Label Switching (GMPLS) User- 1447 Network Interface (UNI): Resource ReserVation Protocol- 1448 Traffic Engineering (RSVP-TE) Support for the Overlay 1449 Model", RFC 4208, October 2005. 1451 [RFC4655] A. Farrel, JP. Vasseur and J. Ash, "A Path Computation 1452 Element (PCE)-Based Architecture", RFC 4655, August 2006. 1454 [RFC4657] J. Ash and J.L. Le Roux (Ed.), "Path Computation Element 1455 (PCE) Communication Protocol Generic Requirements", RFC 1456 4657, September 2006. 1458 [RFC5394] Bryskin, I., Papadimitriou, P., Berger, L., and Ash J, 1459 "Policy-Enabled Path Computation Framework", RFC 5394, 1460 December 2008. 1462 [RFC5440] JP. Vasseur et al, "Path Computation Element (PCE) 1463 communication Protocol (PCEP)" RFC 5440, March 2009. 1465 12. Authors' Addresses 1467 Eiji Oki 1468 University of Electro-Communications 1469 Tokyo 1470 Japan 1471 Email: oki@ice.uec.ac.jp 1473 Tomonori Takeda 1474 NTT 1475 3-9-11 Midori-cho, 1476 Musashino-shi, Tokyo 180-8585, Japan 1477 Email: takeda.tomonori@lab.ntt.co.jp 1479 Jean-Louis Le Roux 1480 France Telecom R&D, 1481 Av Pierre Marzin, 1482 22300 Lannion, France 1483 Email: jeanlouis.leroux@orange-ftgroup.com 1485 Adrian Farrel 1486 Old Dog Consulting 1487 Email: adrian@olddog.co.uk 1489 13. Intellectual Property Statement 1491 The IETF Trust takes no position regarding the validity or scope of 1492 any Intellectual Property Rights or other rights that might be 1493 claimed to pertain to the implementation or use of the technology 1494 described in any IETF Document or the extent to which any license 1495 under such rights might or might not be available; nor does it 1496 represent that it has made any independent effort to identify any 1497 such rights. 1499 Copies of Intellectual Property disclosures made to the IETF 1500 Secretariat and any assurances of licenses to be made available, or 1501 the result of an attempt made to obtain a general license or 1502 permission for the use of such proprietary rights by implementers or 1503 users of this specification can be obtained from the IETF on-line IPR 1504 repository at http://www.ietf.org/ipr 1506 The IETF invites any interested party to bring to its attention any 1507 copyrights, patents or patent applications, or other proprietary 1508 rights that may cover technology that may be required to implement 1509 any standard or specification contained in an IETF Document. Please 1510 address the information to the IETF at ietf-ipr@ietf.org. 1512 The definitive version of an IETF Document is that published by, or 1513 under the auspices of, the IETF. Versions of IETF Documents that are 1514 published by third parties, including those that are translated into 1515 other languages, should not be considered to be definitive versions 1516 of IETF Documents. The definitive version of these Legal Provisions 1517 is that published by, or under the auspices of, the IETF. Versions of 1518 these Legal Provisions that are published by third parties, including 1519 those that are translated into other languages, should not be 1520 considered to be definitive versions of these Legal Provisions. 1522 For the avoidance of doubt, each Contributor to the IETF Standards 1523 Process licenses each Contribution that he or she makes as part of 1524 the IETF Standards Process to the IETF Trust pursuant to the 1525 provisions of RFC 5378. No language to the contrary, or terms, 1526 conditions or rights that differ from or are inconsistent with the 1527 rights and licenses granted under RFC 5378, shall have any effect and 1528 shall be null and void, whether published or posted by such 1529 Contributor, or included with or in such Contribution. 1531 14. Full Copyright Statement 1533 Copyright (c) 2009 IETF Trust and the persons identified as the 1534 document authors. All rights reserved. 1536 This document is subject to BCP 78 and the IETF Trust's Legal 1537 Provisions Relating to IETF Documents in effect on the date of 1538 publication of this document (http://trustee.ietf.org/license-info). 1539 Please review these documents carefully, as they describe your 1540 rights and restrictions with respect to this document. 1542 All IETF Documents and the information contained therein are provided 1543 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 1544 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE 1545 IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL 1546 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY 1547 WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE 1548 ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 1549 FOR A PARTICULAR PURPOSE.