idnits 2.17.1 draft-ravisingh-mpls-el-for-seamless-mpls-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 46 instances of lines with control characters in the document. ** The abstract seems to contain references ([EL-RFC]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. -- The abstract seems to indicate that this document updates RFC6790, but the header doesn't have an 'Updates:' line to match this. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 421 has weird spacing: '...covered by th...' == Line 848 has weird spacing: '...s where a lab...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Preventing insertion of multiple (ELI+EL)s: At an ingress LER, the router SHOULD not insert an (ELI+EL) for an LSP if the packet already contains an ELI. This ensures that for a data packet on a hierarchy of LSPs, there will be only 1 instance of an (ELI+EL). This helps to prevent the issue of section 4.3.1. -- The document date (October 27, 2014) is 3468 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC2119' is mentioned on line 182, but not defined == Unused Reference: 'ISSUE-DEEP' is defined on line 1000, but no explicit reference was found in the text ** Downref: Normative reference to an Informational draft: draft-ietf-mpls-seamless-mpls (ref. 'S-MPLS') -- Obsolete informational reference (is this intentional?): RFC 3107 (ref. 'LU-BGP') (Obsoleted by RFC 8277) Summary: 3 errors (**), 0 flaws (~~), 6 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 MPLS Working Group R. Singh 3 INTERNET-DRAFT Y. Shen 4 Intended Status: Proposed Standard J. Drake 5 Expires: April 30, 2015 Juniper Networks 6 October 27, 2014 8 Entropy label for seamless MPLS 9 draft-ravisingh-mpls-el-for-seamless-mpls-04 11 Abstract 13 This document describes certain optimizations to how entropy labels 14 can be used for load balancing in a seamless MPLS architecture, as 15 enabled by LSP concatenation and LSP hierarchies. 17 The definition of the control plane and data plane behavior at LSP 18 concatenation points; and at the ingress of an LSP in a hierarchy of 19 LSPs, as described in this document, brings the benefits of entropy 20 labels to certain deployment scenarios that may not have had such 21 benefits as specified in [EL-RFC]. 23 This document, if approved, updates RFC 6790. 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF), its areas, and its working groups. Note that 32 other groups may also distribute working documents as 33 Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 The list of current Internet-Drafts can be accessed at 41 http://www.ietf.org/1id-abstracts.html 43 The list of Internet-Draft Shadow Directories can be accessed at 44 http://www.ietf.org/shadow.html 46 This Internet-Draft will expire on January 3, 2015. 48 Copyright and License Notice 50 Copyright (c) 2013 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (http://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 67 3. Key attributes of the entropy label solution: Summary from 68 [EL-RFC] . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 69 4. Problems and Motivation . . . . . . . . . . . . . . . . . . . . 7 70 4.1 EL applicability for seamless MPLS . . . . . . . . . . . . . 7 71 4.2 EL for LSP concatenation . . . . . . . . . . . . . . . . . . 8 72 4.2.1 Spectrum of EL usage behaviors required to be 73 supported for concatenated LSPs . . . . . . . . . . . . 9 74 4.2.1.1 Entropy label for per-segment LSP . . . . . . . . . 9 75 4.2.1.2 Entropy label for notional-segment-LSP(s) . . . . . 10 76 4.2.1.3 Entropy label for e2e LSP . . . . . . . . . . . . . 10 77 4.3 EL for LSP hierarchy . . . . . . . . . . . . . . . . . . . . 10 78 4.3.1 Possibility of unnecessary reduction of max-payload of 79 the LSP: . . . . . . . . . . . . . . . . . . . . . . . . 11 80 4.3.2 Possibility of EL being non-usable for load-balancing: . 12 81 5. EL for LSP concatenation/hierarchy . . . . . . . . . . . . . . 13 82 5.1 Additional EL abstractions: specific to LSP 83 concatenation/hierarchy . . . . . . . . . . . . . . . . . . 13 84 5.2 New abstractions: EL applicability for LSP concatenation . . 13 85 5.2.1 Signaling . . . . . . . . . . . . . . . . . . . . . . . 13 86 5.2.1.1 Signaling ELC at concatenation points (Translation 87 rules) . . . . . . . . . . . . . . . . . . . . . . . 15 88 5.2.2 Data plane aspects . . . . . . . . . . . . . . . . . . . 16 89 5.2.2.1 LSP concatenation: Differing EL dispositions . . . . 16 90 5.3 New abstractions: EL applicability for LSP hierarchy . . . . 18 91 5.3.1 Management plane: . . . . . . . . . . . . . . . . . . . 18 92 5.3.2 Data plane aspects . . . . . . . . . . . . . . . . . . . 19 93 6. Use-cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 94 6.1 Carrier of carrier L3VPN . . . . . . . . . . . . . . . . . . 21 95 6.2 Inter-AS L3VPN: Option C . . . . . . . . . . . . . . . . . . 22 96 7. Manageability . . . . . . . . . . . . . . . . . . . . . . . . . 22 97 8. Security considerations . . . . . . . . . . . . . . . . . . . . 22 98 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 22 99 10. IANA considerations . . . . . . . . . . . . . . . . . . . . . 22 100 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 101 11.1 Normative References . . . . . . . . . . . . . . . . . . . 23 102 11.2 Informative References . . . . . . . . . . . . . . . . . . 23 103 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24 105 1 Introduction 107 [EL-RFC] specifies a way to implement load-balancing in an MPLS 108 network such that sub-flows of an LSP may be identified and sent on 109 different paths through the network. This is achieved by using 110 entropy labels (ELs) to abstract out the flow-identifying information 111 into the entropy label and inserting the entropy label underneath the 112 LSP label. The transit LSRs perform the load-balancing hash- 113 computation, on the label-stack alone, to effect a good load- 114 balancing outcome without a need to parse inner headers. 116 The key feature of [EL-RFC] is that it defines the EL in the context 117 of a given LSP. [EL-RFC] defines the signaling extensions by which 118 entropy label capability might be signaled for LSPs setup by RSVP-TE, 119 LDP or [LU-BGP]. While that works well for individual LSPs, there are 120 additional issues to consider for the seamless MPLS architecture [S- 121 MPLS]. 123 The currently-under-definition framework for seamless MPLS proposes 124 an architecture ([S-MPLS]) that shall enable the setting-up of MPLS 125 LSPs from access nodes to access nodes using a medley of signaling 126 protocols and statically configured LSPs...by essentially leveraging 127 LSP concatenation and hierarchies to carry labeled traffic in larger 128 portions of the network without essentially increasing control plane 129 state. There are special EL-related considerations that need to be 130 dealt with to make EL more suitable for seamless MPLS, on account of 131 its reliance on LSP concatenation and hierarchies. 133 This document defines additional abstractions and rules for the use 134 of entropy-label with LSP concatenation/hierarchy to enable the use 135 of ELs for the seamless MPLS architecture. This document describes 136 how entropy labels may be used when the LSP has been setup by 137 concatenation LSP segments or by tunneling the LSP over other LSPs. 138 It is conceivable that different signaling protocols are in use to 139 create an e2e LSP. 141 LSP stitching is the process of connecting LSP segments in the data 142 plane to form a single e2e data plane LSP. This is achieved by 143 setting up LSP segments through signaling or through management 144 action, and then setting-up an e2e LSP that "uses" these LSP segments 145 as hops in its path. The term "LSP stitching" has potential to be 146 ambiguous. In order to reduce the ambiguity, both meanings of the 147 usage of the term "LSP stitching" are clarified below. 148 One meaning of the term "LSP stitching" is as defined in [GMPLS- 149 STITCHING]. 151 An alternate use of "LSP stitching" as occurs for inter-AS scenarios 152 described in [INTER-AS-VPNS]. When section 10 in [INTER-AS-VPNS] 153 refers to either of the two following statements it is essentially 154 describing "LSP stitching" in the data plane: 155 - In "b)": "This procedure requires that there be a label 156 switched path leading from a packet's ingress PE to its 157 egress PE." 158 - In "c)": "Like the previous procedure, it requires that 159 there be a label switched path leading from a packet's 160 ingress PE to its egress PE." 162 Labeled data traffic flowing over e2e MPLS LSPs, that have been setup 163 by stitching together segments, would benefit from having the entropy 164 label be included in it. The specification of [EL-RFC] can be 165 optimized for usage in such environments. 167 This document specifies optimizations for "LSP stitching" which are 168 applicable to the latter meaning of the term "LSP stitching". For 169 terminology sake, this document shall refer to that latter case as 170 "LSP concatenation". 172 LSP hierarchy is defined in [MPLS-ARCH] and [GMPLS-HIER]. Usage of 173 entropy label in LSP hierarchies has some peculiar practical issues 174 that will benefit from some additional flexibility in inserting ELs 175 for a specific layer in an LSP hierarchy. 177 2. Terminology 179 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 180 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 181 document are to be interpreted as described in RFC 2119 [RFC2119]. 183 The following acronyms/terms are used: 185 e2e: End to end LSP that has been setup by concatenating together LSP 186 segments 188 ECMP: Equal Cost Multi-Path 190 EL: Entropy Label 192 ELC: Entropy Label Capability or Entropy Label Capable 194 ELI: Entropy Label Indicator 196 Intrinsic ELC: Entropy label capability/capable as in [EL-RFC]. In 197 this document, an LSP is considered to be 198 "intrinsically" EL-capable when the: 199 - ingress of the LSP has the ability to compute 200 and PUSH the EL before PUSHing the ELI before 201 PUSHing the LSP label; and 202 - egress/PHR of the LSP-segment has the ability 203 to POP the (ELI+EL) at the egress/PHR while 204 POPping-transport-label/ELI-is-top-label 205 respectively. 207 LAG: Link Aggregation Group 209 LER: Label Edge Router 211 LSP: Label Switched Path 213 LSR: Label Switching Router 215 Notional ingress: Ingress LER for an LSP segment that is inserting 216 the (ELI+EL) on data traffic going over an e2e LSP 218 Notional egress: egress LER for an LSP segment that is removing the 219 (ELI+EL) from data traffic going over an e2e LSP 221 Notional LSP segment: the portion of the e2e LSP between a notional 222 ingress and a notional egress 224 PHP: Penultimate Hop Popping 226 PHR: Penultimate Hop Router 228 UHP: Ultimate Hop Popping 230 NOTE: this document references the (ELI+EL) pair simply as EL when 231 the presence of the ELI is of no significance for the issue 232 being described. The presence of ELI is mandatory as per 233 [EL-RFC] when EL is in use. 235 3. Key attributes of the entropy label solution: Summary from [EL-RFC] 236 - Transport-label-PUSHing router inserts (ELI+EL) 237 The (ELI+EL) insertion is done, if at all, by a router that is 238 PUSHing the transport LSP's label. 240 - Ingress-LER (transport-label-PUSHing-router) inserts (ELI+EL) 241 only if the PHR/egress has signaled ability to strip it off. 243 - Transport-label-POPping router POPs (ELI+EL) PHR/egress of the 244 LSP is responsible for POPping off the (ELI+EL) after it has 245 been exposed as the top label on the packet due to POPping the 246 transport label. The removal of the (ELI+EL) is done either 247 when the ELI is the top label; or when the ELI is next label 248 below the top label being POPed. 250 - Max-payload size for the LSP gets reduced by 8 bytes after the 251 insertion of the (ELI+EL). 253 4. Problems and Motivation 255 [EL-RFC] defines EL signaling/usage suitable for single-segment 256 LSPs. 258 [EL-RFC] does not explicitly specify the EL-signaling-interaction 259 between concatenated LSP segments. Similarly, peculiarities in the 260 data-plane related to LSP concatenation need further specification. 261 Likewise, the signaling and data-plane peculiarities for using EL 262 over LSP hierarchies could be further specified. 264 It is desirable to get the benefits of EL even for concatenated LSPs. 266 Certain aspects peculiar to concatenated LSPs need additional 267 handling to increase the applicability of [EL-RFC]. [EL-RFC] needs to 268 be extended to define the behavior for LSP concatenation and LSP 269 hierarchies (tunneling) when using EL. 271 The sub-sections below list the specific additional requirements for 272 making entropy label more deployable when used with LSP 273 concatenation, and LSP hierarchy. 275 4.1 EL applicability for seamless MPLS 277 The seamless MPLS architecture relies on the use of LSP concatention 278 and hierarchy to signal an e2e LSP between access-nodes, such that 279 the e2e LSP is going over aggregation/transport/cores nodes. 281 The signaling of such e2e LSPs is enabled by using the 282 concatenation/hierarchy mechanisms that exist, using [LU- 283 BGP]/LDP/RSVP. 285 The rules of section 5 provide a general-purpose way for the use of 286 ELs across e2e LSPs by defining: 288 - the rules of ELC propagation at concatenation points; 290 - the data-plane guidelines for the concatenation point LSR; and 291 - the data-plane guidelines for LSP hierarchies for inserting 292 (ELI+EL) at ingress LER of an LSP in an LSP hierarchy. 294 4.2 EL for LSP concatenation 296 A concatenated e2e LSP might be stitched from greater than 2 segment 297 LSPs (along the length of the e2e LSP), with 2 LSPs forming the 298 stitch at each concatenation point. 300 An LSP segment is considered to be intrinsically EL capable when it 301 supports the attributes summarized in section 3. 303 Some of the segment LSPs in the e2e LSP may intrinsically support EL 304 and some may not. So, the e2e LSP may not intrinsically support EL 305 from end to end. However, to obtain the benefits of EL for 306 concatenated LSPs, it is important that an EL should be present on 307 the data packets traversing as many segments of the e2e LSP as is 308 possible within data plane abilities of the routers on the way. 310 In using EL with LSP concatenation, the aims are BOTH of the 311 following: 312 a. Get EL benefits wherever possible: on all segments where 313 possible. Just because a given segment does not support EL 314 is not a reason to deny EL benefits to other segments of the 315 e2e LSP. 317 b. Not run into data-plane problems where an 318 intermediate-segment whose ingress LER can not look deeper 319 to remove EL when the subsequent segment does not support EL. 321 - Independent setup of LSP segments: 323 LSP concatenation typically relies on LSP segments that have been 324 independently setup. In an e2e LSP (made of concatenated segments), 325 it is unlikely that all of the concatenation points (i.e., segment 326 LSP end points) as well as the ultimate ingress and ultimate egress 327 support EL as defined in section 3. 329 However, there would be individual LSP segments that would completely 330 satisfy the requirements of section 2 (i.e. are intrinsically EL 331 capable). This document describes how EL may be used for (portions 332 of) the e2e LSP while still working within the framework for [EL- 333 RFC]. 335 S---A---B---C---D 337 In the above topology, for an e2e LSP from S to D, the segments AB 338 and CD could be intrinsically EL capable while the segments SA, & BC 339 may not be. For data traffic going over the LSP from S to D, using EL 340 on the segments AB and CD would be beneficial for load-balancing over 341 LAGs/ECMP. 343 - Dealing with different protocols being used to setup the segments 344 of the e2e LSP. 346 4.2.1 Spectrum of EL usage behaviors required to be supported for 347 concatenated LSPs 349 To cater for an incremental deployment of intrinsically-ELC routers 350 in a network, the multiple different modes for EL use with LSP 351 concatenation need to be to be supported. 353 The spectrum of supported behaviors are listed below by referencing 354 the following diagram. 356 S1 S2 S3 S4 358 A-----------B-----------C-----------D-----------E 360 LSP segments S1, S2, S3, S4 are between LERs A/B/C/D/E. There may or 361 may not be other routers between the per-segment ingress<->egress 362 LERs. 364 Transport LSP signaling protocol: could be any: LDP/RSVP/([LU-BGP] 365 tunneled over RSVP/LDP). 367 4.2.1.1 Entropy label for per-segment LSP 369 Each of the segments will have their independent EL capability based 370 on BOTH the: 372 - Per-segment ingress having the ability to insert the EL. 374 - Per-segment egress (or PH router) having the ability to strip 375 the EL. 377 This is very similar to [EL-RFC] with the additional data-plane rule 378 of section 5.2.2.1 "A. Rationalizing EL for the outgoing LSP 379 segment:". 381 Reasoning for why per-segment EL may be attractive for certain use 382 scenarios: 384 Opportunity to get benefits on those segments where EL benefits are 385 available. Even though the e2e LSP may not support ELC, this allows 386 the EL benefits on those segments that are EL-capable. 388 4.2.1.2 Entropy label for notional-segment-LSP(s) 390 In the case of concatenated LSPs, it is useful to: 392 - Insert EL at first per-segment ingress LER (per-segment ingress 393 LER closest to the e2e ingress LER) that has the ability to 394 insert EL. 395 - Carry the EL on the data packets as far along the concatenated LSP 397 as the last per-segment egress LER that ability to strip the EL 398 on a series of contiguous EL-supporting segments. 400 The above is enabled by the rules of section "5.2.1.1 Signaling ELC 401 at concatenation points (Translation rules)". 403 The benefit of using EL with notional-segment LSPs: 405 An operator might be able to use EL for the MPLS traffic on its path 406 to a concatenation point even though the concatenation-point router 407 (or its PHR) itself may not have the data-plane capabilities required 408 as in [EL-RFC]. 410 Additionally, even if the concatenation-point (or its PHR) do have 411 the data-plane capabilities of [EL-RFC], it is just more efficient to 412 forward the data packets without having to strip the EL and then 413 reinsert the EL when the downstream segment is also intrinsically 414 ELC. 416 4.2.1.3 Entropy label for e2e LSP 418 This correspond to having the notional-LSP and the e2e LSP being the 419 same. 421 This is covered by the rules of section 5.2.1.1 "Signaling ELC at 422 concatenation points (Translation rules):" with the additional 423 requirement that the data-plane be exactly the same as [EL-RFC], 424 i.e. 425 the (ELI+EL) insertion is done by a label PUSHing router, 426 the (ELI+EL) POP is done by the PHR/egress for the e2e LSP. 428 4.3 EL for LSP hierarchy 430 For the purpose of highlighting the problem to be addressed and the 431 resultant requirements to be met, the following diagram is presented 432 as an example. 434 Let there be an LSP hierarchy with the ingress for the different 435 levels of LSP hierarchy being at different routers, such that each 436 LSP in the hierarchy is intrinsically EL capable. The individual LSPs 437 in the hierarchy could be a single-segment LSP or a concatenated e2e 438 LSP. 440 S1 D1 441 \ --------- / 442 A===B===C===D===E 443 / --------- \ 444 S2 D2 446 In the above topology, let there be the following LSPs: 448 L1: B->D 449 L2: A->E, tunneled through LSP L1 450 L3: S1->D1, tunneled through LSP L2 451 L4: S2->D2, tunneled through LSP L2 453 All of the LSPs above are assumed to be intrinsically EL capable. 455 4.3.1 Possibility of unnecessary reduction of max-payload of the LSP: 457 Even though the aim of using EL is to get better load-balancing 458 support, in some cases the insertion of (ELI+EL) may unnecessarily 459 reduce the effective payload of an LSP. 461 In above diagram, as per [EL-RFC] for a data packet on LSP L3, the 462 insertion of (ELI+EL) for each of the 3 LSPs: L1, L2 and L3 is not 463 explicitly prohibited. As a result it is possible that the packet on 464 LSP L3, might end up with 3 (ELI+EL)s (one for each LSP level in the 465 hierarchy) thus reducing the effective payload of the LSP L3. 466 Likewise for L4. The presence of the (ELI+EL) for the outer LSPs L1 467 and L2 is not strictly useful for load-balancing the data traffic on 468 the LSPs L3 and L4. 470 The solution for this issue is presented in section 5.3.2: it relies 471 on inserting the (ELI+EL) in the context of only 1 LSP in a 472 hierarchy. 474 This issues results in the following requirement for EL usage in the 475 presence of LSP hierarchies: 476 - Desirability of having a single (ELI+EL) on data packets over an 477 LSP hierarchy: The LSP for which the (ELI+EL) is inserted, is 478 preferably the innermost intrinsically EL-capable LSP, as the 479 notion of a user-flow is more specifically indicated by fields 480 deeper inside the packet headers. Having an EL be present deeper 481 in the packet provides load-balancing benefits of EL for the 482 traversal of the packet across a longer stretch of the network. 484 If there is to be only 1 (ELI+EL) in the label stack, it imposes an 485 additional data plane requirement on the ingress LER as described in 486 section 5.3.2. 488 4.3.2 Possibility of EL being non-usable for load-balancing: Even though 489 the aim of using EL is to get better load-balancing, in some cases 490 the insertion of (ELI+EL) may actually offer no load-balancing 491 benefits at all. Whether the presence of an EL offers load-balancing 492 benefits on a given transit router, depends on: 494 - whether the transit router has a LAG or an ECMP as an outgoing 495 interface for the LSP, AND 496 - whether the forwarding ASICs of the transit routers have the 497 ability to include the EL (appearing at a specific position in 498 the label stack) in the hash computation, either by: 499 + looking up the ELI and then picking the EL, or 500 + computing the hash on the maximum number of labels that it can 501 pick from the label-stack for hash-computation which 502 happens to also include the EL. 504 When the EL on a packet is outside the portion-of-the-label-stack 505 that the ASIC of a transit router can use for hash computation, the 506 forwarding hardware may include only the top few labels or the bottom 507 few labels in the hash computation. This may prevent the inclusion of 508 EL for hash-computation. 510 In the above diagram, for a data packet going over LSP L3 let the 511 issue of section 4.3.1 have been resolved by the router S1 inserting 512 the (ELI+EL) underneath the label for LSP L3 and none of the other 513 routers inserting the (ELI+EL). When this data packet arrives at 514 router C, its label stack looks thus: 516 Label-LSP1 | Label-LSP2 | Label-LSP3 | ELI | EL 517 Top-label -> Bottom label 519 Let's say that the router C is able to include only the top 4 labels 520 in a label stack for the hash-computation due to the ability of its 521 forwarding ASICs. 523 So, the router C is not able to get the benefit of the presence of 524 the EL in the data packet. If the only ECMP/LAG in this network is 525 the link between C&D, then the presence of the EL serves no purpose 526 for the above network example and it ends up reducing the payload 527 capacity of the LSPs L3 and L4 by 8 bytes. 529 This example could be further generalized in the case of seamless 530 MPLS, where there may be deeper LSP hierarchies. 532 A transit router that has the ability to hash on an EL (based on its 533 depth in the label stack) does not have multiple paths; while another 534 router that has multiple paths and the ability hash on the EL 535 (appearing at a specific depth in the label stack) is unable to do so 536 as the EL appears outside the depth of the label stack that may be 537 included in the hash. In neither of the foregoing cases is the 538 presence of an EL helpful. 540 This translates into a requirement for EL: Flexibility in choice of 541 LSP tunnel for which EL is inserted: 543 There is a need to have a way by which to include an EL underneath a 544 specific label in a label-hierarchy based on it serving the most 545 useful purpose (i.e. taking into consideration location of multiple- 546 forwarding-paths and stack-depth-concerns). 548 [EL-RFC] has no way of influencing the insertion of (ELI+EL) at a 549 certain LSP level in the stack. Thus, there is a need for a mechanism 550 by which one of the many intrinsically-EL-capable LSPs in an LSP 551 hierarchy could be picked for inserting the (ELI+EL). 553 5. EL for LSP concatenation/hierarchy 554 5.1 Additional EL abstractions: specific to LSP concatenation/hierarchy 556 Given the previous sections, following additional abstractions need 557 to be defined to make EL more useful for LSP concatenation and 558 hierarchy. 560 5.2 New abstractions: EL applicability for LSP concatenation 561 5.2.1 Signaling 563 New abstractions need to be defined to handle the differences in the 564 use of ELs for concatenated-LSPs as compared to their use for single- 565 segment LSPs. 567 The differences are: 569 - Notion of ingress for EL insertion: 570 (ELI+EL) insertion might not necessarily be done by a 571 label-PUSHing router. A concatenation point where the label is 572 being swapped might do the (ELI+EL) insertion, and serves as a 573 "notional ingress". 575 - Notion of egress for EL: 576 "Notional-egress" might not be the segment egress for the segment 577 of the notional-ingress. 578 Even though certain concatenation-points (segment LERs) might not 579 support POPping (ELI+EL), it may be acceptable to let the (ELI+EL) 580 continue to be on the packet since the egress of a subsequent 581 segment has the capability to POP the (ELI+EL) (which may not 582 necessarily be along with POPping the transport label). A 583 "notional-ingress and notional-egress" pair might actually be the 584 segment-ingress and segment-egress for different LSP segments that 585 are part of the same e2e LSP. 587 The portion of the concatenated e2e LSP, between a notional-ingress 588 and a notional-egress is referred to as the "notional-LSP-segment" in 589 this document. 591 As a packet traverses an e2e LSP, it may have an (ELI+EL) imposed on 592 it and then removed at different routers. 594 It is desirable for there to not be more than one instance of an 595 (ELI+EL) to appear on a packet at any given time. However, the 596 insertion followed by removal of an (ELI+EL) may happen more than 597 once as the packet traverses the e2e LSP. Each router doing the 598 (ELI+EL) insertion is the notional-ingress and each router doing the 599 (ELI+EL) removal is the notional-egress (or notion EL-stripping-PH- 600 router). 602 Thus, there may be more than 1 "notional ingress" for EL insertion, 603 and there may be more than 1 "notional egress" for EL removal. 605 For each notional "ingress ingress", there will be a "notional 606 egress" with the "notional ingress"es and "notional egress"es 607 alternating along the path of the e2e LSP when there are more than 1 608 notional ingress and egress for an e2e LSP. 610 In the simplest case, this boils down to the case of there being just 611 one notional ingress and one notional egress; and the notional 612 ingress coincides with the e2e ingress, and the notional-egress 613 coincides with the e2e egress. That is the case that [EL-RFC] 614 addresses. 616 Separation of control/data-plane implies that certain routers 618 - Might be running software that supports signaling ELC and 619 understanding an egress' ELC. 620 - However, might not have the capability to insert (ELI+EL). 622 Such routers should not be allowed to play a spoil-sport in 623 preventing EL benefits for traffic traversing over them via 624 concatenated LSPs. In other words, such routers can not act as 625 notional-ingress or notional-egress. However, the presence of such 626 per-segment ingress/egress routers on the path of a notional segment- 627 LSP should not prevent the notional segment-LSP from benefiting from 628 the use of EL. 630 5.2.1.1 Signaling ELC at concatenation points (Translation rules) 632 The rules for propagating ELC, at concatenation points, from a 633 downstream segment LSP to an upstream segment LSP are listed in this 634 section. 636 The benefit in propagating ELC across concatenation points is to not 637 have to re-compute the EL at different segment ingress for those 638 segments that are EL capable, including when the LSP segments have 639 been setup using different protocols. 641 Additionally, in certain cases it should be possible to get the 642 benefits of (ELI+EL) on LSP segments that are not "intrinsically EL 643 capable", where the lack of "intrinsic EL capability" is due to: 644 - The per-segment ingress does not support EL insertion. 645 - The per-segment PHR/egress does not support POP of the EL. 647 However, such a concatenation point might support ELC signaling. 649 At a concatenation point, when 2 LSP segments: L1 (incoming LSP) and 650 L2 (the outgoing LSP) are being concatenated, the following rules 651 should be followed by the concatenation point in signaling its ELC. 653 A. Segment-egress: 655 1. A segment-egress signals ELC for an LSP-segment L1 when: 656 a. The segment-egress is intrinsically ELC, or 657 b. When it is not intrinsically-ELC, however segment-egress for 658 LSP-segment L2 (downstream of L1)- for which this 659 concatenation-point is segment-ingress - is signaling ELC. 660 [This handles the case: Support the signaling even though it 661 may not support the data-plane behavior.] 663 B. Segment-ingress: 665 The following is relevant only for RSVP as defined in [EL-RFC]. 666 When this router acting as segment-egress for an LSP L1 (that is 667 concatenated to downstream LSP L2) is signaling ELC for L1, then 668 this router must signal ELC in its Path messages using the 669 mechanism defined in [EL-RFC]. 671 This is relevant only in the context of bidirectional LSPs. 673 5.2.2 Data plane aspects 674 5.2.2.1 LSP concatenation: Differing EL dispositions 676 At a concatenation point, when 2 LSP segments: L1 (incoming LSP) and 677 L2 (the outgoing LSP) are being concatenated, the following rules 678 should be followed by the concatenation point in its data-plane 679 behavior. 681 A. Rationalizing EL for the outgoing LSP segment: 682 When the LSP segments L1 and L2 differ in their ELC, the 683 concatenation point router needs to take the following 684 data-plane actions depending on its role for the e2e LSP: 686 a. Notional egress behavior: 687 When L1 intrinsically supports ELC and L2 does not, then 688 the concatenation-point router must remove the (ELI+EL), if 689 present under top label, from the received data packets 690 before effectively SWAPing the top label. In other words, 691 in the presence of the ELI, the operations performed 692 should be: 694 POP(IncomingLabel), POP(ELI+EL), PUSH(OutgoingLabel) 695 or alternately: 696 POP, POP, SWAP(OutgoingLabel) 698 Translation rule "A 2" of section 5.2.1.1 would have ensured that 699 the above is doable at the concatenation point. 701 b. Notional ingress behavior: 702 When L1 does not intrinsically support ELC and L2 does, then the 703 concatenation point router must POP the incoming label, insert 704 (ELI+EL) before PUSHing the label for the LSP segment L2. 706 The label operations performed would be: 707 POP(IncomingLabel), PUSH(EL), PUSH(ELI), PUSH(OutgoingLabel), 708 or 709 SWAP(EL), PUSH(ELI), PUSH(OutgoingLabel) 711 c. Implicit notional ingress behavior: 712 When L1 is intrinsically ELC and so is L2, the arriving data 713 traffic should already have (ELI+EL) on it. 715 However, it is possible that due to local configuration on the 716 notional-ingress, (ELI+EL) is not being inserted. In that 717 case, traffic arriving on L1 will not have (ELI+EL) on it. 719 In that case, this concatenation-point is the "implicit notional 720 ingress" and it should insert (ELI+EL) just as if it were a 721 "notional ingress". 723 B. Preventing multiple (ELI+EL) pairs underneath a given forwarding 724 label in the stack: 726 A segment-ingress that is intrinsically-EL-capable should have 727 the ability to inspect received data packets to check whether an 728 (ELI+EL) already exists on the data packet underneath the top 729 label. 731 Not causing multiple ELs on a data packet: 732 When both the LSP segments L1 and L2 support ELC, the 733 concatenation point router SHOULD insert an (ELI+EL) only if 734 the incoming packet does not contain an (ELI+EL) underneath 735 the top label. In that case, the label operations are as 736 below: 737 POP(IncomingLabel), PUSH(ELI+EL), PUSH(OutgoingLabel) 739 If the incoming packet already contains an (ELI+EL) underneath the 740 top label, an additional (ELI+EL) MUST NOT be inserted on the 741 packet underneath the top label that is being effectively SWAPed. 743 This prevents a segment ingress from inserting an (ELI+EL) when 744 the notional ingress has already inserted an (ELI+EL). 746 C. Rationalizing EL insertion (at concatenation-point) for LSP 747 hierarchy: 749 A concatenation point router that is intrinsically-EL-capable 750 should have the ability to inspect received data packets to check 751 whether an (ELI+EL) already exists, underneath any label in the 752 label-stack. 754 If the router has such a ability, then this router MUST NOT insert 755 an (ELI+EL) as in "A a" above. 757 This helps to prevent multiple (ELI+EL)s on the packet inserted 758 (at a concatenation point) in the context of different transport 759 labels in a label hierarchy. 761 D. Notional ingress role change at a router: 763 This role can change due to local configuration on the router or 764 due to segment egress starting/stopping to signal ELC possibly due 765 to a configuration change at the segment egress or due to a 766 configuration change at this router. 767 When this router becomes a notional ingress, it reacts to the 768 change as in "A b" above. 770 When this router stops being a notional ingress, this router 771 stops inserting the (ELI+EL) underneath the top label that this 772 router is 773 SWAPing(if this router is concatenation point), or 774 PUSHING (if this router is e2e ingress). 776 E. Notional egress role change at a router: 778 This role can change due to local configuration on the router or 779 due the egress of a downstream concatenated LSP segment starting 780 to signal ELC. 782 When this router becomes a notional egress, it reacts to the 783 change as in "A a" above. 785 When this router stops being a notional egress, this router stops 786 performing the label operation described in "A a" above. Instead 787 this router now starts to simply SWAP the top label. 789 5.3 New abstractions: EL applicability for LSP hierarchy 790 5.3.1 Management plane: 792 Moving the (ELI+EL) underneath a different LSP's transport label: 793 There are 2 ways to handle the issue of section 4.3.2: 794 - Configuration at the ingress LER: a configuration option should 795 exist by which an operator can disable the insertion of (ELI+EL) 796 on a per-LSP basis. The specific level in the LSP hierarchy for 797 which to enable this configuration is based on operator knowledge 798 based on: 799 * Knowledge of transit routers that need EL benefits : those 800 routers that have a multi-path (LAG or LSP ECMP) as egress 801 interface. 802 * The label hashing abilities of such routers: information 803 about the specific number of labels in the label-stack that 804 can be used in the hash computation; and any constraints 805 about the position of the labels that can be used for 806 computation when the label stack is larger than a certain 807 ASIC-specific threshold. 809 - Configuration-based rewrite of the label stack at the ingress LER 810 of an intrinsically-EL-capable LSP: 812 An operator will know the forwarding characteristics (with 813 regards to the number of labels that can be included in the hash 814 computation) of the transit routers across the path of the e2e 815 LSP that is part of an LSP hierarchy. 817 By making such a configuration, the operator can ensure that the 818 EL will appear in the label stack such that all transit routers 819 shall be able to include the as part of the hash computation. 821 The configuration would cause the label stack of the outgoing 822 packet to have its extant (ELI+EL) removed, and an (ELI+EL) 823 inserted just underneath the label of the LSP for which this 824 ingress LER is setup to insert (ELI+EL). 826 5.3.2 Data plane aspects 828 Preventing insertion of multiple (ELI+EL)s: 829 At an ingress LER, the router SHOULD not insert an (ELI+EL) for an 830 LSP if the packet already contains an ELI. 831 This ensures that for a data packet on a hierarchy of LSPs, there 832 will be only 1 instance of an (ELI+EL). This helps to prevent the 833 issue of section 4.3.1. 835 This also ensures that when multiple LSPs in an LSP hierarchy are 836 intrinsically-EL-capable, the (ELI+EL) will be inserted just 837 underneath the transport label of the innermost LSP in the hierarchy. 838 However, based on section 5.3.1 there is a way by which to modify the 839 level in the LSP hierarchy for which an (ELI+EL) is inserted. 841 A more specific case of this is already covered in section "5.2.2.1 842 C. Rationalizing EL insertion (at concatenation-point) for LSP 843 hierarchy:". 845 6. Use-cases 847 In this document, the definition of LSP-concatenation refers to those 848 cases where a label advertised by one label distribution protocol 849 being: 851 - removed at one router (by PH POP) followed by a label PUSH for a 852 label distributed by another protocol at the router downstream 853 of the PH router of the previous protocol's LSP. Such a router 854 which is PUSHing a label for a subsequent protocol is 855 referred to as a concatenation-point router in this document. 857 - SWAPed at a router for a label distributed by another protocol 858 is also referred to as a concatenation-point in this document. 860 The list of use-cases of this draft stems from the following possible 861 optimizations: 863 A. Not having to insert/remove (ELI+EL) multiple times along an e2e 864 labeled path, due to the EL capability not getting signaled e2e. 865 In other words, not having to remove (ELI+EL) at a concatenation 866 point only to re-insert it. 867 The lack of e2e EL capability signaling could be either due 868 to lack of support; or due to a label advertised by one 869 label distribution protocol being removed at one router 870 (which also causes the removal of (ELI+EL)) and a label 871 distributed by a subsequent router being PUSHed along 872 with (ELI+EL) at that router. 874 B. On an e2e labeled transport-LSP path, it may be possible 875 to get the load-balancing benefits of EL on (segment of) 876 the e2e LSP even though not every concatenation point router 877 (as defined above) may intrinsically support EL for the 878 LSP terminating at it. 880 6.1 Carrier of carrier L3VPN 881 ----- 882 ======= / A1 883 ---- | | ____CE1 | 884 / \ -------- / \ ------ / | | 885 | A2 CE2- / AS2 \ | | / PE1 \ / 886 \ / \ / \_ / \_ _/ Carrier| ---- 887 ---- -PE2 | | Carrier's | | cust. | 888 | | | carrier | | | 889 | ASBR1|--ASBR2 ASBR3--|ASBR4 | ---- 890 | Carrier | | | | | / \ 891 | cust. / | AS1 | | AS2 | CE5 A5 | 892 \ / \ | | PE4__/\ / 893 ---PE3-------/ | | \ / ----- 894 / | \ / -------- 895 ---- / | ---- | | 896 / \ | / \ ============ 897 | A3 CE3 -CE4 A4| 898 \ / \ / 899 ---- ---- 900 A = Customer site n 901 CE = Customer Edge Device 902 PE = Provider Edge Router 904 In the above figure, the "carrier's carrier" is providing L3VPN 905 service to a carrier customer (carrier cust.) is itself an L3VPN 906 provider. 907 Let the sites A be the sites of the same L3VPN. 908 In order to provide L3VPN service to the sites A, there is 909 effectively an e2e LSP between each pair of PEs. For PEs in the same 910 carrier customer site, the e2e LSP is an RSVP or LDP LSP. eg. Between 911 PE2 and PE3. 912 For PEs that are across the carrier-customer's core, there is an e2e 913 LSP created by advertising a BGP label for the remote PE's loopback 914 address. The BGP label advertised from ASBR2 to ASBR1 rides-on top of 915 the RSVP or LDP label in the carrier's-carrier core. 917 eg. For having an e2e LSP from PE1 to PE2, a BGP label is advertised 918 for PE1's loopback into the carrier customer's site on the left. This 919 label could be dealt with by ASBR1 in two ways: 920 a. Advertising it into LDP in the carrier customer's site (on 921 the left), or 922 b. By advertising it over an iBGP session to PE2. 924 In the former case (LDP advertising a FEC for PE1), this document 925 makes possible for ASBR1 to not have to remove the EL (inserted by 926 PE2) and let it be removed by a either a concatenation point (ASBR2 927 or ASBR3 or ASBR4) or the egress PE1. This is facilitated by the 928 translation rules of section 5.2.1.1. The same also facilitates 929 traffic with EL to be carried over concatenation points such that the 930 EL is eventually removed by the last-EL-capable concatenation point 931 or the EL capable e2e egress. 933 Each carrier (carrier's carrier; and carrier-customer) will have LAGs 934 and LDP ECMP paths in its network. 936 6.2 Inter-AS L3VPN: Option C 938 Option C is conceptually similar to CoC L3VPN from a point of view of 939 setting up the e2e LSP. Therefore, similar EL use-cases also exist 940 for Option C. 942 This applies for both L3VPN and also BGP-VPLS. 944 7. Manageability 946 There are no new MPLS OAM issues opened up by this specification. Any 947 MPLS manageability are the same as those inherited from [EL-RFC] and 948 addressing those is outside the scope of this document. 950 8. Security considerations 952 Security considerations as listed in section 9 of [EL-RFC] apply. 954 9. Acknowledgments 956 Many thanks to Adrian Farrel for his inputs on the stitching 957 scenarios, and suggesting editorial improvements. 959 Thanks to the EL team (Sudharsana Venkataraman, Nitin Singh, Ramji 960 Vijayaraghavan, Jie Yan, Abhishek Tripathi) for discussions on some 961 of these topics. 963 May thanks to John Scudder for discussions on this topic and his 964 editorial comments/corrections. 966 10. IANA considerations 968 None. 970 11. References 972 11.1 Normative References 974 [EL-RFC] Kompella, K., Drake, J., Amante, S., Henderickx, W., 975 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 976 RFC-6790, November 2012. 978 [GMPLS-HIER] Kompella, K., Y. Rekhter, "Label Switched Paths (LSP) 979 Hierarchy with Generalized Multi-Protocol Label 980 Switching (GMPLS)", RFC-4206, October 2005. 982 [MPLS-ARCH] Rosen, E., Viswanathan, A., R. Callon, "Multiprotocol 983 Label Switching Architecture", RFC-3031, January 2001. 985 [S-MPLS] Leymann, N., Decraene, B., Filsfils, C., 986 Konstantynowicz, M., Steinberg, D., "Seamless MPLS 987 Architecture", draft-ietf-mpls-seamless-mpls-07, 988 October 2012. (work-in-progress) 990 [INTER-AS-VPNS] Rosen, E., Y. Rekhter, "BGP/MPLS IP Virtual 991 Private Networks (VPNs)", RFC4364, February 2006. 993 [GMPLS-STITCHING] Ayyangar, A., Kompella, K., Vasseur, JP., 994 A. Farrel, "Label Switched Path Stitching with 995 Generalized Multiprotocol Label Switching Traffic 996 Engineering (GMPLS TE)", RFC 5150, February 2008. 998 11.2 Informative References 1000 [ISSUE-DEEP] K. Kompella, "Deep Label Stacks", 1001 http://tools.ietf.org/agenda/84/slides/slides-84-mpls- 1002 15.pdf, August 2012 1004 [LU-BGP] Rekhter, Y., E. Rosen, "Carrying Label Information in 1005 BGP-4", RFC-3107, May 2001. 1007 Authors' Addresses 1009 Ravi Singh 1010 Juniper Networks 1011 1194 N. Mathilda Ave. 1012 Sunnyvale, CA 94089 1013 US 1015 EMail: ravis@juniper.net 1017 Yimin Shen 1018 Juniper Networks 1019 10 Technology Park Drive 1020 Westford, MA 01886 1021 US 1023 EMail: yshen@juniper.net 1025 John Drake 1026 Juniper Networks 1027 1194 N. Mathilda Ave. 1028 Sunnyvale, CA 94089 1029 US 1031 EMail: jdrake@juniper.net