idnits 2.17.1 draft-ietf-mpls-spring-entropy-label-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The exact meaning of the all-uppercase expression 'MAY NOT' is not defined in RFC 2119. If it is intended as a requirements expression, it should be rewritten using one of the combinations defined in RFC 2119; otherwise it should not be all-uppercase. == The expression 'MAY NOT', while looking like RFC 2119 requirements text, is not defined in RFC 2119, and should not be used. Consider using 'MUST NOT' instead (if that is what you mean). Found 'MAY NOT' in this paragraph: An implementation MAY NOT place an ELI/EL after a regular Adj-SID in order to favor the insertion of ELI/ELs following other segments. == The expression 'MAY NOT', while looking like RFC 2119 requirements text, is not defined in RFC 2119, and should not be used. Consider using 'MUST NOT' instead (if that is what you mean). Found 'MAY NOT' in this paragraph: When the L2 bundle SR extensions [I-D.ietf-isis-l2bundles] are used, adjacency segments may be advertised for each member of the bundle. In this case, an implementation can deduce that load-balancing is not expected on the LSR advertising this segment and MAY NOT insert an ELI/EL after the corresponding label. -- The document date (July 16, 2018) is 2111 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1000' on line 451 -- Looks like a reference, but probably isn't: '1999' on line 451 == Unused Reference: 'I-D.ietf-ospf-segment-routing-msd' is defined on line 1132, but no explicit reference was found in the text == Outdated reference: A later version (-22) exists of draft-ietf-spring-segment-routing-mpls-14 == Outdated reference: A later version (-13) exists of draft-ietf-isis-mpls-elc-03 == Outdated reference: A later version (-15) exists of draft-ietf-ospf-mpls-elc-05 == Outdated reference: A later version (-19) exists of draft-ietf-isis-segment-routing-msd-12 == Outdated reference: A later version (-25) exists of draft-ietf-ospf-segment-routing-msd-14 == Outdated reference: A later version (-18) exists of draft-ietf-idr-bgp-ls-segment-routing-msd-01 Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Kini 3 Internet-Draft 4 Intended status: Standards Track K. Kompella 5 Expires: January 17, 2019 Juniper 6 S. Sivabalan 7 Cisco 8 S. Litkowski 9 Orange 10 R. Shakir 11 Google 12 J. Tantsura 13 July 16, 2018 15 Entropy label for SPRING tunnels 16 draft-ietf-mpls-spring-entropy-label-12 18 Abstract 20 Segment Routing (SR) leverages the source routing paradigm. A node 21 steers a packet through an ordered list of instructions, called 22 segments. Segment Routing can be applied to the Multi Protocol Label 23 Switching (MPLS) data plane. Entropy label (EL) is a technique used 24 in MPLS to improve load-balancing. This document examines and 25 describes how ELs are to be applied to Segment Routing MPLS. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on January 17, 2019. 44 Copyright Notice 46 Copyright (c) 2018 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (https://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 62 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 63 2. Abbreviations and Terminology . . . . . . . . . . . . . . . . 4 64 3. Use-case requiring multipath load-balancing . . . . . . . . . 5 65 4. Entropy Readable Label Depth . . . . . . . . . . . . . . . . 6 66 5. Maximum SID Depth . . . . . . . . . . . . . . . . . . . . . . 8 67 6. LSP stitching using the Binding-SID . . . . . . . . . . . . . 10 68 7. Insertion of entropy labels for SPRING path . . . . . . . . . 11 69 7.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 11 70 7.1.1. Example 1 where the ingress node has a sufficient MSD 12 71 7.1.2. Example 2 where the ingress node does not have a 72 sufficient MSD . . . . . . . . . . . . . . . . . . . 13 73 7.2. Considerations for the placement of entropy labels . . . 14 74 7.2.1. ERLD value . . . . . . . . . . . . . . . . . . . . . 15 75 7.2.2. Segment type . . . . . . . . . . . . . . . . . . . . 16 76 7.2.2.1. Node-SID . . . . . . . . . . . . . . . . . . . . 16 77 7.2.2.2. Adjacency-set SID . . . . . . . . . . . . . . . . 17 78 7.2.2.3. Adjacency-SID representing a single IP link . . . 17 79 7.2.2.4. Adjacency-SID representing a single link within 80 an L2 bundle . . . . . . . . . . . . . . . . . . 17 81 7.2.2.5. Adjacency-SID representing an L2 bundle . . . . . 17 82 7.2.3. Maximizing number of LSRs that will load-balance . . 18 83 7.2.4. Preference for a part of the path . . . . . . . . . . 18 84 7.2.5. Combining criteria . . . . . . . . . . . . . . . . . 18 85 8. A simple example algorithm . . . . . . . . . . . . . . . . . 18 86 9. Deployment Considerations . . . . . . . . . . . . . . . . . . 19 87 10. Options considered . . . . . . . . . . . . . . . . . . . . . 20 88 10.1. Single EL at the bottom of the stack . . . . . . . . . . 20 89 10.2. An EL per segment in the stack . . . . . . . . . . . . . 20 90 10.3. A re-usable EL for a stack of tunnels . . . . . . . . . 21 91 10.4. EL at top of stack . . . . . . . . . . . . . . . . . . . 21 92 10.5. ELs at readable label stack depths . . . . . . . . . . . 22 93 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 94 12. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 95 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 96 14. Security Considerations . . . . . . . . . . . . . . . . . . . 23 97 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 98 15.1. Normative References . . . . . . . . . . . . . . . . . . 23 99 15.2. Informative References . . . . . . . . . . . . . . . . . 24 100 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 102 1. Introduction 104 Segment Routing [I-D.ietf-spring-segment-routing] is based on source 105 routed tunnels to steer a packet along a particular path. This path 106 is encoded as an ordered list of segments. When applied to the MPLS 107 dataplane [I-D.ietf-spring-segment-routing-mpls], each segment is an 108 LSP (Label Switched Path) with an associated MPLS label value. 109 Hence, label stacking is used to represent the ordered list of 110 segments and the label stack associated with an SR tunnel can be seen 111 as nested LSPs (LSP hierarchy) in the MPLS architecture. 113 Using label stacking to encode the list of segments has implications 114 on the label stack depth. 116 Traffic load-balancing over ECMP (Equal Cost Multi Path) or LAGs 117 (Link Aggregation Groups) is usually based on a hashing function. 118 The local node which performs the load-balancing is required to read 119 some header fields in the incoming packets and then computes a hash 120 based on those fields. The result of the hash is finally mapped to a 121 list of outgoing nexthops. The hashing technique is required to 122 perform a per-flow load-balancing and thus prevents packet 123 misordering. For IP traffic, the usual fields that are hashed are 124 the source address, the destination address, the protocol type, and, 125 if provided by the upper layer, the source port and destination port. 127 The MPLS architecture brings some challenges when an LSR tries to 128 look up at header fields. An LSR (Label Switching Router) needs be 129 able to look up at header fields that are beyond the MPLS label stack 130 while the MPLS header does not provide any information about the 131 upper layer protocol. An LSR must perform a deeper inspection 132 compared to an ingress router which could be challenging for some 133 hardware. Entropy label (EL) [RFC6790] is a technique used in the 134 MPLS data plane to provide entropy for load-balancing. The idea 135 behind the entropy label is that the ingress router computes a hash 136 based on several fields from a given packet and places the result in 137 an additional label, named "entropy label". Then, this entropy label 138 can be used as part of the hash keys used by an LSR. Using the 139 entropy label as part of the hash keys reduces the need for deep 140 packet inspection in the LSR while keeping a good level of entropy in 141 the load-balancing. When the entropy label is used, the keys used in 142 the hashing functions are still a local configuration matter and an 143 LSR may use solely the entropy label or a combination of multiple 144 fields from the incoming packet. 146 When using LSP hierarchies, there are implications on how [RFC6790] 147 should be applied. The current document addresses the case where a 148 hierarchy is created at a single LSR as required by Segment Routing. 150 A use-case requiring load-balancing with SR is given in Section 3. A 151 recommended solution is described in Section 7 keeping in 152 consideration the limitations of implementations when applying 153 [RFC6790] to deeper label stacks. Options that were considered to 154 arrive at the recommended solution are documented for historical 155 purposes in Section 10. 157 1.1. Requirements Language 159 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 160 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 161 "OPTIONAL" in this document are to be interpreted as described in BCP 162 14 [RFC2119] [RFC8174] when, and only when, they appear in all 163 capitals, as shown here. 165 2. Abbreviations and Terminology 167 Adj-SID - Adjacency Segment Identifier 169 ECMP - Equal Cost Multi Path 171 EL - Entropy Label 173 ELI - Entropy Label Indicator 175 ELC - Entropy Label Capability 177 ERLD - Entropy Readable Label Depth 179 FEC - Forwarding Equivalent Class 181 LAG - Link Aggregation Group 183 LSP - Label Switched Path 185 LSR - Label Switching Router 187 MPLS - Multiprotocol Label Switching 189 MSD - Maximum SID Depth 191 Node-SID - Node Segment Identifier 193 OAM - Operation, Administration and Maintenance 194 RLD - Readable Label Depth 196 SID - Segment Identifier 198 SPT - Shortest Path Tree 200 SR - Segment Routing 202 SRGB - Segment Routing Global Block 204 VPN - Virtual Private Network 206 3. Use-case requiring multipath load-balancing 208 +------+ 209 | | 210 +-------| P3 |-----+ 211 | +-----| |---+ | 212 L3| |L4 +------+ L1| |L2 +----+ 213 | | | | +--| P4 |--+ 214 +-----+ +-----+ +-----+ | +----+ | +-----+ 215 | S |-----| P1 |------------| P2 |--+ +--| D | 216 | | | | | |--+ +--| | 217 +-----+ +-----+ +-----+ | +----+ | +-----+ 218 +--| P5 |--+ 219 +----+ 220 S=Source LSR, D=Destination LSR, P1,P2,P3,P4,P5=Transit LSRs, 221 L1,L2,L3,L4=Links 223 Figure 1: Traffic engineering use-case 225 Traffic-engineering is one of the applications of MPLS and is also a 226 requirement for Segment Routing [RFC7855]. Consider the topology 227 shown in Figure 1. The LSR S requires data to be sent to LSR D along 228 a traffic-engineered path that goes over the link L1. Good load- 229 balancing is also required across equal cost paths (including 230 parallel links). To steer traffic along a path that crosses link L1, 231 the label stack that LSR S creates consists of a label to the Node- 232 SID of LSR P3, stacked over the label for the Adj-SID (Adjacency 233 Segment Identifier) of link L1 and that in turn is stacked over the 234 label to the Node-SID of LSR D. For simplicity lets assume that all 235 LSRs use the same label space for Segment Routing (as a reminder, it 236 is called the SRGB, Segment Routing Global Block). Let L_N-Px denote 237 the label to be used to reach the Node-SID of LSR Px. Let L_A-Ln 238 denote the label used for the Adj-SID for link Ln. In our example, 239 the LSR S must use the label stack . However, 240 to achieve a good load-balancing over the equal cost paths P2-P4-D, 241 P2-P5-D and the parallel links L3 and L4, a mechanism such as entropy 242 labels [RFC6790] should be adapted for Segment Routing. Indeed, the 243 SPRING architecture with the MPLS dataplane 244 ([I-D.ietf-spring-segment-routing-mpls]) uses nested MPLS LSPs 245 composing the source routed label stack. 247 An MPLS node may have limitations in the number of labels it can 248 push. It may also have a limitation in the number of labels it can 249 inspect when looking for hash keys during load-balancing. While the 250 entropy label is normally inserted at the bottom of the transport 251 tunnel, this may prevent an LSR from taking into account the EL in 252 its load-balancing function if the EL is too deep in the stack. In a 253 Segment Routing environment, it is important to define the 254 considerations that needs to be taken into account when inserting an 255 EL. Multiple ways to apply entropy labels were considered and are 256 documented in Section 10 along with their trade-offs. A recommended 257 solution is described in Section 7. 259 4. Entropy Readable Label Depth 261 The Entropy Readable Label Depth (ERLD) is defined as the number of 262 labels a router can both: 264 a. Read in an MPLS packet received on its incoming interface(s) 265 (starting from the top of the stack). 267 b. Use in its load-balancing function. 269 The ERLD means that the router will perform load-balancing using the 270 EL label if the EL is placed within the first ERLD labels. 272 A router capable of reading N labels but not using an EL located 273 within those N labels MUST consider its ERLD to be 0. 275 In a distributed switching architecture, each linecard may have a 276 different capability in terms of ERLD. For simplicity, an 277 implementation MAY use the minimum ERLD of all linecards as the ERLD 278 value for the system. 280 There may also be a case where a router has a fast switching path 281 (handled by an ASIC or network processor) and a slow switching path 282 (handled by a CPU) with a different ERLD for each switching path. 283 Again, for simplicity's sake, an implementation MAY use the minimum 284 ERLD as the ERLD value for the system. 286 The drawback of using a single ERLD for a system lower than the 287 capability of one or more specific component is that it may increase 288 the number of ELI/ELs inserted. This leads to an increase of the 289 label stack size and may have an impact on the capability of the 290 ingress node to push this label stack. 292 Examples: 294 | Payload | 295 +----------+ 296 | Payload | | EL | P7 297 +----------+ +----------+ 298 | Payload | | EL | | ELI | 299 +----------+ +----------+ +----------+ 300 | Payload | | EL | | ELI | | Label 50 | 301 +----------+ +----------+ +----------+ +----------+ 302 | Payload | | EL | | ELI | | Label 40 | | Label 40 | 303 +----------+ +----------+ +----------+ +----------+ +----------+ 304 | EL | | ELI | | Label 30 | | Label 30 | | Label 30 | 305 +----------+ +----------+ +----------+ +----------+ +----------+ 306 | ELI | | Label 20 | | Label 20 | | Label 20 | | Label 20 | 307 +----------+ +----------+ +----------+ +----------+ +----------+ 308 | Label 16 | | Label 16 | | Label 16 | | Label 16 | | Label 16 | P1 309 +----------+ +----------+ +----------+ +----------+ +----------+ 310 Packet 1 Packet 2 Packet 3 Packet 4 Packet 5 312 Figure 2: Label stacks with ELI/EL 314 In Figure 2, we consider the displayed packets received on a router 315 interface. We consider also a single ERLD value for the router. 317 o If the router has an ERLD of 3, it will be able to load-balance 318 Packet 1 displayed in Figure 2 using the EL as part of the load- 319 balancing keys. The ERLD value of 3 means that the router can 320 read and take into account the entropy label for load-balancing if 321 it is placed between position 1 (top of the MPLS label stack) and 322 position 3. 324 o If the router has an ERLD of 5, it will be able to load-balance 325 Packets 1 to 3 in Figure 2 using the EL as part of the load- 326 balancing keys. Packets 4 and 5 have the EL placed at a position 327 greater than 5, so the router is not able to read it and use as 328 part of the load-balancing keys. 330 o If the router has an ERLD of 10, it will be able to load-balance 331 all the packets displayed in Figure 2 using the EL as part of the 332 load-balancing keys. 334 To allow an efficient load-balancing based on entropy labels, a 335 router running SPRING SHOULD advertise its ERLD (or ERLDs), so all 336 the other SPRING routers in the network are aware of its capability. 338 How this advertisement is done is outside the scope of this document 339 (see Section 7.2.1 for potential approaches). 341 To advertise an ERLD value, a SPRING router: 343 o MUST be entropy label capable and, as a consequence, MUST apply 344 the dataplane procedures defined in [RFC6790]. 346 o MUST be able to read an ELI/EL which is located within its ERLD 347 value. 349 o MUST take into account an EL within the first ERLD labels in its 350 load-balancing function. 352 5. Maximum SID Depth 354 The Maximum SID Depth defines the maximum number of labels that a 355 particular node can impose on a packet. This can include any kind of 356 labels (service, entropy, transport...). In an MPLS network, the MSD 357 is a limit of the head-end of an SR tunnel or a Binding-SID anchor 358 node that performs imposition of additional labels on an existing 359 label stack. 361 Depending on the number of MPLS operations (POP, SWAP...) to be 362 performed before the PUSH, the MSD can vary due to hardware or 363 software limitations. As for the ERLD, different MSD limits can 364 exist within a single node based on the linecard types used in a 365 distributed switching system. Thus, the MSD is a per link and/or per 366 node property. 368 An external controller can be used to program a label stack on a 369 particular node. This node SHOULD advertise its MSD to the 370 controller in order to let the controller know the maximum label 371 stack depth of the path computed that is supported on the head-end. 372 How this advertisement is done is outside the scope of this document 373 ([I-D.ietf-isis-segment-routing-msd], 374 [I-D.ietf-isis-segment-routing-msd] and 375 [I-D.ietf-idr-bgp-ls-segment-routing-msd] provide examples of 376 advertisement of MSD). As the controller does not have the knowledge 377 of the entire label stack to be pushed by the node, in addition to 378 the MSD value, the node SHOULD advertise the type of the MSD. For 379 instance, the MSD value can represent the limit for pushing transport 380 labels only while in reality the node can push an additional service 381 label. As another example, the MSD value can represent the full 382 limit of the node including all label types (transport, service, 383 entropy...). This gives the ability for the controller to program a 384 label stack while leaving room for the local node to add more labels 385 (e.g., service, entropy,...) without reaching the hardware/software 386 limit. If the node does not provide the meaning of the MSD value, 387 the controller could program an LSP using a number of labels equal to 388 the full limit of the node. When receiving this label stack from the 389 controller, the ingress node may not be able to add any service 390 (L2VPN, L3VPN, EVPN...) label on top of this label stack. The 391 consequence could be for the ingress node to drop service packets 392 that should have been forwarded over the LSP. 394 P7 ---- P8 ---- P9 395 / \ 396 PE1 --- P1 --- P2 --- P3 --- P4 --- P5 --- P6 --- PE2 397 | \ | 398 ----> P10 \ | 399 IP Pkt | \ | 400 P11 --- P12 --- P13 401 100 10000 403 Figure 3 405 In Figure 3, an IP packet comes into the MPLS network at PE1. All 406 metrics are considered equal to 1 except P12-P13 which is 10000 and 407 P11-P12 which is 100. PE1 wants to steer the traffic using a SPRING 408 path to PE2 along 409 PE1->P1->P7->P8->P9->P4->P5->P10->P11->P12->P13->PE2. By using Adj- 410 SIDs only, PE1 (acting as an I-LSR) will be required to push 10 411 labels on the IP packet received and thus requires an MSD of 10. If 412 the IP packet should be carried over an MPLS service like a regular 413 layer 3 VPN, an additional service label should be imposed, requiring 414 an MSD of 11 for PE1. In addition, if PE1 wants to insert an ELI/EL 415 for load-balancing purpose, PE1 will need to push 13 labels on the IP 416 packet requiring an MSD of 13. 418 In the SPRING architecture, Node-SIDs or Binding-SIDs can be used to 419 reduce the label stack size. As an example, to steer the traffic on 420 the same path as before, PE1 could use the following label stack: 421 . In this example we 422 consider a combination of Node-SIDs and a Binding-SID advertised by 423 P5 that will stitch the traffic along the path P10->P11->P12->P13. 424 The instruction associated with the Binding-SID at P5 is thus to swap 425 Binding_P5 to Adj_P12-P13 and then push . P5 426 acts as a stitching node that pushes additional labels on an existing 427 label stack, P5's MSD needs also to be taken into account and may 428 limit the number of labels that can be imposed. 430 6. LSP stitching using the Binding-SID 432 The Binding-SID allows binding a segment identifier to an existing 433 LSP. As examples, the Binding-SID can represent an RSVP-TE tunnel, 434 an LDP path (through the mapping server advertisement), or a SPRING 435 path. Each tail-end router of an MPLS LSP associated with a Binding- 436 SID has its own entropy label capability. The entropy label 437 capability of the associated LSP is advertised in the control plane 438 protocol used to signal the LSP. 440 In Figure 4, we consider that: 442 o P6, PE2, P10, P11, P12, P13 are pure LDP routers. 444 o PE1, P1, P2, P3, P4, P7, P8, P9 are pure SPRING routers. 446 o P5 is running SPRING and LDP. 448 o P5 acts as a mapping server and advertises Prefix SIDs for the LDP 449 FECs: an index value of 20 is used for PE2. 451 o All SPRING routers use an SRGB of [1000, 1999]. 453 o P6 advertises label 20 for the PE2 FEC. 455 o Traffic from PE1 to PE2 uses the shortest path. 457 PE1 ----- P1 -- P2 -- P3 -- P4 ---- P5 --- P6 --- PE2 459 --> +----+ +----+ +----+ +----+ 460 IP Pkt | IP | | IP | | IP | | IP | 461 +----+ +----+ +----+ +----+ 462 |1020| |1020| | 20 | 463 +----+ +----+ +----+ 464 SPRING LDP 466 In terms of packet forwarding, by learning the mapping-server 467 advertisement from P5, PE1 imposes a label 1020 to an IP packet 468 destined to PE2. SPRING routers along the shortest path to PE2 will 469 switch the traffic until it reaches P5. P5 will perform the LSP 470 stitching by swapping the SPRING label 1020 to the LDP label 20 471 advertised by the nexthop P6. P6 will finally forward the packet 472 using the LDP label towards PE2. 474 PE1 cannot push an ELI/EL for the Binding-SID without knowing that 475 the tail-end of the LSP associated with the binding (PE2) is entropy 476 label capable. 478 To accommodate the mix of signaling protocols involved during the 479 stitching, the entropy label capability SHOULD be propagated between 480 the signaling domains. Each Binding-SID SHOULD have its own entropy 481 label capability that MUST be inherited from the entropy label 482 capability of the associated LSP. If the router advertising the 483 Binding-SID does not know the ELC state of the target FEC, it MUST 484 NOT set the ELC for the Binding-SID. An ingress node MUST NOT push 485 an ELI/EL associated with a Binding-SID unless this Binding-SID has 486 the entropy label capability. How the entropy label capability is 487 advertised for a Binding-SID is outside the scope of this document 488 (see Section 7.2.1 for potential approaches). 490 In our example, if PE2 is LDP entropy label capable, it will add the 491 entropy label capability in its LDP advertisement. When P5 receives 492 the FEC/label binding for PE2, it learns about the ELC and can set 493 the ELC in the mapping server advertisement. Thus PE1 learns about 494 the ELC of PE2 and may push an ELI/EL associated with the Binding- 495 SID. 497 The proposed solution only works if the SPRING router advertising the 498 Binding-SID is also performing the dataplane LSP stitching. In our 499 example, if the mapping server function is hosted on P8 instead of 500 P5, P8 does not know about the ELC state of PE2's LDP FEC. As a 501 consequence, it does not set the ELC for the associated Binding-SID. 503 7. Insertion of entropy labels for SPRING path 505 7.1. Overview 507 The solution described in this section follows the dataplane 508 processing defined in [RFC6790]. Within a SPRING path, a node may be 509 ingress, egress, transit (regarding the entropy label processing 510 described in [RFC6790]), or it can be any combination of those. For 511 example: 513 o The ingress node of a SPRING domain can be an ingress node from an 514 entropy label perspective. 516 o Any LSR terminating a segment of the SPRING path is an egress node 517 (because it terminates the segment) but can also be a transit node 518 if the SPRING path is not terminated because there is a subsequent 519 SPRING MPLS label in the stack. 521 o Any LSR processing a Binding-SID may be a transit node and an 522 ingress node (because it may push additional labels when 523 processing the Binding-SID). 525 As described earlier, an LSR may have a limitation (the ERLD) on the 526 depth of the label stack that it can read and process in order to do 527 multipath load-balancing based on entropy labels. 529 If an EL does not occur within the ERLD of an LSR in the label stack 530 of an MPLS packet that it receives, then it would lead to poor load- 531 balancing at that LSR. Hence an ELI/EL pair must be within the ERLD 532 of the LSR in order for the LSR to use the EL during load-balancing. 534 Adding a single ELI/EL pair for the entire SPRING path can also lead 535 to poor load-balancing as well because the ELI/EL may not occur 536 within the ERLD of some LSR on the path (if too deep) or may not be 537 present in the stack when it reaches some LSRs (if it is too 538 shallow). 540 In order for the EL to occur within the ERLD of LSRs along the path 541 corresponding to a SPRING label stack, multiple pairs MAY 542 be inserted in this label stack. 544 The insertion of an ELI/EL MUST occur only with a SPRING label 545 advertised by an LSR that advertised an ERLD (the LSR is entropy 546 label capable) or with a SPRING label associated with a Binding-SID 547 that has the ELC set. 549 The ELs among multiple pairs inserted in the stack MAY be 550 the same or different. The LSR that inserts pairs can have 551 limitations on the number of such pairs that it can insert and also 552 the depth at which it can insert them. If, due to limitations, the 553 inserted ELs are at positions such that an LSR along the path 554 receives an MPLS packet without an EL in the label stack within that 555 LSR's ERLD, then the load-balancing performed by that LSR would be 556 poor. An implementation MAY consider multiple criteria when 557 inserting pairs. 559 7.1.1. Example 1 where the ingress node has a sufficient MSD 561 ECMP LAG LAG 562 PE1 --- P1 --- P2 --- P3 --- P4 --- P5 --- P6 --- PE2 564 Figure 5 566 In Figure 5, PE1 wants to forward some MPLS VPN traffic over an 567 explicit path to PE2 resulting in the following label stack to be 568 pushed onto the received IP header: . PE1 is limited 570 to push a maximum of 11 labels (MSD=11). P2, P3 and P6 have an ERLD 571 of 3 while others have an ERLD of 10. 573 PE1 can only add two ELI/EL pairs in the label stack due to its MSD 574 limitation. It should insert them strategically to benefit load- 575 balancing along the longest part of the path. 577 PE1 can take into account multiple parameters when inserting ELs, as 578 examples: 580 o The ERLD value advertised by transit nodes. 582 o The requirement of load-balancing for a particular label value. 584 o Any service provider preference: favor beginning of the path or 585 end of the path. 587 In Figure 5, a good strategy may be to use the following stack 588 . The original stack requests P2 to 590 forward based on a L3 adjacency set that will require load-balancing. 591 Therefore it is important to ensure that P2 can load-balance 592 correctly. As P2 has a limited ERLD of 3, an ELI/EL must be inserted 593 just after the label that P2 will use to forward. On the path to 594 PE2, P3 has also a limited ERLD, but P3 will forward based on a 595 regular adjacency segment that may not require load-balancing. 596 Therefore it does not seem important to ensure that P3 can do load- 597 balancing despite its limited ERLD. The next nodes along the 598 forwarding path have a high ERLD that does not cause any issue, 599 except P6. Moreover, P6 is using some LAGs to PE2 and so is expected 600 to load-balance. It becomes important to insert a new ELI/EL just 601 after the P6 forwarding label. 603 In the case above, the ingress node had a sufficient MSD to ensure 604 end-to-end load-balancing taking into the path attributes. However, 605 there might be cases where the ingress node may not have the 606 necessary label imposition capacity. 608 7.1.2. Example 2 where the ingress node does not have a sufficient MSD 610 ECMP LAG ECMP ECMP 611 PE1 --- P1 --- P2 --- P3 --- P4 --- P5 --- P6 --- P7 --- P8 --- PE2 613 Figure 6 615 In Figure 6, PE1 wants to forward MPLS VPN traffic over an explicit 616 path to PE2 resulting in the following label stack to be pushed onto 617 the IP header: . PE1 is limited to 619 push a maximum of 11 labels. P2, P3 and P6 have an ERLD of 3 while 620 others have an ERLD of 15. 622 Using a similar strategy as the previous case may lead to a dilemma, 623 as PE1 can only push a single ELI/EL while we may need a minimum of 624 three to load-balance the end-to-end path. An optimized stack that 625 would enable end-to-end load-balancing may be: . 629 A decision needs to be taken to favor some part of the path for load- 630 balancing considering that load-balancing may not work on the other 631 parts. A service provider may decide to place the ELI/EL after the 632 P6 forwarding label as it will allow P4 and P6 to load-balance. 633 Placing the ELI/EL at bottom of the stack is also a possibility 634 enabling load-balancing for P4 and P8. 636 7.2. Considerations for the placement of entropy labels 638 The sample cases described in the previous section showed that ELI/EL 639 placement when the maximum number of labels to be pushed is limited 640 is not an easy decision and multiple criteria may be taken into 641 account. 643 This section describes some considerations that an implementation MAY 644 take into account when placing ELI/ELs. This list of criteria is not 645 considered exhaustive and an implementation MAY take into account 646 additional criteria or tie-breakers that are not documented here. As 647 the insertion of ELI/ELs is performed by the ingress node, having 648 ingress nodes that do not use the same criteria does not cause an 649 interoperability issue. However, from a network design and operation 650 perspective, it is better to have all ingress routers using the same 651 criteria. 653 An implementation SHOULD try to maximize the possibility of load- 654 balancing along the path by inserting an ELI/EL where multiple equal 655 cost paths are available and minimize the number of ELI/ELs that need 656 to be inserted. In case of a trade-off, an implementation SHOULD 657 provide flexibility to the operator to select the criteria to be 658 considered when placing ELI/ELs or specify a sub-objective for 659 optimization. 661 2 2 662 PE1 -- P1 -- P2 --P3 --- P4 --- P5 -- ... -- P8 -- P9 -- PE2 663 | | 664 P3'--- P4'--- P5' 666 Figure 7 668 Figure 7 will be used as reference in the following subsections. All 669 metrics are equal to 1, except P3-P4 and P4-P5 which have a metric 2. 670 We consider the MSD of nodes to be the full limit of label imposition 671 (including service labels, entropy labels and transport labels). 673 7.2.1. ERLD value 675 As mentioned in Section 7.1, the ERLD value is an important parameter 676 to consider when inserting an ELI/EL. If an ELI/EL does not fall 677 within the ERLD of a node on the path, the node will not be able to 678 load-balance the traffic efficiently. 680 The ERLD value can be advertised via protocols and those extensions 681 are described in separate documents (for instance, 682 [I-D.ietf-isis-mpls-elc] and [I-D.ietf-ospf-mpls-elc]). 684 Let's consider a path from PE1 to PE2 using the following stack 685 pushed by PE1: . 687 Using the ERLD as an input parameter can help to minimize the number 688 of required ELI/EL pairs to be inserted. An ERLD value must be 689 retrieved for each SPRING label in the label stack. 691 For a label bound to an adjacency segment, the ERLD is the ERLD of 692 the node that has advertised the adjacency segment. In the example 693 above, the ERLD associated with Adj_P1P2 would be the ERLD of router 694 P1 as P1 will perform the forwarding based on the Adj_P1P2 label. 696 For a label bound to a node segment, multiple strategies MAY be 697 implemented. An implementation MAY try to evaluate the minimum ERLD 698 value along the node segment path. If an implementation cannot find 699 the minimum ERLD along the path of the segment or does not support 700 the computation of the minimum ERLD, it SHOULD instead use the ERLD 701 of the tail-end node. Using the ERLD of the tail-end of the node 702 segment mimics the behavior of [RFC6790] where the ingress takes only 703 care of the egress of the LSP. In the example above, if the 704 implementation supports computation of minimum ERLD along the path, 705 the ERLD associated with label Node_P9 would be the minimum ERLD 706 between nodes {P2,P3,P4 ..., P8}. If the implementation does not 707 support the computation of minimum ERLD, it will consider the ERLD of 708 P9 (tail-end node of Node_P9 SID). While providing the more optimal 709 ELI/EL placement, evaluating the minimum ERLD increases the 710 complexity of ELI/EL insertion. As the path to the Node-SID may 711 change over time, a recomputation of the minimum ERLD is required for 712 each topology change. This recomputation may require the positions 713 of the ELI/ELs to change. 715 For a label bound to a binding segment, if the binding segment 716 describes a path, an implementation MAY also try to evaluate the 717 minimum ERLD along this path. If the implementation cannot find the 718 minimum ERLD along the path of the segment or does not support this 719 evaluation, it SHOULD instead use the ERLD of the node advertising 720 the Binding-SID. As for the node segment, evaluating the minimum 721 ERLD adds complexity in the ELI/EL insertion process. 723 7.2.2. Segment type 725 Depending on the type of segment a particular label is bound to, an 726 implementation can deduce that this particular label will be subject 727 to load-balancing on the path. 729 7.2.2.1. Node-SID 731 An MPLS label bound to a Node-SID represents a path that may cross 732 multiple hops. Load-balancing may be needed on the node starting 733 this path but also on any node along the path. 735 In Figure 7, let's consider a path from PE1 to PE2 using the 736 following stack pushed by PE1: . 739 If, for example, PE1 is limited to push 6 labels, it can add a single 740 ELI/EL within the label stack. An operator may want to favor a 741 placement that would allow load-balancing along the Node-SID path. 742 In Figure 7, P3 which is along the Node-SID path requires load- 743 balancing between two equal-cost paths. 745 An implementation MAY try to evaluate if load-balancing is really 746 required within a node segment path. This could be done by running 747 an additional SPT (Shortest Path Tree) computation and analysing of 748 the node segment path to prevent a node segment that does not really 749 require load-balancing from being preferred when placing ELI/ELs. 750 Such inspection may be time consuming for implementations and without 751 a 100% guarantee, as a node segment path may use LAGs that are 752 invisible to the IP topology. As a simpler approach, an 753 implementation MAY consider that a label bound to a Node-SID will be 754 subject to load-balancing and requires an ELI/EL. 756 7.2.2.2. Adjacency-set SID 758 An adjacency-set is an Adj-SID that refers to a set of adjacencies. 759 When an adjacency-set segment is used within a label stack, an 760 implementation can deduce that load-balancing is expected at the node 761 that advertised this adjacency segment. An implementation MAY favor 762 the insertion of an ELI/EL after the Adj-SID representing an 763 adjacency-set. 765 7.2.2.3. Adjacency-SID representing a single IP link 767 When an adjacency segment representing a single IP link is used 768 within a label stack, an implementation can deduce that load- 769 balancing may not be expected at the node that advertised this 770 adjacency segment. 772 An implementation MAY NOT place an ELI/EL after a regular Adj-SID in 773 order to favor the insertion of ELI/ELs following other segments. 775 Readers should note that an adjacency segment representing a single 776 IP link may require load-balancing. This is the case when a LAG (L2 777 bundle) is implemented between two IP nodes and the L2 bundle SR 778 extensions [I-D.ietf-isis-l2bundles] are not implemented. In such a 779 case, it could be useful to insert an ELI/EL in a readable position 780 for the LSR advertising the label associated with the adjacency 781 segment. To communicate the requirement for load-balancing for a 782 particular Adjacency-SID to ingress nodes, a user can enforce the use 783 of the L2 bundle SR extensions defined in [I-D.ietf-isis-l2bundles] 784 or can declare the single adjacency as an adjacency-set. 786 7.2.2.4. Adjacency-SID representing a single link within an L2 bundle 788 When the L2 bundle SR extensions [I-D.ietf-isis-l2bundles] are used, 789 adjacency segments may be advertised for each member of the bundle. 790 In this case, an implementation can deduce that load-balancing is not 791 expected on the LSR advertising this segment and MAY NOT insert an 792 ELI/EL after the corresponding label. 794 7.2.2.5. Adjacency-SID representing an L2 bundle 796 When the L2 bundle SR extensions [I-D.ietf-isis-l2bundles] are used, 797 an adjacency segment may be advertised to represent the bundle. In 798 this case, an implementation can deduce that load-balancing is 799 expected on the LSR advertising this segment and MAY insert an ELI/EL 800 after the corresponding label. 802 7.2.3. Maximizing number of LSRs that will load-balance 804 When placing ELI/ELs, an implementation MAY optimize the number of 805 LSRs that both need to load-balance (i.e., have ECMP paths) and that 806 will be able to perform load-balancing (i.e., the EL label is within 807 their ERLD). 809 Let's consider a path from PE1 to PE2 using the following stack 810 pushed by PE1: . All 811 routers have an ERLD of 10, except P1 and P2 which have an ERLD of 4. 812 PE1 is able to push 6 labels, so only a single ELI/EL can be added. 814 In the example above, adding an ELI/EL after Adj_P1P2 will only allow 815 load-balancing at P1 while inserting it after Adj_PE2P9, will allow 816 load-balancing at P2,P3 ... P9 and maximizing the number of LSRs that 817 can perform load-balancing. 819 7.2.4. Preference for a part of the path 821 An implementation MAY allow the user to favor a part of the end-to- 822 end path when the number of ELI/ELs that can be pushed is not enough 823 to cover the entire path. As an example, a service provider may want 824 to favor load-balancing at the beginning of the path or at the end of 825 path, so the implementation favors putting the ELI/ELs near the top 826 or near of the bottom of the stack. 828 7.2.5. Combining criteria 830 An implementation MAY combine multiple criteria to determine the best 831 ELI/ELs placement. However, combining too many criteria could lead 832 to implementation complexity and high resource consumption. Each 833 time the network topology changes, a new evaluation of the ELI/EL 834 placement will be necessary for each impacted LSPs. 836 8. A simple example algorithm 838 A simple implementation might take into account the ERLD when placing 839 ELI/EL while trying to minimize the number of ELI/ELs inserted and 840 trying to maximize the number of LSRs that can load-balance. 842 The example algorithm is based on the following considerations: 844 o An LSR that can insert a limited number of pairs should 845 insert such pairs deeper in the stack. 847 o An LSR should try to insert pairs at positions to 848 maximize the number of transit LSRs for which the EL occurs within 849 the ERLD of those LSRs. 851 o An LSR should try to insert the minimum number of such pairs while 852 trying to satisfy the above criteria. 854 The pseudocode of the example algorithm is shown below. 856 Initialize the current EL insertion point to the 857 bottom-most label in the stack that is EL-capable 858 while (local-node can push more pairs OR 859 insertion point is not above label stack) { 860 insert an pair below current insertion point 861 move new insertion point up from current insertion point until 862 ((last inserted EL is below the ERLD) AND (ERLD > 2) 863 AND 864 (new insertion point is EL-capable)) 865 set current insertion point to new insertion point 866 } 868 Figure 8: Example algorithm to insert pairs in a label 869 stack 871 When this algorithm is applied to the example described in Section 3, 872 it will result in ELs being inserted in two positions, one after the 873 label L_N-D and another after L_N-P3. Thus, the resulting label 874 stack would be 876 9. Deployment Considerations 878 As long as LSR node dataplane capabilities are limited (number of 879 labels that can be pushed, or number of labels that can be 880 inspected), hop-by-hop load-balancing of SPRING encapsulated flows 881 will require trade-offs. 883 The entropy label is still a good and usable solution as it allows 884 load-balancing without having to perform deep packet inspection on 885 each LSR: it does not seem reasonable to have an LSR inspecting UDP 886 ports within a GRE tunnel carried over a 15 label SPRING tunnel. 888 Due to the limited capacity of reading a deep stack of MPLS labels, 889 multiple ELI/ELs may be required within the stack which directly 890 impacts the capacity of the head-end to push a deep stack: each ELI/ 891 EL inserted requires two additional labels to be pushed. 893 Placement strategies of ELI/ELs are required to find the best trade- 894 off. Multiple criteria could be taken into account and some level of 895 customization (by the user) is required to accommodate different 896 deployments. Since analyzing the path of each destination to 897 determine the best ELI/EL placement may be time consuming for the 898 control plane, we encourage implementations to find the best trade- 899 off between simplicity, resource consumption, and load-balancing 900 efficiency. 902 In the future, hardware and software capacity may increase dataplane 903 capabilities and may remove some of these limitations, increasing 904 load-balancing capability using entropy labels. 906 10. Options considered 908 Different options that were considered to arrive at the recommended 909 solution are documented in this section. 911 These options are detailed here only for historical purposes. 913 10.1. Single EL at the bottom of the stack 915 In this option, a single EL is used for the entire label stack. The 916 source LSR S encodes the entropy label at the bottom of the label 917 stack. In the example described in Section 3, it will result in the 918 label stack at LSR S to look like 919 . Note that the notation in [RFC6790] is 920 used to describe the label stack. An issue with this approach is 921 that as the label stack grows due an increase in the number of SIDs, 922 the EL goes correspondingly deeper in the label stack. Hence, 923 transit LSRs have to access a larger number of bytes in the packet 924 header when making forwarding decisions. In the example described in 925 Section 3, if we consider that the LSR P1 has an ERLD of 3, P1 would 926 load-balance traffic poorly on the parallel links L3 and L4 since the 927 EL is below the ERLD of P1. A load-balanced network design using 928 this approach must ensure that all intermediate LSRs have the 929 capability to read the maximum label stack depth as required for the 930 application that uses source routed stacking. 932 This option was rejected since there exist a number of hardware 933 implementations which have a low maximum readable label depth. 934 Choosing this option can lead to a loss of load-balancing using EL in 935 a significant part of the network when that is a critical requirement 936 in a service-provider network. 938 10.2. An EL per segment in the stack 940 In this option, each segment/label in the stack can be given its own 941 EL. When load-balancing is required to direct traffic on a segment, 942 the source LSR pushes an before pushing the label 943 associated to this segment . In the example described in Section 3, 944 the source LSR S encoded label stack would be where all the ELs can be the same. Accessing the 946 EL at an intermediate LSR is independent of the depth of the label 947 stack and hence independent of the specific application that uses 948 source routed tunnels with label stacking. A drawback is that the 949 depth of the label stack grows significantly, almost 3 times as the 950 number of labels in the label stack. The network design should 951 ensure that source LSRs have the capability to push such a deep label 952 stack. Also, the bandwidth overhead and potential MTU issues of deep 953 label stacks should be considered in the network design. 955 This option was rejected due to the existence of hardware 956 implementations that can push a limited number of labels on the label 957 stack. Choosing this option would result in a hardware requirement 958 to push two additional labels per tunnel label. Hence it would 959 restrict the number of tunnels that can be stacked in a LSP and hence 960 constrain the types of LSPs that can be created. This was considered 961 unacceptable. 963 10.3. A re-usable EL for a stack of tunnels 965 In this option an LSR that terminates a tunnel re-uses the EL of the 966 terminated tunnel for the next inner tunnel. It does this by storing 967 the EL from the outer tunnel when that tunnel is terminated and re- 968 inserting it below the next inner tunnel label during the label swap 969 operation. The LSR that stacks tunnels should insert an EL below the 970 outermost tunnel. It should not insert ELs for any inner tunnels. 971 Also, the penultimate hop LSR of a segment must not pop the ELI and 972 EL even though they are exposed as the top labels since the 973 terminating LSR of that segment would re-use the EL for the next 974 segment. 976 In Section 3 above, the source LSR S encoded label stack would be 977 . At P1, the outgoing label stack 978 would be after it has load-balanced 979 to one of the links L3 or L4. At P3 the outgoing label stack would 980 be . At P2, the outgoing label stack would be and it would load-balance to one of the nexthop LSRs P4 982 or P5. Accessing the EL at an intermediate LSR (e.g., P1) is 983 independent of the depth of the label stack and hence independent of 984 the specific use-case to which the label stack is applied. 986 This option was rejected due to the significant change in label swap 987 operations that would be required for existing hardware. 989 10.4. EL at top of stack 991 A slight variant of the re-usable EL option is to keep the EL at the 992 top of the stack rather than below the tunnel label. In this case, 993 each LSR that is not terminating a segment should continue to keep 994 the received EL at the top of the stack when forwarding the packet 995 along the segment. An LSR that terminates a segment should use the 996 EL from the terminated segment at the top of the stack when 997 forwarding onto the next segment. 999 This option was rejected due to the significant change in label swap 1000 operations that would be required for existing hardware. 1002 10.5. ELs at readable label stack depths 1004 In this option the source LSR inserts ELs for tunnels in the label 1005 stack at depths such that each LSR along the path that must load 1006 balance is able to access at least one EL. Note that the source LSR 1007 may have to insert multiple ELs in the label stack at different 1008 depths for this to work since intermediate LSRs may have differing 1009 capabilities in accessing the depth of a label stack. The label 1010 stack depth access value of intermediate LSRs must be known to create 1011 such a label stack. How this value is determined is outside the 1012 scope of this document. This value can be advertised using a 1013 protocol such as an IGP. 1015 Applying this method to the example in Section 3 above, if LSR P1 1016 needs to have the EL within a depth of 4, then the source LSR S 1017 encoded label stack would be where all the ELs would typically have the same value. 1020 In the case where the ERLD has different values along the path and 1021 the LSR that is inserting pairs has no limit on how many 1022 pairs it can insert, and it knows the appropriate positions in the 1023 stack where they should be inserted, this option is the same as the 1024 recommended solution in Section 7. 1026 Note that a refinement of this solution which balances the number of 1027 pushed labels against the desired entropy is the solution described 1028 in Section 7. 1030 11. Acknowledgements 1032 The authors would like to thank John Drake, Loa Andersson, Curtis 1033 Villamizar, Greg Mirsky, Markus Jork, Kamran Raza, Carlos Pignataro, 1034 Bruno Decraene, Chris Bowers, Nobo Akiya, Daniele Ceccarelli and Joe 1035 Clarke for their review comments and suggestions. 1037 12. Contributors 1038 Xiaohu Xu 1039 Huawei 1041 Email: xuxiaohu@huawei.com 1043 Wim Hendrickx 1044 Nokia 1046 Email: wim.henderickx@nokia.com 1048 Gunter Van De Velde 1049 Nokia 1051 Email: gunter.van_de_velde@nokia.com 1053 Acee Lindem 1054 Cisco 1056 Email: acee@cisco.com 1058 13. IANA Considerations 1060 This memo includes no request to IANA. Note to RFC Editor: Remove 1061 this section before publication. 1063 14. Security Considerations 1065 Compared to [RFC6790], this document introduces the notion of ERLD, 1066 MSD and may require an ingress node to push multiple ELI/EL. These 1067 changes does not introduce any new security considerations beyond 1068 those already listed in [RFC6790]. 1070 15. References 1072 15.1. Normative References 1074 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1075 Requirement Levels", BCP 14, RFC 2119, 1076 DOI 10.17487/RFC2119, March 1997, 1077 . 1079 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 1080 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 1081 RFC 6790, DOI 10.17487/RFC6790, November 2012, 1082 . 1084 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1085 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1086 May 2017, . 1088 [I-D.ietf-spring-segment-routing] 1089 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., 1090 Litkowski, S., and R. Shakir, "Segment Routing 1091 Architecture", draft-ietf-spring-segment-routing-15 (work 1092 in progress), January 2018. 1094 [I-D.ietf-spring-segment-routing-mpls] 1095 Bashandy, A., Filsfils, C., Previdi, S., Decraene, B., 1096 Litkowski, S., and R. Shakir, "Segment Routing with MPLS 1097 data plane", draft-ietf-spring-segment-routing-mpls-14 1098 (work in progress), June 2018. 1100 15.2. Informative References 1102 [I-D.ietf-isis-mpls-elc] 1103 Xu, X., Kini, S., Sivabalan, S., Filsfils, C., and S. 1104 Litkowski, "Signaling Entropy Label Capability and 1105 Readable Label-stack Depth Using IS-IS", draft-ietf-isis- 1106 mpls-elc-03 (work in progress), January 2018. 1108 [I-D.ietf-ospf-mpls-elc] 1109 Xu, X., Kini, S., Sivabalan, S., Filsfils, C., and S. 1110 Litkowski, "Signaling Entropy Label Capability and 1111 Readable Label-stack Depth Using OSPF", draft-ietf-ospf- 1112 mpls-elc-05 (work in progress), January 2018. 1114 [I-D.ietf-isis-l2bundles] 1115 Ginsberg, L., Bashandy, A., Filsfils, C., Nanduri, M., and 1116 E. Aries, "Advertising L2 Bundle Member Link Attributes in 1117 IS-IS", draft-ietf-isis-l2bundles-07 (work in progress), 1118 May 2017. 1120 [RFC7855] Previdi, S., Ed., Filsfils, C., Ed., Decraene, B., 1121 Litkowski, S., Horneffer, M., and R. Shakir, "Source 1122 Packet Routing in Networking (SPRING) Problem Statement 1123 and Requirements", RFC 7855, DOI 10.17487/RFC7855, May 1124 2016, . 1126 [I-D.ietf-isis-segment-routing-msd] 1127 Tantsura, J., Chunduri, U., Aldrin, S., and L. Ginsberg, 1128 "Signaling MSD (Maximum SID Depth) using IS-IS", draft- 1129 ietf-isis-segment-routing-msd-12 (work in progress), May 1130 2018. 1132 [I-D.ietf-ospf-segment-routing-msd] 1133 Tantsura, J., Chunduri, U., Aldrin, S., and P. Psenak, 1134 "Signaling MSD (Maximum SID Depth) using OSPF", draft- 1135 ietf-ospf-segment-routing-msd-14 (work in progress), May 1136 2018. 1138 [I-D.ietf-idr-bgp-ls-segment-routing-msd] 1139 Tantsura, J., Chunduri, U., Mirsky, G., and S. Sivabalan, 1140 "Signaling Maximum SID Depth using Border Gateway Protocol 1141 Link-State", draft-ietf-idr-bgp-ls-segment-routing-msd-01 1142 (work in progress), October 2017. 1144 Authors' Addresses 1146 Sriganesh Kini 1148 EMail: sriganeshkini@gmail.com 1150 Kireeti Kompella 1151 Juniper 1153 EMail: kireeti@juniper.net 1155 Siva Sivabalan 1156 Cisco 1158 EMail: msiva@cisco.com 1160 Stephane Litkowski 1161 Orange 1163 EMail: stephane.litkowski@orange.com 1165 Rob Shakir 1166 Google 1168 EMail: rjs@rob.sh 1169 Jeff Tantsura 1171 EMail: jefftant@gmail.com