idnits 2.17.1 draft-ietf-rtgwg-cl-framework-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (August 12, 2012) is 4268 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-06) exists of draft-ietf-mpls-entropy-label-01 == Outdated reference: A later version (-09) exists of draft-ietf-mpls-tp-security-framework-02 == Outdated reference: A later version (-16) exists of draft-ietf-rtgwg-cl-requirement-05 == Outdated reference: A later version (-06) exists of draft-kompella-mpls-rsvp-ecmp-01 == Outdated reference: A later version (-01) exists of draft-symmvo-rtgwg-cl-use-cases-00 == Outdated reference: A later version (-03) exists of draft-villamizar-mpls-tp-multipath-01 == Outdated reference: A later version (-02) exists of draft-villamizar-mpls-tp-multipath-te-extn-00 -- Obsolete informational reference (is this intentional?): RFC 5316 (Obsoleted by RFC 9346) Summary: 1 error (**), 0 flaws (~~), 8 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RTGWG S. Ning 3 Internet-Draft Tata Communications 4 Intended status: Informational D. McDysan 5 Expires: February 13, 2013 Verizon 6 E. Osborne 7 Cisco 8 L. Yong 9 Huawei USA 10 C. Villamizar 11 Outer Cape Cod Network 12 Consulting 13 August 12, 2012 15 Composite Link Framework in Multi Protocol Label Switching (MPLS) 16 draft-ietf-rtgwg-cl-framework-00 18 Abstract 20 This document specifies a framework for support of composite link in 21 MPLS networks. A composite link consists of a group of homogenous or 22 non-homogenous links that have the same forward adjacency and can be 23 considered as a single TE link or an IP link in routing. A composite 24 link relies on its component links to carry the traffic over the 25 composite link. Applicability is described for a single pair of 26 MPLS-capable nodes, a sequence of MPLS-capable nodes, or a set of 27 layer networks connecting MPLS-capable nodes. 29 Status of this Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at http://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on February 13, 2013. 46 Copyright Notice 48 Copyright (c) 2012 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.1. Architecture Summary . . . . . . . . . . . . . . . . . . . 4 65 1.2. Conventions used in this document . . . . . . . . . . . . 5 66 1.2.1. Terminology . . . . . . . . . . . . . . . . . . . . . 5 67 2. Composite Link Key Characteristics . . . . . . . . . . . . . . 5 68 2.1. Flow Identification . . . . . . . . . . . . . . . . . . . 6 69 2.2. Composite Link in Control Plane . . . . . . . . . . . . . 8 70 2.3. Composite Link in Data Plane . . . . . . . . . . . . . . . 11 71 3. Architecture Tradeoffs . . . . . . . . . . . . . . . . . . . . 11 72 3.1. Scalability Motivations . . . . . . . . . . . . . . . . . 12 73 3.2. Reducing Routing Information and Exchange . . . . . . . . 12 74 3.3. Reducing Signaling Load . . . . . . . . . . . . . . . . . 13 75 3.3.1. Reducing Signaling Load using LDP . . . . . . . . . . 14 76 3.3.2. Reducing Signaling Load using Hierarchy . . . . . . . 14 77 3.3.3. Using Both LDP and RSVP-TE Hierarchy . . . . . . . . . 14 78 3.4. Reducing Forwarding State . . . . . . . . . . . . . . . . 14 79 3.5. Avoiding Route Oscillation . . . . . . . . . . . . . . . . 15 80 4. New Challenges . . . . . . . . . . . . . . . . . . . . . . . . 16 81 4.1. Control Plane Challenges . . . . . . . . . . . . . . . . . 16 82 4.1.1. Delay and Jitter Sensitive Routing . . . . . . . . . . 17 83 4.1.2. Local Control of Traffic Distribution . . . . . . . . 17 84 4.1.3. Path Symmetry Requirements . . . . . . . . . . . . . . 17 85 4.1.4. Requirements for Contained LSP . . . . . . . . . . . . 18 86 4.1.5. Retaining Backwards Compatibility . . . . . . . . . . 19 87 4.2. Data Plane Challenges . . . . . . . . . . . . . . . . . . 19 88 4.2.1. Very Large LSP . . . . . . . . . . . . . . . . . . . . 20 89 4.2.2. Very Large Microflows . . . . . . . . . . . . . . . . 20 90 4.2.3. Traffic Ordering Constraints . . . . . . . . . . . . . 20 91 4.2.4. Accounting for IP and LDP Traffic . . . . . . . . . . 21 92 4.2.5. IP and LDP Limitations . . . . . . . . . . . . . . . . 21 93 5. Existing Mechanisms . . . . . . . . . . . . . . . . . . . . . 22 94 5.1. Link Bundling . . . . . . . . . . . . . . . . . . . . . . 22 95 5.2. Classic Multipath . . . . . . . . . . . . . . . . . . . . 24 97 6. Mechanisms Proposed in Other Documents . . . . . . . . . . . . 24 98 6.1. Loss and Delay Measurement . . . . . . . . . . . . . . . . 24 99 6.2. Link Bundle Extensions . . . . . . . . . . . . . . . . . . 25 100 6.3. Fat PW and Entropy Labels . . . . . . . . . . . . . . . . 26 101 6.4. Multipath Extensions . . . . . . . . . . . . . . . . . . . 26 102 7. Required Protocol Extensions and Mechanisms . . . . . . . . . 27 103 7.1. Brief Review of Requirements . . . . . . . . . . . . . . . 27 104 7.2. Required Document Coverage . . . . . . . . . . . . . . . . 28 105 7.2.1. Component Link Grouping . . . . . . . . . . . . . . . 28 106 7.2.2. Delay and Jitter Extensions . . . . . . . . . . . . . 29 107 7.2.3. Path Selection and Admission Control . . . . . . . . . 29 108 7.2.4. Dynamic Multipath Balance . . . . . . . . . . . . . . 30 109 7.2.5. Frequency of Load Balance . . . . . . . . . . . . . . 30 110 7.2.6. Inter-Layer Communication . . . . . . . . . . . . . . 30 111 7.2.7. Packet Ordering Requirements . . . . . . . . . . . . . 31 112 7.2.8. Minimally Disruption Load Balance . . . . . . . . . . 31 113 7.2.9. Path Symmetry . . . . . . . . . . . . . . . . . . . . 31 114 7.2.10. Performance, Scalability, and Stability . . . . . . . 32 115 7.2.11. IP and LDP Traffic . . . . . . . . . . . . . . . . . . 32 116 7.2.12. LDP Extensions . . . . . . . . . . . . . . . . . . . . 32 117 7.2.13. Pseudowire Extensions . . . . . . . . . . . . . . . . 33 118 7.2.14. Multi-Domain Composite Link . . . . . . . . . . . . . 33 119 7.3. Open Issues Regarding Requirements . . . . . . . . . . . . 34 120 7.4. Framework Requirement Coverage by Protocol . . . . . . . . 34 121 7.4.1. OSPF-TE and ISIS-TE Protocol Extensions . . . . . . . 35 122 7.4.2. PW Protocol Extensions . . . . . . . . . . . . . . . . 35 123 7.4.3. LDP Protocol Extensions . . . . . . . . . . . . . . . 35 124 7.4.4. RSVP-TE Protocol Extensions . . . . . . . . . . . . . 35 125 7.4.5. RSVP-TE Path Selection Changes . . . . . . . . . . . . 35 126 7.4.6. RSVP-TE Admission Control and Preemption . . . . . . . 35 127 7.4.7. Flow Identification and Traffic Balance . . . . . . . 35 128 8. Security Considerations . . . . . . . . . . . . . . . . . . . 35 129 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 36 130 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 36 131 10.1. Normative References . . . . . . . . . . . . . . . . . . . 36 132 10.2. Informative References . . . . . . . . . . . . . . . . . . 37 133 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 40 135 1. Introduction 137 Composite Link functional requirements are specified in 138 [I-D.ietf-rtgwg-cl-requirement]. Composite Link use cases are 139 described in [I-D.symmvo-rtgwg-cl-use-cases]. This document 140 specifies a framework to meet these requirements. 142 Classic multipath, including Ethernet Link Aggregation has been 143 widely used in today's MPLS networks [RFC4385][RFC4928]. Classic 144 multipath using non-Ethernet links are often advertised using MPLS 145 Link bundling. A link bundle [RFC4201] bundles a group of 146 homogeneous links as a TE link to make IGP-TE information exchange 147 and RSVP-TE signaling more scalable. A composite link allows 148 bundling non-homogenous links together as a single logical link. The 149 motivations for using a composite link are descried in 150 [I-D.ietf-rtgwg-cl-requirement] and [I-D.symmvo-rtgwg-cl-use-cases]. 152 This document describes a composite link framework in the context of 153 MPLS networks using an IGP-TE and RSVP-TE MPLS control plane with 154 GMPLS extensions [RFC3209][RFC3630][RFC3945][RFC5305]. 156 A composite link is a single logical link in MPLS network that 157 contains multiple parallel component links between two MPLS LSR. 158 Unlike a link bundle [RFC4201], the component links in a composite 159 link can have different properties such as cost or capacity. 161 Specific protocol solutions are outside the scope of this document, 162 however a framework for the extension of existing protocols is 163 provided. Backwards compatibility is best achieved by extending 164 existing protocols where practical rather than inventing new 165 protocols. The focus is on examining where existing protocol 166 mechanisms fall short with respect to [I-D.ietf-rtgwg-cl-requirement] 167 and on extensions that will be required to accommodate functionality 168 that is called for in [I-D.ietf-rtgwg-cl-requirement]. 170 1.1. Architecture Summary 172 Networks aggregate information, both in the control plane and in the 173 data plane, as a means to achieve scalability. A tradeoff exists 174 between the needs of scalability and the needs to identify differing 175 path and link characteristics and differing requirements among flows 176 contained within further aggregated traffic flows. These tradeoffs 177 are discussed in detail in Section 3. 179 Some aspects of Composite Link requirements present challenges for 180 which multiple solutions may exist. In Section 4 various challenges 181 and potential approaches are discussed. 183 A subset of the functionality called for in 184 [I-D.ietf-rtgwg-cl-requirement] is available through MPLS Link 185 Bundling [RFC4201]. Link bundling and other existing standards 186 applicable to Composite Link are covered in Section 5. 188 The most straightforward means of supporting Composite Link 189 requirements is to extend MPLS protocols and protocol semantics and 190 in particular to extend link bundling. Extensions which have already 191 been proposed in other documents which are applicable to Composite 192 Link are discussed in Section 6. 194 Goals of most new protocol work within IETF is to reuse existing 195 protocol encapsulations and mechanisms where they meet requirements 196 and extend existing mechanisms such that additional complexity is 197 minimized while meeting requirements and such that backwards 198 compatibility is preserved to the extent it is practical to do so. 199 These goals are considered in proposing a framework for further 200 protocol extensions and mechanisms in Section 7. 202 1.2. Conventions used in this document 204 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 205 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 206 document are to be interpreted as described in RFC 2119 [RFC2119]. 208 1.2.1. Terminology 210 Terminology defined in [I-D.ietf-rtgwg-cl-requirement] is used in 211 this document. 213 The abbreviation IGP-TE is used as a shorthand indicating either 214 OSPF-TE [RFC3630] or ISIS-TE [RFC5305]. 216 2. Composite Link Key Characteristics 218 [I-D.ietf-rtgwg-cl-requirement] defines external behavior of 219 Composite Links. The overall framework approach involves extending 220 existing protocols in a backwards compatible manner and reusing 221 ongoing work elsewhere in IETF where applicable, defining new 222 protocols or semantics only where necessary. Given the requirements, 223 and this approach of extending MPLS, Composite Link key 224 characteristics can be described in greater detail than given 225 requirements alone. 227 2.1. Flow Identification 229 Traffic mapping to component links is a data plane operation. 230 Control over how the mapping is done may be directly dictated or 231 constrained by the control plane or by the management plane. When 232 unconstrained by the control plane or management plane, distribution 233 of traffic is entirely a local matter. Regardless of constraints or 234 lack or constraints, the traffic distribution is required to keep 235 packets belonging to individual flows in sequence and meet QoS 236 criteria specified per LSP by either signaling or management 237 [RFC2475][RFC3260]. A key objective of the traffic distribution is 238 to not overload any component link, and be able to perform local 239 recovery when one of component link fails. 241 The network operator may have other objectives such as placing a 242 bidirectional flow or LSP on the same component link in both 243 direction, load balance over component links, composite link energy 244 saving, and etc. These new requirements are described in 245 [I-D.ietf-rtgwg-cl-requirement]. 247 Examples of means to identify a flow may in principle include: 249 1. an LSP identified by an MPLS label, 251 2. a sub-LSP [I-D.kompella-mpls-rsvp-ecmp] identified by an MPLS 252 label, 254 3. a pseudowire (PW) [RFC3985] identified by an MPLS PW label, 256 4. a flow or group of flows within a pseudowire (PW) [RFC6391] 257 identified by an MPLS flow label, 259 5. a flow or flow group in an LSP [I-D.ietf-mpls-entropy-label] 260 identified by an MPLS entropy label, 262 6. all traffic between a pair of IP hosts, identified by an IP 263 source and destination pair, 265 7. a specific connection between a pair of IP hosts, identified by 266 an IP source and destination pair, protocol, and protocol port 267 pair, 269 8. a layer-2 conversation within a pseudowire (PW), where the 270 identification is PW payload type specific, such as Ethernet MAC 271 addresses and VLAN tags within an Ethernet PW (RFC4448). 273 Although in principle a layer-2 conversation within a pseudowire 274 (PW), may be identified by PW payload type specific information, in 275 practice this is impractical at LSP midpoints when PW are carried. 276 The PW ingress may provide equivalent information in a PW flow label 277 [RFC6391]. Therefore, in practice, item #8 above is covered by 278 [RFC6391] and may be dropped from the list. 280 An LSR must at least be capable of identifying flows based on MPLS 281 labels. Most MPLS LSP do not require that traffic carried by the LSP 282 are carried in order. MPLS-TP is a recent exception. If it is 283 assumed that no LSP require strict packet ordering of the LSP itself 284 (only of flows within the LSP), then the entire label stack can be 285 used as flow identification. If some LSP may require strict packet 286 ordering but those LSP cannot be distinguished from others, then only 287 the top label can be used as a flow identifier. If only the top 288 label is used (for example, as specified by [RFC4201] when the "all- 289 ones" component described in [RFC4201] is not used), then there may 290 not be adequate flow granularity to accomplish well balanced traffic 291 distribution and it will not be possible to carry LSP that are larger 292 than any individual component link. 294 The number of flows can be extremely large. This may be the case 295 when the entire label stack is used and is always the case when IP 296 addresses are used in provider networks carrying Internet traffic. 297 Current practice for native IP load balancing at the time of writing 298 were documented in [RFC2991], [RFC2992]. These practices as 299 described, make use of IP addresses. The common practices were 300 extended to include the MPLS label stack and the common practice of 301 looking at IP addresses within the MPLS payload. These extended 302 practices are described in [RFC4385] and [RFC4928] due to their 303 impact on pseudowires without a PWE3 Control Word. Additional detail 304 on current multipath practices can be found in the appendices of 305 [I-D.symmvo-rtgwg-cl-use-cases]. 307 Using only the top label supports too coarse a traffic balance. 308 Using the full label stack or IP addresses as flow identification 309 provides a sufficiently fine traffic balance, but is capable of 310 identifying such a high number of distinct flows, that a technique of 311 grouping flows, such as hashing on the flow identification criteria, 312 becomes essential to reduce the stored state, and is an essential 313 scaling technique. Other means of grouping flows may be possible. 315 In summary: 317 1. Load balancing using only the MPLS label stack provides too 318 coarse a granularity of load balance. 320 2. Tracking every flow is not scalable due to the extremely large 321 number of flows in provider networks. 323 3. Existing techniques, IP source and destination hash in 324 particular, have proven in over two decades of experience to be 325 an excellent way of identifying groups of flows. 327 4. If a better way to identify groups of flows is discovered, then 328 that method can be used. 330 5. IP address hashing is not required, but use of this technique is 331 strongly encouraged given the technique's long history of 332 successful deployment. 334 2.2. Composite Link in Control Plane 336 A composite Link is advertised as a single logical interface between 337 two connected routers, which forms forwarding adjacency (FA) between 338 the routers. The FA is advertised as a TE-link in a link state IGP, 339 using either OSPF-TE or ISIS-TE. The IGP-TE advertised interface 340 parameters for the composite link can be preconfigured by the network 341 operator or be derived from its component links. Composite link 342 advertisement requirements are specified in 343 [I-D.ietf-rtgwg-cl-requirement]. 345 In IGP-TE, a composite link is advertised as a single TE link between 346 two connected routers. This is similar to a link bundle [RFC4201]. 347 Link bundle applies to a set of homogenous component links. 348 Composite link allows homogenous and non-homogenous component links. 349 Due to the similarity, and for backwards compatibility, extending 350 link bundling is viewed as both simple and as the best approach. 352 In order for a route computation engine to calculate a proper path 353 for a LSP, it is necessary for composite link to advertise the 354 summarized available bandwidth as well as the maximum bandwidth that 355 can be made available for single flow (or single LSP where no finer 356 flow identification is available). If a composite link contains some 357 non-homogeneous component links, the composite link also should 358 advertise the summarized bandwidth and the maximum bandwidth for 359 single flow per each homogeneous component link group. 361 Both LDP [RFC5036] and RSVP-TE [RFC3209] can be used to signal a LSP 362 over a composite link. LDP cannot be extended to support traffic 363 engineering capabilities [RFC3468]. 365 When an LSP is signaled using RSVP-TE, the LSP MUST be placed on the 366 component link that meets the LSP criteria indicated in the signaling 367 message. 369 When an LSP is signaled using LDP, the LSP MUST be placed on the 370 component link that meets the LSP criteria, if such a component link 371 is available. LDP does not support traffic engineering capabilities, 372 imposing restrictions on LDP use of Composite Link. See 373 Section 4.2.5 for further details. 375 A composite link may contain non-homogeneous component links. The 376 route computing engine may select one group of component links for a 377 LSP. The routing protocol MUST make this grouping available in the 378 TE-LSDB. The route computation used in RSVP-TE MUST be extended to 379 include only the capacity of groups within a composite link which 380 meet LSP criteria. The signaling protocol MUST be able to indicate 381 either the criteria, or which groups may be used. A composite link 382 MUST place the LSP on a component link or group which meets or 383 exceeds the LSP criteria. 385 Composite link capacity is aggregated capacity. LSP capacity MAY be 386 larger than individual component link capacity. Any aggregated LSP 387 can determine a bounds on the largest microflow that could be carried 388 and this constraint can be handled as follows. 390 1. If no information is available through signaling, management 391 plane, or configuration, the largest microflow is bound by one of 392 the following: 394 A. the largest single LSP if most traffic is RSVP-TE signaled 395 and further aggregated, 397 B. the largest pseudowire if most traffic is carrying pseudowire 398 payloads that are aggregated within RSVP-TE LSP, 400 C. or the largest source and sink interface if a large amount of 401 IP or LDP traffic is contained within the aggregate. 403 If a very large amount of traffic being aggregated is IP or LDP, 404 then the largest microflow is bound by the largest component link 405 on which IP traffic can arrive. For example, if an LSR is acting 406 as an LER and IP and LDP traffic is arrving on 10 Gb/s edge 407 interfaces, then no microflow larger than 10 Gb/s will be present 408 on the RSVP-TE LSP that aggregate traffic across the core, even 409 if the core interfaces are 100 Gb/s interfaces. 411 2. The prior conditions provide a bound on the largest microflow 412 when no signaling extensions indicate a bounds. If an LSP is 413 aggregating smaller LSP for which the largest expected microflow 414 carried by the smaller LSP is signaled, then the largest 415 microflow expected in the containing LSP (the aggregate) is the 416 maximum of the largest expected microflow for any contained LSP. 417 For example, RSVP-TE LSP may be large but aggregate traffic for 418 which the source or sink are all 1 Gb/s or smaller interfaces 419 (such as in mobile applications in which cell sites backhauls are 420 no larger than 1 Gb/s). If this information is carried in the 421 LSP originated at the cell sites, then further aggregates across 422 a core may make use of this information. 424 3. The IGP must provide the bounds on the largest microflow that a 425 composite link can accommodate, which is the maximum capacity on 426 a component link that can be made available by moving other 427 traffic. This information is needed by the ingress LER for path 428 determination. 430 4. A means to signal an LSP whose capacity is larger than individual 431 component link capacity is needed [I-D.ietf-rtgwg-cl-requirement] 432 and also signal the largest microflow expected to be contained in 433 the LSP. If a bounds on the largest microflow is not signaled 434 there is no means to determine if an LSP which is larger than any 435 component link can be subdivided into flows and therefore should 436 be accepted by admission control. 438 When a bidirectional LSP request is signaled over a composite link, 439 if the request indicates that the LSP must be placed on the same 440 component link, the routers of the composite link MUST place the LSP 441 traffic in both directions on a same component link. This is 442 particularly challenging for aggregated capacity which makes use of 443 the label stack for traffic distribution. The two requirements are 444 mutually exclusive for any one LSP. No one LSP may be both larger 445 than any individual component link and require symmetrical paths for 446 every flow. Both requirements can be accommodated by the same 447 composite link for different LSP, with any one LSP requiring no more 448 than one of these two features. 450 Individual component link may fail independently. Upon component 451 link failure, a composite link MUST support a minimally disruptive 452 local repair, preempting any LSP which can no longer be supported. 453 Available capacity in other component links MUST be used to carry 454 impacted traffic. The available bandwidth after failure MUST be 455 advertised immediately to avoid looped crankback. 457 When a composite link is not able to transport all flows, it preempts 458 some flows based upon local management configuration and informs the 459 control plane on these preempted flows. The composite link MUST 460 support soft preemption [RFC5712]. This action ensures the remaining 461 traffic is transported properly. FR#10 requires that the traffic be 462 restored. FR#12 requires that any change be minimally disruptive. 463 These two requirements are interpreted to include preemption among 464 the types of changes that must be minimally disruptive. 466 2.3. Composite Link in Data Plane 468 The data plane must first identify groups of flows. Flow 469 identification is covered in Section 2.1. Having identified groups 470 of flows the groups must be placed on individual component links. 471 This second step is called traffic distribution or traffic placement. 472 The two steps together are known as traffic balancing or load 473 balancing. 475 Traffic distribution may be determined by or constrained by control 476 plane or management plane. Traffic distribution may be changed due 477 to component link status change, subject to constraints imposed by 478 either the management plane or control plane. The distribution 479 function is local to the routers in which a composite link belongs to 480 and is not specified here. 482 When performing traffic placement, a composite link does not 483 differentiate multicast traffic vs. unicast traffic. 485 In order to maintain scalability, existing data plane forwarding 486 retains state associated with the top label only. The use of flow 487 group identification is in a second step in the forwarding process. 488 Data plane forwarding makes use of the top label to select a 489 composite link, or a group of components within a composite link or 490 for the case where an LSP is pinned (see [RFC4201]), a specific 491 component link. For those LSP for which the LSP selects only the 492 composite link or a group of components within a composite link, the 493 load balancing makes use of the flow group identification. 495 The most common traffic placement techniques uses the a flow group 496 identification as an index into a table. The table provides an 497 indirection. The number of bits of hash is constrained to keep table 498 size small. While this is not the best technique, it is the most 499 common. Better techniques exist but they are outside the scope of 500 this document and some are considered proprietary. 502 Requirements to limit frequency of load balancing can be adhered to 503 by keeping track of when a flow group was last moved and imposing a 504 minimum period before that flow group can be moved again. This is 505 straightforward for a table approach. For other approaches it may be 506 less straightforward but is acheivable. 508 3. Architecture Tradeoffs 510 Scalability and stability are critical considerations in protocol 511 design where protocols may be used in a large network such as today's 512 service provider networks. Composite Link is applicable to networks 513 which are large enough to require that traffic be split over multiple 514 paths. Scalability is a major consideration for networks that reach 515 a capacity large enough to require Composite Link. 517 Some of the requirements of Composite Link could potentially have a 518 negative impact on scalability. For example, Composite Link requires 519 additional information to be carried in situations where component 520 links differ in some significant way. 522 3.1. Scalability Motivations 524 In the interest of scalability information is aggregated in 525 situations where information about a large amount of network capacity 526 or a large amount of network demand provides is adequate to meet 527 requirements. Routing information is aggregated to reduce the amount 528 of information exchange related to routing and to simplify route 529 computation (see Section 3.2). 531 In an MPLS network large routing changes can occur when a single 532 fault occurs. For example, a single fault may impact a very large 533 number of LSP traversing a given link. As new LSP are signaled to 534 avoid the fault, resources are consumed elsewhere, and routing 535 protocol announcements must flood the resource changes. If 536 protection is in place, there is less urgency to converging quickly. 537 If multiple faults occur that are not covered by shared risk groups 538 (SRG), then some protection may fail, adding urgency to converging 539 quickly even where protection was deployed. 541 Reducing the amount of information allows the exchange of information 542 during a large routing change to be accomplished more quickly and 543 simplifies route computation. Simplifying route computation improves 544 convergence time after very significant network faults which cannot 545 be handled by preprovisioned or precomputed protection mechanisms. 546 Aggregating smaller LSP into larger LSP is a means to reduce path 547 computation load and reduce RSVP-TE signaling (see Section 3.3). 549 Neglecting scaling issues can result in performance issues, such as 550 slow convergence. Neglecting scaling in some cases can result in 551 networks which perform so poorly as to become unstable. 553 3.2. Reducing Routing Information and Exchange 555 Link bundling at the very least provides a means of aggregating 556 control plane information. Even where the all-ones component link 557 supported by link bundling is not used, the amount of control 558 information is reduced by the average number of component links in a 559 bundle. 561 Fully deaggregating link bundle information would negate this 562 benefit. If there is a need to deaggregate, such as to distinguish 563 between groups of links within specified ranges of delay, then no 564 more deaggregation than is necessary should be done. 566 For example, in supporting the requirement for heterogeneous 567 component links, it makes little sense to fully deaggregate link 568 bundles when adding support for groups of component links with common 569 attributes within a link bundle can maintain most of the benefit of 570 aggregation while adequately supporting the requirement to support 571 heterogeneous component links. 573 Routing information exchange is also reduced by making sensible 574 choices regarding the amount of change to link parameters that 575 require link readvertisement. For example, if delay measurements 576 include queuing delay, then a much more coarse granularity of delay 577 measurement would be called for than if the delay does not include 578 queuing and is dominated by geographic delay (speed of light delay). 580 3.3. Reducing Signaling Load 582 Aggregating traffic into very large hierarchical LSP in the core very 583 substantially reduces the number of LSP that need to be signaled and 584 the number of path computations any given LSR will be required to 585 perform when a major network fault occurs. 587 In the extreme, applying MPLS to a very large network without 588 hierarchy could exceed the 20 bit label space. For example, in a 589 network with 4,000 nodes, with 2,000 on either side of a cutset, 590 would have 4,000,000 LSP crossing the cutset. Even in a degree four 591 cutset, an uneven distribution of LSP across the cutset, or the loss 592 of one link would result in a need to exceed the size of the label 593 space. Among provider networks, 4,000 access nodes is not at all 594 large. 596 In less extreme cases, having each node terminate hundreds of LSP to 597 achieve a full mesh creates a very large computational load. The 598 time complexity of one CSPF computation is order(N log N), where L is 599 proportional to N, and N and L are the number of nodes and number of 600 links, respectively. If each node must perform order(N) computations 601 when a fault occurs, then the computational load increases as 602 order(N^2 log N) as the number of nodes increases. In practice at 603 the time of writing, this imposes a limit of a few hundred nodes in a 604 full mesh of MPLS LSP before the computational load is sufficient to 605 result in unacceptable convergence times. 607 Two solutions are applied to reduce the amount of RSVP-TE signaling. 608 Both involve subdividing the MPLS domain into a core and a set of 609 regions. 611 3.3.1. Reducing Signaling Load using LDP 613 LDP can be used for edge-to-edge LSP, using RSVP-TE to carry the LDP 614 intra-core traffic and also optionally also using RSVP-TE to carry 615 the LDP intra-region traffic within each region. LDP does not 616 support traffic engineering, but does support multipoint-to-point 617 (MPTP) LSP, which require less signaling than edge-to-edge RSVP-TE 618 point-to-point (PTP) LSP. A drawback of this approach is the 619 inability to use RSVP-TE protection (FRR or GMPLS protection) against 620 failure of the border LSR sitting at a core/region boundary. 622 3.3.2. Reducing Signaling Load using Hierarchy 624 When the number of nodes grows too large, the amount of RSVP-TE 625 signaling can be reduced using the MPLS PSC hierarchy [RFC4206]. A 626 core within the hierarchy can divide the topology into M regions of 627 on average N/M nodes. Within a region the computational load is 628 reduced by more than M^2. Within the core, the computational load 629 generally becomes quite small since M is usually a fairly small 630 number (a few tens of regions) and each region is generally attached 631 to the core in typically only two or three places on average. 633 Using hierarchy improves scaling but has two consequences. First, 634 hierarchy effectively forces the use of platform label space. When a 635 containing LSP is rerouted, the labels assigned to the contained LSP 636 cannot be changed but may arrive on a different interface. Second, 637 hierarchy results in much larger LSP. These LSP today are larger 638 than any single component link and therefore force the use of the 639 all-ones component in link bundles. 641 3.3.3. Using Both LDP and RSVP-TE Hierarchy 643 It is also possible to use both LDP and RSVP-TE hierarchy. MPLS 644 networks with a very large number of nodes may benefit from the use 645 of both LDP and RSVP-TE hierarchy. The two techniques are certainly 646 not mutually exclusive. 648 3.4. Reducing Forwarding State 650 Both LDP and MPLS hierarchy have the benefit of reducing the amount 651 of forwarding state. Using the example from Section 3.3, and using 652 MPLS hierarchy, the worst case generally occurs at borders with the 653 core. 655 For example, consider a network with approximately 1,000 nodes 656 divided into 10 regions. At the edges, each node requires 1,000 LSP 657 to other edge nodes. The edge nodes also require 100 intra-region 658 LSP. Within the core, if the core has only 3 attachments to each 659 region the core LSR have less than 100 intra-core LSP. At the border 660 cutset between the core and a given region, in this example there are 661 100 edge nodes with inter-region LSP crossing that cutset, destined 662 to 900 other edge nodes. That yields forwarding state for on the 663 order of 90,000 LSP at the border cutset. These same routers need 664 only reroute well under 200 LSP when a multiple fault occurs, as long 665 as only links are affected and a border LSR does not go down. 667 In the core, the forwarding state is greatly reduced. If inter- 668 region LSP have different characteristics, it makes sense to make use 669 of aggregates with different characteristics. Rather than exchange 670 information about every inter-region LSP within the intra-core LSP it 671 makes more sense to use multiple intra-core LSP between pairs of core 672 nodes, each aggregating sets of inter-region LSP with common 673 characteristics or common requirements. 675 3.5. Avoiding Route Oscillation 677 Networks can become unstable when a feedback loop exists such that 678 moving traffic to a link causes a metric such as delay to increase, 679 which then causes traffic to move elsewhere. For example, the 680 original ARPANET routing used a delay based cost metric and proved 681 prone to route oscillations [DBP]. 683 Delay may be used as a constraint in routing for high priority 684 traffic, where the movement of traffic cannot impact the delay. The 685 safest way to measure delay is to make measurements based on traffic 686 which is prioritized such that it is queued ahead of the traffic 687 which will be affected. This is a reasonable measure of delay for 688 high priority traffic for which constraints have been set which allow 689 this type of traffic to consume only a fraction of link capacities 690 with the remaining capacity available to lower priority traffic. 692 Any measurement of jitter (delay variation) that is used in route 693 decision is likely to cause oscillation. Jitter that is caused by 694 queuing effects and cannot be measured using a very high priority 695 measurement traffic flow. 697 It may be possible to find links with constrained queuing delay or 698 jitter using a theoretical maximum or a probability based bound on 699 queuing delay or jitter at a given priority based on the types and 700 amounts of traffic accepted and combining that theoretical limit with 701 a measured delay at very high priority. 703 Instability can occur due to poor performance and interaction with 704 protocol timers. In this way a computational scaling problem can 705 become a stability problem when a network becomes sufficiently large. 706 For this reason, [I-D.ietf-rtgwg-cl-requirement] has a number of 707 requirements focusing on minimally impacting scalability. 709 4. New Challenges 711 New technical challenges are posed by [I-D.ietf-rtgwg-cl-requirement] 712 in both the control plane and data plane. 714 Among the more difficult challenges are the following. 716 1. requirements related delay or jitter (see Section 4.1.1), 718 2. the combination of ingress control over LSP placement and 719 retaining an ability to move traffic as demands dictate can pose 720 challenges and such requirements can even be conflicting (see 721 target="sect.local-control" />), 723 3. path symmetry requires extensions and is particularly challenging 724 for very large LSP (see Section 4.1.3), 726 4. accommodating a very wide range of requirements among contained 727 LSP can lead to inefficiency if the most stringent requirements 728 are reflected in aggregates, or reduce scalability if a large 729 number of aggregates are used to provide a too fine a reflection 730 of the requirements in the contained LSP (see Section 4.1.4), 732 5. backwards compatibility is somewhat limited due to the need to 733 accommodate legacy multipath interfaces which provide too little 734 information regarding their configured default behavior, and 735 legacy LSP which provide too little information regarding their 736 requirements (see Section 4.1.5), 738 6. data plane challenges include those of accommodating very large 739 LSP, large microflows, traffic ordering constraints imposed by a 740 subsent of LSP, and accounting for IP and LDP traffic (see 741 Section 4.2). 743 4.1. Control Plane Challenges 745 Some of the control plane requirements are particularly challenging. 746 Handling large flows which aggregate smaller flows must be 747 accomplished with minimal impact on scalability. Potentially 748 conflicting are requirements for jitter and requirements for 749 stability. Potentially conflicting are the requirements for ingress 750 control of a large number of parameters, and the requirements for 751 local control needed to achieve traffic balance across a composite 752 link. These challenges and potential solutions are discussed in the 753 following sections. 755 4.1.1. Delay and Jitter Sensitive Routing 757 Delay and jitter sensitive routing are called for in 758 [I-D.ietf-rtgwg-cl-requirement] in requirements FR#2, FR#7, FR#8, 759 FR#9, FR#15, FR#16, FR#17, FR#18. Requirement FR#17 is particularly 760 problematic, calling for constraints on jitter. 762 A tradeoff exists between scaling benefits of aggregating 763 information, and potential benefits of using a finer granularity in 764 delay reporting. To maintain the scaling benefit, measured link 765 delay for any given composite link SHOULD be aggregated into a small 766 number of delay ranges. IGP-TE extensions MUST be provided which 767 advertise the available capacities for each of the selected ranges. 769 For path selection of delay sensitive LSP, the ingress SHOULD bias 770 link metrics based on available capacity and select a low cost path 771 which meets LSP total path delay criteria. To communicate the 772 requirements of an LSP, the ERO MUST be extended to indicate the per 773 link constraints. To communicate the type of resource used, the RRO 774 SHOULD be extended to carry an identification of the group that is 775 used to carry the LSP at each link bundle hop. 777 4.1.2. Local Control of Traffic Distribution 779 Many requirements in [I-D.ietf-rtgwg-cl-requirement] suggest that a 780 node immediately adjacent to a component link should have a high 781 degree of control over how traffic is distributed, as long as network 782 performance objectives are met. Particularly relevant are FR#18 and 783 FR#19. 785 The requirements to allow local control are potentially in conflict 786 with requirement FR#21 which gives full control of component link 787 select to the LSP ingress. While supporting this capability is 788 mandatory, use of this feature is optional per LSP. 790 A given network deployment will have to consider this pair of 791 conflicting requirements and make appropriate use of local control of 792 traffic placement and ingress control of traffic placement to best 793 meet network requirements. 795 4.1.3. Path Symmetry Requirements 797 Requirement FR#21 in [I-D.ietf-rtgwg-cl-requirement] includes a 798 provision to bind both directions of a bidirectional LSP to the same 799 component. This is easily achieved if the LSP is directly signaled 800 across a composite link. This is not as easily achieved if a set of 801 LSP with this requirement are signaled over a large hierarchical LSP 802 which is in turn carried over a composite link. The basis for load 803 distribution in such as case is the label stack. The labels in 804 either direction are completely independent. 806 This could be accommodated if the ingress, egress, and all midpoints 807 of the hierarchical LSP make use of an entropy label in the 808 distribution, and use only that entropy label. A solution for this 809 problem may add complexity with very little benefit. There is little 810 or no true benefit of using symmetrical paths rather than component 811 links of identical characteristics. 813 Traffic symmetry and large LSP capacity are a second pair of 814 conflicting requirements. Any given LSP can meet one of these two 815 requirements but not both. A given network deployment will have to 816 make appropriate use of each of these features to best meet network 817 requirements. 819 4.1.4. Requirements for Contained LSP 821 [I-D.ietf-rtgwg-cl-requirement] calls for new LSP constraints. These 822 constraints include frequency of load balancing rearrangement, delay 823 and jitter, packet ordering constraints, and path symmetry. 825 When LSP are contained within hierarchical LSP, there is no signaling 826 available at midpoint LSR which identifies the contained LSP let 827 alone providing the set of requirements unique to each contained LSP. 828 Defining extensions to provide this information would severely impact 829 scalability and defeat the purpose of aggregating control information 830 and forwarding information into hierarchical LSP. For the same 831 scalability reasons, not aggregating at all is not a viable option 832 for large networks where scalability and stability problems may occur 833 as a result. 835 As pointed out in Section 4.1.3, the benefits of supporting symmetric 836 paths among LSP contained within hierarchical LSP may not be 837 sufficient to justify the complexity of supporting this capability. 839 A scalable solution which accommodates multiple sets of LSP between 840 given pairs of LSR is to provide multiple hierarchical LSP for each 841 given pair of LSR, each hierarchical LSP aggregating LSP with common 842 requirements and a common pair of endpoints. This is a network 843 design technique available to the network operator rather than a 844 protocol extension. This technique can accommodate multiple sets of 845 delay and jitter parameters, multiple sets of frequency of load 846 balancing parameters, multiple sets of packet ordering constraints, 847 etc. 849 4.1.5. Retaining Backwards Compatibility 851 Backwards compatibility and support for incremental deployment 852 requires considering the impact of legacy LSR in the role of LSP 853 ingress, and considering the impact of legacy LSR advertising 854 ordinary links, advertising Ethernet LAG as ordinary links, and 855 advertising link bundles. 857 Legacy LSR in the role of LSP ingress cannot signal requirements 858 which are not supported by their control plane software. The 859 additional capabilities supported by other LSR has no impact on these 860 LSR. These LSR however, being unaware of extensions, may try to make 861 use of scarce resources which support specific requirements such as 862 low delay. To a limited extent it may be possible for a network 863 operator to avoid this issue using existing mechanisms such as link 864 administrative attributes and attribute affinities [RFC3209]. 866 Legacy LSR advertising ordinary links will not advertise attributes 867 needed by some LSP. For example, there is no way to determine the 868 delay or jitter characteristics of such a link. Legacy LSR 869 advertising Ethernet LAG pose additional problems. There is no way 870 to determine that packet ordering constraints would be violated for 871 LSP with strict packet ordering constraints, or that frequency of 872 load balancing rearrangement constraints might be violated. 874 Legacy LSR advertising link bundles have no way to advertise the 875 configured default behavior of the link bundle. Some link bundles 876 may be configured to place each LSP on a single component link and 877 therefore may not be able to accommodate an LSP which requires 878 bandwidth in excess of the size of a component link. Some link 879 bundles may be configured to spread all LSP over the all-ones 880 component. For LSR using the all-ones component link, there is no 881 documented procedure for correctly setting the "Maximum LSP 882 Bandwidth". There is currently no way to indicate the largest 883 microflow that could be supported by a link bundle using the all-ones 884 component link. 886 Having received the RRO, it is possible for an ingress to look for 887 the all-ones component to identify such link bundles after having 888 signaled at least one LSP. Whether any LSR collects this information 889 on legacy LSR and makes use of it to set defaults, is an 890 implementation choice. 892 4.2. Data Plane Challenges 894 Flow identification is briefly discussed in Section 2.1. Traffic 895 distribution is briefly discussed in Section 2.3. This section 896 discusses issues specific to particular requirements specified in 898 [I-D.ietf-rtgwg-cl-requirement]. 900 4.2.1. Very Large LSP 902 Very large LSP may exceed the capacity of any single component of a 903 composite link. In some cases contained LSP may exceed the capacity 904 of any single component. These LSP may the use of the equivalent of 905 the all-ones component of a link bundle, or may use a subset of 906 components which meet the LSP requirements. 908 Very large LSP can be accommodated as long as they can be subdivided 909 (see Section 4.2.2). A very large LSP cannot have a requirement for 910 symetric paths unless complex protocol extensions are proposed (see 911 Section 2.2 and Section 4.1.3). 913 4.2.2. Very Large Microflows 915 Within a very large LSP there may be very large microflows. A very 916 large microflow is a very large flows which cannot be further 917 subdivided. Flows which cannot be subdivided must be no larger that 918 the capacity of any single component. 920 Current signaling provides no way to specify the largest microflow 921 that a can be supported on a given link bundle in routing 922 advertisements. Extensions which address this are discussed in 923 Section 6.4. Absent extensions of this type, traffic containing 924 microflows that are too large for a given composite link may be 925 present. There is no data plane solution for this problem that would 926 not require reordering traffic at the composite link egress. 928 Some techniques are susceptible to statistical collisions where an 929 algorithm to distribute traffic is unable to disambiguate traffic 930 among two or more very large microflow where their sum is in excess 931 of the capacity of any single component. Hash based algorithms which 932 use too small a hash space are particularly susceptible and require a 933 change in hash seed in the event that this were to occur. A change 934 in hash seed is highly disruptive, causing traffic reordering among 935 all traffic flows over which the hash function is applied. 937 4.2.3. Traffic Ordering Constraints 939 Some LSP have strict traffic ordering constraints. Most notable 940 among these are MPLS-TP LSP. In the absence of aggregation into 941 hierarchical LSP, those LSP with strict traffic ordering constraints 942 can be placed on individual component links if there is a means of 943 identifying which LSP have such a constraint. If LSP with strict 944 traffic ordering constraints are aggregated in hierarchical LSP, the 945 hierarchical LSP capacity may exceed the capacity of any single 946 component link. In such a case the load balancing for the containing 947 may be constrained to look only at the top label and the first 948 contained label. This and related issues are discussed further in 949 Section 6.4. 951 4.2.4. Accounting for IP and LDP Traffic 953 Networks which carry RSVP-TE signaled MPLS traffic generally carry 954 low volumes of native IP traffic, often only carrying control traffic 955 as native IP. There is no architectural guarantee of this, it is 956 just how network operators have made use of the protocols. 958 [I-D.ietf-rtgwg-cl-requirement] requires that native IP and native 959 LDP be accommodated. In some networks, a subset of services may be 960 carried as native IP or carried as native LDP. Today this may be 961 accommodated by the network operator estimating the contribution of 962 IP and LDP and configuring a lower set of available bandwidth figures 963 on the RSVP-TE advertisements. 965 The only improvement that Composite Link can offer is that of 966 measuring the IP and LDP traffic levels and automatically reducing 967 the available bandwidth figures on the RSVP-TE advertisements. The 968 measurements would have to be significantly filtered. This is 969 similar to a feature in existing LSR, commonly known as 970 "autobandwidth" with a key difference. In the "autobandwidth" 971 feature, the bandwidth request of an RSVP-TE signaled LSP is adjusted 972 in response to traffic measurements. In this case the IP or LDP 973 traffic measurements are used to reduce the link bandwidth directly, 974 without first encapsulating in an RSVP-TE LSP. 976 This may be a subtle and perhaps even a meaningless distinction if 977 Composite Link is used to form a Sub-Path Maintenance Element (SPME). 978 A SPME is in practice essentially an unsignaled single hop LSP with 979 PHP enabled [RFC5921]. A Composite Link SPME looks very much like 980 classic multipath, where there is no signaling, only management plane 981 configuration creating the multipath entity (of which Ethernet Link 982 Aggregation is a subset). 984 4.2.5. IP and LDP Limitations 986 IP does not offer traffic engineering. LDP cannot be extended to 987 offer traffic engineering [RFC3468]. Therefore there is no traffic 988 engineered fallback to an alternate path for IP and LDP traffic if 989 resources are not adequate for the IP and/or LDP traffic alone on a 990 given link in the primary path. The only option for IP and LDP would 991 be to declare the link down. Declaring a link down due to resource 992 exhaustion would reduce traffic to zero and eliminate the resource 993 exhaustion. This would cause oscillations and is therefore not a 994 viable solution. 996 Congestion caused by IP or LDP traffic loads is a pathologic case 997 that can occur if IP and/or LDP are carried natively and there is a 998 high volume of IP or LDP traffic. This situation can be avoided by 999 carrying IP and LDP within RSVP-TE LSP. 1001 It is also not possible to route LDP traffic differently for 1002 different FEC. LDP traffic engineering is specifically disallowed by 1003 [RFC3468]. It may be possible to support multi-topology IGP 1004 extensions to accommodate more than one set of criteria. If so, the 1005 additional IGP could be bound to the forwarding criteria, and the LDP 1006 FEC bound to a specific IGP instance, inheriting the forwarding 1007 criteria. Alternately, one IGP instance can be used and the LDP SPF 1008 can make use of the constraints, such as delay and jitter, for a 1009 given LDP FEC. [Note: WG needs to discuss this and decide first 1010 whether to solve this at all and then if so, how.] 1012 5. Existing Mechanisms 1014 In MPLS the one mechanisms which support explicit signaling of 1015 multiple parallel links is Link Bundling [RFC4201]. The set of 1016 techniques known as "classis multipath" support no explicit 1017 signaling, except in two cases. In Ethernet Link Aggregation the 1018 Link Aggregation Control Protocol (LACP) coordinates the addition or 1019 removal of members from an Ethernet Link Aggregation Group (LAG). 1020 The use of the "all-ones" component of a link bundle indicates use of 1021 classis multipath, however the ability to determine if a link bundle 1022 makes use of classis multipath is not yet supported. 1024 5.1. Link Bundling 1026 Link bundling supports advertisement of a set of homogenous links as 1027 a single route advertisement. Link bundling supports placement of an 1028 LSP on any single component link, or supports placement of an LSP on 1029 the all-ones component link. Not all link bundling implementations 1030 support the all-ones component link. There is no way for an ingress 1031 LSR to tell which potential midpoint LSR support this feature and use 1032 it by default and which do not. Based on [RFC4201] it is unclear how 1033 to advertise a link bundle for which the all-ones component link is 1034 available and used by default. Common practice is to violate the 1035 specification and set the Maximum LSP Bandwidth to the Available 1036 Bandwidth. There is no means to determine the largest microflow that 1037 could be supported by a link bundle that is using the all-ones 1038 component link. 1040 [RFC6107] extends the procedures for hierarchical LSP but also 1041 extends link bundles. An LSP can be explicitly signaled to indicate 1042 that it is an LSP to be used as a component of a link bundle. Prior 1043 to that the common practice was to simply not advertise the component 1044 link LSP into the IGP, since only the ingress and egress of the link 1045 bundle needed to be aware of their existence, which they would be 1046 aware of due to the RSVP-TE signaling used in setting up the 1047 component LSP. 1049 While link bundling can be the basis for composite links, a 1050 significant number of small extension needs to be added. 1052 1. To support link bundles of heterogeneous links, a means of 1053 advertising the capacity available within a group of homogeneous 1054 needs to be provided. 1056 2. Attributes need to be defined to support the following parameters 1057 for the link bundle or for a group of homogeneous links. 1059 A. delay range 1061 B. jitter (delay variation) range 1063 C. group metric 1065 D. all-ones component capable 1067 E. capable of dynamically balancing load 1069 F. largest supportable microflow 1071 G. abilities to support strict packet ordering requirements 1072 within contained LSP 1074 3. For each of the prior extended attributes, the constraint based 1075 routing path selection needs to be extended to reflect new 1076 constraints based on the extended attributes. 1078 4. For each of the prior extended attributes, LSP admission control 1079 needs to be extended to reflect new constraints based on the 1080 extended attributes. 1082 5. Dynamic load balance must be provided for flows within a given 1083 set of links with common attributes such that NPO are not 1084 violated including frequency of load balance adjustment for any 1085 given flow. 1087 5.2. Classic Multipath 1089 Classic multipath is defined in [I-D.symmvo-rtgwg-cl-use-cases]. 1091 Classic multipath refers to the most common current practice in 1092 implementation and deployment of multipath. The most common current 1093 practice makes use of a hash on the MPLS label stack and if IPv4 or 1094 IPv6 are indicated under the label stack, makes use of the IP source 1095 and destination addresses [RFC4385] [RFC4928]. 1097 Classic multipath provides a highly scalable means of load balancing. 1098 Adaptive multipath has proven value in assuring an even loading on 1099 component link and an ability to adapt to change in offerred load 1100 that occurs over periods of hundreds of milliseconds or more. 1101 Classic multipath scalability is due to the ability to effectively 1102 work with an extremely large number of flows (IP host pairs) using 1103 relatively little resources (a data structure accessed using a hash 1104 result as a key or using ranges of hash results). 1106 Classic multipath meets a small subset of Composite Link 1107 requirements. Due to scalability of the approach, classic multipath 1108 seems to be an excellent candidate for extension to meet the full set 1109 of Composite Link forwarding requirements. 1111 Additional detail can be found in [I-D.symmvo-rtgwg-cl-use-cases]. 1113 6. Mechanisms Proposed in Other Documents 1115 A number of documents which at the time of writing are works in 1116 progress address parts of the requirements of Composite Link, or 1117 assist in making some of the goals achievable. 1119 6.1. Loss and Delay Measurement 1121 Procedures for measuring loss and delay are provided in [RFC6374]. 1122 These are OAM based measurements. This work could be the basis of 1123 delay measurements and delay variation measurement used for metrics 1124 called for in [I-D.ietf-rtgwg-cl-requirement]. 1126 Currently there are two additional Internet-Drafts that address delay 1127 and delay variation metrics. 1129 draft-wang-ccamp-latency-te-metric 1130 [I-D.wang-ccamp-latency-te-metric] is designed specifically to 1131 meet this requirement. OSPF-TE and ISIS-TE extensions are 1132 defined to indicate link delay and delay variance. The RSVP-TE 1133 ERO is extended to include service level requirements. A latency 1134 accumulation object is defined to provide a means of verification 1135 of the service level requirements. This draft is intended to 1136 proceed in the CCAMP WG. It is currently and individual 1137 submission. The 03 version of this draft expired in September 1138 2012. 1140 draft-giacalone-ospf-te-express-path 1141 This document proposes to extend OSPF-TE only. Extensions 1142 support delay, delay variance, loss, residual bandwidth, and 1143 available bandwidth. No extensions to RSVP-TE are proposed. 1144 This draft is intended to proceed in the CCAMP WG. It is 1145 currently and individual submission. The 02 version will expire 1146 in March 2012. 1148 A possible course of action may be to combine these two drafts. The 1149 delay variance, loss, residual bandwidth, and available bandwidth 1150 extensions are particular prone to network instability. The question 1151 as to whether queuing delay and delay variation should be considered, 1152 and if so for which diffserv Per-Hop Service Class (PSC) is not 1153 addressed. 1155 Note to co-authors: The ccamp-latency-te-metric draft refers to 1156 [I-D.ietf-rtgwg-cl-requirement] and is well matched to those 1157 requirements, including stability. The ospf-te-express-path draft 1158 refers to the "Alto Protocol" (draft-ietf-alto-protocol) and 1159 therefore may not be intended for RSVP-TE use. The authors of the 1160 two drafts may be able to resolve this. It may be best to drop ospf- 1161 te-express-path from this framework document. 1163 6.2. Link Bundle Extensions 1165 A set of link bundling extensions are defined in 1166 [I-D.ietf-mpls-explicit-resource-control-bundle]. This document 1167 provides extensions to the ERO and RRO to explicitly control the 1168 labels and resources within a bundle used by an LSP. 1170 The extensions in this document could be further extended to support 1171 indicating a group of component links in the ERO or RRO, where the 1172 group is given an interface identification like the bundle itself. 1173 The extensions could also be further extended to support 1174 specification of the all-ones component link in the ERO or RRO. 1176 [I-D.ietf-mpls-explicit-resource-control-bundle] does not provide a 1177 means to advertise the link bundle components. It is not certain how 1178 the ingress LSR would determine the set of link bundle component 1179 links available for a given link bundle. 1181 [I-D.ospf-cc-stlv] provides a baseline draft for extending link 1182 bundling to advertise components. A new component TVL (C-TLV) is 1183 proposed, which must reference a Composite Link Link TLV. 1184 [I-D.ospf-cc-stlv] is intended for the OSPF WG and submitted for the 1185 "Experimental" track. The 00 version expired in February 2012. 1187 6.3. Fat PW and Entropy Labels 1189 Two documents provide a means to add entropy for the purpose of 1190 improving load balance. MPLS encapsulation can bury information that 1191 is needed to identify microflows. These two documents allow a 1192 pseudowire ingress and LSP ingress respectively to add a label solely 1193 for the purpose of providing a finer granularity of microflow groups. 1195 [RFC6391] allows pseudowires which carry a large volume of traffic, 1196 where microflows can be identified to be load balanced across 1197 multiple members of an Ethernet LAG or an MPLS link bundle. This is 1198 accomplished by adding a flow label below the pseudowire label in the 1199 MPLS label stack. For this to be effective the link bundle load 1200 balance must make use of the label stack up to and including this 1201 flow label. 1203 [I-D.ietf-mpls-entropy-label] provides a means for a LER to put an 1204 additional label known as an entropy label on the MPLS label stack. 1205 As defined, only the LER can add the entropy label. 1207 Core LSR acting as LER for aggregated LSP can add entropy labels 1208 based on deep packet inspection and place an entropy label indicator 1209 (ELI) and entropy label (EL) just below the label being acted on. 1210 This would be helpful in situations where the label stack depth to 1211 which load distribution can operate is limited by implementation or 1212 is limited for other reasons such as carrying both MPLS-TP and MPLS 1213 with entropy labels within the same hierarchical LSP. 1215 6.4. Multipath Extensions 1217 The multipath extensions drafts address one aspect of Composite Link. 1218 These drafts deal with the issue of accommodating LSP which have 1219 strict packet ordering constraints in a network containing multipath. 1220 MPLS-TP has become the one important instance of LSP with strict 1221 packet ordering constraints and has driven this work. 1223 [I-D.villamizar-mpls-tp-multipath] outlines requirements and gives a 1224 number of options for dealing with the apparent incompatibility of 1225 MPLS-TP and multipath. A preferred option is described. 1227 [I-D.villamizar-mpls-tp-multipath-te-extn] provides protocol 1228 extensions needed to implement the preferred option described in 1229 [I-D.villamizar-mpls-tp-multipath]. 1231 Other issues pertaining to multipath are also addressed. Means to 1232 advertise the largest microflow supportable are defined. Means to 1233 indicate the largest expected microflow within an LSP are defined. 1234 Issues related to hierarchy are addressed. 1236 7. Required Protocol Extensions and Mechanisms 1238 Prior sections have reviewed key characteristics, architecture 1239 tradeoffs, new challenges, existing mechanisms, and relevant 1240 mechanisms proposed in existing new documents. 1242 This section first summarizes and groups requirements. A set of 1243 documents coverage groupings are proposed with existing works-in- 1244 progress noted where applicable. The set of extensions are then 1245 grouped by protocol affected as a convenience to implementors. 1247 7.1. Brief Review of Requirements 1249 The following list provides a categorization of requirements 1250 specified in [I-D.ietf-rtgwg-cl-requirement] along with a short 1251 phrase indication what topic the requirement covers. 1253 routing information aggregation 1254 FR#1 (routing summarization), FR#20 (composite link may be a 1255 component of another composite link) 1257 restoration speed 1258 FR#2 (restoration speed meeting NPO), FR#12 (minimally disruptive 1259 load rebalance), DR#6 (fast convergence), DR#7 (fast worst case 1260 failure convergence) 1262 load distribution, stability, minimal disruption 1263 FR#3 (automatic load distribution), FR#5 (must not oscillate), 1264 FR#11 (dynamic placement of flows), FR#12 (minimally disruptive 1265 load rebalance), FR#13 (bounded rearrangement frequency), FR#18 1266 (flow placement must satisfy NPO), FR#19 (flow identification 1267 finer than per top level LSP), MR#6 (operator initiated flow 1268 rebalance) 1270 backward compatibility and migration 1271 FR#4 (smooth incremental deployment), FR#6 (management and 1272 diagnostics must continue to function), DR#1 (extend existing 1273 protocols), DR#2 (extend LDP, no LDP TE) 1275 delay and delay variation 1276 FR#7 (expose lower layer measured delay), FR#8 (precision of 1277 latency reporting), FR#9 (limit latency on per LSP basis), FR#15 1278 (minimum delay path), FR#16 (bounded delay path), FR#17 (bounded 1279 jitter path) 1281 admission control, preemption, traffic engineering 1282 FR#10 (admission control, preemption), FR#14 (packet ordering), 1283 FR#21 (ingress specification of path), FR#22 (path symmetry), 1284 DR#3 (IP and LDP traffic), MR#3 (management specification of 1285 path) 1287 single vs multiple domain 1288 DR#4 (IGP extensions allowed within single domain), DR#5 (IGP 1289 extensions disallowed in multiple domain case) 1291 general network management 1292 MR#1 (polling, configuration, and notification), MR#2 (activation 1293 and de-activation) 1295 path determination, connectivity verification 1296 MR#4 (path trace), MR#5 (connectivity verification) 1298 The above list is not intended as a substitute for 1299 [I-D.ietf-rtgwg-cl-requirement], but rather as a concise grouping and 1300 reminder or requirements to serve as a means of more easily 1301 determining requirements coverage of a set of protocol documents. 1303 7.2. Required Document Coverage 1305 The primary areas where additional protocol extensions and mechanisms 1306 are required include the topics described in the following 1307 subsections. 1309 There are candidate documents for a subset of the topics below. This 1310 grouping of topics does not require that each topic be addressed by a 1311 separate document. In some cases, a document may cover multiple 1312 topics, or a specific topic may be addressed as applicable in 1313 multiple documents. 1315 7.2.1. Component Link Grouping 1317 An extension to link bundling is needed to specify a group of 1318 components with common attributes. This can be a TLV defined within 1319 the link bundle that carries the same encapsulations as the link 1320 bundle. Two interface indices would be needed for each group. 1322 a. An index is needed that if included in an ERO would indicate the 1323 need to place the LSP on any one component within the group. 1325 b. A second index is needed that if included in an ERO would 1326 indicate the need to balance flows within the LSP across all 1327 components of the group. This is equivalent to the "all-ones" 1328 component for the entire bundle. 1330 [I-D.ospf-cc-stlv] can be extended to include multipath treatment 1331 capabilities. An ISIS solution is also needed. An extension of 1332 RSVP-TE signaling is needed to indicate multipath treatment 1333 preferences. 1335 If a component group is allowed to support all of the parameters of a 1336 link bundle, then a group TE metric would be accommodated. This can 1337 be supported with the component TLV (C-TLV) defined in 1338 [I-D.ospf-cc-stlv]. 1340 The primary focus of this document, among the sets of requirements 1341 listed in Section 7.1 is the "routing information aggregation" set of 1342 requirements. The "restoration speed", "backward compatibility and 1343 migration", and "general network management" requirements must also 1344 be considered. 1346 7.2.2. Delay and Jitter Extensions 1348 A extension is needed in the IGP-TE advertisement to support delay 1349 and delay variation for links, link bundles, and forwarding 1350 adjacencies. Whatever mechanism is described must take precautions 1351 that insure that route oscillations cannot occur. 1352 [I-D.wang-ccamp-latency-te-metric] may be a good starting point. 1354 The primary focus of this document, among the sets of requirements 1355 listed in Section 7.1 is the "delay and delay variation" set of 1356 requirements. The "restoration speed", "backward compatibility and 1357 migration", and "general network management" requirements must also 1358 be considered. 1360 7.2.3. Path Selection and Admission Control 1362 Path selection and admission control changes must be documented in 1363 each document that proposes a protocol extension that advertises a 1364 new capability or parameter that must be supported by changes in path 1365 selection and admission control. 1367 The primary focus of this document, among the sets of requirements 1368 listed in Section 7.1 are the "load distribution, stability, minimal 1369 disruption" and "admission control, preemption, traffic engineering" 1370 sets of requirements. The "restoration speed" and "path 1371 determination, connectivity verification" requirements must also be 1372 considered. The "backward compatibility and migration", and "general 1373 network management" requirements must also be considered. 1375 7.2.4. Dynamic Multipath Balance 1377 FR#11 explicitly calls for dynamic load balancing similar to existing 1378 adaptive multipath. In implementations where flow identification 1379 uses a coarse granularity, the adjustments would have to be equally 1380 coarse, in the worst case moving entire LSP. The impact of flow 1381 identification granularity and potential adaptive multipath 1382 approaches may need to be documented in greater detail than provided 1383 here. 1385 The primary focus of this document, among the sets of requirements 1386 listed in Section 7.1 are the "restoration speed" and the "load 1387 distribution, stability, minimal disruption" sets of requirements. 1388 The "path determination, connectivity verification" requirements must 1389 also be considered. The "backward compatibility and migration", and 1390 "general network management" requirements must also be considered. 1392 7.2.5. Frequency of Load Balance 1394 IGP-TE and RSVP-TE extensions are needed to support frequency of load 1395 balancing rearrangement called for in FR#13, and FR#15-FR#17. 1396 Constraints are not defined in RSVP-TE, but could be modeled after 1397 administrative attribute affinities in RFC3209 and elsewhere. 1399 The primary focus of this document, among the sets of requirements 1400 listed in Section 7.1 is the "load distribution, stability, minimal 1401 disruption" set of requirements. The "path determination, 1402 connectivity verification" must also be considered. The "backward 1403 compatibility and migration" and "general network management" 1404 requirements must also be considered. 1406 7.2.6. Inter-Layer Communication 1408 Lower layer to upper layer communication called for in FR#7 and 1409 FR#20. This is addressed for a subset of parameters related to 1410 packet ordering in [I-D.villamizar-mpls-tp-multipath] where layers 1411 are MPLS. Remaining parameters, specifically delay and delay 1412 variation, need to be addressed. Passing information from a lower 1413 non-MPLS layer to an MPLS layer needs to be addressed, though this 1414 may largely be generic advice encouraging a coupling of MPLS to lower 1415 layer management plane or control plane interfaces. This topic can 1416 be addressed in each document proposing a protocol extension, where 1417 applicable. 1419 The primary focus of this document, among the sets of requirements 1420 listed in Section 7.1 is the "restoration speed" set of requirements. 1421 The "backward compatibility and migration" and "general network 1422 management" requirements must also be considered. 1424 7.2.7. Packet Ordering Requirements 1426 A document is needed to define extensions supporting various packet 1427 ordering requirements, ranging from requirements to preservce 1428 microflow ordering only, to requirements to preservce full LSP 1429 ordering (as in MPLS-TP). This is covered by 1430 [I-D.villamizar-mpls-tp-multipath] and 1431 [I-D.villamizar-mpls-tp-multipath-te-extn]. 1433 The primary focus of this document, among the sets of requirements 1434 listed in Section 7.1 are the "admission control, preemption, traffic 1435 engineering" and the "path determination, connectivity verification" 1436 sets of requirements. The "backward compatibility and migration" and 1437 "general network management" requirements must also be considered. 1439 7.2.8. Minimally Disruption Load Balance 1441 The behavior of hash methods used in classic multipath needs to be 1442 described in terms of FR#12 which calls for minimally disruptive load 1443 adjustments. For example, reseeding the hash violates FR#12. Using 1444 modulo operations is significantly disruptive if a link comes or goes 1445 down, as pointed out in [RFC2992]. In addition, backwards 1446 compatibility with older hardware needs to be accommodated. 1448 The primary focus of this document, among the sets of requirements 1449 listed in Section 7.1 is the "load distribution, stability, minimal 1450 disruption" set of requirements. 1452 7.2.9. Path Symmetry 1454 Protocol extensions are needed to support dynamic load balance as 1455 called for to meet FR#22 (path symmetry) and to meet FR#11 (dynamic 1456 placement of flows). Currently path symmetry can only be supported 1457 in link bundling if the path is pinned. When a flow is moved both 1458 ingress and egress must make the move as close to simultaneously as 1459 possible to satisfy FR#22 and FR#12 (minimally disruptive load 1460 rebalance). If a group of flows are identified using a hash, then 1461 the hash must be identical on the pair of LSR at the endpoint, using 1462 the same hash seed and with one side swapping source and destination. 1463 If the label stack is used, then either the entire label stack must 1464 be a special case flow identification, since the set of labels in 1465 either direction are not correlated, or the two LSR must conspire to 1466 use the same flow identifier. For example, using a common entropy 1467 label value, and using only the entropy label in the flow 1468 identification would satisfy this requirement. 1470 The primary focus of this document, among the sets of requirements 1471 listed in Section 7.1 are the "load distribution, stability, minimal 1472 disruption" and the "admission control, preemption, traffic 1473 engineering" sets of requirements. The "backward compatibility and 1474 migration" and "general network management" requirements must also be 1475 considered. Path symetry simplifies support for the "path 1476 determination, connectivity verification" set of requirements, but 1477 with significant complexity added elsewhere. 1479 7.2.10. Performance, Scalability, and Stability 1481 A separate document providing analysis of performance, scalability, 1482 and stability impacts of changes may be needed. The topic of traffic 1483 adjustment oscillation must also be covered. If sufficient coverage 1484 is provided in each document covering a protocol extension, a 1485 separate document would not be needed. 1487 The primary focus of this document, among the sets of requirements 1488 listed in Section 7.1 is the "restoration speed" set of requirements. 1489 This is not a simple topic and not a topic that is well served by 1490 scattering it over multiple documents, therefore it may be best to 1491 put this in a separate document and put citations in documents called 1492 for in Section 7.2.1, Section 7.2.2, Section 7.2.3, Section 7.2.9, 1493 Section 7.2.11, Section 7.2.12, Section 7.2.13, and Section 7.2.14. 1494 Citation may also be helpful in Section 7.2.4, and Section 7.2.5. 1496 7.2.11. IP and LDP Traffic 1498 A document is needed to define the use of measurements native IP and 1499 native LDP traffic levels to reduce link advertised bandwidth 1500 amounts. 1502 The primary focus of this document, among the sets of requirements 1503 listed in Section 7.1 are the "load distribution, stability, minimal 1504 disruption" and the "admission control, preemption, traffic 1505 engineering" set of requirements. The "path determination, 1506 connectivity verification" must also be considered. The "backward 1507 compatibility and migration" and "general network management" 1508 requirements must also be considered. 1510 7.2.12. LDP Extensions 1512 Extending LDP is called for in DR#2. LDP can be extended to couple 1513 FEC admission control to local resource availability without 1514 providing LDP traffic engineering capability. Other LDP extensions 1515 such as signaling a bound on microflow size and LDP LSP requirements 1516 would provide useful information without providing LDP traffic 1517 engineering capability. 1519 The primary focus of this document, among the sets of requirements 1520 listed in Section 7.1 is the "admission control, preemption, traffic 1521 engineering" set of requirements. The "backward compatibility and 1522 migration" and "general network management" requirements must also be 1523 considered. 1525 7.2.13. Pseudowire Extensions 1527 PW extensions such as signaling a bound on microflow size and PW 1528 requirements would provide useful information. 1530 The primary focus of this document, among the sets of requirements 1531 listed in Section 7.1 is the "admission control, preemption, traffic 1532 engineering" set of requirements. The "backward compatibility and 1533 migration" and "general network management" requirements must also be 1534 considered. 1536 7.2.14. Multi-Domain Composite Link 1538 DR#5 calls for Composite Link to span multiple network topologies. 1539 Component LSP may already span multiple network topologies, though 1540 most often in practice these are LDP signaled. Component LSP which 1541 are RSVP-TE signaled may also span multiple network topologies using 1542 at least three existing methods (per domain [RFC5152], BRPC 1543 [RFC5441], PCE [RFC4655]). When such component links are combined in 1544 a Composite Link, the Composite Link spans multiple network 1545 topologies. It is not clear in which document this needs to be 1546 described or whether this description in the framework is sufficient. 1547 The authors and/or the WG may need to discuss this. DR#5 mandates 1548 that IGP-TE extension cannot be used. This would disallow the use of 1549 [RFC5316] or [RFC5392] in conjunction with [RFC5151]. 1551 The primary focus of this document, among the sets of requirements 1552 listed in Section 7.1 are "single vs multiple domain" and "admission 1553 control, preemption, traffic engineering". The "routing information 1554 aggregation" and "load distribution, stability, minimal disruption" 1555 requirements need attention due to their use of the IGP in single 1556 domain Composite Link. Other requirements such as "delay and delay 1557 variation", can more easily be accomodated by carrying metrics within 1558 BGP. The "path determination, connectivity verification" 1559 requirements need attention due to requirements to restrict 1560 disclosure of topology information across domains in multi-domain 1561 deployments. The "backward compatibility and migration" and "general 1562 network management" requirements must also be considered. 1564 7.3. Open Issues Regarding Requirements 1566 Note to co-authors: This section needs to be reduced to an empty 1567 section and then removed. 1569 The following topics in the requirements document are not addressed. 1570 Since they are explicitly mentioned in the requirements document some 1571 mention of how they are supported is needed, even if to say nother 1572 needed to be done. If we conclude any particular topic is 1573 irrelevant, maybe the topic should be removed from the requirement 1574 document. At that point we could add the management requirements 1575 that have come up and were missed. 1577 1. L3VPN RFC 4364, RFC 4797,L2VPN RFC 4664, VPWS, VPLS RFC 4761, RFC 1578 4762 and VPMS VPMS Framework 1579 (draft-ietf-l2vpn-vpms-frmwk-requirements). It is not clear what 1580 additional Composite Link requirements these references imply, if 1581 any. If no additional requirements are implied, then these 1582 references are considered to be informational only. 1584 2. Migration may not be adequately covered in Section 4.1.5. It 1585 might also be necessary to say more here on performance, 1586 scalability, and stability as it related to migration. Comments 1587 on this from co-authors or the WG? 1589 3. We may need a performance section in this document to 1590 specifically address #DR6 (fast convergence), and #DR7 (fast 1591 worst case failure convergence), though we do already have 1592 scalability discussion. The performance section would have to 1593 say "no worse than before, except were there was no alternative 1594 to make it very slightly worse" (in a bit more detail than that). 1595 It would also have to better define the nature of the performance 1596 criteria. 1598 7.4. Framework Requirement Coverage by Protocol 1600 As an aid to implementors, this section summarizes requirement 1601 coverage listed in Section 7.2 by protocol or LSR functionality 1602 affected. 1604 Some documentation may be purely informational, proposing no changes 1605 and proposing usage at most. This includes Section 7.2.3, 1606 Section 7.2.8, Section 7.2.10, and Section 7.2.14. 1608 Section 7.2.9 may require a new protocol. 1610 7.4.1. OSPF-TE and ISIS-TE Protocol Extensions 1612 Many of the changes listed in Section 7.2 require IGP-TE changes, 1613 though most are small extensions to provide additional information. 1614 This set includes Section 7.2.1, Section 7.2.2, Section 7.2.5, 1615 Section 7.2.6, and Section 7.2.7. An adjustment to existing 1616 advertised parameters is suggested in Section 7.2.11. 1618 7.4.2. PW Protocol Extensions 1620 The only suggestion of pseudowire (PW) extensions is in 1621 Section 7.2.13. 1623 7.4.3. LDP Protocol Extensions 1625 Potential LDP extensions are described in Section 7.2.12. 1627 7.4.4. RSVP-TE Protocol Extensions 1629 RSVP-TE protocol extensions are called for in Section 7.2.1, 1630 Section 7.2.5, Section 7.2.7, and Section 7.2.9. 1632 7.4.5. RSVP-TE Path Selection Changes 1634 Section 7.2.3 calls for path selection to be addressed in individual 1635 documents that require change. These changes would include those 1636 proposed in Section 7.2.1, Section 7.2.2, Section 7.2.5, and 1637 Section 7.2.7. 1639 7.4.6. RSVP-TE Admission Control and Preemption 1641 When a change is needed to path selection, a corresponding change is 1642 needed in admission control. The same set of sections applies: 1643 Section 7.2.1, Section 7.2.2, Section 7.2.5, and Section 7.2.7. Some 1644 resource changes such as a link delay change might trigger 1645 preemption. The rules of preemption remain unchanged, still based on 1646 holding priority. 1648 7.4.7. Flow Identification and Traffic Balance 1650 The following describe either the state of the art in flow 1651 identification and traffic balance or propose changes: Section 7.2.4, 1652 Section 7.2.5, Section 7.2.7, and Section 7.2.8. 1654 8. Security Considerations 1656 The security considerations for MPLS/GMPLS and for MPLS-TP are 1657 documented in [RFC5920] and [I-D.ietf-mpls-tp-security-framework]. 1659 The types protocol extensions proposed in this framework document 1660 provide additional information about links, forwarding adjacencies, 1661 and LSP requirements. The protocol semantics changes described in 1662 this framework document propose additional LSP constraints applied at 1663 path computation time and at LSP admission at midpoints LSR. The 1664 additional information and constraints provide no additional security 1665 considerations beyond the security considerations already documented 1666 in [RFC5920] and [I-D.ietf-mpls-tp-security-framework]. 1668 9. Acknowledgments 1670 Authors would like to thank Adrian Farrel, Fred Jounay, Yuji Kamite 1671 for his extensive comments and suggestions regarding early versions 1672 of this document, Ron Bonica, Nabil Bitar, Eric Gray, Lou Berger, and 1673 Kireeti Kompella for their reviews of early versions and great 1674 suggestions. 1676 Authors would like to thank Iftekhar Hussain for review and 1677 suggestions regarding recent versions of this document. 1679 In the interest of full disclosure of affiliation and in the interest 1680 of acknowledging sponsorship, past affiliations of authors are noted. 1681 Much of the work done by Ning So occurred while Ning was at Verizon. 1682 Much of the work done by Curtis Villamizar occurred while at 1683 Infinera. Infinera continues to sponsor this work on a consulting 1684 basis. 1686 10. References 1688 10.1. Normative References 1690 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1691 Requirement Levels", BCP 14, RFC 2119, March 1997. 1693 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1694 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1695 Tunnels", RFC 3209, December 2001. 1697 [RFC3630] Katz, D., Kompella, K., and D. Yeung, "Traffic Engineering 1698 (TE) Extensions to OSPF Version 2", RFC 3630, 1699 September 2003. 1701 [RFC4201] Kompella, K., Rekhter, Y., and L. Berger, "Link Bundling 1702 in MPLS Traffic Engineering (TE)", RFC 4201, October 2005. 1704 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1705 Hierarchy with Generalized Multi-Protocol Label Switching 1706 (GMPLS) Traffic Engineering (TE)", RFC 4206, October 2005. 1708 [RFC5036] Andersson, L., Minei, I., and B. Thomas, "LDP 1709 Specification", RFC 5036, October 2007. 1711 [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic 1712 Engineering", RFC 5305, October 2008. 1714 [RFC5712] Meyer, M. and JP. Vasseur, "MPLS Traffic Engineering Soft 1715 Preemption", RFC 5712, January 2010. 1717 [RFC6107] Shiomoto, K. and A. Farrel, "Procedures for Dynamically 1718 Signaled Hierarchical Label Switched Paths", RFC 6107, 1719 February 2011. 1721 [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay 1722 Measurement for MPLS Networks", RFC 6374, September 2011. 1724 [RFC6391] Bryant, S., Filsfils, C., Drafz, U., Kompella, V., Regan, 1725 J., and S. Amante, "Flow-Aware Transport of Pseudowires 1726 over an MPLS Packet Switched Network", RFC 6391, 1727 November 2011. 1729 10.2. Informative References 1731 [DBP] Bertsekas, D., "Dynamic Behavior of Shortest Path Routing 1732 Algorithms for Communication Networks", IEEE Trans. Auto. 1733 Control 1982. 1735 [I-D.ietf-mpls-entropy-label] 1736 Drake, J., Kompella, K., Yong, L., Amante, S., and W. 1737 Henderickx, "The Use of Entropy Labels in MPLS 1738 Forwarding", draft-ietf-mpls-entropy-label-01 (work in 1739 progress), October 2011. 1741 [I-D.ietf-mpls-explicit-resource-control-bundle] 1742 Zamfir, A., Ali, Z., and P. Dimitri, "Component Link 1743 Recording and Resource Control for TE Links", 1744 draft-ietf-mpls-explicit-resource-control-bundle-10 (work 1745 in progress), April 2011. 1747 [I-D.ietf-mpls-tp-security-framework] 1748 Niven-Jenkins, B., Fang, L., Graveman, R., and S. 1749 Mansfield, "MPLS-TP Security Framework", 1750 draft-ietf-mpls-tp-security-framework-02 (work in 1751 progress), October 2011. 1753 [I-D.ietf-rtgwg-cl-requirement] 1754 Malis, A., Villamizar, C., McDysan, D., Yong, L., and N. 1755 So, "Requirements for MPLS Over a Composite Link", 1756 draft-ietf-rtgwg-cl-requirement-05 (work in progress), 1757 January 2012. 1759 [I-D.kompella-mpls-rsvp-ecmp] 1760 Kompella, K., "Multi-path Label Switched Paths Signaled 1761 Using RSVP-TE", draft-kompella-mpls-rsvp-ecmp-01 (work in 1762 progress), October 2011. 1764 [I-D.ospf-cc-stlv] 1765 Osborne, E., "Component and Composite Link Membership in 1766 OSPF", draft-ospf-cc-stlv-00 (work in progress), 1767 August 2011. 1769 [I-D.symmvo-rtgwg-cl-use-cases] 1770 Malis, A., Villamizar, C., McDysan, D., Yong, L., and N. 1771 So, "Composite Link USe Cases and Design Considerations", 1772 draft-symmvo-rtgwg-cl-use-cases-00 (work in progress), 1773 February 2012. 1775 [I-D.villamizar-mpls-tp-multipath] 1776 Villamizar, C., "Use of Multipath with MPLS-TP and MPLS", 1777 draft-villamizar-mpls-tp-multipath-01 (work in progress), 1778 March 2011. 1780 [I-D.villamizar-mpls-tp-multipath-te-extn] 1781 Villamizar, C., "Multipath Extensions for MPLS Traffic 1782 Engineering", 1783 draft-villamizar-mpls-tp-multipath-te-extn-00 (work in 1784 progress), July 2011. 1786 [I-D.wang-ccamp-latency-te-metric] 1787 Fu, X., Betts, M., Wang, Q., McDysan, D., and A. Malis, 1788 "GMPLS extensions to communicate latency as a traffic 1789 engineering performance metric", 1790 draft-wang-ccamp-latency-te-metric-03 (work in progress), 1791 March 2011. 1793 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1794 and W. Weiss, "An Architecture for Differentiated 1795 Services", RFC 2475, December 1998. 1797 [RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and 1798 Multicast Next-Hop Selection", RFC 2991, November 2000. 1800 [RFC2992] Hopps, C., "Analysis of an Equal-Cost Multi-Path 1801 Algorithm", RFC 2992, November 2000. 1803 [RFC3260] Grossman, D., "New Terminology and Clarifications for 1804 Diffserv", RFC 3260, April 2002. 1806 [RFC3468] Andersson, L. and G. Swallow, "The Multiprotocol Label 1807 Switching (MPLS) Working Group decision on MPLS signaling 1808 protocols", RFC 3468, February 2003. 1810 [RFC3945] Mannie, E., "Generalized Multi-Protocol Label Switching 1811 (GMPLS) Architecture", RFC 3945, October 2004. 1813 [RFC3985] Bryant, S. and P. Pate, "Pseudo Wire Emulation Edge-to- 1814 Edge (PWE3) Architecture", RFC 3985, March 2005. 1816 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1817 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1818 Use over an MPLS PSN", RFC 4385, February 2006. 1820 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 1821 Element (PCE)-Based Architecture", RFC 4655, August 2006. 1823 [RFC4928] Swallow, G., Bryant, S., and L. Andersson, "Avoiding Equal 1824 Cost Multipath Treatment in MPLS Networks", BCP 128, 1825 RFC 4928, June 2007. 1827 [RFC5151] Farrel, A., Ayyangar, A., and JP. Vasseur, "Inter-Domain 1828 MPLS and GMPLS Traffic Engineering -- Resource Reservation 1829 Protocol-Traffic Engineering (RSVP-TE) Extensions", 1830 RFC 5151, February 2008. 1832 [RFC5152] Vasseur, JP., Ayyangar, A., and R. Zhang, "A Per-Domain 1833 Path Computation Method for Establishing Inter-Domain 1834 Traffic Engineering (TE) Label Switched Paths (LSPs)", 1835 RFC 5152, February 2008. 1837 [RFC5316] Chen, M., Zhang, R., and X. Duan, "ISIS Extensions in 1838 Support of Inter-Autonomous System (AS) MPLS and GMPLS 1839 Traffic Engineering", RFC 5316, December 2008. 1841 [RFC5392] Chen, M., Zhang, R., and X. Duan, "OSPF Extensions in 1842 Support of Inter-Autonomous System (AS) MPLS and GMPLS 1843 Traffic Engineering", RFC 5392, January 2009. 1845 [RFC5441] Vasseur, JP., Zhang, R., Bitar, N., and JL. Le Roux, "A 1846 Backward-Recursive PCE-Based Computation (BRPC) Procedure 1847 to Compute Shortest Constrained Inter-Domain Traffic 1848 Engineering Label Switched Paths", RFC 5441, April 2009. 1850 [RFC5920] Fang, L., "Security Framework for MPLS and GMPLS 1851 Networks", RFC 5920, July 2010. 1853 [RFC5921] Bocci, M., Bryant, S., Frost, D., Levrau, L., and L. 1854 Berger, "A Framework for MPLS in Transport Networks", 1855 RFC 5921, July 2010. 1857 Authors' Addresses 1859 So Ning 1860 Tata Communications 1862 Email: ning.so@tatacommunications.com 1864 Dave McDysan 1865 Verizon 1866 22001 Loudoun County PKWY 1867 Ashburn, VA 20147 1869 Email: dave.mcdysan@verizon.com 1871 Eric Osborne 1872 Cisco 1874 Email: eosborne@cisco.com 1876 Lucy Yong 1877 Huawei USA 1878 5340 Legacy Dr. 1879 Plano, TX 75025 1881 Phone: +1 469-277-5837 1882 Email: lucy.yong@huawei.com 1884 Curtis Villamizar 1885 Outer Cape Cod Network Consulting 1887 Email: curtis@occnc.com