MPLS WG Internet Draft Matthew R. Meyer (Ed) Global Crossing Jean-Philippe Vasseur (Ed) Cisco Systems, Inc Denver Maddux Nitrous.net Curtis Villamizar Amir Birjandi Juniper Networks Proposed status: Standard Expires: December 2005 June 2005 MPLS Traffic Engineering Soft Preemption draft-ietf-mpls-soft-preemption-06.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. draft-ietf-mpls-soft-preemption-06.txt June 2005 Abstract This document details MPLS Traffic Engineering Soft Preemption, a suite of protocol modifications extending the concept of preemption with the goal of reducing/eliminating traffic disruption of preempted Traffic Engineering Label Switched Paths (TE LSPs). Initially MPLS RSVP-TE was defined supporting only immediate TE LSP displacement upon preemption. The utilization of a preemption pending flag helps more gracefully mitigate the re-route process of preempted TE LSP. For the brief period soft preemption is activated, reservations (though not necessarily traffic levels) are in effect under- provisioned until the TE LSP(s) can be re-routed. For this reason, the feature is primarily but not exclusively interesting in MPLS enabled IP networks with Differentiated Services and Traffic Engineering capabilities. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [i]. Table of Contents 1. Terminology...............................................3 1.1 Acronyms and Abbreviations............................3 1.2 Nomenclature..........................................3 2. Motivations...............................................4 3. Introduction..............................................4 4. RSVP Extensions...........................................5 4.1 SESSION-ATTRIBUTE Flags...............................5 4.2 RRO IPv4/IPv6 Sub-Object Flags........................5 4.3 Use of the RRO IPv4/IPv6 Sub-Object in Path message...5 5. Theory of Operation.......................................6 6. Elements Of Procedures....................................7 6.1 On a soft preempting LSR..............................7 6.2 On Head-end LSR of soft preempted TE LSP..............8 7. Interoperability..........................................9 8. Management................................................10 9. IANA Considerations.......................................10 10. Security considerations..................................10 11. Acknowledgment...........................................10 12. Intellectual Property Considerations.....................10 13. References...............................................11 13.1 Normative references.................................11 13.2 Informative references...............................11 14. Authors' Addresses.......................................12 Meyer, Vasseur et al. [Page 2] draft-ietf-mpls-soft-preemption-06.txt June 2005 1. Terminology This document follows the nomenclature of the MPLS Architecture defined in [MPLS-ARCH]. 1.1 Acronyms and Abbreviations CSPF Constraint-based Shortest Path First. DS Differentiated Services LER Label Edge Router LSR Label Switching Router LSP Label Switched Path MPLS MultiProtocol Label Switching PPend Preemption Pending RSVP Resource ReSerVation Protocol TE Traffic Engineering TE LSP Traffic Engineering Label Switched Path 1.2 Nomenclature Make Before Break - Technique used to non-intrusively alter the path of a TE LSP. The ingress LER first signals the new path, sharing the bandwidth with the primary TE LSP (to avoid double booking), then switches forwarding over to a new path. Finally the old path state is torn down. Numerically Lower Preemption Priority - TE LSPs have setup and hold preemption priorities of zero (best) through seven (worst). A numerically lower setup priority TE LSP is capable of preempting a numerically higher hold priority TE LSP. Preemption Pending flag - This flag is set on an IPv4 or IPv6 RSVP Resv RRO sub-object to signal to the TE LSP ingress LER that the TE LSP is about to be preempted and must be re-signaled (in a non disruptive fashion, with make before break) along another path. If present in the Path RRO, it is used to alert downstream LSRs that the LSP was soft preempted upstream. Point of Preemption - the midpoint or ingress LSR which due to RSVP provisioning levels is forced to either hard preempt or under- provision and signal soft preemption. Hard Preemption - The (typically default) preemption process in which higher numeric priority TE LSPs are intrusively displaced at the point of preemption by lower numeric priority TE LSPs. In hard preemption the TE LSP is torn down before reestablishment. Soft Preemption - The preemption process in which the point of preemption allows a brief under-provisioning period while the ingress Meyer, Vasseur et al. [Page 3] draft-ietf-mpls-soft-preemption-06.txt June 2005 router is alerted to the requirement for reroute. In soft preemption the TE LSP is reestablished before being torn down. Soft Preemption Desired Flag - This flag is set on the SESSION_ATTRIBUTES Flags in the Path message for the TE LSP indicate to LSRs along the path that, should the LSP need to be preempted, soft preemption should be used if supported. 2. Motivations Initially MPLS RSVP-TE [RSVP-TE] was defined supporting only one method of TE LSP preemption which immediately tears down TE LSPs, disregarding the preempted in-transit traffic. This simple but abrupt process nearly guarantees preempted traffic will be discarded, if only briefly, until the RSVP Path Error message reaches and is processed by the ingress LER and a new forwarding path can be established. In cases of actual resource contention this might be helpful, however preemption may be triggered by mere reservation contention and reservations may not reflect forwarding plane contention up to the moment. The result is that when conditions that promote preemption exist and hard preemption is the default behavior, inferior priority preempted traffic may be needlessly discarded when sufficient bandwidth exists for both the preempted LSP and the preempting TE LSP(s). Hard preemption may be a requirement to protect numerically lower preemption priority traffic in a non Diff-Serv enabled architecture, but in a Diff-Serv enabled architecture, one need not rely exclusively upon preemption to enforce a preference for the most valued traffic since the marking and queuing disciplines should already be aligned for those purposes. Moreover, even in non Diff- Serv aware networks, depending on the TE LSP sizing rules (imagine all LSPs are sized at double their observed traffic level), reservation contention may not accurately reflect the potential for forwarding plane congestion. 3. Introduction In an MPLS RSVP-TE [RSVP-TE] enabled IP network, hard preemption is the default behavior. Hard preemption provides no mechanism to allow preempted TE LSPs to be handled in a make-before-break fashion: the hard preemption scheme instead utilizes a very intrusive method that can cause traffic disruption for a potentially large amount of TE LSPs. Without an alternative, network operators either accept this limitation, or remove functionality by using only one preemption priority or using invalid bandwidth reservation values. Understandably desirable features like ingress LER automated TE reservation adjustments are less palatable when preemption is intrusive and high network stability levels are a concern. Meyer, Vasseur et al. [Page 4] draft-ietf-mpls-soft-preemption-06.txt June 2005 This document defines the use of additional signaling and maintenance mechanisms to alert the ingress LER of the preemption that is pending and allow for temporary under-provisioning while the preempted tunnel is re-routed in a non disruptive fashion (make-before-break) by the ingress LER. During the period that the tunnel is being re-routed, link capacity is under-provisioned on the midpoint where preemption initiated and potentially one or more links upstream along the path where other soft preemptions may have occurred. Optionally the downstream path to the egress LER may be signaled as well to more efficiently deal with any near simultaneous soft preemptions that may have been triggered downstream of the initial preemption. 4. RSVP Extensions 4.1 SESSION-ATTRIBUTE Flags To explicitly signal the desire for a TE LSP to benefit from the soft preemption mechanism (and so not to be hard preempted if the soft preemption mechanism is available), the following flag of the SESSION-ATTRIBUTE object (for both the C-Type 1 and 7) is defined: Soft preemption desired: 0x40 (to be confirmed by IANA) 4.2 RRO IPv4/IPv6 Sub-Object Flags To report that a soft preemption is pending for an LSP, a new flag is defined in the IPv4/IPv6 sub-object carried in the RRO object message defined in [RSVP-TE]. This flag is called the preemption pending (PPend) flag. A compliant LSR MUST support the RRO object, as defined in [RSVP-TE]. Several flags in the RRO IPv4 and IPv6 sub-object have been defined in [RSVP-TE]and [FAST-REROUTE]: This documents defines a new flag for the use of soft preemption named the 'Preemption pending' flag and defined as below: Preemption pending: 0x10 The preempting node sets this flag if a pending preemption is in progress for the TE LSP. This indicates to the ingress LER of this LSP that it SHOULD be re-routed. 4.3 Use of the RRO IPv4/IPv6 Sub-Object in Path message An LSR MAY use the Preemption pending flag in the IPv4/IPv6 RRO sub- object carried in a PATH RRO message to simultaneously alert downstream LSRs that the LSP was soft preempted upstream. This information could be used by the downstream LSR to bias future soft Meyer, Vasseur et al. [Page 5] draft-ietf-mpls-soft-preemption-06.txt June 2005 preemption candidates toward LSPs already soft preempted elsewhere in their path. 5. Theory of Operation Let's consider the following example: R0--1G--R1---155----R2 LSP1: LSP2: | \ | | \ 155 R0-->R1 R1<--R2 | \ | \ | 155 1G R3 V V | \ | R5 R4 | \ 155 | \| R4----1G----R5 Figure 1: example of Soft Preemption Operation In the network depicted above in figure 1, consider the following conditions: -Reservable BW on R0-R1, R1-R5 and R4-R5 is 1Gb/sec. -Reservable BW on R1-R2, R1-R4, R2-R3, R3-R5 is 155 Mb/sec. -Bandwidths and costs are identical in both directions. -Each circuit has an IGP metric of 10 and IGP metric is used by CSPF. -Two TE tunnels are defined: - LSP1: 155 Mb, setup/hold priority 0 tunnel, path R0-R1-R5. - LSP2: 155 Mb, setup/hold priority 7 tunnel, path R2-R1-R4. Both TE LSPs are signaled with the soft preemption desired bit of their SESSION-ATTRIBUTE object set. -Circuit R1-R5 fails. -Soft Preemption is functional. When the circuit R1-R5 fails, R1 detects the failure and sends an updated IGP LSA/LSP and Path Error message to all the head-end LSRs having a TE LSP traversing the failed link (R0 in the example above). Either form of notification may arrive at the head-end LSRs first. Upon receiving the link failure notification, R0 triggers a TE LSP re-route of LSP1, and re-signals LSP1 along shortest path available satisfying the TE LSP constraints: R0-R1-R4-R5 path. The Resv messages for LSP1 travel in the upstream direction (from the destination to the head-end LSR - R5 to R0 in this example). LSP2 is soft preempted at R1 as it has a numerically lower priority value and both bandwidth reservations cannot be satisfied on the R1-R4 link. Instead of sending a path tear for LSP2 upon preemption as with hard preemption (which would result in an immediate traffic disruption for LSP2), R1s local bandwidth accounting for LSP2 is zeroed and a preemption pending flagged Resv RRO for LSP2 is issued. Optionally, Meyer, Vasseur et al. [Page 6] draft-ietf-mpls-soft-preemption-06.txt June 2005 R1 MAY simultaneously send a soft preemption flagged Path RRO notifying downstream LSRs of LSP2s soft preemption. Upon reception of the LSP2's Resv message with the preemption pending flag set, R2 may update the working copy of the TE-DB before running CSPF for the new LSP. In the case that Diff-Serv [DIFF-MPLS] and TE [RSVP-TE] are deployed, receiving preemption pending may imply to a head-end LSR that the available bandwidth for the affected priority level and numerically greater priority levels has been exhausted for the indicated node interface. R2 may choose to reduce or zero available bandwidth for the implied priority range until more accurate information is available (i.e. a new IGP TE update is received). It follows that R2 re-computes a new path and performs a non traffic disruptive rerouting of the new TE LSP T2 by means of the make- before-break procedure. The old path is then torn down. 6. Elements Of Procedures 6.1 On a soft preempting LSR When a new TE LSP is signaled which requires to preempt a set of TE LSP(s) because not all TE LSPs can be accommodated on a specific interface, a node triggers a preemption action which consists of selecting the set of TE LSPs that must be preempted so as to free up some bandwidth in order to satisfy the newly signaled numerically lower preemption TE LSP. For each preempted TE LSP, instead of sending a path tear upon preemption as with hard preemption (which would result in an immediate traffic disruption for the preempted TE LSP), the preempting node's local bandwidth accounting for the preempted TE LSP is zeroed and a preemption pending flagged Resv RRO for that TE LSP is issued upstream toward the head-end LSR. Optionally, the preempting node MAY simultaneously send a soft preemption flagged Path RRO notifying downstream LSRs of soft preemption. If more than one soft preempted TE LSP has the same head-end LSR, these soft preemption Resv (Path) messages may be bundled together. The preempting node MUST immediately send a Resv message with the preemption pending RRO flag set for each soft preempted TE LSP. The node MAY use the occurrence of soft preemption to trigger an immediate IGP update or influence the scheduling of an IGP update. Should a refresh event for a soft preempted TE LSP arrive before the soft preemption timer expires, the soft preempting node MUST continue to refresh the TE LSP. Meyer, Vasseur et al. [Page 7] draft-ietf-mpls-soft-preemption-06.txt June 2005 When the MESSAGE-ID extensions defined in [REFRESH-REDUCTION] are available, Resv messages with the RRO preemption pending flag set SHOULD be sent in reliable mode. In the case that reservation availability is restored at the point of preemption, the point of preemption MAY issue a Resv message with the preemption pending flag unset to signal restoration to the head-end LSR. This implies that a head-end LSR might have delayed or been unsuccessful in re-signaling. To guard against a situation where bandwidth under-provisioning will last forever, a local timer (named the "Soft preemption timer") MUST be started on the preemption node, upon soft preemption. If this timer expires, the preempting node SHOULD send a PathTear and either a ResvTear or a PathErr with the 'Path_State_Removed' flag set. Selection of the preempted TE LSP at a preempting mid-point: when a numerically lower priority TE LSP is signaled that requires the preemption of a set of numerically higher priority LSPs, the node where preemption is to occur has to make a decision on the set of TE LSP(s), candidates for preemption. This decision is a local decision and various algorithms can be used, depending on the objective. See [PREEMPT-EXP]. As already mentioned, soft preemption causes a temporary link under provisioning condition while the soft preempted TE LSPs are rerouted by their respective head-end LSRs. In order to reduce this under provisioning exposure, a soft-preempting LSR MAY check first if there exists soft preempt-able TE LSP bandwidth flagged PPend by another node but still available for soft-preemption locally. If sufficient overlap bandwidth exists the LSR MAY attempt to soft preempt the same LSP. This would help reducing the temporarily elevated under-provisioning ratio on the links where soft preemption occurs and the number of preempted TE LSPs. Optionally, a midpoint LSR upstream or downstream from a soft preempting node MAY choose to flag the LSPs soft preempted state. In the event a local preemption is needed, the relevant priority level LSPs from the cache are soft preempted first, followed by the normal soft and hard preemption selection process for the given priority. Under specific circumstances such as unacceptable link congestion, a node MAY decide to hard preempt a TE LSP (by sending a PathTear and either a ResvTear or a PathErr with the 'Path_State_Removed' flag set) even if its head-end LSR explicitly requested 'soft preemption' ('Soft Preemption desired' flag of the corresponding SESSION- ATTRIBUTE object set). Note that such decision MAY also be taken for TE LSPs under soft preemption state. 6.2 On Head-end LSR of soft preempted TE LSP Meyer, Vasseur et al. [Page 8] draft-ietf-mpls-soft-preemption-06.txt June 2005 Upon reception of a Resv message with the preemption pending flag set, the head-end LSR MAY first update the working copy of the TE-DB before computing a new path (e.g by running CSPF) for the new LSP. In the case that Diff-Serv [DIFF-MPLS] and MPLS Traffic Engineering [RSVP-TE] are deployed, receiving preemption pending may imply to a head-end LSR that the available bandwidth for the affected priority level and numerically greater priority levels has been exhausted for the indicated node interface. A head-end LSR MAY choose to reduce or zero available bandwidth for the implied priority range until more accurate information is available (i.e. a new IGP TE update is received). Once a new path has been computed, the soft preempted TE LSP is rerouted using the non traffic disruptive make-before-break procedure. As a result of soft preemption, no traffic will be needlessly black holed due to mere reservation contention. If loss is to occur, it will be due only to an actual traffic congestion scenario and according to the operators Diff-Serv (if Diff-Serv is deployed) and queuing scheme. 7. Interoperability Backward compatibility should be assured as long as the implementation followed the recommendations set forth in [RSVP-TE]. When processing an RRO, unrecognized sub-objects SHOULD be ignored and passed on. An LSR without soft preemption capabilities but that followed the aforementioned recommendation will simply ignore the RRO Preemption Pending flag and treat the Resv message as a regular Resv refresh message. As a consequence, the soft preempted TE LSP will not be rerouted with make before break by the head-end LSR. As mentioned previously, to guard against a situation where bandwidth under-provisioning will last forever, a local timer (soft preemption timer) MUST be started on the preemption node, upon soft preemption. When this timer expires, the soft preempted TE LSP SHOULD be hard preempted by sending a PathTear and either a ResvTear or a PathErr with the 'Path_State_Removed' flag set. This timer SHOULD be configurable and a default value of 30 seconds is RECOMMENDED. It is RECOMMENDED that configuring the default preemption timer to 0 will cause the implementation to use hard-preemption. Soft Preemption as defined in this document is designed for use in MPLS RSVP-TE enabled IP Networks and may not functionally translate to some GMPLS technologies. As with backward compatibility, if a device does not recognize a flag, it should pass the subobject transparently. Meyer, Vasseur et al. [Page 9] draft-ietf-mpls-soft-preemption-06.txt June 2005 8. Management Both the point of preemption and the ingress LER SHOULD provide some form of accounting internally and to the network operator interface with regard to which TE LSPs and how much capacity is under- provisioned due to soft preemption. Displays of under-provisioning are recommended for the following midpoint, ingress and egress views: - Sum of current bandwidth per preemption priority per local interface - Sum of current bandwidth total per local interface - Sum of current bandwidth total local router (ingress, egress, midpoint) - List current LSPs and bandwidth in PPend status - List current sum bandwidth and session count in PPend status per observed ERO hops (ingress, egress views only). - Cumulative PPend events per observed ERO hops. 9. IANA Considerations IANA [RFC-IANA] will not need to create a new registry. This document requires the assignment of flags related to RFC3209 [RSVP-TE] sections 4.1.1.1, 4.1.1.2, 4.7.1 and 4.7.2. IANA will assign RRO IPv4/IPv6 sub-object flags defined in RFC3209 [RSVP-TE] sec 4.1.1.1 and 4.1.1.2 as detailed in section 4.2 of this document. IANA will assign session attribute flags for both the C-Type 1 and 7 (defined in RFC3209 [RSVP-TE] sec 4.7.1 and 4.7.2) as detailed in section 4.1 of this document. 10. Security Considerations This document does not introduce new security issues. The security considerations pertaining to the original RSVP protocol [RSVP] remain relevant. 11. Acknowledgment The authors would like to thank Carol Iturralde, Dave Cooper, Loa Andersson, Arthi Ayyangar, Ina Minei and George Swallow for their valuable comments. 12. Intellectual Property Considerations The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in Meyer, Vasseur et al. [Page 10] draft-ietf-mpls-soft-preemption-06.txt June 2005 this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf- ipr@ietf.org. 13. References 13.1 Normative references [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels," RFC 2119. [RFC-IANA] T. Narten and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", RFC 2434. [MPLS-ARCH] Rosen, Viswanathan, Callon, "Multiprotocol Label Switching Architecture", RFC3031, January 2001. [RSVP] R. Braden, Ed., et al, "Resource ReSerVation protocol (RSVP) - version 1 functional specification," RFC2205, September 1997. [RSVP-TE] Awduche et al, "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC3209, December 2001. 13.2 Informative references [REFRESH-REDUCTION] Berger et al, "RSVP Refresh Overhead Reduction Extensions", RFC 2961, April 2001. [FAST-REROUTE] P. Pan, Ed., G. Swallow, Ed., A. Atlas, Ed et al., "Fast Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, May 2005. [PREEMPT-EXP]De Oliveira, J., Vasseur, JP., Chen, L. and Scoglio, C., "LSP Preemption Policies for MPLS Traffic Engineering", daft-deoliviera-diff-te-preemption-02.txt, October 2003. Meyer, Vasseur et al. [Page 11] draft-ietf-mpls-soft-preemption-06.txt June 2005 [DIFF-MPLS] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, P., Krishnan, R., Cheval, P. and J. Heinanen, "Multi- Protocol Label Switching (MPLS) Support of Differentiated Services", RFC 3270, May 2002. 14. Authors' Addresses Matthew R. Meyer Global Crossing 3133 Indian Valley Tr. Howell, MI 48855 USA email: mrm@gblx.net, matthew.r.meyer@gmail.com Denver Maddux Nitrous.net 4237 E. Hartford Ave. Phoenix, AZ 85032 USA email: denver@nitrous.net Jean-Philippe Vasseur CISCO Systems, Inc. 300 Beaver Brook Boxborough, MA 01719 USA Email: jpv@cisco.com Curtis Villamizar AVICI curtis@faster-light.net Amir Birjandi Juniper Networks 2251 corporate park dr ste herndon, VA 20171 USA abirjandi@juniper.net Full Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS Meyer, Vasseur et al. [Page 12] draft-ietf-mpls-soft-preemption-06.txt June 2005 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Meyer, Vasseur et al. [Page 13]