idnits 2.17.1 draft-ietf-mpls-ldp-ft-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 53 longer pages, the longest (page 1) being 61 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 1608 has weird spacing: '...ication messa...' == Line 1726 has weird spacing: '... number equal...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: In these cases the node that intends terminating the session can flush acknowledgement using a check-point request as described above. The sender SHOULD not send further label or address-related messages after requesting shutdown check-pointing in order to preserve the integrity of its saved state. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 2002) is 7894 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 3036 (Obsoleted by RFC 5036) -- No information found for draft-ietf-ldp-restart - is the name correct? -- Possible downref: Normative reference to a draft: ref. 'LDP-RESTART' == Outdated reference: A later version (-13) exists of draft-ietf-idr-restart-05 Summary: 3 errors (**), 0 flaws (~~), 6 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 MPLS Working Group Editor 2 Internet Draft Adrian Farrel 3 Document: draft-ietf-mpls-ldp-ft-06.txt Movaz Networks, Inc. 4 Expiration Date: March 2003 September 2002 6 Fault Tolerance for the Label Distribution Protocol (LDP) 8 draft-ietf-mpls-ldp-ft-06.txt 10 Status of this Memo 12 This document is an Internet-Draft and is in full 13 conformance with all provisions of Section 10 of RFC2026 14 [RFC2026]. 16 Internet-Drafts are working documents of the Internet 17 Engineering Task Force (IETF), its areas, and its working 18 groups. Note that other groups may also distribute 19 working documents as Internet-Drafts. Internet-Drafts are 20 draft documents valid for a maximum of six months and may 21 be updated, replaced, or obsoleted by other documents at 22 any time. It is inappropriate to use Internet- Drafts as 23 reference material or to cite them other than as "work in 24 progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt 29 The list of Internet-Draft Shadow Directories can be 30 accessed at http://www.ietf.org/shadow.html. 32 NOTE: The new TLV type numbers, bit values for flags 33 specified in this draft, and new LDP status code values 34 are preliminary suggested values and have yet to be 35 approved by IANA or the MPLS WG. See the section "IANA 36 Considerations" for further details. 38 Abstract 40 Multiprotocol Label Switching (MPLS) systems will be used 41 in core networks where system downtime must be kept to an 42 absolute minimum. Many MPLS Label Switching Routers 43 (LSRs) may, therefore, exploit Fault Tolerant (FT) 44 hardware or software to provide high availability of the 45 core networks. 47 The details of how FT is achieved for the various 48 components of an FT LSR, including Label Distribution 49 Protocol (LDP), the switching hardware and TCP, are 50 implementation specific. This document identifies issues 51 in the LDP specification in RFC 3036 "LDP Specification" 52 that make it difficult to implement an FT LSR using the 53 current LDP protocols, and proposes enhancements to the 54 LDP specification to ease such FT LSR implementations. 56 The issues and extensions described here are equally 57 applicable to RFC 3212, "Constraint-Based LSP Setup Using 58 LDP" (CR-LDP). 60 Contents 62 1. Conventions and Terminology used in this document 3 63 2. Contributing Authors 4 64 3. Introduction 4 65 3.1. Fault Tolerance for MPLS 4 66 3.2. Issues with LDP 5 67 4. Overview of LDP FT Enhancements 7 68 4.1. Establishing an FT LDP Session 8 69 4.1.1 Interoperation with Non-FT LSRs 8 70 4.2. TCP Connection Failure 9 71 4.2.1 Detecting TCP Connection Failures 9 72 4.2.2 LDP Processing after Connection Failure 9 73 4.3. Data Forwarding During TCP Connection Failure 10 74 4.4. FT LDP Session Reconnection 10 75 4.5. Operations on FT Labels 11 76 4.6. Check-Pointing 11 77 4.6.1 Graceful Termination 12 78 4.7. Label Space Depletion and Replenishment 13 79 4.8. Tunneled LSPs 13 80 5. FT Operations 14 81 5.1. FT LDP Messages 14 82 5.1.1 Sequence Numbered FT Label Messages 14 83 5.1.2 FT Address Messages 15 84 5.1.3 Label Resources Available Notifications 15 85 5.2. FT Operation ACKs 17 86 5.3. Preservation of FT State 17 87 5.4. FT Procedure After TCP Failure 19 88 5.4.1 FT LDP Operations During TCP Failure 20 89 5.5. FT Procedure After TCP Re-connection 21 90 5.5.1 Re-Issuing FT Messages 22 91 6. Checkpointing Procedures 22 92 6.1 Checkpointing with the Keepalive Message 23 93 6.1 Quiesce and Keepalive 23 94 7. Changes to Existing Messages 24 95 7.1. LDP Initialization Message 24 96 7.2. LDP Keepalive Messages 24 97 7.3. All Other LDP Session Messages 25 98 8. New Fields and Values 25 99 8.1. Status Codes 25 100 8.2. FT Session TLV 26 101 8.3. FT Protection TLV 29 102 8.4. FT ACK TLV 31 103 8.5. FT Cork TLV 33 104 9. Example Use 34 105 9.1. Session Failure and Recovery - FT Procedures 35 106 9.2. Use of Check-Pointing With FT Procedures 37 107 9.3. Temporary Shutdown With FT Procedures 39 108 9.4. Temporary Shutdown With FT Procedures and Check-Pointing 41 109 9.5. Checkpointing Without FT Procedures 43 110 9.6. Graceful Shutdown With Checkpointing But No FT Procedures 45 111 10. Security Considerations 46 112 11. Implementation Notes 48 113 11.1. FT Recovery Support on Non-FT LSRs 48 114 11.2. ACK generation logic 48 115 11.2.1 Ack Generation Logic When Using Check-Pointing 48 116 11.3 Interactions With Other Label Distribution Mechanisms 49 118 12. Acknowledgments 49 119 13. Intellectual Property Consideration 50 120 14. Full Copyright Statement 50 121 15. IANA Considerations 50 122 15.1. New TLVs 51 123 15.2. New Status Codes 51 124 16. Authors' Addresses 52 125 17. References 52 126 17.1. Normative References 52 127 17.2. Informative References 53 129 1. Conventions and Terminology used in this document 131 Definitions of key words and terms applicable to LDP and 132 CR-LDP are inherited from [RFC3212] and [RFC3036]. 134 The term "FT Label" is introduced in this document to 135 indicated a label for which some fault tolerant operation 136 is used. A "non-FT Label" is not fault tolerant and is 137 handled as specified in [RFC3036]. 139 The term "Sequence Numbered FT Label" is used to indicate 140 an FT label which is secured using the sequence number in 141 the FT Protection TLV described in this document. 143 The term "Checkpointable FT Label" is used to indicate an 144 FT label which is secured by using the checkpointing 145 techniques described in this document. 147 The extensions to LDP specified in this document are 148 collectively referred to as the "LDP FT enhancements". 150 Within the context of this draft, "Checkpointing" refers 151 to a process of messages exchanges that confirm receipt 152 and processing (or secure storage) of specific protocol 153 messages. 155 When talking about the individual bits in the 16-bit FT 156 Flag Field, the words "bit" and "flag" is used 157 interchangeably. 159 In the examples quoted, the following notation is used: 160 Ln : An LSP. For example L1. 161 Pn : An LDP peer. For example P1. 163 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 164 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", 165 "MAY", and "OPTIONAL" in this document are to be 166 interpreted as described in RFC-2119 [RFC2119]. 168 2. Contributing Authors 170 This document was the collective work of several 171 individuals over a period of several years. The text and 172 content of this document was contributed by the editor 173 and the co-authors listed in section 16. 175 3. Introduction 177 High Availability (HA) is typically claimed by equipment 178 vendors when their hardware achieves availability levels 179 of at least 99.999% (five 9s). To implement this, the 180 equipment must be capable of recovering from local 181 hardware and software failures through a process known as 182 fault tolerance (FT). 184 The usual approach to FT involves provisioning backup 185 copies of hardware and/or software. When a primary copy 186 fails, processing is switched to the backup copy. This 187 process, called failover, should result in minimal 188 disruption to the Data Plane. 190 In an FT system, backup resources are sometimes 191 provisioned on a one-to-one basis (1:1), sometimes as one- 192 to-many (1:n), and occasionally as many-to-many (m:n). 193 Whatever backup provisioning is made, the system must 194 switch to the backup automatically on failure of the 195 primary, and the software and hardware state in the 196 backup must be set to replicate the state in the primary 197 at the point of failure. 199 3.1. Fault Tolerance for MPLS 201 MPLS is a technology that will be used in core networks 202 where system downtime must be kept to an absolute 203 minimum. Many MPLS LSRs may, therefore, exploit FT 204 hardware or software to provide high availability of core 205 networks. 207 In order to provide HA, an MPLS system needs to be able 208 to survive a variety of faults with minimal disruption to 209 the Data Plane, including the following fault types: 211 - failure/hot-swap of a physical connection between LSRs 213 - failure/hot-swap of the switching fabric in an LSR 215 - failure of the TCP or LDP stack in an LSR 217 - software upgrade to the TCP or LDP stacks in an LSR. 219 The first two examples of faults listed above are 220 confined to the Data Plane. Such faults can be handled 221 by providing redundancy in the Data Plane which is 222 transparent to LDP operating in the Control Plane. The 223 last two example types of fault require action in the 224 Control Plane to recover from the fault without 225 disrupting traffic in the Data Plane. This is possible 226 because many recent router architectures separate the 227 Control and Data Planes such that forwarding can continue 228 unaffected by recovery action in the Control Plane. 230 3.2. Issues with LDP 232 LDP uses TCP to provide reliable connections between LSRs 233 over which to exchange protocol messages to distribute 234 labels and to set up LSPs. A pair of LSRs that have such 235 a connection are referred to as LDP peers. 237 TCP enables LDP to assume reliable transfer of protocol 238 messages. This means that some of the messages do not 239 need to be acknowledged (for example, Label Release). 241 LDP is defined such that if the TCP connection fails, the 242 LSR should immediately tear down the LSPs associated with 243 the session between the LDP peers, and release any labels 244 and resources assigned to those LSPs. 246 It is notoriously hard to provide a Fault Tolerant 247 implementation of TCP. To do so might involve making 248 copies of all data sent and received. This is an issue 249 familiar to implementers of other TCP applications such 250 as BGP. 252 During failover affecting the TCP or LDP stacks, 253 therefore, the TCP connection may be lost. Recovery from 254 this position is made worse by the fact that LDP control 255 messages may have been lost during the connection 256 failure. Since these messages are unconfirmed, it is 257 possible that LSP or label state information will be 258 lost. 260 This draft describes a solution which involves 262 - negotiation between LDP peers of the intent to support 263 extensions to LDP that facilitate recovery from failover 264 without loss of LSPs 266 - selection of FT survival on a per LSP/label basis 268 - acknowledgement of LDP messages to ensure that a full 269 handshake is performed on those messages either frequently 270 (such as per message) or less frequently as in check- 271 pointing 273 - solicitation of up-to-date acknowledgement 274 (checkpointing) of previous LDP messages to ensure the 275 current state is flushed to disk/NVRAM, with an additional 276 option that allows an LDP partner to request that state is 277 flushed in both directions if graceful shutdown is required. 279 - re-issuing lost messages after failover to ensure that 280 LSP/label state is correctly recovered after reconnection of 281 the LDP session. 283 The issues and objectives described above are equally 284 applicable to CR-LDP. 286 Other objectives of this draft are to 288 - offer backward-compatibility with LSRs that do not 289 implement these extensions to LDP 291 - preserve existing protocol rules described in [RFC3036] 292 for handling unexpected duplicate messages and for 293 processing unexpected messages referring to unknown 294 LSPs/labels 296 - avoid full state refresh solutions (such as those present 297 in RSVP: see [RFC2205], [RFC2961], [RFC3209] and [LDP- 298 RESTART]) whether they be continual, or limited to post- 299 failover recovery. 301 Note that this draft concentrates on the preservation of 302 label state for labels exchanged between a pair of 303 adjacent LSRs when the TCP connection between those LSRs 304 is lost. This is a requirement for Fault Tolerant 305 operation of LSPs, but a full implementation of end-to- 306 end protection for LSPs requires that this is combined 307 with other techniques that are outside the scope of this 308 draft. 310 In particular, this draft does not attempt to describe 311 how to modify the routing of an LSP or the resources 312 allocated to a label or LSP, which is covered by 313 [RFC3214]. This draft also does not address how to 314 provide automatic layer 2 or layer 3 protection switching 315 for a label or LSP, which is a separate area for study. 317 This specification does not preclude an implementation 318 from attempting (or require it to attempt) to use the FT 319 behavior described here to recover from a preemptive 320 failure of a connection on a non-FT system due to, for 321 example, a partial system crash. Note, however, that 322 there are potential issues too numerous to list here - 323 not least the likelihood that the same crash will 324 immediately occur when processing the restored data. 326 4. Overview of LDP FT Enhancements 328 The LDP FT enhancements consist of the following main 329 elements, which are described in more detail in the 330 sections that follow. 332 - The presence of an FT Session TLV on the LDP 333 Initialization message indicates that an LSR supports some 334 form of protection or recovery from session failure. A flag 335 bit within this TLV (the S bit) indicates that the LSR 336 supports the LDP FT enhancements on this session. Another 337 flag (the C bit) indicates that the checkpointing procedures 338 are to be used. 340 - An FT Reconnect Flag in the FT Session TLV (the R bit) 341 indicates whether an LSR has preserved FT Label state across 342 a failure of the TCP connection. 344 - An FT Reconnection Timeout, exchanged on the LDP 345 Initialization message, that indicates the maximum time peer 346 LSRs will preserve FT Label state after a failure of the TCP 347 connection. 349 - An FT Protection TLV used to identify operations that 350 affect LDP labels. All LDP messages carrying the FT 351 Protection TLV need to be secured (e.g. to NVRAM) and ACKed 352 to the sending LDP peer in order that the state for Sequence 353 Numbered FT Labels can be correctly recovered after LDP 354 session reconnection. 356 Note that the implementation within an FT system is left 357 open by this draft. An implementation could choose to 358 secure entire messages relating to Sequence Numbered FT 359 Labels, or it could secure only the relevant state 360 information. 362 - Address advertisement may also be secured by use of the 363 FT Protection TLV. This enables recovery after LDP session 364 reconnection without the need to re-advertise what may be a 365 very large number of addresses. 367 - The FT Protection TLV may also be used on the Keepalive 368 message to flush acknowledgement of all previous FT 369 operations. This enables a check-point for future recovery, 370 either in mid-session or prior to graceful shutdown of an 371 LDP session. This procedure may also be used to checkpoint 372 all (that is both FT and non-FT) operations for future 373 recovery. 375 4.1. Establishing an FT LDP Session 377 In order that the extensions to LDP [RFC3036] described 378 in this draft can be used successfully on an LDP session 379 between a pair of LDP peers, they MUST negotiate that the 380 LDP FT enhancements are to be used on the LDP session. 382 This is done on the LDP Initialization message exchange 383 using a new FT Session TLV. Presence of this TLV 384 indicates that the peer wants to support some form of 385 protection or recovery processing. The S bit within this 386 TLV indicates that the peer wants to support the LDP FT 387 enhancements on this LDP session. The C bit indicates 388 that the peer wants to support the checkpointing 389 functions described in this draft. The S and C bits may 390 be set independently. 392 The relevant LDP FT enhancements MUST be supported on an 393 LDP session if both LDP peers include an FT Session TLV 394 on the LDP Initialization message and have the same 395 setting of the S or C bit. 397 If either LDP Peer does not include the FT Session TLV 398 LDP Initialization message or if there is no match of S 399 and C bits between the peers, the LDP FT enhancements 400 MUST NOT be used during this LDP session. Use of LDP FT 401 enhancements by a sending LDP peer MUST be interpreted by 402 the receiving LDP peer as a serious protocol error 403 causing the session to be terminated. 405 An LSR MAY present different FT/non-FT behavior on 406 different TCP connections, even if those connections are 407 successive instantiations of the LDP session between the 408 same LDP peers. 410 4.1.1 Interoperation with Non-FT LSRs 412 The FT Session TLV on the LDP Initialization message 413 carries the U-bit. If an LSR does not support any 414 protection or recovery mechanisms , it will ignore this 415 TLV. Since such partners also do not include the FT 416 Session TLV, all LDP sessions to such LSRs will not use 417 the LDP FT enhancements. 419 The rest of this draft assumes that the LDP sessions 420 under discussion are between LSRs that do support the LDP 421 FT enhancements, except where explicitly stated 422 otherwise. 424 4.2. TCP Connection Failure 426 4.2.1 Detecting TCP Connection Failures 428 TCP connection failures may be detected and reported to the 429 LDP component in a variety of ways. These should all be 430 treated in the same way by the LDP component. 432 - Indication from the management component that a TCP 433 connection or underlying resource is no longer active. 435 - Notification from a hardware management component of an 436 interface failure. 438 - Sockets keepalive timeout. 440 - Sockets send failure. 442 - New (incoming) Socket opened. 444 - LDP protocol timeout. 446 4.2.2 LDP Processing after Connection Failure 448 If the LDP FT enhancements are not in use on an LDP 449 session, the action of the LDP peers on failure of the 450 TCP connection is as specified in [RFC3036]. 452 All state information and resources associated with non- 453 FT Labels MUST be released on the failure of the TCP 454 connection, including deprogramming the non-FT Label from 455 the switching hardware. This is equivalent to the 456 behavior specified in [RFC3036]. 458 If the LDP FT enhancements are in use on an LDP session, 459 both LDP peers SHOULD preserve state information and 460 resources associated with FT Labels exchanged on the LDP 461 session. Both LDP peers SHOULD use a timer to release 462 the preserved state information and resources associated 463 with FT-labels if the TCP connection is not restored 464 within a reasonable period. The behavior when this timer 465 expires is equivalent to the LDP session failure behavior 466 described in [RFC3036]. 468 The FT Reconnection Timeout each LDP peer intends to 469 apply to the LDP session is carried in the FT Session TLV 470 on the LDP Initialization messages. Both LDP peers MUST 471 use the value that corresponds to the lesser timeout 472 interval of the two proposed timeout values from the LDP 473 Initialization exchange, where a value of zero is treated 474 as positive infinity. 476 4.3. Data Forwarding During TCP Connection Failure 478 An LSR that implements the LDP FT enhancements SHOULD 479 preserve the programming of the switching hardware across 480 a failover. This ensures that data forwarding is 481 unaffected by the state of the TCP connection between 482 LSRs. 484 It is an integral part of FT failover processing in some 485 hardware configurations that some data packets might be 486 lost. If data loss is not acceptable to the applications 487 using the MPLS network, the LDP FT enhancements described 488 in this draft SHOULD NOT be used. 490 4.4. FT LDP Session Reconnection 492 When a new TCP connection is established, the LDP peers 493 MUST exchange LDP Initialization messages. When a new 494 TCP connection is established after failure, the LDP 495 peers MUST re-exchange LDP Initialization messages. 497 If an LDP peer includes the FT Session TLV with the S bit 498 set in the LDP Initialization message for the new 499 instantiation of the LDP session, it MUST also set the FT 500 Reconnect Flag according to whether it has been able to 501 preserve label state. The FT Reconnect Flag is carried 502 in the FT Session TLV. 504 If an LDP peer has preserved all state information for 505 previous instantiations of the LDP session, then it 506 SHOULD set the FT Reconnect Flag to 1 in the FT Session 507 TLV. Otherwise, it MUST set the FT Reconnect Flag to 0. 509 If either LDP peer sets the FT Reconnect Flag to 0, or 510 omits the FT Session TLV, both LDP peers MUST release any 511 state information and resources associated with the 512 previous instantiation of the LDP session between the 513 same LDP peers, including FT Label state and Addresses. 514 This ensures that network resources are not permanently 515 lost by one LSR if its LDP peer is forced to undergo a 516 cold start. 518 If an LDP peer changes any session parameters (for 519 example, the label space bounds) from the previous 520 instantiation the nature of any preserved labels may have 521 changed. In particular, previously allocated labels may 522 now be out of range. For this reason, session 523 reconnection MUST use the same parameters as were in use 524 on the session before the failure. If an LDP peer 525 notices that the parameters have been changed by the 526 other peer it SHOULD send a Notification message with the 527 'FT Session parameters changed' status code. 529 If both LDP peers set the FT Reconnect Flag to 1, both 530 LDP peers MUST use the procedures indicated in this draft 531 to complete any label operations on Sequence Numbered FT 532 Labels that were interrupted by the LDP session failure. 534 If an LDP peer receives an LDP Initialization message 535 with the FT Reconnect Flag set before it sends its own 536 Initialization message, but has retained no information 537 about the previous version of the session, it MUST 538 respond with an Initialization message with the FT 539 Reconnect Flag clear. If an LDP peer receives an LDP 540 Initialization message with the FT Reconnect Flag set in 541 response to an Initialization message that it has sent 542 with the FT Reconnect Flag clear it MUST act as if no 543 state was retained by either peer on the session. 545 4.5. Operations on FT Labels 547 Label operations on Sequence Numbered FT Labels are made 548 Fault Tolerant by providing acknowledgement of all LDP 549 messages that affect Sequence Numbered FT Labels. 550 Acknowledgements are achieved by means of sequence 551 numbers on these LDP messages. 553 The message exchanges used to achieve acknowledgement of 554 label operations and the procedures used to complete 555 interrupted label operations are detailed in the section 556 "FT Operations". 558 Using these acknowledgements and procedures, it is not 559 necessary for LDP peers to perform a complete re- 560 synchronization of state for all Sequence Numbered FT 561 Labels, either on re-connection of the LDP session 562 between the LDP peers or on a timed basis. 564 4.6. Check-Pointing 566 Check-pointing is a useful feature that allows nodes to 567 reduce the amount of processing that they need to do to 568 acknowledge LDP messages. The C bit in the FT Session 569 TLV is used to indicate that checkpointing is supported. 571 Under the normal operation on Sequence Numbered FT 572 Labels, acknowledgments may be deferred during normal 573 processing and only sent periodically. Check-pointing 574 may be used to flush acknowledgement from a peer by 575 including a sequence number on a Keepalive message 576 requesting acknowledgement of that message and all 577 previous messages. In this case, all Sequence Numbered 578 FT Labels are Checkpointable FT Labels. 580 If the S bit is not agreed upon, checkpointing may still 581 be used. In this case it is used to acknowledge all 582 messages exchanged between the peers, and all labels are 583 Checkpointable FT Labels. 585 This offers an approach where acknowledgements need not 586 be sent to every message or even frequently, but are only 587 sent as check-points in response to requests carried on 588 Keepalive messages. Such an approach may be considered 589 optimal in systems that do not show a high degree of 590 change over time (such as targeted LDP sessions) and that 591 are prepared to risk loss of state for the most recent 592 LDP exchanges. More dynamic systems (such as LDP 593 discovery sessions) are more likely to want to 594 acknowledge state changes more frequently so that the 595 maximum amount of state can be preserved over a failure. 597 Note that an important consideration of this draft is 598 that nodes acknowledging messages on a one-for-one basis, 599 nodes deferring acknowledgements, and nodes relying on 600 check-pointing should all interoperate seamlessly and 601 without protocol negotiation beyond session 602 initialization. 604 Further discussion of this feature is provided in the 605 section "FT Operations". 607 4.6.1 Graceful Termination 609 A feature that builds on check-pointing is graceful 610 termination. 612 In some cases, such as controlled failover or software 613 upgrade, it is possible for a node to know in advance 614 that it is going to terminate its session with a peer. 616 In these cases the node that intends terminating the 617 session can flush acknowledgement using a check-point 618 request as described above. The sender SHOULD not send 619 further label or address-related messages after 620 requesting shutdown check-pointing in order to preserve 621 the integrity of its saved state. 623 This, however, only provides for acknowledgement in one 624 direction, and the node that is terminating also requires 625 to know that it has secured all state sent by its peer. 626 This is achieved by a three-way hand shake of the check- 627 point which is requested by an additional TLV (the Cork 628 TLV) in the Keepalive message. 630 Further discussion of this feature is provided in the 631 section "FT Operations". 633 4.7. Label Space Depletion and Replenishment 635 When an LDP peer is unable to satisfy a Label Request 636 message because it has no more available labels, it sends 637 a Notification message carrying the status code 'No label 638 resources'. This warns the requesting LDP peer that 639 subsequent Label Request messages are also likely to fail 640 for the same reason. This message does not need to be 641 acknowledged for FT purposes since Label Request messages 642 sent after session recovery will receive the same 643 response. However, the LDP peer that receives a 'No 644 label resources' Notification stops sending Label Request 645 messages until it receives a 'Label resources available' 646 Notification message. Since this unsolicited 647 Notification might get lost during session failure, it 648 may be protected using the procedures described in this 649 draft. 651 An alternative approach allows that an implementation may 652 always assume that labels are available when a session is 653 re-established. In this case, it is possible that it may 654 throw away the 'No label resources' information from the 655 previous incarnation of the session and may send a batch 656 of LDP messages on session re-establishment that will 657 fail and that it could have known would fail. 659 Note that the sender of a 'Label resources available' 660 Notification message may choose whether to add a sequence 661 number requesting acknowledgement. Conversely, the 662 receiver of 'Label resources available' Notification 663 message may choose to acknowledge the message without 664 actually saving any state. 666 This is an implementation choice made possible by making 667 the FT parameters on the Notification message optional. 668 Implementations will interoperate fully if they take 669 opposite approaches, but additional LDP messages may be 670 sent unnecessarily on session recovery. 672 4.8. Tunneled LSPs 674 The procedures described in this document can be applied 675 to LSPs that are tunneled and to LSPs that are carried by 676 tunnels. Recall that tunneled LSPs are managed by a 677 single LDP session that runs end to end while the tunnel 678 is managed by a different LDP session for each hop along 679 the path. Nevertheless, a break in one of the sessions 680 that manages the tunnel is likely to correspond with a 681 break in the session that manages the tunneled LSP. This 682 is certainly the case when the LDP exchanges share a 683 failed link, but need not be the case if the LDP messages 684 have been routed along a path that is different from that 685 of the tunnel, or if the failure in the tunnel is caused 686 by an LDP software failure at a transit LSR. 688 In order that the forwarding path of a tunneled LSP be 689 preserved, the forwarding path of the tunnel itself must 690 be preserved. This means that the tunnel must not be torn 691 down if there is any session failure along its path. To 692 achieve this the label exchanges between each pair of LDP 693 peers along the path of the tunnel must use one of the 694 procedures in this document or in [LDP-RESTART]. 696 It is perfectly acceptable to mix the restart procedures 697 used for the tunnel and the tunneled LSP. For example, 698 the tunnel could be set up using just check-pointing 699 because it is a stable LSP, but the tunneled LSPs might 700 use full FT procedures so that they can recover active 701 state. 703 Lastly, it is permissible to carry tunneled LSPs that do 704 not have FT protection on an LSP that does have FT 705 protection. 707 5. FT Operations 709 Once an FT LDP session has been established, using the S 710 bit in the FT Session TLV on the Session Initialization 711 message as described in the section "Establishing an FT 712 LDP Session", both LDP peers MUST apply the procedures 713 described in this section for FT LDP message exchanges. 715 If the LDP session has been negotiated to not use the LDP 716 FT enhancements, these procedures MUST NOT be used. 718 5.1. FT LDP Messages 720 5.1.1 Sequence Numbered FT Label Messages 722 A label is identified as being a Sequence Numbered FT 723 Label if the initial Label Request or Label Mapping 724 message relating to that label carries the FT Protection 725 TLV. 727 It is a valid implementation option to flag all labels as 728 Sequence Numbered FT Labels. Indeed this may be a 729 preferred option for implementations wishing to use 730 Keepalive messages carrying the FT Protection TLV to 731 achieve periodic saves of the complete label forwarding 732 state. 734 If a label is a Sequence Numbered FT Label, all LDP 735 messages affecting that label MUST carry the FT 736 Protection TLV in order that the state of the label can 737 be recovered after a failure of the LDP session. 739 A valid option is for no labels to be Sequence Numbered 740 FT Labels. In this case checkpointing using the 741 Keepalive message applies to all messages exchanged on 742 the session. 744 5.1.1.1 Scope of FT Labels 746 The scope of the FT/non-FT status of a label is limited 747 to the LDP message exchanges between a pair of LDP peers. 749 In Ordered Control, when the message is forwarded 750 downstream or upstream, the TLV may be present or absent 751 according to the requirements of the LSR sending the 752 message. 754 If a platform-wide label space is used for FT Labels, an 755 FT Label value MUST NOT be reused until all LDP FT peers 756 to which the label was passed have acknowledged the 757 withdrawal of the FT Label, either by an explicit LABEL 758 WITHDRAW/LABEL RELEASE exchange or implicitly if the LDP 759 session is reconnected after failure but without the FT 760 Reconnect Flag set. In the event that a session is not 761 re-established within the Reconnection Timeout, a label 762 MAY become available for re-use if it is not still in use 763 on some other session. 765 5.1.2 FT Address Messages 767 If an LDP session uses the LDP FT enhancements, both LDP 768 peers MUST secure Address and Address Withdraw messages 769 using FT Operation ACKs, as described below. This avoids 770 any ambiguity over whether an Address is still valid 771 after the LDP session is reconnected. 773 If an LSR determines that an Address message that it sent 774 on a previous instantiation of a recovered LDP session is 775 no longer valid, it MUST explicitly issue an Address 776 Withdraw for that address when the session is 777 reconnected. 779 If the FT Reconnect Flag is not set by both LDP peers on 780 reconnection of an LDP session (i.e. state has not been 781 preserved), both LDP peers MUST consider all Addresses to 782 have been withdrawn. The LDP peers SHOULD issue new 783 Address messages for all their valid addresses as 784 specified in [RFC3036]. 786 5.1.3 Label Resources Available Notifications 788 In LDP, it is possible that a downstream LSR may not have 789 labels available to respond to a Label Request. In this 790 case, as specified in RFC3036, the downstream LSR must 791 respond with a Notification - No Label Resources message. 792 The upstream LSR then suspends asking for new labels 793 until it receives a Notification - Label Resources 794 Available message from the downstream LSR. 796 When the FT extensions are used on a session, 797 implementations may choose whether to secure the label 798 resource state of their peer or not. This choice impacts 799 the number of LDP messages that will be incorrectly 800 routed to a peer with depleted resources on session re- 801 establishment, but does not otherwise impact 802 interoperability. 804 For full preservation of state: 806 - The downstream LSR must preserve the label availability 807 state across a failover so that it remembers to send 808 Notification - Label Resources Available when the resources 809 become available. 811 - The upstream LSR must recall the label availability state 812 across failover so that it can optimize not sending Label 813 Requests when it recovers. 815 - The downstream LSR must use sequence numbers on 816 Notification - Label Resources Available so that it can 817 check that LSR A has received the message and clear its 818 secured state, or resend the message if LSR A recovers 819 without having received it. 821 However, the following options also exist: 823 - The downstream LSR may choose to not include a sequence 824 number on Notification - Label Resources Available. This 825 means that on session re-establishment it does not know what 826 its peer thinks the LSR's resource state is, because the 827 Notification may or may not have been delivered. Such an 828 implementation MUST begin recovered sessions by sending an 829 additional Notification - Label Resources Available to reset 830 its peer. 832 - The upstream node may choose not to secure information 833 about its peer's resource state. It would acknowledge a 834 Notification - Label Resources Available, but would not save 835 the information. Such an implementation MUST assume that 836 its peer's resource state has been reset to Label Resources 837 Available when the session is re-established. 839 If the FT Reconnect Flag is not set by both LDP peers on 840 reconnection of an LDP session (i.e. state has not been 841 preserved), both LDP peers MUST consider the label 842 availability state to have been reset as if the session 843 had been set up for the first time. 845 5.2. FT Operation ACKs 847 Handshaking of FT LDP messages is achieved by use of 848 ACKs. Correlation between the original message and the 849 ACK is by means of the FT Sequence Number contained in 850 the FT Protection TLV, and passed back in the FT ACK TLV. 851 The FT ACK TLV may be carried on any LDP message that is 852 sent on the TCP connection between LDP peers. 854 An LDP peer maintains a separate FT sequence number for 855 each LDP session it participates in. The FT Sequence 856 number is incremented by one for each FT LDP message 857 (i.e. containing the FT Protection TLV) issued by this 858 LSR on the FT LDP session with which the FT sequence 859 number is associated. 861 When an LDP peer receives a message containing the FT 862 Protection TLV, it MUST take steps to secure this message 863 (or the state information derived from processing the 864 message). Once the message is secured, it MUST be ACKed. 865 However, there is no requirement on the LSR to send this 866 ACK immediately. 868 ACKs may be accumulated to reduce the message flow 869 between LDP peers. For example, if an LSR received FT 870 LDP messages with sequence numbers 1, 2, 3, 4, it could 871 send a single ACK with sequence number 4 to ACK receipt 872 and securing of all these messages. There is no protocol 873 reason why the number of ACKs accumulated or the time for 874 which an ACK is deferred should not be allowed to become 875 relatively large. 877 ACKs MUST NOT be sent out of sequence, as this is 878 incompatible with the use of accumulated ACKs. Duplicate 879 ACKs (that is two successive messages that acknowledge 880 the same sequence number) are acceptable. 882 If an LDP peer discovers that its sequence number space 883 for a specific session is full of un-acknowledged 884 sequence numbers (because its partner on the session has 885 not acknowledged them in a timely way) it cannot allocate 886 a new sequence number for any further FT LPD message. It 887 SHOULD send a Notification message with the status code 888 "FT Seq Numbers Exhausted". 890 5.3. Preservation of FT State 892 If the LDP FT enhancements are in use on an LDP session, 893 each LDP peer SHOULD NOT release the state information 894 and resources associated with FT Labels exchanged on that 895 LDP session when the TCP connection fails. This is 896 contrary to [RFC3036], but allows label operations on FT 897 Labels to be completed after re-connection of the TCP 898 connection. 900 Both LDP peers on an LDP session that is using the LDP FT 901 enhancements SHOULD preserve the state information and 902 resources they hold for that LDP session as described 903 below. 905 - An upstream LDP peer SHOULD release the resources (in 906 particular bandwidth) associated with a Sequence Numbered FT 907 Label when it initiates a Label Release or Label Abort 908 message for the label. The upstream LDP peer MUST preserve 909 state information for the Sequence Numbered FT Label, even 910 if it releases the resources associated with the label, as 911 it may need to reissue the label operation if the TCP 912 connection is interrupted. 914 - An upstream LDP peer MUST release the state information 915 and resources associated with a Sequence Numbered FT Label 916 when it receives an acknowledgement to a Label Release or 917 Label Abort message that it sent for the label, or when it 918 sends a Label Release message in response to a Label 919 Withdraw message received from the downstream LDP peer. 921 - A downstream LDP peer SHOULD NOT release the resources 922 associated with a Sequence Numbered FT Label when it sends a 923 Label Withdraw message for the label as it has not yet 924 received confirmation that the upstream LDP peer has ceased 925 to send data using the label. The downstream LDP peer MUST 926 NOT release the state information it holds for the label as 927 it may yet have to reissue the label operation if the TCP 928 connection is interrupted. 930 - A downstream LDP peer MUST release the resources and 931 state information associated with a Sequence Numbered FT 932 Label when it receives an acknowledgement to a Label 933 Withdraw message for the label. 935 - When the FT Reconnection Timeout expires, an LSR SHOULD 936 release all state information and resources from previous 937 instantiations of the (permanently) failed LDP session. 939 - Either LDP peer MAY elect to release state information 940 based on its internal knowledge of the loss of integrity of 941 the state information or an inability to pend (or queue) LDP 942 operations (as described in section 4.4.1) during a TCP 943 failure. That is, the peer is not required to wait for the 944 duration of the FT Reconnection Timeout before releasing 945 state; the timeout provides an upper limit on the 946 persistence of state. However, in the event that a peer 947 releases state before the expiration of the Reconnection 948 Timeout it MUST NOT re-use any label that was in use on the 949 session until the Reconnection Timeout has expired. 951 - When an LSR receives a Status TLV with the E-bit set in 952 - When an LSR receives a Status TLV with the E-bit set in 953 the status code, which causes it to close the TCP 954 connection, the LSR MUST release all state information and 955 resources associated with the session. This behavior is 956 mandated because it is impossible for the LSR to predict the 957 precise state and future behavior of the partner LSR that 958 set the E-bit without knowledge of the implementation of 959 that partner LSR. 961 Note that the "Temporary Shutdown" status code does not have 962 the E-bit set, and MAY be used during maintenance or upgrade 963 operations to indicate that the LSR intends to preserve 964 state across a closure and re-establishment of the TCP 965 session. 967 - If an LSR determines that it must release state for any 968 single FT Label during a failure of the TCP connection on 969 which that label was exchanged, it MUST release all state 970 for all labels on the LDP session. 972 The release of state information and resources associated 973 with non-FT labels is as described in [RFC3036]. 975 Note that a Label Release and the acknowledgement to a 976 Label Withdraw may be received by a downstream LSR in any 977 order. The downstream LSR MAY release its resources on 978 receipt of the first message and MUST release its 979 resources on receipt of the second message. 981 5.4. FT Procedure After TCP Failure 983 When an LSR discovers or is notified of a TCP connection 984 failure it SHOULD start an FT Reconnection Timer to allow 985 a period for re-connection of the TCP connection between 986 the LDP peers. 988 The RECOMMENDED default value for this timer is 5 989 seconds. During this time, failure must be detected and 990 reported, new hardware may need to be activated, software 991 state must be audited, and a new TCP session must be set 992 up. 994 Once the TCP connection between LDP peers has failed, the 995 active LSR SHOULD attempt to re-establish the TCP 996 connection. The mechanisms, timers and retry counts to re- 997 establish the TCP connection are an implementation 998 choice. It is RECOMMENDED that any attempt to re- 999 establish the connection take account of the failover 1000 processing necessary on the peer LSR, the nature of the 1001 network between the LDP peers, and the FT Reconnection 1002 Timeout chosen on the previous instantiation of the TCP 1003 connection (if any). 1005 If the TCP connection cannot be re-established within the 1006 FT Reconnection Timeout period, the LSR detecting this 1007 timeout SHOULD release all state preserved for the failed 1008 LDP session. If the TCP connection is subsequently re- 1009 established (for example, after a further Hello exchange 1010 to set up a new LDP session), the LSR MUST set the FT 1011 Reconnect Flag to 0 if it released the preserved state 1012 information on this timeout event. 1014 If the TCP connection is successfully re-established 1015 within the FT Reconnection Timeout, both peers MUST re- 1016 issue LDP operations that were interrupted by (that is, 1017 un-acknowledged as a result of) the TCP connection 1018 failure. This procedure is described in section "FT 1019 Procedure After TCP Re-connection". 1021 The Hold Timer for an FT LDP Session (see [RFC3036] 1022 section 2.5.5) SHOULD be ignored while the FT 1023 Reconnection Timer is running. The hold timer SHOULD be 1024 restarted when the TCP connection is re-established. 1026 5.4.1 FT LDP Operations During TCP Failure 1028 When the LDP FT enhancements are in use for an LDP 1029 session, it is possible that an LSR may determine that it 1030 needs to send an LDP message to an LDP peer but that the 1031 TCP connection to that peer is currently down. These 1032 label operations affect the state of FT Labels preserved 1033 for the failed TCP connection, so it is important that 1034 the state changes are passed to the LDP peer when the TCP 1035 connection is restored. 1037 If an LSR determines that it needs to issue a new FT LDP 1038 operation to an LDP peer to which the TCP connection is 1039 currently failed, it MUST pend the operation (e.g. on a 1040 queue) and complete that operation with the LDP peer when 1041 the TCP connection is restored, unless the label 1042 operation is overridden by a subsequent additional 1043 operation during the TCP connection failure (see section 1044 "FT Procedure After TCP Re-connection"). 1046 If, during TCP Failure, an LSR determines that it cannot 1047 pend an operation which it cannot simply fail (for 1048 example, a Label Withdraw, Release or Abort operation), 1049 it MUST NOT attempt to re-establish the previous LDP 1050 session. The LSR MUST behave as if the Reconnection 1051 Timer expired and release all state information with 1052 respect to the LDP peer. An LSR may be unable (or 1053 unwilling) to pend operations; for instance, if a major 1054 routing transition occurred while TCP was inoperable 1055 between LDP peers it might result in excessively large 1056 numbers of FT LDP Operations. An LSR that releases state 1057 before the expiration of the Reconnection Timeout MUST 1058 NOT re-use any label that was in use on the session until 1059 the Reconnection Timeout has expired. 1061 In ordered operation, received FT LDP operations that 1062 cannot be correctly forwarded because of a TCP connection 1063 failure MAY be processed immediately (provided sufficient 1064 state is kept to forward the label operation) or pended 1065 for processing when the onward TCP connection is restored 1066 and the operation can be correctly forwarded upstream or 1067 downstream. Operations on existing FT Labels SHOULD NOT 1068 be failed during TCP session failure. 1070 It is RECOMMENDED that Label Request operations for new 1071 FT Labels are not pended awaiting the re-establishment of 1072 TCP connection that is awaiting recovery at the time the 1073 LSR determines that it needs to issue the Label Request 1074 message. Instead, such Label Request operations SHOULD 1075 be failed and, if necessary, a notification message 1076 containing the "No LDP Session" status code sent 1077 upstream. 1079 Label Requests for new non-FT Labels MUST be rejected 1080 during TCP connection failure, as specified in [RFC3036]. 1082 5.5. FT Procedure After TCP Re-connection 1084 The FT operation handshaking described above means that 1085 all state changes for Sequence Numbered FT Labels and 1086 Address messages are confirmed or reproducible at each 1087 LSR. 1089 If the TCP connection between LDP peers fails but is re- 1090 connected within the FT Reconnection Timeout, and both 1091 LSRs have indicated they will be re-establishing the 1092 previous LDP session, both LDP peers on the connection 1093 MUST complete any label operations for Sequence Numbered 1094 FT Labels that were interrupted by the failure and re- 1095 connection of the TCP connection. 1097 The procedures for FT Reconnection Timeout MAY have been 1098 invoked as a result of either LDP peer being unable (or 1099 unwilling) to pend operations which occurred during the 1100 TCP Failure (as described in section 4.4.1). 1102 If, for any reason, an LSR has been unable to pend 1103 operations with respect to an LDP peer, as described in 1104 section 4.4.1, the LSR MUST set the FT Reconnect Flag to 1105 0 on re-connection to that LDP peer indicating that no FT 1106 state has been preserved. 1108 Label operations are completed using the procedure 1109 described below. 1111 5.5.1 Re-Issuing FT Messages 1113 On restoration of the TCP connection between LDP peers, 1114 any LDP messages for Sequence Numbered FT Labels that 1115 were lost because of the TCP connection failure are re- 1116 issued. The LDP peer that receives a re-issued message 1117 processes the message as if received for the first time. 1119 "Net-zero" combinations of messages need not be re-issued 1120 after re-establishment of the TCP connection between LDP 1121 peers. This leads to the following rules for re-issuing 1122 messages that are not ACKed by the LDP peer on the LDP 1123 Initialization message exchange after re-connection of 1124 the TCP session. 1126 - A Label Request message MUST be re-issued unless a Label 1127 Abort would be re-issued for the same Sequence Numbered FT 1128 Label. 1130 - A Label Mapping message MUST be re-issued unless a Label 1131 Withdraw message would be re-issued for the same Sequence 1132 Numbered FT Label. 1134 - All other messages on the LDP session that carried the FT 1135 Protection TLV MUST be re-issued if an acknowledgement had 1136 not previously been received. 1138 Any FT Label operations that were pended (see section 1139 4.4.1) during the TCP connection failure MUST also be 1140 issued on re-establishment of the LDP session, except 1141 where they form part of a "net-zero" combination of 1142 messages according to the above rules. 1144 The determination of "net-zero" FT Label operations 1145 according to the above rules MAY be performed on pended 1146 messages prior to the re-establishment of the TCP 1147 connection in order to optimize the use of queue 1148 resources. Messages that were sent to the LDP peer 1149 before the TCP connection failure, or pended messages 1150 that are paired with them, MUST NOT be subject to such 1151 optimization until an FT ACK TLV is received from the LDP 1152 peer. This ACK allows the LSR to identify which messages 1153 were received by the LDP peer prior to the TCP connection 1154 failure. 1156 6. Checkpointing Procedures 1158 Checkpointing can be selected independently from the FT 1159 procedures described above by using the C bit in the FT 1160 Session TLV on the Session Initialization message. Note, 1161 however, that checkpointing is an integral part of the FT 1162 procedures. Setting the S and the C bit will achieve the 1163 same function as setting just the S bit. 1165 If the C bit is set, but the S bit is not set, no label 1166 is a Sequence Numbered FT Label. Instead, all labels are 1167 Checkpointable FT Labels. Checkpointing is used to 1168 synchronize all labels exchanges. No message apart from 1169 the checkpoint request and acknowledgement carries an 1170 active sequence number. (Note that the Session 1171 Initialization message may carry a sequence number to 1172 confirm that the checkpoint is still in place). 1174 It is an implementation matter to decide the ordering of 1175 received messages and checkpoint requests to ensure that 1176 checkpoint acknowledgements are secured. 1178 If the S and C bits are both set, or only the S bit is 1179 set, checkpointing applies only to Sequence Numbered FT 1180 Labels and to address messages. 1182 The set of all messages that are checkpointed in this way 1183 is called the Checkpointable Messages. 1185 6.1 Checkpointing with the Keepalive Message 1187 If an LSR receives a FT Protection TLV on a Keepalive 1188 message, this is a request to flush the acknowledgements 1189 for all previously received Checkpointable Messages on 1190 the session. 1192 As soon as the LSR has completed securing the 1193 Checkpointable Messages (or state changes consequent on 1194 those messages) received before the Keepalive, it MUST 1195 send an acknowledgement to the sequence number of the 1196 Keepalive message. 1198 In the case where the FT procedures are in use and 1199 acknowledgements have been stored up, this may be 1200 immediately on receipt of the Keepalive. 1202 An example message flow showing this use of the Keepalive 1203 message to perform a periodic check-point of state is 1204 shown in section 8. 1206 An example message flow showing the use of checkpointing 1207 without the FT procedures is shown in section 8. 1209 6.2 Quiesce and Keepalive 1211 If the Keepalive Message also contains the FT Cork TLV, 1212 this indicates that the peer LSR wishes to quiesce the 1213 session prior to a graceful restart. 1215 It is RECOMMENDED that on receiving a Keepalive with the 1216 FT CORK TLV, an LSR should cease to send any further 1217 label or address related messages on the session until it 1218 has been disconnected and reconnected, other than any 1219 messages generated while processing and securing any 1220 previously unacknowledged messages received from the peer 1221 requesting the quiesce. It should also attempt to 1222 complete this processing and return a Keepalive with the 1223 FT ACK TLV as soon as possible in order to allow the 1224 session to be quiesced. 1226 An example message flow showing this use of the FT Cork 1227 TLV to achieves three-way handshake of state 1228 synchronization between two LDP peers is given in section 8. 1230 7. Changes to Existing Messages 1232 7.1. LDP Initialization Message 1234 The LDP FT enhancements add the following optional 1235 parameters to a LDP Initialization message 1237 Optional Parameter Length Value 1239 FT Session TLV 4 See below 1240 FT ACK TLV 4 See below 1242 The encoding for these TLVs is found in Section "New 1243 Fields and Values". 1245 FT Session TLV 1246 If present, specifies the FT behavior of 1247 the LDP session. 1249 FT ACK TLV 1250 If present, specifies the last FT message 1251 that the sending LDP peer was able to 1252 secure prior to the failure of the previous 1253 instantiation of the LDP session. This TLV 1254 is only present if the FT Reconnect flag is 1255 set in the FT Session TLV, in which case 1256 this TLV MUST be present. 1258 7.2. LDP Keepalive Messages 1260 The LDP FT enhancements add the following optional 1261 parameters to a LDP Keepalive message 1263 Optional Parameter Length Value 1265 FT Protection TLV 4 See below 1266 FT Cork TLV 0 See below 1267 FT ACK TLV 4 See below 1269 The encoding for these TLVs is found in Section "New 1270 Fields and Values". 1272 FT Protection TLV 1273 If present, specifies FT Sequence Number 1274 for the LDP message. When present on a 1275 Keepalive message, this indicates a 1276 solicited flush of the acknowledgements to 1277 all previous LDP messages containing 1278 sequence numbers and issued by the sender 1279 of the Keepalive on the same session. 1281 FT Cork TLV 1282 Indicates that the remote LSR wishes to 1283 quiesce the LDP session. See section 5 for 1284 the recommended action in such cases. 1286 FT ACK TLV 1287 If present, specifies the most recent FT 1288 message that the sending LDP peer has been 1289 able to secure. 1291 7.3. All Other LDP Session Messages 1293 The LDP FT enhancements add the following optional 1294 parameters to all other message types that flow on an LDP 1295 session after the LDP Initialization message 1297 Optional Parameter Length Value 1299 FT Protection TLV 4 See below 1300 FT ACK TLV 4 See below 1302 The encoding for these TLVs is found in the section "New 1303 Fields and Values". 1305 FT Protection TLV 1306 If present, specifies FT Sequence Number 1307 for the LDP message. 1309 FT ACK TLV 1310 If present, identifies the most recent FT 1311 LDP message ACKed by the sending LDP peer. 1313 8. New Fields and Values 1315 8.1. Status Codes 1317 The following new status codes are defined to indicate 1318 various conditions specific to the LDP FT enhancements. 1319 These status codes are carried in the Status TLV of a 1320 Notification message. 1322 The "E" column is the required setting of the Status Code 1323 E-bit; the "Status Data" column is the value of the 30- 1324 bit Status Data field in the Status Code TLV. 1326 Note that the setting of the Status Code F-bit is at the 1327 discretion of the LSR originating the Status TLV. 1328 However, it is RECOMMENDED that the F-bit is not set on 1329 Notification messages containing status codes except "No 1330 LDP Session" because the duplication of messages SHOULD 1331 be restricted to being a per-hop behavior. 1333 Status Code E Status Data 1335 No LDP Session 0 0x000000xx 1336 Zero FT seqnum 1 0x000000xx 1337 Unexpected TLV / 1 0x000000xx 1338 Session Not FT 1339 Unexpected TLV / 1 0x000000xx 1340 Label Not FT 1341 Missing FT Protection TLV 1 0x000000xx 1342 FT ACK sequence error 1 0x000000xx 1343 Temporary Shutdown 0 0x000000xx 1345 FT Seq Numbers Exhausted 1 0x000000xx 1346 FT Session parameters / 1 0x000000xx 1347 changed 1348 Unexpected FT Cork TLV 1 0x000000xx 1350 The Temporary Shutdown status code SHOULD be used in 1351 place of the Shutdown status code (which has the E-bit 1352 set) if the LSR that is shutting down wishes to inform 1353 its LDP peer that it expects to be able to preserve FT 1354 Label state and to return to service before the FT 1355 Reconnection Timer expires. 1357 8.2. FT Session TLV 1359 LDP peers can negotiate whether the LDP session between 1360 them supports FT extensions by using a new OPTIONAL 1361 parameter, the FT Session TLV, on LDP Initialization 1362 Messages. 1364 The FT Session TLV is encoded as follows. 1366 0 1 2 3 1367 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1368 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1369 |1|0| FT Session TLV (0x0503) | Length (= 12) | 1370 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1371 | FT Flags | Reserved | 1372 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1373 | FT Reconnect Timeout (in milliseconds) | 1374 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1375 | Recovery Time (in milliseconds) | 1376 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1377 FT Flags 1378 FT Flags: A 16 bit field that indicates 1379 various attributes the FT support on this 1380 LDP session. This fields is formatted as 1381 follows: 1383 0 1 1384 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1386 |R| Reserved |S|A|C|L| 1387 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1389 R: FT Reconnect Flag. 1390 Set to 1 if the sending LSR has 1391 preserved state and resources for all 1392 FT-labels since the previous LDP 1393 session between the same LDP peers, 1394 and set to 0 otherwise. See the 1395 section "FT LDP Session Reconnection" 1396 for details of how this flag is used. 1398 If the FT Reconnect Flag is set, the 1399 sending LSR MUST include an FT ACK TLV 1400 on the LDP Initialization message. 1402 S: Save State Flag. 1403 Set to 1 if the use of the FT 1404 Protection TLV is supported on 1405 messages other than the KeepAlive 1406 message used for chekpointing (see the 1407 C bit). I.e., the S bit indicates 1408 that some label on the session may be 1409 a Sequence Numbered FT Label. 1411 A: All-Label Protection Required 1412 Set to 1 if all labels on the session 1413 MUST be treated as Sequence Numbered 1414 FT Labels. This removes from a node 1415 the option of treating some labels as 1416 FT Labels and some labels as non-FT 1417 Labels. 1419 Passing this information may be 1420 considered helpful to a peer since it 1421 may allow it to make optimizations in 1422 its processing. 1424 The A bit only has meaning if the S 1425 bit is set. 1427 C: Checkpointing Flag. 1428 Set to 1 to indicate that the 1429 checkpointing procedures in this draft 1430 are in use. 1432 If the S bit is also set to 1 then the 1433 C bit indicates that checkpointing is 1434 applied only to Sequence Numbered FT 1435 Labels. 1437 If the S bit is set to 0 (zero) then 1438 the C bit indicates that checkpointing 1439 applies to all labels - all labels are 1440 Checkpointable FT Labels. 1442 L: Learn From Network Flag. 1443 Set to 1 if the Fault Recovery 1444 procedures of [LDP-RESTART] are to be 1445 used to re-learn state from the 1446 network. 1448 It is not valid for all of the S, C and L 1449 bits to be zero. 1451 It is not valid for both the L and either 1452 the S or C bits to be set to 1. 1454 All other bits in this field are currently 1455 reserved and SHOULD be set to zero on 1456 transmission and ignored on receipt. 1458 The following table summarizes the settings 1459 of these bits. 1461 S A C L Comments 1462 ========================= 1463 0 x 0 0 Invalid 1464 0 0 0 1 See [LDP-RESTART] 1465 0 1 0 1 Invalid 1466 0 x 1 0 Checkpointing of all labels 1467 0 x 1 1 Invalid 1468 1 0 0 0 Full FT on selected labels 1469 1 1 0 0 Full FT on all labels 1470 1 x 0 1 Invalid 1471 1 x 1 0 Same as (S=1,A=x,C=0,L=0) 1472 1 x 1 1 Invalid. 1474 FT Reconnection Timeout 1475 If the S bit or C bit in the FT Flags field 1476 is set this indicates the period of time 1477 the sending LSR will preserve state and 1478 resources for FT Labels exchanged on the 1479 previous instantiation of an FT LDP session 1480 that has currently failed. The timeout is 1481 encoded as a 32-bit unsigned integer number 1482 of milliseconds. 1484 A value of zero in this field means that 1485 the sending LSR will preserve state and 1486 resources indefinitely. 1488 See section 4.4 for details of how this 1489 field is used. 1491 If the L bit is set to 1 in the FT Flags 1492 field, the meaning of this field is defined 1493 in [LDP-RESTART]. 1495 Recovery Time 1496 The Recovery Time only has meaning if the L 1497 bit is set in the FT Flags. The meaning is 1498 defined in [LDP-RESTART]. 1500 8.3. FT Protection TLV 1502 LDP peers use the FT Protection TLV to indicate that an 1503 LDP message contains an FT label operation. 1505 The FT Protection TLV MUST NOT be used in messages 1506 flowing on an LDP session that does not support the LDP 1507 FT enhancements. Its presence in such messages SHALL be 1508 treated as a protocol error by the receiving LDP peer 1509 which SHOULD send a Notification message with the 1510 'Unexpected TLV Session Not FT' status code. LSRs that 1511 do not recognize this TLV SHOULD respond with a 1512 Notification message with the 'Unknown TLV' status code. 1514 The FT Protection TLV MAY be carried on an LDP message 1515 transported on the LDP session after the initial exchange 1516 of LDP Initialization messages. In particular, this TLV 1517 MAY optionally be present on the following messages: 1519 - Label Request Messages in downstream on-demand 1520 distribution mode 1522 - Label Mapping messages in downstream unsolicited mode 1524 - Keepalive messages used to request flushing of 1525 acknowledgement of all previous messages that contained this 1526 TLV. 1528 If a label is to be a Sequence Numbered FT Label, then 1529 the Protection TLV MUST be present: 1531 - on the Label Request message in DoD mode 1533 - on the Label Mapping message in DU mode 1535 - on all subsequent messages concerning this label. 1537 Here 'subsequent messages concerning this label' means 1538 any message whose Label TLV specifies this label or whose 1539 Label Request Message ID TLV specifies the initial Label 1540 Request message. 1542 If a label is not to be a Sequence Numbered FT Label, 1543 then the Protection TLV MUST NOT be present on any of 1544 these messages that relate to the label. The presence of 1545 the FT TLV on a message relating to a non-FT Label SHALL 1546 be treated as a protocol error by the receiving LDP peer 1547 which SHOULD send a notification message with the 1548 'Unexpected TLV Label Not FT' status code. 1550 Where a Label Withdraw or Label Release message contains 1551 only a FEC TLV and does not identify a single specific 1552 label, the FT TLV should be included in the message if any 1553 label affected by the message is a Sequence Numbered FT 1554 Label. If there is any doubt as to whether an FT TLVshould 1555 be present, it is RECOMMENDED that the sender add the TLV. 1557 When an LDP peer receives a Label Withdraw Message or 1558 Label Release message that contains only a FEC, it SHALL 1559 accept the FT TLV if it is present regardless of the FT 1560 status of the labels which it affects. 1562 If an LDP session is an FT session as determined by the 1563 presence of the FT Session TLV with the S bit set on the 1564 LDP Initialization messages, the FT Protection TLV MUST 1565 be present on all Address messages on the session. 1567 If the session is an FT session, the FT Protection TLV 1568 may also optionally be present 1570 - on Notification messages on the session that have the 1571 status code 'Label Resources Available' 1573 - on Keepalive messages. 1575 The FT Protection TLV is encoded as follows. 1577 0 1 2 3 1578 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1579 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1580 |0|0| FT Protection (0x0203) | Length (= 4) | 1581 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1582 | FT Sequence Number | 1583 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1585 FT Sequence Number 1586 The sequence number for this Sequence Numbered 1587 FT Label operation. The sequence number is 1588 encoded as a 32-bit unsigned integer. The 1589 initial value for this field on a new LDP 1590 session is 0x00000001 and is incremented by 1591 one for each FT LDP message issued by the 1592 sending LSR on this LDP session. This field 1593 may wrap from 0xFFFFFFFF to 0x00000001. 1595 This field MUST be reset to 0x00000001 if 1596 either LDP peer does not set the FT 1597 Reconnect Flag on re-establishment of the 1598 TCP connection. 1600 See the section "FT Operation Acks" for 1601 details of how this field is used. 1603 The special use of 0x00000000 is discussed 1604 in the section "FT ACK TLV" below. 1606 If an LSR receives an FT Protection TLV on a session that 1607 does not support the FT LDP enhancements, it SHOULD send 1608 a Notification message to its LDP peer containing the 1609 "Unexpected TLV, Session Not FT" status code. LSRs that 1610 do not recognize this TLV SHOULD respond with a 1611 Notification message with the 'Unknown TLV' status code. 1613 If an LSR receives an FT Protection TLV on an operation 1614 affecting a label that it believes is a non-FT Label, it 1615 SHOULD send a Notification message to its LDP peer 1616 containing the "Unexpected TLV, Label Not FT" status 1617 code. 1619 If an LSR receives a message without the FT Protection 1620 TLV affecting a label that it believes is a Sequence 1621 Numbered FT Label, it SHOULD send a Notification message 1622 to its LDP peer containing the "Missing FT Protection 1623 TLV" status code. 1625 If an LSR receives an FT Protection TLV containing a zero 1626 FT Sequence Number, it SHOULD send a Notification message 1627 to its LDP peer containing the "Zero FT Seqnum" status 1628 code. 1630 8.4. FT ACK TLV 1632 LDP peers use the FT ACK TLV to acknowledge FT Label 1633 operations. 1635 The FT ACK TLV MUST NOT be used in messages flowing on an 1636 LDP session that does not support the LDP FT 1637 enhancements. Its presence on such messages SHALL be 1638 treated as a protocol error by the receiving LDP peer. 1640 The FT ACK TLV MAY be present on any LDP message 1641 exchanged on an LDP session after the initial LDP 1642 Initialization messages. It is RECOMMENDED that the FT 1643 ACK TLV is included on all FT Keepalive messages in order 1644 to ensure that the LDP peers do not build up a large 1645 backlog of unacknowledged state information. 1647 The FT ACK TLV is encoded as follows. 1649 0 1 2 3 1650 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1651 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1652 |0|0| FT ACK (0x0504) | Length (= 4) | 1653 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1654 | FT ACK Sequence Number | 1655 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1657 FT ACK Sequence Number 1658 The sequence number for the most recent FT 1659 label message that the sending LDP peer has 1660 received from the receiving LDP peer and 1661 secured against failure of the LDP session. 1662 It is not necessary for the sending peer to 1663 have fully processed the message before 1664 ACKing it. For example, an LSR MAY ACK a 1665 Label Request message as soon as it has 1666 securely recorded the message, without 1667 waiting until it can send the Label Mapping 1668 message in response. 1670 ACKs are cumulative. Receipt of an LDP 1671 message containing an FT ACK TLV with an FT 1672 ACK Sequence Number of 12 is treated as the 1673 acknowledgement of all messages from 1 to 1674 12 inclusive (assuming the LDP session 1675 started with a sequence number of 1). 1677 This field MUST be set to 0 if the LSR 1678 sending the FT ACK TLV has not received any 1679 FT label operations on this LDP session. 1680 This would apply to LDP sessions to new LDP 1681 peers or after an LSR determines that it 1682 must drop all state for a failed TCP 1683 connection. 1685 See the section "FT Operation Acks" for 1686 details of how this field is used. 1688 If an LSR receives a message affecting a label that it 1689 believes is a Sequence Numbered FT Label and that message 1690 does not contain the FT Protection TLV, it SHOULD send a 1691 Notification message to its LDP peer containing the 1692 "Missing FT Protection TLV" status code. 1694 If an LSR receives an FT ACK TLV that contains an FT ACK 1695 Sequence Number that is less than the previously received 1696 FT ACK Sequence Number (remembering to take account of 1697 wrapping), it SHOULD send a Notification message to its 1698 LDP peer containing the "FT ACK Sequence Error" status 1699 code. 1701 8.5. FT Cork TLV 1703 LDP peers use the FT Cork TLV on FT Keepalive messages to 1704 indicate that they wish to quiesce the LDP session prior 1705 to a controlled shutdown and restart, for example during 1706 control-plane software upgrade. 1708 The FT Cork TLV is encoded as follows. 1710 0 1 2 3 1711 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1712 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1713 |0|0| FT Cork (0x0505) | Length (= 0) | 1714 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1716 On receipt of a Keepalive message with the FT Cork TLV 1717 and the FT Protection TLV, an LSR SHOULD perform the 1718 following actions 1720 - Process and secure any messages from the peer LSR that 1721 have sequence numbers less than (accounting for wrap) that 1722 contained in the FT Protection TLV on the Keepalive message. 1724 - Send a Keepalive message back to the peer containing the 1725 FT Cork TLV and the FT ACK TLV specifying the FT ACK 1726 sequence number equal to that in the original Keepalive 1727 message (i.e. ACKing all messages up to that point). 1729 - If this LSR has not yet received an FT ACK to all the 1730 messages it has sent containing the FT Protection TLV, then 1731 also include an FT Protection TLV on the Keepalive sent to 1732 the peer LSR. This tells the remote peer that the local LSR 1733 has saved state prior to quiesce but is still awaiting 1734 confirmation that the remote peer has saved state. 1736 - Cease sending any further state changing messages on this 1737 LDP session until it has been disconnected and recovered. 1739 On receipt of a Keepalive message with the FT Cork TLV 1740 and an FT ACK TLV that acknowledges the previously sent 1741 Keepalive that carried the FT Cork TLV, an LSR knows that 1742 quiesce is complete. If the received Keepalive also 1743 carries the FT Protection TLV, the LSR must respond with 1744 a further Keepalive to complete the 3-way handshake. It 1745 SHOULD now send a "Temporary Shutdown" Notification 1746 message, disconnect the TCP session and perform whatever 1747 control plane actions required this session shutdown. 1749 An example such 3-way handshake for controlled shutdown 1750 is given in section 8. 1752 If an LSR receives a message that should not carry the FT 1753 Cork TLV, or if the FT Cork TLV is used on a Keepalive 1754 message without one of the FT Protection or FT ACK TLVs 1755 present, , it SHOULD send a Notification message to its 1756 LDP peer containing the "Unexpected FR Cork TLV" status code. 1758 9. Example Use 1760 Consider two LDP peers, P1 and P2, implementing LDP over 1761 a TCP connection that connects them, and the message flow 1762 shown below. 1764 The parameters shown on each message shown below are as 1765 follows: 1767 message (label, senders FT sequence number, FT ACK 1768 number) 1770 A "-" for FT ACK number means that the FT ACK TLV is 1771 not included on that message. "n/a" means that the 1772 parameter in question is not applicable to that type 1773 of message. 1775 In the diagrams below, time flows from top to bottom. 1776 The relative position of each message shows when it is 1777 transmitted. See the notes for a description of when 1778 each message is received, secured for FT or processed. 1780 9.1. Session Failure and Recovery - FT Procedures 1782 notes P1 P2 1783 ===== == == 1784 (1) Label Request(L1,27,-) 1785 ---------------------------> 1786 Label Request(L2,28,-) 1787 ---------------------------> 1788 (2) Label Request(L3,93,27) 1789 <--------------------------- 1790 (3) Label Request(L1,123,-) 1791 --------------------------> 1792 Label Request(L2,124,-) 1793 --------------------------> 1794 (4) Label Mapping(L1,57,-) 1795 <-------------------------- 1796 Label Mapping(L1,94,28) 1797 <--------------------------- 1798 (5) Label Mapping(L2,58,-) 1799 <-------------------------- 1800 Label Mapping(L2,95,-) 1801 <--------------------------- 1802 (6) Address(n/a,29,-) 1803 ---------------------------> 1804 (7) Label Request(L4,30,-) 1805 ---------------------------> 1806 (8) Keepalive(n/a,-,94) 1807 ---------------------------> 1808 (9) Label Abort(L3,96,-) 1809 <--------------------------- 1810 (10) ===== TCP Session lost ===== 1811 : 1812 (11) : Label Withdraw(L1,59,-) 1813 : <-------------------------- 1814 : 1815 (12) === TCP Session restored === 1817 LDP Init(n/a,n/a,94) 1818 ---------------------------> 1819 LDP Init(n/a,n/a,29) 1820 <--------------------------- 1821 (13) Label Request(L4,30,-) 1822 ---------------------------> 1823 (14) Label Mapping(L2,95,-) 1824 <--------------------------- 1825 Label Abort(L3,96,30) 1826 <--------------------------- 1827 (15) Label Withdraw(L1,97,-) 1828 <--------------------------- 1830 Notes: 1831 ====== 1833 (1) Assume that the LDP session has already been initialized. 1834 P1 issues 2 new Label Requests using the next sequence numbers. 1836 (2) P2 issues a third Label request to P1. At the time of sending 1837 this request, P2 has secured the receipt of the label request 1838 for L1 from P1, so it includes an ACK for that message. 1840 (3) P2 Processes the Label Requests for L1 and L2 and forwards them 1841 downstream. Details of downstream processing are not shown in 1842 the diagram above. 1844 (4) P2 receives a Label Mapping from downstream for L1, which it 1845 forwards to P1. It includes an ACK to the Label Request for L2, 1846 as that message has now been secured and processed. 1848 (5) P2 receives the Label Mapping for L2, which it forwards to P1. 1849 This time it does not include an ACK as it has not received any 1850 further messages from P1. 1852 (6) Meanwhile, P1 sends a new Address Message to P2 . 1854 (7) P1 also sends a fourth Label Request to P2 1856 (8) P1 sends a Keepalive message to P2, on which it includes an ACK 1857 for the Label Mapping for L1, which is the latest message P1 has 1858 received and secured at the time the Keepalive is sent. 1860 (9) P2 issues a Label Abort for L3. 1862 (10) At this point, the TCP session goes down. 1864 (11) While the TCP session is down, P2 receives a Label Withdraw 1865 Message for L1, which it queues. 1867 (12) The TCP session is reconnected and P1 and P2 exchange LDP 1868 Initialization messages on the recovered session, which include 1869 ACKS for the last message each peer received and secured prior 1870 to the failure. 1872 (13) From the LDP Init exchange, P1 determines that it needs to 1873 re-issue the Label request for L4. 1875 (14) Similarly, P2 determines that it needs to re-issue the Label 1876 Mapping for L2 and the Label Abort. 1878 (15) P2 issues the queued Label Withdraw to P1. 1880 9.2. Use of Check-Pointing With FT Procedures 1882 notes P1 P2 1883 ===== == == 1884 (1) Label Request(L1,27,-) 1885 ---------------------------> 1886 Label Request(L2,28,-) 1887 ---------------------------> 1888 (2) Label Request(L3,93,-) 1889 <--------------------------- 1890 (3) Label Request(L1,123,-) 1891 --------------------------> 1892 Label Request(L2,124,-) 1893 --------------------------> 1894 (4) Label Mapping(L1,57,-) 1895 <-------------------------- 1896 Label Mapping(L1,94,-) 1897 <--------------------------- 1898 (5) Label Mapping(L2,58,-) 1899 <-------------------------- 1900 Label Mapping(L2,95,-) 1901 <--------------------------- 1902 (6) Address(n/a,29,-) 1903 ---------------------------> 1904 (7) Label Request(L4,30,-) 1905 ---------------------------> 1906 (8) Keepalive(n/a,31,-) 1907 ---------------------------> 1908 (9) Keepalive(n/a,-,31) 1909 <--------------------------- 1910 (10) Keepalive(n/a,59,124) 1911 <--------------------------- 1912 (11) Keepalive(n/a,-,59) 1913 ---------------------------> 1915 Notes: 1916 ====== 1918 Notes (1) through (7) are as in the previous example 1919 except note that no acknowledgements are piggy-backed on 1920 reverse direction messages. This means that at note (8) 1921 there are deferred acknowledgements in both directions on 1922 both links. 1924 (8) P1 wishes to synchronize state with P2. It sends a Keepalive 1925 message containing a FT Protection TLV with sequence number 31. 1926 Since it is not interested in P2's perception of the state that 1927 it has stored, it does not include an FT ACK TLV. 1929 (9) P2 responds at once with a Keepalive acknowledging the sequence 1930 number on the received Keepalive. This tells P1 that P2 has 1931 preserved all state/messages previously received on this 1932 session. 1934 (10) P3 wishes to synchronize state with P2. It sends a Keepalive 1935 message containing a FT Protection TLV with sequence number 59. 1936 P3 also takes this opportunity to get up to date with its 1937 acknowledgements to P2 by including an FT ACK TLV acknowledging 1938 up to sequence number 124. 1940 (11) P2 responds at once with a Keepalive acknowledging the sequence 1941 number on the received Keepalive. 1943 9.3. Temporary Shutdown With FT Procedures 1945 notes P1 P2 1946 ===== == == 1947 (1) Label Request(L1,27,-) 1948 ---------------------------> 1949 Label Request(L2,28,-) 1950 ---------------------------> 1951 (2) Label Request(L3,93,27) 1952 <--------------------------- 1953 (3) Label Request(L1,123,-) 1954 --------------------------> 1955 Label Request(L2,124,-) 1956 --------------------------> 1957 (4) Label Mapping(L1,57,-) 1958 <-------------------------- 1959 Label Mapping(L1,94,28) 1960 <--------------------------- 1961 (5) Label Mapping(L2,58,-) 1962 <-------------------------- 1963 Label Mapping(L2,95,-) 1964 <--------------------------- 1965 (6) Address(n/a,29,-) 1966 ---------------------------> 1967 (7) Label Request(L4,30,-) 1968 ---------------------------> 1969 (8) Keepalive(n/a,-,94) 1970 ---------------------------> 1971 (9) Label Abort(L3,96,-) 1972 <--------------------------- 1973 (10) Notification(Temporary shutdown) 1974 ---------------------------> 1975 ===== TCP Session shutdown ===== 1976 : 1977 (11) : Label Withdraw(L1,59,-) 1978 : <-------------------------- 1979 : 1980 ===== TCP Session restored ===== 1981 (12) LDP Init(n/a,n/a,94) 1982 ---------------------------> 1983 LDP Init(n/a,n/a,29) 1984 <--------------------------- 1985 (13) Label Request(L4,30,-) 1986 ---------------------------> 1987 (14) Label Mapping(L2,95,-) 1988 <--------------------------- 1989 Label Abort(L3,96,30) 1990 <--------------------------- 1991 (15) Label Withdraw(L1,97,-) 1992 <--------------------------- 1994 Notes: 1995 ====== 1997 Notes are as in the previous example except as follows. 1999 (10) P1 needs to upgrade the software or hardware that it is running. 2000 It issues a Notification message to terminate the LDP session, 2001 but sets the status code as 'Temporary shutdown' to inform P2 2002 that this is not a fatal error, and P2 should maintain FT state. 2003 The TCP connection may also fail during the period that the LDP 2004 session is down (in which case it will need to be 2005 re-established), but it is also possible that the TCP connection 2006 will be preserved. 2008 9.4. Temporary Shutdown With FT Procedures and Check-Pointing 2010 notes P1 P2 2011 ===== == == 2012 (1) Label Request(L1,27,-) 2013 ---------------------------> 2014 Label Request(L2,28,-) 2015 ---------------------------> 2016 (2) Label Request(L3,93,-) 2017 <--------------------------- 2018 Label Request(L1,123,-) 2019 --------------------------> 2020 Label Request(L2,124,-) 2021 --------------------------> 2022 Label Mapping(L1,57,-) 2023 <-------------------------- 2024 (3) Label Mapping(L1,94,-) 2025 <--------------------------- 2026 Label Mapping(L2,58,-) 2027 <-------------------------- 2028 Label Mapping(L2,95,-) 2029 <--------------------------- 2030 (4) Address(n/a,29,-) 2031 ---------------------------> 2032 (5) Label Request(L4,30,-) 2033 ---------------------------> 2034 (6) Keepalive(n/a,31,95) * with FT Cork TLV * 2035 ---------------------------> 2036 (7) Label Abort(L3,96,-) 2037 <--------------------------- 2038 (8) Keepalive(n/a,97,31) * with FT Cork TLV * 2039 <--------------------------- 2040 (9) Keepalive(n/a,-,97) * with FT Cork TLV * 2041 ---------------------------> 2042 (10) Notification(Temporary shutdown) 2043 ---------------------------> 2044 ===== TCP Session shutdown ===== 2045 : 2046 : Label Withdraw(L1,59,-) 2047 : <-------------------------- 2048 : 2049 ===== TCP Session restored ===== 2050 (11) LDP Init(n/a,n/a,96) 2051 ---------------------------> 2052 LDP Init(n/a,n/a,31) 2053 <--------------------------- 2054 Label Withdraw(L1,97,-) 2055 <--------------------------- 2057 Notes: 2058 ====== 2060 This example operates much as the previous one. However, 2061 at (1), (2), (3), (4) and (5) no acknowledgements are 2062 made. 2064 At (6), P1 determines that graceful shutdown is required 2065 and sends a Keepalive acknowledging all previously 2066 received messages and itself containing a FT Protection 2067 TLV number and the FT Cork TLV. 2069 The Label abort at (7) crosses with this Keepalive, so at 2070 (8) P2 sends a Keepalive that acknowledges all messages 2071 received so far, but also including the FT Protection and 2072 FT Cork TLVs to indicate that there are still messages 2073 outstanding to be acknowledged. 2075 P1 is then able to complete the 3-way handshake at (9) 2076 and close the TCP session at (10). 2078 Upon recovery at (11) there are no messages to be re-sent 2079 because the KeepAlives flushed the acknowledgements. The 2080 only messages sent after recovery is the Label Withdraw 2081 that was pended during the TCP session failure. 2083 9.5. Checkpointing Without FT Procedures 2085 notes P1 P2 2086 ===== == == 2087 (1) Label Request(L1) 2088 ---------------------------> 2089 (2) Label Request(L2) 2090 <--------------------------- 2091 Label Request(L1) 2092 --------------------------> 2093 Label Mapping(L1) 2094 <-------------------------- 2095 (3) Label Mapping(L1) 2096 <--------------------------- 2097 (4) Keepalive(n/a,12,-) 2098 ---------------------------> 2099 (5) Label Request(L3) 2100 ---------------------------> 2101 (6) Keepalive(n/a,-,12) 2102 <--------------------------- 2103 Label Request(L3) 2104 --------------------------> 2105 Label Mapping(L3) 2106 <-------------------------- 2107 (7) Label Mapping(L3) 2108 <--------------------------- 2109 ===== TCP Session failure ===== 2110 : 2111 : 2112 : 2113 ===== TCP Session restored ===== 2114 (8) LDP Init(n/a,n/a,23) 2115 ---------------------------> 2116 LDP Init(n/a,n/a,12) 2117 <--------------------------- 2118 (9) Label Request(L3) 2119 ---------------------------> 2120 Label Request(L3) 2121 --------------------------> 2122 Label Mapping(L3) 2123 <-------------------------- 2124 (10) Label Mapping(L3) 2125 <--------------------------- 2126 (11) Label Request(L2) 2127 <--------------------------- 2129 Notes: 2130 ====== 2132 (1), (2) and (3) show label distribution without FT sequence numbers. 2134 (4) A checkpoint request from P1. It carries the sequence number of 2135 the checkpoint request. 2137 (5) P1 immediately starts a new label distribution request. 2139 (6) P2 confirms that it has secured all previous transactions. 2141 (7) The subsequent (un-acknowledged) label distribution completes. 2143 (8) The session fails and is restarted. Initialization messages 2144 confirm the sequence numbers of the secured checkpoints. 2146 (9) P1 recommences the unacknowledged label distribution request. 2148 (10) P2 recommences an unacknowledged label distribution request. 2150 9.6. Graceful Shutdown With Checkpointing But No FT Procedures 2152 notes P1 P2 2153 ===== == == 2154 (1) Label Request(L1) 2155 ---------------------------> 2156 (2) Label Request(L2) 2157 <--------------------------- 2158 Label Request(L1) 2159 --------------------------> 2160 Label Mapping(L1) 2161 <-------------------------- 2162 (3) Label Mapping(L1) 2163 <--------------------------- 2164 (4) Keepalive(n/a,12,23) * With Cork TLV * 2165 ---------------------------> 2166 (5) : 2167 : 2168 : 2169 (6) Keepalive(n/a,24,12) * With Cork TLV * 2170 <--------------------------- 2171 (7) Keepalive(n/a,-,24) * With Cork TLV * 2172 ---------------------------> 2173 (8) Notification(Temporary shutdown) 2174 ---------------------------> 2175 ===== TCP Session failure ===== 2176 : 2177 : 2178 : 2179 ===== TCP Session restored ===== 2180 (9) LDP Init(n/a,n/a,24) 2181 ---------------------------> 2182 LDP Init(n/a,n/a,12) 2183 <--------------------------- 2184 (10) Label Request(L3) 2185 ---------------------------> 2186 Label Request(L3) 2187 --------------------------> 2188 Label Mapping(L3) 2189 <-------------------------- 2190 (11) Label Mapping(L3) 2191 <--------------------------- 2192 (12) Label Mapping(L2) 2193 ---------------------------> 2195 Notes: 2196 ====== 2198 (1), (2) and (3) show label distribution without FT sequence numbers. 2200 (4) A checkpoint request from P1. It carries the sequence number of 2201 the checkpoint request and a Cork TLV. 2203 (5) P1 has sent a Cork TLV so quieces. 2205 (6) P2 confirms the checkpoint and continues the three-way handshake 2206 by including a Cork TLV itself. 2208 (7) P1 completes the three-way handshake. All operations have now 2209 been checkpointed and the session is quiesced. 2211 (8) The session is gracefully shut down. 2213 (9) The session recovers and the peers exchange the sequence numbers 2214 of the last secured checkpoints. 2216 (10) P1 starts a new label distribution request. 2218 (11) P1 continues processing a previously received label distribution 2219 request. 2221 10. Security Considerations 2223 The LDP FT enhancements inherit similar security 2224 considerations to those discussed in [RFC3036]. 2226 The LDP FT enhancements allow the re-establishment of a 2227 TCP connection between LDP peers without a full re- 2228 exchange of the attributes of established labels, which 2229 renders LSRs that implement the extensions specified in 2230 this draft vulnerable to additional denial-of-service 2231 attacks as follows: 2233 - An intruder may impersonate an LDP peer in order to force 2234 a failure and reconnection of the TCP connection, but where 2235 the intruder does not set the FT Reconnect Flag on re- 2236 connection. This forces all FT labels to be released. 2238 - Similarly, an intruder could set the FT Reconnect Flag on 2239 re-establishment of the TCP session without preserving the 2240 state and resources for FT labels. 2242 - An intruder could intercept the traffic between LDP peers 2243 and override the setting of the FT Label Flag to be set to 0 2244 for all labels. 2246 All of these attacks may be countered by use of an 2247 authentication scheme between LDP peers, such as the MD5- 2248 based scheme outlined in [RFC3036]. 2250 Alternative authentication schemes for LDP peers are 2251 outside the scope of this draft, but could be deployed to 2252 provide enhanced security to implementations of LDP and 2253 the LDP FT enhancements. 2255 As with LDP, a security issue may exist if an LDP 2256 implementation continues to use labels after expiration 2257 of the session that first caused them to be used. This 2258 may arise if the upstream LSR detects the session failure 2259 after the downstream LSR has released and re-used the 2260 label. The problem is most obvious with the platform- 2261 wide label space and could result in mis-forwarding of data 2262 to other than intended destinations and it is conceivable 2263 that these behaviors may be deliberately exploited to 2264 either obtain services without authorization or to deny 2265 services to others. 2267 In this draft, the validity of the session may be 2268 extended by the FT Reconnection Timeout, and the session 2269 may be re-established in this period. After the expiry 2270 of the Reconnection Timeout the session must be 2271 considered to have failed and the same security issue 2272 applies as described above. 2274 However, the downstream LSR may declare the session as 2275 failed before the expiration of its Reconnection Timeout. 2276 This increases the period during which the downstream LSR 2277 might reallocate the label while the upstream LSR 2278 continues to transmit data using the old usage of the 2279 label. To reduce this issue, this draft requires that 2280 labels are not re-used until the Reconnection Timeout has 2281 expired. 2283 A further issue might apply if labels were re-used prior 2284 to the expiration of the FT Reconnection Timeout, but 2285 this is forbidden by this draft. 2287 The issue of re-use of labels extends to labels managed through 2288 other mechanisms including direct configuration through management 2289 applications and distribution through other label distribution 2290 protocols. Avoiding this problem may be contrued as an 2291 implementation issue (see below) but failure to acknowledge it could 2292 result in mis-forwarding of data between LSPs established using 2293 some other mechanism and those recovered using the methods 2294 described in this document. 2296 11. Implementation Notes 2298 11.1. FT Recovery Support on Non-FT LSRs 2300 In order to take full advantage of the FT capabilities of 2301 LSRs in the network, it may be that an LSR that does not 2302 itself contain the ability to recover from local hardware 2303 or software faults still needs to support the LDP FT 2304 enhancements described in this draft. 2306 Consider an LSR, P1, that is an LDP peer of a fully Fault 2307 Tolerant LSR, P2. If P2 experiences a fault in the 2308 hardware or software that serves an LDP session between 2309 P1 and P2, it may fail the TCP connection between the 2310 peers. When the connection is recovered, the LSPs/labels 2311 between P1 and P2 can only be recovered if both LSRs were 2312 applying the FT recovery procedures to the LDP session. 2314 11.2. ACK generation logic 2316 FT ACKs SHOULD be returned to the sending LSR as soon as 2317 is practicable in order to avoid building up a large 2318 quantity of unacknowledged state changes at the LSR. 2319 However, immediate one-for-one acknowledgements would 2320 waste bandwidth unnecessarily. 2322 A possible implementation strategy for sending ACKs to FT 2323 LDP messages is as follows: 2325 - An LSR secures received messages in order and tracks the 2326 sequence number of the most recently secured message, Sr. 2328 - On each LDP KeepAlive that the LSR sends, it attaches an 2329 FT ACK TLV listing Sr 2331 - Optionally, the LSR may attach an FT ACK TLV to any other 2332 LDP message sent between Keepalive messages if, for example, 2333 Sr has increased by more than a threshold value since the 2334 last ACK sent. 2336 This implementation combines the bandwidth benefits of 2337 accumulating ACKs while still providing timely ACKs. 2339 11.2.1 Ack Generation Logic When Using Check-Pointing 2341 If check-pointing is in use, the LSRs need not be 2342 concerned to send ACKs in such a timely manner. 2344 Check-points are solicitations for acknowledgement 2345 conveyed as a sequence number in an FT Protection TLV on 2346 a Keepalive message. Such check-point requests could be 2347 issued on a timer, after a significant amount of change, 2348 or before controlled shutdown of a session. 2350 The use of check-pointing may considerably simplify an 2351 implementation since it does not need to track the 2352 sequence numbers of all received LDP messages. It must, 2353 however, still ensure that all received messages (or the 2354 consequent state changes) are secured before 2355 acknowledging the sequence number on the Keepalive. 2357 This approach may be considered optimal in systems that 2358 do not show a high degree of change over time (such as 2359 targeted LDP sessions) and that are prepared to risk loss 2360 of state for the most recent LDP exchanges. More dynamic 2361 systems (such as LDP discovery sessions) are more likely 2362 to want to acknowledge state changes more frequently so 2363 that the maximum amount of state can be preserved over a 2364 failure. 2366 11.3 Interactions With Other Label Distribution Mechanisms 2368 Many LDP LSRs also run other label distribution mechanisms. These 2369 include management interfaces for configuration of static label 2370 mappings, other distinct instances of LDP, and other label 2371 distribution protocols. The last example includes traffic engineering 2372 label distribution protocol that are used to construct tunnels 2373 through which LDP LSPs are established. 2375 As with re-use of individual labels by LDP within a restarting LDP 2376 system, care must be taken to prevent labels that need to be retained 2377 by a restarting LDP session or protocol component from being used by 2378 another label distribution mechanism since that might compromise 2379 data security amongst other things. 2381 It is a matter for implementations to avoid this issue through the 2382 use of techniques such as a common label management component or 2383 segmented label spaces. 2385 12. Acknowledgments 2387 The work in this draft is based on the LDP ideas 2388 expressed by the authors of [RFC3036]. 2390 The ACK scheme used in this draft was inspired by the 2391 proposal by David Ward and John Scudder for restarting 2392 BGP sessions now included in [BGP-RESTART]. 2394 The authors would also like to acknowledge the careful 2395 review and comments of Nick Weeds, Piers Finlayson, Tim 2396 Harrison, Duncan Archer, Peter Ashwood-Smith, Bob Thomas, 2397 S.Manikantan, Adam Sheppard, Alan Davey, Iftekhar Hussain 2398 and Loa Andersson. 2400 13. Intellectual Property Consideration 2402 The IETF has been notified of intellectual property 2403 rights claimed in regard to some or all of the 2404 specification contained in this document. For more 2405 information, consult the online list of claimed rights. 2407 14. Full Copyright Statement 2409 Copyright (c) The Internet Society (2000, 2001, 2002). 2410 All Rights Reserved. This document and translations of it 2411 may be copied and furnished to others, and derivative 2412 works that comment on or otherwise explain it or assist 2413 in its implementation may be prepared, copied, published 2414 and distributed, in whole or in part, without restriction 2415 of any kind, provided that the above copyright notice and 2416 this paragraph are included on all such copies and 2417 derivative works. However, this document itself may not 2418 be modified in any way, such as by removing the copyright 2419 notice or references to the Internet Society or other 2420 Internet organizations, except as needed for the purpose 2421 of developing Internet standards in which case the 2422 procedures for copyrights defined in the Internet 2423 Standards process must be followed, or as required to 2424 translate it into languages other than English. 2426 The limited permissions granted above are perpetual and 2427 will not be revoked by the Internet Society or its 2428 successors or assigns. 2430 This document and the information contained herein is 2431 provided on an "AS IS" basis and THE INTERNET SOCIETY AND 2432 THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL 2433 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 2434 TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN 2435 WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 2436 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 2438 15. IANA Considerations 2440 This draft requires the use of a number of new TLVs and 2441 status codes from the number spaces within the LDP 2442 protocol. This section explains the logic used by the 2443 authors to choose the most appropriate number space for 2444 each new entity, and is intended to assist in the 2445 determination of any final values assigned by IANA or the 2446 MPLS WG in the event that the MPLS WG chooses to advance 2447 this draft on the standards track. 2449 This section will be removed when the TLV and status code 2450 values have been agreed with IANA. 2452 15.1. New TLVs 2454 The FT Protection TLV carries attributes that affect a 2455 single label exchanged between LDP peers. It is taken 2456 from the 0x02xx range for TLVs that is used in [RFC3036] 2457 by other TLVs carrying label attributes. The next 2458 available value in this range is 0x0203. 2460 The FT Session TLV carries attributes that affect the 2461 entire LDP session between LDP peers. It is taken from 2462 the 0x05xx range for TLVs that is used in [RFC3036] by 2463 other TLVs carrying session-wide attributes. The next 2464 available value in this range is 0x0503. 2466 The FT Protection TLV may ACK many label operations at 2467 once if cumulative ACKS are used. It is taken from the 2468 0x05xx range for TLVs that is used in [RFC3036] by other 2469 TLVs carrying session-wide attributes. The next 2470 available value in this range is 0x0504. 2472 The FT Cork TLV carries attributes that apply to all labels 2473 exchanged between LDP peers. It is taken from the 0x05xx range 2474 for TLVs that is used in [RFC3036] by other TLVs carrying label 2475 attributes. The next available value in this range is 0x0505. 2477 In summary: 2479 FT Protection TLV 0x0203 2480 FT Session TLV 0x0503 2481 FT Ack TLV 0x0504 2482 FT Cork TLV 0x0505 2484 15.2. New Status Codes 2486 LDP status codes are not sub-divided into specific ranges 2487 for different types of error. Hence, the numeric status 2488 code values are selected as the next available. 2490 Section 7.1 lists the new status codes required by this document and 2491 gives interpretative information. The new codes are as follows. 2493 Status Code E Status Data 2495 No LDP Session 0 0x0000001A 2496 Zero FT seqnum 1 0x0000001B 2497 Unexpected TLV / 1 0x0000001C 2498 Session Not FT 2499 Unexpected TLV / 1 0x0000001D 2500 Label Not FT 2501 Missing FT Protection TLV 1 0x0000001E 2502 FT ACK sequence error 1 0x0000001F 2503 Temporary Shutdown 0 0x00000020 2504 FT Seq Numbers Exhausted 1 0x00000021 2505 FT Session parameters / 1 0x00000022 2506 changed 2507 Unexpected FT Cork TLV 1 0x00000023 2509 16. Authors' Addresses 2511 Adrian Farrel (editor) Paul Brittain 2512 Movaz Networks, Inc. Data Connection Ltd. 2513 7926 Jones Branch Drive, Suite 615 Windsor House, Pepper Street, 2514 McLean, VA 22102 Chester, Cheshire 2515 Phone: +1 703-847-1867 CH1 1DF, UK 2516 Email: afarrel@movaz.com Phone: +44-(0)20-8366-1177 2517 Email: pjb@dataconnection.com 2519 Philip Matthews Eric Gray 2520 Hyperchip Celox Networks, Inc. 2521 1800 Rene-Levesque Blvd W 2 Park Central Drive, 2522 Montreal, Quebec H3H 2H2 Southborough, MA 01772 2523 Canada Phone: +1 508 305 7214 2524 Phone: +1 514-906-4965 Email: egray@celoxnetworks.com 2525 Email: pmatthews@hyperchip.com 2527 Jack Shaio Toby Smith 2528 Vivace Networks Laurel Networks, Inc. 2529 2730 Orchard Parkway 1300 Omega Drive 2530 San Jose, CA 95134 Pittsburgh, PA 15205 2531 Phone: +1 408 432 7623 Email: tob@laurelnetworks.com 2532 Email: jack.shaio@vivacenetworks.com 2534 Andrew G. Malis 2535 Vivace Networks 2536 2730 Orchard Parkway 2537 San Jose, CA 95134 2538 Phone: +1 408 383 7223 2539 andy.malis@vivacenetworks.com 2541 17. References 2543 17.1. Normative References 2545 [RFC2026] Bradner, S., "The Internet Standards Process -- 2546 Revision 3", BCP 9, RFC 2026, October 1996. 2548 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2549 Requirement Levels", BCP 14, RFC 2119, March 1997. 2551 [RFC3036] Andersson, L., et. al., LDP Specification, RFC 3036, 2552 January 2001. 2554 [LDP-RESTART] Leelanivas, M., et al., Graceful Restart Mechanism for 2555 LDP, draft-ietf-ldp-restart-05.txt, September 2002, 2556 work in progress. 2558 17.2. Informative References 2560 [RFC2205] Braden, R., et al., Resource ReSerVation Protocol 2561 (RSVP) -- Version 1, Functional Specification, RFC 2562 2205, September 1997. 2564 [RFC2961] Berger, L., et al., RSVP Refresh Reduction Extensions, 2565 RFC 2961, April 2001. 2567 [RFC3209] Awduche, D., et al,. Extensions to RSVP for LSP 2568 Tunnels, RFC 3209, December 2001. 2570 [RFC3212] Jamoussi, B., et. al., Constraint-Based LSP Setup 2571 using LDP, RFC 3212, January 2002. 2573 [RFC3214] Ash, G., et al., LSP Modification Using CR-LDP, RFC 2574 3214, January 2001. 2576 [BGP-RESTART] Sangli, S., et al., Graceful Restart Mechanism 2577 for BGP, draft-ietf-idr-restart-05.txt, June 2002 2578 (work in progress).