idnits 2.17.1 draft-ietf-mpls-ldp-ft-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 54 longer pages, the longest (page 1) being 59 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC3036], [RFC3212]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 1597 has weird spacing: '...ication messa...' == Line 1714 has weird spacing: '... number equal...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: In these cases the node that intends terminating the session can flush acknowledgement using a check-point request as described above. The sender SHOULD not send further label or address-related messages after requesting shutdown check-pointing in order to preserve the integrity of its saved state. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 2002) is 7949 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 3036 (Obsoleted by RFC 5036) == Outdated reference: A later version (-13) exists of draft-ietf-idr-restart-05 -- No information found for draft-ietf-ldp-restart - is the name correct? Summary: 4 errors (**), 0 flaws (~~), 6 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 MPLS Working Group Adrian Farrel 2 Internet Draft Movaz Networks, Inc. 3 Document: draft-ietf-mpls-ldp-ft-04.txt 4 Expiration Date: January 2003 Paul Brittain 5 Data Connection Ltd. 7 Philip Matthews 8 Hyperchip 10 Eric Gray 11 Celox Networks 13 Toby Smith 14 Laurel Networks 16 Andrew G. Malis 17 Jack Shaio 18 Vivace Networks 20 July 2002 22 Fault Tolerance for the Label Distribution Protocol (LDP) 24 draft-ietf-mpls-ldp-ft-04.txt 26 Status of this Memo 28 This document is an Internet-Draft and is in full 29 conformance with all provisions of Section 10 of RFC2026 30 [RFC2026]. 32 Internet-Drafts are working documents of the Internet 33 Engineering Task Force (IETF), its areas, and its working 34 groups. Note that other groups may also distribute 35 working documents as Internet-Drafts. Internet-Drafts are 36 draft documents valid for a maximum of six months and may 37 be updated, replaced, or obsoleted by other documents at 38 any time. It is inappropriate to use Internet- Drafts as 39 reference material or to cite them other than as "work in 40 progress." 42 The list of current Internet-Drafts can be accessed at 43 http://www.ietf.org/ietf/1id-abstracts.txt 45 The list of Internet-Draft Shadow Directories can be 46 accessed at http://www.ietf.org/shadow.html. 48 NOTE: The new TLV type numbers, bit values for flags 49 specified in this draft, and new LDP status code values 50 are preliminary suggested values and have yet to be 51 approved by IANA or the MPLS WG. See the section "IANA 52 Considerations" for further details. 54 Abstract 56 MPLS systems will be used in core networks where system 57 downtime must be kept to an absolute minimum. Many MPLS 58 LSRs may, therefore, exploit Fault Tolerant (FT) hardware 59 or software to provide high availability of the core 60 networks. 62 The details of how FT is achieved for the various 63 components of an FT LSR, including LDP, the switching 64 hardware and TCP, are implementation specific. This 65 document identifies issues in the LDP specification 66 [RFC3036] that make it difficult to implement an FT LSR 67 using the current LDP protocols, and proposes 68 enhancements to the LDP specification to ease such FT LSR 69 implementations. 71 The issues and extensions described here are equally 72 applicable to CR-LDP [RFC3212]. 74 Contents 76 1. Conventions and Terminology used in this document 3 77 2. Introduction 4 78 2.1. Fault Tolerance for MPLS 5 79 2.2. Issues with LDP and CR-LDP 5 80 3. Overview of LDP FT Enhancements 7 81 3.1. Establishing an FT LDP Session 8 82 3.1.1 Interoperation with Non-FT LSRs 8 83 3.2. TCP Connection Failure 9 84 3.2.1 Detecting TCP Connection Failures 9 85 3.2.2 LDP Processing after Connection Failure 9 86 3.3. Data Forwarding During TCP Connection Failure 10 87 3.4. FT LDP Session Reconnection 10 88 3.5. Operations on FT Labels 11 89 3.6. Check-Pointing 12 90 3.6.1 Graceful Termination 13 91 3.7. Label Space Depletion and Replenishment 13 92 4. FT Operations 14 93 4.1. FT LDP Messages 14 94 4.1.1 Sequence Numbered FT Label Messages 14 95 4.1.2 FT Address Messages 15 96 4.1.3 Label Resources Available Notifications 15 97 4.2. FT Operation ACKs 17 98 4.3. Preservation of FT State 17 99 4.4. FT Procedure After TCP Failure 19 100 4.4.1 FT LDP Operations During TCP Failure 20 101 4.5. FT Procedure After TCP Re-connection 21 102 4.5.1 Re-Issuing FT Messages 22 103 4.5.2 Interaction with CR-LDP LSP Modification 22 104 5. Checkpointing Procedures 23 105 5.1. Checkpointing with the Keepalive Message 23 106 5.2. Quiesce and Keepalive 24 108 6. Changes to Existing Messages 24 109 6.1. LDP Initialization Message 24 110 6.2. LDP Keepalive Messages 25 111 6.3. All Other LDP Session Messages 25 112 7. New Fields and Values 26 113 7.1. Status Codes 26 114 7.2. FT Session TLV 27 115 7.3. FT Protection TLV 30 116 7.4. FT ACK TLV 32 117 7.5. FT Cork TLV 34 118 8. Example Use 35 119 8.1. Session Failure and Recovery - FT Procedures 36 120 8.2. Use of Check-Pointing With FT Procedures 38 121 8.3. Temporary Shutdown With FT Procedures 40 122 8.4. Temporary Shutdown With FT Procedures and Check-Pointing 42 123 8.5. Checkpointing Without FT Procedures 44 124 8.6. Graceful Shutdown With Checkpointing But No FT Procedures 46 125 9. Security Considerations 47 126 10. Implementation Notes 49 127 10.1. FT Recovery Support on Non-FT LSRs 49 128 10.2. ACK generation logic 49 129 10.2.1 Ack Generation Logic When Using Check-Pointing 49 130 11. Acknowledgments 50 131 12. Intellectual Property Consideration 50 132 13. Full Copyright Statement 51 133 14. IANA Considerations 51 134 14.1. New TLVs 52 135 14.2. New Status Codes 52 136 15. Authors' Addresses 53 137 16. References 53 138 16.1. Normative References 53 139 16.2. Informative References 53 141 1. Conventions and Terminology used in this document 143 Definitions of key words and terms applicable to LDP and 144 CR-LDP are inherited from [RFC3212] and [RFC3036]. 146 The term "FT Label" is introduced in this document to 147 indicated a label for which some fault tolerant operation 148 is used. A "non-FT Label" is not fault tolerant and is 149 handled as specified in [RFC3212] and [RFC3036]. 151 The term "Sequence Numbered FT Label" is used to indicate 152 an FT label which is secured using the sequence number in 153 the FT Protection TLV described in this document. 155 The term "Checkpointable FT Label" is used to indicate an 156 FT label which is secured by using the checkpointing 157 techniques described in this document. 159 The extensions to LDP specified in this document are 160 collectively referred to as the "LDP FT enhancements". 162 Within the context of this draft, "Checkpointing" refers 163 to a process of messages exchanges that confirm receipt 164 and processing (or secure storage) of specific protocol 165 messages. 167 In the examples quoted, the following notation is used: 169 Ln : An LSP. For example L1. 171 Pn : An LDP peer. For example P1. 173 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 174 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", 175 "MAY", and "OPTIONAL" in this document are to be 176 interpreted as described in RFC-2119 [RFC2119]. 178 2. Introduction 180 High Availability (HA) is typically claimed by equipment 181 vendors when their hardware achieves availability levels 182 of at least 99.999% (five 9s). To implement this, the 183 equipment must be capable of recovering from local 184 hardware and software failures through a process known as 185 fault tolerance (FT). 187 The usual approach to FT involves provisioning backup 188 copies of hardware and/or software. When a primary copy 189 fails, processing is switched to the backup copy. This 190 process, called failover, should result in minimal 191 disruption to the Data Plane. 193 In an FT system, backup resources are sometimes 194 provisioned on a one-to-one basis (1:1), sometimes as 195 many-to-one (1:n), and occasionally as many-to-many 196 (m:n). Whatever backup provisioning is made, the system 197 must switch to the backup automatically on failure of the 198 primary, and the software and hardware state in the 199 backup must be set to replicate the state in the primary 200 at the point of failure. 202 2.1. Fault Tolerance for MPLS 204 MPLS will be used in core networks where system downtime 205 must be kept to an absolute minimum. Many MPLS LSRs may, 206 therefore, exploit FT hardware or software to provide 207 high availability of core networks. 209 In order to provide HA, an MPLS system needs to be able 210 to survive a variety of faults with minimal disruption to 211 the Data Plane, including the following fault types: 213 - failure/hot-swap of a physical connection between LSRs 215 - failure/hot-swap of the switching fabric in the LSR 217 - failure of the TCP or LDP stack in an LSR 219 - software upgrade to the TCP or LDP stacks. 221 The first two examples of faults listed above are 222 confined to the Data Plane. Such faults can be handled 223 by providing redundancy in the Data Plane which is 224 transparent to LDP operating in the Control Plane. The 225 last two example types of fault require action in the 226 Control Plane to recover from the fault without 227 disrupting traffic in the Data Plane. This is possible 228 because many recent router architectures separate the 229 Control and Data Planes such that forwarding can continue 230 unaffected by recovery action in the Control Plane. 232 2.2. Issues with LDP and CR-LDP 234 LDP and CR-LDP use TCP to provide reliable connections 235 between LSRs over which to exchange protocol messages to 236 distribute labels and to set up LSPs. A pair of LSRs that 237 have such a connection are referred to as LDP peers. 239 TCP enables LDP and CR-LDP to assume reliable transfer of 240 protocol messages. This means that some of the messages 241 do not need to be acknowledged (for example, Label 242 Release). 244 LDP and CR-LDP are defined such that if the TCP 245 connection fails, the LSR should immediately tear down 246 the LSPs associated with the session between the LDP 247 peers, and release any labels and resources assigned to 248 those LSPs. 250 It is notoriously hard to provide a Fault Tolerant 251 implementation of TCP. To do so might involve making 252 copies of all data sent and received. This is an issue 253 familiar to implementers of other TCP applications such 254 as BGP. 256 During failover affecting the TCP or LDP stacks, 257 therefore, the TCP connection may be lost. Recovery from 258 this position is made worse by the fact that LDP or CR- 259 LDP control messages may have been lost during the 260 connection failure. Since these messages are 261 unconfirmed, it is possible that LSP or label state 262 information will be lost. 264 This draft describes a solution which involves 266 - negotiation between LDP peers of the intent to support 267 extensions to LDP that facilitate recovery from failover 268 without loss of LSPs 270 - selection of FT survival on a per LSP/label basis 272 - acknowledgement of LDP messages to ensure that a full 273 handshake is performed on those messages either frequently 274 (such as per message) or less frequently as in check- 275 pointing 277 - solicitation of up-to-date acknowledgement 278 (checkpointing) of previous LDP messages to ensure the 279 current state is flushed to disk/NVRAM, with an additional 280 option that allows an LDP partner to request that state is 281 flushed in both directions if graceful shutdown is required. 283 - re-issuing lost messages after failover to ensure that 284 LSP/label state is correctly recovered after reconnection of 285 the LDP session. 287 Other objectives of this draft are to 289 - offer back-compatibility with LSRs that do not implement 290 these proposals 292 - preserve existing protocol rules described in [RFC3212] 293 and [RFC3036] for handling unexpected duplicate messages and 294 for processing unexpected messages referring to unknown 295 LSPs/labels 297 - integrate with the LSP modification function described in 298 [RFC3214] 300 - avoid full state refresh solutions (such as those present 301 in RSVP: see [RFC2205], [RFC2961], [RFC3209] and [LDP- 302 RESTART]) whether they be full-time, or limited to post- 303 failover recovery. 305 Note that this draft concentrates on the preservation of 306 label state for labels exchanged between a pair of 307 adjacent LSRs when the TCP connection between those LSRs 308 is lost. This is a requirement for Fault Tolerant 309 operation of LSPs, but a full implementation of end-to- 310 end protection for LSPs requires that this is combined 311 with other techniques that are outside the scope of this 312 draft. 314 In particular, this draft does not attempt to describe 315 how to modify the routing of an LSP or the resources 316 allocated to a label or LSP, which is covered by 317 [RFC3214]. This draft also does not address how to 318 provide automatic layer 2/3 protection switching for a 319 label or LSP, which is a separate area for study. 321 This specification does not preclude an implementation 322 from attempting (or require it to attempt) to use the FT 323 behavior described here to recover from a preemptive 324 failure of a connection on a non-FT system due to, for 325 example, a partial system crash. Note, however, that 326 there are potential issues too numerous to list here - 327 not least the likelihood that the same crash will 328 immediately occur when processing the restored data. 330 3. Overview of LDP FT Enhancements 332 The LDP FT enhancements consist of the following main 333 elements, which are described in more detail in the 334 sections that follow. 336 - The presence of an FT Session TLV on the LDP 337 Initialization message indicates that an LSR supports some 338 form of protection or recovery from session failure. A flag 339 bit within this TLV (the S bit) indicates that the LSR 340 supports the LDP FT enhancements on this session. Another 341 flag (the C bit) indicates that the checkpointing procedures 342 are to be used. 344 - An FT Reconnect Flag in the FT Session TLV indicates 345 whether an LSR has preserved FT Label state across a failure 346 of the TCP connection. 348 - An FT Reconnection Timeout, exchanged on the LDP 349 Initialization message, that indicates the maximum time peer 350 LSRs will preserve FT Label state after a failure of the TCP 351 connection. 353 - An FT Protection TLV used to identify operations that 354 affect LDP labels. All LDP messages carrying the FT 355 Protection TLV need to be secured (e.g. to NVRAM) and ACKed 356 to the sending LDP peer in order that the state for Sequence 357 Numbered FT Labels can be correctly recovered after LDP 358 session reconnection. 360 Note that the implementation within an FT system is left 361 open by this draft. An implementation could choose to 362 secure entire messages relating to Sequence Numbered FT 363 Labels, or it could secure only the relevant state 364 information. 366 - Address advertisement may also be secured by use of the 367 FT Protection TLV. This enables recovery after LDP session 368 reconnection without the need to re-advertise what may be a 369 very large number of addresses. 371 - The FT Protection TLV may also be used on the Keepalive 372 message to flush acknowledgement of all previous FT 373 operations. This enables a check-point for future recovery, 374 either in mid-session or prior to graceful shutdown of an 375 LDP session. This procedure may also be used to checkpoint 376 all (that is both FT and non-FT) operations for future 377 recovery. 379 3.1. Establishing an FT LDP Session 381 In order that the extensions to LDP [RFC3036] and CR-LDP 382 [RFC3212] described in this draft can be used 383 successfully on an LDP session between a pair of LDP 384 peers, they MUST negotiate that the LDP FT enhancements 385 are to be used on the LDP session. 387 This is done on the LDP Initialization message exchange 388 using a new FT Session TLV. Presence of this TLV 389 indicates that the peer wants to support some form of 390 protection or recovery processing. The S bit within this 391 TLV indicates that the peer wants to support the LDP FT 392 enhancements on this LDP session. The C bit indicates 393 that the peer wants to support the checkpointing 394 functions described in this draft. The S and C bits may 395 be set independently. 397 The relevant LDP FT enhancements MUST be supported on an 398 LDP session if both LDP peers include an FT Session TLV 399 on the LDP Initialization message and have the same 400 setting of the S or C bit. 402 If either LDP Peer does not include the FT Session TLV 403 LDP Initialization message or if there is no match of S 404 and C bits between the peers, the LDP FT enhancements 405 MUST NOT be used during this LDP session. Use of LDP FT 406 enhancements by a sending LDP peer MUST be interpreted by 407 the receiving LDP peer as a serious protocol error 408 causing the session to be terminated. 410 An LSR MAY present different FT/non-FT behavior on 411 different TCP connections, even if those connections are 412 successive instantiations of the LDP session between the 413 same LDP peers. 415 3.1.1 Interoperation with Non-FT LSRs 417 The FT Session TLV on the LDP Initialization message 418 carries the U-bit. If an LSR does not support any 419 protection or recovery mechanisms , it will ignore this 420 TLV. Since such partners also do not include the FT 421 Session TLV, all LDP sessions to such LSRs will not use 422 the LDP FT enhancements. 424 The rest of this draft assumes that the LDP sessions 425 under discussion are between LSRs that do support the LDP 426 FT enhancements, except where explicitly stated 427 otherwise. 429 3.2. TCP Connection Failure 431 3.2.1 Detecting TCP Connection Failures 433 TCP connection failures may be detected and reported to the 434 LDP component in a variety of ways. These should all be 435 treated in the same way by the LDP component. 437 - Indication from the management component that a TCP 438 connection or underlying resource is no longer active. 440 - Notification from a hardware management component of an 441 interface failure. 443 - Sockets keepalive timeout. 445 - Sockets send failure. 447 - New (incoming) Socket opened. 449 - LDP protocol timeout. 451 3.2.2 LDP Processing after Connection Failure 453 If the LDP FT enhancements are not in use on an LDP 454 session, the action of the LDP peers on failure of the 455 TCP connection is as specified in [RFC3212] and 456 [RFC3036]. 458 All state information and resources associated with non- 459 FT Labels MUST be released on the failure of the TCP 460 connection, including deprogramming the non-FT Label from 461 the switching hardware. This is equivalent to the 462 behavior specified in [RFC3036]. 464 If the LDP FT enhancements are in use on an LDP session, 465 both LDP peers SHOULD preserve state information and 466 resources associated with FT Labels exchanged on the LDP 467 session. Both LDP peers SHOULD use a timer to release 468 the preserved state information and resources associated 469 with FT-labels if the TCP connection is not restored 470 within a reasonable period. The behavior when this timer 471 expires is equivalent to the LDP session failure behavior 472 described in [RFC3036]. 474 The FT Reconnection Timeout each LDP peer intends to 475 apply to the LDP session is carried in the FT Session TLV 476 on the LDP Initialization messages. Both LDP peers MUST 477 use the value that corresponds to the lesser timeout 478 interval of the two proposed timeout values from the LDP 479 Initialization exchange, where a value of zero is treated 480 as positive infinity. 482 3.3. Data Forwarding During TCP Connection Failure 484 An LSR that implements the LDP FT enhancements SHOULD 485 preserve the programming of the switching hardware across 486 a failover. This ensures that data forwarding is 487 unaffected by the state of the TCP connection between 488 LSRs. 490 It is an integral part of FT failover processing in some 491 hardware configurations that some data packets might be 492 lost. If data loss is not acceptable to the applications 493 using the MPLS network, the LDP FT enhancements described 494 in this draft SHOULD NOT be used. 496 3.4. FT LDP Session Reconnection 498 When a new TCP connection is established, the LDP peers 499 MUST exchange LDP Initialization messages. When a new 500 TCP connection is established after failure, the LDP 501 peers MUST re-exchange LDP Initialization messages. 503 If an LDP peer includes the FT Session TLV with the S bit 504 set in the LDP Initialization message for the new 505 instantiation of the LDP session, it MUST also set the FT 506 Reconnect Flag according to whether it has been able to 507 preserve label state. The FT Reconnect Flag is carried 508 in the FT Session TLV. 510 If an LDP peer has preserved all state information for 511 previous instantiations of the LDP session, then it 512 SHOULD set the FT Reconnect Flag to 1 in the FT Session 513 TLV. Otherwise, it MUST set the FT Reconnect Flag to 0. 515 If either LDP peer sets the FT Reconnect Flag to 0, or 516 omits the FT Session TLV, both LDP peers MUST release any 517 state information and resources associated with the 518 previous instantiation of the LDP session between the 519 same LDP peers, including FT Label state and Addresses. 520 This ensures that network resources are not permanently 521 lost by one LSR if its LDP peer is forced to undergo a 522 cold start. 524 If an LDP peer changes any session parameters (for 525 example, the label space bounds) from the previous 526 instantiation the nature of any preserved labels may have 527 changed. In particular, previously allocated labels may 528 now be out of range. For this reason, session 529 reconnection MUST use the same parameters as were in use 530 on the session before the failure. If an LDP peer 531 notices that the parameters have been changed by the 532 other peer it SHOULD send a Notification message with the 533 'FT Session parameters changed' status code. 535 If both LDP peers set the FT Reconnect Flag to 1, both 536 LDP peers MUST use the procedures indicated in this draft 537 to complete any label operations on Sequence Numbered FT 538 Labels that were interrupted by the LDP session failure. 540 If an LDP peer receives an LDP Initialization message 541 with the FT Reconnect Flag set before it sends its own 542 Initialization message, but has retained no information 543 about the previous version of the session, it MUST 544 respond with an Initialization message with the FT 545 Reconnect Flag clear. If an LDP peer receives an LDP 546 Initialization message with the FT Reconnect Flag set in 547 response to an Initialization message that it has sent 548 with the FT Reconnect Flag clear it MUST act as if no 549 state was retained by either peer on the session. 551 3.5. Operations on FT Labels 553 Label operations on Sequence Numbered FT Labels are made 554 Fault Tolerant by providing acknowledgement of all LDP 555 messages that affect Sequence Numbered FT Labels. 556 Acknowledgements are achieved by means of sequence 557 numbers on these LDP messages. 559 The message exchanges used to achieve acknowledgement of 560 label operations and the procedures used to complete 561 interrupted label operations are detailed in the section 562 "FT Operations". 564 Using these acknowledgements and procedures, it is not 565 necessary for LDP peers to perform a complete re- 566 synchronization of state for all Sequence Numbered FT 567 Labels, either on re-connection of the LDP session 568 between the LDP peers or on a timed basis. 570 3.6. Check-Pointing 572 Check-pointing is a useful feature that allows nodes to 573 reduce the amount of processing that they need to do to 574 acknowledge LDP messages. The C bit in the FT Session 575 TLV is used to indicate that checkpointing is supported. 577 Under the normal operation on Sequence Numbered FT 578 Labels, acknowledgments may be deferred during normal 579 processing and only sent periodically. Check-pointing 580 may be used to flush acknowledgement from a peer by 581 including a sequence number on a Keepalive message 582 requesting acknowledgement of that message and all 583 previous messages. In this case, all Sequence Numbered 584 FT Labels are Checkpointable FT Labels. 586 If the S bit is not agreed upon, checkpointing may still 587 be used. In this case it is used to acknowledge all 588 messages exchanged between the peers, and all labels are 589 Checkpointable FT Labels. 591 This offers an approach where acknowledgements need not 592 be sent to every message or even frequently, but are only 593 sent as check-points in response to requests carried on 594 Keepalive messages. Such an approach may be considered 595 optimal in systems that do not show a high degree of 596 change over time (such as targeted LDP session or CR-LDP 597 systems) and that are prepared to risk loss of state for 598 the most recent LDP exchanges. More dynamic systems 599 (such as LDP discovery sessions) are more likely to want 600 to acknowledge state changes more frequently so that the 601 maximum amount of state can be preserved over a failure. 603 Note that an important consideration of this draft is 604 that nodes acknowledging messages on a one-for-one basis, 605 nodes deferring acknowledgements, and nodes relying on 606 check-pointing should all interoperate seamlessly and 607 without protocol negotiation beyond session 608 initialization. 610 Further discussion of this feature is provided in the 611 section "FT Operations". 613 3.6.1 Graceful Termination 615 A feature that builds on check-pointing is graceful 616 termination. 618 In some cases, such as controlled failover or software 619 upgrade, it is possible for a node to know in advance 620 that it is going to terminate its session with a peer. 622 In these cases the node that intends terminating the 623 session can flush acknowledgement using a check-point 624 request as described above. The sender SHOULD not send 625 further label or address-related messages after 626 requesting shutdown check-pointing in order to preserve 627 the integrity of its saved state. 629 This, however, only provides for acknowledgement in one 630 direction, and the node that is terminating also requires 631 to know that it has secured all state sent by its peer. 632 This is achieved by a three-way hand shake of the check- 633 point which is requested by an additional TLV (the Cork 634 TLV) in the Keepalive message. 636 Further discussion of this feature is provided in the 637 section "FT Operations". 639 3.7. Label Space Depletion and Replenishment 641 When an LDP peer is unable to satisfy a Label Request 642 message because it has no more available labels, it sends 643 a Notification message carrying the status code 'No label 644 resources'. This warns the requesting LDP peer that 645 subsequent Label Request messages are also likely to fail 646 for the same reason. This message does not need to be 647 acknowledged for FT purposes since Label Request messages 648 sent after session recovery will receive the same 649 response. However, the LDP peer that receives a 'No 650 label resources' Notification stops sending Label Request 651 messages until it receives a 'Label resources available' 652 Notification message. Since this unsolicited 653 Notification might get lost during session failure, it 654 may be protected using the procedures described in this 655 draft. 657 An alternative approach allows that an implementation may 658 always assume that labels are available when a session is 659 re-established. In this case, it is possible that it may 660 throw away the 'No label resources' information from the 661 previous incarnation of the session and may send a batch 662 of LDP messages on session re-establishment that will 663 fail and that it could have known would fail. 665 Note that the sender of a 'Label resources available' 666 Notification message may choose whether to add a sequence 667 number requesting acknowledgement. Conversely, the 668 receiver of 'Label resources available' Notification 669 message may choose to acknowledge the message without 670 actually saving any state. 672 This is an implementation choice made possible by making 673 the FT parameters on the Notification message optional. 674 Implementations will interoperate fully if they take 675 opposite approaches, but additional LDP messages may be 676 sent unnecessarily on session recovery. 678 4. FT Operations 680 Once an FT LDP session has been established, using the S 681 bit in the FT Session TLV on the Session Initialization 682 message as described in the section "Establishing an FT 683 LDP Session", both LDP peers MUST apply the procedures 684 described in this section for FT LDP message exchanges. 686 If the LDP session has been negotiated to not use the LDP 687 FT enhancements, these procedures MUST NOT be used. 689 4.1. FT LDP Messages 691 4.1.1 Sequence Numbered FT Label Messages 693 A label is identified as being a Sequence Numbered FT 694 Label if the initial Label Request or Label Mapping 695 message relating to that label carries the FT Protection 696 TLV. 698 It is a valid implementation option to flag all labels as 699 Sequence Numbered FT Labels. Indeed this may be a 700 preferred option for implementations wishing to use 701 Keepalive messages carrying the FT Protection TLV to 702 achieve periodic saves of the complete label forwarding 703 state. 705 If a label is a Sequence Numbered FT Label, all LDP 706 messages affecting that label MUST carry the FT 707 Protection TLV in order that the state of the label can 708 be recovered after a failure of the LDP session. 710 A valid option is for no labels to be Sequence Numbered 711 FT Labels. In this case checkpointing using the 712 Keepalive message applies to all messages exchanged on 713 the session. 715 4.1.1.1 Scope of FT Labels 717 The scope of the FT/non-FT status of a label is limited 718 to the LDP message exchanges between a pair of LDP peers. 720 In Ordered Control, when the message is forwarded 721 downstream or upstream, the TLV may be present or absent 722 according to the requirements of the LSR sending the 723 message. 725 If a platform-wide label space is used for FT Labels, an 726 FT Label value MUST NOT be reused until all LDP FT peers 727 to which the label was passed have acknowledged the 728 withdrawal of the FT Label, either by an explicit LABEL 729 WITHDRAW/LABEL RELEASE exchange or implicitly if the LDP 730 session is reconnected after failure but without the FT 731 Reconnect Flag set. In the event that a session is not 732 re-established within the Reconnection Timeout, a label 733 MAY become available for re-use if it is not still in use 734 on some other session. 736 4.1.2 FT Address Messages 738 If an LDP session uses the LDP FT enhancements, both LDP 739 peers MUST secure Address and Address Withdraw messages 740 using FT Operation ACKs, as described below. This avoids 741 any ambiguity over whether an Address is still valid 742 after the LDP session is reconnected. 744 If an LSR determines that an Address message that it sent 745 on a previous instantiation of a recovered LDP session is 746 no longer valid, it MUST explicitly issue an Address 747 Withdraw for that address when the session is 748 reconnected. 750 If the FT Reconnect Flag is not set by both LDP peers on 751 reconnection of an LDP session (i.e. state has not been 752 preserved), both LDP peers MUST consider all Addresses to 753 have been withdrawn. The LDP peers SHOULD issue new 754 Address messages for all their valid addresses as 755 specified in [RFC3036]. 757 4.1.3 Label Resources Available Notifications 759 In LDP, it is possible that a downstream LSR may not have 760 labels available to respond to a Label Request. In this 761 case, as specified in RFC3036, the downstream LSR must 762 respond with a Notification - No Label Resources message. 763 The upstream LSR then suspends asking for new labels 764 until it receives a Notification - Label Resources 765 Available message from the downstream LSR. 767 When the FT extensions are used on a session, 768 implementations may choose whether to secure the label 769 resource state of their peer or not. This choice impacts 770 the number of LDP messages that will be incorrectly 771 routed to a peer with depleted resources on session re- 772 establishment, but does not otherwise impact 773 interoperability. 775 For full preservation of state: 777 - The downstream LSR must preserve the label availability 778 state across a failover so that it remembers to send 779 Notification - Label Resources Available when the resources 780 become available. 782 - The upstream LSR must recall the label availability state 783 across failover so that it can optimize not sending Label 784 Requests when it recovers. 786 - The downstream LSR must use sequence numbers on 787 Notification - Label Resources Available so that it can 788 check that LSR A has received the message and clear its 789 secured state, or resend the message if LSR A recovers 790 without having received it. 792 However, the following options also exist: 794 - The downstream LSR may choose to not include a sequence 795 number on Notification - Label Resources Available. This 796 means that on session re-establishment it does not know what 797 its peer thinks the LSR's resource state is, because the 798 Notification may or may not have been delivered. Such an 799 implementation MUST begin recovered sessions by sending an 800 additional Notification - Label Resources Available to reset 801 its peer. 803 - The upstream node may choose not to secure information 804 about its peer's resource state. It would acknowledge a 805 Notification - Label Resources Available, but would not save 806 the information. Such an implementation MUST assume that 807 its peer's resource satte has been reset to Label Resources 808 Available when the session is re-established. 810 If the FT Reconnect Flag is not set by both LDP peers on 811 reconnection of an LDP session (i.e. state has not been 812 preserved), both LDP peers MUST consider the label 813 availability state to have been reset as if the session 814 had been set up for the first time. 816 4.2. FT Operation ACKs 818 Handshaking of FT LDP messages is achieved by use of 819 ACKs. Correlation between the original message and the 820 ACK is by means of the FT Sequence Number contained in 821 the FT Protection TLV, and passed back in the FT ACK TLV. 822 The FT ACK TLV may be carried on any LDP message that is 823 sent on the TCP connection between LDP peers. 825 An LDP peer maintains a separate FT sequence number for 826 each LDP session it participates in. The FT Sequence 827 number is incremented by one for each FT LDP message 828 (i.e. containing the FT Protection TLV) issued by this 829 LSR on the FT LDP session with which the FT sequence 830 number is associated. 832 When an LDP peer receives a message containing the FT 833 Protection TLV, it MUST take steps to secure this message 834 (or the state information derived from processing the 835 message). Once the message is secured, it MUST be ACKed. 836 However, there is no requirement on the LSR to send this 837 ACK immediately. 839 ACKs may be accumulated to reduce the message flow 840 between LDP peers. For example, if an LSR received FT 841 LDP messages with sequence numbers 1, 2, 3, 4, it could 842 send a single ACK with sequence number 4 to ACK receipt 843 and securing of all these messages. There is no protocol 844 reason why the number of ACKs accumulated or the time for 845 which an ACK is deferred should not be allowed to become 846 relatively large. 848 ACKs MUST NOT be sent out of sequence, as this is 849 incompatible with the use of accumulated ACKs. Duplicate 850 ACKs (that is two successive messages that acknowledge 851 the same sequence number) are acceptable. 853 If an LDP peer discovers that its sequence number space 854 for a specific session is full of un-acknowledged 855 sequence numbers (because its partner on the session has 856 not acknowledged them in a timely way) it cannot allocate 857 a new sequence number for any further FT LPD message. It 858 SHOULD send a Notification message with the status code 859 "FT Seq Numbers Exhausted". 861 4.3. Preservation of FT State 863 If the LDP FT enhancements are in use on an LDP session, 864 each LDP peer SHOULD NOT release the state information 865 and resources associated with FT Labels exchanged on that 866 LDP session when the TCP connection fails. This is 867 contrary to [RFC3212] and [RFC3036], but allows label 868 operations on FT Labels to be completed after re- 869 connection of the TCP connection. 871 Both LDP peers on an LDP session that is using the LDP FT 872 enhancements SHOULD preserve the state information and 873 resources they hold for that LDP session as described 874 below. 876 - An upstream LDP peer SHOULD release the resources (in 877 particular bandwidth) associated with a Sequence Numbered FT 878 Label when it initiates a Label Release or Label Abort 879 message for the label. The upstream LDP peer MUST preserve 880 state information for the Sequence Numbered FT Label, even 881 if it releases the resources associated with the label, as 882 it may need to reissue the label operation if the TCP 883 connection is interrupted. 885 - An upstream LDP peer MUST release the state information 886 and resources associated with a Sequence Numbered FT Label 887 when it receives an acknowledgement to a Label Release or 888 Label Abort message that it sent for the label, or when it 889 sends a Label Release message in response to a Label 890 Withdraw message received from the downstream LDP peer. 892 - A downstream LDP peer SHOULD NOT release the resources 893 associated with a Sequence Numbered FT Label when it sends a 894 Label Withdraw message for the label as it has not yet 895 received confirmation that the upstream LDP peer has ceased 896 to send data using the label. The downstream LDP peer MUST 897 NOT release the state information it holds for the label as 898 it may yet have to reissue the label operation if the TCP 899 connection is interrupted. 901 - A downstream LDP peer MUST release the resources and 902 state information associated with a Sequence Numbered FT 903 Label when it receives an acknowledgement to a Label 904 Withdraw message for the label. 906 - When the FT Reconnection Timeout expires, an LSR SHOULD 907 release all state information and resources from previous 908 instantiations of the (permanently) failed LDP session. 910 - Either LDP peer MAY elect to release state information 911 based on its internal knowledge of the loss of integrity of 912 the state information or an inability to pend (or queue) LDP 913 operations (as described in section 4.4.1) during a TCP 914 failure. That is, the peer is not required to wait for the 915 duration of the FT Reconnection Timeout before releasing 916 state; the timeout provides an upper limit on the 917 persistence of state. However, in the event that a peer 918 releases state before the expiration of the Reconnection 919 Timeout it MUST NOT re-use any label that was in use on the 920 session until the Reconnection Timeout has expired. 922 - When an LSR receives a Status TLV with the E-bit set in 923 the status code, which causes it to close the TCP 924 connection, the LSR MUST release all state information and 925 resources associated with the session. This behavior is 926 mandated because it is impossible for the LSR to predict the 927 precise state and future behavior of the partner LSR that 928 set the E-bit without knowledge of the implementation of 929 that partner LSR. 931 Note that the "Temporary Shutdown" status code does not have 932 the E-bit set, and MAY be used during maintenance or upgrade 933 operations to indicate that the LSR intends to preserve 934 state across a closure and re-establishment of the TCP 935 session. 937 - If an LSR determines that it must release state for any 938 single FT Label during a failure of the TCP connection on 939 which that label was exchanged, it MUST release all state 940 for all labels on the LDP session. 942 The release of state information and resources associated 943 with non-FT labels is as described in [RFC3212] and 944 [RFC3036]. 946 Note that a Label Release and the acknowledgement to a 947 Label Withdraw may be received by a downstream LSR in any 948 order. The downstream LSR MAY release its resources on 949 receipt of the first message and MUST release its 950 resources on receipt of the second message. 952 4.4. FT Procedure After TCP Failure 954 When an LSR discovers or is notified of a TCP connection 955 failure it SHOULD start an FT Reconnection Timer to allow 956 a period for re-connection of the TCP connection between 957 the LDP peers. 959 The RECOMMENDED default value for this timer is 5 960 seconds. During this time, failure must be detected and 961 reported, new hardware may need to be activated, software 962 state must be audited, and a new TCP session must be set 963 up. 965 Once the TCP connection between LDP peers has failed, the 966 active LSR SHOULD attempt to re-establish the TCP 967 connection. The mechanisms, timers and retry counts to re- 968 establish the TCP connection are an implementation 969 choice. It is RECOMMENDED that any attempt to re- 970 establish the connection take account of the failover 971 processing necessary on the peer LSR, the nature of the 972 network between the LDP peers, and the FT Reconnection 973 Timeout chosen on the previous instantiation of the TCP 974 connection (if any). 976 If the TCP connection cannot be re-established within the 977 FT Reconnection Timeout period, the LSR detecting this 978 timeout SHOULD release all state preserved for the failed 979 LDP session. If the TCP connection is subsequently re- 980 established (for example, after a further Hello exchange 981 to set up a new LDP session), the LSR MUST set the FT 982 Reconnect Flag to 0 if it released the preserved state 983 information on this timeout event. 985 If the TCP connection is successfully re-established 986 within the FT Reconnection Timeout, both peers MUST re- 987 issue LDP operations that were interrupted by (that is, 988 un-acknowledged as a result of) the TCP connection 989 failure. This procedure is described in section "FT 990 Procedure After TCP Re-connection". 992 The Hold Timer for an FT LDP Session (see [RFC3036] 993 section 2.5.5) SHOULD be ignored while the FT 994 Reconnection Timer is running. The hold timer SHOULD be 995 restarted when the TCP connection is re-established. 997 4.4.1 FT LDP Operations During TCP Failure 999 When the LDP FT enhancements are in use for an LDP 1000 session, it is possible that an LSR may determine that it 1001 needs to send an LDP message to an LDP peer but that the 1002 TCP connection to that peer is currently down. These 1003 label operations affect the state of FT Labels preserved 1004 for the failed TCP connection, so it is important that 1005 the state changes are passed to the LDP peer when the TCP 1006 connection is restored. 1008 If an LSR determines that it needs to issue a new FT LDP 1009 operation to an LDP peer to which the TCP connection is 1010 currently failed, it MUST pend the operation (e.g. on a 1011 queue) and complete that operation with the LDP peer when 1012 the TCP connection is restored, unless the label 1013 operation is overridden by a subsequent additional 1014 operation during the TCP connection failure (see section 1015 "FT Procedure After TCP Re-connection"). 1017 If, during TCP Failure, an LSR determines that it cannot 1018 pend an operation which it cannot simply fail (for 1019 example, a Label Withdraw, Release or Abort operation), 1020 it MUST NOT attempt to re-establish the previous LDP 1021 session. The LSR MUST behave as if the Reconnection 1022 Timer expired and release all state information with 1023 respect to the LDP peer. An LSR may be unable (or 1024 unwilling) to pend operations; for instance, if a major 1025 routing transition occurred while TCP was inoperable 1026 between LDP peers it might result in excessively large 1027 numbers of FT LDP Operations. An LSR that releases state 1028 before the expiration of the Reconnection Timeout MUST 1029 NOT re-use any label that was in use on the session until 1030 the Reconnection Timeout has expired. 1032 In ordered operation, received FT LDP operations that 1033 cannot be correctly forwarded because of a TCP connection 1034 failure MAY be processed immediately (provided sufficient 1035 state is kept to forward the label operation) or pended 1036 for processing when the onward TCP connection is restored 1037 and the operation can be correctly forwarded upstream or 1038 downstream. Operations on existing FT Labels SHOULD NOT 1039 be failed during TCP session failure. 1041 It is RECOMMENDED that Label Request operations for new 1042 FT Labels are not pended awaiting the re-establishment of 1043 TCP connection that is awaiting recovery at the time the 1044 LSR determines that it needs to issue the Label Request 1045 message. Instead, such Label Request operations SHOULD 1046 be failed and, if necessary, a notification message 1047 containing the "No LDP Session" status code sent 1048 upstream. 1050 Label Requests for new non-FT Labels MUST be rejected 1051 during TCP connection failure, as specified in [RFC3212] 1052 and [RFC3036]. 1054 4.5. FT Procedure After TCP Re-connection 1056 The FT operation handshaking described above means that 1057 all state changes for Sequence Numbered FT Labels and 1058 Address messages are confirmed or reproducible at each LSR. 1060 If the TCP connection between LDP peers fails but is re- 1061 connected within the FT Reconnection Timeout, and both 1062 LSRs have indicated they will be re-establishing the 1063 previous LDP session, both LDP peers on the connection 1064 MUST complete any label operations for Sequence Numbered 1065 FT Labels that were interrupted by the failure and re- 1066 connection of the TCP connection. 1068 The procedures for FT Reconnection Timeout MAY have been 1069 invoked as a result of either LDP peer being unable (or 1070 unwilling) to pend operations which occurred during the 1071 TCP Failure (as described in section 4.4.1). 1073 If, for any reason, an LSR has been unable to pend operations 1074 with respect to an LDP peer, as described in section 4.4.1, the 1075 LSR MUST set the FT Reconnect Flag to 0 on re-connection to that 1076 LDP peer indicating that no FT state has been preserved. 1078 Label operations are completed using the procedure described below. 1080 4.5.1 Re-Issuing FT Messages 1082 On restoration of the TCP connection between LDP peers, 1083 any LDP messages for Sequence Numbered FT Labels that 1084 were lost because of the TCP connection failure are re- 1085 issued. The LDP peer that receives a re-issued message 1086 processes the message as if received for the first time. 1088 "Net-zero" combinations of messages need not be re-issued 1089 after re-establishment of the TCP connection between LDP 1090 peers. This leads to the following rules for re-issuing 1091 messages that are not ACKed by the LDP peer on the LDP 1092 Initialization message exchange after re-connection of 1093 the TCP session. 1095 - A Label Request message MUST be re-issued unless a Label Abort 1096 would be re-issued for the same Sequence Numbered FT Label. 1098 - A Label Mapping message MUST be re-issued unless a Label 1099 Withdraw message would be re-issued for the same Sequence 1100 Numbered FT Label. 1102 - All other messages on the LDP session that carried the FT 1103 Protection TLV MUST be re-issued if an acknowledgement had 1104 not previously been received. 1106 Any FT Label operations that were pended (see section 1107 4.4.1) during the TCP connection failure MUST also be 1108 issued on re-establishment of the LDP session, except 1109 where they form part of a "net-zero" combination of 1110 messages according to the above rules. 1112 The determination of "net-zero" FT Label operations 1113 according to the above rules MAY be performed on pended 1114 messages prior to the re-establishment of the TCP 1115 connection in order to optimize the use of queue 1116 resources. Messages that were sent to the LDP peer 1117 before the TCP connection failure, or pended messages 1118 that are paired with them, MUST NOT be subject to such 1119 optimization until an FT ACK TLV is received from the LDP 1120 peer. This ACK allows the LSR to identify which messages 1121 were received by the LDP peer prior to the TCP connection 1122 failure. 1124 4.5.2 Interaction with CR-LDP LSP Modification 1126 Re-issuing LDP messages for FT operation is orthogonal to 1127 the use of duplicate messages marked with the Modify 1128 ActFlg, as specified in [RFC3214]. Each time an LSR uses 1129 the modification procedure for an FT LSP to issue a new 1130 Label Request message, the FT label operation procedures 1131 MUST be separately applied to the new Label Request 1132 message to the same degree that they were applied to the 1133 original Label Request. 1135 5. Checkpointing Procedures 1137 Checkpointing can be selected independently from the FT 1138 procedures described above by using the C bit in the FT 1139 Session TLV on the Session Initialization message. Note, 1140 however, that checkpointing is an integral part of the FT 1141 procedures. Setting the S and the C bit will achieve the 1142 same function as setting just the S bit. 1144 If the C bit is set, but the S bit is not set, no label 1145 is aSequence Numbered FT Label. Instead, all labels are 1146 Checkpointable FT Labels. Checkpointing is used to 1147 synchronize all labels exchanges. No message apart from 1148 the checkpoint request and acknowledgement carries an 1149 active sequence number. (Note that the Session 1150 Initialization message may carry a sequence number to 1151 confirm that the checkpoint is still in place). 1153 It is an implementation matter to decide the ordering of 1154 received messages and checkpoint requests to ensure that 1155 checkpoint acknowledgements are secured. 1157 If the S and C bits are both set, or only the S bit is 1158 set, checkpointing applies only to Sequence Numbered FT 1159 Labels and to address messages. 1161 The set of all messages that are checkpointed in this way 1162 is called the Checkpointable Messages. 1164 5.1. Checkpointing with the Keepalive Message 1166 If an LSR receives a FT Protection TLV on a Keepalive 1167 message, this is a request to flush the acknowledgements 1168 for all previously received Checkpointable Messages on 1169 the session. 1171 As soon as the LSR has completed securing the 1172 Checkpointable Messages (or state changes consequent on 1173 those messages) received before the Keepalive, it MUST 1174 send an acknowledgement to the sequence number of the 1175 Keepalive message. 1177 In the case where the FT procedures are in use and 1178 acknowledgements have been stored up, this may be 1179 immediately on receipt of the Keepalive. 1181 An example message flow showing this use of the Keepalive 1182 message to perform a periodic check-point of state is 1183 shown in section 8. 1185 An example message flow showing the use of checkpointing 1186 without the FT procedures is shown in section 8. 1188 5.2. Quiesce and Keepalive 1190 If the Keepalive Message also contains the FT Cork TLV, 1191 this indicates that the peer LSR wishes to quiesce the 1192 session prior to a graceful restart. 1194 It is RECOMMENDED that on receiving a Keepalive with the 1195 FT CORK TLV, an LSR should cease to send any further 1196 label or address related messages on the session until it 1197 has been disconnected and reconnected, other than any 1198 messages generated while processing and securing any 1199 previously unacknowledged messages received from the peer 1200 requesting the quiesce. It should also attempt to 1201 complete this processing and return a Keepalive with the 1202 FT ACK TLV as soon as possible in order to allow the 1203 session to be quiesced. 1205 An example message flow showing this use of the FT Cork 1206 TLV to achieves three-way handshake of state 1207 synchronization between two LDP peers is given in section 8. 1209 6. Changes to Existing Messages 1211 6.1. LDP Initialization Message 1213 The LDP FT enhancements add the following optional 1214 parameters to a LDP Initialization message 1216 Optional Parameter Length Value 1218 FT Session TLV 4 See below 1219 FT ACK TLV 4 See below 1221 The encoding for these TLVs is found in Section "New 1222 Fields and Values". 1224 FT Session 1225 If present, specifies the FT behavior of 1226 the LDP session. 1228 FT ACK TLV 1229 If present, specifies the last FT message 1230 that the sending LDP peer was able to 1231 secure prior to the failure of the previous 1232 instantiation of the LDP session. This TLV 1233 is only present if the FT Reconnect flag is 1234 set in the FT Session TLV, in which case 1235 this TLV MUST be present. 1237 6.2. LDP Keepalive Messages 1239 The LDP FT enhancements add the following optional 1240 parameters to a LDP Keepalive message 1242 Optional Parameter Length Value 1244 FT Protection TLV 4 See below 1245 FT Cork TLV 0 See below 1246 FT ACK TLV 4 See below 1248 The encoding for these TLVs is found in Section "New 1249 Fields and Values". 1251 FT Protection 1252 If present, specifies FT Sequence Number 1253 for the LDP message. When present on a 1254 Keepalive message, this indicates a 1255 solicited flush of the acknowledgements to 1256 all previous LDP messages containing 1257 sequence numbers and issued by the sender 1258 of the Keepalive on the same session. 1260 FT Cork 1261 Indicates that the remote LSR wishes to 1262 quiesce the LDP session. See section 5 for 1263 the recommended action in such cases. 1265 FT ACK TLV 1266 If present, specifies the most recent FT 1267 message that the sending LDP peer has been 1268 able to secure. 1270 6.3. All Other LDP Session Messages 1272 The LDP FT enhancements add the following optional 1273 parameters to all other message types that flow on an LDP 1274 session after the LDP Initialization message 1276 Optional Parameter Length Value 1278 FT Protection TLV 4 See below 1279 FT ACK TLV 4 See below 1281 The encoding for these TLVs is found in the section "New 1282 Fields and Values". 1284 FT Protection 1285 If present, specifies FT Sequence Number 1286 for the LDP message. 1288 FT ACK 1289 If present, identifies the most recent FT 1290 LDP message ACKed by the sending LDP peer. 1292 7. New Fields and Values 1294 7.1. Status Codes 1296 The following new status codes are defined to indicate 1297 various conditions specific to the LDP FT enhancements. 1298 These status codes are carried in the Status TLV of a 1299 Notification message. 1301 The "E" column is the required setting of the Status Code 1302 E-bit; the "Status Data" column is the value of the 30- 1303 bit Status Data field in the Status Code TLV. 1305 Note that the setting of the Status Code F-bit is at the 1306 discretion of the LSR originating the Status TLV. 1307 However, it is RECOMMENDED that the F-bit is not set on 1308 Notification messages containing status codes except "No 1309 LDP Session" because the duplication of messages SHOULD 1310 be restricted to being a per-hop behavior. 1312 Status Code E Status Data 1314 No LDP Session 0 0x000000xx 1315 Zero FT seqnum 1 0x000000xx 1316 Unexpected TLV / 1 0x000000xx 1317 Session Not FT 1318 Unexpected TLV / 1 0x000000xx 1319 Label Not FT 1320 Missing FT Protection TLV 1 0x000000xx 1321 FT ACK sequence error 1 0x000000xx 1322 Temporary Shutdown 0 0x000000xx 1323 FT Seq Numbers Exhausted 1 0x000000xx 1324 FT Session parameters / 1 0x000000xx 1325 changed 1326 Unexpected FT Cork TLV 1 0x000000xx 1328 The Temporary Shutdown status code SHOULD be used in 1329 place of the Shutdown status code (which has the E-bit 1330 set) if the LSR that is shutting down wishes to inform 1331 its LDP peer that it expects to be able to preserve FT 1332 Label state and to return to service before the FT 1333 Reconnection Timer expires. 1335 7.2. FT Session TLV 1337 LDP peers can negotiate whether the LDP session between 1338 them supports FT extensions by using a new OPTIONAL 1339 parameter, the FT Session TLV, on LDP Initialization 1340 Messages. 1342 The FT Session TLV is encoded as follows. 1344 0 1 2 3 1345 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1346 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1347 |1|0| FT Session TLV (0x0503) | Length (= 12) | 1348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1349 | FT Flags | Reserved | 1350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1351 | FT Reconnect Timeout (in milliseconds) | 1352 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1353 | Recovery Time (in milliseconds) | 1354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1356 FT Flags 1358 FT Flags: A 16 bit field that indicates 1359 various attributes the FT support on this 1360 LDP session. This fields is formatted as 1361 follows: 1363 0 1 1364 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1365 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1366 |R| Reserved |S|A|C|L| 1367 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1369 R: FT Reconnect Flag. 1371 Set to 1 if the sending LSR has 1372 preserved state and resources for all 1373 FT-labels since the previous LDP 1374 session between the same LDP peers, 1375 and set to 0 otherwise. See the 1376 section "FT LDP Session Reconnection" 1377 for details of how this flag is used. 1379 If the FT Reconnect Flag is set, the 1380 sending LSR MUST include an FT ACK TLV 1381 on the LDP Initialization message. 1383 S: Save State Flag. 1385 Set to 1 if the use of the FT 1386 Protection TLV is supported on 1387 messages other than the KeepAlive 1388 message used for chekpointing (see the 1389 C bit). I.e., the S bit indicates hat 1390 some label on the session may be a 1391 Sequence Numbered FT Label. 1393 A: All-Label Protection Required 1395 Set to 1 if all labels on the session 1396 MUST be treated as Sequence Numbered 1397 FT Labels. This removes from a node 1398 the option of treating some labels as 1399 FT Labels and some labels as non-FT 1400 Labels. 1402 Passing this information may be 1403 considered helpful to a peer since it 1404 may allow it to make optimizations in 1405 its processing. 1407 The A bit only has meaning if the S 1408 bit is set. 1410 C: Checkpointing Flag. 1412 Set to 1 to indicate that the 1413 checkpointing procedures in this draft 1414 are in use. 1416 If the S bit is also set to 1 then the 1417 C bit indicates that checkpointing is 1418 applied only to Sequence Numbered FT 1419 Labels. 1421 If the S bit is set to 0 (zero) then 1422 the C bit indicates that checkpointing 1423 applies to all labels - all labels are 1424 Checkpointable FT Labels. 1426 L: Learn From Network Flag. 1428 Set to 1 if the Fault Recovery 1429 procedures of [LDP-RESTART] are to be 1430 used to re-learn state from the 1431 network. 1433 It is not valid for all of the S, C and L 1434 bits to be zero. 1436 It is not valid for both the L and either 1437 the S or C bits to be set to 1. 1439 All other bits in this field are currently 1440 reserved and SHOULD be set to zero on 1441 transmission and ignored on receipt. 1443 The following table summarizes the settings 1444 of these bits. 1446 S A C L Comments 1447 ========================= 1448 0 x 0 0 Invalid 1449 0 x 0 1 See [LDP-RESTART] 1450 0 x 1 0 Checkpointing of all labels 1451 0 x 1 1 Invalid 1452 1 0 0 0 Full FT on selected labels 1453 1 1 0 0 Full FT on all labels 1454 1 x 0 1 Invalid 1455 1 x 1 0 Same as (S=1,A=x,C=0,L=0) 1456 1 x 1 1 Invalid. 1458 FT Reconnection Timeout 1460 If the S bit or C bit in the FT Flags field 1461 is set this indicates the period of time 1462 the sending LSR will preserve state and 1463 resources for FT Labels exchanged on the 1464 previous instantiation of an FT LDP session 1465 that has currently failed. The timeout is 1466 encoded as a 32-bit unsigned integer number 1467 of milliseconds. 1469 A value of zero in this field means that 1470 the sending LSR will preserve state and 1471 resources indefinitely. 1473 See section 4.4 for details of how this 1474 field is used. 1476 If the L bit is set to 1 in the FT Flags 1477 field, the meaning of this field is defined 1478 in [LDP-RESTART]. 1480 Recovery Time 1482 The Recovery Time only has meaning if the L 1483 bit is set in the FT Flags. The meaning is 1484 defined in [LDP-RESTART]. 1486 7.3. FT Protection TLV 1488 LDP peers use the FT Protection TLV to indicate that an 1489 LDP message contains an FT label operation. 1491 The FT Protection TLV MUST NOT be used in messages 1492 flowing on an LDP session that does not support the LDP 1493 FT enhancements. Its presence in such messages SHALL be 1494 treated as a protocol error by the receiving LDP peer 1495 which SHOULD send a Notification message with the 1496 'Unexpected TLV Session Not FT' status code. LSRs that 1497 do not recognize this TLV SHOULD respond with a 1498 Notification message with the 'Unknown TLV' status code. 1500 The FT Protection TLV MAY be carried on an LDP message 1501 transported on the LDP session after the initial exchange 1502 of LDP Initialization messages. In particular, this TLV 1503 MAY optionally be present on the following messages: 1505 - Label Request Messages in downstream on-demand 1506 distribution mode 1508 - Label Mapping messages in downstream unsolicited mode 1510 - Keepalive messages used to request flushing of 1511 acknowledgement of all previous messages that contained this 1512 TLV. 1514 If a label is to be a Sequence Numbered FT Label, then 1515 the Protection TLV MUST be present: 1517 - on the Label Request message in DoD mode 1519 - on the Label Mapping message in DU mode 1521 - on all subsequent messages concerning this label. 1523 Here 'subsequent messages concerning this label' means 1524 any message whose Label TLV specifies this label or whose 1525 Label Request Message ID TLV specifies the initial Label 1526 Request message. 1528 If a label is not to be a Sequence Numbered FT Label, 1529 then the Protection TLV MUST NOT be present on any of 1530 these messages that relate to the label. The presence of 1531 the FT TLV on a message relating to a non-FT Label SHALL 1532 be treated as a protocol error by the receiving LDP peer 1533 which SHOULD send a notification message with the 1534 'Unexpected TLV Label Not FT' status code. 1536 Where a Label Withdraw or Label Release message contains 1537 only a FEC TLV and does not identify a single specific 1538 label, the FT TLV should be included in the message if 1539 any label affected by the message is a Sequence Numbered 1540 FT Label. If there is any doubt as to whether an FT TLV 1541 should be present, it is RECOMMENDED that the sender add 1542 the TLV. 1544 When an LDP peer receives a Label Withdraw Message or 1545 Label Release message that contains only a FEC, it SHALL 1546 accept the FT TLV if it is present regardless of the FT 1547 status of the labels which it affects. 1549 If an LDP session is an FT session as determined by the 1550 presence of the FT Session TLV with the S bit set on the 1551 LDP Initialization messages, the FT Protection TLV MUST 1552 be present on all Address messages on the session. 1554 If the session is an FT session, the FT Protection TLV 1555 may also optionally be present 1557 - on Notification messages on the session that have the 1558 status code 'Label Resources Available' 1560 - on Keepalive messages. 1562 The FT Protection TLV is encoded as follows. 1564 0 1 2 3 1565 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1566 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1567 |0|0| FT Protection (0x0203) | Length (= 4) | 1568 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1569 | FT Sequence Number | 1570 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1572 FT Sequence Number 1574 The sequence number for this Sequence 1575 Numbered FT Label operation. The sequence 1576 number is encoded as a 32-bit unsigned 1577 integer. The initial value for this field 1578 on a new LDP session is 0x00000001 and is 1579 incremented by one for each FT LDP message 1580 issued by the sending LSR on this LDP 1581 session. This field may wrap from 1582 0xFFFFFFFF to 0x00000001. 1584 This field MUST be reset to 0x00000001 if 1585 either LDP peer does not set the FT 1586 Reconnect Flag on re-establishment of the 1587 TCP connection. 1589 See the section "FT Operation Acks" for 1590 details of how this field is used. 1592 The special use of 0x00000000 is discussed 1593 in the section "FT ACK TLV" below. 1595 If an LSR receives an FT Protection TLV on a session that 1596 does not support the FT LDP enhancements, it SHOULD send 1597 a Notification message to its LDP peer containing the 1598 "Unexpected TLV, Session Not FT" status code. LSRs that 1599 do not recognize this TLV SHOULD respond with a 1600 Notification message with the 'Unknown TLV' status code. 1602 If an LSR receives an FT Protection TLV on an operation 1603 affecting a label that it believes is a non-FT Label, it 1604 SHOULD send a Notification message to its LDP peer 1605 containing the "Unexpected TLV, Label Not FT" status 1606 code. 1608 If an LSR receives a message without the FT Protection 1609 TLV affecting a label that it believes is a Sequence 1610 Numbered FT Label, it SHOULD send a Notification message 1611 to its LDP peer containing the "Missing FT Protection 1612 TLV" status code. 1614 If an LSR receives an FT Protection TLV containing a zero 1615 FT Sequence Number, it SHOULD send a Notification message 1616 to its LDP peer containing the "Zero FT Seqnum" status 1617 code. 1619 7.4. FT ACK TLV 1621 LDP peers use the FT ACK TLV to acknowledge FT Label 1622 operations. 1624 The FT ACK TLV MUST NOT be used in messages flowing on an 1625 LDP session that does not support the LDP FT 1626 enhancements. Its presence on such messages SHALL be 1627 treated as a protocol error by the receiving LDP peer. 1629 The FT ACK TLV MAY be present on any LDP message 1630 exchanged on an LDP session after the initial LDP 1631 Initialization messages. It is RECOMMENDED that the FT 1632 ACK TLV is included on all FT Keepalive messages in order 1633 to ensure that the LDP peers do not build up a large 1634 backlog of unacknowledged state information. 1636 The FT ACK TLV is encoded as follows. 1638 0 1 2 3 1639 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1640 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1641 |0|0| FT ACK (0x0504) | Length (= 4) | 1642 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1643 | FT ACK Sequence Number | 1644 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1645 FT ACK Sequence Number 1647 The sequence number for the most recent FT 1648 label message that the sending LDP peer has 1649 received from the receiving LDP peer and 1650 secured against failure of the LDP session. 1651 It is not necessary for the sending peer to 1652 have fully processed the message before 1653 ACKing it. For example, an LSR MAY ACK a 1654 Label Request message as soon as it has 1655 securely recorded the message, without 1656 waiting until it can send the Label Mapping 1657 message in response. 1659 ACKs are cumulative. Receipt of an LDP 1660 message containing an FT ACK TLV with an FT 1661 ACK Sequence Number of 12 is treated as the 1662 acknowledgement of all messages from 1 to 1663 12 inclusive (assuming the LDP session 1664 started with a sequence number of 1). 1666 This field MUST be set to 0 if the LSR 1667 sending the FT ACK TLV has not received any 1668 FT label operations on this LDP session. 1669 This would apply to LDP sessions to new LDP 1670 peers or after an LSR determines that it 1671 must drop all state for a failed TCP 1672 connection. 1674 See the section "FT Operation Acks" for 1675 details of how this field is used. 1677 If an LSR receives a message affecting a label that it 1678 believes is a Sequence Numbered FT Label and that message 1679 does not contain the FT Protection TLV, it SHOULD send a 1680 Notification message to its LDP peer containing the 1681 "Missing FT Protection TLV" status code. 1683 If an LSR receives an FT ACK TLV that contains an FT ACK 1684 Sequence Number that is less than the previously received 1685 FT ACK Sequence Number (remembering to take account of 1686 wrapping), it SHOULD send a Notification message to its 1687 LDP peer containing the "FT ACK Sequence Error" status 1688 code. 1690 7.5. FT Cork TLV 1692 LDP peers use the FT Cork TLV on FT Keepalive messages to 1693 indicate that they wish to quiesce the LDP session prior 1694 to a controlled shutdown and restart, for example during 1695 control-plane software upgrade. 1697 The FT Cork TLV is encoded as follows. 1699 0 1 2 3 1700 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1701 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1702 |0|0| FT Cork (0x0505) | Length (= 0) | 1703 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1705 On receipt of a Keepalive message with the FT Cork TLV and the FT 1706 Protection TLV, an LSR SHOULD perform the following actions. 1708 - Process and secure any messages from the peer LSR that 1709 have sequence numbers less than (accounting for wrap) that 1710 contained in the FT Protection TLV on the Keepalive message. 1712 - Send a Keepalive message back to the peer containing the 1713 FT Cork TLV and the FT ACK TLV specifying the FT ACK 1714 sequence number equal to that in the original Keepalive 1715 message (i.e. ACKing all messages up to that point). 1717 - If this LSR has not yet received an FT ACK to all the 1718 messages it has sent containing the FT Protection TLV, then 1719 also include an FT Protection TLV on the Keepalive sent to 1720 the peer LSR. This tells the remote peer that the local LSR 1721 has saved state prior to quiesce but is still awaiting 1722 confirmation that the remote peer has saved state. 1724 - Cease sending any further state changing messages on this 1725 LDP session until it has been disconnected and recovered. 1727 On receipt of a Keepalive message with the FT Cork TLV 1728 and an FT ACK TLV that acknowledges the previously sent 1729 Keepalive that carried the FT Cork TLV, an LSR knows that 1730 quiesce is complete. If the received Keepalive also 1731 carries the FT Protection TLV, the LSR must respond with 1732 a further Keepalive to complete the 3-way handshake. It 1733 SHOULD now send a "Temporary Shutdown" Notification 1734 message, disconnect the TCP session and perform whatever 1735 control plane actions required this session shutdown. 1737 An example such 3-way handshake for controlled shutdown 1738 is given in section 8. 1740 If an LSR receives a message that should not carry the FT 1741 Cork TLV, or if the FT Cork TLV is used on a Keepalive 1742 message without one of the FT Protection or FT ACK TLVs 1743 present, , it SHOULD send a Notification message to its 1744 LDP peer containing the "Unexpected FR Cork TLV" status code. 1746 8. Example Use 1748 Consider two LDP peers, P1 and P2, implementing LDP over 1749 a TCP connection that connects them, and the message flow 1750 shown below. 1752 The parameters shown on each message shown below are as 1753 follows: 1755 message (label, senders FT sequence number, FT ACK 1756 number) 1758 A "-" for FT ACK number means that the FT ACK TLV is 1759 not included on that message. "n/a" means that the 1760 parameter in question is not applicable to that type 1761 of message. 1763 In the diagrams below, time flows from top to bottom. 1764 The relative position of each message shows when it is 1765 transmitted. See the notes for a description of when 1766 each message is received, secured for FT or processed. 1768 8.1. Session Failure and Recovery - FT Procedures 1770 notes P1 P2 1771 ===== == == 1772 (1) Label Request(L1,27,-) 1773 ---------------------------> 1774 Label Request(L2,28,-) 1775 ---------------------------> 1776 (2) Label Request(L3,93,27) 1777 <--------------------------- 1778 (3) Label Request(L1,123,-) 1779 --------------------------> 1780 Label Request(L2,124,-) 1781 --------------------------> 1782 (4) Label Mapping(L1,57,-) 1783 <-------------------------- 1784 Label Mapping(L1,94,28) 1785 <--------------------------- 1786 (5) Label Mapping(L2,58,-) 1787 <-------------------------- 1788 Label Mapping(L2,95,-) 1789 <--------------------------- 1790 (6) Address(n/a,29,-) 1791 ---------------------------> 1792 (7) Label Request(L4,30,-) 1793 ---------------------------> 1794 (8) Keepalive(n/a,-,94) 1795 ---------------------------> 1796 (9) Label Abort(L3,96,-) 1797 <--------------------------- 1798 (10) ===== TCP Session lost ===== 1799 : 1800 (11) : Label Withdraw(L1,59,-) 1801 : <-------------------------- 1802 : 1803 (12) === TCP Session restored === 1805 LDP Init(n/a,n/a,94) 1806 ---------------------------> 1807 LDP Init(n/a,n/a,29) 1808 <--------------------------- 1809 (13) Label Request(L4,30,-) 1810 ---------------------------> 1811 (14) Label Mapping(L2,95,-) 1812 <--------------------------- 1813 Label Abort(L3,96,30) 1814 <--------------------------- 1815 (15) Label Withdraw(L1,97,-) 1816 <--------------------------- 1818 Notes: 1819 ====== 1821 (1) Assume that the LDP session has already been initialized. 1822 P1 issues 2 new Label Requests using the next sequence numbers. 1824 (2) P2 issues a third Label request to P1. At the time of sending 1825 this request, P2 has secured the receipt of the label request 1826 for L1 from P1, so it includes an ACK for that message. 1828 (3) P2 Processes the Label Requests for L1 and L2 and forwards them 1829 downstream. Details of downstream processing are not shown in 1830 the diagram above. 1832 (4) P2 receives a Label Mapping from downstream for L1, which it 1833 forwards to P1. It includes an ACK to the Label Request for L2, 1834 as that message has now been secured and processed. 1836 (5) P2 receives the Label Mapping for L2, which it forwards to P1. 1837 This time it does not include an ACK as it has not received any 1838 further messages from P1. 1840 (6) Meanwhile, P1 sends a new Address Message to P2. 1842 (7) P1 also sends a fourth Label Request to P2 1844 (8) P1 sends a Keepalive message to P2, on which it includes an ACK 1845 for the Label Mapping for L1, which is the latest message P1 has 1846 received and secured at the time the Keepalive is sent. 1848 (9) P2 issues a Label Abort for L3. 1850 (10) At this point, the TCP session goes down. 1852 (11) While the TCP session is down, P2 receives a Label Withdraw 1853 Message for L1, which it queues. 1855 (12) The TCP session is reconnected and P1 and P2 exchange LDP 1856 Initialization messages on the recovered session, which include 1857 ACKS for the last message each peer received and secured prior 1858 to the failure. 1860 (13) From the LDP Init exchange, P1 determines that it needs to 1861 re-issue the Label request for L4. 1863 (14) Similarly, P2 determines that it needs to re-issue the Label 1864 Mapping for L2 and the Label Abort. 1866 (15) P2 issues the queued Label Withdraw to P1. 1868 8.2. Use of Check-Pointing With FT Procedures 1870 notes P1 P2 1871 ===== == == 1872 (1) Label Request(L1,27,-) 1873 ---------------------------> 1874 Label Request(L2,28,-) 1875 ---------------------------> 1876 (2) Label Request(L3,93,-) 1877 <--------------------------- 1878 (3) Label Request(L1,123,-) 1879 --------------------------> 1880 Label Request(L2,124,-) 1881 --------------------------> 1882 (4) Label Mapping(L1,57,-) 1883 <-------------------------- 1884 Label Mapping(L1,94,-) 1885 <--------------------------- 1886 (5) Label Mapping(L2,58,-) 1887 <-------------------------- 1888 Label Mapping(L2,95,-) 1889 <--------------------------- 1890 (6) Address(n/a,29,-) 1891 ---------------------------> 1892 (7) Label Request(L4,30,-) 1893 ---------------------------> 1894 (8) Keepalive(n/a,31,-) 1895 ---------------------------> 1896 (9) Keepalive(n/a,-,31) 1897 <--------------------------- 1898 (10) Keepalive(n/a,59,124) 1899 <--------------------------- 1900 (11) Keepalive(n/a,-,59) 1901 ---------------------------> 1903 Notes: 1904 ====== 1906 Notes (1) through (7) are as in the previous example 1907 except note that no acknowledgements are piggy-backed on 1908 reverse direction messages. This means that at note (8) 1909 there are deferred acknowledgements in both directions on 1910 both links. 1912 (8) P1 wishes to synchronize state with P2. It sends a Keepalive 1913 message containing a FT Protection TLV with sequence number 31. 1914 Since it is not interested in P2's perception of the state that 1915 it has stored, it does not include an FT ACK TLV. 1917 (9) P2 responds at once with a Keepalive acknowledging the sequence 1918 number on the received Keepalive. This tells P1 that P2 has 1919 preserved all state/messages previously received on this 1920 session. 1921 (10) P3 wishes to synchronize state with P2. It sends a Keepalive 1922 message containing a FT Protection TLV with sequence number 59. 1923 P3 also takes this opportunity to get up to date with its 1924 acknowledgements to P2 by including an FT ACK TLV acknowledging 1925 up to sequence number 124. 1927 (11) P2 responds at once with a Keepalive acknowledging the sequence 1928 number on the received Keepalive. 1930 8.3. Temporary Shutdown With FT Procedures 1932 notes P1 P2 1933 ===== == == 1934 (1) Label Request(L1,27,-) 1935 ---------------------------> 1936 Label Request(L2,28,-) 1937 ---------------------------> 1938 (2) Label Request(L3,93,27) 1939 <--------------------------- 1940 (3) Label Request(L1,123,-) 1941 --------------------------> 1942 Label Request(L2,124,-) 1943 --------------------------> 1944 (4) Label Mapping(L1,57,-) 1945 <-------------------------- 1946 Label Mapping(L1,94,28) 1947 <--------------------------- 1948 (5) Label Mapping(L2,58,-) 1949 <-------------------------- 1950 Label Mapping(L2,95,-) 1951 <--------------------------- 1952 (6) Address(n/a,29,-) 1953 ---------------------------> 1954 (7) Label Request(L4,30,-) 1955 ---------------------------> 1956 (8) Keepalive(n/a,-,94) 1957 ---------------------------> 1958 (9) Label Abort(L3,96,-) 1959 <--------------------------- 1960 (10) Notification(Temporary shutdown) 1961 ---------------------------> 1962 ===== TCP Session shutdown ===== 1963 : 1964 (11) : Label Withdraw(L1,59,-) 1965 : <-------------------------- 1966 : 1967 ===== TCP Session restored ===== 1968 (12) LDP Init(n/a,n/a,94) 1969 ---------------------------> 1970 LDP Init(n/a,n/a,29) 1971 <--------------------------- 1972 (13) Label Request(L4,30,-) 1973 ---------------------------> 1974 (14) Label Mapping(L2,95,-) 1975 <--------------------------- 1976 Label Abort(L3,96,30) 1977 <--------------------------- 1978 (15) Label Withdraw(L1,97,-) 1979 <--------------------------- 1981 Notes: 1982 ====== 1984 Notes are as in the previous example except as follows. 1986 (10) P1 needs to upgrade the software or hardware that it is running. 1987 It issues a Notification message to terminate the LDP session, 1988 but sets the status code as 'Temporary shutdown' to inform P2 1989 that this is not a fatal error, and P2 should maintain FT state. 1990 The TCP connection may also fail during the period that the LDP 1991 session is down (in which case it will need to be 1992 re-established), but it is also possible that the TCP connection 1993 will be preserved. 1995 8.4. Temporary Shutdown With FT Procedures and Check-Pointing 1997 notes P1 P2 1998 ===== == == 1999 (1) Label Request(L1,27,-) 2000 ---------------------------> 2001 Label Request(L2,28,-) 2002 ---------------------------> 2003 (2) Label Request(L3,93,-) 2004 <--------------------------- 2005 Label Request(L1,123,-) 2006 --------------------------> 2007 Label Request(L2,124,-) 2008 --------------------------> 2009 Label Mapping(L1,57,-) 2010 <-------------------------- 2011 (3) Label Mapping(L1,94,-) 2012 <--------------------------- 2013 Label Mapping(L2,58,-) 2014 <-------------------------- 2015 Label Mapping(L2,95,-) 2016 <--------------------------- 2017 (4) Address(n/a,29,-) 2018 ---------------------------> 2019 (5) Label Request(L4,30,-) 2020 ---------------------------> 2021 (6) Keepalive(n/a,31,95) * with FT Cork TLV * 2022 ---------------------------> 2023 (7) Label Abort(L3,96,-) 2024 <--------------------------- 2025 (8) Keepalive(n/a,97,31) *with FT Cork TLV * 2026 <--------------------------- 2027 (9) Keepalive(n/a,-,97) *with FT Cork TLV * 2028 ---------------------------> 2029 (10) Notification(Temporary shutdown) 2030 ---------------------------> 2031 ===== TCP Session shutdown ===== 2032 : 2033 : Label Withdraw(L1,59,-) 2034 : <-------------------------- 2035 : 2036 ===== TCP Session restored ===== 2037 (11) LDP Init(n/a,n/a,96) 2038 ---------------------------> 2039 LDP Init(n/a,n/a,31) 2040 <--------------------------- 2041 Label Withdraw(L1,97,-) 2042 <--------------------------- 2044 Notes: 2045 ====== 2047 This example operates much as the previous one. However, 2048 at (1), (2), (3), (4) and (5) no acknowledgements are 2049 made. 2051 At (6), P1 determines that graceful shutdown is required 2052 and sends a Keepalive acknowledging all previously 2053 received messages and itself containing a FT Protection 2054 TLV number and the FT Cork TLV. 2056 The Label abort at (7) crosses with this Keepalive, so at 2057 (8) P2 sends a Keepalive that acknowledges all messages 2058 received so far, but also including the FT Protection and 2059 FT Cork TLVs to indicate that there are still messages 2060 outstanding to be acknowledged. 2062 P1 is then able to complete the 3-way handshake at (9) 2063 and close the TCP session at (10). 2065 Upon recovery at (11) there are no messages to be re-sent 2066 because the Keepalives flushed the acknowledgements. The 2067 only messages sent after recovery is the Label Withdraw 2068 that was pended during the TCP session failure. 2070 8.5. Checkpointing Without FT Procedures 2072 notes P1 P2 2073 ===== == == 2074 (1) Label Request(L1) 2075 ---------------------------> 2076 (2) Label Request(L2) 2077 <--------------------------- 2078 Label Request(L1) 2079 --------------------------> 2080 Label Mapping(L1) 2081 <-------------------------- 2082 (3) Label Mapping(L1) 2083 <--------------------------- 2084 (4) Keepalive(n/a,12,-) 2085 ---------------------------> 2086 (5) Label Request(L3) 2087 ---------------------------> 2088 (6) Keepalive(n/a,-,12) 2089 <--------------------------- 2090 Label Request(L3) 2091 --------------------------> 2092 Label Mapping(L3) 2093 <-------------------------- 2094 (7) Label Mapping(L3) 2095 <--------------------------- 2096 ===== TCP Session failure ===== 2097 : 2098 : 2099 : 2100 ===== TCP Session restored ===== 2101 (8) LDP Init(n/a,n/a,23) 2102 ---------------------------> 2103 LDP Init(n/a,n/a,12) 2104 <--------------------------- 2105 (9) Label Request(L3) 2106 ---------------------------> 2107 Label Request(L3) 2108 --------------------------> 2109 Label Mapping(L3) 2110 <-------------------------- 2111 (10) Label Mapping(L3) 2112 <--------------------------- 2113 (11) Label Request(L2) 2114 <--------------------------- 2116 Notes: 2117 ====== 2119 (1), (2) and (3) show label distribution without FT sequence numbers. 2121 (4) A checkpoint request from P1. It carries the sequence number of 2122 the checkpoint request. 2124 (5) P1 immediately starts a new label distribution request. 2126 (6) P2 confirms that it has secured all previous transactions. 2128 (7) The subsequent (un-acknowledged) label distribution completes. 2130 (8) The session fails and is restarted. Initialization messages 2131 confirm the sequence numbers of the secured checkpoints. 2133 (9) P1 recommences the unacknowledged label distribution request. 2135 (10) P2 recommences an unacknowledged label distribution request. 2137 8.6. Graceful Shutdown With Checkpointing But No FT Procedures 2139 notes P1 P2 2140 ===== == == 2141 (1) Label Request(L1) 2142 ---------------------------> 2143 (2) Label Request(L2) 2144 <--------------------------- 2145 Label Request(L1) 2146 --------------------------> 2147 Label Mapping(L1) 2148 <-------------------------- 2149 (3) Label Mapping(L1) 2150 <--------------------------- 2151 (4) Keepalive(n/a,12,23) * With Cork TLV 2152 ---------------------------> 2153 (5) : 2154 : 2155 : 2156 (6) Keepalive(n/a,24,12) * With Cork TLV 2157 <--------------------------- 2158 (7) Keepalive(n/a,-,24) * With Cork TLV 2159 ---------------------------> 2160 (8) Notification(Temporary shutdown) 2161 ---------------------------> 2162 ===== TCP Session failure ===== 2163 : 2164 : 2165 : 2166 ===== TCP Session restored ===== 2167 (9) LDP Init(n/a,n/a,24) 2168 ---------------------------> 2169 LDP Init(n/a,n/a,12) 2170 <--------------------------- 2171 (10) Label Request(L3) 2172 ---------------------------> 2173 Label Request(L3) 2174 --------------------------> 2175 Label Mapping(L3) 2176 <-------------------------- 2177 (11) Label Mapping(L3) 2178 <--------------------------- 2179 (12) Label Mapping(L2) 2180 ---------------------------> 2182 Notes: 2183 ====== 2185 (1), (2) and (3) show label distribution without FT sequence numbers. 2187 (4) A checkpoint request from P1. It carries the sequence number of 2188 the checkpoint request and a Cork TLV. 2190 (5) P1 has sent a Cork TLV so quieces. 2192 (6) P2 confirms the checkpoint and continues the three-way handshake 2193 by including a Cork TLV itself. 2195 (7) P1 completes the three-way handshake. All operations have now 2196 been checkpointed and the session is quiesced. 2198 (8) The session is gracefully shut down. 2200 (9) The session recovers and the peers exchange the sequence numbers 2201 of the last secured checkpoints. 2203 (10) P1 starts a new label distribution request. 2205 (11) P1 continues processing a previously received label distribution 2206 request. 2208 9. Security Considerations 2210 The LDP FT enhancements inherit similar security 2211 considerations to those discussed in [RFC3212] and 2212 [RFC3036]. 2214 The LDP FT enhancements allow the re-establishment of a 2215 TCP connection between LDP peers without a full re- 2216 exchange of the attributes of established labels, which 2217 renders LSRs that implement the extensions specified in 2218 this draft vulnerable to additional denial-of-service 2219 attacks as follows: 2221 - An intruder may impersonate an LDP peer in order to force 2222 a failure and reconnection of the TCP connection, but where 2223 the intruder does not set the FT Reconnect Flag on re- 2224 connection. This forces all FT labels to be released. 2226 - Similarly, an intruder could set the FT Reconnect Flag on 2227 re-establishment of the TCP session without preserving the 2228 state and resources for FT labels. 2230 - An intruder could intercept the traffic between LDP peers 2231 and override the setting of the FT Label Flag to be set to 0 2232 for all labels. 2234 All of these attacks may be countered by use of an 2235 authentication scheme between LDP peers, such as the MD5- 2236 based scheme outlined in [RFC3036]. 2238 Alternative authentication schemes for LDP peers are 2239 outside the scope of this draft, but could be deployed to 2240 provide enhanced security to implementations of LDP, CR- 2241 LDP and the LDP FT enhancements. 2243 As with LDP and CR-LDP, a security issue may exist if an 2244 LDP implementation continues to use labels after 2245 expiration of the session that first caused them to be 2246 used. This may arise if the upstream LSR detects the 2247 session failure after the downstream LSR has released and 2248 re-used the label. The problem is most obvious with the 2249 platform-wide label space and could result in mis-routing 2250 of data to other than intended destinations and it is 2251 conceivable that these behaviors may be deliberately 2252 exploited to either obtain services without authorization 2253 or to deny services to others. 2255 In this draft, the validity of the session may be 2256 extended by the FT Reconnection Timeout, and the session 2257 may be re-established in this period. After the expiry 2258 of the Reconnection Timeout the session must be 2259 considered to have failed and the same security issue 2260 applies as described above. 2262 However, the downstream LSR may declare the session as 2263 failed before the expiration of its Reconnection Timeout. 2264 This increases the period during which the downstream LSR 2265 might reallocate the label while the upstream LSR 2266 continues to transmit data using the old usage of the 2267 label. To reduce this issue, this draft requires that 2268 labels are not re-used until the Reconnection Timeout has 2269 expired. 2271 A further issue might apply if labels were re-used prior 2272 to the expiration of the FT Reconnection Timeout, but 2273 this is forbidden by this draft. 2275 10. Implementation Notes 2277 10.1. FT Recovery Support on Non-FT LSRs 2279 In order to take full advantage of the FT capabilities of 2280 LSRs in the network, it may be that an LSR that does not 2281 itself contain the ability to recover from local hardware 2282 or software faults still needs to support the LDP FT 2283 enhancements described in this draft. 2285 Consider an LSR, P1, that is an LDP peer of a fully Fault 2286 Tolerant LSR, P2. If P2 experiences a fault in the 2287 hardware or software that serves an LDP session between 2288 P1 and P2, it may fail the TCP connection between the 2289 peers. When the connection is recovered, the LSPs/labels 2290 between P1 and P2 can only be recovered if both LSRs were 2291 applying the FT recovery procedures to the LDP session. 2293 10.2. ACK generation logic 2295 FT ACKs SHOULD be returned to the sending LSR as soon as 2296 is practicable in order to avoid building up a large 2297 quantity of unacknowledged state changes at the LSR. 2298 However, immediate one-for-one acknowledgements would 2299 waste bandwidth unnecessarily. 2301 A possible implementation strategy for sending ACKs to FT 2302 LDP messages is as follows: 2304 - An LSR secures received messages in order and tracks the 2305 sequence number of the most recently secured message, Sr. 2307 - On each LDP KeepAlive that the LSR sends, it attaches an 2308 FT ACK TLV listing Sr 2310 - Optionally, the LSR may attach an FT ACK TLV to any other 2311 LDP message sent between Keepalive messages if, for example, 2312 Sr has increased by more than a threshold value since the 2313 last ACK sent. 2315 This implementation combines the bandwidth benefits of 2316 accumulating ACKs while still providing timely ACKs. 2318 10.2.1 Ack Generation Logic When Using Check-Pointing 2320 If check-pointing is in use, the LSRs need not be 2321 concerned to send ACKs in such a timely manner. 2323 Check-points are solicitations for acknowledgement 2324 conveyed as a sequence number in an FT Protection TLV on 2325 a Keepalive message. Such check-point requests could be 2326 issued on a timer, after a significant amount of change, 2327 or before controlled shutdown of a session. 2329 The use of check-pointing may considerably simplify an 2330 implementation since it does not need to track the 2331 sequence numbers of all received LDP messages. It must, 2332 however, still ensure that all received messages (or the 2333 consequent state changes) are secured before 2334 acknowledging the sequence number on the Keepalive. 2336 This approach may be considered optimal in systems that 2337 do not show a high degree of change over time (such as 2338 targeted LDP session or CR-LDP systems) and that are 2339 prepared to risk loss of state for the most recent LDP 2340 exchanges. More dynamic systems (such as LDP discovery 2341 sessions) are more likely to want to acknowledge state 2342 changes more frequently so that the maximum amount of 2343 state can be preserved over a failure. 2345 11. Acknowledgments 2347 The work in this draft is based on the LDP and CR-LDP 2348 ideas expressed by the authors of [RFC3212] and []. 2350 The ACK scheme used in this draft was inspired by the 2351 proposal by David Ward and John Scudder for restarting 2352 BGP sessions now included in [BGP-RESTART]. 2354 The authors would also like to acknowledge the careful 2355 review and comments of Nick Weeds, Piers Finlayson, Tim 2356 Harrison, Duncan Archer, Peter Ashwood-Smith, Bob Thomas, 2357 S.Manikantan, Adam Sheppard, Alan Davey and Iftekhar 2358 Hussain. 2360 12. Intellectual Property Consideration 2362 The IETF has been notified of intellectual property 2363 rights claimed in regard to some or all of the 2364 specification contained in this document. For more 2365 information, consult the online list of claimed rights. 2367 13. Full Copyright Statement 2369 Copyright (c) The Internet Society (2000, 2001, 2002). All 2370 Rights Reserved. This document and translations of it may 2371 be copied and furnished to others, and derivative works 2372 that comment on or otherwise explain it or assist in its 2373 implementation may be prepared, copied, published and 2374 distributed, in whole or in part, without restriction of 2375 any kind, provided that the above copyright notice and 2376 this paragraph are included on all such copies and 2377 derivative works. However, this document itself may not 2378 be modified in any way, such as by removing the copyright 2379 notice or references to the Internet Society or other 2380 Internet organizations, except as needed for the purpose 2381 of developing Internet standards in which case the 2382 procedures for copyrights defined in the Internet 2383 Standards process must be followed, or as required to 2384 translate it into languages other than English. 2386 The limited permissions granted above are perpetual and 2387 will not be revoked by the Internet Society or its 2388 successors or assigns. 2390 This document and the information contained herein is 2391 provided on an "AS IS" basis and THE INTERNET SOCIETY AND 2392 THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL 2393 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 2394 TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN 2395 WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 2396 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 2398 14. IANA Considerations 2400 This draft requires the use of a number of new TLVs and 2401 status codes from the number spaces within the LDP 2402 protocol. This section explains the logic used by the 2403 authors to choose the most appropriate number space for 2404 each new entity, and is intended to assist in the 2405 determination of any final values assigned by IANA or the 2406 MPLS WG in the event that the MPLS WG chooses to advance 2407 this draft on the standards track. 2409 This section will be removed when the TLV and status code 2410 values have been agreed with IANA. 2412 14.1. New TLVs 2414 The FT Protection TLV carries attributes that affect a single label 2415 exchanged between LDP peers. It is taken from the 0x02xx range for 2416 TLVs that is used in [RFC3036] by other TLVs carrying label 2417 attributes. The next available value in this range is 0x0203. 2419 The FT Session TLV carries attributes that affect the entire LDP 2420 session between LDP peers. It is taken from the 0x05xx range for 2421 TLVs that is used in [RFC3036] by other TLVs carrying session-wide 2422 attributes. The next available value in this range is 0x0503. 2424 The FT Protection TLV may ACK many label operations at once if 2425 cumulative ACKS are used. It is taken from the 0x05xx range for 2426 TLVs that is used in [RFC3036] by other TLVs carrying session-wide 2427 attributes. The next available value in this range is 0x0504. 2429 The FT Cork TLV carries attributes that apply to all labels exchanged 2430 between LDP peers. It is taken from the 0x05xx range for TLVs that 2431 is used in [RFC3036] by other TLVs carrying label attributes. The 2432 next available value in this range is 0x0505. 2434 In summary: 2436 FT Protection TLV 0x0203 2437 FT Session TLV 0x0503 2438 FT Ack TLV 0x0504 2439 FT Cork TLV 0x0505 2441 14.2. New Status Codes 2443 LDP status codes are not sub-divided into specific ranges 2444 for different types of error. Hence, the numeric status 2445 code values are selected as the next available. 2447 Section 7.1 lists the new status codes required by this 2448 draft and gives interpretative information. The new 2449 codes are as follows. 2451 Status Code E Status Data 2453 No LDP Session 0 0x0000001A 2454 Zero FT seqnum 1 0x0000001B 2455 Unexpected TLV / 1 0x0000001C 2456 Session Not FT 2457 Unexpected TLV / 1 0x0000001D 2458 Label Not FT 2459 Missing FT Protection TLV 1 0x0000001E 2460 FT ACK sequence error 1 0x0000001F 2461 Temporary Shutdown 0 0x00000020 2462 FT Seq Numbers Exhausted 1 0x00000021 2463 FT Session parameters / 1 0x00000022 2464 changed 2465 Unexpected FT Cork TLV 1 0x00000023 2467 15. Authors' Addresses 2469 Adrian Farrel (editor) Paul Brittain 2470 Movaz Networks, Inc. Data Connection Ltd. 2471 7926 Jones Branch Drive, Suite 615 Windsor House, Pepper Street, 2472 McLean, VA 22102 Chester, Cheshire 2473 Phone: +1 703-847-1867 CH1 1DF, UK 2474 Email: afarrel@movaz.com Phone: +44-(0)20-8366-1177 2475 Email: pjb@dataconnection.com 2477 Philip Matthews Eric Gray 2478 Hyperchip Celox Networks, Inc. 2479 1800 Rene-Levesque Blvd W 2 Park Central Drive, 2480 Montreal, Quebec H3H 2H2 Southborough, MA 01772 2481 Canada Phone: +1 508-305-7214 2482 Phone: +1 514-906-4965 Email: egray@celoxnetworks.com 2483 Email: pmatthews@hyperchip.com 2485 Jack Shaio Toby Smith 2486 Vivace Networks Laurel Networks, Inc. 2487 2730 Orchard Parkway 1300 Omega Drive 2488 San Jose, CA 95134 Pittsburgh, PA 15205 2489 Phone: +1 408 432 7623 Email: tob@laurelnetworks.com 2490 Email: jack.shaio@vivacenetworks.com 2492 Andrew G. Malis 2493 Vivace Networks 2494 2730 Orchard Parkway 2495 San Jose, CA 95134 2496 Phone: +1 408 383 7223 2497 andy.malis@vivacenetworks.com 2499 16. References 2501 16.1. Normative References 2503 [RFC3036] Andersson, L., et. al., LDP Specification, RFC 3036, 2504 January 2001. 2506 [RFC3212] Jamoussi, B., et. al., Constraint-Based LSP Setup 2507 using LDP, RFC 3212, January 2002. 2509 16.2. Informative References 2511 [RFC2026] Bradner, S., "The Internet Standards Process -- 2512 Revision 3", BCP 9, RFC 2026, October 1996. 2514 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2515 Requirement Levels", BCP 14, RFC 2119, March 1997. 2517 [RFC2205] Braden, R., et al., Resource ReSerVation Protocol 2518 (RSVP) -- Version 1, Functional Specification, RFC 2519 2205, September 1997. 2521 [RFC2961] Berger, L., et al., RSVP Refresh Reduction Extensions, 2522 RFC 2961, April 2001. 2524 [RFC3209] Awduche, D., et al,. Extensions to RSVP for LSP 2525 Tunnels, RFC 3209, December 2001. 2527 [RFC3214] Ash, G., et al., LSP Modification Using CR-LDP, RFC 2528 3214, January 2001. 2530 [BGP-RESTART] Sangli, S., et al., Graceful Restart Mechanism 2531 for BGP, draft-ietf-idr-restart-05.txt, June 2002 2532 (work in progress). 2534 [LDP-RESTART] Leelanivas, M., et al., Graceful Restart Mechanism for 2535 LDP, draft-ietf-ldp-restart-02.txt, June 2002, work in 2536 progress.