idnits 2.17.1 draft-ietf-mpls-ldp-ft-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 52 longer pages, the longest (page 1) being 59 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [4]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 1297 has weird spacing: '...tted if the F...' == Line 1560 has weird spacing: '...present on al...' == Line 1582 has weird spacing: '...n. The seque...' == Line 1603 has weird spacing: '...ication messa...' == Line 1718 has weird spacing: '... number equal...' == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: In these cases the node that intends terminating the session can flush acknowledgement using a check-point request as described above. The sender SHOULD not send further label or address-related messages after requesting shutdown check-pointing in order to preserve the integrity of its saved state. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2002) is 7958 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 31 looks like a reference -- Missing reference section? '2' on line 2348 looks like a reference -- Missing reference section? '4' on line 2456 looks like a reference -- Missing reference section? '3' on line 212 looks like a reference -- Missing reference section? '5' on line 1164 looks like a reference -- Missing reference section? '6' on line 335 looks like a reference -- Missing reference section? '7' on line 335 looks like a reference -- Missing reference section? '8' on line 335 looks like a reference -- Missing reference section? '12' on line 1495 looks like a reference -- Missing reference section? '9' on line 2352 looks like a reference Summary: 3 errors (**), 0 flaws (~~), 8 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 MPLS Working Group Adrian Farrel 2 Internet Draft Movaz Networks, Inc. 3 Document: draft-ietf-mpls-ldp-ft-03.txt 4 Expiration Date: December 2002 Paul Brittain 5 Data Connection Ltd. 7 Philip Matthews 8 Hyperchip 10 Eric Gray 11 Sandburst 13 Toby Smith 14 Laurel Networks 16 Andy Malis 17 Jack Shaio 18 Vivace Networks 20 June 2002 22 Fault Tolerance for LDP and CR-LDP 24 draft-ietf-mpls-ldp-ft-03.txt 26 Status of this Memo 28 This document is an Internet-Draft and is in full 29 conformance with all provisions of Section 10 of RFC2026 30 [1]. 32 Internet-Drafts are working documents of the Internet 33 Engineering Task Force (IETF), its areas, and its working 34 groups. Note that other groups may also distribute 35 working documents as Internet-Drafts. Internet-Drafts are 36 draft documents valid for a maximum of six months and may 37 be updated, replaced, or obsoleted by other documents at 38 any time. It is inappropriate to use Internet- Drafts as 39 reference material or to cite them other than as "work in 40 progress." 42 The list of current Internet-Drafts can be accessed at 43 http://www.ietf.org/ietf/1id-abstracts.txt 45 The list of Internet-Draft Shadow Directories can be 46 accessed at http://www.ietf.org/shadow.html. 48 NOTE: The new TLV type numbers, bit values for flags 49 specified in this draft, and new LDP status code values 50 are preliminary suggested values and have yet to be 51 approved by IANA or the MPLS WG. See the section "IANA 52 Considerations" for further details. 54 Abstract 56 MPLS systems will be used in core networks where system 57 downtime must be kept to an absolute minimum. Many MPLS 58 LSRs may, therefore, exploit Fault Tolerant (FT) hardware 59 or software to provide high availability of the core 60 networks. 62 The details of how FT is achieved for the various 63 components of an FT LSR, including LDP, CR-LDP, the 64 switching hardware and TCP, are implementation specific. 65 This document identifies issues in the CR-LDP 66 specification [2] and the LDP specification [4] that make 67 it difficult to implement an FT LSR using the current LDP 68 and CR-LDP protocols, and proposes enhancements to the 69 LDP specification to ease such FT LSR implementations. 71 The extensions described here are equally applicable to 72 CR-LDP. 74 Contents 76 1. Conventions and Terminology used in this document 4 77 2. Introduction 5 78 2.1. Fault Tolerance for MPLS 5 79 2.2. Issues with LDP and CR-LDP 6 80 3. Overview of LDP FT Enhancements 7 81 3.1. Establishing an FT LDP Session 8 82 3.1.1 Interoperation with Non-FT LSRs 9 83 3.2. TCP Connection Failure 9 84 3.2.1 Detecting TCP Connection Failures 9 85 3.2.2 LDP Processing after Connection Failure 10 86 3.3. Data Forwarding During TCP Connection Failure 10 87 3.4. FT LDP Session Reconnection 11 88 3.5. Operations on FT Labels 12 89 3.6. Check-Pointing 12 90 3.6.1 Graceful Termination 13 91 3.7. Label Space Depletion and Replenishment 13 92 4. FT Operations 14 93 4.1. FT LDP Messages 14 94 4.1.1 FT Label Messages 14 95 4.1.2 FT Address Messages 15 96 4.1.3 FT Label Resources Available Notification Messages 16 97 4.2. FT Operation ACKs 17 98 4.3. Preservation of FT State 18 99 4.4. FT Procedure After TCP Failure 19 100 4.4.1 FT LDP Operations During TCP Failure 20 101 4.5. FT Procedure After TCP Re-connection 21 102 4.5.1 Re-Issuing FT Messages 22 103 4.5.2 Interaction with CR-LDP LSP Modification 23 104 5. Checkpointing Procedures 23 105 5.1. Checkpointing with the Keepalive Message 24 106 5.2. Quiesce and Keepalive 24 108 6. Changes to Existing Messages 25 109 6.1. LDP Initialization Message 25 110 6.2. LDP Keepalive Messages 25 111 6.3. All Other LDP Session Messages 26 112 7. New Fields and Values 26 113 7.1. Status Codes 26 114 7.2. FT Session TLV 27 115 7.3. FT Protection TLV 29 116 7.4. FT ACK TLV 32 117 7.5. FT Cork TLV 33 118 8. Example Use 34 119 8.1. Session Failure and Recovery - FT Procedures 35 120 8.2. Use of Check-Pointing With FT Procedures 37 121 8.3. Temporary Shutdown With FT Procedures 39 122 8.4. Temporary Shutdown With FT Procedures and Check-Pointing 41 123 8.5. Checkpointing Without FT Procedures 43 124 8.6. Graceful Shutdown With Checkpointing But No FT Procedures 45 125 9. Security Considerations 46 126 10. Implementation Notes 47 127 10.1. FT Recovery Support on Non-FT LSRs 47 128 10.2. ACK Generation Logic 48 129 10.2.1 Ack Generation Logic When Using Check-Pointing 48 130 11. Acknowledgments 49 131 12. Intellectual Property Consideration 49 132 13. Full Copyright Statement 49 133 14. IANA Considerations 50 134 14.1. FT Session TLV 50 135 14.2. FT Protection TLV 50 136 14.3. FT ACK TLV 51 137 14.4. FT Cork TLV 51 138 14.5. Status Codes 51 139 15. Authors' Addresses 51 140 16. Normative References 52 141 17. Informative References 52 143 0. Changes From Version 2 to Version 3 145 This section to be removed before final publication. 147 2.2 Add notes about graceful shutdown and check-pointing. 149 3 Allow sequence number on Keepalive to facilitate check- 150 pointing. 152 3.6 New section to describe the use of check-pointing 154 3.6.1 New sub-section to describe the use of check- 155 pointing for graceful restart 157 3.7 and 4.1.3 Make acknowledgement and request for 158 acknowledgement of Notification messages carrying 'Label 159 Resources Available' optional. 161 4.1.1 Explicitly state it is allowable to make all labels 162 FT labels. 164 5. Add section to describe Checkpointing procedures. 166 6.2 Allow FT Protection TLV on Keepalive message. 168 7.1 Add Unexpected FT Cork TLV status code 170 7.2 Define bits in the FT Session TLV to define which FT 171 processes are in use. 173 7.3 Allow FT Protection TLV on Keepalive message. 175 7.5 Add FT Cork TLV 177 8. Add examples of further message flows. 179 10.2.1 Add description of Ack generation logic when using 180 check-pointing 182 14.4 Add FT Cork TLV 184 1. Conventions and Terminology used in this document 186 Definitions of key words and terms applicable to LDP and 187 CR-LDP are inherited from [2] and [4]. 189 The term "FT label" is introduced in this document to 190 indicated a label for which fault tolerant operation is 191 used. A "non-FT label" is not fault tolerant and is 192 handled as specified in [2] and [4]. 194 The extensions to LDP specified in this document are 195 collectively referred to as the "LDP FT enhancements". 197 Within the context of this draft, "Checkpointing" refers 198 to a process of messages exchanges that confirm receipt 199 and processing (or secure storage) of specific protocol 200 messages. 202 In the examples quoted, the following notation is used. 204 Ln : An LSP. For example L1. 206 Pn : An LDP peer. For example P1. 208 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 209 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", 210 "MAY", and "OPTIONAL" in this document are to be 211 interpreted as described in RFC-2119 [3]. 213 2. Introduction 215 High Availability (HA) is typically claimed by equipment 216 vendors when their hardware achieves availability levels 217 of at least 99.999% (five 9s). To implement this, the 218 equipment must be capable of recovering from local 219 hardware and software failures through a process known as 220 fault tolerance (FT). 222 The usual approach to FT involves provisioning backup 223 copies of hardware and/or software. When a primary copy 224 fails, processing is switched to the backup copy. This 225 process, called failover, should result in minimal 226 disruption to the Data Plane. 228 In an FT system, backup resources are sometimes 229 provisioned on a one-to-one basis (1:1), sometimes as 230 many-to-one (1:n), and occasionally as many-to-many 231 (m:n). Whatever backup provisioning is made, the system 232 must switch to the backup automatically on failure of the 233 primary, and the software and hardware state in the 234 backup must be set to replicate the state in the primary 235 at the point of failure. 237 2.1. Fault Tolerance for MPLS 239 MPLS will be used in core networks where system downtime 240 must be kept to an absolute minimum. Many MPLS LSRs may, 241 therefore, exploit FT hardware or software to provide 242 high availability of core networks. 244 In order to provide HA, an MPLS system needs to be able 245 to survive a variety of faults with minimal disruption to 246 the Data Plane, including the following fault types: 248 - failure/hot-swap of a physical connection between LSRs 250 - failure/hot-swap of the switching fabric in the LSR 252 - failure of the TCP or LDP stack in an LSR 254 - software upgrade to the TCP or LDP stacks. 256 The first two examples of faults listed above are 257 confined to the Data Plane. Such faults can be handled 258 by providing redundancy in the Data Plane which is 259 transparent to LDP operating in the Control Plane. The 260 last two example types of fault require action in the 261 Control Plane to recover from the fault without 262 disrupting traffic in the Data Plane. This is possible 263 because many recent router architectures separate the 264 Control and Data Planes such that forwarding can continue 265 unaffected by recovery action in the Control Plane. 267 2.2. Issues with LDP and CR-LDP 269 LDP and CR-LDP use TCP to provide reliable connections 270 between LSRs over which to exchange protocol messages to 271 distribute labels and to set up LSPs. A pair of LSRs that 272 have such a connection are referred to as LDP peers. 274 TCP enables LDP and CR-LDP to assume reliable transfer of 275 protocol messages. This means that some of the messages 276 do not need to be acknowledged (for example, Label 277 Release). 279 LDP and CR-LDP are defined such that if the TCP 280 connection fails, the LSR should immediately tear down 281 the LSPs associated with the session between the LDP 282 peers, and release any labels and resources assigned to 283 those LSPs. 285 It is notoriously hard to provide a Fault Tolerant 286 implementation of TCP. To do so might involve making 287 copies of all data sent and received. This is an issue 288 familiar to implementers of other TCP applications such 289 as BGP. 291 During failover affecting the TCP or LDP stacks, 292 therefore, the TCP connection may be lost. Recovery from 293 this position is made worse by the fact that LDP or CR- 294 LDP control messages may have been lost during the 295 connection failure. Since these messages are 296 unconfirmed, it is possible that LSP or label state 297 information will be lost. 299 This draft describes a solution which involves 301 - negotiation between LDP peers of the intent to support 302 extensions to LDP that facilitate recovery from failover 303 without loss of LSPs 305 - selection of FT survival on a per LSP/label basis 307 - acknowledgement of LDP messages to ensure that a full 308 handshake is performed on those messages either frequently 309 (such as per message) or less frequently as in check- 310 pointing 312 - solicitation of up-to-date acknowledgement 313 (checkpointing) of previous LDP messages to ensure the 314 current state is flushed to disk/NVRAM, with an additional 315 option that allows an LDP partner to request that state is 316 flushed in both directions if graceful shutdown is required. 318 - re-issuing lost messages after failover to ensure that 319 LSP/label state is correctly recovered after reconnection of 320 the LDP session. 322 Other objectives of this draft are to 324 - offer back-compatibility with LSRs that do not implement 325 these proposals 327 - preserve existing protocol rules described in [2] and [4] 328 for handling unexpected duplicate messages and for 329 processing unexpected messages referring to unknown 330 LSPs/labels 332 - integrate with the LSP modification function described in [5] 334 - avoid full state refresh solutions (such as those present 335 in RSVP: see [6], [7], [8] and [12]) whether they be full- 336 time, or limited to post-failover recovery. 338 Note that this draft concentrates on the preservation of 339 label state for labels exchanged between a pair of 340 adjacent LSRs when the TCP connection between those LSRs 341 is lost. This is a requirement for Fault Tolerant 342 operation of LSPs, but a full implementation of end-to- 343 end protection for LSPs requires that this is combined 344 with other techniques that are outside the scope of this 345 draft. 347 In particular, this draft does not attempt to describe 348 how to modify the routing of an LSP or the resources 349 allocated to a label or LSP, which is covered by [5]. 350 This draft also does not address how to provide automatic 351 layer 2/3 protection switching for a label or LSP, which 352 is a separate area for study. 354 This specification does not preclude an implementation 355 from attempting (or require it to attempt) to use the FT 356 behavior described here to recover from a preemptive 357 failure of a connection on a non-FT system due to, for 358 example, a partial system crash. Note, however, that 359 there are potential issues too numerous to list here - 360 not least the likelihood that the same crash will 361 immediately occur when processing the restored data. 363 3. Overview of LDP FT Enhancements 365 The LDP FT enhancements consist of the following main 366 elements, which are described in more detail in the 367 sections that follow. 369 - The presence of an FT Session TLV on the LDP 370 Initialization message indicates that an LSR supports some 371 form of protection or recovery from session failure. A flag 372 bit within this TLV (the S bit) indicates that the LSR 373 supports the LDP FT enhancements on this session. Another 374 flag (the C bit) indicates that the checkpointing procedures 375 are to be used. 377 - An FT Reconnect Flag in the FT Session TLV indicates 378 whether an LSR has preserved FT label state across a failure 379 of the TCP connection. 381 - An FT Reconnection Timeout, exchanged on the LDP 382 Initialization message, that indicates the maximum time peer 383 LSRs will preserve FT label state after a failure of the TCP 384 connection. 386 - An FT Protection TLV used to identify operations that 387 affect LDP labels. All LDP messages carrying the FT 388 Protection TLV need to be secured (e.g. to NVRAM) and ACKed 389 to the sending LDP peer in order that the state for FT 390 labels can be correctly recovered after LDP session 391 reconnection. 393 Note that the implementation within an FT system is left 394 open by this draft. An implementation could choose to 395 secure entire messages relating to FT labels, or it could 396 secure only the relevant state information. 398 - Address advertisement may also be secured by use of the 399 FT Protection TLV. This enables recovery after LDP session 400 reconnection without the need to re-advertise what may be a 401 very large number of addresses. 403 - The FT Protection TLV may also be used on the Keepalive 404 message to flush acknowledgement of all previous FT 405 operations. This enables a check-point for future recovery, 406 either in mid-session or prior to graceful shutdown of an 407 LDP session. This procedure may also be used to checkpoint 408 all (that is both FT and non-FT) operations for future 409 recovery. 411 3.1. Establishing an FT LDP Session 413 In order that the extensions to LDP [4] and CR-LDP [2] 414 described in this draft can be used successfully on an 415 LDP session between a pair of LDP peers, they MUST 416 negotiate that the LDP FT enhancements are to be used on 417 the LDP session. 419 This is done on the LDP Initialization message exchange 420 using a new FT Session TLV. Presence of this TLV 421 indicates that the peer wants to support some form of 422 protection or recovery processing. The S bit within this 423 TLV indicates that the peer wants to support the LDP FT 424 enhancements on this LDP session. The C bit indicates 425 that the peer wants to support the checkpointing 426 functions described in this draft. The S and C bits may 427 be set independently. 429 The relevant LDP FT enhancements MUST be supported on an 430 LDP session if both LDP peers include an FT Session TLV 431 on the LDP Initialization message and have the same 432 setting of the S or C bit. 434 If either LDP Peer does not include the FT Session TLV 435 LDP Initialization message or if there is no match of S 436 and C bits between the peers, the LDP FT enhancements 437 MUST NOT be used during this LDP session. Use of LDP FT 438 enhancements by a sending LDP peer MUST be interpreted by 439 the receiving LDP peer as a serious protocol error 440 causing the session to be terminated. 442 An LSR MAY present different FT/non-FT behavior on 443 different TCP connections, even if those connections are 444 successive instantiations of the LDP session between the 445 same LDP peers. 447 3.1.1 Interoperation with Non-FT LSRs 449 The FT Session TLV on the LDP Initialization message 450 carries the U-bit. If an LSR does not support any 451 protection or recovery mechanisms , it will ignore this 452 TLV. Since such partners also do not include the FT 453 Session TLV, all LDP sessions to such LSRs will not use 454 the LDP FT enhancements. 456 The rest of this draft assumes that the LDP sessions 457 under discussion are between LSRs that do support the LDP 458 FT enhancements, except where explicitly stated 459 otherwise. 461 3.2. TCP Connection Failure 463 3.2.1 Detecting TCP Connection Failures 465 TCP connection failures may be detected and reported to the 466 LDP component in a variety of ways. These should all be 467 treated in the same way by the LDP component. 469 - Indication from the management component that a TCP 470 connection or underlying resource is no longer active. 472 - Notification from a hardware management component of an 473 interface failure. 475 - Sockets keepalive timeout. 477 - Sockets send failure. 479 - New (incoming) Socket opened. 481 - LDP protocol timeout. 483 3.2.2 LDP Processing after Connection Failure 485 If the LDP FT enhancements are not in use on an LDP 486 session, the action of the LDP peers on failure of the 487 TCP connection is as specified in [2] and [4]. 489 All state information and resources associated with non- 490 FT labels MUST be released on the failure of the TCP 491 connection, including deprogramming the non-FT label from 492 the switching hardware. This is equivalent to the 493 behavior specified in [4]. 495 If the LDP FT enhancements are in use on an LDP session, 496 both LDP peers SHOULD preserve state information and 497 resources associated with FT labels exchanged on the LDP 498 session. Both LDP peers SHOULD use a timer to release 499 the preserved state information and resources associated 500 with FT-labels if the TCP connection is not restored 501 within a reasonable period. The behavior when this timer 502 expires is equivalent to the LDP session failure behavior 503 described in [4]. 505 The FT Reconnection Timeout each LDP peer intends to 506 apply to the LDP session is carried in the FT Session TLV 507 on the LDP Initialization messages. Both LDP peers MUST 508 use the value that corresponds to the lesser timeout 509 interval of the two proposed timeout values from the LDP 510 Initialization exchange, where a value of zero is treated 511 as positive infinity. 513 3.3. Data Forwarding During TCP Connection Failure 515 An LSR that implements the LDP FT enhancements SHOULD 516 preserve the programming of the switching hardware across 517 a failover. This ensures that data forwarding is 518 unaffected by the state of the TCP connection between 519 LSRs. 521 It is an integral part of FT failover processing in some 522 hardware configurations that some data packets might be 523 lost. If data loss is not acceptable to the applications 524 using the MPLS network, the LDP FT enhancements described 525 in this draft SHOULD NOT be used. 527 3.4. FT LDP Session Reconnection 529 When a new TCP connection is established, the LDP peers 530 MUST exchange LDP Initialization messages. When a new 531 TCP connection is established after failure, the LDP 532 peers MUST re-exchange LDP Initialization messages. 534 If an LDP peer includes the FT Session TLV with the S bit 535 set in the LDP Initialization message for the new 536 instantiation of the LDP session, it MUST also set the FT 537 Reconnect Flag according to whether it has been able to 538 preserve label state. The FT Reconnect Flag is carried 539 in the FT Session TLV. 541 If an LDP peer has preserved all state information for 542 previous instantiations of the LDP session, then it 543 SHOULD set the FT Reconnect Flag to 1 in the FT Session 544 TLV. Otherwise, it MUST set the FT Reconnect Flag to 0. 546 If either LDP peer sets the FT Reconnect Flag to 0, or 547 omits the FT Session TLV, both LDP peers MUST release any 548 state information and resources associated with the 549 previous instantiation of the LDP session between the 550 same LDP peers, including FT label state and Addresses. 551 This ensures that network resources are not permanently 552 lost by one LSR if its LDP peer is forced to undergo a 553 cold start. 555 If an LDP peer changes any session parameters (for 556 example, the label space bounds) from the previous 557 instantiation the nature of any preserved labels may have 558 changed. In particular, previously allocated labels may 559 now be out of range. For this reason, session 560 reconnection MUST use the same parameters as were in use 561 on the session before the failure. If an LDP peer 562 notices that the parameters have been changed by the 563 other peer it SHOULD send a Notification message with the 564 'FT Session parameters changed' status code. 566 If both LDP peers set the FT Reconnect Flag to 1, both 567 LDP peers MUST use the FT label operation procedures 568 indicated in this draft to complete any label operations 569 on FT labels that were interrupted by the LDP session 570 failure. 572 If an LDP peer receives an LDP Initialization message 573 with the FT Reconnect Flag set before it sends its own 574 Initialization message, but has retained no information 575 about the previous version of the session, it MUST 576 respond with an Initialization message with the FT 577 Reconnect Flag clear. If an LDP peer receives an LDP 578 Initialization message with the FT Reconnect Flag set in 579 response to an Initialization message that it has sent 580 with the FT Reconnect Flag clear it MUST act as if no 581 state was retained by either peer on the session. 583 3.5. Operations on FT Labels 585 Label operations on FT labels are made Fault Tolerant by 586 providing acknowledgement of all LDP messages that affect 587 FT labels.Acknowledgements are achieved by means of 588 sequence numbers on these LDP messages. 590 The message exchanges used to achieve acknowledgement of 591 label operations and the procedures used to complete 592 interrupted label operations are detailed in the section 593 "FT Operations". 595 Using these acknowledgements and procedures, it is not 596 necessary for LDP peers to perform a complete re- 597 synchronization of state for all FT labels, either on re- 598 connection of the LDP session between the LDP peers or on 599 a timed basis. 601 3.6. Check-Pointing 603 Check-pointing is a useful feature that allows nodes to 604 reduce the amount of processing that they need to do to 605 acknowledge LDP messages. The C bit in the FT Session 606 TLV is used to indicate that checkpointing is supported. 608 Under the normal operation on FT labels, acknowledgments 609 may be deferred during normal processing and only sent 610 periodically. Check-pointing may be used to flush 611 acknowledgement from a peer by including a sequence 612 number on a Keepalive message requesting acknowledgement 613 of that message and all previous messages. 615 If the S bit is not agreed upon, checkpointing may still 616 be used. In this case it is used to acknowledge all 617 messages exchanged between the peers. 619 This offers an approach where acknowledgements need not 620 be sent to every message or even frequently, but are only 621 sent as check-points in response to requests carried on 622 Keepalive messages. Such an approach may be considered 623 optimal in systems that do not show a high degree of 624 change over time (such as targeted LDP session or CR-LDP 625 systems) and that are prepared to risk loss of state for 626 the most recent LDP exchanges. More dynamic systems 627 (such as LDP discovery sessions) are more likely to want 628 to acknowledge state changes more frequently so that the 629 maximum amount of state can be preserved over a failure. 631 Note that an important consideration of this draft is 632 that nodes acknowledging messages on a one-for-one basis, 633 nodes deferring acknowledgements, and nodes relying on 634 check-pointing should all interoperate seamlessly and 635 without protocol negotiation beyond session 636 initialization. 638 Further discussion of this feature is provided in the 639 section "FT Operations". 641 3.6.1 Graceful Termination 643 A feature that builds on check-pointing is graceful 644 termination. 646 In some cases, such as controlled failover or software 647 upgrade, it is possible for a node to know in advance 648 that it is going to terminate its session with a peer. 650 In these cases the node that intends terminating the 651 session can flush acknowledgement using a check-point 652 request as described above. The sender SHOULD not send 653 further label or address-related messages after 654 requesting shutdown check-pointing in order to preserve 655 the integrity of its saved state. 657 This, however, only provides for acknowledgement in one 658 direction, and the node that is terminating also requires 659 to know that it has secured all state sent by its peer. 660 This is achieved by a three-way hand shake of the check- 661 point which is requested by an additional TLV (the Cork 662 TLV) in the Keepalive message. 664 Further discussion of this feature is provided in the 665 section "FT Operations". 667 3.7. Label Space Depletion and Replenishment 669 When an LDP peer is unable to satisfy a Label Request 670 message because it has no more available labels, it sends 671 a Notification message carrying the status code 'No label 672 resources'. This warns the requesting LDP peer that 673 subsequent Label Request messages are also likely to fail 674 for the same reason. This message does not need to be 675 acknowledged for FT purposes since Label Request messages 676 sent after session recovery will receive the same 677 response. However, the LDP peer that receives a 'No 678 label resources' Notification stops sending Label Request 679 messages until it receives a 'Label resources available' 680 Notification message. Since this unsolicited 681 Notification might get lost during session failure, it 682 may be protected using the procedures described in this 683 draft. 685 An alternative approach allows that an implementation may 686 always assume that labels are available when a session is 687 re-established. In this case, it is possible that it may 688 throw away the 'No label resources' information from the 689 previous incarnation of the session and may send a batch 690 of LDP messages on session re-establishment that will 691 fail and that it could have known would fail. 693 Note that the sender of a 'Label resources available' 694 Notification message may choose whether to add a sequence 695 number requesting acknowledgement. Conversely, the 696 receiver of 'Label resources available' Notification 697 message may choose to acknowledge the message without 698 actually saving any state. 700 This is an implementation choice made possible by making 701 the FT parameters on the Notification message optional. 702 Implementations will interoperate fully if they take 703 opposite approaches, but additional LDP messages may be 704 sent unnecessarily on session recovery. 706 4. FT Operations 708 Once an FT LDP session has been established, using the S 709 bit in the FT Session TLV on the Session Initialization 710 message as described in the section "Establishing an FT 711 LDP Session", both LDP peers MUST apply the procedures 712 described in this section for FT LDP message exchanges. 714 If the LDP session has been negotiated to not use the LDP 715 FT enhancements, these procedures MUST NOT be used. 717 4.1. FT LDP Messages 719 4.1.1 FT Label Messages 721 A label is identified as being an FT label if the initial 722 Label Request or Label Mapping message relating to that 723 label carries the FT Protection TLV. 725 It is a valid implementation option to flag all labels as 726 FT labels. Indeed this may be a preferred option for 727 implementations wishing to use Keepalive messages 728 carrying the FT Protection TLV to achieve periodic saves 729 of the complete label forwarding state. 731 If a label is an FT label, all LDP messages affecting 732 that label MUST carry the FT Protection TLV in order that 733 the state of the label can be recovered after a failure 734 of the LDP session. 736 A valid option is for no labels on an FT session to be FT 737 labels. 739 The checkpointing mechanism (see section 5) is an integral 740 part of the FT procedures and is available whenever the FT 741 procedures are selected. When the FT procedures are in use 742 checkpointing applies only to FT labels. If no labels on 743 the session are FT labels but the use of the FT procedures 744 has been negotiated, checkpointing will not secure any 745 message exchanges. 747 If the FT procedures are not in use, checkpointing using the 748 Keepalive message applies to all messages exchanged on the 749 session. 751 4.1.1.1 Scope of FT Labels 753 The scope of the FT/non-FT status of a label is limited 754 to the LDP message exchanges between a pair of LDP peers. 756 In Ordered Control, when the message is forwarded 757 downstream or upstream, the TLV may be present or absent 758 according to the requirements of the LSR sending the 759 message. 761 If a platform-wide label space is used for FT labels, an 762 FT label value MUST NOT be reused until all LDP FT peers 763 to which the label was passed have acknowledged the 764 withdrawal of the FT label, either by an explicit LABEL 765 WITHDRAW/LABEL RELEASE exchange or implicitly if the LDP 766 session is reconnected after failure but without the FT 767 Reconnect Flag set. In the event that a session is not 768 re-established within the Reconnection Timeout, a label 769 MAY become available for re-use if it is not still in use 770 on some other session. 772 4.1.2 FT Address Messages 774 If an LDP session uses the LDP FT enhancements, both LDP 775 peers MUST secure Address and Address Withdraw messages 776 using FT Operation ACKs, as described below. This avoids 777 any ambiguity over whether an Address is still valid 778 after the LDP session is reconnected. 780 If an LSR determines that an Address message that it sent 781 on a previous instantiation of a recovered LDP session is 782 no longer valid, it MUST explicitly issue an Address 783 Withdraw for that address when the session is 784 reconnected. 786 If the FT Reconnect Flag is not set by both LDP peers on 787 reconnection of an LDP session (i.e. state has not been 788 preserved), both LDP peers MUST consider all Addresses to 789 have been withdrawn. The LDP peers SHOULD issue new 790 Address messages for all their valid addresses as 791 specified in [4]. 793 4.1.3 FT Label Resources Available Notification Messages 795 In LDP, it is possible that a downstream LSR may not have 796 labels available to respond to a Label Request. In this 797 case, as specified in RFC3036, the downstream LSR must 798 respond with a Notification - No Label Resources message. 799 The upstream LSR then suspends asking for new labels 800 until it receives a Notification - Label Resources 801 Available message from the downstream LSR. 803 When the FT extensions are used on a session 804 implementations may choose whether to secure the label 805 resource state of their peer or not. This choice impacts 806 the number of LDP messages that will be incorrectly 807 routed to a peer with depleted resources on session re- 808 establishment, but does not otherwise impact 809 interoperability. 811 For full preservation of state: 813 - The downstream LSR must preserve the label availability 814 state across a failover so that it remembers to send 815 Notification - Label Resources Available when the resources 816 become available. 818 - The upstream LSR must recall the label availability state 819 across failover so that it can optimize not sending Label 820 Requests when it recovers. 822 - The downstream LSR must use sequence numbers on 823 Notification - Label Resources Available so that it can 824 check that LSR A has received the message and clear its 825 secured state, or resend the message if LSR A recovers 826 without having received it. 828 However, the following options also exist: 830 - The downstream LSR may choose to not include a sequence 831 number on Notification - Label Resources Available. This 832 means that on session re-establishment it does not know what 833 its peer thinks the LSR's resource state is, because the 834 Notification may or may not have been delivered. Such an 835 implementation MUST begin recovered sessions by sending an 836 additional Notification - Label Resources Available to reset 837 its peer. 839 - The upstream node may choose not to secure information 840 about its peer's resource state. It would acknowledge a 841 Notification - Label Resources Available, but would not save 842 the information. Such an implementation MUST assume that 843 its peer's resource satte has been reset to Label Resources 844 Available when the session is re-established. 846 If the FT Reconnect Flag is not set by both LDP peers on 847 reconnection of an LDP session (i.e. state has not been 848 preserved), both LDP peers MUST consider the label 849 availability state to have been reset as if the session 850 had been set up for the first time. 852 4.2. FT Operation ACKs 854 Handshaking of FT LDP messages is achieved by use of 855 ACKs. Correlation between the original message and the 856 ACK is by means of the FT Sequence Number contained in 857 the FT Protection TLV, and passed back in the FT ACK TLV. 858 The FT ACK TLV may be carried on any LDP message that is 859 sent on the TCP connection between LDP peers. 861 An LDP peer maintains a separate FT sequence number for 862 each LDP session it participates in. The FT Sequence 863 number is incremented by one for each FT LDP message 864 (i.e. containing the FT Protection TLV) issued by this 865 LSR on the FT LDP session with which the FT sequence 866 number is associated. 868 When an LDP peer receives a message containing the FT 869 Protection TLV, it MUST take steps to secure this message 870 (or the state information derived from processing the 871 message). Once the message is secured, it MUST be ACKed. 872 However, there is no requirement on the LSR to send this 873 ACK immediately. 875 ACKs may be accumulated to reduce the message flow 876 between LDP peers. For example, if an LSR received FT 877 LDP messages with sequence numbers 1, 2, 3, 4, it could 878 send a single ACK with sequence number 4 to ACK receipt 879 and securing of all these messages. There is no protocol 880 reason why the number of ACKs accumulated or the time for 881 which an ACK is deferred should not be allowed to become 882 relatively large. 884 ACKs MUST NOT be sent out of sequence, as this is 885 incompatible with the use of accumulated ACKs. Duplicate 886 ACKs (that is two successive messages that acknowledge 887 the same sequence number) are acceptable. 889 If an LDP peer discovers that its sequence number space 890 for a specific session is full of un-acknowledged 891 sequence numbers (because its partner on the session has 892 not acknowledged them in a timely way) it cannot allocate 893 a new sequence number for any further FT LPD message. It 894 SHOULD send a Notification message with the status code 895 "FT Seq Numbers Exhausted". 897 4.3. Preservation of FT State 899 If the LDP FT enhancements are in use on an LDP session, 900 each LDP peer SHOULD NOT release the state information 901 and resources associated with FT labels exchanged on that 902 LDP session when the TCP connection fails. This is 903 contrary to [2] and [4], but allows label operations on 904 FT labels to be completed after re-connection of the TCP 905 connection. 907 Both LDP peers on an LDP session that is using the LDP FT 908 enhancements SHOULD preserve the state information and 909 resources they hold for that LDP session as described 910 below. 912 - An upstream LDP peer SHOULD release the resources (in 913 particular bandwidth) associated with an FT label when it 914 initiates a Label Release or Label Abort message for the 915 label. The upstream LDP peer MUST preserve state 916 information for the label, even if it releases the resources 917 associated with the label, as it may need to reissue the 918 label operation if the TCP connection is interrupted. 920 - An upstream LDP peer MUST release the state information 921 and resources associated with an FT label when it receives 922 an acknowledgement to a Label Release or Label Abort message 923 that it sent for the label, or when it sends a Label Release 924 message in response to a Label Withdraw message received 925 from the downstream LDP peer. 927 - A downstream LDP peer SHOULD NOT release the resources 928 associated with an FT label when it sends a Label Withdraw 929 message for the label as it has not yet received 930 confirmation that the upstream LDP peer has ceased to send 931 data using the label. The downstream LDP peer MUST NOT 932 release the state information it holds for the label as it 933 may yet have to reissue the label operation if the TCP 934 connection is interrupted. 936 - A downstream LDP peer MUST release the resources and 937 state information associated with an FT label when it 938 receives an acknowledgement to a Label Withdraw message for 939 the label. 941 - When the FT Reconnection Timeout expires, an LSR SHOULD 942 release all state information and resources from previous 943 instantiations of the (permanently) failed LDP session. 945 - Either LDP peer MAY elect to release state information 946 based on its internal knowledge of the loss of integrity of 947 the state information or an inability to pend (or queue) LDP 948 operations (as described in section 4.4.1) during a TCP 949 failure. That is, the peer is not required to wait for the 950 duration of the FT Reconnection Timeout before releasing 951 state; the timeout provides an upper limit on the 952 persistence of state. However, in the event that a peer 953 releases state before the expiration of the Reconnection 954 Timeout it MUST NOT re-use any label that was in use on the 955 session until the Reconnection Timeout has expired. 957 - When an LSR receives a Status TLV with the E-bit set in 958 the status code, which causes it to close the TCP 959 connection, the LSR MUST release all state information and 960 resources associated with the session. This behavior is 961 mandated because it is impossible for the LSR to predict the 962 precise state and future behavior of the partner LSR that 963 set the E-bit without knowledge of the implementation of 964 that partner LSR. 966 Note that the "Temporary Shutdown" status code does not have 967 the E-bit set, and MAY be used during maintenance or upgrade 968 operations to indicate that the LSR intends to preserve 969 state across a closure and re-establishment of the TCP 970 session. 972 - If an LSR determines that it must release state for any 973 single FT label during a failure of the TCP connection on 974 which that label was exchanged, it MUST release all state 975 for all labels on the LDP session. 977 The release of state information and resources associated 978 with non-FT labels is as described in [2] and [4]. 980 Note that a Label Release and the acknowledgement to a 981 Label Withdraw may be received by a downstream LSR in any 982 order. The downstream LSR MAY release its resources on 983 receipt of the first message and MUST release its 984 resources on receipt of the second message. 986 4.4. FT Procedure After TCP Failure 988 When an LSR discovers or is notified of a TCP connection 989 failure it SHOULD start an FT Reconnection Timer to allow 990 a period for re-connection of the TCP connection between 991 the LDP peers. 993 The RECOMMENDED default value for this timer is 5 994 seconds. During this time, failure must be detected and 995 reported, new hardware may need to be activated, software 996 state must be audited, and a new TCP session must be set 997 up. 999 Once the TCP connection between LDP peers has failed, the 1000 active LSR SHOULD attempt to re-establish the TCP 1001 connection. The mechanisms, timers and retry counts to re- 1002 establish the TCP connection are an implementation 1003 choice. It is RECOMMENDED that any attempt to re- 1004 establish the connection take account of the failover 1005 processing necessary on the peer LSR, the nature of the 1006 network between the LDP peers, and the FT Reconnection 1007 Timeout chosen on the previous instantiation of the TCP 1008 connection (if any). 1010 If the TCP connection cannot be re-established within the 1011 FT Reconnection Timeout period, the LSR detecting this 1012 timeout SHOULD release all state preserved for the failed 1013 LDP session. If the TCP connection is subsequently re- 1014 established (for example, after a further Hello exchange 1015 to set up a new LDP session), the LSR MUST set the FT 1016 Reconnect Flag to 0 if it released the preserved state 1017 information on this timeout event. 1019 If the TCP connection is successfully re-established 1020 within the FT Reconnection Timeout, both peers MUST re- 1021 issue LDP operations that were interrupted by (that is, 1022 un-acknowledged as a result of) the TCP connection 1023 failure. This procedure is described in section "FT 1024 Procedure After TCP Re-connection". 1026 The Hold Timer for an FT LDP Session (see [4] section 1027 2.5.5) SHOULD be ignored while the FT Reconnection Timer 1028 is running. The hold timer SHOULD be restarted when the 1029 TCP connection is re-established. 1031 4.4.1 FT LDP Operations During TCP Failure 1033 When the LDP FT enhancements are in use for an LDP 1034 session, it is possible that an LSR may determine that it 1035 needs to send an LDP message to an LDP peer but that the 1036 TCP connection to that peer is currently down. These 1037 label operations affect the state of FT labels preserved 1038 for the failed TCP connection, so it is important that 1039 the state changes are passed to the LDP peer when the TCP 1040 connection is restored. 1042 If an LSR determines that it needs to issue a new FT LDP 1043 operation to an LDP peer to which the TCP connection is 1044 currently failed, it MUST pend the operation (e.g. on a 1045 queue) and complete that operation with the LDP peer when 1046 the TCP connection is restored, unless the label 1047 operation is overridden by a subsequent additional 1048 operation during the TCP connection failure (see section 1049 "FT Procedure After TCP Re-connection"). 1051 If, during TCP Failure, an LSR determines that it cannot 1052 pend an operation which it cannot simply fail (for 1053 example a Label Withdraw, Release, or Abort operation), 1054 it MUST NOT attempt to re-establish the previous LDP 1055 session. The LSR MUST behave as if the Reconnection 1056 Timer expired and release all state information with 1057 respect to the LDP peer. An LSR may be unable (or 1058 unwilling) to pend operations; for instance, if a major 1059 routing transition occurred while TCP was inoperable 1060 between LDP peers it might result in excessively large 1061 numbers of FT LDP Operations. An LSR that releases state 1062 before the expiration of the Reconnection Timeout MUST 1063 NOT re-use any label that was in use on the session until 1064 the Reconnection Timeout has expired. 1066 In ordered operation, received FT LDP operations that 1067 cannot be correctly forwarded because of a TCP connection 1068 failure MAY be processed immediately (provided sufficient 1069 state is kept to forward the label operation) or pended 1070 for processing when the onward TCP connection is restored 1071 and the operation can be correctly forwarded upstream or 1072 downstream. Operations on existing FT labels SHOULD NOT 1073 be failed during TCP session failure. 1075 It is RECOMMENDED that Label Request operations for new 1076 FT labels are not pended awaiting the re-establishment of 1077 TCP connection that is awaiting recovery at the time the 1078 LSR determines that it needs to issue the Label Request 1079 message. Instead, such Label Request operations SHOULD 1080 be failed and, if necessary, a notification message 1081 containing the "No LDP Session" status code sent 1082 upstream. 1084 Label Requests for new non-FT labels MUST be rejected 1085 during TCP connection failure, as specified in [2] and 1086 [4]. 1088 4.5. FT Procedure After TCP Re-connection 1090 The FT operation handshaking described above means that 1091 all state changes for FT labels and Address messages are 1092 confirmed or reproducible at each LSR. 1094 If the TCP connection between LDP peers fails but is re- 1095 connected within the FT Reconnection Timeout, and both 1096 LSRs have indicated they will be re-establishing the 1097 previous LDP session, both LDP peers on the connection 1098 MUST complete any label operations for FT labels that 1099 were interrupted by the failure and re-connection of the 1100 TCP connection. 1102 The procedures for FT Reconnection Timeout MAY have been 1103 invoked as a result of either LDP peer being unable (or 1104 unwilling) to pend operations which occurred during the 1105 TCP Failure (as described in section 4.4.1). 1107 If, for any reason, an LSR has been unable to pend 1108 operations with respect to an LDP peer, as described in 1109 section 4.4.1, the LSR MUST set the FT Reconnect Flag to 1110 0 on re-connection to that LDP peer indicating that no FT 1111 state has been preserved. 1113 Label operations are completed using the procedure 1114 described below. 1116 4.5.1 Re-Issuing FT Messages 1118 On restoration of the TCP connection between LDP peers, 1119 any FT LDP messages that were lost because of the TCP 1120 connection failure are re-issued. The LDP peer that 1121 receives a re-issued message processes the message as if 1122 received for the first time. 1124 "Net-zero" combinations of messages need not be re-issued 1125 after re-establishment of the TCP connection between LDP 1126 peers. This leads to the following rules for re-issuing 1127 messages that are not ACKed by the LDP peer on the LDP 1128 Initialization message exchange after re-connection of 1129 the TCP session. 1131 - A Label Request message MUST be re-issued unless a Label 1132 Abort would be re-issued for the same FT label. 1134 - A Label Mapping message MUST be re-issued unless a Label 1135 Withdraw message would be re-issued for the same FT label. 1137 - All other messages on the LDP session that carried the FT 1138 Protection TLV MUST be re-issued if an acknowledgement had 1139 not previously been received. 1141 Any FT label operations that were pended (see section "FT 1142 Label Operations During TCP Failure") during the TCP 1143 connection failure MUST also be issued on re- 1144 establishment of the LDP session, except where they form 1145 part of a "net-zero" combination of messages according to 1146 the above rules. 1148 The determination of "net-zero" FT label operations 1149 according to the above rules MAY be performed on pended 1150 messages prior to the re-establishment of the TCP 1151 connection in order to optimize the use of queue 1152 resources. Messages that were sent to the LDP peer 1153 before the TCP connection failure, or pended messages 1154 that are paired with them, MUST NOT be subject to such 1155 optimization until an FT ACK TLV is received from the LDP 1156 peer. This ACK allows the LSR to identify which messages 1157 were received by the LDP peer prior to the TCP connection 1158 failure. 1160 4.5.2 Interaction with CR-LDP LSP Modification 1162 Re-issuing LDP messages for FT operation is orthogonal to 1163 the use of duplicate messages marked with the Modify 1164 ActFlg, as specified in [5]. Each time an LSR uses the 1165 modification procedure for an FT LSP to issue a new Label 1166 Request message, the FT label operation procedures MUST 1167 be separately applied to the new Label Request message. 1169 5. Checkpointing Procedures 1171 Checkpointing can be selected independently from the FT 1172 procedures described above by using the C bit in the FT 1173 Session TLV on the Session Initialization message. Note, 1174 however, that checkpointing is an integral part of the FT 1175 procedures. Setting the S and the C bit will achieve the 1176 same function as setting just the S bit. 1178 If the C bit is set, but the S bit is not set, no label 1179 is an FT label. Checkpointing is used to synchronize all 1180 labels exchanges for labels requested or distributed by 1181 the LDP peer requesting the exchange up to the time of 1182 requesting the checkpoint. No message apart from the 1183 checkpoint request and acknowledgement carries an active 1184 sequence number. (Note that the Session Initialization 1185 message may carry a sequence number to confirm that the 1186 checkpoint is still in place). 1188 It is an implementation matter to decide the ordering of 1189 received messages and checkpoint requests to ensure that 1190 checkpoint acknowledgements are secured. 1192 If the S and C bits are both set, or only the S bit is 1193 set, checkpointing applies only to FT labels and to 1194 address messages. 1196 The set of all messages that are checkpointed in this way 1197 is called the Checkpointable Messages. 1199 5.1 Checkpointing with the Keepalive Message 1201 If an LSR receives a FT Protection TLV on a Keepalive 1202 message, this is a request to flush the acknowledgements 1203 for all previously received Checkpointable Messages on 1204 the session. 1206 As soon as the LSR has completed securing the 1207 Checkpointable Messages (or state changes consequent on 1208 those messages) received before the Keepalive, it MUST 1209 send an acknowledgement to the sequence number of the 1210 Keepalive message. 1212 In the case where the FT procedures are in use and 1213 acknowledgements have been stored up, this may be 1214 immediately on receipt of the Keepalive. 1216 An example message flow showing this use of the Keepalive 1217 message to perform a periodic check-point of state is 1218 shown in section 8. 1220 An example message flow showing the use of checkpointing 1221 without the FT procedures is shown in section 8. 1223 5.2 Quiesce and Keepalive 1225 If the Keepalive Message also contains the FT Cork TLV, 1226 this indicates that the peer LSR wishes to quiesce the 1227 session prior to a graceful restart. 1229 It is RECOMMENDED that on receiving a Keepalive with the 1230 FT CORK TLV, an LSR should cease to send any further 1231 label or address related messages on the session until it 1232 has been disconnected and reconnected, other than any 1233 messages generated while processing and securing any 1234 previously unacknowledged messages received from the peer 1235 requesting the quiesce. It should also attempt to 1236 complete this processing and return a Keepalive with the 1237 FT ACK TLV as soon as possible in order to allow the 1238 session to be quiesced. 1240 An example message flow showing this use of the FT Cork 1241 TLV to achieves three-way handshake of state 1242 synchronization between two LDP peers is given in section 8. 1244 6. Changes to Existing Messages 1246 6.1. LDP Initialization Message 1248 The LDP FT enhancements add the following optional 1249 parameters to a LDP Initialization message 1251 Optional Parameter Length Value 1253 FT Session TLV 4 See below 1254 FT ACK TLV 4 See below 1256 The encoding for these TLVs is found in Section "New 1257 Fields and Values". 1259 FT Session 1260 If present, specifies the FT behavior of 1261 the LDP session. 1263 FT ACK TLV 1264 If present, specifies the last FT message 1265 that the sending LDP peer was able to 1266 secure prior to the failure of the previous 1267 instantiation of the LDP session. This TLV 1268 is only present if the FT Reconnect flag is 1269 set in the FT Session TLV, in which case 1270 this TLV MUST be present. 1272 6.2. LDP Keepalive Messages 1274 The LDP FT enhancements add the following optional 1275 parameters to a LDP Keepalive message 1277 Optional Parameter Length Value 1279 FT Protection TLV 4 See below 1280 FT Cork TLV 0 See below 1282 FT ACK TLV 4 See below 1284 The encoding for these TLVs is found in Section "New 1285 Fields and Values". 1287 FT Protection 1288 If present, specifies FT Sequence Number 1289 for the LDP message. When present on a 1290 Keepalive message, this indicates a 1291 solicited flush of the acknowledgements to 1292 all previous LDP messages containing 1293 sequence numbers and issued by the sender 1294 of the Keepalive on the same session. 1296 FT Cork 1297 Only permitted if the FT Protection TLV is 1298 also present. Indicates that the remote 1299 LSR wishes to quiesce the LDP session. See 1300 section 4.2 for the recommended action in 1301 such cases. 1303 FT ACK TLV 1304 If present, specifies the most recent FT 1305 message that the sending LDP peer has been 1306 able to secure. 1308 6.3. All Other LDP Session Messages 1310 The LDP FT enhancements add the following optional 1311 parameters to all other message types that flow on an LDP 1312 session after the LDP Initialization message 1314 Optional Parameter Length Value 1316 FT Protection TLV 4 See below 1317 FT ACK TLV 4 See below 1319 The encoding for these TLVs is found in the section "New 1320 Fields and Values". 1322 FT Protection 1323 If present, specifies FT Sequence Number 1324 for the LDP message. 1326 FT ACK 1327 If present, identifies the most recent FT 1328 LDP message ACKed by the sending LDP peer. 1330 7. New Fields and Values 1332 7.1. Status Codes 1334 The following new status codes are defined to indicate 1335 various conditions specific to the LDP FT enhancements. 1336 These status codes are carried in the Status TLV of a 1337 Notification message. 1339 The "E" column is the required setting of the Status Code 1340 E-bit; the "Status Data" column is the value of the 30- 1341 bit Status Data field in the Status Code TLV. 1343 Note that the setting of the Status Code F-bit is at the 1344 discretion of the LSR originating the Status TLV. 1345 However, it is RECOMMENDED that the F-bit is not set on 1346 Notification messages containing status codes except "No 1347 LDP Session" because the duplication of messages SHOULD 1348 be restricted to being a per-hop behavior. 1350 Status Code E Status Data 1352 No LDP Session 0 0x000000xx 1353 Zero FT seqnum 1 0x000000xx 1354 Unexpected TLV / 1 0x000000xx 1355 Session Not FT 1356 Unexpected TLV / 1 0x000000xx 1357 Label Not FT 1358 Missing FT Protection TLV 1 0x000000xx 1359 FT ACK sequence error 1 0x000000xx 1360 Temporary Shutdown 0 0x000000xx 1361 FT Seq Numbers Exhausted 1 0x000000xx 1362 FT Session parameters / 1 0x000000xx 1363 changed 1364 Unexpected FT Cork TLV 1 0x000000xx 1366 The Temporary Shutdown status code SHOULD be used in 1367 place of the Shutdown status code (which has the E-bit 1368 set) if the LSR that is shutting down wishes to inform 1369 its LDP peer that it expects to be able to preserve FT 1370 label state and to return to service before the FT 1371 Reconnection Timer expires. 1373 7.2. FT Session TLV 1375 LDP peers can negotiate whether the LDP session between 1376 them supports FT extensions by using a new OPTIONAL 1377 parameter, the FT Session TLV, on LDP Initialization 1378 Messages. 1380 The FT Session TLV is encoded as follows. 1382 0 1 2 3 1383 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1384 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1385 |1|0| FT Session TLV (0x0503) | Length (= 12) | 1386 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1387 | FT Flags | Reserved | 1388 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1389 | FT Reconnect Timeout (in milliseconds) | 1390 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1391 | Recovery Time (in milliseconds) | 1392 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1394 FT Flags 1395 FT Flags: A 16 bit field that indicates various 1396 attributes the FT support on this LDP session. 1397 This fields is formatted as follows: 1399 0 1 1400 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 1401 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1402 |R| Reserved |S|A|C|L| 1403 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1404 R: FT Reconnect Flag. 1406 Set to 1 if the sending LSR has 1407 preserved state and resources for all 1408 FT-labels since the previous LDP 1409 session between the same LDP peers, 1410 and set to 0 otherwise. See the 1411 section "FT LDP Session Reconnection" 1412 for details of how this flag is used. 1414 If the FT Reconnect Flag is set, the 1415 sending LSR MUST include an FT ACK TLV 1416 on the LDP Initialization message. 1418 S: Save State Flag. 1420 Set to 1 if the use of the FT 1421 Protection TLV is supported on 1422 messages other than the KeepAlive 1423 message used for chekpointing (see the 1424 C bit) 1426 A: All-Label Protection Required 1428 Set to 1 if all labels on the session 1429 MUST be treated as FT Labels. This 1430 removes from a node the option of 1431 treating some labels as FT labels and 1432 some labels as non-FT labels. 1434 Passing this information may be 1435 considered helpful to a peer since it 1436 may allow it to make optimizations in 1437 its processing. 1439 The A bit only has meaning if the S 1440 bit is set. 1442 C: Checkpointing Flag. 1444 Set to 1 to indicate that the 1445 checkpointing procedures in this draft 1446 are in use. 1448 If the S bit is also set to 1 then the 1449 C bit indicates that checkpointing is 1450 applied only to those message 1451 exchanges that carry the FT Protection 1452 TLV. 1454 If the S bit is set to 0 (zero) then 1455 the C bit indicates that checkpointing 1456 applies to all message exchanges on 1457 the session. 1459 L: Learn From Network Flag. 1461 Set to 1 if the Fault Recovery 1462 procedures of [12] are to be used to 1463 re-learn state from the network. 1465 It is not valid for all of the S, C and L bits to be zero. 1467 It is not valid for both the L and either the S or C bits 1468 to be set to 1. 1470 FT Reconnection Timeout 1471 If the S bit or C bit in the FT Flags field 1472 is set this indicates the period of time 1473 the sending LSR will preserve state and 1474 resources for FT labels exchanged on the 1475 previous instantiation of an FT LDP session 1476 that has currently failed. The timeout is 1477 encoded as a 32-bit unsigned integer number 1478 of milliseconds. 1480 A value of zero in this field means that 1481 the sending LSR will preserve state and 1482 resources indefinitely. 1484 See the section "FT Procedure After TCP 1485 Failure" for details of how this field is 1486 used. 1488 If the L bit is set to 1 in the FT Flags 1489 field, the meaning of this field is defined 1490 in [12]. 1492 Recovery Time 1493 The Recovery Time only has meaning if the L 1494 bit is set in the FT Flags. The meaning is 1495 defined in [12]. 1497 7.3. FT Protection TLV 1499 LDP peers use the FT Protection TLV to indicate that an 1500 LDP message contains an FT label operation. 1502 The FT Protection TLV MUST NOT be used in messages 1503 flowing on an LDP session that does not support the LDP 1504 FT enhancements. Its presence in such messages SHALL be 1505 treated as a protocol error by the receiving LDP peer 1506 which SHOULD send a Notification message with the 1507 'Unexpected TLV Session Not FT' status code. 1509 The FT Protection TLV MAY be carried on an LDP message 1510 transported on the LDP session after the initial exchange 1511 of LDP Initialization messages. In particular, this TLV 1512 MAY optionally be present on the following messages: 1514 - Label Request Messages in downstream on-demand 1515 distribution mode 1517 - Label Mapping messages in downstream unsolicited mode 1519 - Keepalive messages used to request flushing of 1520 acknowledgement of all previous messages that contained this 1521 TLV. 1523 If a label is to be an FT label, then the Protection TLV 1524 MUST be present: 1526 - on the Label Request message in DoD mode 1528 - on the Label Mapping message in DU mode 1530 - on all subsequent messages concerning this label. 1532 Here 'subsequent messages concerning this label' means 1533 any message whose Label TLV specifies this label or whose 1534 Label Request Message ID TLV specifies the initial Label 1535 Request message. 1537 If a label is not to be an FT label, then the Protection 1538 TLV MUST NOT be present on any of these messages. The 1539 presence of the FT TLV on a message relating to a non-FT 1540 label SHALL be treated as a protocol error by the 1541 receiving LDP peer which SHOULD send a notification 1542 message with the 'Unexpected TLV Label Not FT' status 1543 code. 1545 Where a Label Withdraw or Label Release message contains 1546 only a FEC TLV and does not identify a single specific 1547 label, the FT TLV should be included in the message if 1548 any label affected by the message is an FT label. If 1549 there is any doubt as to whether an FT TLV should be 1550 present, it is RECOMMENDED that the sender add the TLV. 1552 When an LDP peer receives a Label Withdraw Message or 1553 Label Release message that contains only a FEC, it SHALL 1554 accept the FT TLV if it is present regardless of the FT 1555 status of the labels which it affects. 1557 If an LDP session is an FT session as determined by the 1558 presence of the FT Session TLV with the S bit set on the 1559 LDP Initialization messages, the FT Protection TLV MUST 1560 be present on all Address messages on the session. 1562 If the session is an FT session, the FT Protection TLV 1563 may also optionally be present 1565 - on Notification messages on the session that have the 1566 status code 'Label Resources Available' 1568 - on Keepalive messages. 1570 The FT Protection TLV is encoded as follows. 1572 0 1 2 3 1573 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1574 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1575 |0|0| FT Protection (0x0203) | Length (= 4) | 1576 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1577 | FT Sequence Number | 1578 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1580 FT Sequence Number 1581 The sequence number for this FT label 1582 operation. The sequence number is encoded 1583 as a 32-bit unsigned integer. The initial 1584 value for this field on a new LDP session 1585 is 0x00000001 and is incremented by one for 1586 each FT LDP message issued by the sending 1587 LSR on this LDP session. This field may 1588 wrap from 0xFFFFFFFF to 0x00000001. 1590 This field MUST be reset to 0x00000001 if 1591 either LDP peer does not set the FT 1592 Reconnect Flag on re-establishment of the 1593 TCP connection. 1595 See the section "FT Operation Acks" for 1596 details of how this field is used. 1598 The special use of 0x00000000 is discussed 1599 in the section "FT ACK TLV" below. 1601 If an LSR receives an FT Protection TLV on a session that 1602 does not support the FT LDP enhancements, it SHOULD send 1603 a Notification message to its LDP peer containing the 1604 "Unexpected TLV, Session Not FT" status code. 1606 If an LSR receives an FT Protection TLV on an operation 1607 affecting a label that it believes is a non-FT label, it 1608 SHOULD send a Notification message to its LDP peer 1609 containing the "Unexpected TLV, Label Not FT" status 1610 code. 1612 If an LSR receives a message without the FT Protection 1613 TLV affecting a label that it believes is an FT label, it 1614 SHOULD send a Notification message to its LDP peer 1615 containing the "Missing FT Protection TLV" status code. 1617 If an LSR receives an FT Protection TLV containing a zero 1618 FT Sequence Number, it SHOULD send a Notification message 1619 to its LDP peer containing the "Zero FT Seqnum" status 1620 code. 1622 7.4. FT ACK TLV 1624 LDP peers use the FT ACK TLV to acknowledge FT label 1625 operations. 1627 The FT ACK TLV MUST NOT be used in messages flowing on an 1628 LDP session that does not support the LDP FT 1629 enhancements. Its presence on such messages SHALL be 1630 treated as a protocol error by the receiving LDP peer. 1632 The FT ACK TLV MAY be present on any LDP message 1633 exchanged on an LDP session after the initial LDP 1634 Initialization messages. It is RECOMMENDED that the FT 1635 ACK TLV is included on all FT Keepalive messages in order 1636 to ensure that the LDP peers do not build up a large 1637 backlog of unacknowledged state information. 1639 The FT ACK TLV is encoded as follows. 1641 0 1 2 3 1642 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1643 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1644 |0|0| FT ACK (0x0504) | Length (= 4) | 1645 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1646 | FT ACK Sequence Number | 1647 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1649 FT ACK Sequence Number 1650 The sequence number for this most recent FT 1651 label message that the sending LDP peer has 1652 received from the receiving LDP peer and 1653 secured against failure of the LDP session. 1654 It is not necessary for the sending peer to 1655 have fully processed the message before 1656 ACKing it. For example, an LSR MAY ACK a 1657 Label Request message as soon as it has 1658 securely recorded the message, without 1659 waiting until it can send the Label Mapping 1660 message in response. 1662 ACKs are cumulative. Receipt of an LDP 1663 message containing an FT ACK TLV with an FT 1664 ACK Sequence Number of 12 is treated as the 1665 acknowledgement of all messages from 1 to 1666 12 inclusive (assuming the LDP session 1667 started with a sequence number of 1). 1669 This field MUST be set to 0 if the LSR 1670 sending the FT ACK TLV has not received any 1671 FT label operations on this LDP session. 1672 This would apply to LDP sessions to new LDP 1673 peers or after an LSR determines that it 1674 must drop all state for a failed TCP 1675 connection. 1677 See the section "FT Operation Acks" for 1678 details of how this field is used. 1680 If an LSR receives a message affecting a label that it 1681 believes is an FT label and that message does not contain 1682 the FT Protection TLV, it SHOULD send a Notification 1683 message to its LDP peer containing the "Missing FT 1684 Protection TLV" status code. 1686 If an LSR receives an FT ACK TLV that contains an FT ACK 1687 Sequence Number that is less than the previously received 1688 FT ACK Sequence Number (remembering to take account of 1689 wrapping), it SHOULD send a Notification message to its 1690 LDP peer containing the "FT ACK Sequence Error" status 1691 code. 1693 7.5. FT Cork TLV 1695 LDP peers use the FT Cork TLV on FT Keepalive messages to 1696 indicate that they wish to quiesce the LDP session prior 1697 to a controlled shutdown and restart, for example during 1698 control-plane software upgrade. 1700 The FT Cork TLV is encoded as follows. 1702 0 1 2 3 1703 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1704 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1705 |0|0| FT Cork (0x0505) | Length (= 0) | 1706 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1708 On receipt of a Keepalive message with the FT Cork TLV 1709 and the FT Protection TLV, an LSR SHOULD perform the 1710 following actions 1712 - Process and secure any messages from the peer LSR that 1713 have sequence numbers less than (accounting for wrap) that 1714 contained in the FT Protection TLV on the Keepalive message. 1716 - Send a Keepalive message back to the peer containing the 1717 FT Cork TLV and the FT ACK TLV specifying the FT ACK 1718 sequence number equal to that in the original Keepalive 1719 message (i.e. ACKing all messages up to that point). 1721 - If this LSR has not yet received an FT ACK to all the 1722 messages it has sent containing the FT Protection TLV, then 1723 also include an FT Protection TLV on the Keepalive sent to 1724 the peer LSR. This tells the remote peer that the local LSR 1725 has saved state prior to quiesce but is still awaiting 1726 confirmation that the remote peer has saved state. 1728 - Cease sending any further state changing messages on this 1729 LDP session until it has been disconnected and recovered. 1731 On receipt of a Keepalive message with the FT Cork TLV 1732 and the FT ACK TLV (but NOT the FT Protection TLV), an 1733 LSR knows that 3-way handshake it initiated is complete 1734 and it SHOULD now send a "Temporary Shutdown" 1735 Notification message, disconnect the TCP session and 1736 perform whatever control plane actions required this 1737 session shutdown. 1739 An example such 3-way handshake for controlled shutdown 1740 is given in section 8. 1742 If an LSR receives a message that should not carry the FT 1743 Cork TLV, or if the FT Cork TLV is used on a Keepalive 1744 message without one of the FT Protection or FT ACK TLVs 1745 present, , it SHOULD send a Notification message to its 1746 LDP peer containing the "Unexpected FR Cork TLV" status 1747 code. 1749 8. Example Use 1751 Consider two LDP peers, P1 and P2, implementing LDP over 1752 a TCP connection that connects them, and the message flow 1753 shown below. 1755 The parameters shown on each message shown below are as 1756 follows: 1758 message (label, senders FT sequence number, FT ACK 1759 number) 1761 A "-" for FT ACK number means that the FT ACK TLV is 1762 not included on that message. "n/a" means that the 1763 parameter in question is not applicable to that type 1764 of message. 1766 In the diagrams below, time flows from top to bottom. 1767 The relative position of each message shows when it is 1768 transmitted. See the notes for a description of when 1769 each message is received, secured for FT or processed. 1771 8.1. Session Failure and Recovery - FT Procedures 1773 notes P1 P2 1774 ===== == == 1775 (1) Label Request(L1,27,-) 1776 ---------------------------> 1777 Label Request(L2,28,-) 1778 ---------------------------> 1779 (2) Label Request(L3,93,27) 1780 <--------------------------- 1781 (3) Label Request(L1,123,-) 1782 --------------------------> 1783 Label Request(L2,124,-) 1784 --------------------------> 1785 (4) Label Mapping(L1,57,-) 1786 <-------------------------- 1787 Label Mapping(L1,94,28) 1788 <--------------------------- 1789 (5) Label Mapping(L2,58,-) 1790 <-------------------------- 1791 Label Mapping(L2,95,-) 1792 <--------------------------- 1793 (6) Address(n/a,29,-) 1794 ---------------------------> 1795 (7) Label Request(L4,30,-) 1796 ---------------------------> 1797 (8) Keepalive(n/a,-,94) 1798 ---------------------------> 1799 (9) Label Abort(L3,96,-) 1800 <--------------------------- 1801 (10) ===== TCP Session lost ===== 1802 : 1803 (11) : Label Withdraw(L1,59,-) 1804 : <-------------------------- 1805 : 1806 (12) === TCP Session restored === 1808 LDP Init(n/a,n/a,94) 1809 ---------------------------> 1810 LDP Init(n/a,n/a,29) 1811 <--------------------------- 1812 (13) Label Request(L4,30,-) 1813 ---------------------------> 1814 (14) Label Mapping(L2,95,-) 1815 <--------------------------- 1816 Label Abort(L3,96,30) 1817 <--------------------------- 1818 (15) Label Withdraw(L1,97,-) 1819 <--------------------------- 1821 Notes: 1822 ====== 1824 (1) Assume that the LDP session has already been initialized. 1825 P1 issues 2 new Label Requests using the next sequence numbers. 1827 (2) P2 issues a third Label request to P1. At the time of sending 1828 this request, P2 has secured the receipt of the label request 1829 for L1 from P1, so it includes an ACK for that message. 1831 (3) P2 Processes the Label Requests for L1 and L2 and forwards them 1832 downstream. Details of downstream processing are not shown in 1833 the diagram above. 1835 (4) P2 receives a Label Mapping from downstream for L1, which it 1836 forwards to P1. It includes an ACK to the Label Request for L2, 1837 as that message has now been secured and processed. 1839 (5) P2 receives the Label Mapping for L2, which it forwards to P1. 1840 This time it does not include an ACK as it has not received any 1841 further messages from P1. 1843 (6) Meanwhile, P1 sends a new Address Message to P2 . 1845 (7) P1 also sends a fourth Label Request to P2 1847 (8) P1 sends a Keepalive message to P2, on which it includes an ACK 1848 for the Label Mapping for L1, which is the latest message P1 has 1849 received and secured at the time the Keepalive is sent. 1851 (9) P2 issues a Label Abort for L3. 1853 (10) At this point, the TCP session goes down. 1855 (11) While the TCP session is down, P2 receives a Label Withdraw 1856 Message for L1, which it queues. 1858 (12) The TCP session is reconnected and P1 and P2 exchange LDP 1859 Initialization messages on the recovered session, which include 1860 ACKS for the last message each peer received and secured prior 1861 to the failure. 1863 (13) From the LDP Init exchange, P1 determines that it needs to 1864 re-issue the Label request for L4. 1866 (14) Similarly, P2 determines that it needs to re-issue the Label 1867 Mapping for L2 and the Label Abort. 1869 (15) P2 issues the queued Label Withdraw to P1. 1871 8.2. Use of Check-Pointing With FT Procedures 1873 notes P1 P2 1874 ===== == == 1875 (1) Label Request(L1,27,-) 1876 ---------------------------> 1877 Label Request(L2,28,-) 1878 ---------------------------> 1879 (2) Label Request(L3,93,-) 1880 <--------------------------- 1881 (3) Label Request(L1,123,-) 1882 --------------------------> 1883 Label Request(L2,124,-) 1884 --------------------------> 1885 (4) Label Mapping(L1,57,-) 1886 <-------------------------- 1887 Label Mapping(L1,94,-) 1888 <--------------------------- 1889 (5) Label Mapping(L2,58,-) 1890 <-------------------------- 1891 Label Mapping(L2,95,-) 1892 <--------------------------- 1893 (6) Address(n/a,29,-) 1894 ---------------------------> 1895 (7) Label Request(L4,30,-) 1896 ---------------------------> 1897 (8) Keepalive(n/a,31,-) 1898 ---------------------------> 1899 (9) Keepalive(n/a,-,31) 1900 <--------------------------- 1901 (10) Keepalive(n/a,59,124) 1902 <--------------------------- 1903 (11) Keepalive(n/a,-,59) 1904 ---------------------------> 1906 Notes: 1907 ====== 1909 Notes (1) through (7) are as in the previous example 1910 except note that no acknowledgements are piggy-backed on 1911 reverse direction messages. This means that at note (8) 1912 there are deferred acknowledgements in both directions on 1913 both links. 1915 (8) P1 wishes to synchronize state with P2. It sends a Keepalive 1916 message containing a FT Protection TLV with sequence number 31. 1917 Since it is not interested in P2's perception of the state that 1918 it has stored, it does not include an FT ACK TLV. 1920 (9) P2 responds at once with a Keepalive acknowledging the sequence 1921 number on the received Keepalive. This tells P1 that P2 has 1922 preserved all state/messages previously received on this 1923 session. 1925 (10) P3 wishes to synchronize state with P2. It sends a Keepalive 1926 message containing a FT Protection TLV with sequence number 59. 1927 P3 also takes this opportunity to get up to date with its 1928 acknowledgements to P2 by including an FT ACK TLV acknowledging 1929 up to sequence number 124. 1931 (11) P2 responds at once with a Keepalive acknowledging the sequence 1932 number on the received Keepalive. 1934 8.3. Temporary Shutdown With FT Procedures 1936 notes P1 P2 1937 ===== == == 1938 (1) Label Request(L1,27,-) 1939 ---------------------------> 1940 Label Request(L2,28,-) 1941 ---------------------------> 1942 (2) Label Request(L3,93,27) 1943 <--------------------------- 1944 (3) Label Request(L1,123,-) 1945 --------------------------> 1946 Label Request(L2,124,-) 1947 --------------------------> 1948 (4) Label Mapping(L1,57,-) 1949 <-------------------------- 1950 Label Mapping(L1,94,28) 1951 <--------------------------- 1952 (5) Label Mapping(L2,58,-) 1953 <-------------------------- 1954 Label Mapping(L2,95,-) 1955 <--------------------------- 1956 (6) Address(n/a,29,-) 1957 ---------------------------> 1958 (7) Label Request(L4,30,-) 1959 ---------------------------> 1960 (8) Keepalive(n/a,-,94) 1961 ---------------------------> 1962 (9) Label Abort(L3,96,-) 1963 <--------------------------- 1964 (10) Notification(Temporary shutdown) 1965 ---------------------------> 1966 ===== TCP Session shutdown ===== 1967 : 1968 (11) : Label Withdraw(L1,59,-) 1969 : <-------------------------- 1970 : 1971 ===== TCP Session restored ===== 1972 (12) LDP Init(n/a,n/a,94) 1973 ---------------------------> 1974 LDP Init(n/a,n/a,29) 1975 <--------------------------- 1976 (13) Label Request(L4,30,-) 1977 ---------------------------> 1978 (14) Label Mapping(L2,95,-) 1979 <--------------------------- 1980 Label Abort(L3,96,30) 1981 <--------------------------- 1982 (15) Label Withdraw(L1,97,-) 1983 <--------------------------- 1985 Notes: 1986 ====== 1988 Notes are as in the previous example except as follows. 1990 (10) P1 needs to upgrade the software or hardware that it is running. 1991 It issues a Notification message to terminate the LDP session, 1992 but sets the status code as 'Temporary shutdown' to inform P2 1993 that this is not a fatal error, and P2 should maintain FT state. 1994 The TCP connection may also fail during the period that the LDP 1995 session is down (in which case it will need to be 1996 re-established), but it is also possible that the TCP connection 1997 will be preserved. 1999 8.4. Temporary Shutdown With FT Procedures and Check-Pointing 2001 notes P1 P2 2002 ===== == == 2003 (1) Label Request(L1,27,-) 2004 ---------------------------> 2005 Label Request(L2,28,-) 2006 ---------------------------> 2007 (2) Label Request(L3,93,-) 2008 <--------------------------- 2009 Label Request(L1,123,-) 2010 --------------------------> 2011 Label Request(L2,124,-) 2012 --------------------------> 2013 Label Mapping(L1,57,-) 2014 <-------------------------- 2015 (3) Label Mapping(L1,94,-) 2016 <--------------------------- 2017 Label Mapping(L2,58,-) 2018 <-------------------------- 2019 Label Mapping(L2,95,-) 2020 <--------------------------- 2021 (4) Address(n/a,29,-) 2022 ---------------------------> 2023 (5) Label Request(L4,30,-) 2024 ---------------------------> 2025 (6) Keepalive(n/a,31,95) * with FT Cork TLV * 2026 ---------------------------> 2027 (7) Label Abort(L3,96,-) 2028 <--------------------------- 2029 (8) Keepalive(n/a,97,31) * with FT Cork TLV * 2030 <--------------------------- 2031 (9) Keepalive(n/a,-,97) * with FT Cork TLV * 2032 ---------------------------> 2033 (10) Notification(Temporary shutdown) 2034 ---------------------------> 2035 ===== TCP Session shutdown ===== 2036 : 2037 : Label Withdraw(L1,59,-) 2038 : <-------------------------- 2039 : 2040 ===== TCP Session restored ===== 2041 (11) LDP Init(n/a,n/a,96) 2042 ---------------------------> 2043 LDP Init(n/a,n/a,31) 2044 <--------------------------- 2045 Label Withdraw(L1,97,-) 2046 <--------------------------- 2048 Notes: 2050 This example operates much as the previous one. However, 2051 at (1), (2), (3), (4) and (5) no acknowledgements are 2052 made. 2054 At (6), P1 determines that graceful shutdown is required 2055 and sends a Keepalive acknowledging all previously 2056 received messages and itself containing a FT Protection 2057 TLV number and the FT Cork TLV. 2059 The Label abort at (7) crosses with this Keepalive, so at 2060 (8) P2 sends a Keepalive that acknowledges all messages 2061 received so far, but also including the FT Protection and 2062 FT Cork TLVs to indicate that there are still messages 2063 outstanding to be acknowledged. 2065 P1 is then able to complete the 3-way handshake at (9) 2066 and close the TCP session at (10). 2068 Upon recovery at (11) there are no messages to be re-sent 2069 because the Keepalives flushed the acknowledgements. The 2070 only messages sent after recovery is the Label Withdraw 2071 that was pended during the TCP session failure. 2073 8.5. Checkpointing Without FT Procedures 2075 notes P1 P2 2076 ===== == == 2077 (1) Label Request(L1) 2078 ---------------------------> 2079 (2) Label Request(L2) 2080 <--------------------------- 2081 Label Request(L1) 2082 --------------------------> 2083 Label Mapping(L1) 2084 <-------------------------- 2085 (3) Label Mapping(L1) 2086 <--------------------------- 2087 (4) Keepalive(n/a,12,-) 2088 ---------------------------> 2089 (5) Label Request(L3) 2090 ---------------------------> 2091 (6) Keepalive(n/a,-,12) 2092 <--------------------------- 2093 Label Request(L3) 2094 --------------------------> 2095 Label Mapping(L3) 2096 <-------------------------- 2097 (7) Label Mapping(L3) 2098 <--------------------------- 2099 ===== TCP Session failure ===== 2100 : 2101 : 2102 : 2103 ===== TCP Session restored ===== 2104 (8) LDP Init(n/a,n/a,23) 2105 ---------------------------> 2106 LDP Init(n/a,n/a,12) 2107 <--------------------------- 2108 (9) Label Request(L3) 2109 ---------------------------> 2110 Label Request(L3) 2111 --------------------------> 2112 Label Mapping(L3) 2113 <-------------------------- 2114 Label Mapping(L3) 2115 <--------------------------- 2116 (10) Label Request(L2) 2117 <--------------------------- 2119 Notes: 2121 (1), (2) and (3) show label distribution without FT sequence numbers. 2123 (4) A checkpoint request from P1. It carries the sequence number of 2124 the checkpoint request. 2126 (5) P1 immediately starts a new label distribution request. 2128 (6) P2 confirms that it has secured all previous transactions. 2130 (7) The subsequent (un-acknowledged) label distribution completes. 2132 (8) The session fails and is restarted. Initialization messages 2133 confirm the sequence numbers of the secured checkpoints. 2135 (9) P1 recommences the unacknowledged label distribution request. 2137 (10) P2 recommences an unacknowledged label distribution request. 2139 8.6. Graceful Shutdown With Checkpointing But No FT Procedures 2141 notes P1 P2 2142 ===== == == 2143 (1) Label Request(L1) 2144 ---------------------------> 2145 (2) Label Request(L2) 2146 <--------------------------- 2147 Label Request(L1) 2148 --------------------------> 2149 Label Mapping(L1) 2150 <-------------------------- 2151 (3) Label Mapping(L1) 2152 <--------------------------- 2153 (4) Keepalive(n/a,12,23) * With FT Cork TLV * 2154 ---------------------------> 2155 (5) : 2156 : 2157 : 2158 (6) Keepalive(n/a,24,12) * With FT Cork TLV * 2159 <--------------------------- 2160 (7) Keepalive(n/a,-,24) * With FT Cork TLV * 2161 ---------------------------> 2162 (8) Notification(Temporary shutdown) 2163 ---------------------------> 2164 ===== TCP Session failure ===== 2165 : 2166 : 2167 : 2168 ===== TCP Session restored ===== 2169 (9) LDP Init(n/a,n/a,24) 2170 ---------------------------> 2171 LDP Init(n/a,n/a,12) 2172 <--------------------------- 2173 (10) Label Request(L3) 2174 ---------------------------> 2175 Label Request(L3) 2176 --------------------------> 2177 Label Mapping(L3) 2178 <-------------------------- 2179 Label Mapping(L3) 2180 <--------------------------- 2181 (11) Label Mapping(L2) 2182 ---------------------------> 2184 Notes: 2186 (1), (2) and (3) show label distribution without FT sequence numbers. 2188 (4) A checkpoint request from P1. It carries the sequence number of 2189 the checkpoint request and a Cork TLV. 2191 (5) P1 has sent a Cork TLV so quieces. 2193 (6) P2 confirms the checkpoint and continues the three-way handshake 2194 by including a Cork TLV itself. 2196 (7) P1 completes the three-way handshake. All operations have now 2197 been checkpointed and the session is quiesced. 2199 (8) The session is gracefully shut down. 2201 (9) The session recovers and the peers exchange the sequence numbers 2202 of the last secured checkpoints. 2204 (10) P1 starts a new label distribution request. 2206 (11) P1 continues processing a previously received label distribution 2207 request. 2209 9. Security Considerations 2211 The LDP FT enhancements inherit similar security 2212 considerations to those discussed in [2] and [4]. 2214 The LDP FT enhancements allow the re-establishment of a 2215 TCP connection between LDP peers without a full re- 2216 exchange of the attributes of established labels, which 2217 renders LSRs that implement the extensions specified in 2218 this draft vulnerable to additional denial-of-service 2219 attacks as follows: 2221 - An intruder may impersonate an LDP peer in order to force 2222 a failure and reconnection of the TCP connection, but where 2223 the intruder does not set the FT Reconnect Flag on re- 2224 connection. This forces all FT labels to be released. 2226 - Similarly, an intruder could set the FT Reconnect Flag on 2227 re-establishment of the TCP session without preserving the 2228 state and resources for FT labels. 2230 - An intruder could intercept the traffic between LDP peers 2231 and override the setting of the FT Label Flag to be set to 0 2232 for all labels. 2234 All of these attacks may be countered by use of an 2235 authentication scheme between LDP peers, such as the MD5- 2236 based scheme outlined in [4]. 2238 Alternative authentication schemes for LDP peers are 2239 outside the scope of this draft, but could be deployed to 2240 provide enhanced security to implementations of LDP, CR- 2241 LDP and the LDP FT enhancements. 2243 As with LDP and CR-LDP, a security issue may exist if an 2244 LDP implementation continues to use labels after 2245 expiration of the session that first caused them to be 2246 used. This may arise if the upstream LSR detects the 2247 session failure after the downstream LSR has released and 2248 re-used the label. The problem is most obvious with the 2249 platform-wide label space and could result in mis-routing 2250 of data to other than intended destinations and it is 2251 conceivable that these behaviors may be deliberately 2252 exploited to either obtain services without authorization 2253 or to deny services to others. 2255 In this draft, the validity of the session may be 2256 extended by the FT Reconnection Timeout, and the session 2257 may be re-established in this period. After the expiry 2258 of the Reconnection Timeout the session must be 2259 considered to have failed and the same security issue 2260 applies as described above. 2262 However, the downstream LSR may declare the session as 2263 failed before the expiration of its Reconnection Timeout. 2264 This increases the period during which the downstream LSR 2265 might reallocate the label while the upstream LSR 2266 continues to transmit data using the old usage of the 2267 label. To reduce this issue, this draft requires that 2268 labels are not re-used until the Reconnection Timeout has 2269 expired. 2271 A further issue might apply if labels were re-used prior 2272 to the expiration of the FT Reconnection Timeout, but 2273 this is forbidden by this draft. 2275 10. Implementation Notes 2277 10.1. FT Recovery Support on Non-FT LSRs 2279 In order to take full advantage of the FT capabilities of 2280 LSRs in the network, it may be that an LSR that does not 2281 itself contain the ability to recover from local hardware 2282 or software faults still needs to support the LDP FT 2283 enhancements described in this draft. 2285 Consider an LSR, P1, that is an LDP peer of a fully Fault 2286 Tolerant LSR, P2. If P2 experiences a fault in the 2287 hardware or software that serves an LDP session between 2288 P1 and P2, it may fail the TCP connection between the 2289 peers. When the connection is recovered, the LSPs/labels 2290 between P1 and P2 can only be recovered if both LSRs were 2291 applying the FT recovery procedures to the LDP session. 2293 10.2. ACK Generation Logic 2295 FT ACKs SHOULD be returned to the sending LSR as soon as 2296 is practicable in order to avoid building up a large 2297 quantity of unacknowledged state changes at the LSR. 2298 However, immediate one-for-one acknowledgements would 2299 waste bandwidth unnecessarily. 2301 A possible implementation strategy for sending ACKs to FT 2302 LDP messages is as follows: 2304 - An LSR secures received messages in order and tracks the 2305 sequence number of the most recently secured message, Sr. 2307 - On each LDP KeepAlive that the LSR sends, it attaches an 2308 FT ACK TLV listing Sr 2310 - Optionally, the LSR may attach an FT ACK TLV to any other 2311 LDP message sent between Keepalive messages if, for example, 2312 Sr has increased by more than a threshold value since the 2313 last ACK sent. 2315 This implementation combines the bandwidth benefits of 2316 accumulating ACKs while still providing timely ACKs. 2318 10.2.1 Ack Generation Logic When Using Check-Pointing 2320 If check-pointing is in use, the LSRs need not be 2321 concerned to send ACKs in such a timely manner. 2323 Check-points are solicitations for acknowledgement 2324 conveyed as a sequence number in an FT Protection TLV on 2325 a Keepalive message. Such check-point requests could be 2326 issued on a timer, after a significant amount of change, 2327 or before controlled shutdown of a session. 2329 The use of check-pointing may considerably simplify an 2330 implementation since it does not need to track the 2331 sequence numbers of all received LDP messages. It must, 2332 however, still ensure that all received messages (or the 2333 consequent state changes) are secured before 2334 acknowledging the sequence number on the Keepalive. 2336 This approach may be considered optimal in systems that 2337 do not show a high degree of change over time (such as 2338 targeted LDP session or CR-LDP systems) and that are 2339 prepared to risk loss of state for the most recent LDP 2340 exchanges. More dynamic systems (such as LDP discovery 2341 sessions) are more likely to want to acknowledge state 2342 changes more frequently so that the maximum amount of 2343 state can be preserved over a failure. 2345 11. Acknowledgments 2347 The work in this draft is based on the LDP and CR-LDP 2348 ideas expressed by the authors of [2] and [4]. 2350 The ACK scheme used in this draft was inspired by the 2351 proposal by David Ward and John Scudder for restarting 2352 BGP sessions now included in [9]. 2354 The authors would also like to acknowledge the careful 2355 review and comments of Nick Weeds, Piers Finlayson, Tim 2356 Harrison, Duncan Archer, Peter Ashwood-Smith, Bob Thomas, 2357 S.Manikantan, Adam Sheppard and Alan Davey. 2359 12. Intellectual Property Consideration 2361 The IETF has been notified of intellectual property 2362 rights claimed in regard to some or all of the 2363 specification contained in this document. For more 2364 information, consult the online list of claimed rights. 2366 13. Full Copyright Statement 2368 Copyright (c) The Internet Society (2000, 2001). All 2369 Rights Reserved. This document and translations of it may 2370 be copied and furnished to others, and derivative works 2371 that comment on or otherwise explain it or assist in its 2372 implementation may be prepared, copied, published and 2373 distributed, in whole or in part, without restriction of 2374 any kind, provided that the above copyright notice and 2375 this paragraph are included on all such copies and 2376 derivative works. However, this document itself may not 2377 be modified in any way, such as by removing the copyright 2378 notice or references to the Internet Society or other 2379 Internet organizations, except as needed for the purpose 2380 of developing Internet standards in which case the 2381 procedures for copyrights defined in the Internet 2382 Standards process must be followed, or as required to 2383 translate it into languages other than English. 2385 The limited permissions granted above are perpetual and 2386 will not be revoked by the Internet Society or its 2387 successors or assigns. 2389 This document and the information contained herein is 2390 provided on an "AS IS" basis and THE INTERNET SOCIETY AND 2391 THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL 2392 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED 2393 TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN 2394 WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 2395 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 2397 14. IANA Considerations 2399 This draft requires the use of a number of new TLVs and 2400 status codes from the number spaces within the LDP 2401 protocol. This section explains the logic used by the 2402 authors to choose the most appropriate number space for 2403 each new entity, and is intended to assist in the 2404 determination of any final values assigned by IANA or the 2405 MPLS WG in the event that the MPLS WG chooses to advance 2406 this draft on the standards track. 2408 This section will be removed when the TLV and status code 2409 values have been agreed with IANA. 2411 14.1. FT Session TLV 2413 The FT Session TLV carries attributes that affect the 2414 entire LDP session between LDP peers. It is suggested 2415 that the type for this TLV should be chosen from the 2416 0x05xx range for TLVs that is used in [4] by other TLVs 2417 carrying session-wide attributes. At the time of this 2418 writing, the next available number in this range is 2419 0x0503. 2421 14.2. FT Protection TLV 2423 The FT Protection TLV carries attributes that affect a 2424 single label exchanged between LDP peers. It is 2425 suggested that the type for this TLV should be chosen 2426 from the 0x02xx range for TLVs that is used in [4] by 2427 other TLVs carrying label attributes. At the time of 2428 this writing, the next available number in this range is 2429 0x0203. 2431 Consideration was given to using the message number field 2432 instead of a new FT Sequence Number field. However, the 2433 authors felt this placed unacceptable implementation 2434 constraints on the use of message number (e.g. it could 2435 no longer be used to reference a control block). 2437 14.3. FT ACK TLV 2439 The FT Protection TLV may ACK many label operations at 2440 once if cumulative ACKS are used. It is suggested that 2441 the type for this TLV should be chosen from the 0x05xx 2442 range for TLVs that is used in [4] by other TLVs carrying 2443 session-wide attributes. At the time of this writing, 2444 the next available number in this range is 0x0504. 2446 Consideration was given to carrying the FT ACK Number in 2447 the FT Protection TLV, but the authors felt this would be 2448 inappropriate as many implementations may wish to carry 2449 the ACKs on Keepalive messages. 2451 14.4. FT Cork TLV 2453 The FT Cork TLV carries attributes that affect a single 2454 label exchanged between LDP peers. It is suggested that 2455 the type for this TLV should be chosen from the 0x05xx 2456 range for TLVs that is used in [4] by other TLVs carrying 2457 label attributes. At the time of this writing, the next 2458 available number in this range is 0x0505. 2460 14.5. Status Codes 2462 The authors' current understanding is that MPLS status 2463 codes are not sub-divided into specific ranges for 2464 different types of error. Hence, the numeric status code 2465 values assigned for this draft should simply be the next 2466 available values at the time of writing and may be 2467 substituted for other numeric values. 2469 See section "Status Codes" for details of the status 2470 codes defined in this draft. 2472 15. Authors' Addresses 2474 Adrian Farrel (editor) Paul Brittain 2475 Movaz Networks, Inc. Data Connection Ltd. 2476 7926 Jones Branch Drive, Suite 615 Windsor House, Pepper Street, 2477 McLean, VA 22102 Chester, Cheshire 2478 Phone: +1 703-847-1867 CH1 1DF, UK 2479 Email: afarrel@movaz.com Phone: +44-(0)20-8366-1177 2480 Email: pjb@dataconnection.com 2482 Philip Matthews Eric Gray 2483 Hyperchip Sandburst Corporation 2484 1800 Rene-Levesque Blvd W 600 Federal Street 2485 Montreal, Quebec H3H 2H2 Andover, MA 01810 2486 Canada Phone: +1 978-689-1600 2487 Phone: +1 514-906-4965 Email:eric.gray@sandburst.com 2488 Email: pmatthews@hyperchip.com 2489 Jack Shaio Toby Smith 2490 Vivace Networks Laurel Networks, Inc. 2491 2730 Orchard Parkway 1300 Omega Drive 2492 San Jose, CA 95134 Pittsburgh, PA 15205 2493 Phone: +1 408 432 7623 Email: tob@laurelnetworks.com 2494 Email: jack.shaio@vivacenetworks.com 2496 Andy Malis 2497 Vivace Networks 2498 2730 Orchard Parkway 2499 San Jose, CA 95134 2500 Phone: +1 408 383 7223 2501 andy.malis@vivacenetworks.com 2503 16. Normative References 2505 4 Andersson, L., et. al., LDP Specification, RFC 3036, January 2001. 2507 17. Informative References 2509 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 2510 9, RFC 2026, October 1996. 2512 2 Jamoussi, B., et. al., Constraint-Based LSP Setup using LDP, 2513 draft-ietf-mpls-cr-ldp-06.txt, November 2001, (work in progress). 2515 3 Bradner, S., "Key words for use in RFCs to Indicate Requirement 2516 Levels", BCP 14, RFC 2119, March 1997. 2518 5 Ash, G., et al., LSP Modification Using CR-LDP, draft-ietf-mpls- 2519 crlsp-modify-03.txt, March 2001 (work in progress). 2521 6 Braden, R., et al., Resource ReSerVation Protocol (RSVP) -- 2522 Version 1, Functional Specification, RFC 2205, September 1997. 2524 7 Berger, L., et al., RSVP Refresh Reduction Extensions, RFC 2961 2525 April 2001. 2527 8 Awduche, D., et al,. Extensions to RSVP for LSP Tunnels, RFC 3209, 2528 December 2001. 2530 9 Ramachandra, S., et al., Graceful Restart Mechanism for BGP, 2531 draft-ietf-idr-restart-00.txt, December 2000 (work in progress). 2533 10 Stewart, R., et al., Stream Control Transmission Protocol, 2534 RFC 2906, October 2000. 2536 11 Moy, J., Hitless OSPF Restart, draft-ietf-ospf-hitless-restart- 2537 00.txt, February 2001 (work in progress). 2539 12 Leelanivas, M., et al., Graceful Restart Mechanism for LDP, draft- 2540 ietf-ldp-restart-00.txt, January 2002, work in progress.