idnits 2.17.1 draft-ietf-forces-ceha-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 317 has weird spacing: '... |try v...' == Line 566 has weird spacing: '...ociated v...' == Line 567 has weird spacing: '...)|CE or retry...' -- The document date (February 19, 2013) is 4081 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group K. Ogawa 3 Internet-Draft NTT Corporation 4 Intended status: Standards Track W. M. Wang 5 Expires: August 23, 2013 Zhejiang Gongshang University 6 E. Haleplidis 7 University of Patras 8 J. Hadi Salim 9 Mojatatu Networks 10 February 19, 2013 12 ForCES Intra-NE High Availability 13 draft-ietf-forces-ceha-06 15 Abstract 17 This document discusses CE High Availability within a ForCES NE. 19 Status of this Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at http://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on August 23, 2013. 36 Copyright Notice 38 Copyright (c) 2013 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 55 2.1. Document Scope . . . . . . . . . . . . . . . . . . . . . . 5 56 2.2. Quantifying Problem Scope . . . . . . . . . . . . . . . . 5 57 3. RFC5810 CE HA Framework . . . . . . . . . . . . . . . . . . . 6 58 3.1. RFC 5810 CE High Availability Support . . . . . . . . . . 6 59 3.1.1. Cold Standby Interaction with ForCES Protocol . . . . 7 60 3.1.2. Responsibilities for HA . . . . . . . . . . . . . . . 9 61 4. CE HA Hot Standby . . . . . . . . . . . . . . . . . . . . . . 10 62 4.1. Changes to the FEPO model . . . . . . . . . . . . . . . . 10 63 4.2. FEPO processing . . . . . . . . . . . . . . . . . . . . . 12 64 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 65 6. Security Considerations . . . . . . . . . . . . . . . . . . . 16 66 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16 67 7.1. Normative References . . . . . . . . . . . . . . . . . . . 16 68 7.2. Informative References . . . . . . . . . . . . . . . . . . 17 69 Appendix A. New FEPO version . . . . . . . . . . . . . . . . . . 17 70 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 72 1. Definitions 74 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 75 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 76 document are to be interpreted as described in RFC 2119. 78 The following definitions are taken from [RFC3654]and [RFC3746]: 80 o Logical Functional Block (LFB) -- A template that represents a 81 fine-grained, logically separate aspects of FE processing. 83 o ForCES Protocol -- The protocol used at the Fp reference point in 84 the ForCES Framework in [RFC3746]. 86 o ForCES Protocol Layer (ForCES PL) -- A layer in the ForCES 87 architecture that embodies the ForCES protocol and the state 88 transfer mechanisms as defined in [RFC5810]. 90 o ForCES Protocol Transport Mapping Layer (ForCES TML) -- A layer in 91 ForCES protocol architecture that specifically addresses the 92 protocol message transportation issues, such as how the protocol 93 messages are mapped to different transport media (like SCTP, IP, 94 TCP, UDP, ATM, Ethernet, etc), and how to achieve and implement 95 reliability, security, etc. 97 2. Introduction 99 Figure 1 illustrates a ForCES NE controlled by a set of redundant CEs 100 with CE1 being active and CE2 and CEN being a backup. 102 ----------------------------------------- 103 | ForCES Network Element | 104 | +-----------+ | 105 | | CEN | | 106 | | (Backup) | | 107 -------------- Fc | +------------+ +------------+ | | 108 | CE Manager |--------+-| CE1 |------| CE2 |-+ | 109 -------------- | | (Active) | Fr | (Backup) | | 110 | | +-------+--+-+ +---+---+----+ | 111 | Fl | | | Fp / | | 112 | | | +---------+ / | | 113 | | Fp| |/ |Fp | 114 | | | | | | 115 | | | Fp /+--+ | | 116 | | | +-------+ | | | 117 | | | | | | | 118 -------------- Ff | --------+--+-- ----+---+----+ | 119 | FE Manager |--------+-| FE1 | Fi | FE2 | | 120 -------------- | | |------| | | 121 | -------------- -------------- | 122 | | | | | | | | | | 123 ----+--+--+--+----------+--+--+--+------- 124 | | | | | | | | 125 | | | | | | | | 126 Fi/f Fi/f 128 Fp: CE-FE interface 129 Fi: FE-FE interface 130 Fr: CE-CE interface 131 Fc: Interface between the CE Manager and a CE 132 Ff: Interface between the FE Manager and an FE 133 Fl: Interface between the CE Manager and the FE Manager 134 Fi/f: FE external interface 136 Figure 1: ForCES Architecture 138 The ForCES architecture allows FEs to be aware of multiple CEs but 139 enforces that only one CE be the master controller. This is known in 140 the industry as 1+N redundancy. The master CE controls the FEs via 141 the ForCES protocol operating in the Fp interface. If the master CE 142 becomes faulty, a backup CE takes over and NE operation continues. 143 By definition, the current documented setup is known as cold-standby. 144 The set of CEs controlling an FE is static and is passed to the FE by 145 the FE Manager (FEM) via the Ff interface and to each CE by the CE 146 Manager (CEM) in the Fc interface during the pre-association phase. 148 From an FE perspective, the knobs of control for a CE set are defined 149 by the FEPO LFB in [RFC5810], Appendix B. Section 3.1 of this 150 document details these knobs further. 152 2.1. Document Scope 154 It is assumed that the reader is aware of the ForCES architecture to 155 make sense of the changes being described in this document. This 156 document provides background information to set the context of the 157 discussion in Section 4. 159 At the time this document is being written, the Fr interface is out 160 of scope for the ForCES architecture. However, it is expected that 161 organizations implementing a set of CEs will need to have the CEs 162 communicate to each other via the Fr interface in order to achieve 163 the synchronization necessary for controlling the FEs. 165 The problem scope addressed by this document falls into 2 areas: 167 1. To describe with more clarity (than [RFC5810]) how current cold- 168 standby approach operates within the NE cluster. 170 2. To describe how to evolve the [RFC5810] cold-standby setup to a 171 hot-standby redundancy setup so as to improve the failover time 172 and NE availability. 174 2.2. Quantifying Problem Scope 176 The NE recovery and availability is dependent on several time- 177 sensitive metrics: 179 1. How fast the CE plane failure is detected by the FE. 181 2. How fast a backup CE becomes operational. 183 3. How fast the FEs associate with the new master CE. 185 4. How fast the FEs recover their state and become operational. 187 The design intent of the current [RFC5810] as well as this document 188 to meet the above goals are driven by desire for simplicity. 190 To quantify the above criteria with the current prescribed ForCES CE 191 setup in [RFC5810]: 193 1. How fast the FE side detects a CE failure is left undefined. To 194 illustrate an extreme scenario, we could have a human operator 195 acting as the monitoring entity to detect faulty CEs. How fast 196 such detection happens could be in the range of seconds to days. 197 A more active monitor on the Fr interface could improve this 198 detection. 200 2. How fast the backup CE becomes operational is also currently out 201 of scope. In the current setup, a backup CE need not be 202 operational at all (for example, to save power) and therefore it 203 is feasible for a monitoring entity to boot up a backup CE after 204 it detects the failure of the master CE. In this document 205 Section 4 we suggest that at least one backup CE be online so as 206 to improve this metric. 208 3. How fast an FE associates with new master CE is also currently 209 undefined. The cost of an FE connecting and associating adds to 210 the recovery overhead. As mentioned above we suggest having at 211 least one backup CE online. In Section 4 we propose to zero out 212 the connection and association cost on failover by having each FE 213 associate with all online backup CEs after associating to an 214 active/master CE. Note that if an FE pre-associates with at 215 least one backup CE, then the system will be technically 216 operating in hot-standby mode. 218 4. And last: How fast an FE recovers its state depends on how much 219 NE state exists. By ForCES current definition, the new master CE 220 assumes zero state on the FE and starts from scratch to update 221 the FE. So the larger the state, the longer the recovery. 223 3. RFC5810 CE HA Framework 225 To achieve CE High Availabilty, FEs and CEs MUST inter-operate per 226 [RFC5810] definition which is repeated for contextual reasons in 227 Section 3.1. It should be noted that in this default setup, which 228 MUST be implemented by CEs and FEs needing HA, the Fr plane is out of 229 scope (and if available is proprietary to an implementation). 231 3.1. RFC 5810 CE High Availability Support 233 As mentioned earlier, although there can be multiple redundant CEs, 234 only one CE actively controls FEs in a ForCES NE. In practice there 235 may be only one backup CE. At any moment in time, only one master CE 236 can control an FE. In addition, the FE connects and associates to 237 only the master CE. The FE and the CE are aware of the primary and 238 one or more secondary CEs. This information (primary, secondary CEs) 239 is configured on the FE and the CE during pre-association by the FEM 240 and the CEM respectively. 242 Figure 2 below illustrates the Forces message sequences that the FE 243 uses to recover the connection in current defined cold-standby 244 scheme. 246 FE CE Primary CE Secondary 247 | | | 248 | Asso Estb,Caps exchg | | 249 1 |<--------------------->| | 250 | | | 251 | state update | | 252 2 |<--------------------->| | 253 | | | 254 | | | 255 | FAILURE | 256 | | 257 | Asso Estb,Caps exchange | 258 3 |<------------------------------------------>| 259 | | 260 | Event Report (pri CE down) | 261 4 |------------------------------------------->| 262 | | 263 | state update | 264 5 |<------------------------------------------>| 266 Figure 2: CE Failover for Cold Standby 268 3.1.1. Cold Standby Interaction with ForCES Protocol 270 High Availability parameterization in an FE is driven by configuring 271 the FE Protocol Object (FEPO) LFB. 273 The FEPO CEID component identifies the current master CE and the 274 component table BackupCEs identifies the configured backup CEs. The 275 FEPO FE Heartbeat Interval, CE Heartbeat Dead Interval, and CE 276 Heartbeat policy help in detecting connectivity problems between an 277 FE and CE. The CE Failover policy defines how the FE should react on 278 a detected failure. The FEObject FEState component [RFC5812] defines 279 the operational forwarding status and control. The CE can turn off 280 the FE's forwarding operations by setting the FEState to AdminDisable 281 and can turn it on by setting it to OperEnable. Note: [RFC5812] 282 section 5.1 has an erratta which describes the FEState as read-only 283 when it should be read-write. 285 Figure 3 illustrates the defined state machine that facilitates 286 connection recovery. 288 The FE connects to the CE specified on FEPO CEID component. If it 289 fails to connect to the defined CE, it moves it to the bottom of 290 table BackupCEs and sets its CEID component to be the first CE 291 retrieved from table BackupCEs. The FE then attempts to associate 292 with the CE designated as the new primary CE. The FE continues 293 through this procedure until it successfully connects to one of the 294 CEs. 296 FE tries to associate 297 +-->-----+ 298 | | 299 (CE changes master || | | 300 CE issues Teardown || +---+--------v----+ 301 Lost association) && | Pre-Association | 302 CE failover policy = 0 | (Association | 303 +------------>-->-->| in +<----+ 304 | | progress) | | 305 | | | | 306 | +--------+--------+ | 307 | CE Association | | CEFTI 308 | Response V | timer 309 | +------------------+ | expires 310 | |FE issue CEPrimaryDown ^ 311 | V | 312 +-+-----------+ +------+-----+ 313 | | (CE changes master || | Not | 314 | | CE issues Teardown || | Associated | 315 | | Lost association) && | +->---+ 316 | Associated | CE Failover Policy = 1 |(May | FE | 317 | | | Continue |try v 318 | |-------->------->------>| Forwarding)|assn | 319 | | Start CEFTI timer | |-<---+ 320 | | | | 321 +-------------+ +-------+-----+ 322 ^ | 323 | Successful V 324 | Association | 325 | Setup | 326 | (Cancel CEFTI Timer) | 327 +_________________________________________+ 328 FE issue CEPrimaryDown event 330 Figure 3: FE State Machine considering HA 332 There are several events that trigger mastership changes: The master 333 CE may issue a mastership change (by changing the CEID value), or 334 teardown an existing association; and last, connectivity may be lost 335 between the CE and FE. 337 When communication fails between the FE and CE (which can be caused 338 by either the CE or link failure but not FE related), either the TML 339 on the FE will trigger the FE PL regarding this failure or it will be 340 detected using the HB messages between FEs and CEs. The 341 communication failure, regardless of how it is detected, MUST be 342 considered as a loss of association between the CE and corresponding 343 FE. 345 If the FE's FEPO CE Failover Policy is configured to mode 0 (the 346 default), it will immediately transition to the pre-association 347 phase. This means that if association is later re-established with a 348 CE, all FE state will need to be re-created. 350 If the FE's FEPO CE Failover Policy is configured to mode 1, it 351 indicates that the FE will run in HA restart recovery. In such a 352 case, the FE transitions to the Not Associated state and the CEFTI 353 timer [RFC5810] is started. The FE MAY continue to forward packets 354 during this state. The FE recycles through any configured backup CEs 355 in a round-robin fashion. It first adds its primary CE to the bottom 356 of table BackupCEs and sets its CEID component to be the first 357 secondary retrieved from table BackupCEs. The FE then attempts to 358 associate with the CE designated as the new primary CE. If it fails 359 to re-associate with any CE and the CEFTI expires, the FE then 360 transitions to the pre-association state and FE will operationally 361 bring down its forwarding path (and set the [RFC5812] FEObject 362 FEState component to OperDisable). 364 If the FE, while in the not associated state, manages to reconnect to 365 a new primary CE before CEFTI expires it transitions to the 366 Associated state. Once re-associated, the CE may try to synchronize 367 any state that the FE may have lost during disconnection. How the CE 368 re-synchronizes such state is out of scope for the current ForCES 369 architecture but would typically constitute the issuing of new 370 configs and queries. 372 An explicit message (a Config message setting Primary CE component in 373 ForCES Protocol object) from the primary CE, can also be used to 374 change the Primary CE for an FE during normal protocol operation. In 375 this case, the FE transitions to the Not Associated State and 376 attempts to Associate with the new CE. 378 3.1.2. Responsibilities for HA 380 TML Level: 382 1. The TML controls logical connection availability and failover. 384 2. The TML also controls peer HA management. 386 At this level, control of all lower layers, for example transport 387 level (such as IP addresses, MAC addresses etc) and associated links 388 going down are the role of the TML. 390 PL Level: 391 All other functionality, including configuring the HA behavior during 392 setup, the CE IDs used to identify primary and secondary CEs, 393 protocol messages used to report CE failure (Event Report), Heartbeat 394 messages used to detect association failure, messages to change the 395 primary CE (Config), and other HA related operations described in 396 Section 3.1, are the PL's responsibility. 398 To put the two together, if a path to a primary CE is down, the TML 399 would take care of failing over to a backup path, if one is 400 available. If the CE is totally unreachable then the PL would be 401 informed and it would take the appropriate actions described before. 403 4. CE HA Hot Standby 405 In this section we describe small extensions to the existing scheme 406 to enable hot standby HA. To achieve hot standby HA, we target to 407 improve the specific goals defined in Section 2.2, namely: 409 o How fast a backup CE becomes operational. 411 o How fast the FEs associate with the new master CE. 413 As described in Section 3.1, in the pre-association phase the FEM 414 configures the FE to make it aware of all the CEs in the NE. The FEM 415 MUST configure the FE to make it aware which CE is the master and MAY 416 specify any backup CE(s). 418 4.1. Changes to the FEPO model 420 In order for the above to be achievable there is a need to make a few 421 changes in the FEPO model. Appendix A contains the xml definition of 422 the new version 1.1 of the FEPO LFB. 424 Changes from the version 1 of FEPO are: 426 1. Added four new datatypes: 428 1. CEStatusType an unsigned char to specify status of a 429 connection with a CE. Special values are: 431 + 0 (Disconnected) represents that no connection attempt has 432 been made with the CE yet 434 + 1 (Connected) represents that the FE connection with the 435 CE at the TML has completed successfully 437 + 2 (Associated) represents that the FE has successfully 438 associated with the CE 440 + 3 (IsMaster) represents that the FE has associated with 441 the CE and is the master of the FE 443 + 4 (LostConnection) represents that the FE was associated 444 with the CE at one point but lost the connection 446 + 5 (Unreachable) represents the FE deems this CE 447 unreachable. i.e the FE has tried over a period to connect 448 to it but has failed. 450 2. HAModeValues an unsigned char to specify selected HA mode. 451 Special values are: 453 + 0 (No HA Mode) The FE is not running in HA mode 455 + 1 (HA Mode - Cold Standby) The FE is in HA mode cold 456 Standby 458 + and 2 (HA Mode - Hot Standby) The FE is in HA mode hot 459 Standby 461 3. Statistics, a complex structure, representing the 462 communication statistics between the FE and CE. The 463 components are: 465 + RecvPackets representing the packet count received from 466 the CE 468 + RecvBytes representing the byte count received from the CE 470 + RecvErrPackets representing the erronous packets received 471 from the CE. This component logs badly formatted packets 472 as well as good packets sent to the FE by the CE to set 473 components whilst that CE is not the master. Erronous 474 packets are dropped(i.e not responded to). 476 + RecvErrBytes representing the RecvErrPackets byte count 477 received from the CE 479 + TxmitPackets representing the packet count transmitted to 480 the CE 482 + TxmitErrPackets representing the error packet count 483 transmitted to the CE. Typically these would be failures 484 due to communication. 486 + TxmitBytes representing the byte count transmitted to the 487 CE 489 + TxmitErrBytes representing the byte count of errors from 490 transmit to the CE 492 4. AllCEType, a complex structure constituing the CE ID, 493 Statistics and CEStatusType to reflect connection information 494 for one CE. Used in the AllCEs component array. 496 2. Appended two new components: 498 1. Read-only AllCEs to hold status for all CEs. AllCEs is an 499 Array of the AllCEType. 501 2. Read-write HAMode of type HAModeValues to carry the HA mode 502 used by the FE. 504 3. Added one additional Event, PrimaryCEChanged, reporting the new 505 master CEID when there is a mastership change. 507 Since no component from the FEPO v1 has been changed FEPO v1.1 508 retains backwards compatibility with CEs that know only version 1.0. 509 These CEs however cannot make use of the High Availability options 510 that the new FEPO provides. 512 4.2. FEPO processing 514 The FE's FEPO LFB version 1.1 AllCEs table contains all the CEIDs 515 that the FE may connect and associate with. The ordering of the CE 516 IDs in this table defines the priority order in which an FE will 517 connect to the CEs. This table is provisioned initially from the 518 configuration plane (FEM). In the pre-association phase, the first 519 CE (lowest table index) in the AllCEs table MUST be the first CE that 520 the FE will attempt to connect and associate with. If the FE fails 521 to connect and associate with the first listed CE, it will attempt to 522 connect to the second CE and so forth, and cycles back to the 523 beggining of the list until there is a successful association. The 524 FE MUST associate with at least one CE. Upon a successful 525 association, the FEPO's CEID component identifies the current 526 associated master CE. 528 While it would be much simpler to have the FE not respond to any 529 messages from a CE other than the master, in practise it has been 530 found to be useful to respond to queries and hearbeats from backup 531 CEs. For this reason, we allow backup CEs to issues queries to the 532 FE. Configuration messages (SET/DEL) from backup CEs MUST be dropped 533 by the FE and logged as received errors. 535 Asynchronous events that the master CE has subscribed to, as well as 536 heartbeats are sent to all associated-to CEs. Packet redirects 537 continue to be sent only to the master CE. The Heartbeat Interval, 538 the CEHB Policy and the FEHB Policy are global for all CEs(and 539 changed only by the master CE). 541 Figure 4 illustrates the state machine that facilitates connection 542 recovery with High Availability enabled. 544 FE tries to associate 545 +-->-----+ 546 | | 547 (CE changes master || | | 548 CE issues Teardown || +---+--------v----+ 549 Lost association) && | Pre-Association | 550 CE failover policy = 0 | (Association | 551 +------------>-->-->| in +<----+ 552 | | progress) | | 553 | | | | 554 | +--------+--------+ | 555 | CE Association | | CEFTI 556 | Response V | timer 557 | +------------------+ | expires 558 | |FE issue CEPrimaryDown ^ 559 | |FE issue PrimaryCEChanged ^ 560 | V | 561 +-+-----------+ +------+-----+ 562 | | (CE changes master || | Not | 563 | | CE issues Teardown || | Associated | 564 | | Lost association) && | +->-----------+ 565 | Associated | CE Failover Policy = 1 |(May |find first | 566 | | | Continue |associated v 567 | |-------->------->------>| Forwarding)|CE or retry | 568 | | Start CEFTI timer | |associating | 569 | | | |-<-----------+ 570 | | | | 571 +----+--------+ +-------+----+ 572 | | 573 ^ Found | associated CE 574 | or newly | associated CE 575 | V 576 | (Cancel CEFTI Timer) | 577 +_________________________________________+ 578 FE issue CEPrimaryDown event 579 FE issue PrimaryCEChanged event 581 Figure 4: FE State Machine considering HA 583 Once the FE has associated with a master CE it moves to the post- 584 association phase (Associated state). It is assumed that the master 585 CE will communicate with other CEs within the NE for the purpose of 586 synchronization via the CE-CE interface. The CE-CE interface is out 587 of scope for this document. An election result amongst CEs may 588 result in desire to change mastership to a different associated CE; 589 at which point current assumed master CE will instruct the FE to use 590 a different master CE. 592 FE CE#1 CE#2 ... CE#N 593 | | | | 594 | Asso Estb,Caps exchg | | | 595 1 |<-------------------->| | | 596 | | | | 597 | state update | | | 598 2 |<-------------------->| | | 599 | | | | 600 | Asso Estb,Caps exchg | | 601 3I|<--------------------------------->| | 602 ... ... ... ... 603 | Asso Estb,Caps exchg | 604 3N|<------------------------------------------>| 605 | | | | 606 4 |<-------------------->| | | 607 . . . . 608 4x|<-------------------->| | | 609 | FAILURE | | 610 | | | | 611 | Event Report (LastCEID changed) | | 612 5 |---------------------------------->|------->| 613 | Event Report (CE#2 is new master) | | 614 6 |---------------------------------->|------->| 615 | | | 616 7 |<--------------------------------->| | 617 . . . . 618 7x|<--------------------------------->| | 619 . . . . 621 Figure 5: CE Failover for Hot Standby 623 While in the post-association phase, if the CE Failover Policy is set 624 to 1 and HAMode set to 2 (HotStandby) then the FE, after succesfully 625 associating with the master CE, MUST attempt to connect and associate 626 with all the CEs that it is aware of. Figure 5 steps #1 and #2 627 illustrates the FE associating with CE#1 as the master and then 628 proceeding to steps #3I to #3N the association with backup CEs CE#2 629 to CE#N. If the FE fails to connect or associate with some CEs, the 630 FE MAY flag them as unreachable to avoid continuous attempts to 631 connect. The FE MAY retry to reassociate with unreachable CEs when 632 possible. 634 When the master CE for any reason is considered to be down, then the 635 FE MUST try to find the first associated CE from the list of all CEs 636 in a round-robin fashion. 638 If the FE is unable to find an associated FE in its list of CEs, then 639 it MUST attempt to connect and associate with the first from the list 640 of all CEs and continue in a round-robin fashion until it connects 641 and associates with a CE. 643 Once the FE selects an associated CE to use as the new master, the FE 644 issues a PrimaryCEDown Event Notification to all associated CEs to 645 notify them that the last primary CE went down (and what its identity 646 was); a second event PrimaryCEChanged identifying the new master CE 647 is sent as well to identify which CE the reporting FE considers to be 648 the new master. 650 In most High Availability architectures there exists the possibility 651 of split-brain. However, since in our setup the FE will never accept 652 any configuration messages from any other than the master CE, we 653 consider the FE as fenced against data corruption from the other CEs 654 that consider themselves as the master. The split-brain issue 655 becomes mostly a CE-CE communication problem which is considered to 656 be out of scope. 658 By virtue of having multiple CE connections, the FE switchover to a 659 new master CE will be relatively much faster. The overall effect is 660 improving the NE recovery time in case of communication failure or 661 faults of the master CE. This satisfies the requirement we set to 662 achieve. 664 5. IANA Considerations 666 XXX: This document updates an IANA registered FE Protocol object 667 Logical Functional Block (LFB). At minimal when it becomes RFC we 668 should update https://www.iana.org/assignments/forces/forces.xml 669 section on FEPO. 671 6. Security Considerations 673 Security consideration as defined in section 9 of [RFC5810] applies. 675 7. References 677 7.1. Normative References 679 [RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, 680 W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and 681 Control Element Separation (ForCES) Protocol 682 Specification", RFC 5810, March 2010. 684 7.2. Informative References 686 [RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation 687 of IP Control and Forwarding", RFC 3654, November 2003. 689 [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, 690 "Forwarding and Control Element Separation (ForCES) 691 Framework", RFC 3746, April 2004. 693 [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control 694 Element Separation (ForCES) Forwarding Element Model", 695 RFC 5812, March 2010. 697 Appendix A. New FEPO version 699 702 703 704 705 CEHBPolicyValues 706 707 The possible values of CE heartbeat policy 708 709 710 uchar 711 712 713 CEHBPolicy0 714 715 The CE will send heartbeats to the FE 716 every CEHDI timeout if no other messages 717 have been sent since. 718 719 720 721 CEHBPolicy1 722 723 The CE will not send heartbeats to the FE 724 725 726 727 728 729 730 FEHBPolicyValues 731 732 The possible values of FE heartbeat policy 733 734 735 uchar 736 737 738 FEHBPolicy0 739 740 The FE will not generate any heartbeats to the CE 741 742 743 744 FEHBPolicy1 745 746 The FE generates heartbeats to the CE every FEHI 747 if no other messages have been sent to the CE. 748 749 750 751 752 753 754 FERestartPolicyValues 755 756 The possible values of FE restart policy 757 758 759 uchar 760 761 762 FERestartPolicy0 763 764 The FE restart restats its state from scratch 765 766 767 768 769 770 771 HAModeValues 772 773 The possible values of HA modes 774 775 776 uchar 777 778 779 NoHA 780 781 The FE is not running in HA mode 782 783 784 785 ColdStandby 786 787 The FE is running in HA mode cold Standby 788 789 790 791 HotStandby 792 793 The FE is running in HA mode hot Standby 794 795 796 797 798 799 800 CEFailoverPolicyValues 801 802 The possible values of CE failover policy 803 804 805 uchar 806 807 808 CEFailoverPolicy0 809 810 The FE should stop functioning immediate and 811 transition to the FE OperDisable state 812 813 814 815 CEFailoverPolicy1 816 817 The FE should continue forwarding even without an 818 associated CE for CEFTI. The FE goes to FE 819 OperDisable when the CEFTI expires and no 820 association. Requires graceful restart support. 821 822 823 824 825 826 827 FEHACapab 828 829 The supported HA features 830 831 832 uchar 833 834 835 GracefullRestart 836 837 The FE supports Graceful Restart 838 839 840 841 HA 842 843 The FE supports HA 844 845 846 847 848 849 850 CEStatusType 851 Status values. Status for each CE 852 853 uchar 854 855 856 Disconnected 857 No connection attempt with the CE yet 858 859 860 861 Connected 862 The FE connection with the CE at the TML 863 has been completed 864 865 866 867 Associated 868 The FE has associated with the CE 869 870 871 872 IsMaster 873 The CE is the master (and associated) 874 876 877 878 LostConnection 879 The FE was associated with the CE but 880 lost the connection 881 882 883 884 Unreachable 885 The CE is deemed as unreachable by the FE 886 887 888 889 890 891 892 StatisticsType 893 Statistics Definition 894 895 896 RecvPackets 897 Packets Received 898 uint64 899 900 901 RecvErrPackets 902 Packets Received from CE with errors 903 904 uint64 905 906 907 RecvBytes 908 Bytes Received from CE 909 uint64 910 911 912 RecvErrBytes 913 Bytes Received from CE in Error 914 uint64 915 916 917 TxmitPackets 918 Packets Transmitted to CE 919 uint64 920 921 922 TxmitErrPackets 923 924 Packets Transmitted to CE that incurred 925 errors 926 927 uint64 928 929 930 TxmitBytes 931 Bytes Transmitted to CE 932 uint64 933 934 935 TxmitErrBytes 936 Bytes Transmitted to CE incurring errors 937 938 uint64 939 940 941 942 943 AllCEType 944 Table Type for AllCE component 945 946 947 CEID 948 ID of the CE 949 uint32 950 951 952 Statistics 953 Statistics per CE 954 StatisticsType 955 956 957 CEStatus 958 Status of the CE 959 CEStatusType 960 961 962 963 964 965 966 FEPO 967 968 The FE Protocol Object, with new CEHA 969 970 1.1 971 972 973 CurrentRunningVersion 974 Currently running ForCES version 975 uchar 976 977 978 FEID 979 Unicast FEID 980 uint32 981 982 983 MulticastFEIDs 984 985 the table of all multicast IDs 986 987 988 uint32 989 990 991 992 CEHBPolicy 993 994 The CE Heartbeat Policy 995 996 CEHBPolicyValues 997 998 999 CEHDI 1000 1001 The CE Heartbeat Dead Interval in millisecs 1002 1003 uint32 1004 1005 1006 FEHBPolicy 1007 1008 The FE Heartbeat Policy 1009 1010 FEHBPolicyValues 1011 1012 1013 FEHI 1014 1015 The FE Heartbeat Interval in millisecs 1016 1017 uint32 1018 1019 1020 CEID 1021 1022 The Primary CE this FE is associated with 1023 1024 uint32 1025 1026 1027 BackupCEs 1028 1029 The table of all backup CEs other than the 1030 primary 1031 1032 1033 uint32 1034 1035 1036 1037 CEFailoverPolicy 1038 1039 The CE Failover Policy 1040 1041 CEFailoverPolicyValues 1042 1043 1044 CEFTI 1045 1046 The CE Failover Timeout Interval in millisecs 1047 1048 uint32 1049 1050 1051 FERestartPolicy 1052 1053 The FE Restart Policy 1054 1055 FERestartPolicyValues 1056 1057 1058 LastCEID 1059 1060 The Primary CE this FE was last associated 1061 with 1062 1063 uint32 1064 1065 1066 HAMode 1067 1068 The HA mode used 1069 1070 HAModeValues 1071 1072 1073 AllCEs 1074 The table of all CEs 1075 1076 AllCEType 1077 1078 1079 1080 1081 1082 SupportableVersions 1083 1084 the table of ForCES versions that FE supports 1085 1086 1087 uchar 1088 1089 1090 1091 HACapabilities 1092 1093 the table of HA capabilities the FE supports 1094 1095 1096 FEHACapab 1097 1098 1099 1100 1101 1102 PrimaryCEDown 1103 1104 The primary CE has changed 1105 1106 1107 LastCEID 1108 1109 1110 1111 1112 LastCEID 1113 1114 1115 1116 1117 PrimaryCEChanged 1118 A New primary CE has been selected 1119 1120 1121 CEID 1122 1123 1124 1125 1126 CEID 1127 1128 1129 1130 1131 1132 1133 1135 Authors' Addresses 1137 Kentaro Ogawa 1138 NTT Corporation 1139 3-9-11 Midori-cho 1140 Musashino-shi, Tokyo 180-8585 1141 Japan 1143 Email: ogawa.kentaro@lab.ntt.co.jp 1145 Weiming Wang 1146 Zhejiang Gongshang University 1147 149 Jiaogong Road 1148 Hangzhou 310035 1149 P.R.China 1151 Phone: +86-571-88057712 1152 Email: wmwang@mail.zjgsu.edu.cn 1153 Evangelos Haleplidis 1154 University of Patras 1155 Panepistimioupoli Patron 1156 Patras 26504 1157 Greece 1159 Email: ehalep@ece.upatras.gr 1161 Jamal Hadi Salim 1162 Mojatatu Networks 1163 Suite 400, 303 Moodie Dr. 1164 Ottawa, Ontario K2H 9R4 1165 Canada 1167 Email: hadi@mojatatu.com