idnits 2.17.1 draft-ietf-rtgwg-uloop-delay-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 12, 2017) is 2328 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO10589' == Outdated reference: A later version (-10) exists of draft-ietf-rtgwg-backoff-algo-06 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Routing Area Working Group S. Litkowski 3 Internet-Draft B. Decraene 4 Intended status: Standards Track Orange 5 Expires: May 16, 2018 C. Filsfils 6 Cisco Systems 7 P. Francois 8 Individual 9 November 12, 2017 11 Micro-loop prevention by introducing a local convergence delay 12 draft-ietf-rtgwg-uloop-delay-09 14 Abstract 16 This document describes a mechanism for link-state routing protocols 17 to prevent local transient forwarding loops in case of link failure. 18 This mechanism proposes a two-step convergence by introducing a delay 19 between the convergence of the node adjacent to the topology change 20 and the network wide convergence. 22 As this mechanism delays the IGP convergence it may only be used for 23 planned maintenance or when fast reroute protects the traffic between 24 the link failure time and the IGP convergence. 26 The proposed mechanism is limited to the link down event in order to 27 keep the mechanism simple. 29 Simulations using real network topologies have been performed and 30 show that local loops are a significant portion (>50%) of the total 31 forwarding loops. 33 Requirements Language 35 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 36 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 37 document are to be interpreted as described in [RFC2119]. 39 Status of This Memo 41 This Internet-Draft is submitted in full conformance with the 42 provisions of BCP 78 and BCP 79. 44 Internet-Drafts are working documents of the Internet Engineering 45 Task Force (IETF). Note that other groups may also distribute 46 working documents as Internet-Drafts. The list of current Internet- 47 Drafts is at https://datatracker.ietf.org/drafts/current/. 49 Internet-Drafts are draft documents valid for a maximum of six months 50 and may be updated, replaced, or obsoleted by other documents at any 51 time. It is inappropriate to use Internet-Drafts as reference 52 material or to cite them other than as "work in progress." 54 This Internet-Draft will expire on May 16, 2018. 56 Copyright Notice 58 Copyright (c) 2017 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (https://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with respect 66 to this document. Code Components extracted from this document must 67 include Simplified BSD License text as described in Section 4.e of 68 the Trust Legal Provisions and are provided without warranty as 69 described in the Simplified BSD License. 71 Table of Contents 73 1. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . 3 74 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 75 3. Transient forwarding loops side effects . . . . . . . . . . . 4 76 3.1. Fast reroute inefficiency . . . . . . . . . . . . . . . . 4 77 3.2. Network congestion . . . . . . . . . . . . . . . . . . . 7 78 4. Overview of the solution . . . . . . . . . . . . . . . . . . 7 79 5. Specification . . . . . . . . . . . . . . . . . . . . . . . . 8 80 5.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 8 81 5.2. Regular IGP reaction . . . . . . . . . . . . . . . . . . 8 82 5.3. Local events . . . . . . . . . . . . . . . . . . . . . . 9 83 5.4. Local delay for link down . . . . . . . . . . . . . . . . 10 84 6. Applicability . . . . . . . . . . . . . . . . . . . . . . . . 10 85 6.1. Applicable case: local loops . . . . . . . . . . . . . . 10 86 6.2. Non applicable case: remote loops . . . . . . . . . . . . 11 87 7. Simulations . . . . . . . . . . . . . . . . . . . . . . . . . 11 88 8. Deployment considerations . . . . . . . . . . . . . . . . . . 12 89 9. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 13 90 9.1. Local link down . . . . . . . . . . . . . . . . . . . . . 14 91 9.2. Local and remote event . . . . . . . . . . . . . . . . . 18 92 9.3. Aborting local delay . . . . . . . . . . . . . . . . . . 19 93 10. Comparison with other solutions . . . . . . . . . . . . . . . 23 94 10.1. PLSN . . . . . . . . . . . . . . . . . . . . . . . . . . 23 95 10.2. OFIB . . . . . . . . . . . . . . . . . . . . . . . . . . 23 96 11. Implementation Status . . . . . . . . . . . . . . . . . . . . 24 97 12. Security Considerations . . . . . . . . . . . . . . . . . . . 25 98 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 99 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 100 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 26 101 15.1. Normative References . . . . . . . . . . . . . . . . . . 26 102 15.2. Informative References . . . . . . . . . . . . . . . . . 26 103 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 105 1. Acronyms 107 FIB: Forwarding Information Base 109 FRR: Fast ReRoute 111 IGP: Interior Gateway Protocol 113 LFA: Loop Free Alternate 115 LSA: Link State Advertisement 117 LSP: Link State Packet 119 MRT: Maximum Redundant Trees 121 OFIB: Ordered FIB 123 PLSN: Path Locking via Safe Neighbor 125 RIB: Routing Information Base 127 RLFA: Remote Loop Free Alternate 129 SPF: Shortest Path First 131 TTL: Time To Live 133 2. Introduction 135 Micro-forwarding loops and some potential solutions are well 136 described in [RFC5715]. This document describes a simple targeted 137 mechanism that prevents micro-loops that are local to the failure. 138 Based on network analysis, local failures make up a significant 139 portion of the micro-forwarding loops. A simple and easily 140 deployable solution for these local micro-loops is critical because 141 these local loops cause some traffic loss after a fast-reroute 142 alternate has been used (see Section 3.1). 144 Consider the case in Figure 1 where S does not have an LFA (Loop Free 145 Alternate) to protect its traffic to D when the S-D link fails. That 146 means that all non-D neighbors of S on the topology will send to S 147 any traffic destined to D; if a neighbor did not, then that neighbor 148 would be loop-free. Regardless of the advanced fast-reroute (FRR) 149 technique used, when S converges to the new topology, it will send 150 its traffic to a neighbor that was not loop-free and thus cause a 151 local micro-loop. The deployment of advanced fast-reroute techniques 152 motivates this simple router-local mechanism to solve this targeted 153 problem. This solution can work with the various techniques 154 described in [RFC5715]. 156 D ------ C 157 | | 158 | | 5 159 | | 160 S ------ B 162 Figure 1 164 In the Figure 1, all links have a metric of 1 except B-C which has a 165 metric of 5. When S-D fails, a transient forwarding loop may appear 166 between S and B if S updates its forwarding entry to D before B does. 168 3. Transient forwarding loops side effects 170 Even if they are very limited in duration, transient forwarding loops 171 may cause significant network damage. 173 3.1. Fast reroute inefficiency 175 D 176 1 | 177 | 1 178 A ------ B 179 | | ^ 180 10 | | 5 | T 181 | | | 182 E--------C 183 | 1 184 1 | 185 S 187 Figure 2 - RSVP-TE FRR case 189 In the Figure 2, we consider an IP/LDP routed network. An RSVP-TE 190 tunnel T, provisioned on C and terminating on B, is used to protect 191 the traffic against C-B link failure (the IGP shortcut feature, 192 defined in [RFC3906], is activated on C ). The primary path of T is 193 C->B and FRR is activated on T providing an FRR bypass or detour 194 using path C->E->A->B. On router C, the next hop to D is the tunnel 195 T thanks to the IGP shortcut. When C-B link fails: 197 1. C detects the failure, and updates the tunnel path using a 198 preprogrammed FRR path. The traffic path from S to D becomes: 199 S->E->C->E->A->B->A->D. 201 2. In parallel, on router C, both the IGP convergence and the TE 202 tunnel convergence (tunnel path recomputation) are occurring: 204 * The Tunnel T path is recomputed and now uses C->E->A->B. 206 * The IGP path to D is recomputed and now uses C->E->A->D. 208 3. On C, the tail-end of the TE tunnel (router B) is no longer on 209 the shortest-path tree (SPT) to D, so C does not continue to 210 encapsulate the traffic to D using the tunnel T and updates its 211 forwarding entry to D using the nexthop E. 213 If C updates its forwarding entry to D before router E, there would 214 be a transient forwarding loop between C and E until E has converged. 216 The table 1 below describes a theoretical sequence of events 217 happening when the B-C link fails. This theoretical sequence of 218 events should only be read as an example. 220 +-----------+------------+------------------+-----------------------+ 221 | Network | Time | Router C events | Router E events | 222 | condition | | | | 223 +-----------+------------+------------------+-----------------------+ 224 | S->D | | | | 225 | Traffic | | | | 226 | OK | | | | 227 | | | | | 228 | S->D | t0 | Link B-C fails | Link B-C fails | 229 | Traffic | | | | 230 | lost | | | | 231 | | | | | 232 | | t0+20msec | C detects the | | 233 | | | failure | | 234 | | | | | 235 | S->D | t0+40msec | C activates FRR | | 236 | Traffic | | | | 237 | OK | | | | 238 | | | | | 239 | | t0+50msec | C updates its | | 240 | | | local LSP/LSA | | 241 | | | | | 242 | | t0+60msec | C schedules SPF | | 243 | | | (100ms) | | 244 | | | | | 245 | | t0+70msec | C floods its | | 246 | | | local updated | | 247 | | | LSP/LSA | | 248 | | | | | 249 | | t0+87msec | | E receives LSP/LSA | 250 | | | | from C and schedules | 251 | | | | SPF (100ms) | 252 | | | | | 253 | | t0+117msec | | E floods LSP/LSA from | 254 | | | | C | 255 | | | | | 256 | | t0+160msec | C computes SPF | | 257 | | | | | 258 | | t0+165msec | C starts | | 259 | | | updating its | | 260 | | | RIB/FIB | | 261 | | | | | 262 | | t0+193msec | | E computes SPF | 263 | | | | | 264 | | t0+199msec | | E starts updating its | 265 | | | | RIB/FIB | 266 | | | | | 267 | S->D | t0+255msec | C updates its | | 268 | Traffic | | RIB/FIB for D | | 269 | lost | | | | 270 | | | | | 271 | | t0+340msec | C convergence | | 272 | | | ends | | 273 | | | | | 274 | S->D | t0+443msec | | E updates its RIB/FIB | 275 | Traffic | | | for D | 276 | OK | | | | 277 | | | | | 278 | | t0+470msec | | E convergence ends | 279 +-----------+------------+------------------+-----------------------+ 281 Table 1 - Route computation event time scale 283 The issue described here is completely independent of the fast- 284 reroute mechanism involved (TE FRR, LFA/rLFA, MRT ...) when the 285 primary path uses hop-by-hop routing. The protection enabled by 286 fast-reroute is working perfectly, but ensures a protection, by 287 definition, only until the PLR has converged (as soon as the PLR has 288 converged, it replaces its FRR path by a new primary path). When 289 implementing FRR, a service provider wants to guarantee a very 290 limited loss of connectivity time. The previous example shows that 291 the benefit of FRR may be completely lost due to a transient 292 forwarding loop appearing when PLR has converged. Delaying FIB 293 updates after the IGP convergence may allow to keep the fast-reroute 294 path until the neighbors have converged and preserves the customer 295 traffic. 297 3.2. Network congestion 299 1 300 D ------ C 301 | | 302 1 | | 5 303 | | 304 A -- S ------ B 305 / | 1 306 F E 308 Figure 3 310 In the figure above, as presented in Section 2, when the link S-D 311 fails, a transient forwarding loop may appear between S and B for 312 destination D. The traffic on the S-B link will constantly increase 313 due to the looping traffic to D. Depending on the TTL of the 314 packets, the traffic rate destined to D, and the bandwidth of the 315 link, the S-B link may become congested in a few hundreds of 316 milliseconds and will stay congested until the loop is eliminated. 318 The congestion introduced by transient forwarding loops is 319 problematic as it can affect traffic that is not directly affected by 320 the failing network component. In the example, the congestion of the 321 S-B link will impact some customer traffic that is not directly 322 affected by the failure: e.g. A to B, F to B, E to B. Class of 323 service may mitigate the congestion for some traffic. However, some 324 traffic not directly affected by the failure will still be dropped as 325 a router is not able to distinguish the looping traffic from the 326 normally forwarded traffic. 328 4. Overview of the solution 330 This document defines a two-step convergence initiated by the router 331 detecting a failure and advertising the topological changes in the 332 IGP. This introduces a delay between network-wide convergence and 333 the convergence of the local router. 335 The proposed solution is limited to local link down events in order 336 to keep the solution simple. 338 This ordered convergence is similar to the ordered FIB proposed 339 defined in [RFC6976], but it is limited to only a "one hop" distance. 340 As a consequence, it is more simple and becomes a local-only feature 341 that does not require interoperability. This benefit comes with the 342 limitation of eliminating transient forwarding loops involving the 343 local router only. The proposed mechanism also reuses some concepts 344 described in [I-D.ietf-rtgwg-microloop-analysis]. 346 5. Specification 348 5.1. Definitions 350 This document will refer to the following existing IGP timers. These 351 timers may be standardized or implemented as a vendor specific local 352 feature. 354 o LSP_GEN_TIMER: The delay between two consecutives local LSP/LSA 355 generation. From an operational point of view, this delay is 356 usually tuned to batch multiple local events in one single local 357 LSP/LSA update. In IS-IS, this timer is defined as 358 minimumLSPGenerationInterval in [ISO10589]. In OSPF version 2, 359 this timer is defined as MinLSInterval in [RFC2328]. It is often 360 associated with a vendor specific damping mechanism to slow down 361 reactions by incrementing the timer when multiple consecutive 362 events are detected. 364 o SPF_DELAY: The delay between the first IGP event triggering a new 365 routing table computation and the start of that routing table 366 computation. It is often associated with a damping mechanism to 367 slow down reactions by incrementing the timer when the IGP becomes 368 unstable. As an example, [I-D.ietf-rtgwg-backoff-algo] defines a 369 standard SPF (Shortest Path First) delay algorithm. 371 This document introduces the following new timer: 373 o ULOOP_DELAY_DOWN_TIMER: used to slow down the local node 374 convergence in case of link down events. 376 5.2. Regular IGP reaction 378 Upon a change of the status of an adjacency/link, the regular IGP 379 convergence behavior of the router advertising the event involves the 380 following main steps: 382 1. IGP is notified of the Up/Down event. 384 2. The IGP processes the notification and postpones the reaction for 385 LSP_GEN_TIMER msec. 387 3. Upon LSP_GEN_TIMER expiration, the IGP updates its LSP/LSA and 388 floods it. 390 4. The SPF computation is scheduled in SPF_DELAY msec. 392 5. Upon SPF_DELAY timer expiration, the SPF is computed, then the 393 RIB and FIB are updated. 395 5.3. Local events 397 The mechanism described in this document assumes that there has been 398 a single link failure as seen by the IGP area/level. If this 399 assumption is violated (e.g. multiple links or nodes failed), then 400 regular IP convergence must be applied (as described in Section 5.2). 402 To determine if the mechanism can be applicable or not, an 403 implementation SHOULD implement logic to correlate the protocol 404 messages (LSP/LSA) received during the SPF scheduling period in order 405 to determine the topology changes that occured. This is necessary as 406 multiple protocol messages may describe the same topology change and 407 a single protocol message may describe multiple topology changes. As 408 a consequence, determining a particular topology change MUST be 409 independent of the order of reception of those protocol messages. 410 How the logic works is left to the implementation. 412 Using this logic, if an implementation determines that the associated 413 topology change is a single local link failure, then the router MAY 414 use the mechanism described in this document, otherwise the regular 415 IP convergence MUST be used. 417 Example: 419 +--- E ----+--------+ 420 | | | 421 A ---- B -------- C ------ D 423 Figure 4 425 Let router B be the computing router when the link B-C fails. B 426 updates its local LSP/LSA describing the link B->C as down, C does 427 the same, and both start flooding their updated LSP/LSAs. During the 428 SPF_DELAY period, B and C learn all the LSPs/LSAs to consider. B 429 sees that C is flooding an advertisement that indicates that a link 430 is down, and B is the other end of that link. B determines that B 431 and C are describing the same single event. Since B receives no 432 other changes, B can determine that this is a local link failure and 433 may decide to activate the mechanism described in this document. 435 5.4. Local delay for link down 437 Upon an adjacency/link down event, this document introduces a change 438 in step 5 (Section 5.2) in order to delay the local convergence 439 compared to the network wide convergence. The new step 5 is 440 described below: 442 5. Upon SPF_DELAY timer expiration, the SPF is computed. If the 443 condition of a single local link-down event has been met, then an 444 update of the RIB and the FIB MUST be delayed for 445 ULOOP_DELAY_DOWN_TIMER msecs. Otherwise, the RIB and FIB SHOULD 446 be updated immediately. 448 If a new convergence occurs while ULOOP_DELAY_DOWN_TIMER is running, 449 ULOOP_DELAY_DOWN_TIMER is stopped and the RIB/FIB SHOULD be updated 450 as part of the new convergence event. 452 As a result of this addition, routers local to the failure will 453 converge slower than remote routers. Hence it SHOULD only be done 454 for a non-urgent convergence, such as for administrative de- 455 activation (maintenance) or when the traffic is protected by fast- 456 reroute. 458 6. Applicability 460 As previously stated, this mechanism only avoids the forwarding loops 461 on the links between the node local to the failure and its neighbors. 462 Forwarding loops may still occur on other links. 464 6.1. Applicable case: local loops 466 A ------ B ----- E 467 | / | 468 | / | 469 G---D------------C F All the links have a metric of 1 471 Figure 5 473 Let us consider the traffic from G to F. The primary path is 474 G->D->C->E->F. When link C-E fails, if C updates its forwarding 475 entry for F before D, a transient loop occurs. This is sub-optimal 476 as C has FRR enabled and it breaks the FRR forwarding while all 477 upstream routers are still forwarding the traffic to itself. 479 By implementing the mechanism defined in this document on C, when the 480 C-E link fails, C delays the update of its forwarding entry to F, in 481 order to allow some time for D to converge. FRR on C keeps 482 protecting the traffic during this period. When the timer expires on 483 C, its forwarding entry to F is updated. There is no transient 484 forwarding loop on the link C-D. 486 6.2. Non applicable case: remote loops 488 A ------ B ----- E --- H 489 | | 490 | | 491 G---D--------C ------F --- J ---- K 493 All the links have a metric of 1 except BE=15 495 Figure 6 497 Let us consider the traffic from G to K. The primary path is 498 G->D->C->F->J->K. When the C-F link fails, if C updates its 499 forwarding entry to K before D, a transient loop occurs between C and 500 D. 502 By implementing the mechanism defined in this document on C, when the 503 link C-F fails, C delays the update of its forwarding entry to K, 504 allowing time for D to converge. When the timer expires on C, its 505 forwarding entry to F is updated. There is no transient forwarding 506 loop between C and D. However, a transient forwarding loop may still 507 occur between D and A. In this scenario, this mechanism is not 508 enough to address all the possible forwarding loops. However, it 509 does not create additional traffic loss. Besides, in some cases 510 -such as when the nodes update their FIB in the following order C, A, 511 D, for example because the router A is quicker than D to converge- 512 the mechanism may still avoid the forwarding loop that would have 513 otherwise occurred. 515 7. Simulations 517 Simulations have been run on multiple service provider topologies. 519 +----------+------+ 520 | Topology | Gain | 521 +----------+------+ 522 | T1 | 71% | 523 | T2 | 81% | 524 | T3 | 62% | 525 | T4 | 50% | 526 | T5 | 70% | 527 | T6 | 70% | 528 | T7 | 59% | 529 | T8 | 77% | 530 +----------+------+ 532 Table 2 - Number of Repair/Dst that may loop 534 We evaluated the efficiency of the mechanism on eight different 535 service provider topologies (different network size, design). The 536 benefit is displayed in the table above. The benefit is evaluated as 537 follows: 539 o We consider a tuple (link A-B, destination D, PLR S, backup 540 nexthop N) as a loop if upon link A-B failure, the flow from a 541 router S upstream from A (A could be considered as PLR also) to D 542 may loop due to convergence time difference between S and one of 543 his neighbors N. 545 o We evaluate the number of potential loop tuples in normal 546 conditions. 548 o We evaluate the number of potential loop tuples using the same 549 topological input but taking into account that S converges after 550 N. 552 o The gain is how many loops (both remote and local) we succeed to 553 suppress. 555 On topology 1, 71% of the transient forwarding loops created by the 556 failure of any link are prevented by implementing the local delay. 557 The analysis shows that all local loops are prevented and only remote 558 loops remain. 560 8. Deployment considerations 562 Transient forwarding loops have the following drawbacks: 564 o They limit FRR efficiency: even if FRR is activated within 50msec, 565 as soon as PLR has converged, the traffic may be affected by a 566 transient loop. 568 o They may impact traffic not directly affected by the failure (due 569 to link congestion). 571 This local delay proposal is a transient forwarding loop avoidance 572 mechanism (like OFIB). Even if it only addresses local transient 573 loops, the efficiency versus complexity comparison of the mechanism 574 makes it a good solution. It is also incrementally deployable with 575 incremental benefits, which makes it an attractive option both for 576 vendors to implement and service providers to deploy. Delaying the 577 convergence time is not an issue if we consider that the traffic is 578 protected during the convergence. 580 The ULOOP_DELAY_DOWN_TIMER value should be set according to the 581 maximum IGP convergence time observed in the network (usually 582 observed in the slowest node). 584 The proposed mechanism is limited to link down events. When a link 585 goes down, it eventually goes back up. As a consequence, with the 586 proposed mechanism deployed, only the link down event will be 587 protected against transient forwarding loops while the link up event 588 will not. If the operator wants to limit the impact of the transient 589 forwarding loops during the link up event, it should take care of 590 using specific procedures to bring the link back online. As 591 examples, the operator can decide to put back the link online out of 592 business hours or it can use some incremental metric changes to 593 prevent loops (as proposed in [RFC5715]). 595 9. Examples 597 We will consider the following figure for the associated examples : 599 D 600 1 | F----X 601 | 1 | 602 A ------ B 603 | | 604 10 | | 5 605 | | 606 E--------C 607 | 1 608 1 | 609 S 611 Figure 7 613 The network above is considered to have a convergence time about 1 614 second, so ULOOP_DELAY_DOWN_TIMER will be adjusted to this value. We 615 also consider that FRR is running on each node. 617 9.1. Local link down 619 The table 3 describes the events and associated timing that happen on 620 router C and E when link B-C goes down. It is based on a theoretical 621 sequence of event that should only been read as an example. As C 622 detects a single local event corresponding to a link down (its LSP + 623 LSP from B received), it applies the local delay down behavior and no 624 microloop is formed. 626 +-----------+-------------+------------------+----------------------+ 627 | Network | Time | Router C events | Router E events | 628 | condition | | | | 629 +-----------+-------------+------------------+----------------------+ 630 | S->D | | | | 631 | Traffic | | | | 632 | OK | | | | 633 | | | | | 634 | S->D | t0 | Link B-C fails | Link B-C fails | 635 | Traffic | | | | 636 | lost | | | | 637 | | | | | 638 | | t0+20msec | C detects the | | 639 | | | failure | | 640 | | | | | 641 | S->D | t0+40msec | C activates FRR | | 642 | Traffic | | | | 643 | OK | | | | 644 | | | | | 645 | | t0+50msec | C updates its | | 646 | | | local LSP/LSA | | 647 | | | | | 648 | | t0+60msec | C schedules SPF | | 649 | | | (100ms) | | 650 | | | | | 651 | | t0+67msec | C receives | | 652 | | | LSP/LSA from B | | 653 | | | | | 654 | | t0+70msec | C floods its | | 655 | | | local updated | | 656 | | | LSP/LSA | | 657 | | | | | 658 | | t0+87msec | | E receives LSP/LSA | 659 | | | | from C and schedules | 660 | | | | SPF (100ms) | 661 | | | | | 662 | | t0+117msec | | E floods LSP/LSA | 663 | | | | from C | 664 | | | | | 665 | | t0+160msec | C computes SPF | | 666 | | | | | 667 | | t0+165msec | C delays its | | 668 | | | RIB/FIB update | | 669 | | | (1 sec) | | 670 | | | | | 671 | | t0+193msec | | E computes SPF | 672 | | | | | 673 | | t0+199msec | | E starts updating | 674 | | | | its RIB/FIB | 675 | | | | | 676 | | t0+443msec | | E updates its | 677 | | | | RIB/FIB for D | 678 | | | | | 679 | | t0+470msec | | E convergence ends | 680 | | | | | 681 | | t0+1165msec | C starts | | 682 | | | updating its | | 683 | | | RIB/FIB | | 684 | | | | | 685 | | t0+1255msec | C updates its | | 686 | | | RIB/FIB for D | | 687 | | | | | 688 | | t0+1340msec | C convergence | | 689 | | | ends | | 690 +-----------+-------------+------------------+----------------------+ 692 Table 3 - Route computation event time scale 694 Similarly, upon B-C link down event, if LSP/LSA from B is received 695 before C detects the link failure, C will apply the route update 696 delay if the local detection is part of the same SPF run. The table 697 4 describes the associated theoretical sequence of events. It should 698 only been read as an example. 700 +-----------+-------------+------------------+----------------------+ 701 | Network | Time | Router C events | Router E events | 702 | condition | | | | 703 +-----------+-------------+------------------+----------------------+ 704 | S->D | | | | 705 | Traffic | | | | 706 | OK | | | | 707 | | | | | 708 | S->D | t0 | Link B-C fails | Link B-C fails | 709 | Traffic | | | | 710 | lost | | | | 711 | | | | | 712 | | t0+32msec | C receives | | 713 | | | LSP/LSA from B | | 714 | | | | | 715 | | t0+33msec | C schedules SPF | | 716 | | | (100ms) | | 717 | | | | | 718 | | t0+50msec | C detects the | | 719 | | | failure | | 720 | | | | | 721 | S->D | t0+55msec | C activates FRR | | 722 | Traffic | | | | 723 | OK | | | | 724 | | | | | 725 | | t0+55msec | C updates its | | 726 | | | local LSP/LSA | | 727 | | | | | 728 | | t0+70msec | C floods its | | 729 | | | local updated | | 730 | | | LSP/LSA | | 731 | | | | | 732 | | t0+87msec | | E receives LSP/LSA | 733 | | | | from C and schedules | 734 | | | | SPF (100ms) | 735 | | | | | 736 | | t0+117msec | | E floods LSP/LSA | 737 | | | | from C | 738 | | | | | 739 | | t0+160msec | C computes SPF | | 740 | | | | | 741 | | t0+165msec | C delays its | | 742 | | | RIB/FIB update | | 743 | | | (1 sec) | | 744 | | | | | 745 | | t0+193msec | | E computes SPF | 746 | | | | | 747 | | t0+199msec | | E starts updating | 748 | | | | its RIB/FIB | 749 | | | | | 750 | | t0+443msec | | E updates its | 751 | | | | RIB/FIB for D | 752 | | | | | 753 | | t0+470msec | | E convergence ends | 754 | | | | | 755 | | t0+1165msec | C starts | | 756 | | | updating its | | 757 | | | RIB/FIB | | 758 | | | | | 759 | | t0+1255msec | C updates its | | 760 | | | RIB/FIB for D | | 761 | | | | | 762 | | t0+1340msec | C convergence | | 763 | | | ends | | 764 +-----------+-------------+------------------+----------------------+ 766 Table 4 - Route computation event time scale 768 9.2. Local and remote event 770 The table 5 describes the events and associated timing that happen on 771 router C and E when link B-C goes down, in addition F-X link will 772 fail in the same time window. C will not apply the local delay 773 because a non local topology change is also received. The table 5 is 774 based on a theoretical sequence of event that should only been read 775 as an example. 777 +-----------+------------+-----------------+------------------------+ 778 | Network | Time | Router C events | Router E events | 779 | condition | | | | 780 +-----------+------------+-----------------+------------------------+ 781 | S->D | | | | 782 | Traffic | | | | 783 | OK | | | | 784 | | | | | 785 | S->D | t0 | Link B-C fails | Link B-C fails | 786 | Traffic | | | | 787 | lost | | | | 788 | | | | | 789 | | t0+20msec | C detects the | | 790 | | | failure | | 791 | | | | | 792 | | t0+36msec | Link F-X fails | Link F-X fails | 793 | | | | | 794 | S->D | t0+40msec | C activates FRR | | 795 | Traffic | | | | 796 | OK | | | | 797 | | | | | 798 | | t0+50msec | C updates its | | 799 | | | local LSP/LSA | | 800 | | | | | 801 | | t0+54msec | C receives | | 802 | | | LSP/LSA from F | | 803 | | | and floods it | | 804 | | | | | 805 | | t0+60msec | C schedules SPF | | 806 | | | (100ms) | | 807 | | | | | 808 | | t0+67msec | C receives | | 809 | | | LSP/LSA from B | | 810 | | | | | 811 | | t0+69msec | | E receives LSP/LSA | 812 | | | | from F, floods it and | 813 | | | | schedules SPF (100ms) | 814 | | | | | 815 | | t0+70msec | C floods its | | 816 | | | local updated | | 817 | | | LSP/LSA | | 818 | | | | | 819 | | t0+87msec | | E receives LSP/LSA | 820 | | | | from C | 821 | | | | | 822 | | t0+117msec | | E floods LSP/LSA from | 823 | | | | C | 824 | | | | | 825 | | t0+160msec | C computes SPF | | 826 | | | | | 827 | | t0+165msec | C starts | | 828 | | | updating its | | 829 | | | RIB/FIB (NO | | 830 | | | DELAY) | | 831 | | | | | 832 | | t0+170msec | | E computes SPF | 833 | | | | | 834 | | t0+173msec | | E starts updating its | 835 | | | | RIB/FIB | 836 | | | | | 837 | S->D | t0+365msec | C updates its | | 838 | Traffic | | RIB/FIB for D | | 839 | lost | | | | 840 | | | | | 841 | S->D | t0+443msec | | E updates its RIB/FIB | 842 | Traffic | | | for D | 843 | OK | | | | 844 | | | | | 845 | | t0+450msec | C convergence | | 846 | | | ends | | 847 | | | | | 848 | | t0+470msec | | E convergence ends | 849 | | | | | 850 +-----------+------------+-----------------+------------------------+ 852 Table 5 - Route computation event time scale 854 9.3. Aborting local delay 856 The table 6 describes the events and associated timing that happen on 857 router C and E when link B-C goes down. In addition, we consider 858 what happens when F-X link fails during local delay of the FIB 859 update. C will first apply the local delay, but when the new event 860 happens, it will fall back to the standard convergence mechanism 861 without further delaying route insertion. In this example, we 862 consider a ULOOP_DELAY_DOWN_TIMER configured to 2 seconds. The table 863 6 is based on a theoretical sequence of event that should only been 864 read as an example. 866 +-----------+------------+-------------------+----------------------+ 867 | Network | Time | Router C events | Router E events | 868 | condition | | | | 869 +-----------+------------+-------------------+----------------------+ 870 | S->D | | | | 871 | Traffic | | | | 872 | OK | | | | 873 | | | | | 874 | S->D | t0 | Link B-C fails | Link B-C fails | 875 | Traffic | | | | 876 | lost | | | | 877 | | | | | 878 | | t0+20msec | C detects the | | 879 | | | failure | | 880 | | | | | 881 | S->D | t0+40msec | C activates FRR | | 882 | Traffic | | | | 883 | OK | | | | 884 | | | | | 885 | | t0+50msec | C updates its | | 886 | | | local LSP/LSA | | 887 | | | | | 888 | | t0+60msec | C schedules SPF | | 889 | | | (100ms) | | 890 | | | | | 891 | | t0+67msec | C receives | | 892 | | | LSP/LSA from B | | 893 | | | | | 894 | | t0+70msec | C floods its | | 895 | | | local updated | | 896 | | | LSP/LSA | | 897 | | | | | 898 | | t0+87msec | | E receives LSP/LSA | 899 | | | | from C and schedules | 900 | | | | SPF (100ms) | 901 | | | | | 902 | | t0+117msec | | E floods LSP/LSA | 903 | | | | from C | 904 | | | | | 905 | | t0+160msec | C computes SPF | | 906 | | | | | 907 | | t0+165msec | C delays its | | 908 | | | RIB/FIB update (2 | | 909 | | | sec) | | 910 | | | | | 911 | | t0+193msec | | E computes SPF | 912 | | | | | 913 | | t0+199msec | | E starts updating | 914 | | | | its RIB/FIB | 915 | | | | | 916 | | t0+254msec | Link F-X fails | Link F-X fails | 917 | | | | | 918 | | t0+300msec | C receives | | 919 | | | LSP/LSA from F | | 920 | | | and floods it | | 921 | | | | | 922 | | t0+303msec | C schedules SPF | | 923 | | | (200ms) | | 924 | | | | | 925 | | t0+312msec | E receives | | 926 | | | LSP/LSA from F | | 927 | | | and floods it | | 928 | | | | | 929 | | t0+313msec | E schedules SPF | | 930 | | | (200ms) | | 931 | | | | | 932 | | t0+502msec | C computes SPF | | 933 | | | | | 934 | | t0+505msec | C starts updating | | 935 | | | its RIB/FIB (NO | | 936 | | | DELAY) | | 937 | | | | | 938 | | t0+514msec | | E computes SPF | 939 | | | | | 940 | | t0+519msec | | E starts updating | 941 | | | | its RIB/FIB | 942 | | | | | 943 | S->D | t0+659msec | C updates its | | 944 | Traffic | | RIB/FIB for D | | 945 | lost | | | | 946 | | | | | 947 | S->D | t0+778msec | | E updates its | 948 | Traffic | | | RIB/FIB for D | 949 | OK | | | | 950 | | | | | 951 | | t0+781msec | C convergence | | 952 | | | ends | | 953 | | | | | 954 | | t0+810msec | | E convergence ends | 955 +-----------+------------+-------------------+----------------------+ 957 Table 6 - Route computation event time scale 959 10. Comparison with other solutions 961 As stated in Section 4, the proposed solution reuses some concepts 962 already introduced by other IETF proposals but tries to find a 963 tradeoff between efficiency and simplicity. This section tries to 964 compare behaviors of the solutions. 966 10.1. PLSN 968 PLSN ([I-D.ietf-rtgwg-microloop-analysis]) describes a mechanism 969 where each node in the network tries to avoid transient forwarding 970 loops upon a topology change by always keeping traffic on a loop-free 971 path for a defined duration (locked path to a safe neighbor). The 972 locked path may be the new primary nexthop, another neighbor, or the 973 old primary nexthop depending how the safety condition is satisfied. 975 PLSN does not solve all transient forwarding loops (see 976 [I-D.ietf-rtgwg-microloop-analysis] Section 4 for more details). 978 Our solution reuses some concept of PLSN but in a more simple 979 fashion: 981 o PLSN has three different behaviors: keep using old nexthop, use 982 new primary nexthop if it is safe, or use another safe nexthop, 983 while the proposed solution only has one: keep using the current 984 nexthop (old primary, or already activated FRR path). 986 o PLSN may cause some damage while using a safe nexthop which is not 987 the new primary nexthop in case the new safe nexthop does not 988 provide enough bandwidth (see [RFC7916]). This solution may not 989 experience this issue as the service provider may have control on 990 the FRR path being used preventing network congestion. 992 o PLSN applies to all nodes in a network (remote or local changes), 993 while the proposed mechanism applies only on the nodes connected 994 to the topology change. 996 10.2. OFIB 998 OFIB ([RFC6976]) describes a mechanism where the convergence of the 999 network upon a topology change is ordered in order to prevent 1000 transient forwarding loops. Each router in the network must deduce 1001 the failure type from the LSA/LSP received and computes/applies a 1002 specific FIB update timer based on the failure type and its rank in 1003 the network considering the failure point as root. 1005 This mechanism allows to solve all the transient forwarding loop in a 1006 network at the price of introducing complexity in the convergence 1007 process that may require a strong monitoring by the service provider. 1009 Our solution reuses the OFIB concept but limits it to the first hop 1010 that experiences the topology change. As demonstrated, the mechanism 1011 proposed in this document allows to solve all the local transient 1012 forwarding loops that represents an high percentage of all the loops. 1013 Moreover limiting the mechanism to one hop allows to keep the 1014 network-wide convergence behavior. 1016 11. Implementation Status 1018 At this time, there are three different implementations of this 1019 mechanism. 1021 o Implementation 1: 1023 * Organization: Cisco 1025 * Implementation name: Local Microloop Protection 1027 * Operating system: IOS-XE 1029 * Level of maturity: production release 1031 * Coverage: all the specification is implemented 1033 * Protocols supported: ISIS and OSPF 1035 * Implementation experience: tested in lab and works as expected 1037 * Comment: the feature gives the ability to choose to apply the 1038 delay to FRR protected entry only 1040 * Report last update: 10-11-2017 1042 o Implementation 2: 1044 * Organization: Cisco 1046 * Implementation name: Local Microloop Protection 1048 * Operating system: IOS-XR 1050 * Level of maturity: deployed 1052 * Coverage: all the specification is implemented 1053 * Protocols supported: ISIS and OSPF 1055 * Implementation experience: deployed and works as expected 1057 * Comment: the feature gives the ability to choose to apply the 1058 delay to FRR protected entry only 1060 * Report last update: 10-11-2017 1062 o Implementation 3: 1064 * Organization: Juniper Networks 1066 * Implementation name: Microloop avoidance when IS-IS link fails 1068 * Operating system: JUNOS 1070 * Level of maturity: deployed (hidden command) 1072 * Coverage: all the specification is implemented 1074 * Protocols supported: ISIS only 1076 * Implementation experience: deployed and works as expected 1078 * Comment: the feature applies to all the ISIS routes 1080 * Report last update: 10-11-2017 1082 12. Security Considerations 1084 This document does not introduce any change in term of IGP security. 1085 The operation is internal to the router. The local delay does not 1086 increase the number of attack vectors as an attacker could only 1087 trigger this mechanism if he already has be ability to disable or 1088 enable an IGP link. The local delay does not increase the negative 1089 consequences. If an attacker has the ability to disable or enable an 1090 IGP link, it can already harm the network by creating instability and 1091 harm the traffic by creating forwarding packet loss and forwarding 1092 loss for the traffic crossing that link. 1094 13. Acknowledgements 1096 We would like to thanks the authors of [RFC6976] for introducing the 1097 concept of ordered convergence: Mike Shand, Stewart Bryant, Stefano 1098 Previdi, and Olivier Bonaventure. 1100 14. IANA Considerations 1102 This document has no actions for IANA. 1104 15. References 1106 15.1. Normative References 1108 [ISO10589] 1109 "Intermediate System to Intermediate System intra-domain 1110 routeing information exchange protocol for use in 1111 conjunction with the protocol for providing the 1112 connectionless-mode network service (ISO 8473)", 1113 ISO 10589, 2002. 1115 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1116 Requirement Levels", BCP 14, RFC 2119, 1117 DOI 10.17487/RFC2119, March 1997, 1118 . 1120 [RFC2328] Moy, J., "OSPF Version 2", STD 54, RFC 2328, 1121 DOI 10.17487/RFC2328, April 1998, 1122 . 1124 15.2. Informative References 1126 [I-D.ietf-rtgwg-backoff-algo] 1127 Decraene, B., Litkowski, S., Gredler, H., Lindem, A., 1128 Francois, P., and C. Bowers, "SPF Back-off algorithm for 1129 link state IGPs", draft-ietf-rtgwg-backoff-algo-06 (work 1130 in progress), October 2017. 1132 [I-D.ietf-rtgwg-microloop-analysis] 1133 Zinin, A., "Analysis and Minimization of Microloops in 1134 Link-state Routing Protocols", draft-ietf-rtgwg-microloop- 1135 analysis-01 (work in progress), October 2005. 1137 [RFC3906] Shen, N. and H. Smit, "Calculating Interior Gateway 1138 Protocol (IGP) Routes Over Traffic Engineering Tunnels", 1139 RFC 3906, DOI 10.17487/RFC3906, October 2004, 1140 . 1142 [RFC5715] Shand, M. and S. Bryant, "A Framework for Loop-Free 1143 Convergence", RFC 5715, DOI 10.17487/RFC5715, January 1144 2010, . 1146 [RFC6976] Shand, M., Bryant, S., Previdi, S., Filsfils, C., 1147 Francois, P., and O. Bonaventure, "Framework for Loop-Free 1148 Convergence Using the Ordered Forwarding Information Base 1149 (oFIB) Approach", RFC 6976, DOI 10.17487/RFC6976, July 1150 2013, . 1152 [RFC7916] Litkowski, S., Ed., Decraene, B., Filsfils, C., Raza, K., 1153 Horneffer, M., and P. Sarkar, "Operational Management of 1154 Loop-Free Alternates", RFC 7916, DOI 10.17487/RFC7916, 1155 July 2016, . 1157 Authors' Addresses 1159 Stephane Litkowski 1160 Orange 1162 Email: stephane.litkowski@orange.com 1164 Bruno Decraene 1165 Orange 1167 Email: bruno.decraene@orange.com 1169 Clarence Filsfils 1170 Cisco Systems 1172 Email: cfilsfil@cisco.com 1174 Pierre Francois 1175 Individual 1177 Email: pfrpfr@gmail.com