idnits 2.17.1 draft-ietf-grow-bgp-gshut-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 3, 2017) is 2489 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Francois 3 Internet-Draft Individual Contributor 4 Intended status: Informational B. Decraene 5 Expires: January 4, 2018 Orange 6 C. Pelsser 7 Strasbourg University 8 K. Patel 9 Arrcus, Inc. 10 C. Filsfils 11 Cisco Systems 12 July 3, 2017 14 Graceful BGP session shutdown 15 draft-ietf-grow-bgp-gshut-09 17 Abstract 19 This draft describes operational procedures aimed at reducing the 20 amount of traffic lost during planned maintenances of routers or 21 links, involving the shutdown of BGP peering sessions. It defines a 22 well-known BGP community, called GRACEFUL_SHUTDOWN, to signal the 23 graceful shutdown of paths. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on January 4, 2018. 42 Copyright Notice 44 Copyright (c) 2017 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 3. Packet loss upon manual EBGP session shutdown . . . . . . . . 4 62 4. Practices to avoid packet losses . . . . . . . . . . . . . . 4 63 4.1. Improving availability of alternate paths . . . . . . . . 4 64 4.2. Make before break convergence: graceful shutdown . . . . 5 65 4.3. Forwarding modes and transient forwarding loops during 66 convergence . . . . . . . . . . . . . . . . . . . . . . . 5 67 5. EBGP graceful shutdown procedure . . . . . . . . . . . . . . 5 68 5.1. Pre-configuration . . . . . . . . . . . . . . . . . . . . 5 69 5.2. Operations at maintenance time . . . . . . . . . . . . . 6 70 5.3. BGP implementation support for g-Shut . . . . . . . . . . 6 71 6. Beyond EBGP graceful shutdown . . . . . . . . . . . . . . . . 7 72 6.1. IBGP graceful shutdown . . . . . . . . . . . . . . . . . 7 73 6.2. Link Up cases . . . . . . . . . . . . . . . . . . . . . . 7 74 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 75 8. Security Considerations . . . . . . . . . . . . . . . . . . . 9 76 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 9 77 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 78 10.1. Normative References . . . . . . . . . . . . . . . . . . 9 79 10.2. Informative References . . . . . . . . . . . . . . . . . 9 80 Appendix A. Alternative techniques with limited applicability . 10 81 A.1. Multi Exit Discriminator tweaking . . . . . . . . . . . . 10 82 A.2. IGP distance Poisoning . . . . . . . . . . . . . . . . . 10 83 Appendix B. Configuration Examples . . . . . . . . . . . . . . . 10 84 B.1. Cisco IOS XR . . . . . . . . . . . . . . . . . . . . . . 11 85 B.2. BIRD . . . . . . . . . . . . . . . . . . . . . . . . . . 11 86 B.3. OpenBGPD . . . . . . . . . . . . . . . . . . . . . . . . 12 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 89 1. Introduction 91 Routing changes in BGP can be caused by planned, maintenance 92 operations. This document discusses operational procedures to be 93 applied in order to reduce or eliminate losses of packets during the 94 maintenance. These losses come from the transient lack of 95 reachability during the BGP convergence following the shutdown of an 96 EBGP peering session between two Autonomous System Border Routers 97 (ASBR). 99 This document presents procedures for the cases where the forwarding 100 plane is impacted by the maintenance, hence when the use of Graceful 101 Restart does not apply. 103 The procedures described in this document can be applied to reduce or 104 avoid packet loss for outbound and inbound traffic flows initially 105 forwarded along the peering link to be shut down. These procedures 106 trigger, in both involved ASes, rerouting to the alternate path, 107 while allowing routers to keep using old paths until alternate ones 108 are learned, installed in the RIB and in the FIB. This ensures that 109 routers always have a valid route available during the convergence 110 process. 112 The goal of the document is to meet the requirements described in 113 [RFC6198] at best, without changing the BGP protocol. 115 This document defines a well-known community [RFC1997], called 116 GRACEFUL_SHUTDOWN, for the purpose of reducing the management 117 overhead of gracefully shutting down BGP sessions. The well-known 118 community allows implementers to provide an automated graceful 119 shutdown mechanism that does not require any router reconfiguration 120 at maintenance time. 122 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 123 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 124 document are to be interpreted as described in RFC 2119 [RFC2119]. 126 2. Terminology 128 graceful shutdown initiator: a router on which the session shutdown 129 is performed for the maintenance. 131 graceful shutdown receiver: a router that has a BGP session, to be 132 shutdown, with the graceful shutdown initiator. 134 Initiator AS: the Autonomous System of the graceful shutdown 135 initiator. 137 Receiver AS: the Autonomous System of the graceful shutdown receiver. 139 Loss of Connectivity (LoC: the state when a router has no path toward 140 an affected prefix. 142 3. Packet loss upon manual EBGP session shutdown 144 Packets can be lost during a manual shutdown of an EBGP session for 145 two reasons. 147 First, routers involved in the convergence process can transiently 148 lack of paths toward an affected prefix, and drop traffic destined to 149 this prefix. This is because alternate paths can be hidden by nodes 150 of an AS. This happens when the paths are not selected as best by 151 the ASBR that receive them on an EBGP session, or by Route Reflectors 152 that do not propagate them further in the IBGP topology because they 153 do not select them as best. 155 Second, within the AS, the FIB of routers can be transiently 156 inconsistent during the BGP convergence and packets toward affected 157 prefixes can loop and be dropped. Note that these loops only happen 158 when ASBR-to-ASBR encapsulation is not used within the AS. 160 This document only addresses the first reason. 162 4. Practices to avoid packet losses 164 This section describes means for an ISP to reduce the transient loss 165 of packets upon a manual shutdown of a BGP session. 167 4.1. Improving availability of alternate paths 169 All solutions that increase the availability of alternate BGP paths 170 at routers performing packet lookups in BGP tables such as 171 [I-D.ietf-idr-best-external] and [RFC7911] help in reducing the LoC 172 bound with manual shutdown of EBGP sessions. 174 One of such solutions increasing diversity in such a way that, at any 175 single step of the convergence process following the EBGP session 176 shutdown, a BGP router does not receive a message withdrawing the 177 only path it currently knows for a given NLRI, allows for a 178 simplified graceful shutdown procedure. 180 Note that the LoC for the inbound traffic of the maintained router, 181 induced by a lack of alternate path propagation within the IBGP 182 topology of a receiver AS is not under the control of the operator 183 performing the maintenance. The part of the procedure aimed at 184 avoiding LoC for incoming paths can thus be applied even if no LoC 185 are expected for the outgoing paths. 187 4.2. Make before break convergence: graceful shutdown 189 The goal of this procedure is to retain the paths to be shutdown 190 between the peers, but with a lower LOCAL_PREF value, allowing the 191 paths to remain in use while alternate paths are selected and 192 propagated, rather than simply withdrawing the paths. 194 Section 5 describes configurations and actions to be performed for 195 the graceful shutdown of BGP sessions. 197 4.3. Forwarding modes and transient forwarding loops during convergence 199 The graceful shutdown procedure or the solutions improving the 200 availability of alternate paths, do not change the fact that BGP 201 convergence and the subsequent FIB updates are run independently on 202 each router of the ASes. If the AS applying the solution does not 203 rely on encapsulation to forward packets from the Ingress Border 204 Router to the Egress Border Router, then transient forwarding loops 205 and consequent packet losses can occur during the convergence 206 process. If zero LoC is required, encapsulation is required between 207 ASBRs of the AS. 209 5. EBGP graceful shutdown procedure 211 This section describes configurations and actions to be performed for 212 the graceful shutdown of EBGP peering links. 214 5.1. Pre-configuration 216 On each ASBR supporting the graceful shutdown receiver procedure, an 217 inbound BGP route policy is applied on all EBGP sessions of the ASBR, 218 that: 220 o matches the GRACEFUL_SHUTDOWN community 222 o sets the LOCAL_PREF attribute of the paths tagged with the 223 GRACEFUL_SHUTDOWN community to a low value 225 Note that in the case where an AS is aggregating multiple routes 226 under a covering prefix, it is recommended to filter out the 227 GRACEFUL_SHUTDOWN community from the resulting aggregate BGP route. 228 By doing so, the setting of the GRACEFUL_SHUTDOWN community on one of 229 the aggregated routes will not let the entire aggregate inherit the 230 community. Not doing so would let the entire aggregate undergo the 231 graceful shutdown behavior. 233 5.2. Operations at maintenance time 235 On the graceful shutdown initiator, upon maintenance time, it is 236 required to: 238 o apply an outbound BGP route policy on the EBGP session to be 239 shutdown. This policy tags the paths propagated over the session 240 with the GRACEFUL_SHUTDOWN community. This will trigger the BGP 241 implementation to re-advertise all active routes previously 242 advertised, and tag them with the GRACEFUL_SHUTDOWN community. 244 o apply an inbound BGP route policy on the maintained EBGP session 245 to tag the paths received over the session with the 246 GRACEFUL_SHUTDOWN community. 248 o wait for convergence to happen. 250 o shutdown the EBGP session, optionally using 251 [I-D.ietf-idr-shutdown] to communicate the reason of the shutdown. 253 In the case of a shutdown of the whole router, in addition to the 254 graceful shutdown of all EBGP sessions, there is a need to graceful 255 shutdown the routes originated by this router (e.g, BGP aggregates 256 redistributed from other protocols, including static routes). This 257 can be performed by tagging such routes with the GRACEFUL_SHUTDOWN 258 community. 260 5.3. BGP implementation support for g-Shut 262 A BGP router implementation MAY provide features aimed at automating 263 the application of the graceful shutdown procedures described above. 265 Upon a session shutdown specified as graceful by the operator, a BGP 266 implementation supporting a graceful shutdown feature SHOULD: 268 1. Update all the paths propagated over the corresponding EBGP 269 session, tagging the GRACEFUL_SHUTDOWN community to them. Any 270 subsequent update sent over the session being gracefully shut 271 down would be tagged with the GRACEFUL_SHUTDOWN community. 273 2. Lower the LOCAL_PREF value of the paths received over the EBGP 274 session being shut down. 276 3. Optionally shut down the session after a configured time. 278 4. Prevent the GRACEFUL_SHUTDOWN community from being inherited by a 279 path that would aggregate some paths tagged with the GSHUT 280 community. This behavior avoids the GSHUT procedure to be 281 applied to the aggregate upon the graceful shutdown of one of its 282 covered prefixes. 284 A BGP implementation supporting a graceful shutdown feature SHOULD 285 also automatically install the BGP policies that are supposed to be 286 configured, as described in Section 5.1 for sessions over which 287 graceful shutdown is to be supported. 289 6. Beyond EBGP graceful shutdown 291 6.1. IBGP graceful shutdown 293 For the shutdown of an IBGP session, provided the IBGP topology is 294 viable after the maintenance of the session, i.e, if all BGP speakers 295 of the AS have an IBGP signaling path for all prefixes advertised on 296 this graceful shutdown IBGP session, then the shutdown of an IBGP 297 session does not lead to transient unreachability. As a consequence, 298 no specific graceful shutdown action is required. 300 6.2. Link Up cases 302 We identify two potential causes for transient packet losses upon an 303 EBGP link up event. The first one is local to the graceful no-shut 304 initiator, the second one is due to the BGP convergence following the 305 injection of new best paths within the IBGP topology. 307 6.2.1. Unreachability local to the ASBR 309 An ASBR that selects as best a path received over a newly brought up 310 EBGP session may transiently drop traffic. This can typically happen 311 when the NEXT_HOP attribute differs from the IP address of the EBGP 312 peer, and the receiving ASBR has not yet resolved the MAC address 313 associated with the IP address of that "third party" NEXT_HOP. 315 A BGP speaker implementation could avoid such losses by ensuring that 316 "third party" NEXT_HOPs are resolved before installing paths using 317 these in the RIB. 319 If the link up event corresponds to an EBGP session that is being 320 manually brought up, over an already up multi-access link, then the 321 operator can ping third party NEXT_HOP that are expected to be used 322 before actually bringing the session up, or ping directed broadcast 323 the subnet IP address of the link. By proceeding like this, the MAC 324 addresses associated with these third party NEXT_HOP will be resolved 325 by the graceful no-shut initiator. 327 6.2.2. IBGP convergence 329 Corner cases leading to LoC can occur during an EBGP link up event. 331 A typical example for such transient unreachability for a given 332 prefix is the following: 334 Let's consider 3 route reflectors RR1, RR2, RR3. There is a full 335 mesh of IBGP session between them. 337 1. RR1 is initially advertising the current best path to the 338 members of its IBGP RR full-mesh. It propagated that path within 339 its RR full-mesh. RR2 knows only that path toward the prefix. 341 2. RR3 receives a new best path originated by the "graceful no- 342 shut" initiator, being one of its RR clients. RR3 selects it as 343 best, and propagates an UPDATE within its RR full-mesh, i.e., to 344 RR1 and RR2. 346 3. RR1 receives that path, reruns its decision process, and picks 347 this new path as best. As a result, RR1 withdraws its previously 348 announced best-path on the IBGP sessions of its RR full-mesh. 350 4. If, for any reason, RR3 processes the withdraw generated in 351 step 3, before processing the update generated in step 2, RR3 352 transiently suffers from unreachability for the affected prefix. 354 The use of [I-D.ietf-idr-best-external] among the RR of the IBGP 355 full-mesh can solve these corner cases by ensuring that within an AS, 356 the advertisement of a new route is not translated into the withdraw 357 of a former route. 359 Indeed, "best-external" ensures that an ASBR does not withdraw a 360 previously advertised (EBGP) path when it receives an additional, 361 preferred path over an IBGP session. Also, "best-intra-cluster" 362 ensures that a RR does not withdraw a previously advertised (IBGP) 363 path to its non clients (e.g. other RRs in a mesh of RR) when it 364 receives a new, preferred path over an IBGP session. 366 7. IANA Considerations 368 The IANA has assigned the community value 0xFFFF0000 to the planned- 369 shut community in the "BGP Well-known Communities" registry. IANA is 370 requested to change the name planned-shut to GRACEFUL_SHUTDOWN and 371 set this document as the reference. 373 8. Security Considerations 375 By providing the graceful shutdown service to a neighboring AS, an 376 ISP provides means to this neighbor and possibly its downstream ASes 377 to lower the LOCAL_PREF value assigned to the paths received from 378 this neighbor. 380 The neighbor could abuse the technique and do inbound traffic 381 engineering by declaring some prefixes as undergoing a maintenance so 382 as to switch traffic to another peering link. 384 If this behavior is not tolerated by the ISP, it SHOULD monitor the 385 use of the graceful shutdown community by this neighbor. 387 9. Acknowledgments 389 The authors wish to thank Olivier Bonaventure, Pradosh Mohapatra and 390 Job Snijders for their useful comments on this work. 392 10. References 394 10.1. Normative References 396 [RFC1997] Chandra, R., Traina, P., and T. Li, "BGP Communities 397 Attribute", RFC 1997, DOI 10.17487/RFC1997, August 1996, 398 . 400 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 401 Requirement Levels", BCP 14, RFC 2119, 402 DOI 10.17487/RFC2119, March 1997, 403 . 405 [RFC6198] Decraene, B., Francois, P., Pelsser, C., Ahmad, Z., 406 Elizondo Armengol, A., and T. Takeda, "Requirements for 407 the Graceful Shutdown of BGP Sessions", RFC 6198, 408 DOI 10.17487/RFC6198, April 2011, 409 . 411 10.2. Informative References 413 [I-D.ietf-idr-best-external] 414 Marques, P., Fernando, R., Chen, E., Mohapatra, P., and H. 415 Gredler, "Advertisement of the best external route in 416 BGP", draft-ietf-idr-best-external-05 (work in progress), 417 January 2012. 419 [I-D.ietf-idr-shutdown] 420 Snijders, J., Heitz, J., and J. Scudder, "BGP 421 Administrative Shutdown Communication", draft-ietf-idr- 422 shutdown-10 (work in progress), June 2017. 424 [RFC7911] Walton, D., Retana, A., Chen, E., and J. Scudder, 425 "Advertisement of Multiple Paths in BGP", RFC 7911, 426 DOI 10.17487/RFC7911, July 2016, 427 . 429 Appendix A. Alternative techniques with limited applicability 431 A few alternative techniques have been considered to provide graceful 432 shutdown capabilities but have been rejected due to their limited 433 applicability. This section describe them for possible reference. 435 A.1. Multi Exit Discriminator tweaking 437 The MED attribute of the paths to be avoided can be increased so as 438 to force the routers in the neighboring AS to select other paths. 440 The solution only works if the alternate paths are as good as the 441 initial ones with respect to the Local-Pref value and the AS Path 442 Length value. In the other cases, increasing the MED value will not 443 have an impact on the decision process of the routers in the 444 neighboring AS. 446 A.2. IGP distance Poisoning 448 The distance to the BGP NEXT_HOP corresponding to the maintained 449 session can be increased in the IGP so that the old paths will be 450 less preferred during the application of the IGP distance tie-break 451 rule. However, this solution only works for the paths whose 452 alternates are as good as the old paths with respect to their Local- 453 Pref value, their AS Path length, and their MED value. 455 Also, this poisoning cannot be applied when nexthop self is used as 456 there is no nexthop specific to the maintained session to poison in 457 the IGP. 459 Appendix B. Configuration Examples 461 This appendix is non-normative. 463 Example routing policy configurations to honor the GRACEFUL_SHUTDOWN 464 well-known BGP community. 466 B.1. Cisco IOS XR 468 community-set comm-graceful-shutdown 469 65535:0 470 end-set 471 ! 472 route-policy AS64497-ebgp-inbound 473 ! normally this policy would contain much more 474 if community matches-any comm-graceful-shutdown then 475 set local-preference 0 476 endif 477 end-policy 478 ! 479 router bgp 64496 480 neighbor 2001:db8:1:2::1 481 remote-as 64497 482 description a fantastic EBGP neighbor 483 address-family ipv6 unicast 484 send-community-ebgp 485 route-policy AS64497-ebgp-inbound in 486 route-policy AS65040v6-bgp-out out 487 ! 488 ! 489 ! 491 B.2. BIRD 493 function honor_graceful_shutdown() { 494 if (65535, 0) ~ bgp_community then { 495 bgp_local_pref = 0; 496 } 497 } 498 filter AS64497_ebgp_inbound 499 { 500 # normally this policy would contain much more 501 honor_graceful_shutdown(); 502 } 503 protocol bgp peer_64497_1 { 504 description "a fantastic EBGP neighbor"; 505 neighbor 2001:db8:1:2::1 as 64497; 506 local as 64496; 507 import keep filtered; 508 import filter AS64497_ebgp_inbound; 509 export filter AS64497_ebgp_outbound; 510 } 512 B.3. OpenBGPD 514 AS 64496 515 router-id 192.0.2.1 516 neighbor 2001:db8:1:2::1 { 517 descr "a fantastic EBGP neighbor" 518 remote-as 64497 519 } 520 # normally this policy would contain much more 521 match from any community GRACEFUL_SHUTDOWN set { localpref 0 } 523 Authors' Addresses 525 Pierre Francois 526 Individual Contributor 528 Email: pfrpfr@gmail.com 530 Bruno Decraene 531 Orange 533 Email: bruno.decraene@orange.com 535 Cristel Pelsser 536 Strasbourg University 538 Email: pelsser@unistra.fr 540 Keyur Patel 541 Arrcus, Inc. 543 Email: keyur@arrcus.com 545 Clarence Filsfils 546 Cisco Systems 548 Email: cfilsfil@cisco.com