idnits 2.17.1 draft-ietf-grow-as-path-prepending-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? -- The draft header indicates that this document updates RFC7454, but the abstract doesn't seem to mention this, which it should. -- The draft header indicates that this document updates RFC8195, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (July 8, 2021) is 1016 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 492 -- Looks like a reference, but probably isn't: '2' on line 494 ** Downref: Normative reference to an Informational RFC: RFC 5398 ** Downref: Normative reference to an Informational RFC: RFC 5737 ** Downref: Normative reference to an Informational RFC: RFC 8195 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. McBride 3 Internet-Draft Futurewei 4 Updates: 7454, 8195 (if approved) D. Madory 5 Intended status: Best Current Practice Kentik 6 Expires: January 9, 2022 J. Tantsura 7 Microsoft 8 R. Raszuk 9 NTT Network Innovations 10 H. Li 11 HPE 12 J. Heitz 13 Cisco 14 G. Mishra 15 Verizon Inc. 16 July 8, 2021 18 AS Path Prepending 19 draft-ietf-grow-as-path-prepending-04 21 Abstract 23 AS Path Prepending provides a tool to manipulate the BGP AS_Path 24 attribute through prepending multiple entries of an AS. AS Path 25 Prepending is used to deprioritize a route or alternate path. By 26 prepending the local ASN multiple times, ASs can make advertised AS 27 paths appear artificially longer. Excessive AS Path Prepending has 28 caused routing issues in the internet. This document provides 29 guidance with the use of AS Path Prepending, including alternative 30 solutions, in order to avoid negatively affecting the internet. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at https://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on January 9, 2022. 49 Copyright Notice 51 Copyright (c) 2021 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents 56 (https://trustee.ietf.org/license-info) in effect on the date of 57 publication of this document. Please review these documents 58 carefully, as they describe your rights and restrictions with respect 59 to this document. Code Components extracted from this document must 60 include Simplified BSD License text as described in Section 4.e of 61 the Trust Legal Provisions and are provided without warranty as 62 described in the Simplified BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 67 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 68 2. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 3 69 3. Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 3.1. Cascading and ripple affect of prepending across the 71 internet . . . . . . . . . . . . . . . . . . . . . . . . 4 72 3.2. Excessive Prepending . . . . . . . . . . . . . . . . . . 5 73 3.3. Prepending during a routing leak . . . . . . . . . . . . 6 74 3.4. Prepending to All . . . . . . . . . . . . . . . . . . . . 7 75 3.5. Memory . . . . . . . . . . . . . . . . . . . . . . . . . 7 76 3.6. Errant announcement . . . . . . . . . . . . . . . . . . . 7 77 4. Alternatives to AS Path Prepend . . . . . . . . . . . . . . . 8 78 5. Best Practices . . . . . . . . . . . . . . . . . . . . . . . 9 79 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 80 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 81 8. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 11 82 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 83 9.1. Normative References . . . . . . . . . . . . . . . . . . 11 84 9.2. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 11 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 87 1. Introduction 89 The Border Gateway Protocol (BGP) [RFC4271] specifies the AS_PATH 90 attribute which enumerates ASs a route update has traversed. If the 91 UPDATE message is propagated over an external link, then the local AS 92 number is prepended to the AS_PATH attribute, and the NEXT_HOP 93 attribute is updated with an IP address of the router that should be 94 used as a next hop to the network. If the UPDATE message is 95 propagated over an internal link, then the AS_PATH attribute and the 96 NEXT_HOP attribute are passed unmodified. 98 A common practice among operators is to prepend multiple entries of 99 an AS (known as AS Path Prepending) in order to deprioritize a route 100 or a path. This has worked well in practice but the practice is 101 increasing, with both IPv4 and IPv6, and there are inherit risks to 102 the global internet especially with excessive AS Path Prepending. 103 Prepending is frequently employed in an excessive manner such that it 104 renders routes vulnerable to disruption or misdirection. AS Path 105 Prepending is discussed in Use of BGP Large Communities [RFC8195] and 106 this document provides additional, and specific, guidance to 107 operators on how to be a good internet citizen with the proper use of 108 AS Path Prepending. 110 1.1. Requirements Language 112 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 113 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 114 document are to be interpreted as described in RFC 2119 [RFC2119]. 116 2. Use Cases 118 There are various reasons that AS Path Prepending is in use today 119 including: 121 o Preferring one ISP over another ISP on the same ASBR or across 122 different ASBRs. 124 o Preferring one ASBR over another ASBR in the same site or between 125 sister sites. 127 o Utilize one path exclusively and another path solely as a backup. 129 o Signal to indicate that one path may have a different amount of 130 capacity than another where the lower capacity link still takes 131 traffic. 133 o Conditionally prefer one ASBR over another at the same site or 134 between sites for lowest latency path based on geographic 135 location. 137 o An ISP doesn't accept traffic engineering using BGP communities. 138 Prepending is the only option. 140 The following illustration, from Geoff Hustons Path Prepending in BGP 141 [1], shows how AS Prepending is typically used: 143 +---+ +---+ 144 +---| D |----| F | 145 | +---+ +---+ 146 +---+ +---+ | 147 | A |---| B | | 148 +---+ +---+ 2x<- | 149 | +---+ +---+ 150 +---| C |----| E | 151 +---+ +---+ 153 In the diagram above, A, B, C, D, E, and F all have a different AS. 154 B will normally prefer the path via C to send traffic to E, as this 155 represents the shorter AS path for B. If E were to prepend a further 156 two instances of its own AS number when advertising its routes to C, 157 then B will now see a different situation, where the AS Path via D 158 represents the shorter path. Through the use of selective prepending 159 E is able to alter the routing decision of B, even though B is not an 160 adjacent neighbour of E. The result is that traffic from A and B 161 will be passed via D and F to reach E, rather than via C. In this 162 way prepending implements action at a distance where the routing 163 decisions made by non-adjacent ASs can be influenced by selective AS 164 Path prepending. 166 3. Problems 168 Since it is so commonly used, what is the problem with the excessive 169 use of AS Path Prepending? Here are a few examples: 171 3.1. Cascading and ripple affect of prepending across the internet 173 Care must be taken in prepending, as prepending can result in a 174 ripple affect with multiple AS's performing the same set of prepend 175 in the same direction can result in route leaks where the valid 176 preferred path becomes now de-preferred. 178 <-5x <-5x <-5x 179 +---+ +---+ +---+ +---+ 180 +---| D |----| F |----| H |----| J | 181 | +---+ +---+ +---+ +---+ 182 +---+ +---+ | | 183 | A |---| B | | | 184 +---+ +---+ 13x<-| | 185 | +---+ +---+ +---+ +---+ 186 +---| C |----| E |----| G |----| I | 187 +---+ +---+ +---+ +---+ 189 In the diagram above A, B, C, D, E, F G, H, I, J are all part of a 190 different AS. B will normally prefer the path via D to send traffic 191 to J, as this represents the preferred path to B, due to E prepending 192 13 instances of its own AS number when advertising routes to C. ISP 193 J decides to prepend 5 instances of its own AS when advertising H, 194 and ISP H decides to do the same and prepends 5 instances of its own 195 AS when advertising to F. ISP F decides to as well prepend 5 196 instances of its own AS when advertising to D. B now sees 19 AS hops 197 for prefixes coming from D to get to J which should be the preferred 198 path compare to 18 AS hops coming from C which is now preferred. We 199 now have a route leak to I as B now sends all of its traffic through 200 I to reach J. This is the typical scenario where route leaks occur 201 where providers decide to de-prefer a path, however as the same de- 202 prefer of a path gets cascaded in the same direction, as a result, 203 the path that should never be preferred as its as-path is very high 204 in this case 18 AS hops ends up being the preferred path resulting in 205 a route leak. BGP large communties along with conditional 206 prepending, along with care being taken when prepending is performed 207 between providers can help mitigate the adverse impacts of 208 prepending. 210 3.2. Excessive Prepending 212 The risk of excessive use of AS Path Prepending can be illustrated 213 with real-world examples that have been anonymized using 214 documentation prefixes [RFC5737] and ASs [RFC5398] . Consider the 215 prefix 198.51.100.0/24 which is normally announced with an inordinate 216 amount of prepending. A recent analysis revealed that 217 198.51.100.0/24 is announced to the world along the following AS 218 path: 220 64496 64511 64511 64511 64511 64511 64511 64511 64511 64511 64511 221 64511 64511 64511 64511 64511 64511 64511 64511 64511 64511 64511 222 64511 64511 223 In this example, the origin AS64511 appears 23 consecutive times 224 before being passed on to a single upstream (AS64496), which passes 225 it on to the global internet, prepended-to-all. An attacker, wanting 226 to intercept or manipulate traffic to this prefix, could enlist a 227 datacenter to allow announcements of the same prefix with a 228 fabricated AS path such as 999999 64496 64511. Here the fictional 229 AS999999 represents the shady datacenter. This malicious route would 230 be preferred due to the shortened AS path length and might go 231 unnoticed by the true origin, even if route-monitoring had been 232 implemented. Standard BGP route monitoring checks a route's origin 233 and upstream and both would be intact in this scenario. The length 234 of the prepending gives the attacker room to craft an AS path that 235 would appear plausible to the casual observer, comply with origin 236 validation mechanisms, and not be detected by off-the-shelf route 237 monitoring. 239 3.3. Prepending during a routing leak 241 In April 2010, a service provider experienced a routing leak. While 242 analyzing the leak something peculiar was noticed. When we ranked 243 the approximately 50,000 prefixes involved in the leak based on how 244 many ASs accepted the leaked routes, most of the impact was 245 constrained to Country A routes. However, two of the top five most- 246 propagated leaked routes (listed in the table below) were Country B 247 routes. 249 During the routing leak, nearly all of the ASs of the internet 250 preferred the Country A leaked routes for 192.0.2.0/21 and 251 198.51.100.0/22 because, at the time, these two Country B prefixes 252 were being announced to the entire internet along the following 253 excessively prepended AS path: 64496 64500 64511 64511 64511 64511 254 64511 64511. Virtually any illegitimate route would be preferred 255 over the legitimate route. In this case, the victim is all but 256 ensuring their victimhood. 258 There was only a single upstream seen in the prepending example from 259 above, so the prepending was achieving nothing except incurring risk. 260 You would think such mistakes would be relatively rare, especially 261 now, 10 years later. As it turns out, there is quite a lot of 262 prepending-to-all going on right now and during leaks, it doesn't go 263 well for those who make this mistake. While one can debate the 264 merits of prepending to a subset of multiple transit providers, it is 265 difficult to see the utility in prepending to every provider. In 266 this configuration, the prepending is no longer shaping route 267 propagation. It is simply incentivizing ASs to choose another origin 268 if one were to suddenly appear whether by mistake or otherwise. 270 3.4. Prepending to All 272 Based on analysis done in 2019, Excessive AS Path Prepending [2], out 273 of approximately 750,000 routes in the IPv4 global routing table, 274 nearly 60,000 BGP routes are prepended to 95% or more of hundreds of 275 BGP sources. About 8% of the global routing table, or 1 out of every 276 12 BGP routes, is configured with prepends to virtually the entire 277 internet. The 60,000 routes include entities of every stripe: 278 governments, financial institutions, even important parts of internet 279 infrastructure. 281 Much of the worst propagation of leaked routes during big leak events 282 have been due to routes being prepended-to-all. AS64505 leak of 283 April 2014 (>320,000 prefixes) was prepended-to-all. And the AS64506 284 leak of June 2015 (>260,000 prefixes) was also prepended-to-all. 285 Prepended-to-all prefixes are those seen as prepended by all (or 286 nearly all) of the ASs of the internet. In this configuration, 287 prepending is no longer shaping route propagation but is simply 288 incentivizing ASs to choose another origin if one were to suddenly 289 appear whether by mistake or otherwise. The percentage of the IPv4 290 table that is prepended-to-all is growing at 0.5% per year. The IPv6 291 table is growing slower at 0.2% per year. The reasons for using 292 prepend-to-all appears to be due to 1) the AS forgetting to remove 293 the prepending for one of its transit providers when it is no longer 294 needed and 2) the AS attempting to de-prioritize traffic from transit 295 providers over settlement-free peers and 3) there are simply a lot of 296 errors in BGP routing. Consider the prepended AS path below: 298 64496 64501 64501 64510 64510 64501 64510 64501 64501 64510 64510 299 64501 64501 64510 301 The prepending here involves a mix of two distinct ASNs (64501 and 302 64510) with the last two digits transposed. 304 3.5. Memory 306 Long AS Paths cause an increase in memory usage by BGP speakers. A 307 concern about an AS Path longer than 255 is the extra complexity 308 required to process it, because it needs to be encoded in more than 309 one AS_SEQUENCE in the AS_PATH BGP path attribute. 311 3.6. Errant announcement 313 There was an Internet-wide outage caused by a single errant routing 314 announcement. In this incident, AS64496 announced its one prefix 315 with an extremely long AS path. Someone entered their ASN instead of 316 the prepend count 64496 modulo 256 = 252 prepends and when a path 317 lengths exceeded 255, routers crashed 319 4. Alternatives to AS Path Prepend 321 Various options, to provide path preference without needing to use AS 322 Path Prepend, include: 324 o Use predefined communities that are mapped to a particular 325 behavior when propagated. 327 o Announce more specific routes on the preferred path. 329 o The BGP Origin Code is an attribute that is used for path 330 selection and can be used as a high order tie-breaker. The three 331 origin codes are IGP, EGP and INCOMPLETE. When AS Paths are of 332 equivalent length, users could advertise paths, with IGP or EGP 333 origin, over the preferred path while the other ASBRs (which would 334 otherwise need to prepend N times) advertises with an INCOMPLETE 335 origin code. 337 o The Multi Exit Discriminator (MED) is an optional non-transitive 338 attribute that can be used to influence path preference instead of 339 using as-path. MED is non transitive so it cannot influence an AS 340 more then 1 AS hop away. 342 o Local-preference optional non-transitive attribute, above as-path 343 in bgp path selection, can be used to influence route preference 344 within the local operators AS administrative domain. Local- 345 preference can shield the operator domain from traffic shifts off 346 the preferred path to a de-preferred path caused by excess 347 prepending done by service providers across the internet. If all 348 service providers across the internet set local-preference inbound 349 conditionally with Large Community set on preferred paths, 350 essentially the impacts of route leaks as well as other routing 351 issues resulting from excess prepending can be mitigated. 353 <-5x <-5x <-5x 354 LP 110 +---+ +---+ +---+ +---+ 355 +---| D |----| F |----| H |----| J | 356 | +---+ +---+ +---+ +---+ 357 +---+ +---+ | | 358 | A |---| B | | | 359 +---+ +---+ 13x<-| | 360 | +---+ +---+ +---+ +---+ 361 +---| C |----| E |----| G |----| I | 362 +---+ +---+ +---+ +---+ 364 In the diagram above A, B, C, D, E, F G, H, I, J are all part of a 365 different AS. B will normally prefer the path via D to send traffic 366 to J, as this represents the preferred path to B, due to E prepending 367 13 instances of its own AS number when advertising routes to C. ISP 368 J decides to prepend 5 instances of its own AS when advertising to H, 369 and ISP H decides to do the same and prepends 5 instances of its own 370 AS when advertising to F. ISP F decides to also prepend 5 instances 371 of its own AS when advertising to D. B now sees 19 AS hops for 372 prefixes coming from D to get to J which should be the preferred path 373 compare to 18 AS hops coming from C which is now preferred. We now 374 have a route leak to I as B now sends all of its traffic through I to 375 reach J. Route leak on B can be prevented locally within the 376 operator domain by setting local-preference inbound, which is above 377 as-path length in the best path selection, to higher then default 100 378 to 110 inbound from D, thus shielding the operator B from being 379 influenced by the excessive prepend cascading ripple affect by F, H, 380 J. Note that A still sees the cascading ripple affect of excess 381 prepending, however A, or any service provider AS downstream of B, 382 ingressing B, will be shunted to D via local-preference and the route 383 leak is now mitigated for all downstream AS to the left of B that 384 prefer the path through B. 386 5. Best Practices 388 Many of the best practices, or lack thereof, can be illustrated from 389 the preceeding examples. Here's a summary of the best current 390 practices when using AS Path Prepending: 392 o Network operators should ensure prepending is absolutely necessary 393 as many networks have excessive prepending. It is best to 394 innumerate what the routing policies are intended to achieve 395 before concluding that prepending is a solution 397 o The neighbor you are prepending may have an unconditional 398 preference for customer routes and prepending doesn't work. It's 399 helpful to check with neighbors to see if they will honor the 400 prepend to avoid wasting effort and potentially causing further 401 vulnerabilities. 403 o Use of local-preference inbound on preferred paths between service 404 providers to help mitigate the adverse affects of prepending 406 o There is no need to prepend more than 5 ASs. The following 407 diagram, from the previously referenced AS Path Prepending 408 analysis from 2019, shows that 90% of AS path lengths are 5 ASNs 409 or fewer in length. 411 +------------------------------------+ 412 90| | 413 | X | 414 80| X X | 415 | X X | 416 70| X X | 417 | X X | 418 60| X X | 419 | X X | 420 50| X X | 421 | X X | 422 40| X X | 423 | X X | 424 30| X X | 425 | X X | 426 20| X XX | 427 | XX XX | 428 10| XX XXXX | 429 |XX XXXXXXXXXXXXXXXXX| 430 +------------------------------------+ 431 5 10 15 432 AS Path Length in IPv4 434 X Axis = unique AS Paths in millions 436 o Don't prepend ASNs that you don't own. 438 o Prepending-to-all is a self-inflicted and needless risk that 439 serves little purpose. Those excessively prepending their routes 440 should consider this risk and adjust their routing configuration. 442 o The Internet is typically around 5 ASs deep with the largest 443 AS_PATH being 16-20 ASNs. Some have added 100 or more AS Path 444 Prepends and operators should therefore consider limiting the 445 maximum AS-path length being accepted through aggressive filter 446 policies. 448 6. IANA Considerations 450 7. Security Considerations 452 Long prepending may make a network more vulernable to route hijacking 453 which will exist whenever there is a well connected peer that is 454 willing to forge their AS_PATH or allows announcements with a 455 fabricated AS path. 457 8. Acknowledgement 459 The authors would like to thank Greg Skinner, Randy Bush, Dave 460 Farmer, Nick Hilliard, Martijn Schmidt, Michael Still, Geoff Huston 461 and Jeffrey Haas for contributing to this document. 463 9. References 465 9.1. Normative References 467 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 468 Requirement Levels", BCP 14, RFC 2119, 469 DOI 10.17487/RFC2119, March 1997, 470 . 472 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 473 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 474 DOI 10.17487/RFC4271, January 2006, 475 . 477 [RFC5398] Huston, G., "Autonomous System (AS) Number Reservation for 478 Documentation Use", RFC 5398, DOI 10.17487/RFC5398, 479 December 2008, . 481 [RFC5737] Arkko, J., Cotton, M., and L. Vegoda, "IPv4 Address Blocks 482 Reserved for Documentation", RFC 5737, 483 DOI 10.17487/RFC5737, January 2010, 484 . 486 [RFC8195] Snijders, J., Heasley, J., and M. Schmidt, "Use of BGP 487 Large Communities", RFC 8195, DOI 10.17487/RFC8195, June 488 2017, . 490 9.2. URIs 492 [1] https://labs.apnic.net/?p=1264 494 [2] https://blogs.oracle.com/internetintelligence/excessive-as-path- 495 prepending-is-a-self-inflicted-vulnerability 497 Authors' Addresses 499 Mike McBride 500 Futurewei 502 Email: michael.mcbride@futurewei.com 503 Doug Madory 504 Kentik 506 Email: dmadory@kentik.com 508 Jeff Tantsura 509 Microsoft 511 Email: jefftant.ietf@gmail.com 513 Robert Raszuk 514 NTT Network Innovations 515 940 Stewart Dr 516 Sunnyvale, CA 94085 517 USA 519 Email: robert@raszuk.net 521 Hongwei Li 522 HPE 524 Email: flycoolman@gmail.com 526 Jakob Heitz 527 Cisco 528 170 West Tasman Drive 529 San Jose, CA 95134 530 USA 532 Email: jheitz@cisco.com 534 Gyan Mishra 535 Verizon Inc. 537 Email: gyan.s.mishra@verizon.com