idnits 2.17.1 draft-ietf-grow-irr-routing-policy-considerations-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (August 27, 2014) is 3530 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 GROW Working Group D. McPherson 3 Internet-Draft Verisign, Inc. 4 Intended status: Informational S. Amante 5 Expires: February 28, 2015 Level 3 Communications 6 E. Osterweil 7 Verisign, Inc. 8 L. Blunk 9 Merit Network, Inc. 10 D. Mitchell 11 Twitter, Inc. 12 August 27, 2014 14 IRR & Routing Policy Configuration Considerations 15 17 Abstract 19 The purpose of this document is to catalog past issues influencing 20 the efficacy of Internet Routing Registries (IRR) for inter-domain 21 routing policy specification and application in the global routing 22 system over the past two decades. Additionally, it provides a 23 discussion regarding which of these issues are still problematic in 24 practice, and which are simply artifacts that are no longer 25 applicable but continue to stifle inter-provider policy-based 26 filtering adoption and IRR utility to this day. 28 Status of this Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at http://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on February 28, 2015. 45 Copyright Notice 47 Copyright (c) 2014 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 3. Historical Artifacts Influencing IRR Efficacy . . . . . . . . 4 68 4. Accuracy and Integrity of Data Contained within the IRR . . . 5 69 4.1. Lack of Resource Certification . . . . . . . . . . . . . . 5 70 4.2. Incentives to Maintain Data within the IRR . . . . . . . . 6 71 4.3. Inability for Third Parties to Remove (Stale) 72 Information from the IRRs . . . . . . . . . . . . . . . . 7 73 4.4. Lack of Authoritative IRR for Resources . . . . . . . . . 8 74 4.5. Conclusions with respect to Data in the IRR . . . . . . . 9 76 5. Operation of the IRR Infrastructure . . . . . . . . . . . . . 9 77 5.1. Replication of Resources Among IRRs . . . . . . . . . . . 9 78 5.2. Updating Routing Policies from Updated IRR Resources . . . 10 80 6. Historical BGP Protocol Limitations . . . . . . . . . . . . . 12 82 7. Historical Limitations of Routers . . . . . . . . . . . . . . 13 83 7.1. Incremental Updates to Policy on Routers . . . . . . . . . 13 84 7.2. Storage Requirements for Policy on Routers . . . . . . . . 14 85 7.3. Updating Configuration on Routers . . . . . . . . . . . . 14 87 8. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 89 9. Security Considerations . . . . . . . . . . . . . . . . . . . 16 91 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 93 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16 95 12. Informative References . . . . . . . . . . . . . . . . . . . . 16 96 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18 98 1. Introduction 100 The purpose of this document is to catalog past issues influencing 101 the efficacy of Internet Routing Registries (IRR) for inter-domain 102 routing policy specification and application in the global routing 103 system over the past two decades. Additionally, it provides a 104 discussion regarding which of these issues still pose problems in 105 practice, and which are no longer obstacles, but whose perceived 106 drawbacks continue to stifle inter-provider policy-based filtering 107 support and IRR utility to this day. 109 2. Background 111 IRRs can be used to express a multitude of Internet number bindings 112 and policy objectives, to include bindings between an origin AS and a 113 given prefix, AS and community import and export policies for a given 114 AS, as well as AS macros (as-sets in RPSL-speak) that convey the set 115 of ASes for which a given AS intends to include in some common group. 117 As quoted from Section 7 of [RFC1787], Routing in a Multi-Provider 118 Internet: 120 While ensuring Internet-wide coordination may be more and more 121 difficult, as the Internet continues to grow, stability and 122 consistency of the Internet-wide routing could significantly 123 benefit if the information about routing requirements of various 124 organizations could be shared across organizational boundaries. 125 Such information could be used in a wide variety of situations 126 ranging from troubleshooting to detecting and eliminating 127 conflicting routing requirements. The scale of the Internet 128 implies that the information should be distributed. Work is 129 currently underway to establish depositories of this information 130 (Routing Registries), as well as to develop tools that analyze, as 131 well as utilize this information. 133 3. Historical Artifacts Influencing IRR Efficacy 135 The term IRR is often used, incorrectly, as a broad catch-all term to 136 categorize issues related to the accuracy of data in the IRR, the 137 Routing Policy Specification Language (RPSL) and the operational 138 deployment of policy (derived from RPSL contained within the IRR) to 139 routers. It is important to classify these issues into distinct 140 categories so that the reader can understand which categories of 141 issues are historical artifacts that are no longer applicable and 142 which categories of issues still exist and might be addressed by the 143 IETF. 145 The following sections will separate out challenges related to the 146 IRR into the following categories. First, accuracy and integrity of 147 data contained within the IRR. Second, the Resource Policy 148 Specification Language (RPSL) used to represent various types of data 149 in the IRR. Third, operation of the IRR infrastructure, i.e.: 150 synchronization of resources from one IRR to other IRRs. Finally, 151 the methods related to extraction of policy from the IRR and the 152 input plus activation of that policy within routers. 154 4. Accuracy and Integrity of Data Contained within the IRR 156 The following section will examine issues related to accuracy and 157 integrity of data contained within the IRR. 159 4.1. Lack of Resource Certification 161 Internet number resources include IPv4 addresses, IPv6 addresses, 162 Autonomous System Numbers (ASNs), and more. While these resources 163 are generally allocated by hierarchical authorities, a general 164 mechanism for formally verifying (such as through cryptographic 165 mechanisms) when parties have been allocated resource remains an open 166 challenge. We generally define such a system a Resource 167 Certification System, and we note that some candidate examples of how 168 such a general system might be implemented and deployed exist 169 [TASRS], [RC_HotNetsX], [RFC6480]. 171 One of the largest weaknesses often cited with the IRR system is that 172 the data contained within the IRRs is out of date or lacks integrity. 173 This is largely attributable to the fact that existing IRR mechanisms 174 do not provide ways for a relying party to (cryptographically) verify 175 the validity of an IRR object. That is, there has never existed a 176 resource certification infrastructure that enables a resource holder 177 to authorize a particular autonomous system to originate network 178 layer reachability advertisements for a given IPv4 or IPv6 prefix. 179 It should be noted that this is not a weakness of the underlying 180 Routing Policy Specification Language (RPSL) [RFC2622], but rather, 181 was largely the result of no clear demand by the operator community 182 for Internet Number Resource Registries to provide sufficient 183 resource certification infrastructure that would enable a resource 184 holder to develop a cryptographic binding between, for example, a 185 given AS number and an IP prefix. 187 Another noteworthy (but slightly different) deficiency in the IRR 188 system is the absence of a tangible tie between the resource and the 189 resource holder. That is, generally there is no assurance of the 190 validity of objects at their creation time (except for a subset of, 191 for example, the RIPE IRR where RPSS [RFC2725] attests for RIPE 192 address holders and RIPE ASN holders). If a resource holder's 193 authorization cannot be certified, then consumers cannot verify 194 attestations made. In effect, without resource certification, 195 consumers are basically only certifying the assertions that the 196 creator / maintainer of the resource object has made (not if that 197 assertion is valid). 199 The RIPE community addressed this last issue by putting in a 200 foundation policy [RIPE2007-01], which requires a contractual link 201 between the RIPE NCC and the end user in direct assignment + ASN 202 assignment cases, which weren't previously covered by LIR contracts 203 for address allocations. There were a couple of intentions with this 204 policy: 206 1. There was an encumbrance placed in the policy for the LIR to 207 charge the end-user for provider independent resources. This 208 action created a collection mechanism for PI address resources 209 (IPv4 / IPv6 space, ASNs). 211 2. It guaranteed that all RIPE NCC allocated/assigned space would be 212 subject to a contractual link, and that this contractual chain 213 might end up actually meaning something when it came to the issue 214 of who made what claim about what number resource. 216 3. It tied into the RIPE NCC's object grandfathering policy which 217 ties the registration details of the end-user to the object 218 registered in the IRR database. 220 While this policy specifically addressed PI/portable space holders, 221 other regions address this issue, too. Further, it is indeed a 222 prerequisite for resource certification, though it does not directly 223 address the IRR deficiencies. 225 One of the central observations of this policy was that without a 226 chain-of-ownership functionality in IRR databases, the discussion of 227 certifying their contents becomes moot. 229 4.2. Incentives to Maintain Data within the IRR 231 A second problem with data contained in the IRRs is that the 232 incentives for resource holders to maintain both accurate and up-to- 233 date information in one, or more, IRRs; including deletion of out-of- 234 date or stale data from the IRRs can diminish rapidly when changing 235 their peering policies (such as switching transit providers). 236 Specifically, there is a very strong incentive for an ISP's customers 237 to register new routing information in the IRR, because some ISPs 238 enforce a strict policy that they will only build or update a 239 customer's prefix-lists applied to the customer's inbound eBGP 240 sessions based off information found within the IRRs. Unfortunately, 241 there is little incentive for an ISP's customers to remove out-of- 242 date information from an IRR, most likely attributed to the fact that 243 some ISPs do not use, or enforce use of, data contained within the 244 IRRs to automatically build incoming policy applied to customer's 245 eBGP sessions. For example, if a customer is terminating service 246 from one ISP that requires use of IRR data to build incoming policy 247 and, at the same time, enabling service with another ISP that does 248 not require use of IRR data, then that customer will likely let the 249 data in the IRR become stale or inaccurate. 251 Further, policy filters are almost exclusively generated based on the 252 origin AS information contained within IRR route objects and used by 253 providers to filter downstream transit customers. Since providers 254 only look for route objects containing the origin AS of their 255 downstream customer(s), stale route objects with historical and, 256 possibly, incorrect origin AS information are ignored. Assuming that 257 the downstream customer(s) do not continue to announce those routes 258 with historical, or incorrect, origin AS information in BGP to the 259 upstream provider, there is no significant incentive to remove them 260 as they do not impact offline policy filter generation nor routing on 261 the Internet. On the other hand, the main incentive that causes the 262 Service Provider to work with downstream customer(s) is when the 263 resultant filter list becomes too large that it is difficult for it 264 to be stored on-board PE routers; however, this is more practically 265 an operational issue with very old, legacy PE routers, not more 266 modern PE router hardware with more robust Control Plane Engines. 268 4.3. Inability for Third Parties to Remove (Stale) Information from the 269 IRRs 271 A third challenge with data contained in IRRs is that it is not 272 possible for IRR operators, and ISPs who use them, to proactively 273 remove (perceived) out-of-date or "stale" resources in an IRR, on 274 behalf of resource holders who may not be diligent in maintaining 275 this information themselves. The reason is that, according to the 276 Resource Policy Specification Language (RPSL) [RFC2622], only the 277 resource holder ('mntner') specified in a 'mnt-by' value field of an 278 RPSL resource is authorized to add, modify or delete their own 279 resources within the IRR. To address this issue, the 'auth-override' 280 mechanism [RFC2725] was later developed that would have enabled a 281 third party to update and/or remove "stale" resources from the IRR. 282 While it is unclear if this was ever implemented or deployed, it does 283 provide language semantics needed to overcome this obstacle. 285 Nevertheless, with such a mechanism in place, there is still a risk 286 that the original RPSL resource holder would not receive 287 notifications (via the 'notify' attribute in various RPSL resources) 288 about the pending or actual removal of a resource from the IRR in 289 time to halt that action if those resources were still being used. 290 In this case, if the removal of a resource was not suspended, it 291 could potentially result in an unintentional denial of service for 292 the RPSL resource holder when, for example, ISPs automatically 293 generated and deployed a new policy based on the newly removed 294 resources from the IRR, thus dropping any reachability announcement 295 for the removed resource in eBGP. 297 4.4. Lack of Authoritative IRR for Resources 299 According to [RFC2622], within an RPSL resource "the source attribute 300 specifies the registry where the object is registered." Note that 301 this source attribute only exists within the RPSL resource itself. 302 Unfortunately, given a specific resource, (e.g.: a specific IPv4 or 303 IPv6 prefix), most of the time it is impossible to tell a priori the 304 authoritative IRR where to query and retrieve an authoritative copy 305 of that resource. 307 This makes it difficult for consumers of data from the IRR to 308 automatically know the authoritative IRR of a resource holder, which 309 will contain their most up-to-date set of resources. This is 310 typically not a problem for an ISP that has a direct (customer) 311 relationship with the resource holder, because the ISP will ask the 312 resource holder which (authoritative) IRR to pull their resources 313 from on, for example, a "Customer BGP Order Form". However, third 314 parties that do not have a direct relationship with the resource 315 holder, have a difficult time attempting to infer the authoritative 316 IRR, preferred by the resource holder, that likely contains the most 317 up-to-date set of resources. As a result, it would be helpful for 318 third parties if there was a robust referral mechanism so that a 319 query to one IRR would be automatically redirected toward the 320 authoritative IRR for the most up-to-date and authoritative copy of 321 that particular resource. This problem is worked around by 322 individual IRR operators storing a local copy of other IRR's 323 resources, through periodic mirroring, which allows the individual 324 IRR to respond to a client's query with all registered instances of a 325 particular IRR resource that exist in both the local IRR and all 326 other IRRs. Of course, the problem with this approach is that an 327 individual IRR operator is under no obligation to mirror all other 328 IRRs and, in practice, some IRRs do not mirror the resources from all 329 other IRRs. This could lead to the false impression that a 330 particular resource is not registered or maintained at a particular 331 IRR. Furthermore, the authentication process of accepting updates by 332 any given IRR may, or may not, be robust enough to overcome 333 impersonation attacks. As a result, there is no rigorous assurance 334 that a mirrored RPSL statement was actually made by the authorized 335 resource holder. 337 4.5. Conclusions with respect to Data in the IRR 339 All of the aforementioned issues related to integrity and accuracy of 340 data within the IRR stem from a distinct lack of resource 341 certification for resources contained within the IRR. Only now is an 342 experimental test bed that reports to provide this function (under 343 the auspices of the Resource PKI [RFC6480]) being formally discussed, 344 which could also aid in certification of resources within the IRR. 345 It should be noted that the RPKI is only currently able to support 346 signing of Route Origin Authorization (ROA) resources that are the 347 equivalent of 'route' resources in the IRR. It is unclear if, in the 348 future, the RPKI will be extended to support additional resources 349 that already exist in the IRR, e.g.: aut-num, as-net, route-set, etc. 350 Finally, a seemingly equivalent resource certification specification 351 for all resources in the IRR has already been developed [RFC2725], 352 however it is unclear how widely it was ever implemented or deployed. 354 5. Operation of the IRR Infrastructure 356 5.1. Replication of Resources Among IRRs 358 Currently, several IRRs [IRR_LIST] use a Near-Real-Time Mirroring 359 (NRTM) protocol to replicate each others contents. However, this 360 protocol has a several weaknesses. Namely, there is no way to 361 validate that the copy of mirror is correct and synchronization 362 issues have often resulted. Furthermore, the NRTM protocol does not 363 employ any security mechanisms. The NRTM protocol relies on a pull 364 mechanism and is generally configured with a poll interval of 5 to 10 365 minutes. There is currently no mechanism to notify an IRR when an 366 update has occurred in a mirrored IRR so that an immediate update can 367 be made. 369 Some providers employ a process of mirroring an instance of an IRR 370 that involves downloading a flat text file copy of the IRR that is 371 made available via FTP. These FTP files are exported at regular 372 intervals of typically anywhere between 2 and 24 hours by the IRRs. 373 When a provider fetches those text files, it will process them to 374 identify any updates and reflect changes within its own internally 375 maintained database. The use of an internally maintained database is 376 out-of-scope of this document, but is generally used to assist with 377 more straightforward access to or modification of data by the IRR 378 operator. Providers typically employ a 24 hour cycle to pull updated 379 resources from IRRs. Thus, depending on when resource holders 380 submitted their changes to an IRR, it may take up to 24 hours for 381 those changes to be reflected in their policy configurations. In 382 practice, it appears that the RPKI will also employ a 24 hour cycle 383 whereby changes in resources are pushed out to other RPKI caches 385 [RPKI_SIZING]. 387 IRRs originated from Section 7 of [RFC1787], specifically: "The scale 388 of the Internet implies that the [routing requirements] information 389 should be distributed." Regardless, the practical effect of an 390 organization maintaining its own local cache of IRR resources is an 391 increase in resource resiliency (due to multiple copies of the same 392 resource being geographically distributed), a reduction in query time 393 for resources and, likely, a reduction in inter-domain bandwidth 394 consumption and associated costs. This is particularly beneficial 395 when, for example, an ISP is evaluating resources in an IRR just to 396 determine if there are any modifications to those resources that will 397 ultimately be reflected in a new routing policy that will get 398 propagated to (Edge) routers in the ISP's network. Cache locality 399 results in reduced inter-domain bandwidth utilization for each round 400 trip. 402 On the other hand, it is unclear from where the current provider 403 replication interval of 24 hours originated or even whether it still 404 provides enough freshness in the face of available resources, 405 particularly in today's environment where a typical IRR system 406 consists of a (multi-core) multi-GHz CPU connected to a network via a 407 physical connection of 100 Mbps or, more likely, higher bandwidth. 408 In addition, it can be observed that the most common circuit size 409 used by ISPs is now 10 Gbps, thus eliminating bandwidth as a 410 significant factor in the transfer of data between IRRs. 411 Furthermore, it should be noted that Merit's Internet Routing 412 Registry Daemon (IRRd) [MERIT-IRRD], uses 10 minutes as its default 413 for "irr_mirror_interval". 415 Lastly, it should be noted that [RFC2769], Routing Policy System 416 Replication, attempted to offer a more methodical solution for 417 distributed replication of resources between IRRs. It is unclear why 418 this RFC failed to gain traction, but it is suspected that this was 419 due to that specification's reliance on [RFC2725], Routing Policy 420 System Security, that addressed "the need to assure integrity of the 421 data by providing an authentication and authorization model." 423 5.2. Updating Routing Policies from Updated IRR Resources 425 Ultimately, the length of time it takes to replicate resources among 426 IRRs is, generally, the dominant factor in reflecting changes to 427 resources in policy that is eventually applied within the Control 428 Plane of routers. The length of time to update network elements will 429 vary considerably depending on the size of the ISP and the number of 430 IRR resources that were updated during any given interval. However, 431 there are a variety of common techniques, that are outside the scope 432 of this document, that allow for this automated process to be 433 optimized to considerably reduce the length of time it takes to 434 update policies in the ISP's network. 436 An ISP will begin the process of updating the policy in their 437 network, by first fetching IRR resources associated with, for 438 example, a customer ASN attached to their network. Next, the ISP 439 constructs a new policy associated to that customer and then 440 evaluates if that new policy is different from existing policy 441 associated with that same customer. If there are no changes between 442 the new and existing policy associated with that customer, then the 443 ISP does not make any changes to the policy in their routers specific 444 to that customer. On the other hand, if the new policy does reflect 445 changes from the existing policy for that customer, then the ISP 446 begins a process of uploading the new policy to the routers attached 447 to that customer. 449 The process of constructing a new policy involves use of a set of 450 programs, e.g.: IRRtoolset, that performs recursive expansion of an 451 RPSL aut-num resource, that comprises an arbitrary combination of 452 other RPSL aut-num, as-set, route and route-set resources, according 453 to procedures defined by RPSL. The end result of this process is, 454 traditionally, a router vendor dependent configuration snippet that 455 defines the routing policy for that customer. This routing policy 456 may consist of the set of IPv4 or IPv6 prefixes, associated prefix 457 lengths and AS_PATH's that are supposed to be accepted from a 458 customer's eBGP session. However, if indicated in the appropriate 459 RPSL resource, the policy may also set certain BGP Attributes, such 460 as MED, AS_PATH prepend value, LOCAL_PREF, etc., at either the 461 incoming eBGP session from the customer or on static routes that are 462 originated by the resource holder. 464 An ISP's customers may not adequately plan for pre-planned 465 maintenance or, more likely, need to rapidly begin announcing a new 466 IP prefix as a result of, for example, an emergency turn-up of the 467 ISP customer's new downstream customer. Unfortunately, the routine, 468 automated process employed by the ISP means that they may not begin 469 updating their routing policy on their network for up to 24 hours, 470 because the ISP or the IRRs the ISP uses might only mirror changes to 471 IRR resources once every 24 hours. The time interval for the 472 routine/automated process is not responsive to the needs of directly 473 paying customer(s) who need rapid changes in their policy in rare 474 situations. In these situations, when a customer has an urgent need 475 for updates to take affect immediately, they will call up the Network 476 Operations Center (NOC) of their ISP and request that the ISP 477 immediately fetch new IRR objects and push those changes out to their 478 network. This is often accomplished in as little as five minutes 479 from the time a customer contacts their ISP's NOC to the time a new 480 filtering policy is pushed out to the PE routers that are attached to 481 that customer's Attachment Circuits (AC's). It is conceivable that 482 some ISPs automate this using out-of-band mechanisms as well, 483 although the authors are unaware of any existing mechanisms that 484 support this. 486 Ultimately, the aforementioned latency with respect to "emergency 487 changes" in IRR resources that need to be reflected in near-real-time 488 in the network is compounded if the IRR resources were being used by 489 third party ISPs to perform filtering on their peering circuits, 490 where typically no such policies are employed today for this very 491 reason. It is likely that the length of time at which IRRs mirror 492 changes will have to be dramatically reduced with a corresponding 493 reduction in time at the ISPs that, in turn, need to evaluate whether 494 changes in a set of IRR resources need to get reflected in updated 495 router policies that will get pushed out to ASBR's, connected to 496 peering circuits, on their network. 498 6. Historical BGP Protocol Limitations 500 As mentioned previously, after a resource holder made changes to 501 their resources in an IRR, those changes would automatically be 502 distributed to other IRRs, ISPs would then learn of those changes, 503 generate a new BGP policies, and then apply those to the appropriate 504 subset of routers in their ASes. However, in the past, one 505 additional step is necessary in order to have any of those new BGP 506 policies take effect in the Control Plane and to allow/deny the 507 updated resource from a customer to their ISP and from their 508 immediately upstream ISP to the ISP's peers. It was necessary (often 509 manually) to actually induce BGP on each router to apply the new 510 policy within the Control Plane, thus learning of a newly announced/ 511 changed IP prefix (or, dropping a deleted IP prefix). Unfortunately, 512 most of these methods were not only highly operationally impactful, 513 but also affected traffic forwarding to IP destinations that were 514 unrelated to the most recent changes to the BGP policy. 516 Historically, a customer would have to (re-)announce the new IP 517 prefix toward their ISP, but only after the ISP had placed the new 518 BGP policies into affect. Alternatively, the ISP would have to reset 519 the entire PE-CE eBGP session either by: a) bouncing the entire 520 interface toward the customer, (e.g.: shutdown/no shutdown); or, b) 521 clearing the eBGP session toward the customer, (e.g.: clear ip bgp 522 neighbor >IP address of CE router<). The latter two cases were, of 523 course, the most highly impactful and thus could generally only be 524 performed off-hours during a maintenance window. 526 Once the new IP prefix has been successfully announced by the 527 customer and permitted by the newly updated policy at the ISP's PE's 528 (attached to that customer), it would be propagated to that ISP's 529 ASBR's, attached to peers, at the perimeter of the ISP network. 530 Unfortunately, if those peers had either not yet learned of the 531 changes to resources in the IRR or pushed out new BGP policies (and, 532 reset their BGP sessions immediately afterward) incorporating those 533 changes, then the newly announced route would also get dropped at the 534 ingress ASBR's of the peers. 536 Ultimately, either of the two scenarios above further lengthen the 537 effective time it would take for changes in IRR resources to take 538 affect within BGP in the network. Fortunately, BGP has been enhanced 539 over the last several years in order that changes within BGP policy 540 will take affect without requiring a service impacting reset of BGP 541 sessions. Specifically, BGP soft-reconfiguration [REF?] and, later, 542 Route Refresh Capability for BGP-4 [RFC2918] were developed so that 543 ISPs, or their customers, could induce BGP to apply a new policy 544 while leaving both the existing eBGP session active as well as 545 (unaffected) routes active in both the Loc-RIB and, more importantly, 546 FIB of the router. Thus, using either of these mechanisms, an ISP or 547 its peers currently will deploy a newly generated BGP policy, based 548 on changes to resources within the IRR, and immediately trigger a 549 non-service impacting notification to the BGP process to have those 550 changes take affect in their Control Plane, either allowing a new IP 551 prefix to be announced or an old IP prefix to be dropped. This 552 dramatically reduces the length of time after changes are propagated 553 throughout the IRRs to when those changes are actually operational 554 within BGP policy in an ISP's network. 556 7. Historical Limitations of Routers 558 7.1. Incremental Updates to Policy on Routers 560 Routers in the mid 1990's rarely supported incrementally updatable 561 prefix filters for BGP, and therefore, if new information was 562 received from a policy or internal configuration database that would 563 impact a policy applied to a given eBGP peer, the entire prefix list 564 or access list would need to be deleted and rewritten, compiled, and 565 installed. This was very tedious and commonly led to leaked routes 566 during the time when the policy was being rewritten, compiled, and 567 applied on a router. Furthermore, application of a new policy would 568 not automatically result in new ingress or egress reachability 569 advertisements, from that new policy, because routers at the time 570 would require a reset of the eBGP sessions for routing information to 571 be evaluated by the new policy. Of course, resetting of an eBGP 572 session had implications on traffic forwarding during the time the 573 eBGP session was reestablished and new routing information was 574 learned. 576 Routers now support the ability to perform incremental, and in situ, 577 updates to filter lists consisting of IP prefixes and/or AS_PATH's 578 that are used within an ingress or egress BGP policy. In addition, 579 routers also can apply those incremental updates to policy, with no 580 traffic disruption, using BGP soft-reconfiguration or BGP Route 581 Refresh, as discussed in the previous section. 583 7.2. Storage Requirements for Policy on Routers 585 Historically, routers had very limited storage capacity and would 586 have difficulty in storing an extremely large BGP policy on-board. 587 This was typically the result of router hardware vendors using an 588 extremely limited amount of NVRAM for storage of router 589 configurations. 591 Another challenge with historical router hardware was that writing to 592 NVRAM was extremely slow. Thus, when the router configuration had 593 changed as a result of, for example, updating a BGP policy to 594 accommodate changes in IRR resources, this would result in extremely 595 long times to write out these configuration changes and, sometimes, 596 due to bugs would result in loss of protocol keep-alives, thus 597 causing an impact to routing or forwarding of packets through the 598 platform. 600 The above limitations have largely been resolved with more modern 601 equipment from the last few years shipping with increasing amounts of 602 non-volatile storage such as: PCMCIA or USB Flash Cards, Hard Disk 603 Drives or Solid State Disk Drives. 605 However, as capacities and technologies have evolved on modern 606 routing hardware, so have the some of the scaling requirements of the 607 data. In some large networks, configuration growth has begun to 608 "pose challenges" [IEPG89_NTT]. While the enhancements of hardware 609 have overcome some historical limitations, evidence suggests that 610 further optimizations in configuration processing may be needed in 611 some cases. Some of the more recent operational issues include 612 scheduler slips and protracted commit times. This suggests that even 613 though many historical hurdles have been overcome, there are still 614 motivations to optimize and modernize IRR technologies. 616 7.3. Updating Configuration on Routers 618 Historically, there has not been a standardized network configuration 619 modeling language or an associated method to update router 620 configurations. When an ISP detected a change in resources within 621 the IRR, it would fashion a router vendor dependent BGP policy and 622 upload that to the router usually via the following method. 624 First, an updated BGP policy configuration 'snippet' is generated via 625 processes running on an offline server. Next, the operator uses 626 either telnet or SSH to login to the CLI of a target router and issue 627 router vendor dependent CLI commands that will trigger the target 628 router to fetch the new configuration snippet via TFTP, FTP or Secure 629 Copy (SCP) stored on the offline server. The target router will then 630 perform syntax checking on that configuration snippet and, if that 631 passes, merge that configuration snippet into the running 632 configuration of the router's Control Software. At this point, the 633 new BGP policy configuration 'snippet' is active within the Control 634 Plane of the router. One last step remains, which is the operator 635 will issue a CLI command to induce the router to perform a "soft 636 reset", via BGP soft-reconfiguration or BGP Route Refresh, of the 637 associated BGP session in order to trigger the router to apply the 638 new policy to routes learned from that BGP session without disrupting 639 traffic forwarding. 641 More recently, operators have the ability to use NETCONF/SSH (or, 642 TLS) from an offline server to push a BGP configuration 'snippet' 643 from an offline server toward a target router that has that 644 capability. However, this activity is still dependent on generating, 645 via the offline server, a router vendor dependent XML configuration 646 snippet that would get uploaded via SSH or TLS to the target router. 648 In the future, the ability to upload new Route Origin Authorization 649 (ROA) information may be provided from the RPKI to routers via the 650 RPKI-RTR [RFC6810] protocol. However, this will not allow operators 651 the ability to upload other configuration information such as BGP 652 policy information, such as AS_PATH's, BGP communities, etc. that 653 might be associated with that ROA information, like they can from IRR 654 generated BGP policies. 656 8. Summary 658 As discussed above, many of the problems that have historically 659 stifled IRR deployment have, themselves, become historical. However, 660 there are still real operational considerations that limit IRR usage 661 from realizing its full effectiveness. The potential for IRRs to 662 express inter-domain routing policy, and to allow relying parties to 663 learn policy, has always positioned it as a strong candidate to aid 664 the security postures of operators. However, while routing density 665 and complexity have grown, so have some of the challenges facing IRRs 666 (even today). Because of this state increase, the potential to model 667 all policies for all ASes in all routers may still remain illusive. 668 In addition, without an operationally deployed resource certification 669 framework that can tie policies to resource holders, there is a 670 fundamental limitation that still exists. 672 9. Security Considerations 674 One of the central concerns with IRRs is the ability of an IRR 675 operator to remotely influence the routing operations of an external 676 consumer. Specifically, if the processing of IRR contents can become 677 burdensome, or if the policy statements can be crafted to introduce 678 routing problems or anomalies, then operators may want to be 679 circumspect about ingesting contents from external parties. However, 680 in regards to routing anomalies, as mentioned in Section 4.1 a 681 resource certification framework should be used to address the 682 authorization issues of IRR statements to make attestations and 683 assertions. 685 Additionally, the external and systemic dependencies introduced by 686 IRRs and other such systems employed to inform routing policy, and 687 how tightly or loosely coupled those systems are to the global 688 routing system and operational networks, introduces additional 689 vectors that operators and system architects should consider when 690 evaluating attack surface and service dependencies associated with 691 those elements. These attributes and concerns are certainly not 692 unique to IRRs, and operators should evaluate the implications of 693 external systems and varying degrees of coupling and operational 694 buffers that might be installed in their environments. 696 10. IANA Considerations 698 There are no IANA considerations. 700 11. Acknowledgements 702 The editors would like to acknowledge and thank Chris Morrow, Jeff 703 Haas, and Wes George for their help and constructive feedback. 705 12. Informative References 707 [IEPG89_NTT] 708 "NTT BGP Configuration Size and Scope", IEPG 89 http:// 709 iepg.org/2014-03-02-ietf89/ietf89_iepg_jmauch.pdf. 711 [IRR_LIST] 712 Merit Network, Inc., "List of Routing Registries", List of 713 Routing Registries http://www.irr.net/docs/list.html. 715 [MERIT-IRRD] 716 Merit, "IRRd - Internet Routing Registry Daemon", 717 http://www.irrd.net. 719 [RC_HotNetsX] 720 Osterweil, E., Amante, S., McPherson, D., and D. Massey, 721 "The Great IPv4 Land Grab: Resource Certification for the 722 IPv4 Grey Market", 723 HotNets-X http://dl.acm.org/citation.cfm?id=2070574. 725 [RFC1787] Rekhter, Y., "Routing in a Multi-provider Internet", 726 RFC 1787, April 1995. 728 [RFC2622] Alaettinoglu, C., Villamizar, C., Gerich, E., Kessens, D., 729 Meyer, D., Bates, T., Karrenberg, D., and M. Terpstra, 730 "Routing Policy Specification Language (RPSL)", RFC 2622, 731 June 1999. 733 [RFC2725] Villamizar, C., Alaettinoglu, C., Meyer, D., and S. 734 Murphy, "Routing Policy System Security", RFC 2725, 735 December 1999. 737 [RFC2769] Villamizar, C., Alaettinoglu, C., Govindan, R., and D. 738 Meyer, "Routing Policy System Replication", RFC 2769, 739 February 2000. 741 [RFC2918] Chen, E., "Route Refresh Capability for BGP-4", RFC 2918, 742 September 2000. 744 [RFC6480] Lepinski, M. and S. Kent, "An Infrastructure to Support 745 Secure Internet Routing", RFC 6480, February 2012. 747 [RFC6810] Bush, R. and R. Austein, "The Resource Public Key 748 Infrastructure (RPKI) to Router Protocol", RFC 6810, 749 January 2013. 751 [RIPE2007-01] 752 RIPE NCC, "DRAFT: Autonomous System (AS) Number Assignment 753 Policies and Procedures", Foundation 754 Policy http://www.ripe.net/ripe/docs/ripe-452. 756 [RPKI_SIZING] 757 Osterweil, E., Manderson, T., White, R., and D. McPherson, 758 "Sizing Estimates for a Fully Deployed RPKI", Verisign 759 Labs Technical Report 1120005 version 2 http:// 760 techreports.verisignlabs.com/ 761 tr-lookup.cgi?trid=1120005&rev=2. 763 [TASRS] Osterweil, E., Amante, S., and D. McPherson, "TASRS: 764 Towards a Secure Routing System Through Internet Number 765 Resource Certification", Verisign Labs Technical Report 766 1130009 http://techreports.verisignlabs.com /tr- 767 lookup.cgi?trid=1130009&rev=1. 769 Authors' Addresses 771 Danny McPherson 772 Verisign, Inc. 774 Email: dmcpherson@verisign.com 776 Shane Amante 777 Level 3 Communications 778 1025 Eldorado Blvd 779 Broomfield, CO 80021 780 USA 782 Email: shane@level3.net 784 Eric Osterweil 785 Verisign, Inc. 787 Email: eosterweil@verisign.com 789 Larry J. Blunk 790 Merit Network, Inc. 792 Email: ljb@merit.edu 794 Dave Mitchell 795 Twitter, Inc. 797 Email: dave@twitter.com