idnits 2.17.1 draft-ietf-idr-bgp4-op-experience-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-03-19) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 13, 1996) is 10172 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '6' is defined on line 383, but no explicit reference was found in the text ** Obsolete normative reference: RFC 1264 (ref. '1') (Obsoleted by RFC 4794) == Outdated reference: A later version (-26) exists of draft-ietf-idr-bgp4-02 ** Obsolete normative reference: RFC 1657 (ref. '4') (Obsoleted by RFC 4273) ** Obsolete normative reference: RFC 1519 (ref. '5') (Obsoleted by RFC 4632) ** Downref: Normative reference to an Informational RFC: RFC 1773 (ref. '6') ** Downref: Normative reference to an Informational RFC: RFC 1774 (ref. '7') Summary: 14 errors (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Inter-Domain Routing Working Group Paul Traina 2 INTERNET DRAFT cisco Systems 3 May 13, 1996 5 Operational Experience with the BGP-4 protocol 7 Status of this Memo 9 This memo provides information for the Internet community. It does 10 not specify an Internet standard. Distribution of this memo is 11 unlimited. 13 This document is an Internet Draft. Internet Drafts are working 14 documents of the Internet Engineering Task Force (IETF), its Areas, 15 and its Working Groups. Note that other groups may also distribute 16 working documents as Internet Drafts. 18 Internet Drafts are draft documents valid for a maximum of six 19 months. Internet Drafts may be updated, replaced, or obsoleted by 20 other documents at any time. It is not appropriate to use Internet 21 Drafts as reference material or to cite them other than as a "working 22 draft" or "work in progress". 24 Introduction 26 The purpose of this memo is to document how the requirements for 27 advancing a routing protocol to Full Standard have been satisfied by 28 Border Gateway Protocol version 4 (BGP-4). This report documents 29 experience with BGP. It is the second of two reports on the BGP 30 protocol. 32 The remaining sections of this memo document how BGP satisfies 33 General Requirements specified in Section 3.0, as well as the 34 Requirements for Full Standard as specified in Section 6.0 of the 35 "Internet Routing Protocol Standardization Criteria" document [1]. 37 Please send comments to bgp@ans.net. 39 Documentation 41 BGP is an inter-autonomous system routing protocol designed for 42 TCP/IP networks. Version 1 of the BGP protocol was published in RFC 43 1105. Since then BGP Versions 2, 3, and 4 have been developed. 44 Version 2 was documented in RFC 1163. Version 3 is documented in RFC 45 1267. The changes between versions 1, 2 and 3 are explained in 46 Appendix 2 of [2]. All of the functionality that was present in the 47 previous versions is present in version 4. 49 BGP version 2 removed from the protocol the concept of "up", "down", 50 and "horizontal" relations between autonomous systems that were 51 present in version 1. BGP version 2 introduced the concept of path 52 attributes. In addition, BGP version 2 clarified parts of the 53 protocol that were "under-specified". 55 BGP version 3 lifted some of the restrictions on the use of the 56 NEXT_HOP path attribute, and added the BGP Identifier field to the 57 BGP OPEN message. It also clarifies the procedure for distributing 58 BGP routes between the BGP speakers within an autonomous system. 60 BGP version 4 redefines the (previously class-based) network layer 61 reachability portion of the updates to specify prefixes of arbitrary 62 length in order to represent multiple classful networks in a single 63 entry as discussed in [5]. BGP version 4 has also modified the AS- 64 PATH attribute so that sets of autonomous systems, as well as 65 individual ASs may be described. In addition, BGP version 4 has re- 66 described the INTER-AS METRIC attribute as the MULTI-EXIT 67 DISCRIMINATOR and added new LOCAL-PREFERENCE and AGGREGATOR 68 attributes. 70 Possible applications of BGP in the Internet are documented in [3]. 72 The BGP protocol was developed by the IDR Working Group of the 73 Internet Engineering Task Force. This Working Group has a mailing 74 list, bgp@ans.net, where discussions of protocol features and 75 operation are held. The IDR Working Group meets regularly during the 76 quarterly Internet Engineering Task Force conferences. Reports of 77 these meetings are published in the IETF's Proceedings. 79 MIB 81 A BGP-4 Management Information Base has been published [4]. The MIB 82 was written by Steve Willis (Bay), John Burruss (Bay), and John Chu 83 (IBM). 85 Apart from a few system variables, the BGP MIB is broken into two 86 tables: the BGP Peer Table and the BGP Received Path Attribute Table. 87 The Peer Table reflects information about BGP peer connections, such 88 as their state and current activity. The Received Path Attribute 89 Table contains all attributes received from all peers before local 90 routing policy has been applied. The actual attributes used in 91 determining a route are a subset of the received attribute table. 93 Security Considerations 95 BGP provides flexible and extendible mechanism for authentication and 96 security. The mechanism allows the support of schemes with various 97 degree of complexity. All BGP sessions are authenticated based on 98 the BGP Identifier of a peer. In addition, all BGP sessions are 99 authenticated based on the autonomous system number advertised by a 100 peer. As part of the BGP authentication mechanism, the protocol 101 allows the carriage of an encrypted digital signature in every BGP 102 message. All authentication failures result in the sending of a 103 NOTIFICATION message and immediate termination of the BGP connection. 105 Since BGP runs over TCP and IP, BGP's authentication scheme may be 106 augmented by any authentication or security mechanism provided by 107 either TCP or IP. 109 However, since BGP runs over TCP and IP, BGP is vulnerable to the 110 same denial of service or authentication attacks that are present in 111 any other TCP based protocol. 113 One method for improving the security of TCP connections for use with 114 BGP has been documented in [7]. 116 Operational experience 118 This section discusses operational experience with BGP-4, which has 119 involved the use of several independent implementations of BGP. 121 BGP has been used in the Internet since 1989, BGP-4 since 1993. This 122 use has involved at least three independant implementations. 123 Production use of BGP has included utilization of all significant 124 features of the protocol. The present production environment, where 125 BGP is used as the inter-autonomous system routing protocol, is 126 highly heterogeneous. 128 This environment includes link bandwidths which vary from from 28 129 Kbits/sec to 150 Mbits/sec. 131 Routers which run BGP range from relatively low-performanced IBM 132 PC/RTs to those equiped with high performance RISC based CPUs, and 133 includes both the special purpose routers and the general purpose 134 workstations running UNIX. 136 Topologies in the production environment vary from the very sparse 137 (e.g. the spanning tree of the ICM network) to quite dense (e.g. 138 Sprintlink, Alternet, and MCI backbones). 140 At the time of this writing BGP-4 is used as an inter-autonomous 141 system routing protocol between all significant autonomous systems, 142 including, but by all means not limited to: Alternet, ANS, Ebone, 143 ICM, IIJ, MCI, and Sprint. The smallest know backbone consists of 144 one BGP speaker, whereas the largest contains nearly 120 BGP 145 speakers. All together, there are several thousand known BGP 146 speaking routers. 148 BGP is used both for the exchange of routing information between a 149 transit and a stub autonomous system, and for the exchange of routing 150 information between multiple transit autonomous systems. There is no 151 distinction between sites historically considered backbones vs those 152 considered "local" networks. 154 Within most transit networks, BGP is used as the exclusive carrier of 155 exterior routing information. At the time of this writing, few sites 156 propogate all exterior routing information into their interior 157 routing protocols. 159 The full set of exterior routes that is carried by BGP in the 160 production Internet is well over 30,000 distinct classless prefixes 161 representing several times that number of connected networks. 163 Operational experience with BGP-4 has exercised all basic features of 164 the protocol, including authentication, routing loop suppression and 165 the new features of BGP-4: enhanced metrics and route aggregation. 167 Bandwidth consumed by BGP has been measured at the interconnection 168 points between CA*Net and T1 NSFNET Backbone. The results of these 169 measurements were presented by Dennis Ferguson during the Twenty- 170 first IETF, and are available from the IETF Proceedings. These 171 results showed clear superiority of BGP over EGP when protocol 172 bandwidth consumption is compared. Observations on the CA*Net by 173 Dennis Ferguson, and on the T1 NSFNET Backbone by Susan Hares 174 confirmed clear superiority of the BGP protocol family as compared 175 with EGP in the area of CPU requirements. 177 Migration to BGP version 4 179 On multiple occasions some members of IETF expressed concern about 180 the migration path from classful protocols to classless protocols 181 such as BGP-4. 183 BGP-4 was rushed into production use on the Internet because of the 184 exponential growth of routing tables and the increase of memory and 185 CPU utilization required by BGP. As such, migration issues that 186 normally would have stalled deployment were cast aside in favor of 187 pragmatic and intelligent deployment of BGP-4 by network operators. 189 There was much discussion about creating "prefix exploders" which 190 would enumerate individual class-based networks of CIDR allocations 191 to BGP-3 speaking routers, however a cursory examination showed that 192 this would vastly hasten the requirement for more CPU and memory 193 resources for these older implementations. There would be no way 194 internal to BGP to differentiate between known used destinations and 195 the unused portions of advertised CIDR allocations. 197 The migration path chosen by the operators was known as "CIDR, 198 default, or die." 200 To test BGP-4 operation, a virtual "shadow" Internet was created by 201 linking Alternet, Ebone, ICM, and cisco over GRE based tunnels. 202 Experimentation was done with actual live routing information by 203 establishing BGP version 3 connections with the production networks 204 at those sites. This allowed extensive regression testing before 205 deploying BGP-4 on production equipment. 207 After testing using the shadow network, BGP-4 implementations were 208 deployed on production transit networks at those sites. BGP-4 209 capable routers negotiated BGP-4 connections and inter-operated with 210 other sites by speaking BGP-3. Several test aggregate routes were 211 injected into this network in addition to classful destinations for 212 compatibility with BGP-3 speakers. 214 At this point, the shadow-Internet was re-chartered as an 215 "operational experience" network. Tunnel connections were 216 established with most major transit service operators so that 217 operators could gain some understanding of how the introduction of 218 aggregate destinations would affect routing. 220 After being satisfied with the initial deployment of BGP-4, a number 221 of sites chose to withdraw their class-based advertisements and rely 222 only on their CIDR aggregate advertisements. This supplied 223 motivation for transit providers who had not migrated to either do 224 so, accept a default route, or lose connectivity to several popular 225 destinations. 227 Currently, BGP-4 is the default choice for carrying exterior routing 228 information in the production Internet. 230 Metrics 232 BGP version 4 re-defined the INTER-AS metric as a MULTI-EXIT- 233 DISCRIMINATOR. This value may be used in the tie breaking process 234 when selecting a preferred path to a given address space. The "MED" 235 is intended to be used only when comparing paths received from 236 different external peers in the same AS to indicate the preference of 237 the originating AS. The MED was purposely designed to be a "weak" 238 metric that would only be used late in the best-path decision 239 process. 241 The IDR working wanted to insure that any metric specified by a 242 remote operator would only affect routing in a local AS if no other 243 preference was specified. A paramount goal of the design of the MED 244 was insure that neighboring autonomous systems could not "shed" or 245 "absorb" traffic for destinations that they advertise. 247 The LOCAL-PREFERENCE attribute was added so a local operator could 248 easily configure a policy that overrode the standard best path 249 determination mechanism without requiring the manual configuration on 250 every router in the AS. 252 One shortcoming in the BGP-4 specification was a suggestion for a 253 default value of LOCAL-PREFERENCE to be assumed if none was provided. 254 Defaults of 0 or the maximum value each have range limitations, so a 255 common default would have aided in the interoperation of different 256 BGP implementations in the same AS (since LOCAL-PREFERENCE is a local 257 administration knob, there is no interoperability drawback across AS 258 boundaries). 260 Another area where more exploration is required is a method whereby 261 an originating or remote AS may influence the best path selection 262 process. For example, a dual-connected site may select one AS as a 263 primary transit service provider and have one as a backup. 265 In a topology where the multiple transit service providers connect to 266 additional autonomous systems, there is no formal mechanism for 267 indicating a path selection preference should a remote autonomous 268 system wish to respect that preference. 270 In BGP implementations where the total length of the sequence 271 portions of the AS path attribute may be used as part of the path 272 selection criteria, one practice in use today is to prepend 273 additional copies of the originator's autonomous system number to the 274 AS path. 276 /--- transit A ---\ 277 / \ 278 end-customer transit C---- 279 109 \ / 280 \--- transit B ---/ 282 Using the example above, if the "end customer" advertises routes 283 originating in its autonomous system as having an AS path of "109" to 284 transit A, and a path of "109 109" to transit B, transit provider C 285 may be influenced by the difference in AS sequence lengths and prefer 286 the path via transit A. 288 There has been some discussion of the creation of an optional 289 transitive attribute which would represent a sequence of (AS, 290 preference) entries to indicate a preference value for a given path. 291 Cooperating ASs would chose traffic based upon comparison of 292 "interesting" portions of this sequence according to local routing 293 policy. 295 Additional suggestions have been made suggesting a less flexible 296 "destination provider selection" attribute to indicate desired 297 preferences. 299 While protecting a given autonomous system's routing policy is of 300 paramount concern, avoiding extensive hand configuration of routing 301 policies needs to be examined more carefully in future protocol 302 varients. 304 Internal BGP in large autonomous systems 306 While not strictly a protocol issue, one other concern has been 307 raised by network operators who need to maintain autonomous systems 308 with a large number of peers. Each speaker peering with an external 309 router is responsible for propagating reachability and path 310 information to all other transit and border routers within that AS. 311 This is typically done by establishing internal BGP connections to 312 all transit and border routers in the local AS. 314 This practice leads to an O(n^2) mesh of TCP connections and requires 315 some method of configuring and maintaining those connections. BGP 316 does not regulate how this information is to be propagated, so 317 alternatives, such as injecting BGP attribute information into the 318 local IGP have been suggested. Also, internal BGP "route 319 reflectors", and "autonomous system confederation" mechanisms have 320 been implemented and demonstrate a significant improvement in 321 configuration, memory and CPU requirements necessary to convey 322 information to all other BGP peers in an autonomous system. 324 Internet Dynamics 326 As discussed in [7], the driving force in CPU and bandwidth 327 utilization is the dynamic nature of routing in the Internet. As the 328 net has grown, the number of changes per second has increased. We 329 receive some level of damping when more specific reachability 330 information is aggregated into larger blocks, however this isn't 331 sufficient. 333 At least one current implementation of BGP provides route update 334 dampening that includes routing hysteresis. This allows fast 335 convergence for routes that flap relatively infrequently while 336 suppressing instabilities caused by frequently flapping paths. 337 Operational experience in the Internet shows that large-scale 338 deployment of this dampening technique has proven to be highly 339 beneficial for the stability of the routing system. 341 Acknowledgments 343 The BGP-4 protocol has been developed by the IDR/BGP Working Group of 344 the Internet Engineering Task Force. I would like to express thanks 345 to Yakov Rekhter for providing RFC 1266 from which this document is 346 based. I'd like to thank Yakov Rekhter, John Hawkinson, and Vince 347 Fuller for the review of this document as well as constructive and 348 valuable comments. This report is based on the initial work of Peter 349 Lothberg (STUPI), Andrew Partan (UUNET), and several others. 351 Author's Address: 353 Paul Traina 354 cisco Systems, Inc. 355 170 W. Tasman Dr. 356 San Jose, CA 95134 357 pst@cisco.com 359 References 361 [1] RFC1264 362 Hinden, R., "Internet Routing Protocol Standardization Criteria", 363 October 1991. 365 [2] draft-ietf-idr-bgp4-02.txt 366 Rekhter, Y., and Li, T., "A Border Gateway Protocol 4 (BGP-4)", 367 January 1996. 369 [3] RFC1772 370 Rekhter, Y., and P. Gross, Editors, "Application of the Border 371 Gateway Protocol in the Internet", March 1995. 373 [4] RFC1657 374 S. Willis, J. Burruss, J. Chu, "Definitions of Managed Objects for 375 the Fourth Version of the Border Gateway Protocol (BGP-4) using 376 SMIv2", July 1994. 378 [5] RFC1519 379 Fuller V.; Li. T; Yu J.; Varadhan, K., "Classless Inter-Domain 380 Routing (CIDR): an Address Assignment and Aggregation Strategy", 381 September 1993. 383 [6] RFC1773 384 Traina P., "Experience with the BGP-4 protocol." March 1995. 386 [7] RFC1774 387 Traina P., "BGP Version 4 Protocol Analysis", March 1995.