idnits 2.17.1 draft-ietf-idr-symm-multi-prov-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-20) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 9 instances of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 256 has weird spacing: '...rs) and defau...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 1995) is 10537 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '2' is defined on line 482, but no explicit reference was found in the text == Unused Reference: '5' is defined on line 492, but no explicit reference was found in the text == Unused Reference: '6' is defined on line 495, but no explicit reference was found in the text ** Obsolete normative reference: RFC 1771 (ref. '1') (Obsoleted by RFC 4271) -- Possible downref: Non-RFC (?) normative reference: ref. '3' -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' ** Downref: Normative reference to an Informational RFC: RFC 1787 (ref. '6') -- Possible downref: Non-RFC (?) normative reference: ref. '7' Summary: 11 errors (**), 0 flaws (~~), 5 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT Enke Chen 3 Tony Bates 4 Expires in six months MCI 5 June 1995 7 Current Practice of Implementing 8 Symmetric Routing and Load Sharing 9 in the Multi-Provider Internet 10 12 Status of this Memo 14 This document is an Internet Draft. Internet Drafts are working 15 documents of the Internet Engineering Task Force (IETF), its Areas, 16 and its Working Groups. Note that other groups may also distribute 17 working documents as Internet Drafts. 19 Internet Drafts are draft documents valid for a maximum of six 20 months. Internet Drafts may be updated, replaced, or obsoleted by 21 Internet Drafts are draft documents valid for a maximum of six 22 months. Internet Drafts may be updated, replaced, or obsoleted by 23 other documents at any time. It is not appropriate to use Internet 24 Drafts as reference material or to cite them other than as a "working 25 draft" or "work in progress". 27 Please check the I-D abstract listing contained in each Internet 28 Draft directory to learn the current status of this or any other 29 Internet Draft. 31 Abstract 33 In the current multi-provider Internet, it is common for an entity to 34 have multiple service providers. Symmetric routing becomes 35 increasingly important for various reasons. This memo documents and 36 analyzes the current practice in implementing symmetric inter-domain 37 routing using BGP for several representative topologies of Internet 38 connections. 40 1. Introduction 42 In the multi-provider Internet, it is common for an entity to have 43 multiple connections to the Internet. For example, 44 o A Regional Service Provider (RSP) may be connected to multiple 45 transit Internet Service Providers (ISPs). 47 o A service subscriber may be connected to multiple RSPs or ISPs. 49 o Subscribers of different providers may wish to backup each 50 other. 52 These connections would provide for the capability of load sharing, 53 path diversification and backup. The Internet is a mesh of ISPs, 54 RSPs and service subscribers and is generally sparsely connected. 56 Symmetric routing is generally preferred as it facilitates problem 57 resolution, and provides for better resource (especially network 58 capacity) planning and utilization. Routing symmetry is also desir- 59 able in achieving optimal traffic flow in terms of reliability, delay 60 character, cost and other QoS metrics. Several applications such as 61 NTP, RSVP and MBONE rely upon routing symmetry. In the multi- 62 provider Internet, routing asymmetry, especially at the inter-domain 63 level, may have serious economic and legal ramifications. 65 This paper presents several representative topologies of Internet 66 connection and their inter-domain routing requirements. It then doc- 67 uments and analyzes the current practice in implementing symmetric 68 inter-domain routing in these cases using BGP. 70 This paper assumes that in general an ISP treats other ISPs equally 71 (in terms of the "local_pref" parameter) in the route selection pro- 72 cess. It also assumes that the following order of preference is fol- 73 lowed for the purpose of route selection: first the "local_pref" 74 parameter, followed by the shortest AS-path, the MED, and the IGP 75 metric. 77 It is noted that the length of the AS-path has not been specified in 78 the BGP document [1] as a route selection criteria. However, it has 79 been included in more than one implementations, and has been widely 80 used as such. 82 2. Internet Connection and Routing 84 The Internet is a mesh of transit Internet Service Providers (ISPs), 85 Regional Service Providers (RSPs), and service subscribers. In gen- 86 eral this mesh is rather sparsely connected with loose hierarchy. In 87 the multi-provider Internet, a good routing plan for an entity (i.e., 88 autonomous system) requires good understanding of its internal net- 89 work topology, its connection to direct providers (and neighboring 90 ASs), and its path to the major interconnection points (or network 91 access points, NAPs). 93 In this section, we present several typical topologies of Internet 94 connections, and their inter-domain routing requirements. Although 95 these cases are not meant to be exhaustive, they are expected to 96 cover the vast majority of Internet connection topologies. 98 2.1 An Entity with a Single Direct Provider 100 +-----+ +-----+ +-----+ 101 | ISP | | ISP | | ISP | 102 +-----+ +-----+ +-----+ 103 | | | 104 +-----+ +-----+ +-----+ 105 | AS1 | | RSP | | RSP | 106 +-----+ +-----+ +-----+ 107 | 108 +-----+ 109 | AS1 | 110 +-----+ 112 (a) (b) (c) 114 Figure 1 116 The routing is always symmetric at the inter-domain level. Routing 117 policies can be achieved using the current version of BGP. AS1 can 118 either take full routing or use default. 120 2.2 Backup of Entities with Different Direct Providers 122 Several topologies are shown in Figure 2. Both AS1 and AS2 have 123 their direct provider(s), and they would like to backup each other. 124 That is, if the link between AS1 and its direct provider is down, the 125 link between AS1 and AS2 would be used to reach the Internet. 127 +-----+ +------+ +------+ 128 | ISP | | ISP3 | | ISP3 | 129 +-----+ +------+ +------+ 130 / \ / \ / \ 131 / \ / (NAP) \ / (NAP) \ 132 +-----+ +-----+ +------+ +------+ +------+ +------+ 133 | AS1 |----| AS2 | | ISP1 |------| ISP2 | | ISP1 |-------| ISP2 | 134 +-----+ +-----+ +------+ +------+ +------+ +------+ 135 | | | | 136 +------+ +------+ +------+ +------+ 137 | AS1 |------| AS2 | | RSP1 | | RSP2 | 138 +------+ +------+ +------+ +------+ 139 | | 140 +------+ +------+ 141 | AS1 |------| AS2 | 142 +------+ +------+ 144 (a) (b) (c) 146 Figure 2 148 Note that in Figures 2(a)-2(b), AS1 and AS2 could be RSPs. 150 In all cases of Figure 2, in order to provide for backup, AS1 shall 151 permit the acceptance of AS2's routes from both AS2 and AS1's direct 152 provider, and permit their announcements to its direct providers. 153 Similar configuration is required for AS2. 155 There are two common routing policies depending upon how the link 156 between AS1 and AS2 is used. 158 Policy 1: Used solely as a backup link 160 The routing policy can be implemented by coordinating AS-based 161 "local_pref" values between neighboring ASs. 163 In all cases of Figure 2, the "local_pref" value for the peer of 164 AS1 or AS2 with its direct provider shall be higher than that for 165 the peer between AS1 and AS2. Either full routing or partial 166 routing can be configured. 168 For example, in Figure 2(a), AS1 can take full routing from ISP 169 and AS2. An alternative is for AS1 to take only AS2's routes 170 from ISP and AS2, and configure default routes (with different 171 weights) at its border routers and then propagate them into its 172 own AS (via, e.g., iBGP). The ISP needs to make sure that the 173 "local_pref" values are equal for the peers with AS1 and AS2 so 174 that the shorter AS-path would be selected. 176 Policy 2: Used for traffic between AS1 and AS2, and as backup in gen- 177 eral 179 In general this routing policy can be implemented by coordinating 180 AS-based "local_pref" values among the neighboring ASs and direct 181 providers. 183 In Figure 2(a), equal "local_pref" values could be configured for 184 all the peers. Then the length of AS path would be used as tie- 185 breaker in the route selection. AS1 can either take full routing 186 from AS2 and its direct provider. It can also choose to take only 187 AS2's routes from its direct provider and AS2, and configure 188 default routes (with a different weights) at its border routers 189 and then propagate them into its own AS (via, e.g., iBGP). Simi- 190 lar configuration for AS2. 192 In Figures 2(b)-(c), AS1 can either take full routing from AS2 and 193 its direct provider, and configure the "local_pref" parameter so 194 that traffic to AS2 prefers the AS1 - AS2 link over the link to 195 its direct provider. AS1 can choose to take only AS2's routes from 196 its direct provider and AS2, and configure default routes (with 197 different weight) at its border routers and then propagate them 198 into its own AS (via, e.g., iBGP). Similar configuration for AS2. 200 AS1's direct provider (and possible its ISP) needs to configure 201 the "local_pref" parameter so that traffic to AS2 does not prefer 202 the link to AS1. Similar configuration is required for AS2's 203 direct provider. 205 2.3 An Entity with Multiple Direct Providers 207 As shown in Figure 3, AS1 has two direct providers. X and Y are 208 routes of AS1. Note that AS1 could be an RSP. 210 +------+ +-----+ +------+ 211 | ISP3 | | ISP | | ISP3 | 212 +------+ +-----+ +------+ 213 / \ / \ / \ 214 / (NAP) \ / \ / (NAP)\ 215 +------+ +------+ +------+ +------+ +------+ +-----+ 216 | ISP1 |------| ISP2 | | RSP1 | | RSP2 | | ISP1 |-----| ISP2| 217 +------+ +------+ +------+ +------+ +------+ +-----+ 218 \ / \ / | | 219 \L1 /L2 \L1 /L2 | | 220 +------+ +------+ +------+ +-----+ 221 | AS1 | | AS1 | | RSP1 | | RSP2| 222 |X Y| |X Y| +------+ +-----+ 223 +------+ +------+ \ / 224 \L1 /L2 225 +------+ 226 | AS1 | 227 |X Y| 228 +------+ 230 (a) (b) (c) 232 Figure 3 234 Depending upon the quality of these links and the internal network 235 topology of the AS, there are several common routing policies. 237 Policy 1: One link is used as primary, the other as pure backup 239 This policy is common when the quality of these links differ dra- 240 matically. 242 This policy can be implemented by coordinating AS-based 243 "local_pref" values between the entity and its direct providers. 244 The AS can either take full routing or use default routes. In the 245 case of default, each border router can configure a default route 246 and then propagate it into the AS (via, e.g., iBGP). 248 Policy 2: Each link is used for traffic with the respective direct 249 provider. In general one link is used as primary, and the other as 250 backup. 252 If the traffic between AS1 and its direct providers (and their 253 customers) shall take the direct link, AS1 needs to be configured: 255 o either with partial routing (only routes of the direct 256 providers and their customers) and defaults with different 257 weights. 259 o or with full routing and configure AS-based "local_pref" val- 260 ues. 262 The difficulty is that the indirect providers (e.g., ISP3 in 263 Figure 3(a)) may have to be involved to achieve symmetric rout- 264 ing. More specifically, 266 o In Figure 3(a), ISP3 would receive routes X and Y from both 267 ISP1 and ISP2 with identical length of AS paths. In order 268 for L1 to be favored by ISP3 to AS1, ISP1 would need to 269 manipulate the AS-path length, which is discussed in Section 270 3.1. Another approach is for ISP3 to configure AS-based 271 "local_pref" parameter, which certainly does not scale well 272 as there are many ISPs at an NAP. In addition, it is almost 273 impossible to do as AS1 is not a customer of ISP3. 275 o In Figure 3(b), ISP would receive routes X and Y from both 276 RSP1 and RSP2 with identical length of AS paths. Either RSP1 277 needs to manipulate the AS-path length, or ISP needs to con- 278 figure AS-based "local_pref" parameter. 280 o In Figure 3(c), ISP1 would receive routes X and Y from both 281 RSP1 and ISP2. In order for traffic from ISP1 to AS1 to 282 favor the ISP1 - ISP2 link, either RSP1 needs to manipulate 283 the AS-path length, or ISP1 needs to configure AS-based 284 "local_pref" parameter. 286 Another problem is that it is difficult for AS1 to implement 287 "full routing", as AS1 needs to update the AS list for the 288 "local_pref" parameter each time its direct providers acquire a 289 new AS as a customer. Nevertheless, some entities still prefer 290 full routing. 292 Policy 3: Partial load-sharing among these links 294 That is, the direct link is used for traffic between AS1 and its 295 direct providers including its customers. However, the closest 296 exit point would be taken for traffic beyond these direct 297 providers and their customers. For example, in Figure 3(a) traffic 298 between AS1 and ISP1 (and its customers) would use the direct link 299 between AS1 and ISP1; traffic between AS1 and ISP2 (and its cus- 300 tomers) would use the direct link between AS1 and ISP2. For traf- 301 fic destined to ISP3, either ISP1 or ISP2 would be used depending 302 on where the traffic is originated in AS1. 304 AS1 can take full routing. It can also take partial routing 305 (routes of direct providers and their customers), and configure 306 equal-weight default routes at its border routers and propagate 307 them into its AS. 309 The problem is how to make sure the return traffic from a 4th 310 party (e.g., ISP3 in Figure 3(a)) is symmetric. 312 Policy 4: Complete load-sharing among these links 314 That is, each network in AS1 sends packets to the closer (in terms 315 of internal route preference) border router that peers with a 316 direct providers. The return traffic is expected to take a sym- 317 metric path. For example, in Figure 3(a) a packet, which is orig- 318 inated from network X and is destined outside AS1, would be for- 319 warded to ISP1, even when the destination is in ISP2. 321 The simpler approach is for AS1 to use default. That is, AS1 322 would first configure default route at each connection to a 323 provider and propagate (e.g., via iBGP) them into the AS. Then, 324 each network in AS1 choose the closest exit point (determined by 325 IGP metric). The problem is how to make sure the return traffic 326 to X and Y takes symmetric paths. Currently this is achieved by 327 manipulating the AS-path length or other approaches detailed in 328 the following section. 330 If AS1 still prefers to take full routing, more coordination would 331 be required for using the AS path manipulation or other techniques 332 as described in Section 3. 334 3. Current Practices 336 Currently there are mainly three approaches to implement Polices 2-4 337 for Figure 3. This section presents analysis and critique of these 338 approaches. Without loss of generality, Figure 3(a) is used as an 339 example. 341 3.1 Manipulation of AS Path Length 343 Although the length of the AS path was not specified in [1] as a 344 parameter in the route selection process, it has been widely used as 345 such. 347 Some router software offers the ability of prepending AS numbers to 348 the AS path for the purpose of influencing the route selection. Here 349 is how the feature can be used. First, AS1 categorizes all of its 350 routes and prepends an AS number (either its own AS or a different AS 351 number): 353 AS1 Prepend AS Path 354 Route To ISP1 To ISP2 ISP1 ISP2 355 ===== ======= ======= ======= ====== 356 X AS1 AS1 AS1 AS1 357 Y AS1 AS1 AS1 AS1 359 In general the different AS paths can be used by ISP1 and ISP2 to 360 configure AS-based "local_pref" values to implement the desired rout- 361 ing policy. The "local_pref" configuration would not be necessary if 362 there are sufficient number of ASs inserted. 364 With this approach the AS that originates the preference has full 365 control, and only that AS needs to manipulate the AS path on a per- 366 route basis. 368 The drawbacks of this approach includes: 370 o It extends the AS path with superfluous information. In par- 371 ticular, the superfluous information in the AS path would be 372 propagated upstream and to the whole Internet. 374 o The number of ASs that need to be preprended is in general 375 proportional to the number of direct providers. 377 o Compatibility with other BGP implementation may be a problem. 379 3.2 Splitting AS 381 This approach requires an AS to be split into multiple ASs and run 382 external BGP between these ASs (possibly with MEDs configured for 383 load balancing among these ASs). Then the cases in Figure 3 can be 384 reduced to the cases in Figure 2, which have been discussed in Sec- 385 tion 2. 387 This is probably the cleanest approach the current BGP version can 388 offer. However, there is a great deal of reluctance in using this 389 approach. Practical problems with this approach include: 391 o It does not work with the partial load-sharing case where the 392 connections to multiple providers originate from one router. 394 o Splitting ASs and having them maintained could be quite 395 involved depending upon the internal network topology. 397 o Extra AS numbers are required [7]. It would be necessary to 398 apply for AS numbers at the InterNIC. 400 o The number of split ASs is proportional to the number of 401 direct providers. 403 o Wasting of AS numbers. The exhaustion of the AS number space 404 could become real with the ever-increasing number of such 405 needs. 407 o The increased number of ASs would add complexity to the Inter- 408 net topology, and therefore complicate problem resolution. 410 o An AS number has been traditionally tied to an organization. 411 Splitting AS means loss of coherence for some customers. 413 3.3 NLRI-based Preference Specification 415 This is the approach that has been used for the NSFNET. Here is how 416 it is done with Figure 3(a). ISP1 and ISP2 configure net-based pref- 417 erence on their routers, according to preference provided by AS1. 418 For example, for route X, ISP1 would configure higher preference for 419 its direct link with AS1, and lower preference for its direct link 420 with ISP2. 422 This approach requires NRLI-based customization with the direct 423 providers and sometimes indirect providers as well as the originating 424 AS. The NSFNET configuration experience has shown that this approach 425 requires non-trivial administrative coordination and full topology 426 information. In addition, it places a burden on routers with limited 427 memory capacity. For these reasons, the NLRI-based preference con- 428 figuration should be avoided at the provider level if possible. 429 Instead, such a configuration should be pushed as close to the origi- 430 nating AS as possible. 432 3.4 Perfect Aggregation and Addressing 434 In the cases that all routes in the AS are covered by an aggregate 435 and address assignment is completely consistent with the network 436 topology, the rule of the longest prefix match can be used to help 437 achieve routing symmetry and load sharing. That is, a portion of the 438 aggregate, along with the aggregate itself, can be announced to one 439 direct provider. The remaining portion of the aggregate, along with 440 the aggregate itself, can be announced to the other direct provider. 442 It does not work with the partial load-sharing case where the connec- 443 tions to multiple providers originate from one router. More impor- 444 tantly, the requirement of this approach is not likely to be met in 445 practice. So, this approach is listed just for the sake of complete- 446 ness. 448 4. Discussion 450 As has been illustrated in Section 3, it is not easy to implement 451 routing symmetry and load sharing for an entity with multiple direct 452 providers using the current functionality of BGP. There are many 453 drawbacks with the current practice of implementation. Even the 454 implementation of the AS-based "local_pref" parameter can sometimes 455 be quite involved. The difficulty is caused by the lack of a glob- 456 ally transitive preference an AS (with multiple direct providers) can 457 specify, and be used in the route selection process. 459 A new BGP attribute termed "Destination Preference Attribute" (DPA) 460 has been proposed in [3] to address such need. As illustrated in 461 [4], the routing policies presented in Section 2 can be implemented 462 with ease by using the DPA attribute. In particular, only the AS 463 that originates this preference needs to specify this preference on a 464 per-route basis. 466 5. Security Considerations 468 Security considerations are not discussed in this memo. 470 6. Acknowledgments 472 The authors would like to thank Roy Alcala, Dennis Ferguson, John 473 Stewart, and Jack Waters of MCI for the many interesting hallway dis- 474 cussions related to this work. We also acknowledge helpful comments 475 and suggestions by Yakov Rekhter of Cisco. 477 7. References 479 [1] Rekhter, Y., and Li, T., "A Border Gateway Protocol 4 (BGP-4)", 480 RFC1771, March 1995. 482 [2] Y. Rekhter, and P. Gross, "Application of the Border Gateway Pro- 483 tocol in the Internet", RFC1772, March 1995. 485 [3] Chen, E., and Bates, T., "Destination Preference Attribute for 486 BGP", INTERNET-DRAFT, , June 1995. 488 [4] Chen, E., and Bates, T., "Application of the BGP Destination 489 Preference Attribute in Implementing Symmetric Routing", INTERNET- 490 DRAFT, , June 1995. 492 [5] Antonov, V., "BGP AS Path Metrics", INTERNET DRAFT, , March 1995. 495 [6] Rekhter, Y., "Routing in a Multi-provider Internet", RFC1787, 496 April 1995. 498 [7] Hawkinsin, J., and Bates, T., "Guidelines for creation, selec- 499 tion, and registration of an Autonomous System (AS)", INTERNET-DRAFT, 500 , May 1995. 502 8. Author's Addresses 504 Enke Chen 505 MCI 506 2100 Reston Parkway 507 Reston, VA 22091 509 phone: +1 703 715 7087 510 email: enke@mci.net 512 Tony Bates 513 MCI 514 2100 Reston Parkway 515 Reston, VA 22091 517 phone: +1 703 715 7521 518 email: Tony.Bates@mci.net