idnits 2.17.1 draft-ietf-idr-symm-multi-prov-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 9 instances of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 258 has weird spacing: '...rs) and defau...' == Line 431 has weird spacing: '...bute to allow...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (January 1996) is 10328 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '2' is defined on line 484, but no explicit reference was found in the text == Unused Reference: '5' is defined on line 494, but no explicit reference was found in the text == Unused Reference: '6' is defined on line 497, but no explicit reference was found in the text ** Obsolete normative reference: RFC 1771 (ref. '1') (Obsoleted by RFC 4271) -- Possible downref: Non-RFC (?) normative reference: ref. '3' -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' ** Downref: Normative reference to an Informational RFC: RFC 1787 (ref. '6') -- Possible downref: Non-RFC (?) normative reference: ref. '7' -- Possible downref: Non-RFC (?) normative reference: ref. '8' Summary: 11 errors (**), 0 flaws (~~), 6 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT Enke Chen 3 Tony Bates 4 MCI 5 January 1996 6 Expires in six months 8 Current Practice of Implementing 9 Symmetric Routing and Load Sharing 10 in the Multi-Provider Internet 11 13 Status of this Memo 15 This document is an Internet Draft. Internet Drafts are working 16 documents of the Internet Engineering Task Force (IETF), its Areas, 17 and its Working Groups. Note that other groups may also distribute 18 working documents as Internet Drafts. 20 Internet Drafts are draft documents valid for a maximum of six 21 months. Internet Drafts may be updated, replaced, or obsoleted by 22 Internet Drafts are draft documents valid for a maximum of six 23 months. Internet Drafts may be updated, replaced, or obsoleted by 24 other documents at any time. It is not appropriate to use Internet 25 Drafts as reference material or to cite them other than as a "working 26 draft" or "work in progress". 28 Please check the I-D abstract listing contained in each Internet 29 Draft directory to learn the current status of this or any other 30 Internet Draft. 32 Abstract 34 In the current multi-provider Internet, it is common for an entity to 35 have multiple service providers. Symmetric routing becomes 36 increasingly important for various reasons. This memo documents and 37 analyzes the current practice in implementing symmetric inter-domain 38 routing using BGP for several representative topologies of Internet 39 connections. 41 1. Introduction 43 In the multi-provider Internet, it is common for an entity to have 44 multiple connections to the Internet. For example, 45 o A Regional Service Provider (RSP) may be connected to multiple 46 transit Internet Service Providers (ISPs). 48 o A service subscriber may be connected to multiple RSPs or ISPs. 50 o Subscribers of different providers may wish to backup each 51 other. 53 These connections would provide for the capability of load sharing, 54 path diversification and backup. The Internet is a mesh of ISPs, 55 RSPs and service subscribers and is generally sparsely connected. 57 Symmetric routing is generally preferred as it facilitates problem 58 resolution, and provides for better resource (especially network 59 capacity) planning and utilization. Routing symmetry is also desir- 60 able in achieving optimal traffic flow in terms of reliability, delay 61 character, cost and other QoS metrics. Several applications such as 62 NTP, RSVP and MBONE rely upon routing symmetry. In the multi- 63 provider Internet, routing asymmetry, especially at the inter-domain 64 level, may have serious economic and legal ramifications. 66 This paper presents several representative topologies of Internet 67 connection and their inter-domain routing requirements. It then doc- 68 uments and analyzes the current practice in implementing symmetric 69 inter-domain routing in these cases using BGP. 71 This paper assumes that in general an ISP treats other ISPs equally 72 (in terms of the "local_pref" parameter) in the route selection pro- 73 cess. It also assumes that the following order of preference is fol- 74 lowed for the purpose of route selection: first the "local_pref" 75 parameter, followed by the shortest AS-path, the MED, and the IGP 76 metric. 78 It is noted that the length of the AS-path has not been specified in 79 the BGP document [1] as a route selection criteria. However, it has 80 been included in more than one implementations, and has been widely 81 used as such. 83 2. Internet Connection and Routing 85 The Internet is a mesh of transit Internet Service Providers (ISPs), 86 Regional Service Providers (RSPs), and service subscribers. In gen- 87 eral this mesh is rather sparsely connected with loose hierarchy. In 88 the multi-provider Internet, a good routing plan for an entity (i.e., 89 autonomous system) requires good understanding of its internal net- 90 work topology, its connection to direct providers (and neighboring 91 ASs), and its path to the major interconnection points (or network 92 access points, NAPs). 94 In this section, we present several typical topologies of Internet 95 connections, and their inter-domain routing requirements. Although 96 these cases are not meant to be exhaustive, they are expected to 97 cover the vast majority of Internet connection topologies. 99 2.1 An Entity with a Single Direct Provider 101 +-----+ +-----+ +-----+ 102 | ISP | | ISP | | ISP | 103 +-----+ +-----+ +-----+ 104 | | | 105 +-----+ +-----+ +-----+ 106 | AS1 | | RSP | | RSP | 107 +-----+ +-----+ +-----+ 108 | 109 +-----+ 110 | AS1 | 111 +-----+ 113 (a) (b) (c) 115 Figure 1 117 The routing is always symmetric at the inter-domain level. Routing 118 policies can be achieved using the current version of BGP. AS1 can 119 either take full routing or use default. 121 2.2 Backup of Entities with Different Direct Providers 123 Several topologies are shown in Figure 2. Both AS1 and AS2 have 124 their direct provider(s), and they would like to backup each other. 125 That is, if the link between AS1 and its direct provider is down, the 126 link between AS1 and AS2 would be used to reach the Internet. 128 +-----+ +------+ +------+ 129 | ISP | | ISP3 | | ISP3 | 130 +-----+ +------+ +------+ 131 / \ / \ / \ 132 / \ / (NAP) \ / (NAP) \ 133 +-----+ +-----+ +------+ +------+ +------+ +------+ 134 | AS1 |----| AS2 | | ISP1 |------| ISP2 | | ISP1 |-------| ISP2 | 135 +-----+ +-----+ +------+ +------+ +------+ +------+ 136 | | | | 137 +------+ +------+ +------+ +------+ 138 | AS1 |------| AS2 | | RSP1 | | RSP2 | 139 +------+ +------+ +------+ +------+ 140 | | 141 +------+ +------+ 142 | AS1 |------| AS2 | 143 +------+ +------+ 145 (a) (b) (c) 147 Figure 2 149 Note that in Figures 2(a)-2(b), AS1 and AS2 could be RSPs. 151 In all cases of Figure 2, in order to provide for backup, AS1 shall 152 permit the acceptance of AS2's routes from both AS2 and AS1's direct 153 provider, and permit their announcements to its direct providers. 154 Similar configuration is required for AS2. 156 There are two common routing policies depending upon how the link 157 between AS1 and AS2 is used. 159 Policy 1: Used solely as a backup link 161 The routing policy can be implemented by coordinating "LOCAL_PREF" 162 values between neighboring ASs. This can be acheived using AS- 163 based configuration or comunity based configuration [8]. 165 In all cases of Figure 2, the "LOCAL_PREF" value for the peer of 166 AS1 or AS2 with its direct provider shall be higher than that for 167 the peer between AS1 and AS2. Either full routing or partial 168 routing can be configured. 170 For example, in Figure 2(a), AS1 can take full routing from ISP 171 and AS2. An alternative is for AS1 to take only AS2's routes 172 from ISP and AS2, and configure default routes (with different 173 weights) at its border routers and then propagate them into its 174 own AS (via, e.g., iBGP). The ISP needs to make sure that the 175 "LOCAL_PREF" values are equal for the peers with AS1 and AS2 so 176 that the shorter AS-path would be selected. 178 Policy 2: Used for traffic between AS1 and AS2, and as backup in gen- 179 eral 181 In general this routing policy can be implemented by coordinating 182 "LOCAL_PREF" values [8] among the neighboring ASs and direct 183 providers. 185 In Figure 2(a), equal "LOCAL_PREF" values could be configured for 186 all the peers. Then the length of AS path would be used as tie- 187 breaker in the route selection. AS1 can either take full routing 188 from AS2 and its direct provider. It can also choose to take only 189 AS2's routes from its direct provider and AS2, and configure 190 default routes (with a different weights) at its border routers 191 and then propagate them into its own AS (via, e.g., iBGP). Simi- 192 lar configuration for AS2. 194 In Figures 2(b)-(c), AS1 can either take full routing from AS2 and 195 its direct provider, and configure the "LOCAL_PREF" parameter so 196 that traffic to AS2 prefers the AS1 - AS2 link over the link to 197 its direct provider. AS1 can choose to take only AS2's routes from 198 its direct provider and AS2, and configure default routes (with 199 different weight) at its border routers and then propagate them 200 into its own AS (via, e.g., iBGP). Similar configuration for AS2. 202 AS1's direct provider (and possible its ISP) needs to configure 203 the "LOCAL_PREF" parameter so that traffic to AS2 does not prefer 204 the link to AS1. Similar configuration is required for AS2's 205 direct provider. 207 2.3 An Entity with Multiple Direct Providers 209 As shown in Figure 3, AS1 has two direct providers. X and Y are 210 routes of AS1. Note that AS1 could be an RSP. 212 +------+ +-----+ +------+ 213 | ISP3 | | ISP | | ISP3 | 214 +------+ +-----+ +------+ 215 / \ / \ / \ 216 / (NAP) \ / \ / (NAP)\ 217 +------+ +------+ +------+ +------+ +------+ +-----+ 218 | ISP1 |------| ISP2 | | RSP1 | | RSP2 | | ISP1 |-----| ISP2| 219 +------+ +------+ +------+ +------+ +------+ +-----+ 220 \ / \ / | | 221 \L1 /L2 \L1 /L2 | | 222 +------+ +------+ +------+ +-----+ 223 | AS1 | | AS1 | | RSP1 | | RSP2| 224 |X Y| |X Y| +------+ +-----+ 225 +------+ +------+ \ / 226 \L1 /L2 227 +------+ 228 | AS1 | 229 |X Y| 230 +------+ 232 (a) (b) (c) 234 Figure 3 236 Depending upon the quality of these links and the internal network 237 topology of the AS, there are several common routing policies. 239 Policy 1: One link is used as primary, the other as pure backup 241 This policy is common when the quality of these links differ dra- 242 matically. 244 This policy can be implemented by coordinating AS-based 245 "LOCAL_PREF" values between the entity and its direct providers. 246 The AS can either take full routing or use default routes. In the 247 case of default, each border router can configure a default route 248 and then propagate it into the AS (via, e.g., iBGP). 250 Policy 2: Each link is used for traffic with the respective direct 251 provider. In general one link is used as primary, and the other as 252 backup. 254 If the traffic between AS1 and its direct providers (and their 255 customers) shall take the direct link, AS1 needs to be configured: 257 o either with partial routing (only routes of the direct 258 providers and their customers) and defaults with different 259 weights. 261 o or with full routing and configure "LOCAL_PREF" values. 263 The difficulty is that the indirect providers (e.g., ISP3 in 264 Figure 3(a)) may have to be involved to achieve symmetric rout- 265 ing. More specifically, 267 o In Figure 3(a), ISP3 would receive routes X and Y from both 268 ISP1 and ISP2 with identical length of AS paths. In order 269 for L1 to be favored by ISP3 to AS1, ISP1 would need to 270 manipulate the AS-path length, which is discussed in Section 271 3.1. Another approach is for ISP3 to configure "LOCAL_PREF" 272 parameter, which certainly does not scale well as there are 273 many ISPs at an NAP. In addition, it is almost impossible to 274 do as AS1 is not a customer of ISP3. 276 o In Figure 3(b), ISP would receive routes X and Y from both 277 RSP1 and RSP2 with identical length of AS paths. Either RSP1 278 needs to manipulate the AS-path length, or ISP needs to con- 279 figure "LOCAL_PREF" parameter. 281 o In Figure 3(c), ISP1 would receive routes X and Y from both 282 RSP1 and ISP2. In order for traffic from ISP1 to AS1 to 283 favor the ISP1 - ISP2 link, either RSP1 needs to manipulate 284 the AS-path length, or ISP1 needs to configure "LOCAL_PREF" 285 parameter. 287 Another problem is that it is difficult for AS1 to implement 288 "full routing", as AS1 needs to update the AS list for the 289 "LOCAL_PREF" parameter each time its direct providers acquire a 290 new AS as a customer. Nevertheless, some entities still prefer 291 full routing. 293 Policy 3: Partial load-sharing among these links 295 That is, the direct link is used for traffic between AS1 and its 296 direct providers including its customers. However, the closest 297 exit point would be taken for traffic beyond these direct 298 providers and their customers. For example, in Figure 3(a) traffic 299 between AS1 and ISP1 (and its customers) would use the direct link 300 between AS1 and ISP1; traffic between AS1 and ISP2 (and its cus- 301 tomers) would use the direct link between AS1 and ISP2. For traf- 302 fic destined to ISP3, either ISP1 or ISP2 would be used depending 303 on where the traffic is originated in AS1. 305 AS1 can take full routing. It can also take partial routing 306 (routes of direct providers and their customers), and configure 307 equal-weight default routes at its border routers and propagate 308 them into its AS. 310 The problem is how to make sure the return traffic from a 4th 311 party (e.g., ISP3 in Figure 3(a)) is symmetric. 313 Policy 4: Complete load-sharing among these links 315 That is, each network in AS1 sends packets to the closer (in terms 316 of internal route preference) border router that peers with a 317 direct providers. The return traffic is expected to take a sym- 318 metric path. For example, in Figure 3(a) a packet, which is orig- 319 inated from network X and is destined outside AS1, would be for- 320 warded to ISP1, even when the destination is in ISP2. 322 The simpler approach is for AS1 to use default. That is, AS1 323 would first configure default route at each connection to a 324 provider and propagate (e.g., via iBGP) them into the AS. Then, 325 each network in AS1 choose the closest exit point (determined by 326 IGP metric). The problem is how to make sure the return traffic 327 to X and Y takes symmetric paths. Currently this is achieved by 328 manipulating the AS-path length or other approaches detailed in 329 the following section. 331 If AS1 still prefers to take full routing, more coordination would 332 be required for using the AS path manipulation or other techniques 333 as described in Section 3. 335 3. Current Practices 337 Currently there are mainly three approaches to implement Polices 2-4 338 for Figure 3. This section presents analysis and critique of these 339 approaches. Without loss of generality, Figure 3(a) is used as an 340 example. 342 3.1 Manipulation of AS Path Length 343 Although the length of the AS path was not specified in [1] as a 344 parameter in the route selection process, it has been widely used as 345 such. 347 Some router software offers the ability of prepending AS numbers to 348 the AS path for the purpose of influencing the route selection. Here 349 is how the feature can be used. First, AS1 categorizes all of its 350 routes and prepends an AS number (either its own AS or a different AS 351 number): 353 AS1 Prepend AS Path 354 Route To ISP1 To ISP2 ISP1 ISP2 355 ===== ======= ======= ======= ====== 356 X AS1 AS1 AS1 AS1 357 Y AS1 AS1 AS1 AS1 359 In general the different AS paths can be used by ISP1 and ISP2 to 360 configure AS-based "LOCAL_PREF" values to implement the desired rout- 361 ing policy. The "LOCAL_PREF" configuration would not be necessary if 362 there are sufficient number of ASs inserted. 364 With this approach the AS that originates the preference has full 365 control, and only that AS needs to manipulate the AS path on a per- 366 route basis. 368 The drawbacks of this approach includes: 370 o It extends the AS path with superfluous information. In par- 371 ticular, the superfluous information in the AS path would be 372 propagated upstream and to the whole Internet. 374 o The number of ASs that need to be preprended is in general 375 proportional to the number of direct providers. 377 o Compatibility with other BGP implementation may be a problem. 379 3.2 Splitting AS 381 This approach requires an AS to be split into multiple ASs and run 382 external BGP between these ASs (possibly with MEDs configured for 383 load balancing among these ASs). Then the cases in Figure 3 can be 384 reduced to the cases in Figure 2, which have been discussed in Sec- 385 tion 2. 387 This is probably the cleanest approach the current BGP version can 388 offer. However, there is a great deal of reluctance in using this 389 approach. Practical problems with this approach include: 391 o It does not work with the partial load-sharing case where the 392 connections to multiple providers originate from one router. 394 o Splitting ASs and having them maintained could be quite 395 involved depending upon the internal network topology. 397 o Extra AS numbers are required [7]. It would be necessary to 398 apply for AS numbers at the InterNIC. 400 o The number of split ASs is proportional to the number of 401 direct providers. 403 o Wasting of AS numbers. The exhaustion of the AS number space 404 could become real with the ever-increasing number of such 405 needs. 407 o The increased number of ASs would add complexity to the Inter- 408 net topology, and therefore complicate problem resolution. 410 o An AS number has been traditionally tied to an organization. 411 Splitting AS means loss of coherence for some customers. 413 3.3 NLRI-based Preference Specification 415 This is the approach that has been used for the NSFNET. Here is how 416 it is done with Figure 3(a). ISP1 and ISP2 configure net-based pref- 417 erence on their routers, according to preference provided by AS1. 418 For example, for route X, ISP1 would configure higher preference for 419 its direct link with AS1, and lower preference for its direct link 420 with ISP2. 422 This approach requires NRLI-based customization with the direct 423 providers and sometimes indirect providers as well as the originating 424 AS. The NSFNET configuration experience has shown that this approach 425 requires non-trivial administrative coordination and full topology 426 information. In addition, it places a burden on routers with limited 427 memory capacity. For these reasons, the NLRI-based preference con- 428 figuration should be avoided at the provider level if possible. 429 Instead, such a configuration should be pushed as close to the origi- 430 nating AS as possible. The technique presented in [8] makes use of 431 the BGP community attribute to allow one to acheive NLRI based 432 LOCAL_PREF configuration without provider level customization. 434 3.4 Perfect Aggregation and Addressing 436 In the cases that all routes in the AS are covered by an aggregate 437 and address assignment is completely consistent with the network 438 topology, the rule of the longest prefix match can be used to help 439 achieve routing symmetry and load sharing. That is, a portion of the 440 aggregate, along with the aggregate itself, can be announced to one 441 direct provider. The remaining portion of the aggregate, along with 442 the aggregate itself, can be announced to the other direct provider. 444 It does not work with the partial load-sharing case where the connec- 445 tions to multiple providers originate from one router. More impor- 446 tantly, the requirement of this approach is not likely to be met in 447 practice. So, this approach is listed just for the sake of complete- 448 ness. 450 4. Discussion 452 As has been illustrated in Section 3, it is not easy to implement 453 routing symmetry and load sharing for an entity with multiple direct 454 providers using the current functionality of BGP. There are many 455 drawbacks with the current practice of implementation. Even the 456 implementation of the AS-based "LOCAL_PREF" parameter can sometimes 457 be quite involved. The difficulty is caused by the lack of a glob- 458 ally transitive preference an AS (with multiple direct providers) can 459 specify, and be used in the route selection process. 461 A new BGP attribute termed "Destination Preference Attribute" (DPA) 462 has been proposed in [3] to address such need. As illustrated in 463 [4], the routing policies presented in Section 2 can be implemented 464 with ease by using the DPA attribute. In particular, only the AS 465 that originates this preference needs to specify this preference on a 466 per-route basis. 468 5. Security Considerations 470 Security considerations are not discussed in this memo. 472 6. Acknowledgments 474 The authors would like to thank Roy Alcala, Dennis Ferguson, John 475 Stewart, and Jack Waters of MCI for the many interesting hallway dis- 476 cussions related to this work. We also acknowledge helpful comments 477 and suggestions by Yakov Rekhter of Cisco. 479 7. References 481 [1] Rekhter, Y., and Li, T., "A Border Gateway Protocol 4 (BGP-4)", 482 RFC1771, March 1995. 484 [2] Y. Rekhter, and P. Gross, "Application of the Border Gateway Pro- 485 tocol in the Internet", RFC1772, March 1995. 487 [3] Chen, E., and Bates, T., "Destination Preference Attribute for 488 BGP", INTERNET-DRAFT, , January 1996. 490 [4] Chen, E., and Bates, T., "Application of the BGP Destination 491 Preference Attribute in Implementing Symmetric Routing", INTERNET- 492 DRAFT, , January 1996. 494 [5] Antonov, V., "BGP AS Path Metrics", INTERNET DRAFT, , March 1995. 497 [6] Rekhter, Y., "Routing in a Multi-provider Internet", RFC1787, 498 April 1995. 500 [7] Hawkinsin, J., and Bates, T., "Guidelines for creation, selec- 501 tion, and registration of an Autonomous System (AS)", INTERNET-DRAFT, 502 , December 1995. 504 [8] Chen, E., and Bates, T., "An Application of the BGP Community 505 Attribute in Multi-home Routing", INTERNET-DRAFT, , January 1996. 508 8. Author's Addresses 510 Enke Chen 511 MCI 512 2100 Reston Parkway 513 Reston, VA 22091 515 phone: +1 703 715 7087 516 email: enke@mci.net 518 Tony Bates 519 MCI 520 2100 Reston Parkway 521 Reston, VA 22091 523 phone: +1 703 715 7521 524 email: Tony.Bates@mci.net