idnits 2.17.1 draft-dskc-bess-bgp-car-problem-statement-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 4 instances of too long lines in the document, the longest one being 3 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (May 23, 2021) is 1067 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'S1' is mentioned on line 794, but not defined == Missing Reference: 'S2' is mentioned on line 794, but not defined == Missing Reference: 'S3' is mentioned on line 797, but not defined == Missing Reference: 'RFC8604' is mentioned on line 882, but not defined == Unused Reference: 'I-D.agrawal-spring-srv6-mpls-interworking' is defined on line 1162, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-bess-srv6-services' is defined on line 1167, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-idr-bgp-ipv6-rt-constrain' is defined on line 1173, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-spring-sr-service-programming' is defined on line 1195, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-spring-srv6-network-programming' is defined on line 1202, but no explicit reference was found in the text == Unused Reference: 'I-D.voyer-pim-sr-p2mp-policy' is defined on line 1208, but no explicit reference was found in the text == Unused Reference: 'RFC2119' is defined on line 1214, but no explicit reference was found in the text == Unused Reference: 'RFC4360' is defined on line 1219, but no explicit reference was found in the text == Unused Reference: 'RFC4684' is defined on line 1223, but no explicit reference was found in the text == Unused Reference: 'RFC4760' is defined on line 1230, but no explicit reference was found in the text == Unused Reference: 'RFC5512' is defined on line 1235, but no explicit reference was found in the text == Unused Reference: 'RFC5701' is defined on line 1241, but no explicit reference was found in the text == Unused Reference: 'RFC7311' is defined on line 1245, but no explicit reference was found in the text == Unused Reference: 'RFC7606' is defined on line 1250, but no explicit reference was found in the text == Unused Reference: 'RFC8126' is defined on line 1255, but no explicit reference was found in the text == Unused Reference: 'RFC8174' is defined on line 1260, but no explicit reference was found in the text == Unused Reference: 'RFC8277' is defined on line 1264, but no explicit reference was found in the text == Unused Reference: 'RFC8402' is defined on line 1268, but no explicit reference was found in the text == Unused Reference: 'RFC8664' is defined on line 1273, but no explicit reference was found in the text == Unused Reference: 'RFC8669' is defined on line 1279, but no explicit reference was found in the text == Unused Reference: 'I-D.filsfils-spring-sr-policy-considerations' is defined on line 1287, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-idr-performance-routing' is defined on line 1293, but no explicit reference was found in the text == Unused Reference: 'RFC3906' is defined on line 1304, but no explicit reference was found in the text == Unused Reference: 'RFC4271' is defined on line 1309, but no explicit reference was found in the text == Unused Reference: 'RFC4272' is defined on line 1314, but no explicit reference was found in the text == Unused Reference: 'RFC4364' is defined on line 1318, but no explicit reference was found in the text == Unused Reference: 'RFC6952' is defined on line 1322, but no explicit reference was found in the text == Unused Reference: 'RFC7911' is defined on line 1328, but no explicit reference was found in the text == Outdated reference: A later version (-14) exists of draft-agrawal-spring-srv6-mpls-interworking-05 == Outdated reference: A later version (-15) exists of draft-ietf-bess-srv6-services-07 == Outdated reference: A later version (-26) exists of draft-ietf-lsr-flex-algo-15 == Outdated reference: A later version (-22) exists of draft-ietf-spring-segment-routing-policy-11 == Outdated reference: A later version (-09) exists of draft-ietf-spring-sr-service-programming-04 ** Obsolete normative reference: RFC 5512 (Obsoleted by RFC 9012) == Outdated reference: A later version (-09) exists of draft-filsfils-spring-sr-policy-considerations-07 Summary: 4 errors (**), 0 flaws (~~), 40 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS WorkGroup D. Rao 3 Internet-Draft S. Agrawal 4 Intended status: Informational C. Filsfils 5 Expires: November 24, 2021 K. Talaulikar 6 Cisco Systems 7 B. Decraene 8 Orange 9 D. Steinberg 10 Lapishills Consulting Limited 11 L. Jalil 12 Verizon 13 J. Guichard 14 Futurewei 15 K. Patel 16 Arrcus, Inc 17 W. Henderickx 18 Nokia 19 May 23, 2021 21 BGP Color-Aware Routing Problem Statement 22 draft-dskc-bess-bgp-car-problem-statement-03 24 Abstract 26 This document explores the scope, use-cases and requirements for a 27 BGP based routing solution to establish end-to-end intent-aware paths 28 across a multi-domain service provider network environment. 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on November 24, 2021. 47 Copyright Notice 49 Copyright (c) 2021 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents 54 (https://trustee.ietf.org/license-info) in effect on the date of 55 publication of this document. Please review these documents 56 carefully, as they describe your rights and restrictions with respect 57 to this document. Code Components extracted from this document must 58 include Simplified BSD License text as described in Section 4.e of 59 the Trust Legal Provisions and are provided without warranty as 60 described in the Simplified BSD License. 62 Table of Contents 64 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 65 1.1. Objective . . . . . . . . . . . . . . . . . . . . . . . . 3 66 1.2. Color-Aware Routing . . . . . . . . . . . . . . . . . . . 3 67 1.2.1. Intent . . . . . . . . . . . . . . . . . . . . . . . 4 68 1.2.2. Color . . . . . . . . . . . . . . . . . . . . . . . . 4 69 1.2.3. Colored Service Route . . . . . . . . . . . . . . . . 4 70 1.2.4. Color-Aware Route . . . . . . . . . . . . . . . . . . 4 71 1.2.5. Service Route Automated Steering on color-aware route 5 72 1.2.6. Inter-Domain color-aware routing with SR Policy . . . 5 73 1.2.7. Need for a BGP-based color-aware routing solution . . 5 74 1.2.8. BGP Color-Aware Routing . . . . . . . . . . . . . . . 5 75 1.2.9. Architectural consistency among color-aware routing 76 solutions . . . . . . . . . . . . . . . . . . . . . . 5 77 1.2.10. Color Domains . . . . . . . . . . . . . . . . . . . . 7 78 1.2.11. Per-Destination and Per-Flow Steering with BGP CAR . 7 79 2. Intent bound to a Color . . . . . . . . . . . . . . . . . . . 8 80 3. BGP CAR Use-cases . . . . . . . . . . . . . . . . . . . . . . 8 81 3.1. BGP Transport CAR . . . . . . . . . . . . . . . . . . . . 8 82 3.1.1. Use-case of minimization of a cost metric vs a 83 latency metric . . . . . . . . . . . . . . . . . . . 9 84 3.1.2. Use-case of exclusion/inclusion of link affinity . . 11 85 3.1.3. Use-case of exclusion/inclusion of domains . . . . . 11 86 3.1.4. Use-case of virtual network function chains in local 87 and core domains . . . . . . . . . . . . . . . . . . 12 88 3.2. BGP VPN CAR . . . . . . . . . . . . . . . . . . . . . . . 13 89 3.2.1. Use-case of minimization of a cost metric vs a 90 latency metric . . . . . . . . . . . . . . . . . . . 16 91 3.2.2. Use-case of exclusion/inclusion of link affinity . . 17 92 3.2.3. Use-case of virtual network function chains in local 93 and core domains . . . . . . . . . . . . . . . . . . 18 94 4. Deployment Requirements . . . . . . . . . . . . . . . . . . . 19 95 5. Scalability . . . . . . . . . . . . . . . . . . . . . . . . . 20 96 5.1. Scale Requirements . . . . . . . . . . . . . . . . . . . 20 97 5.2. Scale Analysis . . . . . . . . . . . . . . . . . . . . . 22 98 6. Network Availability . . . . . . . . . . . . . . . . . . . . 24 99 7. BGP Protocol Requirements . . . . . . . . . . . . . . . . . . 25 100 8. Future Considerations . . . . . . . . . . . . . . . . . . . . 26 101 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 102 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 27 103 10.1. Normative References . . . . . . . . . . . . . . . . . . 27 104 10.2. Informative References . . . . . . . . . . . . . . . . . 30 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 107 1. Introduction 109 1.1. Objective 111 This document explores the scope, use-cases and requirements for a 112 BGP based routing solution to establish end-to-end intent-aware paths 113 across a multi-domain service provider network environment. 115 The targeted design outcome is to define the technology and protocol 116 extensions that may be required in a manner that addresses the widest 117 application. 119 The problem that the document initially focuses on is the BGP-based 120 delivery of an intent across several transport domains. To do this, 121 it describes existing intent-aware routing solutions that are 122 deployed and then extends the solution scope and architecture to BGP. 124 The problem space is then widened to include any intent (including 125 NFV chains and their location), any dataplane and the application of 126 the intent-based routing to the Service/VPN routes. All of this is 127 detailed in the rest of the document. 129 1.2. Color-Aware Routing 131 Color-Aware Routing (CAR) establishes routed paths that satisfy 132 specific intent in a network. This section describes the basic 133 concepts that define CAR and the protocols that currently support it. 135 The figure below is used as reference. 137 +-----------------------------------+ 138 |----+ +----| 139 | E1 | | E2 |- V/v with C 140 |----+ +----| 141 +-----------------------------------+ 143 Figure 1: Color-aware routing reference topology 145 1.2.1. Intent 147 Intent in routing may be any combination of the following behaviors: 149 o Topology path selection (e.g. minimize metric, avoid resource) 151 o NFV service insertion (e.g. service chain steering) 153 o Per-hop behavior (e.g. QoS for 5G slice) 155 An intent-aware routed path may be within a single network domain or 156 across multiple domains. 158 1.2.2. Color 160 Color is a 32-bit numerical value that is associated with an intent, 161 as defined in [I-D.ietf-spring-segment-routing-policy] 163 1.2.3. Colored Service Route 165 An Egress PE E2 colors a BGP service (e.g., VPN) route V/v to 166 indicate the particular intent that E2 requests for the traffic bound 167 to V/v. The color (C) is encoded as a BGP Color Extended community 168 [I-D.ietf-idr-tunnel-encaps]. 170 1.2.4. Color-Aware Route 172 (E2, C) is a color-aware route to E2 which satisfies the intent 173 associated with color C. 175 Multiple technologies already provide color-aware paths in solutions 176 that are widely deployed. 178 o SR Policy [I-D.ietf-spring-segment-routing-policy] 180 o IGP Flex-Algo [I-D.ietf-lsr-flex-algo] 182 In the context of large-scale SR-MPLS networks, SR Policy is 183 applicable to both intra-domain and inter-domain deployments; whereas 184 IGP Flex-Algo is better suited to intra-domain scenarios. 186 1.2.5. Service Route Automated Steering on color-aware route 188 An ingress PE E1 automatically steers V-destined packets onto a 189 Color-Aware path bound to (E2, C). If several such paths exist, a 190 preference scheme is used to select the best path: E.g. IGP Flex- 191 Algo first, then SR Policy. 193 1.2.6. Inter-Domain color-aware routing with SR Policy 195 If E1 and E2 are in different domains, E1 may request an SR-PCE in 196 its domain for a path to (E2, C). The SR-PCE (or a set of them) 197 computes the end-to-end path and installs it at E1 as an SR Policy. 198 The end-to-end color-aware path may seamlessly cross multiple 199 domains. 201 1.2.7. Need for a BGP-based color-aware routing solution 203 o An operator with an existing Seamless-MPLS/BGP-LU inter-domain 204 deployment [I-D.ietf-mpls-seamless-mpls] may prefer a BGP based 205 extension as a more incremental approach 207 o There may be an expectation that BGP would support a larger scale 209 o Trust boundaries in an inter-domain deployment leads to a 210 preference for a BGP peering based solution 212 1.2.8. BGP Color-Aware Routing 214 BGP Color-Aware Routing (CAR) is a new BGP solution which signals 215 intent-aware routes to reach a given destination (e.g., E2). (E2, C) 216 represents a BGP hop-by-hop distributed route that builds an inter- 217 domain color-aware path to E2 for color C. 219 1.2.9. Architectural consistency among color-aware routing solutions 221 As seen above, multiple technologies exist that provide color-aware 222 routing in a network. A BGP based solution must be compliant with 223 the existing principles that apply to them: 225 o Service routes MUST be colored using BGP Color Extended-Community 226 to request intent 228 * V/v via E, colored with C 230 o Colored service routes MUST be automatically steered on an 231 appropriate color-aware path 233 * V/v via E with C is steered via (E, C) 234 * (E, C) provided by any color-aware technology or protocol 236 o Color-aware routes MAY resolve recursively via other color-aware 237 routes 239 * (E, C) via N recursively resolves via (N, C) 241 Here is a brief example that illustrates these principles. 243 +----------------+ +----------------+ +----------------+ 244 | | | | | | V/v with C1 245 |----+ +----+ +----+ +----|/ 246 | E1 | | N1 | | N2 | | E2 |\ 247 |----+ +----+ +----+ +----| W/w with C2 248 | | | | | | 249 | Domain 1 | | Domain 2 | | Domain 3 | 250 +----------------+ +----------------+ +--- ------------+ 252 Figure 2: Color-aware routing inter-domain reference topology 254 In the figure above, all the nodes are part of an inter-domain 255 network under a single authority and with a consistent color-to- 256 intent mapping: 258 o Color C1 is mapped to "low delay" 260 * Flex-Algo FA1 is mapped to "low delay" and hence to C1 262 o Color C2 is mapped to "low delay and avoid resource R" 264 * Flex-Algo FA2 is mapped to "low delay and avoid resource R" and 265 hence to C2 267 E1 receives two BGP colored service routes from E2: 269 o V/v with BGP Color Extended community C1 271 o W/w with BGP Color Extended community C2 273 E1 has the following inter-domain color-aware paths: 275 o (E2, C1) provided by BGP CAR which recursively resolves via intra- 276 domain color-aware paths: 278 * (N1, C1) provided by IGP FA1 in Domain1 280 * (N2, C1) provided by SR Policy bound to color C1 in Domain2 282 o (E2, C2) provided by SR Policy 284 E1 automatically steers the received colored service routes as 285 follows: 287 o V/v via (E2, C1) provided by BGP CAR 289 o W/w via (E2, C2) provided by SR Policy 291 The example illustrates the benefits provided by leveraging the 292 architectural principles: 294 o Seamless co-existence of multiple color-aware technologies, e.g., 295 BGP CAR and SR Policy 297 * V/v is steered on BGP CAR color-aware path 299 * W/w is steered on SR Policy color-aware path 301 o Seamless and complementary interworking between different color- 302 aware technologies 304 * V/v is steered on a BGP CAR color-aware path that is itself 305 resolved within domain 2 onto an SR Policy bound to the color 306 of V/v 308 1.2.10. Color Domains 310 o A color domain represents a collection of one or more network 311 (IGP/BGP) domains with a single, consistent color-to-intent 312 mapping 314 o Color re-mapping may happen at color domain boundaries 316 1.2.11. Per-Destination and Per-Flow Steering with BGP CAR 318 Ingress PE E1 steers packets destined for a service (VPN) route V/v 319 via BGP Color-Aware Route R/r to E2 321 o Per-Destination Steering: Incoming packets on E1 match BGP service 322 route V/v to be steered based on the destination IP address of the 323 packets. 325 o Per-Flow Steering: Incoming packets on E1 match BGP service route 326 V/v to be steered based on the combination of the destination IP 327 address and additional elements in the packet header (i.e., IP 328 flow). Such a packet lookup may recurse on a forwarding array 329 where some of the entries are BGP color-aware routes to E2. A 330 given flow is mapped to a specific entry in this array i.e. via a 331 specific BGP color-aware route to E2. 333 2. Intent bound to a Color 335 The BGP CAR solution must support the following intents bound to a 336 color: 338 o Minimization of a cost metric vs a latency metric 340 * Minimization of different metric types, static and dynamic 342 o Exclusion/Inclusion of SRLG and/or Link Affinity and/or minimum 343 MTU/number of hops 345 o Bandwidth management 347 o In the inter-domain context, exclusion/inclusion of entire 348 domains, and border routers 350 o Inclusion of one or several virtual network function chains 352 * Located in a regional domain and/or core domain, in a DC 354 o Localization of the virtual network function chains 356 * Some functions may be desired in the regional DC or vice versa 358 o Per-Destination and Per-Flow steering 360 3. BGP CAR Use-cases 362 The BGP CAR route may be a transport route or a service route (in 363 this document, we use the term VPN instead of service for 364 simplicity). 366 3.1. BGP Transport CAR 368 o Transport Intent 370 * Intent-aware routing between PEs connected across multiple 371 transit domains 373 + Set up BGP based end-to-end paths stitching intent-aware 374 intra-domain segments 376 o The network diagram below illustrates the reference network 377 topology used in this section for Transport CAR: 379 +-----+ +-----+ +-----+ 380 .....|S-RR1| <............. |S-RR2| <............... |S-RR3| <... 381 : +-----+ +-----+ +-----+ : 382 : : 383 : : 384 : : 385 +--:--------------+ +-----------------+ +--------------:--+ 386 | : | | | | : | 387 | : | | | | : | 388 | : +---| D=20 |---+ +---| D=25 |---+ : | 389 | : |121|-------|211| |231|-------|321| : | 390 | : +---| \ / |---+ +---| \ / |---+ : | 391 |----+ | \ / | | \ / | +----| 392 |PE11| | V | | V | |PE31| 393 |----+ | / \ | | / \ | +----| 394 | +---| / \ |---+ +---| / \ |---+ | 395 | |122|-------|212| |232|-------|322| | 396 | +---| D=15 |---+ +---| D=10 |---+ | 397 | | | | | | 398 | Domain 1 | | Domain 2 | | Domain 3 | 399 +-----------------+ +-----------------+ +------------------+ 401 Figure 3: Transport CAR Reference Topology 403 The following network design assumptions apply to the reference 404 topology above, as an example: 406 * Independent ISIS/OSPF SR instance in each domain. 408 * eBGP peering link between ASBRs (121-211, 121-212, 122-211, 409 122-212, 231-321, 231-322, 232-321 and 232-322). 411 * Peering links have equal cost metric. 413 * Peering links have delay configured or measured as shown by 414 "D". D=50 for cross peering links. 416 * VPN service is running from PE31 to PE11 via service RRs (S-RRn 417 in figure). 419 o The following sections illustrate a few examples of intent use- 420 cases applicable to transport routes. 422 3.1.1. Use-case of minimization of a cost metric vs a latency metric 424 o In the reference topology of Figure 3 426 Each domain has Algo 0 and Flex Algo 128 427 Algo 0 is for minimum cost metric(cost optimized). 429 Flex Algo 128 definition is for minimum delay (low latency). 431 o Cost Optimized 433 * Color C1 - Minimum cost intent. (Here, a BGP CAR route with 434 Color C1 is being used, instead of BGP-LU.) 436 * On PE11, VPN routes colored with C1 are steered via (C1, PE31) 437 BGP CAR route 439 + BGP CAR for C1 sets up path(s) between PEs for end-to-end 440 minimum cost. 442 + (2) These paths traverse over intra-domain Algo 0 in each 443 domain and account for the peering link cost between ASBRs. 445 + Example: PE11 learns (C1, PE31) CAR route via several equal 446 paths: 448 1. One such path is through FA0 to node 121, links 121-211, 449 FA0 to 231, link 231-321, FA0 to PE31 451 2. Another such path is through FA0 to node 122, link 452 122-212, FA0 to 232, link 232-322, FA0 to PE31. 454 o Minimize latency 456 * Color C2 - Minimum latency intent. 458 * On PE11, VPN routes colored with C2 are steered via (C2, PE31) 459 BGP CAR route. 461 + BGP CAR for C2 advertises paths between PEs for minimum end- 462 to-end delay. 464 + (2) These paths traverse over intra-domain Flex Algo 128 in 465 each domain and account for peering link delay between 466 ASBRs. 468 + (3) Example: PE11 learns (C2, PE31) BGP CAR route and best 469 path is through FA128 to node 122, link 122-212, FA128 to 470 232, link 232-322, FA128 to PE31. 472 3.1.2. Use-case of exclusion/inclusion of link affinity 474 o Color C3 - Intent to Minimize cost metric and avoid purple links 476 o In the reference topology of Figure 3 478 Each domain has Flex Algo 129 and some links have purple 479 affinity. 481 Flex Algo 129 definition is set to minimum cost metric and 482 avoid purple links (within domain). 484 Peering cross links are colored purple by policy. 486 o On PE11, VPN routes colored with C3 are steered via (C3, PE31) BGP 487 CAR route. 489 * BGP CAR for C3 sets up paths between PEs for minimum end-to-end 490 cost and avoiding purple link affinity. 492 * These paths traverse over intra domain Flex Algo 129 in each 493 domain and accounts for peering link cost between ASBR and 494 avoiding purple links. 496 * Example: PE11 learns (C3, PE31) BGP CAR route via 2 paths. 498 1. First path is through FA 129 to node 121, link 121-211, 499 FA129 to 231, link 231-321, FA129 to PE31. 501 2. Second path is through FA129 to node 122, link 122-212, 502 FA129 to 232, link 232-322, FA129 to PE31. 504 3.1.3. Use-case of exclusion/inclusion of domains 505 +-----+ +-----+ +-----+ 506 ....|S-RR1| <............. |S-RR2| <.............. |S-RR3| <.... 507 : +-----+ +-----+ +-----+ : 508 : : 509 : +----------------+ : 510 : | | : 511 +--:--------------+ |---+ +---| +--------------:--+ 512 | : | |---|211| |241|---| | : | 513 | : | | |---+ +---| | | : | 514 | : +---| | | Domain 2 | | |---+ : | 515 | : |121|---| +----------------+ |---|421| : | 516 | : +---| |---+ : | 517 |----+ | | +----| 518 |PE11| | | |PE41| 519 |----+ | | +----| 520 | +---| |---+ | 521 | |131|---| +----------------+ |---|431| | 522 | +---| | | | | |---+ | 523 | | | |---+ +---| | | | 524 | Domain 1 | |---|311| |341|---| | Domain 4 | 525 +-----------------+ |---+ +---| +-----------------+ 526 | Domain 3 | 527 +----------------+ 529 Figure 4 531 Color C4 - Avoid sending selected traffic via Domain 3 533 o VPN routes advertised from PEs with Color C4 535 o BGP CAR for Color C4 should only set up paths between PE11 and 536 PE41 that exclude Domain 3 538 3.1.4. Use-case of virtual network function chains in local and core 539 domains 540 ____ ____ 541 / \ / \ 542 | NFV1 | | NFV2 | 543 \ / \ / 544 +---------------------+ +--------------------+ +-------------------+ 545 | |E11| | | |E21| | | | 546 | +---+ | | +---+ | | | 547 | | | | | | 548 | | | | | | 549 |----+ +------+ +------+ +----| 550 |PE11| | 121 | | 231 | |PE31| 551 |----+ +------+ +------+ +----| 552 | | | | | | 553 | | | | | | 554 | | | | | | 555 | | | | | | 556 | Domain 1 | | Domain 2 | | Domain 3 | 557 +---------------------+ +--------------------+ +-------------------+ 559 Figure 5 561 o Color intent 563 * C5 - Routing via min-cost paths 565 * C6 - Routing via a local NFV service chain situated at E11 567 * C7 - Routing via a centrally located NFV service chain situated 568 at E21 570 o Forwarding of packets from PE11 towards PE31: 572 * (C5, PE31) mapped packets are sent via nodes 121, 231 to PE31 574 * (C6, PE31) mapped packets are sent to E11 and then post-service 575 chain, via 121, 231 to PE31 577 * (C7, PE31) mapped packets are sent via 121 to E21 and then 578 post-service chain, via 231 to PE31 580 3.2. BGP VPN CAR 582 o VPN (Service layer) intent 584 * Extend the signaling of intent awareness end-to-end: CE site to 585 CE site across provider networks 586 + Provide ability for a CE to select paths through specific 587 PEs for a given intent 589 - Example-1: Certain intent in transport not available via 590 specific PEs 592 - Example-2: Certain CE-PE connection does not support 593 specific intent 595 - Example-3: Site access via certain CE does not support 596 specific intent. For instance, link connecting a 597 specific CE to a DC hosting loss-sensitive service may 598 have better quality than a link from another CE 600 + Provide ability for a CE to send traffic indicating a 601 specific intent (via suitable encapsulation) to the PE for 602 optimal steering. 604 * Intent aware routing support for multiple service (VPN) 605 interworking models 607 + Beyond options such as iBGP or Inter-AS Option C that 608 inherently extend from PE to PE 610 1. Inter-AS Option A 612 2. Inter-AS Option B 614 3. GW based interworking(L3VPN, EVPN) 616 + Interworking with existing L3VPN deployments, both PEs and 617 CEs 619 o The network diagram below illustrates the reference network 620 topology used in this section for VPN CAR. 622 +-------------------+ +-------------------+ 623 | ....|S-RR|.... | | ....|S-RR|..... | 624 | : +----+ : | | : +----+ : | 625 | : : : : | | : : : : | 626 |----+ : : +---| D=20 |---+ : : +----| 627 /|PE11| : : |121|-----------|211| : : |PE21|\ 628 D=25/ |----+ : : +---| X X |---+ : : +----| \ D=25 629 / | : : | X X | : : | \ V/24 630 CE1 | : : | X D=50| : : | CE2<--- 631 X | : : | X X | : : | X 632 D=15 X |----+ : : +---| X X |---+ : : +----| X D=10 633 X|PE12|...: :...|212|-----------|232|...: :...|PE22|X 634 |----+ +---| D=10 |---+ +----| 635 | | | | 636 | AS 1 | | AS 2 | 637 +-------------------+ +-------------------+ 639 Figure 6: VPN CAR reference topology 641 The following network design assumptions apply to the reference 642 topology above, as an example: 644 * Independent ISIS/OSPF SR instance in each AS. 646 * eBGP peering link between VPN ASBRs 121-211, 121-212, 122-211, 647 122-212. 649 * VPN service is running between PEs via service RRs in each AS 650 to local ASBRs. Between ASBRs, its Option-B i.e. next hop self 651 for VPN SAFI. 653 * CE1 is dual homed to PE11 and PE12. Similarly, CE2 is dual 654 homed to PE21 and PE22. 656 * Peering links have equal cost metric 658 * Peering links have delay configured or measured as shown by 659 "D". 661 * CE2 advertises prefix V/24 to CE1. It is advertised as RD:V/24 662 between PEs, including color-awareness 664 o The following sections illustrate a few examples of intent use- 665 cases applicable to VPN (service) routes. 667 3.2.1. Use-case of minimization of a cost metric vs a latency metric 669 o In the reference topology of Figure 6 671 Each AS has Flex Algo 0 and 128. 673 Flex Algo 0 is for minimum cost metric(cost optimized). 675 Flex Algo 128 definition is for minimum delay (low latency). 677 o Cost Optimized 679 * Color C1 - Minimum cost intent. 681 * On CE1, flows requiring cost optimized paths to V/24 are 682 steered over (C1, V/24) route. 684 + BGP CAR for C1 sets up paths between CEs for minimum end-to- 685 end cost. 687 + This advertisement needs BGP CAR between PE-CE for V/24 688 prefix and color C1 awareness. 690 + It also needs BGP VPN CAR between PEs and ASBRs for RD:V/24 691 prefix and color C1 awareness (C1, RD:V/24). 693 + Paths traverse over PE-CE links, intra-domain Flex Algo 0 in 694 each AS and peering links between ASBRs, minimizing cost for 695 VPN. 697 + Example: CE1 learns (C1, V/24) CAR route through several 698 equal cost paths: 700 1. One path is through link CE1-PE11, FA0 to 121, link 701 121-211, FA0 to PE21 and link PE21-CE2. 703 2. Another such path is through CE1-PE12, FA0 to node 122, 704 link 122-212, FA0 to PE22, link PE22-CE2. 706 o Minimize latency 708 * Color C2 - Minimum latency intent 710 * On CE1, flows requiring low latency paths to prefix V/24 are 711 steered over (C2, V/24) CAR route. 713 + BGP CAR for C2 sets up paths between CEs for minimum end-to- 714 end delay. 716 + This advertisement needs BGP CAR between PE-CE for V/24 717 prefix and color C2 awareness. 719 + It also needs BGP VPN CAR between PEs and ASBR for RD:V/24 720 prefix and color C2 awareness (C2, RD:V/24). 722 + Paths traverse over intra-domain Flex Algo 128 in each AS 723 and accounts for inter ASBR link delays and PE-CE link 724 delays for the VPN. 726 + Example: CE1 learns (C2, V/24) CAR best route through link 727 CE1-PE12, FA128 to 122, link 122-212, FA128 to PE22 and link 728 PE22-CE2. 730 3.2.2. Use-case of exclusion/inclusion of link affinity 732 o Color C3 - Intent to Minimize cost metric and avoid purple links 734 o In the reference topology of Figure 6 736 Each AS has Flex Algo 129 and some links have purple affinity. 738 Flex Algo 129 definition is set to minimum cost metric and 739 avoid purple links (within AS). 741 ASBR cross links are colored purple by policy. Bottom PE-CE 742 links are colored purple as well by policy 744 o On CE1, flows requiring minimum cost path avoiding purple links to 745 V/24 are steered over (C3, V/24) BGP CAR route. 747 * BGP CAR for C3 setup paths between CEs for minimum end-to-end 748 cost and avoiding purple link affinity. 750 * This advertisement needs BGP CAR between PE-CE for V/24 prefix 751 and color C3 awareness 753 * It also needs BGP VPN CAR between PEs and ASBRs for RD:V/24 754 prefix and color C3 awareness (C3, RD:V/24). 756 * The path avoids purple PE-CE links, traverses over intra-domain 757 Flex Algo 129 in each AS and avoids purple links between VPN 758 ASBRs. 760 * Example: CE1 learns (C3, V/24) CAR route through link CE1-PE11, 761 FA129 to 121, link 121-211, FA129 to PE21 and link PE21-CE2. 763 3.2.3. Use-case of virtual network function chains in local and core 764 domains 766 +-----+ 767 ........................|S-RR | <................. 768 : +-----+ <........... : 769 : : : 770 : : : 771 : ___ ___ ___ : : 772 : / \ / \ / \ : : 773 :| S1 | | S2 | | S3 | : : 774 : \ / \ / \ / : : 775 +-:---------------+ +--------------+ +------:----:--+ 776 | : |E11| |E12| | | |E21| | | : : | 777 | : +---+ +---+ | | +---+ | | : +----|<-(V1/24,C1) 778 | : | | | | : |PE31|--CE2 779 | V | | | | : +----| 780 |----+ +------+ +------+ : | 781 CE1--|PE11| | 121 | | 231 | : | 782 |----+ +------+ +------+ : +----|<-(V2/24/C2) 783 | | | | | :..|PE32|--CE3 784 | | | | | +----| 785 | | | | | | 786 | | | | | | 787 | Domain 1 | | Domain 2 | | Domain 3 | 788 +-----------------+ +--------------+ +--------------+ 790 Figure 7 792 o Color intent 794 * C1 - Routing via NFV service chain comprising of [S1, S2] 795 attached to E11 and E12 797 * C2 - Routing via NFV service [S3] attached to E21 799 o CE1, CE2, CE3 are sites of VPN1. 801 o Prefix V1/24 colored with C1 from CE2, and advertised as RD:V1/24 802 with C1 by PE31 to PE11 via S-RR 804 o Prefix V2/24 colored with C2 from CE3, and advertised as RD:V2/24 805 with C2 by PE32 to PE11 via SS-RR 807 o From PE11: 809 * [V1/24, C1] mapped packets are sent via S1, S2 and then routed 810 to PE31, CE2 812 * [V2/24, C2] mapped packets are sent via S3 and then routed to 813 PE32, CE3 815 4. Deployment Requirements 817 The figure below shows a reference large-scale multi-domain network 818 topology for targeted deployments. E1 and E2 are PEs; the other 819 nodes are border routers between domains in different tiers of the 820 network. A VPN route is advertised via service RRs (S-RR) between an 821 egress PE (E2) and an ingress PE (E1). BGP must provide reachability 822 from E1 to E2 based on various intent. 824 +-----+ +-----+ V/v via E2 +-----+ 825 .......|S-RR1|<...............|S-RR2|<.................|S-RR3| <.. 826 : +-----+ +-----+ Color C +-----+ : 827 : : 828 : : 829 +--:----------+-------------+--------------+-------------+----------:--+ 830 | : | | | | : | 831 | : | | | | : | 832 | : +---+ +---+ +---+ +---+ : | 833 | : |121| |231| |341| |451| : | 834 | : +---+ +---+ +---+ +---+ : | 835 |----+ | | | | +----| 836 | E1 | | | | | | E2 | 837 |----+ | | | | +----| 838 | +---+ +---+ +---+ +---+ | 839 | |122| |232| |342| |452| | 840 | +---+ +---+ +---+ +---+ | 841 | Access | Metro | Core | Metro | Access | 842 | domain 1 | domain 2 | domain 3 | domain 4 | domain 5 | 843 +-------------+-------------+--------------+-------------+-------------+ 844 iPE iBRM iBRC eBRC eBRM ePE 846 Figure 8: Reference large-scale multi-domain network topology 848 The solution must support the following : 850 o Co-existence, compatibility and interworking with currently 851 deployed SR-PCE based multi-domain color-aware solution 853 o Support different multi-domain deployment designs 855 * Multiple IGP domains within a single AS (Seamless MPLS) 856 + Inter-connect at node level (ABR) 858 * Multiple BGP AS domains 860 + Inter-connect via peering links (ASBR) 862 o Support end-to-end path crossing transport domains with different 863 technologies and encapsulations 865 * LDP-MPLS 867 * RSVP-TE-MPLS 869 * SR-MPLS 871 * SRv6 873 * IPv4/IPv6 875 o Support interworking between domains with different encapsulations 876 (e.g, SR-MPLS and SRv6) 878 o Support multiple transport encapsulations within a domain for co- 879 existence and migration 881 o Provide a BGP-based control-plane solution for the use-case 882 illustrated in [RFC8604] together with deployment design 883 guidelines for the leverage of anycast and binding SIDs. 885 5. Scalability 887 5.1. Scale Requirements 889 o Support for massive scaled transport network 891 * Number of Remote PE's: >= 300k 893 * Number of Colors C: >= 5 895 o Scalable MPLS dataplane solution 897 * With one label per (C, Remote PE), the 1M MPLS dataplane does 898 not work. 900 * A notion of hierarchy or segment list is required. 902 + E.g. the SR-PCE builds the end-to-end path as a list of 903 segments such that no single node needs to support a data- 904 plane scaling in the order of (Remote PE * C) 906 + The solution is thus not a direct extension of BGP-LU 908 * Additionally, PE and transit nodes (ABRs) may be devices with 909 limited forwarding table space 911 * Devices may have constraints on packet processing (e.g., label 912 operations, number of labels pushed) and performance 914 o Ability to abstract the topology from remote domains - for scale, 915 stability and faster convergence 917 * Abstracting PE and/or ABR related state and network events 919 o Support for an Emulated-PULL model for the BGP signaling 921 * The SR-PCE solution natively supports a PULL model: when PE1 922 installs a VPN route V/v via (C, PE2), PE1 requests its serving 923 SR-PCE to compute the SR Policy to (C, PE2). I.e. PE1 does 924 not learn unneeded SR policies. 926 * BGP Signaling is natively a PUSH model. 928 * Emulated-PULL refers to the ability for a BGP CAR node PE1 to 929 "subscribe" to (C, PE2) route such that only the related paths 930 are signaled to PE1. 932 * The subscription and related filtering solution must apply to 933 any BGP CAR node 935 + Transport CAR routes 937 1. Ability for a node (PE/ABR/RR) to signal interest for 938 routes of specific colors. 940 2. PEs only learn routes that they need - remote VPN 941 endpoints (PEs/ASBRs) or transit nodes (ABRs, ASBRs). 943 3. ABRs also only learn and propagate routes they need 944 locally in domain 946 + Service/VPN CAR routes 948 1. Ability for a node (PE) to signal interest for a 949 specific (Egress PE, Color) transport route 951 2. CEs learn routes that they need - interested colors 953 3. PEs learn routes that they need - interested VPNs, 954 colors 956 + Automation of the subscription/filter route 958 1. Similar to the SR-PCE solution, when an ingress PE1 959 installs VPN V/v via (C, PE2), PE1 originates its 960 subscription/filter route for (C, PE2). 962 + Efficient propagation and processing of subscription/filter 963 routes. 965 + Ability to perform aggregation and suppression of 966 subscription/filter routes at nodes in the route propagation 967 path to reduce explosion and churn in propagation of the 968 filter routes themselves. 970 + The solution may be optional for networks that do not have 971 the large scaling requirements 973 5.2. Scale Analysis 975 It is useful to analyze the multiple scaling requirements and 976 specifically the data plane constraints in the context of a few 977 common reference designs and use-cases. 979 A couple of example scenarios are listed below for reference. 981 o Seamless-MPLS design, with IGP Flex-Algo in each domain 982 +-----+ 983 .............................. |S-RR | <........................ 984 : +-----+ : 985 : : 986 : : 987 +--:-----------------+ +--------------------+ +-----------------:--+ 988 | : | | | | : | 989 | : | | | | : | 990 | : +------+ +------+ : | 991 | : | 121 | | 231 | : | 992 | V +------+ +------+ : | 993 |----+ | | | | +----| 994 |PE11| | | | | |PE31| 995 |----+ | | | | +----| 996 | +------+ +------+ | 997 | | 122 | | 232 | | 998 | +------+ +------+ | 999 | | | | | | 1000 | Domain 1 | | Domain 2 | | Domain 3 | 1001 +--------------------+ +--------------------+ +--------------------+ 1003 Figure 9 1005 o Inter-AS Option C VPN design, with IGP Flex-Algo in each domain, 1006 and eBGP peering between domains 1007 +-----+ +-----+ +-----+ 1008 ....|S-RR1| <............. |S-RR2| <............... |S-RR3| <... 1009 : +-----+ +-----+ +-----+ : 1010 : : 1011 : : 1012 : : 1013 +--:---------------+ +------------------+ +--------------:--+ 1014 | : | | | | : | 1015 | : | | | | : | 1016 | : +---| |---+ +---| |---+ : | 1017 | : |121|------|211| |231|------|321| : | 1018 | : +---| |---+ +---| |---+ : | 1019 |----+ | | | | +----| 1020 |PE11| | | | | |PE31| 1021 |----+ | | | | +----| 1022 | +---| |---+ +---| |---+ | 1023 | |122|------|212| |232|------|322| | 1024 | +---| |---+ +---| |---+ | 1025 | | | | | | 1026 | Domain 1 | | Domain 2 | | Domain 3 | 1027 +------------------+ +------------------+ +-----------------+ 1029 Figure 10 1031 6. Network Availability 1033 o The BGP CAR solution should provide high network availability for 1034 typical deployment topologies, with minimum loss of connectivity 1035 in different network failure scenarios. 1037 o The network failure scenarios, applicable technologies and design 1038 options described in [I-D.ietf-mpls-seamless-mpls] should be used 1039 as a reference. 1041 o In the Seamless-MPLS reference topology in previous section: 1043 * Failure of intra-domain links should limit loss of connectivity 1044 (LoC) to < 50ms. E.g., PE11 to a P node (not shown), 121 to a 1045 P node in Domain1 or Domain2) 1047 * Failure of an intra-domain node (P node in any domain) should 1048 limit LoC to < 50ms 1050 * Failure of an ABR node (e.g., 121, 231) should limit LoC to < 1051 1sec 1053 * Failure of a remote PE node (e.g., PE3) should limit LoC to < 1054 1sec 1056 o In the Inter-AS Option C VPN reference topology in previous 1057 section: 1059 * Failure of intra-domain links should limit LoC to < 50ms. 1060 E.g., PE11 to a P node (not shown), 121 to a P node in Domain1 1061 or Domain2) 1063 * Failure of an intra-domain node (P node in any domain) should 1064 limit LoC to < 50ms 1066 * Failure of an ASBR node (e.g., 121, 211) should limit LoC to < 1067 1sec 1069 * Failure of a remote PE node (e.g., PE3) should limit LoC to < 1070 1sec 1072 * Failure of an external link (e.g., 121-211) should limit LoC to 1073 < 1sec 1075 o The solution should explore and describe additional techniques and 1076 design options that are applicable to further improve handling of 1077 the failure cases listed above. 1079 7. BGP Protocol Requirements 1081 o Support signaling and distribution of different Color-Aware routes 1082 to reach a participating node, e.g., a PE. Intent should be 1083 indicated by the notion of a Color as defined in SR Policy 1084 Architecture. 1086 * Signal different instances of a prefix distinguished by color 1088 * Signal intent associated with a given route 1090 o Support for a flexible NLRI definition to accommodate both 1091 efficiency of processing (e.g., packing) and future extensibility 1093 * Avoid limitations associated with existing SAFI NLRI 1094 definitions. For example, 24-bit label. 1096 o Support for validation of paths 1098 * Reachability of next-hop in control plane 1100 * Availability and programming of encapsulation in data plane 1101 * Validation of intent 1103 o Next-hop resolution for Color-Aware route 1105 * Flexibility to use different intra-domain and inter-domain 1106 mechanisms - IGP-FA, SR-TE, RSVP-TE, IGP, BGP-LU etc. 1108 * Recursive resolution over BGP Color-Aware routes 1110 * Ability to carry end-to-end cumulative metric for a given color 1112 * Support setting up an end-to-end Color-Aware path using a 1113 different/less preferred or best-effort paths in domains where 1114 a particular intent is not available 1116 o Separation of transport and VPN service semantics. 1118 * Allow for different route distribution planes for service vs 1119 transport routes. 1121 o Support signaling of different transport encapsulations 1123 o Support for signaling multiple encapsulations for co-existence and 1124 migration 1126 o Generation of BGP Color-Aware routes sourced from IGP-FA, SR-TE 1127 policies and BGP-LU from a domain 1129 o Support signaling across domains with different color mappings for 1130 a given intent. 1132 8. Future Considerations 1134 Multicast service intent 1136 9. Acknowledgements 1138 Many people contributed to this document. 1140 The authors would especially like to thank Jim Uttaro for his 1141 guidance on the work and feedback on many aspects of the problem 1142 statement. We would also like to thank Daniel Voyer, Luay Jalil and 1143 Robert Raszuk for their review and valuable suggestions. 1145 We also express our appreciation to Bruno Decreane, Keyur Patel, Jim 1146 Guichard, Alex Bogdanov, Dirk Steinberg, Hannes Gredler and Xiaohu Hu 1147 for discussions on several topics that have helped provide input to 1148 the document. We also thank Huaimo Chen for his valuable review 1149 comments. 1151 The authors would like to thank Stephane Litkowski for his detailed 1152 review and for making valuable suggestions to improve the quality of 1153 the document. We would also like to thank Kamran Raza and Kris 1154 Michelson for their review and comments on the document and to Simon 1155 Spraggs, Jose Liste and Jiri Chaloupka for their early inputs on the 1156 problem statement. 1158 10. References 1160 10.1. Normative References 1162 [I-D.agrawal-spring-srv6-mpls-interworking] 1163 Agrawal, S., ALI, Z., Filsfils, C., Voyer, D., and Z. Li, 1164 "SRv6 and MPLS interworking", draft-agrawal-spring-srv6- 1165 mpls-interworking-05 (work in progress), February 2021. 1167 [I-D.ietf-bess-srv6-services] 1168 Dawra, G., Filsfils, C., Talaulikar, K., Raszuk, R., 1169 Decraene, B., Zhuang, S., and J. Rabadan, "SRv6 BGP based 1170 Overlay Services", draft-ietf-bess-srv6-services-07 (work 1171 in progress), April 2021. 1173 [I-D.ietf-idr-bgp-ipv6-rt-constrain] 1174 Patel, K., Raszuk, R., Djernaes, M., Dong, J., and M. 1175 Chen, "IPv6 Extensions for Route Target Distribution", 1176 draft-ietf-idr-bgp-ipv6-rt-constrain-12 (work in 1177 progress), April 2018. 1179 [I-D.ietf-idr-tunnel-encaps] 1180 Patel, K., Velde, G. V. D., Sangli, S. R., and J. Scudder, 1181 "The BGP Tunnel Encapsulation Attribute", draft-ietf-idr- 1182 tunnel-encaps-22 (work in progress), January 2021. 1184 [I-D.ietf-lsr-flex-algo] 1185 Psenak, P., Hegde, S., Filsfils, C., Talaulikar, K., and 1186 A. Gulko, "IGP Flexible Algorithm", draft-ietf-lsr-flex- 1187 algo-15 (work in progress), April 2021. 1189 [I-D.ietf-spring-segment-routing-policy] 1190 Filsfils, C., Talaulikar, K., Voyer, D., Bogdanov, A., and 1191 P. Mattes, "Segment Routing Policy Architecture", draft- 1192 ietf-spring-segment-routing-policy-11 (work in progress), 1193 April 2021. 1195 [I-D.ietf-spring-sr-service-programming] 1196 Clad, F., Xu, X., Filsfils, C., Bernier, D., Li, C., 1197 Decraene, B., Ma, S., Yadlapalli, C., Henderickx, W., and 1198 S. Salsano, "Service Programming with Segment Routing", 1199 draft-ietf-spring-sr-service-programming-04 (work in 1200 progress), March 2021. 1202 [I-D.ietf-spring-srv6-network-programming] 1203 Filsfils, C., Garvia, P. C., Leddy, J., Voyer, D., 1204 Matsushima, S., and Z. Li, "Segment Routing over IPv6 1205 (SRv6) Network Programming", draft-ietf-spring-srv6- 1206 network-programming-28 (work in progress), December 2020. 1208 [I-D.voyer-pim-sr-p2mp-policy] 1209 Voyer, D., Filsfils, C., Parekh, R., Bidgoli, H., and Z. 1210 Zhang, "Segment Routing Point-to-Multipoint Policy", 1211 draft-voyer-pim-sr-p2mp-policy-02 (work in progress), July 1212 2020. 1214 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1215 Requirement Levels", BCP 14, RFC 2119, 1216 DOI 10.17487/RFC2119, March 1997, 1217 . 1219 [RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended 1220 Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, 1221 February 2006, . 1223 [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 1224 R., Patel, K., and J. Guichard, "Constrained Route 1225 Distribution for Border Gateway Protocol/MultiProtocol 1226 Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual 1227 Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, 1228 November 2006, . 1230 [RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter, 1231 "Multiprotocol Extensions for BGP-4", RFC 4760, 1232 DOI 10.17487/RFC4760, January 2007, 1233 . 1235 [RFC5512] Mohapatra, P. and E. Rosen, "The BGP Encapsulation 1236 Subsequent Address Family Identifier (SAFI) and the BGP 1237 Tunnel Encapsulation Attribute", RFC 5512, 1238 DOI 10.17487/RFC5512, April 2009, 1239 . 1241 [RFC5701] Rekhter, Y., "IPv6 Address Specific BGP Extended Community 1242 Attribute", RFC 5701, DOI 10.17487/RFC5701, November 2009, 1243 . 1245 [RFC7311] Mohapatra, P., Fernando, R., Rosen, E., and J. Uttaro, 1246 "The Accumulated IGP Metric Attribute for BGP", RFC 7311, 1247 DOI 10.17487/RFC7311, August 2014, 1248 . 1250 [RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K. 1251 Patel, "Revised Error Handling for BGP UPDATE Messages", 1252 RFC 7606, DOI 10.17487/RFC7606, August 2015, 1253 . 1255 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 1256 Writing an IANA Considerations Section in RFCs", BCP 26, 1257 RFC 8126, DOI 10.17487/RFC8126, June 2017, 1258 . 1260 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1261 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1262 May 2017, . 1264 [RFC8277] Rosen, E., "Using BGP to Bind MPLS Labels to Address 1265 Prefixes", RFC 8277, DOI 10.17487/RFC8277, October 2017, 1266 . 1268 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1269 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1270 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1271 July 2018, . 1273 [RFC8664] Sivabalan, S., Filsfils, C., Tantsura, J., Henderickx, W., 1274 and J. Hardwick, "Path Computation Element Communication 1275 Protocol (PCEP) Extensions for Segment Routing", RFC 8664, 1276 DOI 10.17487/RFC8664, December 2019, 1277 . 1279 [RFC8669] Previdi, S., Filsfils, C., Lindem, A., Ed., Sreekantiah, 1280 A., and H. Gredler, "Segment Routing Prefix Segment 1281 Identifier Extensions for BGP", RFC 8669, 1282 DOI 10.17487/RFC8669, December 2019, 1283 . 1285 10.2. Informative References 1287 [I-D.filsfils-spring-sr-policy-considerations] 1288 Filsfils, C., Talaulikar, K., Krol, P., Horneffer, M., and 1289 P. Mattes, "SR Policy Implementation and Deployment 1290 Considerations", draft-filsfils-spring-sr-policy- 1291 considerations-07 (work in progress), April 2021. 1293 [I-D.ietf-idr-performance-routing] 1294 Xu, X., Hegde, S., Talaulikar, K., Boucadair, M., and C. 1295 Jacquenet, "Performance-based BGP Routing Mechanism", 1296 draft-ietf-idr-performance-routing-03 (work in progress), 1297 December 2020. 1299 [I-D.ietf-mpls-seamless-mpls] 1300 Leymann, N., Decraene, B., Filsfils, C., Konstantynowicz, 1301 M., and D. Steinberg, "Seamless MPLS Architecture", draft- 1302 ietf-mpls-seamless-mpls-07 (work in progress), June 2014. 1304 [RFC3906] Shen, N. and H. Smit, "Calculating Interior Gateway 1305 Protocol (IGP) Routes Over Traffic Engineering Tunnels", 1306 RFC 3906, DOI 10.17487/RFC3906, October 2004, 1307 . 1309 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 1310 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 1311 DOI 10.17487/RFC4271, January 2006, 1312 . 1314 [RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis", 1315 RFC 4272, DOI 10.17487/RFC4272, January 2006, 1316 . 1318 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 1319 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 1320 2006, . 1322 [RFC6952] Jethanandani, M., Patel, K., and L. Zheng, "Analysis of 1323 BGP, LDP, PCEP, and MSDP Issues According to the Keying 1324 and Authentication for Routing Protocols (KARP) Design 1325 Guide", RFC 6952, DOI 10.17487/RFC6952, May 2013, 1326 . 1328 [RFC7911] Walton, D., Retana, A., Chen, E., and J. Scudder, 1329 "Advertisement of Multiple Paths in BGP", RFC 7911, 1330 DOI 10.17487/RFC7911, July 2016, 1331 . 1333 Authors' Addresses 1335 Dhananjaya Rao 1336 Cisco Systems 1337 USA 1339 Email: dhrao@cisco.com 1341 Swadesh Agrawal 1342 Cisco Systems 1343 USA 1345 Email: swaagraw@cisco.com 1347 Clarence Filsfils 1348 Cisco Systems 1349 Belgium 1351 Email: cfilsfil@cisco.com 1353 Ketan Talaulikar 1354 Cisco Systems 1355 India 1357 Email: ketant@cisco.com 1359 Bruno Decraene 1360 Orange 1361 France 1363 Email: bruno.decraene@orange.com 1365 Dirk Steinberg 1366 Lapishills Consulting Limited 1367 Germany 1369 Email: dirk@lapishills.com 1370 Luay Jalil 1371 Verizon 1372 USA 1374 Email: luay.jalil@verizon.com 1376 Jim Guichard 1377 Futurewei 1378 USA 1380 Email: james.n.guichard@futurewei.com 1382 Keyur Patel 1383 Arrcus, Inc 1384 USA 1386 Email: keyur@arrcus.com 1388 Wim Henderickx 1389 Nokia 1390 Belgium 1392 Email: wim.henderickx@nokia.com