idnits 2.17.1 draft-dskc-bess-bgp-car-problem-statement-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 4 instances of too long lines in the document, the longest one being 3 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (22 November 2021) is 886 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'S1' is mentioned on line 794, but not defined == Missing Reference: 'S2' is mentioned on line 794, but not defined == Missing Reference: 'S3' is mentioned on line 797, but not defined == Missing Reference: 'RFC8604' is mentioned on line 882, but not defined == Unused Reference: 'I-D.agrawal-spring-srv6-mpls-interworking' is defined on line 1162, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-bess-srv6-services' is defined on line 1169, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-idr-bgp-ipv6-rt-constrain' is defined on line 1177, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-spring-sr-service-programming' is defined on line 1207, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-spring-srv6-network-programming' is defined on line 1216, but no explicit reference was found in the text == Unused Reference: 'I-D.voyer-pim-sr-p2mp-policy' is defined on line 1224, but no explicit reference was found in the text == Unused Reference: 'RFC2119' is defined on line 1231, but no explicit reference was found in the text == Unused Reference: 'RFC4360' is defined on line 1236, but no explicit reference was found in the text == Unused Reference: 'RFC4684' is defined on line 1240, but no explicit reference was found in the text == Unused Reference: 'RFC4760' is defined on line 1247, but no explicit reference was found in the text == Unused Reference: 'RFC5512' is defined on line 1252, but no explicit reference was found in the text == Unused Reference: 'RFC5701' is defined on line 1258, but no explicit reference was found in the text == Unused Reference: 'RFC7311' is defined on line 1262, but no explicit reference was found in the text == Unused Reference: 'RFC7606' is defined on line 1267, but no explicit reference was found in the text == Unused Reference: 'RFC8126' is defined on line 1272, but no explicit reference was found in the text == Unused Reference: 'RFC8174' is defined on line 1277, but no explicit reference was found in the text == Unused Reference: 'RFC8277' is defined on line 1281, but no explicit reference was found in the text == Unused Reference: 'RFC8402' is defined on line 1285, but no explicit reference was found in the text == Unused Reference: 'RFC8664' is defined on line 1290, but no explicit reference was found in the text == Unused Reference: 'RFC8669' is defined on line 1296, but no explicit reference was found in the text == Unused Reference: 'I-D.filsfils-spring-sr-policy-considerations' is defined on line 1304, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-idr-performance-routing' is defined on line 1312, but no explicit reference was found in the text == Unused Reference: 'RFC3906' is defined on line 1327, but no explicit reference was found in the text == Unused Reference: 'RFC4271' is defined on line 1332, but no explicit reference was found in the text == Unused Reference: 'RFC4272' is defined on line 1337, but no explicit reference was found in the text == Unused Reference: 'RFC4364' is defined on line 1341, but no explicit reference was found in the text == Unused Reference: 'RFC6952' is defined on line 1345, but no explicit reference was found in the text == Unused Reference: 'RFC7911' is defined on line 1351, but no explicit reference was found in the text == Outdated reference: A later version (-14) exists of draft-agrawal-spring-srv6-mpls-interworking-06 == Outdated reference: A later version (-15) exists of draft-ietf-bess-srv6-services-08 == Outdated reference: A later version (-26) exists of draft-ietf-lsr-flex-algo-18 == Outdated reference: A later version (-22) exists of draft-ietf-spring-segment-routing-policy-14 == Outdated reference: A later version (-09) exists of draft-ietf-spring-sr-service-programming-05 ** Obsolete normative reference: RFC 5512 (Obsoleted by RFC 9012) == Outdated reference: A later version (-09) exists of draft-filsfils-spring-sr-policy-considerations-08 Summary: 4 errors (**), 0 flaws (~~), 40 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 BESS WorkGroup D. Rao 3 Internet-Draft S. Agrawal 4 Intended status: Informational C. Filsfils 5 Expires: 26 May 2022 K. Talaulikar 6 Cisco Systems 7 B. Decraene 8 Orange 9 D. Steinberg 10 Lapishills Consulting Limited 11 L. Jalil 12 Verizon 13 J. Guichard 14 Futurewei 15 K. Patel 16 Arrcus, Inc 17 W. Henderickx 18 Nokia 19 22 November 2021 21 BGP Color-Aware Routing Problem Statement 22 draft-dskc-bess-bgp-car-problem-statement-04 24 Abstract 26 This document explores the scope, use-cases and requirements for a 27 BGP based routing solution to establish end-to-end intent-aware paths 28 across a multi-domain service provider network environment. 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 26 May 2022. 47 Copyright Notice 49 Copyright (c) 2021 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Revised BSD License text as 58 described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Revised BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 1.1. Objective . . . . . . . . . . . . . . . . . . . . . . . . 3 65 1.2. Color-Aware Routing . . . . . . . . . . . . . . . . . . . 3 66 1.2.1. Intent . . . . . . . . . . . . . . . . . . . . . . . 4 67 1.2.2. Color . . . . . . . . . . . . . . . . . . . . . . . . 4 68 1.2.3. Colored Service Route . . . . . . . . . . . . . . . . 4 69 1.2.4. Color-Aware Route . . . . . . . . . . . . . . . . . . 4 70 1.2.5. Service Route Automated Steering on color-aware 71 route . . . . . . . . . . . . . . . . . . . . . . . . 5 72 1.2.6. Inter-Domain color-aware routing with SR Policy . . . 5 73 1.2.7. Need for a BGP-based color-aware routing solution . . 5 74 1.2.8. BGP Color-Aware Routing . . . . . . . . . . . . . . . 5 75 1.2.9. Architectural consistency among color-aware routing 76 solutions . . . . . . . . . . . . . . . . . . . . . . 5 77 1.2.10. Color Domains . . . . . . . . . . . . . . . . . . . . 7 78 1.2.11. Per-Destination and Per-Flow Steering with BGP CAR . 7 79 2. Intent bound to a Color . . . . . . . . . . . . . . . . . . . 8 80 3. BGP CAR Use-cases . . . . . . . . . . . . . . . . . . . . . . 8 81 3.1. BGP Transport CAR . . . . . . . . . . . . . . . . . . . . 8 82 3.1.1. Use-case of minimization of a cost metric vs a latency 83 metric . . . . . . . . . . . . . . . . . . . . . . . 10 84 3.1.2. Use-case of exclusion/inclusion of link affinity . . 11 85 3.1.3. Use-case of exclusion/inclusion of domains . . . . . 11 86 3.1.4. Use-case of virtual network function chains in local 87 and core domains . . . . . . . . . . . . . . . . . . 12 88 3.2. BGP VPN CAR . . . . . . . . . . . . . . . . . . . . . . . 13 89 3.2.1. Use-case of minimization of a cost metric vs a latency 90 metric . . . . . . . . . . . . . . . . . . . . . . . 16 91 3.2.2. Use-case of exclusion/inclusion of link affinity . . 17 92 3.2.3. Use-case of virtual network function chains in local 93 and core domains . . . . . . . . . . . . . . . . . . 18 94 4. Deployment Requirements . . . . . . . . . . . . . . . . . . . 19 95 5. Scalability . . . . . . . . . . . . . . . . . . . . . . . . . 20 96 5.1. Scale Requirements . . . . . . . . . . . . . . . . . . . 20 97 5.2. Scale Analysis . . . . . . . . . . . . . . . . . . . . . 22 98 6. Network Availability . . . . . . . . . . . . . . . . . . . . 24 99 7. BGP Protocol Requirements . . . . . . . . . . . . . . . . . . 25 100 8. Future Considerations . . . . . . . . . . . . . . . . . . . . 26 101 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 102 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 27 103 10.1. Normative References . . . . . . . . . . . . . . . . . . 27 104 10.2. Informative References . . . . . . . . . . . . . . . . . 30 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 107 1. Introduction 109 1.1. Objective 111 This document explores the scope, use-cases and requirements for a 112 BGP based routing solution to establish end-to-end intent-aware paths 113 across a multi-domain service provider network environment. 115 The targeted design outcome is to define the technology and protocol 116 extensions that may be required in a manner that addresses the widest 117 application. 119 The problem that the document initially focuses on is the BGP-based 120 delivery of an intent across several transport domains. To do this, 121 it describes existing intent-aware routing solutions that are 122 deployed and then extends the solution scope and architecture to BGP. 124 The problem space is then widened to include any intent (including 125 NFV chains and their location), any dataplane and the application of 126 the intent-based routing to the Service/VPN routes. All of this is 127 detailed in the rest of the document. 129 1.2. Color-Aware Routing 131 Color-Aware Routing (CAR) establishes routed paths that satisfy 132 specific intent in a network. This section describes the basic 133 concepts that define CAR and the protocols that currently support it. 135 The figure below is used as reference. 137 +-----------------------------------+ 138 |----+ +----| 139 | E1 | | E2 |- V/v with C 140 |----+ +----| 141 +-----------------------------------+ 143 Figure 1: Color-aware routing reference topology 145 1.2.1. Intent 147 Intent in routing may be any combination of the following behaviors: 149 * Topology path selection (e.g. minimize metric, avoid resource) 151 * NFV service insertion (e.g. service chain steering) 153 * Per-hop behavior (e.g. QoS for 5G slice) 155 An intent-aware routed path may be within a single network domain or 156 across multiple domains. 158 1.2.2. Color 160 Color is a 32-bit numerical value that is associated with an intent, 161 as defined in [I-D.ietf-spring-segment-routing-policy] 163 1.2.3. Colored Service Route 165 An Egress PE E2 colors a BGP service (e.g., VPN) route V/v to 166 indicate the particular intent that E2 requests for the traffic bound 167 to V/v. The color (C) is encoded as a BGP Color Extended community 168 [I-D.ietf-idr-tunnel-encaps]. 170 1.2.4. Color-Aware Route 172 (E2, C) is a color-aware route to E2 which satisfies the intent 173 associated with color C. 175 Multiple technologies already provide color-aware paths in solutions 176 that are widely deployed. 178 * SR Policy [I-D.ietf-spring-segment-routing-policy] 180 * IGP Flex-Algo [I-D.ietf-lsr-flex-algo] 182 In the context of large-scale SR-MPLS networks, SR Policy is 183 applicable to both intra-domain and inter-domain deployments; whereas 184 IGP Flex-Algo is better suited to intra-domain scenarios. 186 1.2.5. Service Route Automated Steering on color-aware route 188 An ingress PE E1 automatically steers V-destined packets onto a 189 Color-Aware path bound to (E2, C). If several such paths exist, a 190 preference scheme is used to select the best path: E.g. IGP Flex- 191 Algo first, then SR Policy. 193 1.2.6. Inter-Domain color-aware routing with SR Policy 195 If E1 and E2 are in different domains, E1 may request an SR-PCE in 196 its domain for a path to (E2, C). The SR-PCE (or a set of them) 197 computes the end-to-end path and installs it at E1 as an SR Policy. 198 The end-to-end color-aware path may seamlessly cross multiple 199 domains. 201 1.2.7. Need for a BGP-based color-aware routing solution 203 * An operator with an existing Seamless-MPLS/BGP-LU inter-domain 204 deployment [I-D.ietf-mpls-seamless-mpls] may prefer a BGP based 205 extension as a more incremental approach 207 * There may be an expectation that BGP would support a larger scale 209 * Trust boundaries in an inter-domain deployment leads to a 210 preference for a BGP peering based solution 212 1.2.8. BGP Color-Aware Routing 214 BGP Color-Aware Routing (CAR) is a new BGP solution which signals 215 intent-aware routes to reach a given destination (e.g., E2). (E2, C) 216 represents a BGP hop-by-hop distributed route that builds an inter- 217 domain color-aware path to E2 for color C. 219 1.2.9. Architectural consistency among color-aware routing solutions 221 As seen above, multiple technologies exist that provide color-aware 222 routing in a network. A BGP based solution must be compliant with 223 the existing principles that apply to them: 225 * Service routes MUST be colored using BGP Color Extended-Community 226 to request intent 228 - V/v via E, colored with C 230 * Colored service routes MUST be automatically steered on an 231 appropriate color-aware path 233 - V/v via E with C is steered via (E, C) 234 - (E, C) provided by any color-aware technology or protocol 236 * Color-aware routes MAY resolve recursively via other color-aware 237 routes 239 - (E, C) via N recursively resolves via (N, C) 241 Here is a brief example that illustrates these principles. 243 +----------------+ +----------------+ +----------------+ 244 | | | | | | V/v with C1 245 |----+ +----+ +----+ +----|/ 246 | E1 | | N1 | | N2 | | E2 |\ 247 |----+ +----+ +----+ +----| W/w with C2 248 | | | | | | 249 | Domain 1 | | Domain 2 | | Domain 3 | 250 +----------------+ +----------------+ +--- ------------+ 252 Figure 2: Color-aware routing inter-domain reference topology 254 In the figure above, all the nodes are part of an inter-domain 255 network under a single authority and with a consistent color-to- 256 intent mapping: 258 * Color C1 is mapped to "low delay" 260 - Flex-Algo FA1 is mapped to "low delay" and hence to C1 262 * Color C2 is mapped to "low delay and avoid resource R" 264 - Flex-Algo FA2 is mapped to "low delay and avoid resource R" and 265 hence to C2 267 E1 receives two BGP colored service routes from E2: 269 * V/v with BGP Color Extended community C1 271 * W/w with BGP Color Extended community C2 273 E1 has the following inter-domain color-aware paths: 275 * (E2, C1) provided by BGP CAR which recursively resolves via intra- 276 domain color-aware paths: 278 - (N1, C1) provided by IGP FA1 in Domain1 279 - (N2, C1) provided by SR Policy bound to color C1 in Domain2 281 * (E2, C2) provided by SR Policy 283 E1 automatically steers the received colored service routes as 284 follows: 286 * V/v via (E2, C1) provided by BGP CAR 288 * W/w via (E2, C2) provided by SR Policy 290 The example illustrates the benefits provided by leveraging the 291 architectural principles: 293 * Seamless co-existence of multiple color-aware technologies, e.g., 294 BGP CAR and SR Policy 296 - V/v is steered on BGP CAR color-aware path 298 - W/w is steered on SR Policy color-aware path 300 * Seamless and complementary interworking between different color- 301 aware technologies 303 - V/v is steered on a BGP CAR color-aware path that is itself 304 resolved within domain 2 onto an SR Policy bound to the color 305 of V/v 307 1.2.10. Color Domains 309 * A color domain represents a collection of one or more network 310 (IGP/BGP) domains with a single, consistent color-to-intent 311 mapping 313 * Color re-mapping may happen at color domain boundaries 315 1.2.11. Per-Destination and Per-Flow Steering with BGP CAR 317 Ingress PE E1 steers packets destined for a service (VPN) route V/v 318 via BGP Color-Aware Route R/r to E2 320 * Per-Destination Steering: Incoming packets on E1 match BGP service 321 route V/v to be steered based on the destination IP address of the 322 packets. 324 * Per-Flow Steering: Incoming packets on E1 match BGP service route 325 V/v to be steered based on the combination of the destination IP 326 address and additional elements in the packet header (i.e., IP 327 flow). Such a packet lookup may recurse on a forwarding array 328 where some of the entries are BGP color-aware routes to E2. A 329 given flow is mapped to a specific entry in this array i.e. via a 330 specific BGP color-aware route to E2. 332 2. Intent bound to a Color 334 The BGP CAR solution must support the following intents bound to a 335 color: 337 * Minimization of a cost metric vs a latency metric 339 - Minimization of different metric types, static and dynamic 341 * Exclusion/Inclusion of SRLG and/or Link Affinity and/or minimum 342 MTU/number of hops 344 * Bandwidth management 346 * In the inter-domain context, exclusion/inclusion of entire 347 domains, and border routers 349 * Inclusion of one or several virtual network function chains 351 - Located in a regional domain and/or core domain, in a DC 353 * Localization of the virtual network function chains 355 - Some functions may be desired in the regional DC or vice versa 357 * Per-Destination and Per-Flow steering 359 3. BGP CAR Use-cases 361 The BGP CAR route may be a transport route or a service route (in 362 this document, we use the term VPN instead of service for 363 simplicity). 365 3.1. BGP Transport CAR 367 * Transport Intent 369 - Intent-aware routing between PEs connected across multiple 370 transit domains 372 o Set up BGP based end-to-end paths stitching intent-aware 373 intra-domain segments 375 * The network diagram below illustrates the reference network 376 topology used in this section for Transport CAR: 378 +-----+ +-----+ +-----+ 379 .....|S-RR1| <............. |S-RR2| <............... |S-RR3| <... 380 : +-----+ +-----+ +-----+ : 381 : : 382 : : 383 : : 384 +--:--------------+ +-----------------+ +--------------:--+ 385 | : | | | | : | 386 | : | | | | : | 387 | : +---| D=20 |---+ +---| D=25 |---+ : | 388 | : |121|-------|211| |231|-------|321| : | 389 | : +---| \ / |---+ +---| \ / |---+ : | 390 |----+ | \ / | | \ / | +----| 391 |PE11| | V | | V | |PE31| 392 |----+ | / \ | | / \ | +----| 393 | +---| / \ |---+ +---| / \ |---+ | 394 | |122|-------|212| |232|-------|322| | 395 | +---| D=15 |---+ +---| D=10 |---+ | 396 | | | | | | 397 | Domain 1 | | Domain 2 | | Domain 3 | 398 +-----------------+ +-----------------+ +------------------+ 400 Figure 3: Transport CAR Reference Topology 402 The following network design assumptions apply to the reference 403 topology above, as an example: 405 - Independent ISIS/OSPF SR instance in each domain. 407 - eBGP peering link between ASBRs (121-211, 121-212, 122-211, 408 122-212, 231-321, 231-322, 232-321 and 232-322). 410 - Peering links have equal cost metric. 412 - Peering links have delay configured or measured as shown by 413 "D". D=50 for cross peering links. 415 - VPN service is running from PE31 to PE11 via service RRs (S-RRn 416 in figure). 418 * The following sections illustrate a few examples of intent use- 419 cases applicable to transport routes. 421 3.1.1. Use-case of minimization of a cost metric vs a latency metric 423 * In the reference topology of Figure 3 425 Each domain has Algo 0 and Flex Algo 128 427 Algo 0 is for minimum cost metric(cost optimized). 429 Flex Algo 128 definition is for minimum delay (low latency). 431 * Cost Optimized 433 - Color C1 - Minimum cost intent. (Here, a BGP CAR route with 434 Color C1 is being used, instead of BGP-LU.) 436 - On PE11, VPN routes colored with C1 are steered via (C1, PE31) 437 BGP CAR route 439 o BGP CAR for C1 sets up path(s) between PEs for end-to-end 440 minimum cost. 442 o (2) These paths traverse over intra-domain Algo 0 in each 443 domain and account for the peering link cost between ASBRs. 445 o Example: PE11 learns (C1, PE31) CAR route via several equal 446 paths: 448 1. One such path is through FA0 to node 121, links 121-211, 449 FA0 to 231, link 231-321, FA0 to PE31 451 2. Another such path is through FA0 to node 122, link 452 122-212, FA0 to 232, link 232-322, FA0 to PE31. 454 * Minimize latency 456 - Color C2 - Minimum latency intent. 458 - On PE11, VPN routes colored with C2 are steered via (C2, PE31) 459 BGP CAR route. 461 o BGP CAR for C2 advertises paths between PEs for minimum end- 462 to-end delay. 464 o (2) These paths traverse over intra-domain Flex Algo 128 in 465 each domain and account for peering link delay between 466 ASBRs. 468 o (3) Example: PE11 learns (C2, PE31) BGP CAR route and best 469 path is through FA128 to node 122, link 122-212, FA128 to 470 232, link 232-322, FA128 to PE31. 472 3.1.2. Use-case of exclusion/inclusion of link affinity 474 * Color C3 - Intent to Minimize cost metric and avoid purple links 476 * In the reference topology of Figure 3 478 Each domain has Flex Algo 129 and some links have purple 479 affinity. 481 Flex Algo 129 definition is set to minimum cost metric and 482 avoid purple links (within domain). 484 Peering cross links are colored purple by policy. 486 * On PE11, VPN routes colored with C3 are steered via (C3, PE31) BGP 487 CAR route. 489 - BGP CAR for C3 sets up paths between PEs for minimum end-to-end 490 cost and avoiding purple link affinity. 492 - These paths traverse over intra domain Flex Algo 129 in each 493 domain and accounts for peering link cost between ASBR and 494 avoiding purple links. 496 - Example: PE11 learns (C3, PE31) BGP CAR route via 2 paths. 498 1. First path is through FA 129 to node 121, link 121-211, 499 FA129 to 231, link 231-321, FA129 to PE31. 501 2. Second path is through FA129 to node 122, link 122-212, 502 FA129 to 232, link 232-322, FA129 to PE31. 504 3.1.3. Use-case of exclusion/inclusion of domains 505 +-----+ +-----+ +-----+ 506 ....|S-RR1| <............. |S-RR2| <.............. |S-RR3| <.... 507 : +-----+ +-----+ +-----+ : 508 : : 509 : +----------------+ : 510 : | | : 511 +--:--------------+ |---+ +---| +--------------:--+ 512 | : | |---|211| |241|---| | : | 513 | : | | |---+ +---| | | : | 514 | : +---| | | Domain 2 | | |---+ : | 515 | : |121|---| +----------------+ |---|421| : | 516 | : +---| |---+ : | 517 |----+ | | +----| 518 |PE11| | | |PE41| 519 |----+ | | +----| 520 | +---| |---+ | 521 | |131|---| +----------------+ |---|431| | 522 | +---| | | | | |---+ | 523 | | | |---+ +---| | | | 524 | Domain 1 | |---|311| |341|---| | Domain 4 | 525 +-----------------+ |---+ +---| +-----------------+ 526 | Domain 3 | 527 +----------------+ 529 Figure 4 531 Color C4 - Avoid sending selected traffic via Domain 3 533 * VPN routes advertised from PEs with Color C4 535 * BGP CAR for Color C4 should only set up paths between PE11 and 536 PE41 that exclude Domain 3 538 3.1.4. Use-case of virtual network function chains in local and core 539 domains 540 ____ ____ 541 / \ / \ 542 | NFV1 | | NFV2 | 543 \ / \ / 544 +---------------------+ +--------------------+ +-------------------+ 545 | |E11| | | |E21| | | | 546 | +---+ | | +---+ | | | 547 | | | | | | 548 | | | | | | 549 |----+ +------+ +------+ +----| 550 |PE11| | 121 | | 231 | |PE31| 551 |----+ +------+ +------+ +----| 552 | | | | | | 553 | | | | | | 554 | | | | | | 555 | | | | | | 556 | Domain 1 | | Domain 2 | | Domain 3 | 557 +---------------------+ +--------------------+ +-------------------+ 559 Figure 5 561 * Color intent 563 - C5 - Routing via min-cost paths 565 - C6 - Routing via a local NFV service chain situated at E11 567 - C7 - Routing via a centrally located NFV service chain situated 568 at E21 570 * Forwarding of packets from PE11 towards PE31: 572 - (C5, PE31) mapped packets are sent via nodes 121, 231 to PE31 574 - (C6, PE31) mapped packets are sent to E11 and then post-service 575 chain, via 121, 231 to PE31 577 - (C7, PE31) mapped packets are sent via 121 to E21 and then 578 post-service chain, via 231 to PE31 580 3.2. BGP VPN CAR 582 * VPN (Service layer) intent 584 - Extend the signaling of intent awareness end-to-end: CE site to 585 CE site across provider networks 586 o Provide ability for a CE to select paths through specific 587 PEs for a given intent 589 + Example-1: Certain intent in transport not available via 590 specific PEs 592 + Example-2: Certain CE-PE connection does not support 593 specific intent 595 + Example-3: Site access via certain CE does not support 596 specific intent. For instance, link connecting a 597 specific CE to a DC hosting loss-sensitive service may 598 have better quality than a link from another CE 600 o Provide ability for a CE to send traffic indicating a 601 specific intent (via suitable encapsulation) to the PE for 602 optimal steering. 604 - Intent aware routing support for multiple service (VPN) 605 interworking models 607 o Beyond options such as iBGP or Inter-AS Option C that 608 inherently extend from PE to PE 610 1. Inter-AS Option A 612 2. Inter-AS Option B 614 3. GW based interworking(L3VPN, EVPN) 616 o Interworking with existing L3VPN deployments, both PEs and 617 CEs 619 * The network diagram below illustrates the reference network 620 topology used in this section for VPN CAR. 622 +-------------------+ +-------------------+ 623 | ....|S-RR|.... | | ....|S-RR|..... | 624 | : +----+ : | | : +----+ : | 625 | : : : : | | : : : : | 626 |----+ : : +---| D=20 |---+ : : +----| 627 /|PE11| : : |121|-----------|211| : : |PE21|\ 628 D=25/ |----+ : : +---| X X |---+ : : +----| \ D=25 629 / | : : | X X | : : | \ V/24 630 CE1 | : : | X D=50| : : | CE2<--- 631 X | : : | X X | : : | X 632 D=15 X |----+ : : +---| X X |---+ : : +----| X D=10 633 X|PE12|...: :...|212|-----------|232|...: :...|PE22|X 634 |----+ +---| D=10 |---+ +----| 635 | | | | 636 | AS 1 | | AS 2 | 637 +-------------------+ +-------------------+ 639 Figure 6: VPN CAR reference topology 641 The following network design assumptions apply to the reference 642 topology above, as an example: 644 - Independent ISIS/OSPF SR instance in each AS. 646 - eBGP peering link between VPN ASBRs 121-211, 121-212, 122-211, 647 122-212. 649 - VPN service is running between PEs via service RRs in each AS 650 to local ASBRs. Between ASBRs, its Option-B i.e. next hop self 651 for VPN SAFI. 653 - CE1 is dual homed to PE11 and PE12. Similarly, CE2 is dual 654 homed to PE21 and PE22. 656 - Peering links have equal cost metric 658 - Peering links have delay configured or measured as shown by 659 "D". 661 - CE2 advertises prefix V/24 to CE1. It is advertised as RD:V/24 662 between PEs, including color-awareness 664 * The following sections illustrate a few examples of intent use- 665 cases applicable to VPN (service) routes. 667 3.2.1. Use-case of minimization of a cost metric vs a latency metric 669 * In the reference topology of Figure 6 671 Each AS has Flex Algo 0 and 128. 673 Flex Algo 0 is for minimum cost metric(cost optimized). 675 Flex Algo 128 definition is for minimum delay (low latency). 677 * Cost Optimized 679 - Color C1 - Minimum cost intent. 681 - On CE1, flows requiring cost optimized paths to V/24 are 682 steered over (C1, V/24) route. 684 o BGP CAR for C1 sets up paths between CEs for minimum end-to- 685 end cost. 687 o This advertisement needs BGP CAR between PE-CE for V/24 688 prefix and color C1 awareness. 690 o It also needs BGP VPN CAR between PEs and ASBRs for RD:V/24 691 prefix and color C1 awareness (C1, RD:V/24). 693 o Paths traverse over PE-CE links, intra-domain Flex Algo 0 in 694 each AS and peering links between ASBRs, minimizing cost for 695 VPN. 697 o Example: CE1 learns (C1, V/24) CAR route through several 698 equal cost paths: 700 1. One path is through link CE1-PE11, FA0 to 121, link 701 121-211, FA0 to PE21 and link PE21-CE2. 703 2. Another such path is through CE1-PE12, FA0 to node 122, 704 link 122-212, FA0 to PE22, link PE22-CE2. 706 * Minimize latency 708 - Color C2 - Minimum latency intent 710 - On CE1, flows requiring low latency paths to prefix V/24 are 711 steered over (C2, V/24) CAR route. 713 o BGP CAR for C2 sets up paths between CEs for minimum end-to- 714 end delay. 716 o This advertisement needs BGP CAR between PE-CE for V/24 717 prefix and color C2 awareness. 719 o It also needs BGP VPN CAR between PEs and ASBR for RD:V/24 720 prefix and color C2 awareness (C2, RD:V/24). 722 o Paths traverse over intra-domain Flex Algo 128 in each AS 723 and accounts for inter ASBR link delays and PE-CE link 724 delays for the VPN. 726 o Example: CE1 learns (C2, V/24) CAR best route through link 727 CE1-PE12, FA128 to 122, link 122-212, FA128 to PE22 and link 728 PE22-CE2. 730 3.2.2. Use-case of exclusion/inclusion of link affinity 732 * Color C3 - Intent to Minimize cost metric and avoid purple links 734 * In the reference topology of Figure 6 736 Each AS has Flex Algo 129 and some links have purple affinity. 738 Flex Algo 129 definition is set to minimum cost metric and 739 avoid purple links (within AS). 741 ASBR cross links are colored purple by policy. Bottom PE-CE 742 links are colored purple as well by policy 744 * On CE1, flows requiring minimum cost path avoiding purple links to 745 V/24 are steered over (C3, V/24) BGP CAR route. 747 - BGP CAR for C3 setup paths between CEs for minimum end-to-end 748 cost and avoiding purple link affinity. 750 - This advertisement needs BGP CAR between PE-CE for V/24 prefix 751 and color C3 awareness 753 - It also needs BGP VPN CAR between PEs and ASBRs for RD:V/24 754 prefix and color C3 awareness (C3, RD:V/24). 756 - The path avoids purple PE-CE links, traverses over intra-domain 757 Flex Algo 129 in each AS and avoids purple links between VPN 758 ASBRs. 760 - Example: CE1 learns (C3, V/24) CAR route through link CE1-PE11, 761 FA129 to 121, link 121-211, FA129 to PE21 and link PE21-CE2. 763 3.2.3. Use-case of virtual network function chains in local and core 764 domains 766 +-----+ 767 ........................|S-RR | <................. 768 : +-----+ <........... : 769 : : : 770 : : : 771 : ___ ___ ___ : : 772 : / \ / \ / \ : : 773 :| S1 | | S2 | | S3 | : : 774 : \ / \ / \ / : : 775 +-:---------------+ +--------------+ +------:----:--+ 776 | : |E11| |E12| | | |E21| | | : : | 777 | : +---+ +---+ | | +---+ | | : +----|<-(V1/24,C1) 778 | : | | | | : |PE31|--CE2 779 | V | | | | : +----| 780 |----+ +------+ +------+ : | 781 CE1--|PE11| | 121 | | 231 | : | 782 |----+ +------+ +------+ : +----|<-(V2/24/C2) 783 | | | | | :..|PE32|--CE3 784 | | | | | +----| 785 | | | | | | 786 | | | | | | 787 | Domain 1 | | Domain 2 | | Domain 3 | 788 +-----------------+ +--------------+ +--------------+ 790 Figure 7 792 * Color intent 794 - C1 - Routing via NFV service chain comprising of [S1, S2] 795 attached to E11 and E12 797 - C2 - Routing via NFV service [S3] attached to E21 799 * CE1, CE2, CE3 are sites of VPN1. 801 * Prefix V1/24 colored with C1 from CE2, and advertised as RD:V1/24 802 with C1 by PE31 to PE11 via S-RR 804 * Prefix V2/24 colored with C2 from CE3, and advertised as RD:V2/24 805 with C2 by PE32 to PE11 via SS-RR 807 * From PE11: 809 - [V1/24, C1] mapped packets are sent via S1, S2 and then routed 810 to PE31, CE2 812 - [V2/24, C2] mapped packets are sent via S3 and then routed to 813 PE32, CE3 815 4. Deployment Requirements 817 The figure below shows a reference large-scale multi-domain network 818 topology for targeted deployments. E1 and E2 are PEs; the other 819 nodes are border routers between domains in different tiers of the 820 network. A VPN route is advertised via service RRs (S-RR) between an 821 egress PE (E2) and an ingress PE (E1). BGP must provide reachability 822 from E1 to E2 based on various intent. 824 +-----+ +-----+ V/v via E2 +-----+ 825 .......|S-RR1|<...............|S-RR2|<.................|S-RR3| <.. 826 : +-----+ +-----+ Color C +-----+ : 827 : : 828 : : 829 +--:----------+-------------+--------------+-------------+----------:--+ 830 | : | | | | : | 831 | : | | | | : | 832 | : +---+ +---+ +---+ +---+ : | 833 | : |121| |231| |341| |451| : | 834 | : +---+ +---+ +---+ +---+ : | 835 |----+ | | | | +----| 836 | E1 | | | | | | E2 | 837 |----+ | | | | +----| 838 | +---+ +---+ +---+ +---+ | 839 | |122| |232| |342| |452| | 840 | +---+ +---+ +---+ +---+ | 841 | Access | Metro | Core | Metro | Access | 842 | domain 1 | domain 2 | domain 3 | domain 4 | domain 5 | 843 +-------------+-------------+--------------+-------------+-------------+ 844 iPE iBRM iBRC eBRC eBRM ePE 846 Figure 8: Reference large-scale multi-domain network topology 848 The solution must support the following : 850 * Co-existence, compatibility and interworking with currently 851 deployed SR-PCE based multi-domain color-aware solution 853 * Support different multi-domain deployment designs 855 - Multiple IGP domains within a single AS (Seamless MPLS) 856 o Inter-connect at node level (ABR) 858 - Multiple BGP AS domains 860 o Inter-connect via peering links (ASBR) 862 * Support end-to-end path crossing transport domains with different 863 technologies and encapsulations 865 - LDP-MPLS 867 - RSVP-TE-MPLS 869 - SR-MPLS 871 - SRv6 873 - IPv4/IPv6 875 * Support interworking between domains with different encapsulations 876 (e.g, SR-MPLS and SRv6) 878 * Support multiple transport encapsulations within a domain for co- 879 existence and migration 881 * Provide a BGP-based control-plane solution for the use-case 882 illustrated in [RFC8604] together with deployment design 883 guidelines for the leverage of anycast and binding SIDs. 885 5. Scalability 887 5.1. Scale Requirements 889 * Support for massive scaled transport network 891 - Number of Remote PE's: >= 300k 893 - Number of Colors C: >= 5 895 * Scalable MPLS dataplane solution 897 - With one label per (C, Remote PE), the 1M MPLS dataplane does 898 not work. 900 - A notion of hierarchy or segment list is required. 902 o E.g. the SR-PCE builds the end-to-end path as a list of 903 segments such that no single node needs to support a data- 904 plane scaling in the order of (Remote PE * C) 906 o The solution is thus not a direct extension of BGP-LU 908 - Additionally, PE and transit nodes (ABRs) may be devices with 909 limited forwarding table space 911 - Devices may have constraints on packet processing (e.g., label 912 operations, number of labels pushed) and performance 914 * Ability to abstract the topology from remote domains - for scale, 915 stability and faster convergence 917 - Abstracting PE and/or ABR related state and network events 919 * Support for an Emulated-PULL model for the BGP signaling 921 - The SR-PCE solution natively supports a PULL model: when PE1 922 installs a VPN route V/v via (C, PE2), PE1 requests its serving 923 SR-PCE to compute the SR Policy to (C, PE2). I.e. PE1 does 924 not learn unneeded SR policies. 926 - BGP Signaling is natively a PUSH model. 928 - Emulated-PULL refers to the ability for a BGP CAR node PE1 to 929 "subscribe" to (C, PE2) route such that only the related paths 930 are signaled to PE1. 932 - The subscription and related filtering solution must apply to 933 any BGP CAR node 935 o Transport CAR routes 937 1. Ability for a node (PE/ABR/RR) to signal interest for 938 routes of specific colors. 940 2. PEs only learn routes that they need - remote VPN 941 endpoints (PEs/ASBRs) or transit nodes (ABRs, ASBRs). 943 3. ABRs also only learn and propagate routes they need 944 locally in domain 946 o Service/VPN CAR routes 948 1. Ability for a node (PE) to signal interest for a 949 specific (Egress PE, Color) transport route 951 2. CEs learn routes that they need - interested colors 953 3. PEs learn routes that they need - interested VPNs, 954 colors 956 o Automation of the subscription/filter route 958 1. Similar to the SR-PCE solution, when an ingress PE1 959 installs VPN V/v via (C, PE2), PE1 originates its 960 subscription/filter route for (C, PE2). 962 o Efficient propagation and processing of subscription/filter 963 routes. 965 o Ability to perform aggregation and suppression of 966 subscription/filter routes at nodes in the route propagation 967 path to reduce explosion and churn in propagation of the 968 filter routes themselves. 970 o The solution may be optional for networks that do not have 971 the large scaling requirements 973 5.2. Scale Analysis 975 It is useful to analyze the multiple scaling requirements and 976 specifically the data plane constraints in the context of a few 977 common reference designs and use-cases. 979 A couple of example scenarios are listed below for reference. 981 * Seamless-MPLS design, with IGP Flex-Algo in each domain 982 +-----+ 983 .............................. |S-RR | <........................ 984 : +-----+ : 985 : : 986 : : 987 +--:-----------------+ +--------------------+ +-----------------:--+ 988 | : | | | | : | 989 | : | | | | : | 990 | : +------+ +------+ : | 991 | : | 121 | | 231 | : | 992 | V +------+ +------+ : | 993 |----+ | | | | +----| 994 |PE11| | | | | |PE31| 995 |----+ | | | | +----| 996 | +------+ +------+ | 997 | | 122 | | 232 | | 998 | +------+ +------+ | 999 | | | | | | 1000 | Domain 1 | | Domain 2 | | Domain 3 | 1001 +--------------------+ +--------------------+ +--------------------+ 1003 Figure 9 1005 * Inter-AS Option C VPN design, with IGP Flex-Algo in each domain, 1006 and eBGP peering between domains 1007 +-----+ +-----+ +-----+ 1008 ....|S-RR1| <............. |S-RR2| <............... |S-RR3| <... 1009 : +-----+ +-----+ +-----+ : 1010 : : 1011 : : 1012 : : 1013 +--:---------------+ +------------------+ +--------------:--+ 1014 | : | | | | : | 1015 | : | | | | : | 1016 | : +---| |---+ +---| |---+ : | 1017 | : |121|------|211| |231|------|321| : | 1018 | : +---| |---+ +---| |---+ : | 1019 |----+ | | | | +----| 1020 |PE11| | | | | |PE31| 1021 |----+ | | | | +----| 1022 | +---| |---+ +---| |---+ | 1023 | |122|------|212| |232|------|322| | 1024 | +---| |---+ +---| |---+ | 1025 | | | | | | 1026 | Domain 1 | | Domain 2 | | Domain 3 | 1027 +------------------+ +------------------+ +-----------------+ 1029 Figure 10 1031 6. Network Availability 1033 * The BGP CAR solution should provide high network availability for 1034 typical deployment topologies, with minimum loss of connectivity 1035 in different network failure scenarios. 1037 * The network failure scenarios, applicable technologies and design 1038 options described in [I-D.ietf-mpls-seamless-mpls] should be used 1039 as a reference. 1041 * In the Seamless-MPLS reference topology in previous section: 1043 - Failure of intra-domain links should limit loss of connectivity 1044 (LoC) to < 50ms. E.g., PE11 to a P node (not shown), 121 to a 1045 P node in Domain1 or Domain2) 1047 - Failure of an intra-domain node (P node in any domain) should 1048 limit LoC to < 50ms 1050 - Failure of an ABR node (e.g., 121, 231) should limit LoC to < 1051 1sec 1053 - Failure of a remote PE node (e.g., PE3) should limit LoC to < 1054 1sec 1056 * In the Inter-AS Option C VPN reference topology in previous 1057 section: 1059 - Failure of intra-domain links should limit LoC to < 50ms. 1060 E.g., PE11 to a P node (not shown), 121 to a P node in Domain1 1061 or Domain2) 1063 - Failure of an intra-domain node (P node in any domain) should 1064 limit LoC to < 50ms 1066 - Failure of an ASBR node (e.g., 121, 211) should limit LoC to < 1067 1sec 1069 - Failure of a remote PE node (e.g., PE3) should limit LoC to < 1070 1sec 1072 - Failure of an external link (e.g., 121-211) should limit LoC to 1073 < 1sec 1075 * The solution should explore and describe additional techniques and 1076 design options that are applicable to further improve handling of 1077 the failure cases listed above. 1079 7. BGP Protocol Requirements 1081 * Support signaling and distribution of different Color-Aware routes 1082 to reach a participating node, e.g., a PE. Intent should be 1083 indicated by the notion of a Color as defined in SR Policy 1084 Architecture. 1086 - Signal different instances of a prefix distinguished by color 1088 - Signal intent associated with a given route 1090 * Support for a flexible NLRI definition to accommodate both 1091 efficiency of processing (e.g., packing) and future extensibility 1093 - Avoid limitations associated with existing SAFI NLRI 1094 definitions. For example, 24-bit label. 1096 * Support for validation of paths 1098 - Reachability of next-hop in control plane 1100 - Availability and programming of encapsulation in data plane 1101 - Validation of intent 1103 * Next-hop resolution for Color-Aware route 1105 - Flexibility to use different intra-domain and inter-domain 1106 mechanisms - IGP-FA, SR-TE, RSVP-TE, IGP, BGP-LU etc. 1108 - Recursive resolution over BGP Color-Aware routes 1110 - Ability to carry end-to-end cumulative metric for a given color 1112 - Support setting up an end-to-end Color-Aware path using a 1113 different/less preferred or best-effort paths in domains where 1114 a particular intent is not available 1116 * Separation of transport and VPN service semantics. 1118 - Allow for different route distribution planes for service vs 1119 transport routes. 1121 * Support signaling of different transport encapsulations 1123 * Support for signaling multiple encapsulations for co-existence and 1124 migration 1126 * Generation of BGP Color-Aware routes sourced from IGP-FA, SR-TE 1127 policies and BGP-LU from a domain 1129 * Support signaling across domains with different color mappings for 1130 a given intent. 1132 8. Future Considerations 1134 Multicast service intent 1136 9. Acknowledgements 1138 Many people contributed to this document. 1140 The authors would especially like to thank Jim Uttaro for his 1141 guidance on the work and feedback on many aspects of the problem 1142 statement. We would also like to thank Daniel Voyer, Luay Jalil and 1143 Robert Raszuk for their review and valuable suggestions. 1145 We also express our appreciation to Bruno Decreane, Keyur Patel, Jim 1146 Guichard, Alex Bogdanov, Dirk Steinberg, Hannes Gredler and Xiaohu Hu 1147 for discussions on several topics that have helped provide input to 1148 the document. We also thank Huaimo Chen for his valuable review 1149 comments. 1151 The authors would like to thank Stephane Litkowski for his detailed 1152 review and for making valuable suggestions to improve the quality of 1153 the document. We would also like to thank Kamran Raza and Kris 1154 Michelson for their review and comments on the document and to Simon 1155 Spraggs, Jose Liste and Jiri Chaloupka for their early inputs on the 1156 problem statement. 1158 10. References 1160 10.1. Normative References 1162 [I-D.agrawal-spring-srv6-mpls-interworking] 1163 Agrawal, S., ALI, Z., Filsfils, C., Voyer, D., and Z. Li, 1164 "SRv6 and MPLS interworking", Work in Progress, Internet- 1165 Draft, draft-agrawal-spring-srv6-mpls-interworking-06, 22 1166 August 2021, . 1169 [I-D.ietf-bess-srv6-services] 1170 Dawra, G., Filsfils, C., Talaulikar, K., Raszuk, R., 1171 Decraene, B., Zhuang, S., and J. Rabadan, "SRv6 BGP based 1172 Overlay Services", Work in Progress, Internet-Draft, 1173 draft-ietf-bess-srv6-services-08, 10 November 2021, 1174 . 1177 [I-D.ietf-idr-bgp-ipv6-rt-constrain] 1178 Patel, K., Raszuk, R., Djernaes, M., Dong, J., and M. 1179 Chen, "IPv6 Extensions for Route Target Distribution", 1180 Work in Progress, Internet-Draft, draft-ietf-idr-bgp-ipv6- 1181 rt-constrain-12, 26 April 2018, 1182 . 1185 [I-D.ietf-idr-tunnel-encaps] 1186 Patel, K., Velde, G. V. D., Sangli, S. R., and J. Scudder, 1187 "The BGP Tunnel Encapsulation Attribute", Work in 1188 Progress, Internet-Draft, draft-ietf-idr-tunnel-encaps-22, 1189 7 January 2021, . 1192 [I-D.ietf-lsr-flex-algo] 1193 Psenak, P., Hegde, S., Filsfils, C., Talaulikar, K., and 1194 A. Gulko, "IGP Flexible Algorithm", Work in Progress, 1195 Internet-Draft, draft-ietf-lsr-flex-algo-18, 25 October 1196 2021, . 1199 [I-D.ietf-spring-segment-routing-policy] 1200 Filsfils, C., Talaulikar, K., Voyer, D., Bogdanov, A., and 1201 P. Mattes, "Segment Routing Policy Architecture", Work in 1202 Progress, Internet-Draft, draft-ietf-spring-segment- 1203 routing-policy-14, 25 October 2021, 1204 . 1207 [I-D.ietf-spring-sr-service-programming] 1208 Clad, F., Xu, X., Filsfils, C., Bernier, D., Li, C., 1209 Decraene, B., Ma, S., Yadlapalli, C., Henderickx, W., and 1210 S. Salsano, "Service Programming with Segment Routing", 1211 Work in Progress, Internet-Draft, draft-ietf-spring-sr- 1212 service-programming-05, 10 September 2021, 1213 . 1216 [I-D.ietf-spring-srv6-network-programming] 1217 Filsfils, C., Garvia, P. C., Leddy, J., Voyer, D., 1218 Matsushima, S., and Z. Li, "Segment Routing over IPv6 1219 (SRv6) Network Programming", Work in Progress, Internet- 1220 Draft, draft-ietf-spring-srv6-network-programming-28, 29 1221 December 2020, . 1224 [I-D.voyer-pim-sr-p2mp-policy] 1225 Voyer, D., Filsfils, C., Parekh, R., Bidgoli, H., and Z. 1226 Zhang, "Segment Routing Point-to-Multipoint Policy", Work 1227 in Progress, Internet-Draft, draft-voyer-pim-sr-p2mp- 1228 policy-02, 10 July 2020, . 1231 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1232 Requirement Levels", BCP 14, RFC 2119, 1233 DOI 10.17487/RFC2119, March 1997, 1234 . 1236 [RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended 1237 Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, 1238 February 2006, . 1240 [RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, 1241 R., Patel, K., and J. Guichard, "Constrained Route 1242 Distribution for Border Gateway Protocol/MultiProtocol 1243 Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual 1244 Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, 1245 November 2006, . 1247 [RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter, 1248 "Multiprotocol Extensions for BGP-4", RFC 4760, 1249 DOI 10.17487/RFC4760, January 2007, 1250 . 1252 [RFC5512] Mohapatra, P. and E. Rosen, "The BGP Encapsulation 1253 Subsequent Address Family Identifier (SAFI) and the BGP 1254 Tunnel Encapsulation Attribute", RFC 5512, 1255 DOI 10.17487/RFC5512, April 2009, 1256 . 1258 [RFC5701] Rekhter, Y., "IPv6 Address Specific BGP Extended Community 1259 Attribute", RFC 5701, DOI 10.17487/RFC5701, November 2009, 1260 . 1262 [RFC7311] Mohapatra, P., Fernando, R., Rosen, E., and J. Uttaro, 1263 "The Accumulated IGP Metric Attribute for BGP", RFC 7311, 1264 DOI 10.17487/RFC7311, August 2014, 1265 . 1267 [RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K. 1268 Patel, "Revised Error Handling for BGP UPDATE Messages", 1269 RFC 7606, DOI 10.17487/RFC7606, August 2015, 1270 . 1272 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 1273 Writing an IANA Considerations Section in RFCs", BCP 26, 1274 RFC 8126, DOI 10.17487/RFC8126, June 2017, 1275 . 1277 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1278 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1279 May 2017, . 1281 [RFC8277] Rosen, E., "Using BGP to Bind MPLS Labels to Address 1282 Prefixes", RFC 8277, DOI 10.17487/RFC8277, October 2017, 1283 . 1285 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1286 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1287 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1288 July 2018, . 1290 [RFC8664] Sivabalan, S., Filsfils, C., Tantsura, J., Henderickx, W., 1291 and J. Hardwick, "Path Computation Element Communication 1292 Protocol (PCEP) Extensions for Segment Routing", RFC 8664, 1293 DOI 10.17487/RFC8664, December 2019, 1294 . 1296 [RFC8669] Previdi, S., Filsfils, C., Lindem, A., Ed., Sreekantiah, 1297 A., and H. Gredler, "Segment Routing Prefix Segment 1298 Identifier Extensions for BGP", RFC 8669, 1299 DOI 10.17487/RFC8669, December 2019, 1300 . 1302 10.2. Informative References 1304 [I-D.filsfils-spring-sr-policy-considerations] 1305 Filsfils, C., Talaulikar, K., Krol, P., Horneffer, M., and 1306 P. Mattes, "SR Policy Implementation and Deployment 1307 Considerations", Work in Progress, Internet-Draft, draft- 1308 filsfils-spring-sr-policy-considerations-08, 22 October 1309 2021, . 1312 [I-D.ietf-idr-performance-routing] 1313 Xu, X., Hegde, S., Talaulikar, K., Boucadair, M., and C. 1314 Jacquenet, "Performance-based BGP Routing Mechanism", Work 1315 in Progress, Internet-Draft, draft-ietf-idr-performance- 1316 routing-03, 22 December 2020, 1317 . 1320 [I-D.ietf-mpls-seamless-mpls] 1321 Leymann, N., Decraene, B., Filsfils, C., Konstantynowicz, 1322 M., and D. Steinberg, "Seamless MPLS Architecture", Work 1323 in Progress, Internet-Draft, draft-ietf-mpls-seamless- 1324 mpls-07, 28 June 2014, . 1327 [RFC3906] Shen, N. and H. Smit, "Calculating Interior Gateway 1328 Protocol (IGP) Routes Over Traffic Engineering Tunnels", 1329 RFC 3906, DOI 10.17487/RFC3906, October 2004, 1330 . 1332 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 1333 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 1334 DOI 10.17487/RFC4271, January 2006, 1335 . 1337 [RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis", 1338 RFC 4272, DOI 10.17487/RFC4272, January 2006, 1339 . 1341 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 1342 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 1343 2006, . 1345 [RFC6952] Jethanandani, M., Patel, K., and L. Zheng, "Analysis of 1346 BGP, LDP, PCEP, and MSDP Issues According to the Keying 1347 and Authentication for Routing Protocols (KARP) Design 1348 Guide", RFC 6952, DOI 10.17487/RFC6952, May 2013, 1349 . 1351 [RFC7911] Walton, D., Retana, A., Chen, E., and J. Scudder, 1352 "Advertisement of Multiple Paths in BGP", RFC 7911, 1353 DOI 10.17487/RFC7911, July 2016, 1354 . 1356 Authors' Addresses 1358 Dhananjaya Rao 1359 Cisco Systems 1360 United States of America 1362 Email: dhrao@cisco.com 1364 Swadesh Agrawal 1365 Cisco Systems 1366 United States of America 1368 Email: swaagraw@cisco.com 1370 Clarence Filsfils 1371 Cisco Systems 1372 Belgium 1374 Email: cfilsfil@cisco.com 1375 Ketan Talaulikar 1376 Cisco Systems 1377 India 1379 Email: ketant@cisco.com 1381 Bruno Decraene 1382 Orange 1383 France 1385 Email: bruno.decraene@orange.com 1387 Dirk Steinberg 1388 Lapishills Consulting Limited 1389 Germany 1391 Email: dirk@lapishills.com 1393 Luay Jalil 1394 Verizon 1395 United States of America 1397 Email: luay.jalil@verizon.com 1399 Jim Guichard 1400 Futurewei 1401 United States of America 1403 Email: james.n.guichard@futurewei.com 1405 Keyur Patel 1406 Arrcus, Inc 1407 United States of America 1409 Email: keyur@arrcus.com 1411 Wim Henderickx 1412 Nokia 1413 Belgium 1415 Email: wim.henderickx@nokia.com