idnits 2.17.1 draft-cth-rtgwg-bgp-control-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (January 24, 2021) is 1159 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC4271' is defined on line 742, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-idr-bgpls-segment-routing-epe' is defined on line 765, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-idr-flowspec-path-redirect' is defined on line 771, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-isis-segment-routing-extensions' is defined on line 782, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-rtgwg-bgp-routing-large-dc' is defined on line 794, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-spring-segment-routing' is defined on line 799, but no explicit reference was found in the text ** Obsolete normative reference: RFC 1771 (Obsoleted by RFC 4271) ** Obsolete normative reference: RFC 3107 (Obsoleted by RFC 8277) ** Obsolete normative reference: RFC 5575 (Obsoleted by RFC 8955) ** Obsolete normative reference: RFC 7752 (Obsoleted by RFC 9552) == Outdated reference: A later version (-12) exists of draft-ietf-idr-flowspec-path-redirect-11 == Outdated reference: A later version (-26) exists of draft-ietf-idr-segment-routing-te-policy-11 Summary: 4 errors (**), 0 flaws (~~), 10 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Y. Luo 3 Internet-Draft L. Qu 4 Intended status: Informational China Telcom Co., Ltd. 5 Expires: July 28, 2021 X. Huang 6 Tencent 7 G. Mishra 8 Verizon Inc. 9 H. Chen 10 Futurewei 11 S. Zhuang 12 Z. Li 13 Huawei 14 January 24, 2021 16 Architecture for Use of BGP as Central Controller 17 draft-cth-rtgwg-bgp-control-06 19 Abstract 21 BGP is a core part of a network including Software-Defined Networking 22 (SDN) system. It has the traffic engineering information on the 23 network topology and can compute optimal paths for a given traffic 24 flow across the network. 26 This document describes some reference architectures for BGP as a 27 central controller. A BGP-based central controller can simplify the 28 operations on the network and use network resources efficiently for 29 providing services with high quality. 31 Requirements Language 33 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 34 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 35 document are to be interpreted as described in RFC 2119 [RFC2119]. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at https://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on July 28, 2021. 54 Copyright Notice 56 Copyright (c) 2021 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (https://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 72 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 3. Architectures . . . . . . . . . . . . . . . . . . . . . . . . 5 74 3.1. Building Blocks . . . . . . . . . . . . . . . . . . . . . 5 75 3.1.1. TEDB . . . . . . . . . . . . . . . . . . . . . . . . 5 76 3.1.2. SLDB . . . . . . . . . . . . . . . . . . . . . . . . 5 77 3.1.3. TPDB . . . . . . . . . . . . . . . . . . . . . . . . 5 78 3.1.4. CSPF . . . . . . . . . . . . . . . . . . . . . . . . 6 79 3.1.5. TM . . . . . . . . . . . . . . . . . . . . . . . . . 6 80 3.2. One Controller . . . . . . . . . . . . . . . . . . . . . 6 81 3.3. Controller Cluster . . . . . . . . . . . . . . . . . . . 8 82 3.4. Hierarchical Controllers . . . . . . . . . . . . . . . . 10 83 4. Application Scenarios . . . . . . . . . . . . . . . . . . . . 12 84 4.1. Business-oriented Traffic Steering . . . . . . . . . . . 12 85 4.1.1. Preferential Users . . . . . . . . . . . . . . . . . 12 86 4.1.2. Preferential Services . . . . . . . . . . . . . . . . 13 87 4.2. Traffic Congestion Mitigation . . . . . . . . . . . . . . 14 88 4.2.1. Congestion Mitigation in Core . . . . . . . . . . . . 15 89 4.2.2. Congestion Mitigation among ISPs . . . . . . . . . . 15 90 4.2.3. Congestion Mitigation at International Edge . . . . . 16 91 5. Security Considerations . . . . . . . . . . . . . . . . . . . 17 92 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 93 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 94 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 17 95 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 96 9.1. Normative References . . . . . . . . . . . . . . . . . . 18 97 9.2. Informative References . . . . . . . . . . . . . . . . . 18 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 100 1. Introduction 102 Border Gateway Protocol (BGP) [RFC1771] is an exterior gateway 103 protocol (EGP). It is developed to exchange routing information 104 among routers in different autonomous systems (ASes). Along its 105 developments, BGP has been extended to provide numerous new 106 functions. It collects the link states including traffic engineering 107 (TE) information from other protocols such as IGP and distributes 108 them among routers in different ASes [RFC7752]. It also controls the 109 redirection of traffic flows [RFC5575]. Furthermore, it distributes 110 MPLS labels [RFC3107]. For scalability, BGP is extended to have 111 Route Reflector (RR) [RFC4456]. 113 For segment routing (SR), BGP is extended to advertise SR policies 114 with candidate paths to the policy headend routers, which are 115 typically ingress routers [I-D.ietf-idr-segment-routing-te-policy]. 116 The SR specific PCEP extensions are defined in 117 [I-D.ietf-pce-segment-routing]. A stateful PCE can compute an SR 118 traffic engineering (SR-TE) path satisfying a set of constraints, and 119 initiate an SR-TE path on a headend router using the extensions. 121 An SDN controller (or controller for short) is the core of an SDN 122 system or network. It is between network elements (NEs) such as 123 routers or switches at one end and applications such as Operational 124 Support System (OSS) or Network Management System (NMS) at the other 125 end. The essential function of a controller is to steer traffic 126 flows across the network for providing more services with higher 127 quality. It manages network resources such as link bandwidth, 128 computes expected paths for carrying traffic flows based on available 129 network resources, programs the network elements for the creation of 130 tunnels along the paths, and redirects traffic flows into 131 corresponding tunnels. 133 Based on the current BGP, it is natural, beneficial and relatively 134 simple to extend BGP to become a controller. Using BGP as a 135 controller for a network will greatly simplify the operations on the 136 network. It avoids deploying, operating and maintaining a new extra 137 component or protocol such as PCE as a controller in the network. 139 This document describes some reference architectures for BGP as a 140 central controller and introduces some scenarios to which the BGP 141 controller can be applied. 143 2. Terminology 145 o SR: Segment Routing 147 o RR: Route Reflector 149 o SID: Segment Identifier 151 o SR-Path: Segment Routing Path 153 o SR-Tunnel: Segment Routing Tunnel 155 o TEDB: Traffic Engineering Database 157 o LSDB: Link State Database 159 o SLDB: SID/Label Database 161 o TPDB: Tunnel and Path Database 163 o CSPF: Constrained Shortest Path First 165 o TM: Tunnel Manager 167 o NMS: Network Management System 169 o SRLB: SR Local Block 171 o NE: Network Element 173 o PCE: Path Computation Element 175 o AS: Autonomous System 177 o QoS: Quality of Service 179 o ISP: Internet Service Provider 181 o MAN: Metropolitan Area Network 183 o OTT: Over the Top 185 o OTTSP: Over the Top Service Provider, or Content Operator 187 o AR: Access Router 189 3. Architectures 191 An architecture for the use of BGP as a central controller is based 192 on the essential function of a controller. It is constructed from 193 some building blocks or components. After introduction to building 194 blocks, a few of reference architectures are described in this 195 section. 197 3.1. Building Blocks 199 Some critical building blocks are briefed. They are Traffic 200 Engineering Database (TEDB or TED for short), SID/Label Database 201 (SLDB), Tunnel and Path Database (TPDB), Constrained Shortest Path 202 First (CSPF), and Tunnel Manager (TM). 204 3.1.1. TEDB 206 The Traffic Engineering Database (TEDB) stores the Traffic 207 Engineering (TE) information about the network. It includes the 208 unreserved bandwidth at each of eight priority levels for every link 209 in the network. 211 TEDB can be an individual block, which is constructed from the link 212 state information received. It may be embedded into the link state 213 database (LSDB) in the BGP when the BGP creates/updates the LSDB from 214 the link state information it receives. 216 3.1.2. SLDB 218 The SID/Label Database (SLDB) records and maintains the status of 219 every Segment Identifier (SID) and label for every node, interface/ 220 link and/or prefix in the network, which the controller controls. 221 The status of SID/label indicates whether the SID/Label is assigned. 222 If it is assigned, then the object such as the node, link or prefix, 223 to which it is assigned, is recorded. 225 SLDB can be an individual block, which is constructed from the link 226 state information such as SR Local Block (SRLB) that the BGP 227 receives. It may be embedded into the link state database (LSDB) in 228 the BGP when the BGP creates the LSDB from the link state information 229 it receives. 231 3.1.3. TPDB 233 The Tunnel and Path Database (TPDB) stores the information for every 234 tunnel, which includes: 236 o the parameters received for the tunnel from a user/application, 237 o the path computed for the tunnel, 239 o the resources such as link bandwidth reserved along the path for 240 the tunnel, 242 o the SID/labels assigned along the path for the tunnel, and 244 o the status of the tunnel. 246 3.1.4. CSPF 248 The Constrained Shortest Path First (CSPF) computes a path for a 249 tunnel such as SR tunnel or LSP tunnel that satisfies a set of given 250 constraints using the information in TEDB. 252 3.1.5. TM 254 The Tunnel Manager (TM) receives a request for an operation on a 255 tunnel from a user or an application such as Network Management 256 System (NMS). The operation may be a creation of a new tunnel, a 257 deletion of an existing tunnel, or a change to an existing tunnel. 259 When receiving a request for creating a new tunnel, the TM asks the 260 CSPF to compute a path for the tunnel that satisfies the constraints 261 given for the tunnel. 263 After obtaining the path for the tunnel from the CSPF, the TM 264 requests the SLDB to assign SID/labels along the path for the tunnel 265 and asks the TEDB to reserve the resources such as link bandwidth 266 along the path for the tunnel. 268 The TM in a central controller may set up the tunnel along the path 269 in the network by programming each of the NEs along the path through 270 the API to the network. In a SR network, the TM initiates a SR 271 tunnel in the network by sending a sequence of SID/labels to the 272 source NE of the tunnel. 274 The TM records the information for the tunnel in the Tunnel and Path 275 Database (TPDB). The information includes the path computed for the 276 tunnel, the resources such as bandwidth reserved along the path, the 277 SID/labels assigned along the path for the tunnel, and the status of 278 the tunnel. 280 3.2. One Controller 282 Figure below illustrates a reference architecture for using the BGP 283 as a central controller, which controls a network. The BGP as a 284 controller in the reference architecture controls a network through 285 an API to the network such as BGP+/RR+ (extensions to BGP for central 286 controller). The BGP controller is responsible for creating and 287 maintaining every tunnel in the network. It also controls the 288 redirection of traffic flow to each tunnel. 290 The BGP controller comprises a number of modules, including a TM, a 291 CSPF, a TEDB, a SLDB and a TPDB. The interfaces among these modules 292 are listed as follows: 294 +------------------------------------------+ 295 | Users/Applications(Orchestrator/OSS/NMS) | 296 +------------------------------------------+ 297 | 298 +----------------------------------------------+ 299 | BGP as Controller | 300 | +---------------+ | 301 | /------------| TM | | 302 | / Ia +---------------+ | 303 | +--------+ | | | \ | 304 | | CSPF | ________| | | \Id | 305 | +--------+ / Ib /Ic | +---------+ | 306 | \Ie / / | | TPDB | | 307 | +---------+ +-------+ | +---------+ | 308 | | TEDB | | SLDB | | | 309 | +---------+ +-------+ | | 310 | \ \ |In | 311 +----------------API to Network(RR+)-----------+ 312 / \ 313 / \____ 314 / \ \____ 315 /\ .---. .---+ \ 316 | \( ' |'.---. | 317 |---\ Network | '+. 318 (o \ | | ) 319 ( | | o) 320 ( | | ) 321 ( o o .-' 322 ' ) 323 '---._.-. ) 324 '---' 326 o Interface Ia between the TM and the CSPF. Through this interface, 327 the TM requests the CSPF to compute a path for a tunnel with a set 328 of constraints, and the CSPF responses the TM with the path 329 computed that satisfies the constraints. 331 o Interface Ib between the TM and the TEDB. When a tunnel is to be 332 created, through this interface, the TM reserves in the TEDB the 333 TE resources such as link bandwidths on every link along the path 334 computed for the tunnel. When a tunnel is deleted, the TM 335 releases the TE resources such as link bandwidths on every link 336 along the path for the tunnel. 338 o Interface Ic between the TM and the SLDB. When a tunnel is to be 339 created, through this interface, the TM reserves in the SLDB a 340 SID/label for every link or some links along the path computed for 341 the tunnel. When a tunnel is deleted, the TM releases the SID/ 342 label for every link or some links along the path for the tunnel. 344 o Interface Id between the TM and the TPDB. the TM updates the 345 information for every tunnel in the TPDB through this interface. 347 o Interface Ie between the CSPF and the TEDB. Through this 348 interface, the CSPF accesses the traffic engineering information 349 such as link bandwidths when it computes a path for a tunnel. 351 There is an interface In between the BGP controller and the network. 352 In fact, there is a control channel (or interface) between the BGP 353 controller and every (edge) node in the network. 355 Initially, the TEDB obtains the original traffic engineering (TE) 356 information such as link bandwidths from the network through the 357 interface In (i.e., API to network) for every link in the network. 358 The SLDB gets the original SID/label resources from the network 359 through the interface for every node, link and prefix in the network. 361 3.3. Controller Cluster 363 A critical issue in a network with a central controller is the 364 failure of the controller, which is a single point of failure (SPOF). 365 If the controller fails, the entire network may not work. 367 A controller cluster (i.e., a group of controllers) works as a single 368 controller from user's point of view. A simple controller cluster 369 consists of two controllers. One works as a active (or say primary) 370 controller, and the other as a standby (or say secondary) controller. 371 In normal operations, the active controller is responsible for the 372 network it controls. It also synchronizes with the standby 373 controller. When the active controller fails, the standby controller 374 becomes a new active controller, which controls the network. 376 The Figure below illustrates a simple controller cluster containing 377 two BGP-based controllers: Active BGP-based Controller and Standby 378 BGP-based Controller. In normal operations, the active controller 379 interacts with users and/or applications. For example, it receives 380 configurations for tunnels and the traffic flows to tunnels from 381 users. The active controller instructs the network elements in the 382 network to provide the services requested by users and/or 383 applications. For example, after receiving the configurations for a 384 tunnel and a traffic flow to the tunnel, the active controller 385 computes a path for the tunnel, programs (or say instructs) the 386 network elements along the path for creating the tunnel, and 387 instructs the ingress of the tunnel to direct the traffic flow into 388 the tunnel. 390 +-------------------------------------------+ 391 | Users/Applications(Orchestrator/OSS/NMS) | 392 +-------------------------------------------+ 393 ^ 394 | 395 +--------------------------+------------------------+ 396 | Controller ______________|_____________ | 397 | Cluster | | | 398 | | ___________________ | | 399 | | | Synchronization | | | 400 | v v v v | 401 | +------------+ +------------+ | 402 | | Active | | Standby | | 403 | | BGP-based | | BGP-based | | 404 | | Controller | | Controller | | 405 | +------------+ +------------+ | 406 | ^ ^ | 407 | |____________________________| | 408 | | | 409 | v | 410 +-----------------API to Network(RR+)---------------+ 411 / \ 412 / \____ 413 / \ \____ 414 /\ .---. .---+ \ 415 | \( ' |'.---. | 416 |---\ Network | '+. 417 (o \ | | ) 418 ( | | o) 419 ( | | ) 420 ( o o .-' 421 ' ) 422 '---._.-. ) 423 '---' 425 During this process, the status information about the network is 426 updated in the active controller. The information includes: the 427 traffic engineering information in their TEDBs, the SID/label 428 information in their SLDBs, and the configurations, paths, resources 429 and status for tunnels in their TPDBs. The active controller 430 synchronizes this information with the standby controller. Thus 431 these two controllers have the same status information about the 432 network. When the active controller fails, the standby controller 433 takes over the role of the active controller smoothly and becomes 434 active controller. 436 3.4. Hierarchical Controllers 438 The Figure below illustrates a system with hierarchical controllers. 439 There is one Parent Controller and four Child Controllers: Child 440 Controller 1, Child Controller 2, Child Controller 3 and Child 441 Controller 4. 443 +-------------------------------------------+ 444 | Users/Applications(Orchestrator/OSS/NMS) | 445 +----------------------+--------------------+ 446 | 447 +---------+---------+ 448 | Parent Controller | 449 +--+---------+----+-+ 450 _/| \ \____ 451 _/ | \ \____ 452 _/ | \ \__ 453 __/ | +---------+---------+ \ 454 __/ | |Child Controller 3 | | 455 / | +-------------------+ | 456 +---------+---------+ | / \ | 457 |Child Controller 1 | | .---. .---,\ | 458 +-------------------+ | ( ' ') | 459 / \ | ( Domain 3 ) | 460 .---. .---,\ | ( ) +---------+---------+ 461 ( ' ') | '-o-.--o) |Child Controller 4 | 462 ( Domain 1 ) | | +-------------------+ 463 ( ) | | / \____ 464 '-o-.---) +--------+----------+ \ / \ \____ 465 | |Child Controller 2 | \ /\ .---. .---+ \ 466 | +-------------------+ \ | \( ' |'.---. | 467 | / \____ \_ |---\ Domain 4 | '+, 468 \ / \ \____ (o \ | | ) 469 \ /\ .---. .---+ \ ( | | o) 470 \ | \( ' |'.---. | ( | | ) 471 \ |---\ Domain 2 | '+. ( o o .-' 472 \____(o \ | | ) ' ) 473 ( | | o)-------o---._.-.-----) 474 ( | | ) 475 ( o o .-' 476 ' ) 477 '---._.-.-----) 479 The parent controller communicates with these four child controllers 480 and controls them, each of which controls (or is responsible for) a 481 domain. Child controller 1 controls domain 1, Child controller 2 482 controls domain 2, Child controller 3 controls domain 3, and Child 483 controller 4 controls domain 4. 485 One level of hierarchy of controllers is illustrated in the figure 486 above. There is one parent controller at top level, which is not a 487 child controller. Under the parent controller, there are four child 488 controllers, which are not parent controllers. 490 In a general case, at top level there is one parent controller that 491 is not a child controller, there are some controllers that are both 492 parent controllers and child controllers, and there are a number of 493 child controllers that are not parent controllers. This is a system 494 of multiple levels of hierarchies, in which one parent controller 495 controls or communicates with a first number of child controllers, 496 some of which are also parent controllers, each of which controls or 497 communicates with a second number of child controllers, and so on. 499 The parent controller receives requests for creating end to end 500 tunnels from users or applications. For each request, the parent 501 controller is responsible for obtaining a path for the tunnel and 502 creating the tunnel along the path through sending instructions to 503 the corresponding child controllers. 505 4. Application Scenarios 507 This section introduces a set of scenarios to which the controller 508 can be applied. 510 4.1. Business-oriented Traffic Steering 512 It is reasonable in commercial sense to provide multiple paths to the 513 same destination with differentiated experiences for preferential 514 users/services. This is an efficient approach to maximize providers' 515 network resource usage as well as their profit and offer more choices 516 to network users. 518 4.1.1. Preferential Users 520 In the Figure below for an ISP network, there are three kinds of 521 users in Sydney, saying Gold, Silver and Bronze, and they wish to 522 visit website located in HongKong. The ISP provides three different 523 paths with different experiences according to users' priority. The 524 Gold Users may use Path1 with less latency and loss. The Silver 525 Users may use the Path2 through Singapore with less latency but maybe 526 some congestion there. The Bronze Users may use Path3 through LA 527 with some latency and loss. 529 +----------+ 530 | HongKong | 531 --+----------+-- 532 --- | --- 533 --- | --- 534 -- | -- 535 +----------+ | +----------+ 536 |Singapore | | | LA | 537 +----------+ | +----------+ 538 -- |Path1 -- 539 --- | --- 540 Path2 --- | --- Path3 541 --+----------+-- 542 | Sydney | 543 +----------+ 544 | 545 | 546 +-----------+-----------+ 547 | | | 548 +-------+ +-------+ +-------+ 549 |Silver | |Gold | |Bronze | 550 |Users | |Users | |Users | 551 +-------+ +-------+ +-------+ 553 4.1.2. Preferential Services 555 As depicted in the Figure below, the OTTSP has 3 exits with one ISP, 556 which are located in City A, City B and City C. The content is 557 obtained from Content Server and send to the exits through AR. An 558 OTTSP may make its steering strategy based on different services. 559 For example, the OTTSP in the Figure may choose exit R21 for video 560 service and exit R22 for web service, which REQUIREs a mechanism/ 561 system exists to identify different services from traffic flow. 563 * * 564 City A * City B * City C 565 * * 566 * +-----+ * 567 * |Users| * 568 * +-----+ * 569 * | * 570 +-----------+-----------+ 571 | * | * | 572 +-----+ * +-----+ * +-----+ 573 | R11 |-----| R12 |-----| R13 | 574 +-----+ * +-----+ * +-----+ ISP 575 | * | * | 576 *****|***********|***********|********* 577 | * | * | 578 | * | * | OTT 579 +-----+ * +-----+ * +-----+ 580 | R21 |-----| R22 |-----| R23 | 581 +-----+ * +-----+ * +-----+ 582 | * | * | 583 +-----------+-----------+ 584 * | * 585 * +-----+ * +-------+ 586 * | AR |--------|Content| 587 * +-----+ * |Server | 588 +-------+ 590 4.2. Traffic Congestion Mitigation 592 It is a persistent goal for providers to increase the utilization 593 ratio of their current network resources, and to mitigate the traffic 594 congestion. Traffic congestion is possible to happen anywhere in the 595 ISP network(MAN, IDC, core and the links between them), because 596 internet traffic is hard to predict. For example, there might be 597 some local online events that the network operators didn't know 598 beforehand, or some sudden attack just happened. Even for the big 599 events that can be predicted, such as annual online discount of 600 e-commerce company, or IOS update of Apple Inc, we could not 601 guarantee there is no congestion. Since the network capacity 602 expansion is usually an annual operation, there could be delay on any 603 links of the engineering. As a result, the temporary traffic 604 steering is always needed. The same thing happens to the OTT 605 networks as well. 607 It should be noted that, the traffic steering is absolutely not a 608 global behavior. It just acts on part of the network, and it's 609 temporary. 611 4.2.1. Congestion Mitigation in Core 613 As depicted in the Figure below, traffic from MAN C1 to MAN D2 614 follows the path Core C->Core B->Core D as the primary path, but 615 somehow the load ratio becomes too much. It is reasonable to 616 transfer some traffic load to less utilized path Core C->Core A->Core 617 D when the primary path has congestion. 619 Core 621 +----------+ 622 | Core A | 623 +------+ --+----------+-- +------+ 624 |MAN C1|-+ --- --- +-|MAN D1| 625 +------+ | --- --- | +------+ 626 | -- -- | 627 | +----------+ +----------+ | 628 +-| Core C | | Core D |-+ 629 | +----------+ +----------+ | 630 | -- -- | 631 +------+ | --- --- | +------+ 632 |MAN C2|-+ --- --- +-|MAN D2| 633 +------+ --+----------+-- +------+ 634 | Core B | 635 +----------+ 637 4.2.2. Congestion Mitigation among ISPs 639 As depicted in the Figure below, ISP1 and ISP2 are interconnect by 3 640 exits which are located in 3 cities respectively. The links between 641 ISP1 and ISP2 in the same city are called local links, and the rest 642 are long distance links. Traffic from IXP C1 to Core A in ISP 2 643 usually passes through link IXP C1->IXP A2->Core A. This is a long 644 distant route, directly connecting city C and city A. Part of 645 traffic could be transferred to link IXP. 647 * * 648 City A * City B * City C 649 * * 650 +-------+ * +-------+ * +-------+ 651 |IXP A1 |----|IXP B1|---|IXP C1 | 652 +-------+ * +-------+ * +-------+ ISP 1 653 | * | * | | 654 *******|*************|*********|**|********** 655 | +----------|---------+ | 656 | | * | * | ISP 2 657 | | * | * | 658 +------+ * +------+ * +------+ 659 |IXP A2|----|IXP B2|----|IXP C2| 660 +------+ * +------+ * +------+ 661 | * | * | 662 | * | * | 663 +-------+ * +-------+ * +-------+ 664 |Core A |----|Core B |---|Core C | 665 +-------+ * +-------+ * +-------+ 667 4.2.3. Congestion Mitigation at International Edge 669 An ISP usually interconnects with more than 2 transit networks at the 670 international edge, so it is quite common that multiple paths may 671 exist for the same foreign destination. Usually those paths with 672 better QoS properties such as latency, loss, jitter and etc are often 673 preferred. Since these properties keep changing from time to time, 674 the decision of path selection has to be made dynamically. 676 As depicted in the Figure below, the traffic to the foreign 677 destination H from IP core network (AS C1) has two choices on transit 678 network, saying Transit A and Transit B. Under normal conditions, 679 Transit B is the primary choice, but Transit A will be preferred when 680 the QoS of Transit B gets worse. As a result, the same traffic will 681 go through Transit A instead. 683 * * 684 City A * City B * City C 685 * * 686 +-------+ * +-------+ * +-------+ 687 |IXP A1 |----|IXP B1|---|IXP C1 | 688 +-------+ * +-------+ * +-------+ ISP 1 689 | * | * | | 690 *******|*************|*********|**|********** 691 | +----------|---------+ | 692 | | * | * | ISP 2 693 | | * | * | 694 +------+ * +------+ * +------+ 695 |IXP A2|----|IXP B2|----|IXP C2| 696 +------+ * +------+ * +------+ 697 | * | * | 698 | * | * | 699 +-------+ * +-------+ * +-------+ 700 |Core A |----|Core B |---|Core C | 701 +-------+ * +-------+ * +-------+ 703 5. Security Considerations 705 The interactions with a BGP-based controller are similar to those 706 with any other SDN controller. The security implications of SDN 707 controller have not been fully discussed or described. Therefore, 708 protocol and applicability for solutions around this architecture 709 must take proper account of these concerns. 711 6. IANA Considerations 713 This document does not require any IANA actions. 715 7. Acknowledgements 717 The authors would like to thank Chris Bowers, Jeff Tantsura for their 718 valuable suggestions and comments on this draft. 720 8. Contributors 722 Nan Wu 723 Huawei 724 Email: eric.wu@huawei.com 726 9. References 727 9.1. Normative References 729 [RFC1771] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP- 730 4)", RFC 1771, DOI 10.17487/RFC1771, March 1995, 731 . 733 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 734 Requirement Levels", BCP 14, RFC 2119, 735 DOI 10.17487/RFC2119, March 1997, 736 . 738 [RFC3107] Rekhter, Y. and E. Rosen, "Carrying Label Information in 739 BGP-4", RFC 3107, DOI 10.17487/RFC3107, May 2001, 740 . 742 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 743 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 744 DOI 10.17487/RFC4271, January 2006, 745 . 747 [RFC4456] Bates, T., Chen, E., and R. Chandra, "BGP Route 748 Reflection: An Alternative to Full Mesh Internal BGP 749 (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006, 750 . 752 [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., 753 and D. McPherson, "Dissemination of Flow Specification 754 Rules", RFC 5575, DOI 10.17487/RFC5575, August 2009, 755 . 757 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 758 S. Ray, "North-Bound Distribution of Link-State and 759 Traffic Engineering (TE) Information Using BGP", RFC 7752, 760 DOI 10.17487/RFC7752, March 2016, 761 . 763 9.2. Informative References 765 [I-D.ietf-idr-bgpls-segment-routing-epe] 766 Previdi, S., Talaulikar, K., Filsfils, C., Patel, K., Ray, 767 S., and J. Dong, "BGP-LS extensions for Segment Routing 768 BGP Egress Peer Engineering", draft-ietf-idr-bgpls- 769 segment-routing-epe-19 (work in progress), May 2019. 771 [I-D.ietf-idr-flowspec-path-redirect] 772 Velde, G., Patel, K., and Z. Li, "Flowspec Indirection-id 773 Redirect", draft-ietf-idr-flowspec-path-redirect-11 (work 774 in progress), May 2020. 776 [I-D.ietf-idr-segment-routing-te-policy] 777 Previdi, S., Filsfils, C., Talaulikar, K., Mattes, P., 778 Rosen, E., Jain, D., and S. Lin, "Advertising Segment 779 Routing Policies in BGP", draft-ietf-idr-segment-routing- 780 te-policy-11 (work in progress), November 2020. 782 [I-D.ietf-isis-segment-routing-extensions] 783 Previdi, S., Ginsberg, L., Filsfils, C., Bashandy, A., 784 Gredler, H., and B. Decraene, "IS-IS Extensions for 785 Segment Routing", draft-ietf-isis-segment-routing- 786 extensions-25 (work in progress), May 2019. 788 [I-D.ietf-pce-segment-routing] 789 Sivabalan, S., Filsfils, C., Tantsura, J., Henderickx, W., 790 and J. Hardwick, "PCEP Extensions for Segment Routing", 791 draft-ietf-pce-segment-routing-16 (work in progress), 792 March 2019. 794 [I-D.ietf-rtgwg-bgp-routing-large-dc] 795 Lapukhov, P., Premji, A., and J. Mitchell, "Use of BGP for 796 routing in large-scale data centers", draft-ietf-rtgwg- 797 bgp-routing-large-dc-11 (work in progress), June 2016. 799 [I-D.ietf-spring-segment-routing] 800 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., 801 Litkowski, S., and R. Shakir, "Segment Routing 802 Architecture", draft-ietf-spring-segment-routing-15 (work 803 in progress), January 2018. 805 Authors' Addresses 807 Yujia 808 China Telcom Co., Ltd. 809 109 West Zhongshan Ave,Tianhe District 810 Guangzhou 510630 811 China 813 Email: luoyuj@sdu.edu.cn 815 Liang 816 China Telcom Co., Ltd. 817 109 West Zhongshan Ave,Tianhe District 818 Guangzhou 510630 819 China 821 Email: ouliang@chinatelecom.cn 822 Xiang 823 Tencent 825 Email: terranhuang@tencent.com 827 Gyan S. Mishra 828 Verizon Inc. 829 13101 Columbia Pike 830 Silver Spring MD 20904 831 USA 833 Phone: 301 502-1347 834 Email: gyan.s.mishra@verizon.com 836 Huaimo Chen 837 Futurewei 838 Boston, MA 839 USA 841 Email: Huaimo.chen@futurewei.com 843 Shunwan Zhuang 844 Huawei 845 Huawei Bld., No.156 Beiqing Rd. 846 Beijing 100095 847 China 849 Email: zhuangshunwan@huawei.com 851 Zhenbin Li 852 Huawei 853 Huawei Bld., No.156 Beiqing Rd. 854 Beijing 100095 855 China 857 Email: lizhenbin@huawei.com