idnits 2.17.1 draft-dunbar-lsr-5g-edge-compute-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (January 7, 2022) is 841 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'LSR-FlexAlgo' is mentioned on line 120, but not defined == Missing Reference: 'A-ER' is mentioned on line 169, but not defined == Missing Reference: 'RFC8174' is mentioned on line 228, but not defined == Missing Reference: 'RFC8362' is mentioned on line 280, but not defined == Missing Reference: 'RFC8920' is mentioned on line 330, but not defined ** Obsolete undefined reference: RFC 8920 (Obsoleted by RFC 9492) == Missing Reference: '5GC' is mentioned on line 430, but not defined == Missing Reference: 'RFC5305' is mentioned on line 570, but not defined == Unused Reference: 'RFC2328' is defined on line 365, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 367, but no explicit reference was found in the text == Unused Reference: 'RFC8200' is defined on line 375, but no explicit reference was found in the text == Unused Reference: 'RFC8326' is defined on line 378, but no explicit reference was found in the text == Unused Reference: 'RFC9012' is defined on line 382, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 387, but no explicit reference was found in the text == Unused Reference: '5G-StickyService' is defined on line 394, but no explicit reference was found in the text == Unused Reference: 'LSR-Flex-Algo' is defined on line 406, but no explicit reference was found in the text == Unused Reference: 'LSR-Flex-Algo-BW' is defined on line 409, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 413, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 5521 ** Downref: Normative reference to an Proposed Standard RFC: RFC 7684 ** Downref: Normative reference to an Proposed Standard RFC: RFC 8362 (ref. 'RFC8326') ** Downref: Normative reference to an Proposed Standard RFC: RFC 9012 -- No information found for draft-dunbar-6man-5g-ec-sticky-service - is the name correct? == Outdated reference: A later version (-14) exists of draft-dunbar-idr-5g-edge-compute-app-meta-data-03 == Outdated reference: A later version (-26) exists of draft-ietf-lsr-flex-algo-17 == Outdated reference: A later version (-10) exists of draft-ietf-lsr-flex-algo-bw-con-01 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 Summary: 5 errors (**), 0 flaws (~~), 23 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft H. Chen 3 Intended status: Standard Futurewei 4 Expires: June 16, 2022 Aijun Wang 5 China Telecom 6 January 7, 2022 8 IGP Extension for 5G Edge Computing Service 9 draft-dunbar-lsr-5g-edge-compute-03 11 Abstract 12 Routers in 5G Local Data Network (LDN) can use additional 13 site-costs, preference, and other application related 14 metrics in addition to the network routing distance to 15 compute constraint-based SPF within the 5G LDN to enhance 16 performance for selected services. This draft describes 17 using application server related metrics to influence the 18 SPF and Flexible Algorithms to indicate the constraints. 20 Status of this Memo 21 This Internet-Draft is submitted in full conformance with 22 the provisions of BCP 78 and BCP 79. 24 This Internet-Draft is submitted in full conformance with 25 the provisions of BCP 78 and BCP 79. This document may not 26 be modified, and derivative works of it may not be created, 27 except to publish it as an RFC and to translate it into 28 languages other than English. 30 Internet-Drafts are working documents of the Internet 31 Engineering Task Force (IETF), its areas, and its working 32 groups. Note that other groups may also distribute working 33 documents as Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of 36 six months and may be updated, replaced, or obsoleted by 37 other documents at any time. It is inappropriate to use 38 Internet-Drafts as reference material or to cite them other 39 than as "work in progress." 41 The list of current Internet-Drafts can be accessed at 42 http://www.ietf.org/ietf/1id-abstracts.txt 44 Internet-Draft LSR Extension for 5G EC Service 46 The list of Internet-Draft Shadow Directories can be 47 accessed at http://www.ietf.org/shadow.html 49 This Internet-Draft will expire on April 7, 2021. 51 Copyright Notice 53 Copyright (c) 2021 IETF Trust and the persons identified as 54 the document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's 57 Legal Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the 59 date of publication of this document. Please review these 60 documents carefully, as they describe your rights and 61 restrictions with respect to this document. Code Components 62 extracted from this document must include Simplified BSD 63 License text as described in Section 4.e of the Trust Legal 64 Provisions and are provided without warranty as described 65 in the Simplified BSD License. 67 Table of Contents 69 1. Introduction........................................... 3 70 1.1. Unbalanced Distribution due to UE Mobility........ 3 71 1.2. ANYCAST in 5G EC Environment...................... 4 73 2. Conventions used in this document...................... 4 75 3. Solution Overview...................................... 6 77 4. New Flags added to FAD Flags Sub-TLV................... 6 79 5. "Site-Cost" Advertisement in OSPF...................... 7 81 6. "Site-Cost" Advertisement in IS-IS..................... 7 83 7. Alternative method for Distributing Aggregated Cost.... 7 85 8. Manageability Considerations........................... 8 87 Internet-Draft LSR Extension for 5G EC Service 89 9. Security Considerations................................ 8 91 10. IANA Considerations................................... 8 93 11. References............................................ 8 94 11.1. Normative References............................. 9 95 11.2. Informative References........................... 9 97 12. Appendix:5G Edge Computing Background................ 11 99 13. 5G EC LDN Characteristics for the Constraint SPF..... 12 100 13.1. IP Layer Metrics to Gauge EC Server Running Status 101 ...................................................... 12 102 13.2. App Metrics Constrained Shortest Path First..... 14 103 13.3. Reason for using IGP Based Solution............. 15 104 13.4. Flow Affinity to an ANYCAST server.............. 15 106 14. Acknowledgments...................................... 16 108 1. Introduction 110 In 5G Edge Computing (EC) environment, it is common for an 111 application that needs low latency to be instantiated on 112 multiple servers close in proximity to UEs (User 113 Equipment). When those multiple server instances share one 114 IP address (ANYCAST), the transient network and load 115 conditions can be incorporated in selecting an optimal path 116 among server instances for UEs. 118 Flexible algorithms provide mechanisms for topologies to 119 use different IGP path algorithms. This draft describes 120 using Flexible Algorithms [LSR-FlexAlgo] to indicate the 121 desired constrained SPF behavior for a subset of prefixes, 122 in addition to the encodings for advertising the IP Layer 123 App related metrics that can impact application servers' 124 performance. 126 1.1. Unbalanced Distribution due to UE Mobility 128 UEs' frequent moving from one 5G site to another can make 129 it difficult to plan where the App Servers should be 130 hosted. When one App server is heavily utilized, other App 131 servers of the same address close by can be under-utilized. 133 Internet-Draft LSR Extension for 5G EC Service 135 The difference in the routing distance to reach multiple 136 Application Servers might be relatively small. The traffic 137 load at the router where the App Server is attached and the 138 site capacity, when combined, can be more significant than 139 the routing distance from the latency and performance 140 perspective. 142 Since the condition can be short-lived, it is difficult for 143 the application controller to anticipate the moving and 144 adjusting. 146 1.2. ANYCAST in 5G EC Environment 148 ANYCAST is assigning the same IP address for multiple 149 servers in different locations. Using ANYCAST can eliminate 150 the single point of failure and bottleneck at load 151 balancers or DNS. Another benefit is removing the 152 dependency on how UEs resolve IP addresses for their 153 applications. Some UEs (or clients) might use stale cached 154 IP addresses for an extended period. 156 But having the same IP address in multiple locations of the 157 5G Edge Computing environment can be problematic because 158 all those locations can be close in proximity. There might 159 be a very small difference in the routing distance to reach 160 an Application Server attached to a different edge router. 162 Note: for the ease of description, the EC (Edge Computing) 163 server, Application server, App server are used 164 interchangeably throughout this document. 166 2. Conventions used in this document 168 A-ER: Egress Edge Router to an Application Server, 169 [A-ER] is used to describe the last router that 170 the Application Server is attached. For 5G EC 171 environment, the A-ER can be the gateway router 172 to a (mini) Edge Computing Data Center. 174 Application Server: An application server is a physical or 175 virtual server that hosts the software system 176 for the application. 178 Internet-Draft LSR Extension for 5G EC Service 180 Application Server Location: Represent a cluster of servers 181 at one location serving the same Application. 182 One application may have a Layer 7 Load 183 balancer, whose address(es) are reachable from 184 an external IP network, in front of a set of 185 application servers. From IP network 186 perspective, this whole group of servers is 187 considered as the Application server at the 188 location. 190 Edge Application Server: used interchangeably with 191 Application Server throughout this document. 193 EC: Edge Computing 195 Edge Hosting Environment: An environment providing the 196 support required for Edge Application Server's 197 execution. 199 NOTE: The above terminologies are the same as 200 those used in 3GPP TR 23.758 202 Edge DC: Edge Data Center, which provides the Edge 203 Computing Hosting Environment. It might be co- 204 located with 5G Base Station and not only host 205 5G core functions, but also host frequently 206 used Edge server instances. 208 gNB next generation Node B 210 LDN: Local Data Network 212 PSA: PDU Session Anchor (UPF) 214 SSC: Session and Service Continuity 216 UE: User Equipment 218 UPF: User Plane Function 220 Internet-Draft LSR Extension for 5G EC Service 222 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 223 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT 224 RECOMMENDED", "MAY", and "OPTIONAL" in this document are to 225 be interpreted as described in BCP 14 [RFC2119] [RFC8174] 226 when, and only when, they appear in all capitals, as shown 227 here. 229 3. Solution Overview 231 The proposed solution is for the egress edge router (A-ER) 232 with the EC Servers directly attached to 234 - advertise the "Site-Cost" via IP prefix reachability 235 TLV associated with the (anycast) prefix. 237 - use a Flag in the Flexible Algorithm TLV to indicate 238 that "site-cost" needs to be included for the 239 constrained SPF to reach the Prefix 241 The "Site-Cost" associated with an EC server (i.e., ANYCAST 242 prefix) is computed based on the IP layer App-related 243 metrics [Section 12.1], such as Load Measurement, the 244 Capacity Index, the Preference Index, and other constraints 245 by a consistent algorithm across all A-ERs. 247 The solution assumes that the 5G EC controller or 248 management system is aware of the EC ANYCAST addresses that 249 need optimized forwarding. To minimize the processing, only 250 the addresses that match with the ACLs configured by the 5G 251 EC controller will have their Site-Cost collected and 252 advertised. 254 4. New Flags added to FAD Flags Sub-TLV 256 A New flag is added to indicate a constrained SPF compute 257 method is needed for the prefix. 259 Flags: 261 0 1 2 3 4 5 6 7... 262 +-+-+-+-+-+-+-+-+... 263 |M|P| | ... 264 +-+-+-+-+-+-+-+-+... 266 Internet-Draft LSR Extension for 5G EC Service 268 P-flag: Site-Cost Metrics is included in deriving 269 Constrained IGP path to the prefix. 271 5. "Site-Cost" Advertisement in OSPF 273 - IPv4: OSPFv2 274 A new Aggregated Cost Sub-TLV needs to be added to 275 OSPFv2 Extended Prefix TLV [RFC7684] 277 - IPv6: OSPFv3 278 A new sub-TLV can be appended to the E-Intra-Area- 279 Prefix-LSA, E-Inter-Area-Prefix-LSA, E-AS-External- 280 LSA, and E-Type-7-LSA [RFC8362] to carry the 281 Aggregated Cost. 283 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 284 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 285 |AggCostSubTLV | Length | 286 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 287 | AggCost to the App Server | 288 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 289 Figure 1: Aggregated cost Advertisement in IS-IS 291 6. "Site-Cost" Advertisement in IS-IS 293 Aggregated Cost appended to the IP Reachability TLV: 128, 294 130, or 135. 296 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 297 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 298 |AggCostSubTLV | Length | 299 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 300 | AggCost to the App Server | 301 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 302 | PrefixLength | PrefixOptions | 0 | 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 | Address Prefix | 305 | ... | 306 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 307 Figure 2: Aggregated cost Advertisement in IS-IS 309 7. Alternative method for Distributing Aggregated Cost 311 Section 6 and Section 7 demonstrate different ways for 312 OSPFv2, OSPFv3, and ISIS to propagate the aggregated cost. 314 Internet-Draft LSR Extension for 5G EC Service 316 It would be better if the aggregated cost could be 317 advertised the same way, regardless of OSPFv2, OSPFv3, or 318 ISIS. 320 Draft [draft-wang-lsr-stub-link-attributes] introduces the 321 Stub-Link TLV for OSPFv2/v3 and ISIS protocol respectively. 322 Considering the interfaces on an edge router that connects 323 to the EC servers are normally configured as passive 324 interfaces, these IP-layer App-metrics can also be 325 advertised as the attributes of the passive/stub link. The 326 associated prefixes can then be advertised in the "Stub- 327 Link TLV" that is defined in [draft-wang-lsr-stub-link- 328 attributes]. All the associated prefixes share the same 329 characteristic of the link. Other link related sub-TLVs 330 defined in [RFC8920] can also be attached and applied to 331 the calculation of path to the associated prefixes." 333 Section 6 for the advertisement of AppMetaData Metric can 334 also utilize the Stub-Link TLV that defined in [draft-wang- 335 lsr-stub-link-attributes] 337 8. Manageability Considerations 339 To be added. 341 9. Security Considerations 343 To be added. 345 10. IANA Considerations 347 The following Sub-TLV types need to be added by IANA to 348 FlexAlgo. 350 - AggCostSubTLV Type for ISIS, OSPF (TBD1): IPv4 or IPv6 352 P-flag added to FAD Flags Sub-TLV to indicate that the 353 Site-Cost Metrics is included in deriving Constrained IGP 354 path to the prefix. 356 11. References 357 Internet-Draft LSR Extension for 5G EC Service 359 11.1. Normative References 361 [RFC2119] Bradner, S., "Key words for use in RFCs to 362 Indicate Requirement Levels", BCP 14, RFC 2119, 363 March 1997. 365 [RFC2328] J. Moy, "OSPF Version 2", RFC 2328, April 1998. 367 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 368 Subsequent Address Family Identifier (SAFI) and 369 the BGP Tunnel Encapsulation Attribute", April 370 2009. 372 [RFC7684] P. Psenak, et al, "OSPFv2 Prefix/Link Attribute 373 Advertisement", RFC 7684, Nov. 2015. 375 [RFC8200] S. Deering R. Hinden, "Internet Protocol, Version 376 6 (IPv6) Specification", July 2017. 378 [RFC8326] A. Lindem, et al, "OSPFv3 Link State 379 advertisement (LSA0 Extensibility", RFC 8362, 380 April 2018. 382 [RFC9012] E. Rosen, et al "The BGP Tunnel Encapsulation 383 Attribute", April 2021. 385 11.2. Informative References 387 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 388 Partnership Project; Technical Specification 389 Group Services and System Aspects; Study on 390 enhancement of support for Edge Computing in 5G 391 Core network (5GC)", Release 17 work in progress, 392 Aug 2020. 394 [5G-StickyService] L. Dunbar, J. Kaippallimalil, "IPv6 395 Solution for 5G Edge Computing Sticky Service", 396 draft-dunbar-6man-5g-ec-sticky-service-00, work- 397 in-progress, Oct 2020. 399 Internet-Draft LSR Extension for 5G EC Service 401 [BGP-5G-AppMetaData] L. Dunbar, K. Majumdar, H. Wang, "BGP 402 App Metadata for 5G Edge Computing Service", 403 draft-dunbar-idr-5g-edge-compute-app-meta-data- 404 03, work-in-progress, Sept 2020. 406 [LSR-Flex-Algo] P. Psenak, et al, "IGP Flexible Algorithm", 407 draft-ietf-lsr-flex-algo-17, July 2021. 409 [LSR-Flex-Algo-BW] S. Hegde, et al, "Flexible Algorithms: 410 Bandwidth, Delay, Metrics and Constraints", 411 draft-ietf-lsr-flex-algo-bw-con-01, July 2021. 413 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 414 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 415 draft-dunbar-idr-sdwan-edge-discovery-00, work- 416 in-progress, July 2020. 418 Internet-Draft LSR Extension for 5G EC Service 420 12. Appendix:5G Edge Computing Background 422 The network connecting the 5G EC servers with the 5G Base 423 stations consists of a small number of dedicated routers 424 that form the 5G Local Data Network (LDN) to enhance the 425 performance of the EC services. 427 When a User Equipment (UE) initiates application packets 428 using the destination address from a DNS reply or its 429 cache, the packets from the UE are carried in a PDU session 430 through 5G Core [5GC] to the 5G UPF-PSA (User Plan Function 431 - PDU Session Anchor). The UPF-PSA decapsulates the 5G GTP 432 outer header, performs NAT sometimes, before handing the 433 packets from the UEs to the adjacent router, also known as 434 the ingress router to the EC LDN, which is responsible for 435 forwarding the packets to the intended destinations. 437 When the UE moves out of coverage of its current gNB (next- 438 generation Node B) (gNB1), the handover procedure is 439 initiated, which includes the 5G SMF (Session Management 440 Function) selecting a new UPF-PSA [3GPP TS 23.501 and TS 441 23.502]. When the handover process is complete, the IP 442 point of attachment is to the new UPF-PSA. The UE's IP 443 address stays the same unless moving to different operator 444 domain. 5GC may maintain a path from the old UPF to the new 445 UPF for a short time for SSC [Session and Service 446 Continuity] mode 3 to make the handover process more 447 seamless. 449 Internet-Draft LSR Extension for 5G EC Service 451 +--+ 452 |UE|---\+---------+ +------------------+ 453 +--+ | 5G | +---------+ | S1: aa08::4450 | 454 +--+ | Site +--++---+ +----+ | 455 |UE|----| A |PSA| Ra| | R1 | S2: aa08::4460 | 456 +--+ | +---+---+ +----+ | 457 +---+ | | | | | S3: aa08::4470 | 458 |UE1|---/+---------+ | | +------------------+ 459 +---+ |IP Network | L-DN1 460 |(3GPP N6) | 461 | | | +------------------+ 462 | UE1 | | | S1: aa08::4450 | 463 | moves to | +----+ | 464 | Site B | | R3 | S2: aa08::4460 | 465 v | +----+ | 466 | | | S3: aa08::4470 | 467 | | +------------------+ 468 | | L-DN3 469 +--+ | | 470 |UE|---\+---------+ | | +------------------+ 471 +--+ | 5G | | | | S1: aa08::4450 | 472 +--+ | Site +--++-+--+ +----+ | 473 |UE|----| B |PSA| Rb | | R2 | S2: aa08::4460 | 474 +--+ | +--++----+ +----+ | 475 +--+ | | +-----------+ | S3: aa08::4470 | 476 |UE|---/+---------+ +------------------+ 477 +--+ L-DN2 478 Figure 10: App Servers in different edge DCs 480 13. 5G EC LDN Characteristics for the Constraint SPF 482 13.1. IP Layer Metrics to Gauge EC Server Running Status 484 Most applications do not expose their internal logic to the 485 network. Their communications are generally encrypted. Most 486 of them do not even respond to PING or ICMP messages 487 initiated by routers. 489 Internet-Draft LSR Extension for 5G EC Service 491 Here are some IP Layer App related Metrics that can gauge 492 the servers running status and environment: 494 - Capacity Index: 495 a numeric number, configured on all A-ERs in the 496 domain consistently, is used to represent the capacity 497 of an EC server attached to an A-ER. The IP addresses 498 exposed to the A-ER can be the App Layer Load 499 balancers that have many instances attached. At other 500 sites, the IP address exposed is the server itself. 501 - Site preference index: 502 Is used to describe some sites are more preferred than 503 others. For example, a site with less leasing cost has 504 a higher preference value. Note: the preference value 505 is configured on all A-ERs in the domain consistently 506 by the Domain Controller. 508 - Load Measurement for gauging the load of the attached 509 prefix (i.e., EC Server): 510 The Load Measurement for an EC Server is a weighted 511 combination of the number of packets/bytes to the EC 512 server (i.e., its IP address) and the number of 513 packets/bytes from the EC server. The Load Measurement 514 are collected by the A-ER that has the EC Server 515 directly attached. 517 An A-ER only collects those measurement for the 518 prefixes instructed by the Domain Controller. 520 For ease of description, those metrics with more to be 521 added later are called IP Layer App Metrics (or Site-Cost) 522 throughout the document. 524 Internet-Draft LSR Extension for 5G EC Service 526 13.2. App Metrics Constrained Shortest Path First 528 The main benefit of using ANYCAST is to leverage the 529 network layer information to balance the traffic among 530 multiple locations of one application server. 532 For the 5G EC environment, the routers in the LDN need to 533 take consideration of various measurements of the EC 534 servers attached to each A-ER in addition to TE metrics to 535 compute ECMP paths to the servers. 537 Here is one algorithm that computes the cost to reach the 538 App Servers attached to Site-i relative to another site, 539 say Site-j. When the reference site, Site-j, is plugged in 540 the formula, the cost is 1. So, if the formula returns a 541 value less than 1, the cost to reach Site-i is less than 542 reaching the reference site (Site-j). 544 CP-j * Load-i Pref-j * Network-Delay-i 545 Cost-i= (w *(----------------) + (1-w) *(-------------------------)) 546 CP-i * Load-j Pref-i * Network-Delay-j 548 Load-i: Load Index at Site-i, it is the weighted 549 combination of the total packets or/and bytes sent to 550 and received from the Application Server at Site-i 551 during a fixed time period. 553 CP-i: capacity index at site i, a higher value means 554 higher capacity. 556 Network Delay-i: Network latency measurement (RTT) to 557 the A-ER that has the Application Server attached at the 558 site-i. 559 Noted: Ingress nodes can easily measure RTT to all the 560 egress edge nodes by existing IPPM metrics. But it is 561 not so easy for ingress nodes to measure RTT to all the 562 App Servers. Therefore, "Network-Delay-i", a.k.a. 563 Network latency measurement (RTT), is between the 564 Ingress and egress edge nodes. The cost for the egress 565 edge nodes to reach to their attached servers is 566 embedded in the "capacity index". 568 Pref-i: Preference index for site-i, a higher value 569 means higher preference. Preference can be derived from 570 the total path cost to reach the A-ER [RFC5305], as 571 calculated below: 1/(total-path-cost). 573 Internet-Draft LSR Extension for 5G EC Service 575 w: Weight for load and site information, which is a 576 value between 0 and 1. If smaller than 0.5, Network 577 latency and the site Preference have more influence; 578 otherwise, Server load and its capacity have more 579 influence. 581 13.3. Reason for using IGP Based Solution 583 Here are some benefits of using IGP to propagate the IP 584 Layer App-Metrics: 585 - Intermediate routers can utilize the aggregated cost to 586 reach the EC Servers attached to different egress edge 587 nodes, especially: 588 - The path to the optimal egress edge node can be 589 more accurate or shorter. 590 - Convergence is shorter when there is any failure 591 along the way towards the optimal ANYCAST server. 592 - When there is any failure at the intended ANYCAST 593 server, all the packets in transit can be optimally 594 forwarded to another App Server attached to a 595 different egress edge router. 596 - Doesn't need the ingress nodes to establish tunnels with 597 egress edge nodes. 599 There are limitations of using IGP too, such as: 601 - The IGP approach might not suit well to 5G EC LDN 602 operated by multiple ISPs. 603 For LDN operated by multiple IPSs, BGP should be used. 604 [BGP-5G-AppMetaData] describes the BGP UPDATE message to 605 propagate IP Layer App-Metrics crossing multiple ISPs. 607 13.4. Flow Affinity to an ANYCAST server 609 When multiple servers with the same IP address (ANYCAST) 610 are attached to different A-ERs, Flow Affinity means 611 routers sending the packets of the same flow to the same A- 612 ER even if the cost towards the A-ER is no longer optimal. 614 Many commercial routers support some forms of flow affinity 615 to ensure packets belonging to one flow be forwarded along 616 the same path. 618 Editor's note: for IPv6 traffic, Flow Affinity can be 619 achieved by routers forwarding the packets with the same 621 Internet-Draft LSR Extension for 5G EC Service 623 Flow Label extracted from the IPv6 Header along the same 624 path. 626 14. Acknowledgments 628 Acknowledgements to Peter Psenak, Acee Lindem, Shraddha 629 Hegde, Tony Li, Gyan Mishra, Jeff Tantsura, and Donald 630 Eastlake for their review and suggestions. 632 This document was prepared using 2-Word-v2.0.template.dot. 634 Authors' Addresses 636 Linda Dunbar 637 Futurewei 638 Email: ldunbar@futurewei.com 640 Huaimo Chen 641 Futurewei 642 Email: huaimo.chen@futurewei.com 644 Aijun Wang 645 China Telecom 646 Email: wangaj3@chinatelecom.cn