idnits 2.17.1 draft-dunbar-lsr-5g-edge-compute-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 11, 2021) is 921 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: '5GC' is mentioned on line 126, but not defined == Missing Reference: 'A-ER' is mentioned on line 223, but not defined == Missing Reference: 'RFC8174' is mentioned on line 280, but not defined == Missing Reference: 'LSR-FlexAlgo' is mentioned on line 286, but not defined == Missing Reference: 'RFC5305' is mentioned on line 401, but not defined == Missing Reference: 'RFC8920' is mentioned on line 833, but not defined ** Obsolete undefined reference: RFC 8920 (Obsoleted by RFC 9492) == Missing Reference: 'RFC8919' is mentioned on line 733, but not defined ** Obsolete undefined reference: RFC 8919 (Obsoleted by RFC 9479) == Missing Reference: 'Figure 2' is mentioned on line 762, but not defined == Unused Reference: 'RFC5521' is defined on line 877, but no explicit reference was found in the text == Unused Reference: 'RFC7684' is defined on line 882, but no explicit reference was found in the text == Unused Reference: 'RFC8200' is defined on line 885, but no explicit reference was found in the text == Unused Reference: 'RFC8326' is defined on line 888, but no explicit reference was found in the text == Unused Reference: 'RFC9012' is defined on line 892, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 899, but no explicit reference was found in the text == Unused Reference: '5G-StickyService' is defined on line 906, but no explicit reference was found in the text == Unused Reference: 'LSR-Flex-Algo' is defined on line 916, but no explicit reference was found in the text == Unused Reference: 'LSR-Flex-Algo-BW' is defined on line 919, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 923, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 5521 ** Downref: Normative reference to an Proposed Standard RFC: RFC 7684 ** Downref: Normative reference to an Proposed Standard RFC: RFC 8362 (ref. 'RFC8326') ** Downref: Normative reference to an Proposed Standard RFC: RFC 9012 -- No information found for draft-dunbar-6man-5g-ec-sticky-service - is the name correct? == Outdated reference: A later version (-14) exists of draft-dunbar-idr-5g-edge-compute-app-meta-data-03 == Outdated reference: A later version (-26) exists of draft-ietf-lsr-flex-algo-17 == Outdated reference: A later version (-09) exists of draft-ietf-lsr-flex-algo-bw-con-01 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 Summary: 6 errors (**), 0 flaws (~~), 23 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft H. Chen 3 Intended status: Standard Futurewei 4 Expires: April 11, 2022 Aijun Wang 5 China Telecom 6 October 11, 2021 8 Flex Algo Extension for 5G Edge Computing Service 9 draft-dunbar-lsr-5g-edge-compute-01 11 Abstract 12 Routers in 5G Local Data Network (LDN) can use additional 13 site-costs, preference, and other application related 14 metrics on top of the network condition to compute 15 constraint-based SPF within the 5G LDN to enhance 16 performance for selected services. This draft describes 17 those application server related metrics to be used in 18 Flexible Algorithms. 20 Status of this Memo 21 This Internet-Draft is submitted in full conformance with 22 the provisions of BCP 78 and BCP 79. 24 This Internet-Draft is submitted in full conformance with 25 the provisions of BCP 78 and BCP 79. This document may not 26 be modified, and derivative works of it may not be created, 27 except to publish it as an RFC and to translate it into 28 languages other than English. 30 Internet-Drafts are working documents of the Internet 31 Engineering Task Force (IETF), its areas, and its working 32 groups. Note that other groups may also distribute working 33 documents as Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of 36 six months and may be updated, replaced, or obsoleted by 37 other documents at any time. It is inappropriate to use 38 Internet-Drafts as reference material or to cite them other 39 than as "work in progress." 41 The list of current Internet-Drafts can be accessed at 42 http://www.ietf.org/ietf/1id-abstracts.txt 44 Internet-Draft LSR Extension for 5G EC Service 46 The list of Internet-Draft Shadow Directories can be 47 accessed at http://www.ietf.org/shadow.html 49 This Internet-Draft will expire on April 7, 2021. 51 Copyright Notice 53 Copyright (c) 2021 IETF Trust and the persons identified as 54 the document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's 57 Legal Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the 59 date of publication of this document. Please review these 60 documents carefully, as they describe your rights and 61 restrictions with respect to this document. Code Components 62 extracted from this document must include Simplified BSD 63 License text as described in Section 4.e of the Trust Legal 64 Provisions and are provided without warranty as described 65 in the Simplified BSD License. 67 Table of Contents 69 1. Introduction........................................... 3 70 1.1. 5G Edge Computing Background...................... 3 71 1.2. Problem#1: ANYCAST in 5G EC Environment........... 5 72 1.3. Problem #2: Unbalanced Anycast Distribution due to 73 UE Mobility............................................ 5 74 2. Conventions used in this document...................... 5 75 3. Solution Overview...................................... 7 76 3.1. Flow Affinity to an ANYCAST server................ 7 77 3.2. IP Layer Metrics to Gauge EC Server Running Status 8 78 3.3. App Metrics Constrained Shortest Path First....... 9 79 3.4. Reason for using IGP Based Solution.............. 10 80 4. IS-IS FAD with AppMetaData Constraint sub-TLVs........ 10 81 4.1. ISIS FAD sub-TLV for the Aggregated cost......... 10 82 4.2. IS-IS FAD for AppMetaData Metrics Advertisements. 11 83 4.3. ISIS AppMetaData Sub-TLV......................... 13 84 4.4. OSPF AppMetaData Sub-TLV......................... 14 85 5. AppMetaData SubSub-TLVs............................... 15 86 6. AppMetaData Metric Advertisement...................... 17 87 7. Aggregated Cost Advertisement in ISIS................. 18 88 8. Aggregated Cost Advertisement in OSPF................. 19 90 Internet-Draft LSR Extension for 5G EC Service 92 8.1. OSPFv3 LSA to carry the Aggregated Cost.......... 19 93 8.2. OSPFv2 LSA to carry the Aggregated Cost.......... 19 94 9. Alternative method for Distributing Aggregated Cost... 20 95 10. Manageability Considerations......................... 20 96 11. Security Considerations.............................. 20 97 12. IANA Considerations.................................. 20 98 13. References........................................... 21 99 13.1. Normative References............................ 21 100 13.2. Informative References.......................... 22 101 14. Acknowledgments...................................... 22 103 1. Introduction 105 In 5G Edge Computing (EC) environment, it is common for one 106 application to be instantiated on multiple servers close in 107 proximity. Those multiple server instances can share one IP 108 address (ANYCAST) so that the transient network and load 109 conditions can be considered when computing the IGP path. 111 Flexible algorithms provide mechanisms to create 112 constraint-based paths in IGP. This draft describes some 113 specific metrics, that can impact application servers' 114 performance, to be used in the Flexible Algorithms. 116 1.1. 5G Edge Computing Background 118 The network connecting the 5G EC servers with the 5G Base 119 stations consists of a small number of dedicated routers 120 that form the 5G Local Data Network (LDN) to enhance the 121 performance of the EC services. 123 When a User Equipment (UE) initiates application packets 124 using the destination address from a DNS reply or its 125 cache, the packets from the UE are carried in a PDU session 126 through 5G Core [5GC] to the 5G UPF-PSA (User Plan Function 127 - PDU Session Anchor). The UPF-PSA decapsulates the 5G GTP 128 outer header, performs NAT sometimes, before handing the 129 packets from the UEs to the adjacent router, also known as 130 the ingress router to the EC LDN, which is responsible for 131 forwarding the packets to the intended destinations. 133 Internet-Draft LSR Extension for 5G EC Service 135 When the UE moves out of coverage of its current gNB (next- 136 generation Node B) (gNB1), the handover procedure is 137 initiated, which includes the 5G SMF (Session Management 138 Function) selecting a new UPF-PSA [3GPP TS 23.501 and TS 139 23.502]. When the handover process is complete, the IP 140 point of attachment is to the new UPF-PSA. The UE's IP 141 address stays the same unless moving to different operator 142 domain. 5GC may maintain a path from the old UPF to the new 143 UPF for a short time for SSC [Session and Service 144 Continuity] mode 3 to make the handover process more 145 seamless. 146 +--+ 147 |UE|---\+---------+ +------------------+ 148 +--+ | 5G | +---------+ | S1: aa08::4450 | 149 +--+ | Site +--++---+ +----+ | 150 |UE|----| A |PSA| Ra| | R1 | S2: aa08::4460 | 151 +--+ | +---+---+ +----+ | 152 +---+ | | | | | S3: aa08::4470 | 153 |UE1|---/+---------+ | | +------------------+ 154 +---+ |IP Network | L-DN1 155 |(3GPP N6) | 156 | | | +------------------+ 157 | UE1 | | | S1: aa08::4450 | 158 | moves to | +----+ | 159 | Site B | | R3 | S2: aa08::4460 | 160 v | +----+ | 161 | | | S3: aa08::4470 | 162 | | +------------------+ 163 | | L-DN3 164 +--+ | | 165 |UE|---\+---------+ | | +------------------+ 166 +--+ | 5G | | | | S1: aa08::4450 | 167 +--+ | Site +--++-+--+ +----+ | 168 |UE|----| B |PSA| Rb | | R2 | S2: aa08::4460 | 169 +--+ | +--++----+ +----+ | 170 +--+ | | +-----------+ | S3: aa08::4470 | 171 |UE|---/+---------+ +------------------+ 172 +--+ L-DN2 173 Figure 1: App Servers in different edge DCs 175 Internet-Draft LSR Extension for 5G EC Service 177 1.2. Problem#1: ANYCAST in 5G EC Environment 179 ANYCAST makes it possible to load balance across server 180 locations based on network conditions dynamically. With 181 multiple servers having the same IP address, it eliminates 182 the single point of failure and bottleneck at the 183 application layer load balancers. Another benefit of using 184 ANYCAST address is removing the dependency on how UEs get 185 the IP addresses for their applications. Some UEs (or 186 clients) might use stale cached IP addresses for an 187 extended period. 189 But, having multiple locations of the same IP address in 190 the 5G Edge Computing environment can be problematic 191 because all those edge computing Data Centers can be close 192 in proximity. There might be very little difference in the 193 routing distance to reach the Application Servers attached 194 to a different edge router, which can cause packets from 195 one flow to be forwarded to different locations, resulting 196 in service glitches. 198 1.3. Problem #2: Unbalanced Anycast Distribution due to UE 199 Mobility 201 UEs' frequent moving from one 5G site to another can make 202 it difficult to plan where the App Servers should be 203 hosted. When one App server is heavily utilized, other App 204 servers of the same address close by can be under-utilized. 205 The difference in the routing distance to reach multiple 206 Application Servers might be relatively small. The traffic 207 load at the router where the App Server is attached and the 208 site capacity, when combined, can be more significant from 209 the latency and performance perspective. 211 Since the condition can be short-lived, it is difficult for 212 the application controller to anticipate the moving and 213 adjusting. 215 Note: for the ease of description, the EC (Edge Computing) 216 server, Application server, App server are used 217 interchangeably throughout this document. 219 2. Conventions used in this document 220 Internet-Draft LSR Extension for 5G EC Service 222 A-ER: Egress Edge Router to an Application Server, 223 [A-ER] is used to describe the last router that 224 the Application Server is attached. For 5G EC 225 environment, the A-ER can be the gateway router 226 to a (mini) Edge Computing Data Center. 228 Application Server: An application server is a physical or 229 virtual server that hosts the software system 230 for the application. 232 Application Server Location: Represent a cluster of servers 233 at one location serving the same Application. 234 One application may have a Layer 7 Load 235 balancer, whose address(es) are reachable from 236 an external IP network, in front of a set of 237 application servers. From IP network 238 perspective, this whole group of servers is 239 considered as the Application server at the 240 location. 242 Edge Application Server: used interchangeably with 243 Application Server throughout this document. 245 EC: Edge Computing 247 Edge Hosting Environment: An environment providing the 248 support required for Edge Application Server's 249 execution. 251 NOTE: The above terminologies are the same as 252 those used in 3GPP TR 23.758 254 Edge DC: Edge Data Center, which provides the Edge 255 Computing Hosting Environment. It might be co- 256 located with 5G Base Station and not only host 257 5G core functions, but also host frequently 258 used Edge server instances. 260 gNB next generation Node B 262 LDN: Local Data Network 264 Internet-Draft LSR Extension for 5G EC Service 266 PSA: PDU Session Anchor (UPF) 268 SSC: Session and Service Continuity 270 UE: User Equipment 272 UPF: User Plane Function 274 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 275 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT 276 RECOMMENDED", "MAY", and "OPTIONAL" in this document are to 277 be interpreted as described in BCP 14 [RFC2119] [RFC8174] 278 when, and only when, they appear in all capitals, as shown 279 here. 281 3. Solution Overview 283 The proposed solution is for the egress edge router (A-ER) 284 with the EC Servers directly attached to advertise the 285 "Site-Cost" [Section 3.2] within the 5G EC LDN via Flexible 286 algorithms [LSR-FlexAlgo], so that constrained IGP path can 287 be computed. 289 The solution assumes that the 5G EC controller or 290 management system is aware of the EC ANYCAST addresses that 291 need optimized forwarding. To minimize the processing, only 292 the addresses that match with the ACLs configured by the 5G 293 EC controller will have their Site-Cost collected and 294 advertised. 296 3.1. Flow Affinity to an ANYCAST server 298 When multiple servers with the same IP address (ANYCAST) 299 are attached to different A-ERs, Flow Affinity means 300 routers sending the packets of the same flow to the same A- 301 ER even if the cost towards the A-ER is no longer optimal. 303 Many commercial routers support some forms of flow affinity 304 to ensure packets belonging to one flow be forwarded along 305 the same path. 307 Editor's note: for IPv6 traffic, Flow Affinity can be 308 achieved by routers forwarding the packets with the same 310 Internet-Draft LSR Extension for 5G EC Service 312 Flow Label extracted from the IPv6 Header along the same 313 path. 315 3.2. IP Layer Metrics to Gauge EC Server Running Status 317 Most applications do not expose their internal logic to the 318 network. Their communications are generally encrypted. Most 319 of them do not even respond to PING or ICMP messages 320 initiated by routers. 322 Here are some IP Layer Metrics that can gauge the servers 323 running status and environment: 325 - Capacity Index: 326 a numeric number, configured on all A-ERs in the 327 domain consistently, is used to represent the capacity 328 of an EC server attached to an A-ER. The IP addresses 329 exposed to the A-ER can be the App Layer Load 330 balancers that have many instances attached. At other 331 sites, the IP address exposed is the server itself. 332 - Site preference index: 333 Is used to describe some sites are more preferred than 334 others. For example, a site with less leasing cost has 335 a higher preference value. Note: the preference value 336 is configured on all A-ERs in the domain consistently 337 by the Domain Controller. 339 - Load Measurement for gauging the load of the attached 340 prefix (i.e., EC Server): 341 The Load Measurement for an EC Server is a weighted 342 combination of the number of packets/bytes to the EC 343 server (i.e., its IP address) and the number of 344 packets/bytes from the EC server. The Load Measurement 345 are collected by the A-ER that has the EC Server 346 directly attached. 348 An A-ER only collects those measurement for the 349 prefixes instructed by the Domain Controller. 351 For ease of description, those metrics with more to be 352 added later are called IP Layer Site-Cost throughout the 353 document. 355 Internet-Draft LSR Extension for 5G EC Service 357 3.3. App Metrics Constrained Shortest Path First 359 The main benefit of using ANYCAST is to leverage the 360 network layer information to balance the traffic among 361 multiple locations of one application server. 363 For the 5G EC environment, the routers in the LDN need to 364 take consideration of various measurements of the App 365 Servers attached to each A-ER in addition to TE metrics to 366 compute the shortest path to the A-ER. 368 Here is one algorithm that computes the cost to reach the 369 App Servers attached to Site-i relative to another site, 370 say Site-b. When the reference site, Site-b, is plugged in 371 the formula, the cost is 1. So, if the formula returns a 372 value less than 1, the cost to reach Site-i is less than 373 reaching the reference site (Site-b). 375 CP-b * Load-i Pref-b * Network-Delay-i 376 Cost-i= (w *(----------------) + (1-w) *(-------------------------)) 377 CP-i * Load-b Pref-i * Network-Delay-b 379 Load-i: Load Index at Site-i, it is the weighted 380 combination of the total packets or/and bytes sent to 381 and received from the Application Server at Site-i 382 during a fixed time period. 384 CP-i: capacity index at site i, a higher value means 385 higher capacity. 387 Network Delay-i: Network latency measurement (RTT) to 388 the A-ER that has the Application Server attached at the 389 site-i. 390 Noted: Ingress nodes can easily measure RTT to all the 391 egress edge nodes by existing IPPM metrics. But it is 392 not so easy for ingress nodes to measure RTT to all the 393 App Servers. Therefore, "Network-Delay-i", a.k.a. 394 Network latency measurement (RTT), is between the 395 Ingress and egress edge nodes. The link cost between the 396 egress edge nodes to their attached servers is embedded 397 in the "capacity index". 399 Pref-i: Preference index for site-i, a higher value 400 means higher preference. Preference can be derived from 401 the total path cost to reach the A-ER [RFC5305], as 402 calculated below: 1/(total-path-cost). 404 Internet-Draft LSR Extension for 5G EC Service 406 w: Weight for load and site information, which is a 407 value between 0 and 1. If smaller than 0.5, Network 408 latency and the site Preference have more influence; 409 otherwise, Server load and its capacity have more 410 influence. 412 3.4. Reason for using IGP Based Solution 414 Here are some benefits of using IGP to propagate the IP 415 Layer App-Metrics: 416 - Intermediate routers can derive the aggregated cost to 417 reach the EC Servers attached to different egress edge 418 nodes, especially: 419 - The path to the optimal egress edge node can be 420 more accurate or shorter. 421 - Convergence is shorter when there is any failure 422 along the way towards the optimal ANYCAST server. 423 - When there is any failure at the intended ANYCAST 424 server, all the packets in transit can be optimally 425 forwarded to another App Server attached to a 426 different egress edge router. 427 - Doesn't need the ingress nodes to establish tunnels with 428 egress edge nodes. 430 There are limitations of using IGP too, such as: 432 - The IGP approach might not suit well to 5G EC LDN 433 operated by multiple ISPs. 434 For LDN operated by multiple IPSs, BGP should be used. 435 [BGP-5G-AppMetaData] describes the BGP UPDATE message to 436 propagate IP Layer App-Metrics crossing multiple ISPs. 438 4. IS-IS FAD with AppMetaData Constraint sub-TLVs 440 4.1. ISIS FAD sub-TLV for the Aggregated cost 442 If egress edge routers with EC servers directly attached 443 can compute the aggregated cost, they can append the 444 Aggregated Cost sub-sub-TLV directly to the IS-IS FAD Sub- 445 TLV: 447 0 1 2 3 448 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 | Type | Length |Flex-Algorithm | Metric-Type | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 453 Internet-Draft LSR Extension for 5G EC Service 455 | Calc-Type | Priority | 456 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 457 | Aggregated cost Sub-sub-TLVs | 458 + + 459 | ... | 460 | | 461 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 463 Flex-Algorithm: SPF. 465 Metric-Type: 466 A new value to be assigned by IANA to indicate the 467 Aggregated Cost AppMetaData Metrics included in computing 468 the constrained SPF. 470 Calc-Type: 471 A value chosen by the IGP operator to indicate a 472 constrained SPF algorithm that takes the Aggregated Cost 473 into the SPF computation across the routers in the 5G 474 LDN. 476 The aggregated cost is computed based on the Load 477 Measurement, the Capacity value, the Preference value and 478 other constraints by a consistent algorithm across all A- 479 ERs. 481 0 1 2 3 482 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 483 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 484 |AggCostSubTLV | Length | AggCost to the App Server | 485 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 486 | PrefixLength | PrefixOptions | 0 | 487 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 488 | Address Prefix | 489 | ... | 490 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 491 Figure 3: AppMetaData Aggregated Cost subsub TLV 493 4.2. IS-IS FAD for AppMetaData Metrics Advertisements 495 This section describes the sub-sub TLVs that carry the 496 detailed IP Layer Metrics for other routers in the 5G LDN 497 to compute the constrained SPF. 499 0 1 2 3 500 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 501 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 503 Internet-Draft LSR Extension for 5G EC Service 505 | Type | Length |Flex-Algorithm | Metric-Type | 506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 507 | Calc-Type | Priority | 508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 | AppMetaData Metrics SubSub-TLVs | 510 + + 511 | ... | 512 | | 513 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 515 Flex-Algorithm: SPF. 517 Metric-Type: 518 A new value to be assigned by IANA to indicate the 519 additional AppMetaData Metrics included in computing the 520 constrained SPF. 522 Calc-Type: 523 A value chosen by the IGP operator to indicate a specific 524 constrained SPF algorithm that takes the AppMetaData 525 attributes into the path computation across the routers 526 in the IGP domain. 528 It worth noting that not all hosts (prefix) attached to an 529 A-ER are EC servers that need network optimization. An A- 530 ER only needs to advertise the site-Cost Metrics for the 531 EC server addresses requested by the Controller. 533 Draft [draft-wang-lsr-passive-interface-attribute] 534 introduces the Stub-Link TLV for OSPFv2/v3 and ISIS 535 protocol respectively. Considering the interfaces on an 536 edge router that connects to the EC servers are normally 537 configured as passive interfaces, these IP-layer App- 538 metrics can also be advertised as the attributes of the 539 passive/stub link. The associated prefixes can then be 540 advertised in the "Stub-Link Prefix Sub-TLV" that is 541 defined in [draft-wang-lsr-passive-interface-attribute]. 542 All the associated prefixes share the same characteristic 543 of the link. Other link related sub-TLVs defined in 544 [RFC8920] can also be attached and applied to the 545 calculation of path to the associated prefixes. 547 Internet-Draft LSR Extension for 5G EC Service 549 4.3. ISIS AppMetaData Sub-TLV 551 For EC Servers using IPv6, the AppMetaData Sub-TLV is 552 encoded as below: 554 0 1 2 3 555 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 556 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 557 |AppMetaDataType| Length | 558 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 559 | IPv6 or IPv4 AppServer (ANYCAST) address | 560 ~ ~ 561 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 562 | Load measurement SubSub-TLV | 563 ~ ~ 564 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 565 | Capability SubSub-TLV | 566 ~ ~ 567 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 568 | Preference SubSub-TLV | 569 ~ ~ 570 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 571 Figure 3: AppMetaData sub TLV 573 AppMetaData Type (TBD1): ISIS-IPv4 or ISIS-IPv6. 575 Internet-Draft LSR Extension for 5G EC Service 577 4.4. OSPF AppMetaData Sub-TLV 579 For EC Servers using IPv6, the AppMetaData Sub-TLV is 580 encoded as below: 582 0 1 2 3 583 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 584 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 585 |AppMetaDataType | Length | 586 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 587 | IPv6 or IPv4 AppServer (ANYCAST) address | 588 ~ ~ 589 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 590 | Load measurement SubSub-TLV | 591 ~ ~ 592 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 593 | Capability SubSub-TLV | 594 ~ ~ 595 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 596 | Preference SubSub-TLV | 597 ~ ~ 598 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 599 Figure 4: AppMetaData sub TLV 601 AppMetaData Type (TBD2): OSPF-IPv4 or OSPF-IPv6. 603 Internet-Draft LSR Extension for 5G EC Service 605 5. AppMetaData SubSub-TLVs 607 Two types of Load Measurement SubSub-TLVs are specified: 609 a) The Aggregated Load Index based on a weighted 610 combination of the collected measurements. 611 b) The raw measurements of packets/bytes to/from the App 612 Server address. The raw measurement is useful when the 613 egress edge routers cannot be configured with a 614 consistent algorithm to compute the aggregated load 615 index or the raw measurements are needed by a central 616 analytic system. 618 The Aggregated Load Index Sub-TLV has the following format: 620 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 621 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 622 | Type (TBD3) | Length | 623 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 624 | Measurement Period | 625 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 626 | Aggregated Load Index to reach the App Server | 627 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 628 Figure 5: Aggregated Load Index Sub-TLV 630 Type=TBD2 (to be assigned by IANA) indicates that the 631 sub-TLV carries the Load Measurement Index derived from 632 the Weighted combination of bytes/packets sent 633 to/received from the App server: 635 Index=w1*ToPackets+w2*FromPackes+w3*ToBytes+w4*FromBytes 637 Where wi is a value between 0 and 1; w1+ w2+ w3+ w4 = 1. 639 Internet-Draft LSR Extension for 5G EC Service 641 The Raw Load Measurement sub-TLV has the following format: 643 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 644 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 645 | Type (TBD4) | Length | 646 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 647 | Measurement Period | 648 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 649 | total number of packets to the AppServer | 650 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 651 | total number of packets from the AppServer | 652 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 653 | total number of bytes to the AppServer | 654 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 655 | total number of bytes from the AppServer | 656 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 657 Figure 6: Raw Load Measurement Sub-TLV 659 Type= TBD3 (to be assigned by IANA) indicates that the 660 sub-TLV carries the Raw measurements of packets/bytes 661 to/from the App Server ANYCAST address. 663 Measurement Period: A user-specified period in seconds, 664 default is 3600 seconds. 666 The Capacity Index sub-TLV has the following format: 668 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 670 | Type (TBD5) | Length | 671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 672 | Capacity Index | 673 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 674 Figure 7: Capacity Index Sub-TLV 676 Internet-Draft LSR Extension for 5G EC Service 678 The Preference Index sub-TLV has the following format: 680 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 681 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 682 | Type (TBD6) | Length | 683 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 684 | Preference Index | 685 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 686 Figure 8: Preference Index Sub-TLV 688 Note: "Capacity Index" and "Site preference" can be 689 different for different attached server addresses. For 690 Figure 1, the address S1 can have higher Site Preference 691 when attached to R1 than R2. 693 6. AppMetaData Metric Advertisement 695 With Flex-Algorithm, the network administrator can define a 696 function that compute the SPF with consideration of the 697 AppMetaData metrics advertised by the routers to which the 698 EC servers are directly attached. 700 This document defines a new standard metric type, 701 AppMetaData, for this purpose. The AppMetaData Metric MAY 702 be advertised in the Generic Metric sub-TLV with the metric 703 type set to "AppMetaData Metric". ISIS and OSPF will 704 advertise this new type of metric in their link 705 advertisements. AppMetaData metric is a link attribute and 706 for advertisement and processing of this attribute for 707 Flex-algorithm purposes, MUST follow the section 12 of [I- 708 D.ietf-lsr-flex-algo] 710 Flex-Algorithm uses this metric type by specifying the 711 AppMetaData as the metric type in a FAD TLV. A FAD TLV may 712 also specify an automatic computation of the AppMetaData 713 metric based on a links advertised bandwidth. An explicit 714 advertisement of a link's AppMetaData metric using the 715 Generic Metric sub-TLV overrides this automatic 716 computation. The automatic AppMetaData metric calculation 717 sub-TLVs are advertised in FAD TLV and these parameters are 718 applicable to applications such as Flex-algorithm that make 719 use of the FAD TLV. 721 Internet-Draft LSR Extension for 5G EC Service 723 7. Aggregated Cost Advertisement in ISIS 725 When A-ER can compute the aggregated cost to an attached EC 726 server, the Aggregated Cost sub-sub-TLV can be directly 727 appended to the App Specific Bit Mask. The aggregated cost 728 computing algorithm can be from EC controller that take the 729 Load Measurement, Capacity value, and Preference value into 730 consideration across all A-ERs. 732 The Application-Specific Link Attribute sub-TLV described 733 in [RFC8919] can be used to carry the "Aggregated-Cost" for 734 the EC server directly attached. 736 When carrying the "Aggregate-Cost" sub-sub TLVs, the App 737 specific Link Attribute sub-TLV can be included in TLV 22 738 (extended IS reachability), 23 (IS Neighbor Attribute), or 739 25(L2 bundle Member Attribute). 741 The Aggregate-Cost bit is added to the Standard 742 Applications Bit Mask (SABM). 744 0 1 2 3 4 5 6 7 ... 745 +-+-+-+-+-+-+-+-+... 746 |R|S|F|C| ... 747 +-+-+-+-+-+-+-+-+... 748 Figure 9: Extended Application Identifier Bit Mask 750 C-bit: set to specify the Site Cost related sub-sub TLVs, 751 described in the Section 3.2, are included in the App- 752 Specific Sub-TLV. 754 The R-bit, S-bit, F-bit are specified by the RFC8919. 756 The Extended App Specific Link Attributes Sub-TLV is as 757 following: 759 Type: 16 760 Length: (1 octet) 761 Value: 762 Extended Application Identifier Bit Mask [Figure 2] 763 Aggregate-Cost sub-sub-TLVs. 765 Internet-Draft LSR Extension for 5G EC Service 767 0 1 2 3 768 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 769 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 770 |AggCostSubTLV | Length | AggCost to the EC Server | 771 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 772 | PrefixLength | PrefixOptions | 0 | 773 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 774 | Address Prefix | 775 | ... | 776 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 777 Figure 10: Aggregated Cost to EC Server 779 8. Aggregated Cost Advertisement in OSPF 781 When all egress edge routers with directly attached EC 782 servers can compute the aggregated cost that takes into 783 consideration the Load Measurement, Capacity value, and 784 Preference value, this aggregated cost can be considered as 785 the Metric of the link to the EC Server. 787 8.1. OSPFv3 LSA to carry the Aggregated Cost 789 For EC servers using IPv6 address, the aggregated cost 790 computed by the A-ERs can be encoded in the Metric field 791 [the interface cost] of Intra-Area-Prefix-LSA specified by 792 Section 3.7 of the [ RFC5340]. 793 0 1 2 3 794 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 795 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 796 | 6 (Intra-Area Prefix) | TLV Length | 797 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 798 | 0 | Aggregated Cost to the EC Server | 799 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 800 | PrefixLength | PrefixOptions | 0 | 801 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 802 | Address Prefix | 803 | ... | 804 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 805 Figure 11: Aggregated Cost to App Server 807 8.2. OSPFv2 LSA to carry the Aggregated Cost 809 For EC servers in IPv4 address, the aggregated cost can be 810 encoded in the "Metric" field of the Stub Link LSA [Link 811 type =3] specified by Section 12.4 of the [RFC2328]. 813 Internet-Draft LSR Extension for 5G EC Service 815 9. Alternative method for Distributing Aggregated Cost 817 Section 7 and Section 8 demonstrate different ways for 818 OSPFv2, OSPFv3, and ISIS to propagate the aggregated cost. 819 It would be better if the aggregated cost could be 820 advertised the same way, regardless of OSPFv2, OSPFv3, or 821 ISIS. 823 Draft [draft-wang-lsr-stub-link-attributes] introduces the 824 Stub-Link TLV for OSPFv2/v3 and ISIS protocol respectively. 825 Considering the interfaces on an edge router that connects 826 to the EC servers are normally configured as passive 827 interfaces, these IP-layer App-metrics can also be 828 advertised as the attributes of the passive/stub link. The 829 associated prefixes can then be advertised in the "Stub- 830 Link TLV" that is defined in [draft-wang-lsr-stub-link- 831 attributes]. All the associated prefixes share the same 832 characteristic of the link. Other link related sub-TLVs 833 defined in [RFC8920] can also be attached and applied to 834 the calculation of path to the associated prefixes." 836 Section 6 for the advertisement of AppMetaData Metric can 837 also utilize the Stub-Link TLV that defined in [draft-wang- 838 lsr-stub-link-attributes] 840 10. Manageability Considerations 842 To be added. 844 11. Security Considerations 846 To be added. 848 12. IANA Considerations 850 The following Sub-TLV types need to be added by IANA to 851 FlexAlgo. 853 - AppMetaData Type for ISIS (TBD1): IPv4 or IPv6 854 - AppMetaData Type for OSPF (TBD2): IPv4 or IPv6 856 Internet-Draft LSR Extension for 5G EC Service 858 The following SubSub-TLV types need to be added by IANA, to 859 be included in FAD sub-TLV, ISIS Extended-LSA Sub-TLVs, or 860 OSPFv2 Extended Link Opaque LSA TLVs Registry. 862 - Aggregated Load Index Sub-TLV type (TBD3) 863 - Raw Load Measurement Sub-TLV type (TBD4) 864 - Capacity Index Sub-TLV type (TBD5) 865 - Preference Index Sub-TLV type (TBD6) 867 13. References 869 13.1. Normative References 871 [RFC2119] Bradner, S., "Key words for use in RFCs to 872 Indicate Requirement Levels", BCP 14, RFC 2119, 873 March 1997. 875 [RFC2328] J. Moy, "OSPF Version 2", RFC 2328, April 1998. 877 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 878 Subsequent Address Family Identifier (SAFI) and 879 the BGP Tunnel Encapsulation Attribute", April 880 2009. 882 [RFC7684] P. Psenak, et al, "OSPFv2 Prefix/Link Attribute 883 Advertisement", RFC 7684, Nov. 2015. 885 [RFC8200] S. Deering R. Hinden, "Internet Protocol, Version 886 6 (IPv6) Specification", July 2017. 888 [RFC8326] A. Lindem, et al, "OSPFv3 Link State 889 advertisement (LSA0 Extensibility", RFC 8362, 890 April 2018. 892 [RFC9012] E. Rosen, et al "The BGP Tunnel Encapsulation 893 Attribute", April 2021. 895 Internet-Draft LSR Extension for 5G EC Service 897 13.2. Informative References 899 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 900 Partnership Project; Technical Specification 901 Group Services and System Aspects; Study on 902 enhancement of support for Edge Computing in 5G 903 Core network (5GC)", Release 17 work in progress, 904 Aug 2020. 906 [5G-StickyService] L. Dunbar, J. Kaippallimalil, "IPv6 907 Solution for 5G Edge Computing Sticky Service", 908 draft-dunbar-6man-5g-ec-sticky-service-00, work- 909 in-progress, Oct 2020. 911 [BGP-5G-AppMetaData] L. Dunbar, K. Majumdar, H. Wang, "BGP 912 App Metadata for 5G Edge Computing Service", 913 draft-dunbar-idr-5g-edge-compute-app-meta-data- 914 03, work-in-progress, Sept 2020. 916 [LSR-Flex-Algo] P. Psenak, et al, "IGP Flexible Algorithm", 917 draft-ietf-lsr-flex-algo-17, July 2021. 919 [LSR-Flex-Algo-BW] S. Hegde, et al, "Flexible Algorithms: 920 Bandwidth, Delay, Metrics and Constraints", 921 draft-ietf-lsr-flex-algo-bw-con-01, July 2021. 923 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 924 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 925 draft-dunbar-idr-sdwan-edge-discovery-00, work- 926 in-progress, July 2020. 928 14. Acknowledgments 930 Acknowledgements to Acee Lindem, Shraddha Hegde, Tony Li, 931 Gyan Mishra, Jeff Tantsura, and Donald Eastlake for their 932 review and suggestions. 934 This document was prepared using 2-Word-v2.0.template.dot. 936 Internet-Draft LSR Extension for 5G EC Service 938 Authors' Addresses 940 Linda Dunbar 941 Futurewei 942 Email: ldunbar@futurewei.com 944 Huaimo Chen 945 Futurewei 946 Email: huaimo.chen@futurewei.com 948 Aijun Wang 949 China Telecom 950 Email: wangaj3@chinatelecom.cn