idnits 2.17.1 draft-dunbar-lsr-5g-edge-compute-ospf-ext-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (March 8, 2021) is 1138 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: '5GC' is mentioned on line 125, but not defined == Missing Reference: 'A-ER' is mentioned on line 224, but not defined == Missing Reference: 'RFC8174' is mentioned on line 285, but not defined == Missing Reference: 'IPv6-StickyService' is mentioned on line 364, but not defined == Missing Reference: 'RFC8920' is mentioned on line 528, but not defined ** Obsolete undefined reference: RFC 8920 (Obsoleted by RFC 9492) == Unused Reference: 'RFC8200' is defined on line 795, but no explicit reference was found in the text == Unused Reference: 'RFC8326' is defined on line 798, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 804, but no explicit reference was found in the text == Unused Reference: '5G-StickyService' is defined on line 821, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 826, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 833, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 838, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 843, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 7684 ** Downref: Normative reference to an Proposed Standard RFC: RFC 8362 (ref. 'RFC8326') == Outdated reference: A later version (-14) exists of draft-dunbar-idr-5g-edge-compute-app-meta-data-01 == Outdated reference: A later version (-02) exists of draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-01 -- No information found for draft-dunbar-6man-5g-ec-sticky-service - is the name correct? == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 3 errors (**), 0 flaws (~~), 19 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft H. Chen 3 Intended status: Standard Futurewei 4 Expires: September 7, 2021 Aijun Wang 5 China Telecom 6 March 8, 2021 8 OSPF extension for 5G Edge Computing Service 9 draft-dunbar-lsr-5g-edge-compute-ospf-ext-03 11 Abstract 12 This draft describes an OSPF extension for routers to 13 advertise the running status and environment of the 14 directly attached 5G Edge Computing servers. The 15 AppMetaData can be used by the routers in the 5G Local Data 16 Network to make intelligent decision to optimize the 17 forwarding of flows from UEs. The goal is to improve 18 latency and performance for 5G Edge Computing services. 20 Status of this Memo 21 This Internet-Draft is submitted in full conformance with 22 the provisions of BCP 78 and BCP 79. 24 This Internet-Draft is submitted in full conformance with 25 the provisions of BCP 78 and BCP 79. This document may not 26 be modified, and derivative works of it may not be created, 27 except to publish it as an RFC and to translate it into 28 languages other than English. 30 Internet-Drafts are working documents of the Internet 31 Engineering Task Force (IETF), its areas, and its working 32 groups. Note that other groups may also distribute working 33 documents as Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of 36 six months and may be updated, replaced, or obsoleted by 37 other documents at any time. It is inappropriate to use 38 Internet-Drafts as reference material or to cite them other 39 than as "work in progress." 41 The list of current Internet-Drafts can be accessed at 42 http://www.ietf.org/ietf/1id-abstracts.txt 44 Internet-Draft OSPF Extension for 5G EC Service 46 The list of Internet-Draft Shadow Directories can be 47 accessed at http://www.ietf.org/shadow.html 49 This Internet-Draft will expire on April 7, 2021. 51 Copyright Notice 53 Copyright (c) 2021 IETF Trust and the persons identified as 54 the document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's 57 Legal Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the 59 date of publication of this document. Please review these 60 documents carefully, as they describe your rights and 61 restrictions with respect to this document. Code Components 62 extracted from this document must include Simplified BSD 63 License text as described in Section 4.e of the Trust Legal 64 Provisions and are provided without warranty as described 65 in the Simplified BSD License. 67 Table of Contents 69 1. Introduction........................................... 3 70 1.1. 5G Edge Computing Background...................... 3 71 1.2. Problem#1: ANYCAST in 5G EC Environment........... 4 72 1.3. Problem #2: Unbalanced Anycast Distribution due to 73 UE Mobility............................................ 5 74 1.4. Problem 3: Application Server Relocation.......... 5 75 2. Conventions used in this document...................... 5 76 3. Solution Overview...................................... 7 77 3.1. Flow Affinity to an ANYCAST server................ 7 78 3.2. IP Layer Metrics to Gauge App Server Running Status 79 ....................................................... 9 80 3.3. To Equalize traffic among Multiple ANYCAST 81 Locations............................................. 10 82 3.4. Reason for using IGP Based Solution.............. 11 83 4. Aggregated Cost Computed by Egress routers............ 11 84 4.1. OSPFv3 LSA to carry the Aggregated Cost.......... 11 85 4.2. OSPFv2 LSA to carry the Aggregated Cost.......... 12 86 5. IP Layer App-Metrics Advertisements................... 12 87 5.1. OSPFv3 Extension to carry the App-Metrics........ 13 89 Internet-Draft OSPF Extension for 5G EC Service 91 5.2. OSPFv2 Extension to advertise the IP Layer App- 92 Metrics............................................... 13 93 5.3. IP Layer App-Metrics Sub-TLVs.................... 15 94 6. Soft Anchoring of an ANYCAST Flow..................... 17 95 7. Manageability Considerations.......................... 19 96 8. Security Considerations............................... 19 97 9. IANA Considerations................................... 19 98 10. References........................................... 19 99 10.1. Normative References............................ 19 100 10.2. Informative References.......................... 20 101 11. Acknowledgments...................................... 21 103 1. Introduction 105 This document describes an OSPF extension to distribute the 106 5G Edge Computing App running status and environment, so 107 that other routers in the 5G Local Data Network (LDN) can 108 make intelligent decision to optimize forwarding of flows 109 from UEs. The goal is to improve latency and performance 110 for 5G Edge Computing services. 112 1.1. 5G Edge Computing Background 114 As described in [5G-EC-Metrics], one Application can have 115 multiple Application Servers hosted in different Edge 116 Computing data centers that can be close in proximity. 117 Those Edge Computing (mini) data centers are usually very 118 close to, or co-located with, 5G base stations, with the 119 goal to minimize latency and to optimize the user 120 experience. 122 When a UE (User Equipment) initiates application packets 123 using the destination address from a DNS reply or its own 124 cache, the packets from the UE are carried in a PDU session 125 through 5G Core [5GC] to the 5G UPF-PSA (User Plan Function 126 - PDU Session Anchor). The UPF-PSA decapsulates the 5G GTP 127 outer header and forwards the packets from the UEs to the 128 Ingress router of the Edge Computing (EC) Local Data 129 Network (LDN) which is responsible for forwarding the 130 packets to the intended destinations. 132 When the UE moves out of coverage of its current gNB (next 133 generation Node B) (gNB1), the handover procedure is 134 initiated which includes the 5G SMF (Session Management 136 Internet-Draft OSPF Extension for 5G EC Service 138 Function) selecting a new UPF-PSA [3GPP TS 23.501 and TS 139 23.502]. When the handover process is complete, the UE has 140 a new IP address and the IP point of attachment is to the 141 new UPF-PSA. 5GC may maintain a path from the old UPF to 142 new the UPF for a short period of time for SSC [Session and 143 Service Continuity] mode 3 to make the handover process 144 more seamless. 145 +--+ 146 |UE|---\+---------+ +------------------+ 147 +--+ | 5G | +---------+ | S1: aa08::4450 | 148 +--+ | Site +--++---+ +----+ | 149 |UE|----| A |PSA| Ra| | R1 | S2: aa08::4460 | 150 +--+ | +---+---+ +----+ | 151 +---+ | | | | | S3: aa08::4470 | 152 |UE1|---/+---------+ | | +------------------+ 153 +---+ |IP Network | L-DN1 154 |(3GPP N6) | 155 | | | +------------------+ 156 | UE1 | | | S1: aa08::4450 | 157 | moves to | +----+ | 158 | Site B | | R3 | S2: aa08::4460 | 159 v | +----+ | 160 | | | S3: aa08::4470 | 161 | | +------------------+ 162 | | L-DN3 163 +--+ | | 164 |UE|---\+---------+ | | +------------------+ 165 +--+ | 5G | | | | S1: aa08::4450 | 166 +--+ | Site +--++-+--+ +----+ | 167 |UE|----| B |PSA| Rb | | R2 | S2: aa08::4460 | 168 +--+ | +--++----+ +----+ | 169 +--+ | | +-----------+ | S3: aa08::4470 | 170 |UE|---/+---------+ +------------------+ 171 +--+ L-DN2 172 Figure 1: App Servers in different edge DCs 174 1.2. Problem#1: ANYCAST in 5G EC Environment 176 Increasingly, ANYCAST is used extensively by various 177 application providers and CDNs because ANYCAST makes it 178 possible to dynamically load balance across server 179 locations based on network conditions. With multiple 181 Internet-Draft OSPF Extension for 5G EC Service 183 servers having the same ANYCAST address, it eliminates the 184 single point of failure and bottleneck at the DNS resolvers 185 and application layer load balancers. Another benefit of 186 using ANYCAST address is removing the dependency on how UEs 187 get the IP addresses for their Applications. Some UEs (or 188 clients) might use stale cached IP addresses for extended 189 period. 191 But, having multiple locations of the same ANYCAST address 192 in 5G Edge Computing environment can be problematic because 193 all those edge computing Data Centers can be close in 194 proximity. There might be very little difference in the 195 routing cost to reach the Application Servers in different 196 Edge DCs, which can cause packets from one flow to be 197 forwarded to different locations, resulting in service 198 glitches. 200 1.3. Problem #2: Unbalanced Anycast Distribution due to UE 201 Mobility 203 UEs' frequent moving from one 5G site to another can make 204 it difficult to plan where the App Servers should be 205 hosted. When one App server is heavily utilized, other App 206 servers of the same address close-by can be very under- 207 utilized. Since the condition can be short lived, it is 208 difficult for the application controller to anticipate the 209 move and adjust. 211 1.4. Problem 3: Application Server Relocation 213 When an Application Server is added to, moved, or deleted 214 from a 5G Edge Computing Data Center, not only the 215 reachability changes but also the utilization and capacity 216 for the Data Center might change. 218 Note: for the ease of description, the Edge Computing 219 server, Application server, App server are used 220 interchangeably throughout this document. 222 2. Conventions used in this document 224 A-ER: Egress Router to an Application Server, [A-ER] 225 is used to describe the last router that the 227 Internet-Draft OSPF Extension for 5G EC Service 229 Application Server is attached. For 5G EC 230 environment, the A-ER can be the gateway router 231 to a (mini) Edge Computing Data Center. 233 Application Server: An application server is a physical or 234 virtual server that host the software system 235 for the application. 237 Application Server Location: Represent a cluster of servers 238 at one location serving the same Application. 239 One application may have a Layer 7 Load 240 balancer, whose address(es) are reachable from 241 external IP network, in front of a set of 242 application servers. From IP network 243 perspective, this whole group of servers are 244 considered as the Application server at the 245 location. 247 Edge Application Server: used interchangeably with 248 Application Server throughout this document. 250 EC: Edge Computing 252 Edge Hosting Environment: An environment providing support 253 required for Edge Application Server's 254 execution. 256 NOTE: The above terminologies are the same as 257 those used in 3GPP TR 23.758 259 Edge DC: Edge Data Center, which provides the Edge 260 Computing Hosting Environment. It might be co- 261 located with 5G Base Station and not only host 262 5G core functions, but also host frequently 263 used Edge server instances. 265 gNB next generation Node B 267 LDN: Local Data Network 269 PSA: PDU Session Anchor (UPF) 271 Internet-Draft OSPF Extension for 5G EC Service 273 SSC: Session and Service Continuity 275 UE: User Equipment 277 UPF: User Plane Function 279 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 280 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT 281 RECOMMENDED", "MAY", and "OPTIONAL" in this document are to 282 be interpreted as described in BCP 14 [RFC2119] [RFC8174] 283 when, and only when, they appear in all capitals, as shown 284 here. 286 3. Solution Overview 288 From IP Layer, the Application Servers are identified by 289 their IP (ANYCAST) addresses. The 5G Edge Computing 290 controller or management system is aware of the ANYCAST 291 addresses that need the optimized forwarding. The 5G Edge 292 Computing controller or management system can configure the 293 ACLs on selected routers in LDN to filter out packets for 294 those applications. 296 The proposed solution is for the egress router (A-ER), to 297 which the Application Servers are directly attached to 298 collect various measurements about the Servers' running 299 status [5G-EC-Metrics] and advertise the metrics to other 300 routers in 5G EC LDN. 302 3.1. Flow Affinity to an ANYCAST server 304 When there are multiple Edge Computing Servers with the 305 same ANYCAST address located in different mini Edge 306 Computing Data Centers, each location is identified by its 307 A-ER unicast address(es). To the routers in an LDN, 308 achieving Flow Affinity is to send the packets of the same 309 flow to the same A-ER's unicast address. A-ER, e.g. R1 in 310 Figure 1, should deliver the packets destined towards the 311 ANYCAST address to its directly attached server even 312 through the same address is also reachable from other 313 routers, unless the directly attached server is no longer 314 reachable due to Server or Link failure. 316 Internet-Draft OSPF Extension for 5G EC Service 318 Many commercial routers today support some forms of flow 319 affinity to ensure packets belonging to one flow be 320 forwarded along the same path. For servers with the same 321 ANYCAST address attached to 3 different egress routers, 322 routers supporting the flow affinity feature should forward 323 the packets of one flow to the same egress router even if 324 the cost towards the egress router changes. 326 Editor's note: for IPv6 traffic, Flow Affinity can be 327 supported by the routers of the Local Data Network (LDN) 328 forwarding the packets with the same Flow Label in the 329 packets' IPv6 Header along the same path towards the same 330 egress router. 332 Internet-Draft OSPF Extension for 5G EC Service 334 3.2. IP Layer Metrics to Gauge App Server Running Status 336 Most application don't expose their internal logics to 337 network. Their communications are generally encrypted. Most 338 of them do not even respond to PING or ICMP messages 339 initiated by routers or network gears. 341 [5G-EC-Metrics] describes the IP Layer Metrics that can 342 gauge the application servers running status and 343 environment: 345 - IP-Layer Metric for App Server Load Measurement: 346 The Load Measurement to an App Server is a weighted 347 combination of the number of packets/bites to the App 348 Server and the number of packets/bytes from the App 349 Server which are collected by the A-ER that has the 350 direct connection to the App Server. 351 The A-ER is configured with an ACL that can filter out 352 the packets for the Application Server. 353 - Capacity Index: 354 Capacity Index is used to differentiate the running 355 environment of the attached application server. Some 356 data centers can have hundreds, or thousands, of 357 servers behind an application server's App Layer Load 358 Balancer. Other data centers can have very small 359 number of servers for the application. "Capacity 360 Index", which is a numeric number, is used to 361 represent the capacity of the application server 362 attached to an A-ER. 363 - Site preference index: 364 [IPv6-StickyService] describes a scenario that some 365 sites are more preferred for handling an application 366 than others for flows from a specific UE. 368 For ease of description, those metrics, more may be added 369 later, are called IP Layer App-Metrics throughout the 370 document. 372 Internet-Draft OSPF Extension for 5G EC Service 374 3.3. To Equalize traffic among Multiple ANYCAST Locations 376 The main benefit of using ANYCAST is to leverage the 377 network layer information to balance the traffic among 378 multiple Application Server locations. 380 For 5G Edge Computing environment, the routers in the LDN 381 need to be notified of various measurement of the App 382 Servers attached to each A-ER to make the intelligent 383 decision on where to forward the traffic for the 384 application from UEs. 386 [5G-EC-Metrics] describes the algorithms that can be used 387 by the routers in LDN to compare the cost to reach the App 388 Servers between the Site-i or Site-j: 390 Load-i * CP-j Pref-j * Network-Delay-i 391 Cost-i=min(w *(----------------) + (1-w) *(-------------------------)) 392 Load-j * CP-i Pref-i * Network-Delay-j 394 Load-i: Load Index at Site-i, it is the weighted 395 combination of the total packets or/and bytes sent to 396 and received from the Application Server at Site-i 397 during a fixed time period. 399 CP-i: capacity index at the site I, higher value means 400 higher capacity. 402 Network Delay-i: Network latency measurement (RTT) to 403 the A-ER that has the Application Server attached at the 404 site-i. 405 Noted: Ingress nodes can easily measure RTT to all the 406 egress nodes by existing IPPM metrics. But it is not so 407 easy for ingress nodes to measure RTT to all the App 408 Servers. Therefore, "Network-Delay-i", a.k.a. Network 409 latency measurement (RTT), is between the Ingress nodes 410 and egress nodes. The link cost between the egress nodes 411 to their attached servers are embedded in the "capacity 412 index". 414 Pref-i: Preference index for the site-i, higher value 415 means higher preference. 417 w: Weight for load and site information, which is a 418 value between 0 and 1. If smaller than 0.5, Network 419 latency and the site Preference have more influence; 421 Internet-Draft OSPF Extension for 5G EC Service 423 otherwise, Server load and its capacity have more 424 influence. 426 3.4. Reason for using IGP Based Solution 428 Here are some benefits in using IGP to propagate the IP 429 Layer App-Metrics: 430 - Intermediate routers can derive the aggregated cost to 431 reach the Application Servers attached to different 432 egress nodes, especially: 433 - The path to the optimal egress node can be more 434 accurate or shorter 435 - Convergence is shorter when there is any failure 436 along the way towards the optimal ANYCAST server. 437 - When there is any failure at the intended ANYCAST 438 server, all the transient packets can be optimally 439 forwarded to another App Server attached to a 440 different egress router. 441 - Doesn't need ingress node to establish tunnels with 442 egress nodes. 444 There are limitations of using IGP too, such as: 446 - The IGP approach might not suit well to 5G EC LDN 447 operated by multiple ISPs networks. 448 For LDN operated by multiple IPSs, BGP should be used. 449 AppMetaData NLRI Path Attribute [5G-AppMetaData] 450 describes the BGP UPDATE message to propagate IP Layer 451 App-Metrics crossing multiple ISPs. 453 4. Aggregated Cost Computed by Egress Routers 455 If all egress routers that have direct connection to the 456 App Servers can be configured with a consistent algorithm 457 to compute an aggregated cost that take into consideration 458 the Load Measurement, Capacity value and Preference value, 459 this aggregated cost can be considered as the Metric of the 460 link to the App Server. 462 In this scenario, there is no protocol extension needed. 464 4.1. OSPFv3 LSA to carry the Aggregated Cost 466 If the App Servers use IPv6 ANYCAST address, the aggregated 467 cost computed by the egress routers can be encoded in the 469 Internet-Draft OSPF Extension for 5G EC Service 471 Metric field [the interface cost] of Intra-Area-Prefix-LSA 472 specified by the Section 3.7 of the [ RFC5340]. 474 0 1 2 3 475 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 476 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 477 | 6 (Intra-Area Prefix) | TLV Length | 478 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 479 | 0 | Aggregated Cost to the App Server | 480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 481 | PrefixLength | PrefixOptions | 0 | 482 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 483 | Address Prefix | 484 | ... | 485 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 486 Figure 2: Aggregated Cost to App Server 488 4.2. OSPFv2 LSA to carry the Aggregated Cost 490 For App Servers in IPv4 address, the Aggregated Cost can be 491 encoded in the "Metric" field of the Stub Link LSA [Link 492 type =3] specified by the Section 12.4 of the [RFC2328]. 494 5. IP Layer App-Metrics Advertisements 496 This section describes the OSPF extension that can carry 497 the detailed IP Layer Metrics when it is not possible for 498 all the egress routers to have a consistent algorithm to 499 compute the aggregated cost or some routers need all the 500 detailed IP Layer metrics for the App Servers for other 501 purposes. 503 Since only a subset of routers within an IGP domain need to 504 know those detailed metrics, it makes sense to use the 505 OSPFv2 Extended Prefix Opaque LSA for IPv4 and OSPFv3 506 Extended LSA with Intra-Area-Prefix TLV to carry the 507 detailed sub-TLVs. For routers that don't care about those 508 metrics, they can ignore them very easily. 510 It worth noting that not all hosts (prefix) attached to an 511 A-ER are ANYCAST servers that need network optimization. 512 An A-ER only needs to advertise the App-Metrics for the 513 ANYCAST addresses that match with the configured ACLs. 515 Draft [draft-wang-lsr-passive-interface-attribute] 516 introduces the Stub-Link TLV for OSPFv2/v3 and ISIS 518 Internet-Draft OSPF Extension for 5G EC Service 520 protocol respectively. Considering the interfaces on an 521 edge router that connects to the App servers are normally 522 configured as passive interfaces, these IP-layer App- 523 metrics can also be advertised as the attributes of the 524 passive/stub link. The associated prefixes can then be 525 advertised in the "Stub-Link Prefix Sub-TLV" that defined 526 in [draft-wang-lsr-passive-interface-attribute]. All the 527 associated prefixes share the same characteristic of the 528 link. Other link related sub-TLVs defined in [RFC8920] can 529 also be attached and applied to the calculation of path to 530 the associated prefixes. 532 5.1. OSPFv3 Extension to carry the App-Metrics 534 For App Servers using IPv6, the OSPFv3 Extended LSA with 535 the Intra-Area-Prefix Address TLV specified by the Section 536 3.7 of RFC8362 can be used to carry the App-Metrics for the 537 attached App Servers. 539 0 1 2 3 540 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 541 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 542 |7 (IPv6 Local-Local Address) | Length | 543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 544 | IPv6 AppServer (ANYCAST) address | 545 ~ ~ 546 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 547 | Load measurement sub-TLV | 548 ~ ~ 549 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 550 | Capability sub-TLV | 551 ~ ~ 552 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 553 | Preference sub-TLV | 554 ~ ~ 555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 556 Figure 3: IPv6 App Server App-Metrics Encoding 558 5.2. OSPFv2 Extension to advertise the IP Layer App-Metrics 560 For App Servers using IPv4 addresses, the OSPFv2 Extended 561 Prefix Opaque LSA with the extended Prefix TLV can be used 563 Internet-Draft OSPF Extension for 5G EC Service 565 to carry the App Metrics sub-TLVs, as specified by the 566 Section 2.1 [RFC7684]. 568 Here is the proposed encoding: 570 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 571 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 572 | Type | Length | 573 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 574 | Route Type | Prefix Length | AF | Flags | 575 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 576 | Address Prefix (variable) | 577 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 578 | Load Measurement Sub-TLV | 579 ~ ~ 580 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 581 | capacity Index Sub-TLV | 582 ~ ~ 583 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 584 | Site Preference Sub-TLV | 585 ~ ~ 586 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 587 Figure 4: App-Metrix Sub-TLVs in OSPFv2 Extended Prefix TLV 589 Internet-Draft OSPF Extension for 5G EC Service 591 5.3. IP Layer App-Metrics Sub-TLVs 593 Two types of Load Measurement Sub-TLVs are specified: 595 a) The Aggregated Load Index based on weighted combination 596 of the collected measurements; 597 b) The raw measurements of packets/bytes to/from the App 598 Server address. The raw measurement is useful when the 599 egress routers cannot be configured with a consistent 600 algorithm to compute the aggregated load index or the 601 raw measurements are needed by a central analytic 602 system. 604 The Aggregated Load Index Sub-TLV has the following format: 606 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 607 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 608 | Type (TBD2) | Length | 609 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 610 | Measurement Period | 611 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 612 | Aggregated Load Index to reach the App Server | 613 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 614 Figure 5: Aggregated Load Index Sub-TLV 616 Type=TBD2 (to be assigned by IANA) indicates that the 617 sub-TLV carries the Aggregated Load Measurement Index 618 derived from the Weighted combination of bytes/packets 619 sent to/received from the App server: 621 Index=w1*ToPackets+w2*FromPackes+w3*ToBytes+w4*FromBytes 623 Where wi is a value between 0 and 1; w1+ w2+ w3+ w4 = 1. 625 Internet-Draft OSPF Extension for 5G EC Service 627 The Raw Load Measurement sub-TLV has the following format: 629 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 630 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 631 | Type (TBD3) | Length | 632 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 633 | Measurement Period | 634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 635 | total number of packets to the AppServer | 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 | total number of packets from the AppServer | 638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 639 | total number of bytes to the AppServer | 640 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 641 | total number of bytes from the AppServer | 642 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 643 Figure 6: Raw Load Measurement Sub-TLV 645 Type= TBD3 (to be assigned by IANA) indicates that the 646 sub-TLV carries the Raw measurements of packets/bytes 647 to/from the App Server ANYCAST address. 649 Measurement Period: A user specified period in seconds, 650 default is 3600 seconds. 652 The Capacity Index sub-TLV has the following format: 654 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 655 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 656 | Type (TBD3) | Length | 657 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 658 | Capacity Index | 659 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 660 Figure 7: Capacity Index Sub-TLV 662 Internet-Draft OSPF Extension for 5G EC Service 664 The Preference Index sub-TLV has the following format: 666 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 668 | Type (TBD4) | Length | 669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 670 | Preference Index | 671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 672 Figure 8: Preference Index Sub-TLV 674 Note: "Capacity Index" and "Site preference" can be more 675 stable for each site. If those values are configured to 676 nodes, they might not need to be included in every OSPF 677 LSA. 679 6. Soft Anchoring of an ANYCAST Flow 680 This section describes a solution that can anchor an 681 application flow from a UE to a specific ANYCAST Server 682 location even when the UE moves from one 5G Site to 683 another. This is called "Sticky Service" in the 3GPP Edge 684 Computing specification. 686 Lets' assume one application "App.net" is instantiated on 687 four servers that are attached to four different A-ERs: 688 R1, R2, R3, and R4 respectively. The "App.net" needs to be 689 Sticky means that the packets to the "App.net" from UE-1 690 needs to stick with one server, say the "App.net" Server 691 attached to R1, even when the UE moves from one 5G site to 692 another. When there is failure at R1 or the Application 693 Server attached to R1, the packets of the flow "App.net" 694 from UE-1 need to be forwarded to the Application Server 695 attached to R2, R3, or R4. 697 We call this kind of sticky service "Soft Anchoring", 698 meaning that anchoring to the site of R1 is preferred, but 699 other sites can be chosen when the preferred site 700 encounters failure. 702 Here are the steps to achieving "Soft Anchoring": 704 - Assign a group of ANYCAST addresses to one 705 application. For example, "App.net" is assigned with 707 Internet-Draft OSPF Extension for 5G EC Service 709 4 ANYCAST addresses, L1, L2, L3, and L4. L1/L2/L3/L4 710 represents the location preferred ANYCAST addresses. 711 - For the App.net Server attached to a router, the 712 router has four Stub links to the same Server, L1, 713 L2, L3, and L4 respectively. The cost to L1, L2, L3 714 and L4 is assigned differently for different routers. 715 For example, 716 o When attached to R1, the L1 has the lowest cost, 717 say 10, when attached to R2, R3, and R4, the L1 718 can have higher cost, say 30. 719 o ANYCAST L2 has the lowest cost when attached to 720 R2, higher cost when attached to R1, R3, R4 721 respectively. 722 o ANYCAST L3 has the lowest cost when attached to 723 R3, higher cost when attached to R1, R2, R4 724 respectively, and 725 o ANYCAST L4 has the lowest cost when attached to 726 R4, higher cost when attached to R1, R2, R3 727 respectively 728 - When a UE queries for the "App.net" for the first 729 time, the DNS replies the location preferred ANYCAST 730 address, say L1, based on where the query is 731 initiated. 732 - When the UE moves from one 5G site-A to Site-B, UE 733 continues sending packets of the "App.net" to ANYCAST 734 address L1. The routers will continue sending packets 735 to R1 because the total cost for the App.net instance 736 for ANYCAST L1 is lowest at R1. If any failure occurs 737 making R1 not reachable, the packets of the "App.net" 738 from UE-1 will be sent to R2, R3, or R4 (depending on 739 the total cost to reach each of them). 741 If the Application Server supports the HTTP redirect, more 742 optimal forwarding can be achieved. 744 - When a UE queries for the "App.net" for the first 745 time, the global DNS replies the ANYCAST address G1, 747 Internet-Draft OSPF Extension for 5G EC Service 749 which has the same cost regardless where the 750 Application Servers are attached. 751 - When the UE initiates the communication to G1, the 752 packets from the UE will be sent to the Application 753 Server that has the lowest cost, say the Server 754 attached to R1. The Application server is instructed 755 with HTTPs Redirect to respond back a location 756 specific URL, say App.net-Loc1. The client on the UE 757 will query the DNS for App.net-Loc1 and get the 758 response of ANYCAST L1. The subsequent packets from 759 the UE-1 for App.net are sent to L1. 761 7. Manageability Considerations 763 To be added. 765 8. Security Considerations 767 To be added. 769 9. IANA Considerations 771 The following Sub-TLV types need to be added by IANA 772 to OSPFv4 Extended-LSA Sub-TLVs and OSPFv2 Extended 773 Link Opaque LSA TLVs Registry. 775 - Aggregated Load Index Sub-TLV type 776 - Raw Load Measurement Sub-TLV type 777 - Capacity Index Sub-TLV type 778 - Preference Index Sub-TLV type 780 10. References 782 10.1. Normative References 784 [RFC2119] Bradner, S., "Key words for use in RFCs to 785 Indicate Requirement Levels", BCP 14, RFC 2119, 786 March 1997. 788 Internet-Draft OSPF Extension for 5G EC Service 790 [RFC2328] J. Moy, "OSPF Version 2", RFC 2328, April 1998. 792 [RFC7684] P. Psenak, et al, "OSPFv2 Prefix/Link Attribute 793 Advertisement", RFC 7684, Nov. 2015. 795 [RFC8200] S. Deering R. Hinden, "Internet Protocol, Version 796 6 (IPv6) Specification", July 2017. 798 [RFC8326] A. Lindem, et al, "OSPFv3 Link State 799 advertisement (LSA0 Extensibility", RFC 8362, 800 April 2018. 802 10.2. Informative References 804 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 805 Partnership Project; Technical Specification 806 Group Services and System Aspects; Study on 807 enhancement of support for Edge Computing in 5G 808 Core network (5GC)", Release 17 work in progress, 809 Aug 2020. 811 [5G-AppMetaData] L. Dunbar, K. Majumdar, H. Wang, "BGP NLRI 812 App Meta Data for 5G Edge Computing Service", 813 draft-dunbar-idr-5g-edge-compute-app-meta-data- 814 01, work-in-progress, Nov 2020. 816 [5G-EC-Metrics] L. Dunbar, H. Song, J. Kaippallimalil, "IP 817 Layer Metrics for 5G Edge Computing Service", 818 draft-dunbar-ippm-5g-edge-compute-ip-layer- 819 metrics-01, work-in-progress, Nov 2020. 821 [5G-StickyService] L. Dunbar, J. Kaippallimalil, "IPv6 822 Solution for 5G Edge Computing Sticky Service", 823 draft-dunbar-6man-5g-ec-sticky-service-00, work- 824 in-progress, Oct 2020. 826 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 827 Subsequent Address Family Identifier (SAFI) and 828 the BGP Tunnel Encapsulation Attribute", April 829 2009. 831 Internet-Draft OSPF Extension for 5G EC Service 833 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension 834 for SDWAN Overlay Networks", draft-dunbar-idr- 835 bgp-sdwan-overlay-ext-03, work-in-progress, Nov 836 2018. 838 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 839 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 840 draft-dunbar-idr-sdwan-edge-discovery-00, work- 841 in-progress, July 2020. 843 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel 844 Encapsulation Attribute", draft-ietf-idr-tunnel- 845 encaps-10, Aug 2018. 847 11. Acknowledgments 849 Acknowledgements to Acee Lindem, Jeff Tantsura, and Donald 850 Eastlake for their review and suggestions. 852 This document was prepared using 2-Word-v2.0.template.dot. 854 Authors' Addresses 856 Linda Dunbar 857 Futurewei 858 Email: ldunbar@futurewei.com 860 Huaimo Chen 861 Futurewei 862 Email: huaimo.chen@futurewei.com 864 Aijun Wang 865 China Telecom 866 Email: wangaj3@chinatelecom.cn