idnits 2.17.1 draft-dunbar-idr-5g-edge-compute-app-meta-data-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC9012]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (September 23, 2021) is 945 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC9012' is mentioned on line 578, but not defined == Missing Reference: '5GC' is mentioned on line 136, but not defined == Missing Reference: 'A-ER' is mentioned on line 414, but not defined == Missing Reference: '5G-Sticky-Service' is mentioned on line 544, but not defined == Missing Reference: 'Section 5' is mentioned on line 652, but not defined == Unused Reference: 'RFC4364' is defined on line 786, but no explicit reference was found in the text == Unused Reference: 'RFC8200' is defined on line 796, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 801, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 817, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 821, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 825, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 830, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 4364 == Outdated reference: A later version (-02) exists of draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-00 -- No information found for draft-dunbar-6man-5g-ec-sticky-service - is the name correct? == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 2 errors (**), 0 flaws (~~), 17 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft Futurewei 3 Intended status: Standard K. Majumdar 4 Expires: March 23, 2022 CommScope 5 H. Wang 6 Huawei 7 September 23, 2021 9 BGP App Metadata for 5G Edge Computing Service 10 draft-dunbar-idr-5g-edge-compute-app-meta-data-03 12 Abstract 13 This draft describes a new AppMetaData subTLV carried by 14 Tunnel Encap[RFC9012] Path Attribute for egress router to 15 advertise the running status and environment of the directly 16 attached 5G Edge Computing servers. The AppMetaData can be 17 used by the ingress routers in the 5G Local Data Network to 18 make intelligent path selection for flows from UEs. The goal 19 is to improve latency and performance for 5G Edge Computing 20 services. 22 The extension enables a feature, called soft anchoring, which 23 makes one Edge Computing Server at one specific location to be 24 more preferred than others for the same application to receive 25 packets from a specific source (UE). 27 Status of this Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. This document may not be 34 modified, and derivative works of it may not be created, 35 except to publish it as an RFC and to translate it into 36 languages other than English. 38 Internet-Drafts are working documents of the Internet 39 Engineering Task Force (IETF), its areas, and its working 40 groups. Note that other groups may also distribute working 41 documents as Internet-Drafts. 43 Internet-Draft AppMetaData for 5G EC Service 45 Internet-Drafts are draft documents valid for a maximum of six 46 months and may be updated, replaced, or obsoleted by other 47 documents at any time. It is inappropriate to use Internet- 48 Drafts as reference material or to cite them other than as 49 "work in progress." 51 The list of current Internet-Drafts can be accessed at 52 http://www.ietf.org/ietf/1id-abstracts.txt 54 The list of Internet-Draft Shadow Directories can be accessed 55 at http://www.ietf.org/shadow.html 57 This Internet-Draft will expire on April 7, 2021. 59 Copyright Notice 61 Copyright (c) 2021 IETF Trust and the persons identified as 62 the document authors. All rights reserved. 64 This document is subject to BCP 78 and the IETF Trust's Legal 65 Provisions Relating to IETF Documents 66 (http://trustee.ietf.org/license-info) in effect on the date 67 of publication of this document. Please review these documents 68 carefully, as they describe your rights and restrictions with 69 respect to this document. Code Components extracted from this 70 document must include Simplified BSD License text as described 71 in Section 4.e of the Trust Legal Provisions and are provided 72 without warranty as described in the Simplified BSD License. 74 Table of Contents 76 1. Introduction.............................................. 3 77 1.1. 5G Edge Computing Background......................... 3 78 1.2. 5G Edge Computing Network Properties................. 4 79 1.3. Problem#1: ANYCAST in 5G EC Environment.............. 6 80 1.4. Problem #2: Unbalanced Anycast Distribution due to UE 81 Mobility.................................................. 7 82 1.5. Problem 3: Application Server Relocation............. 7 83 2. Conventions used in this document......................... 7 84 3. Usage of App-Meta-Data for 5G Edge Computing.............. 9 86 Internet-Draft AppMetaData for 5G EC Service 88 3.1. Assumptions.......................................... 9 89 3.2. IP Layer Metrics to Gauge Application Behavior....... 9 90 3.3. AppMetaData Constrained Optimal Path Selection...... 10 91 3.4. BGP Protocol Extension to advertise Load & Capacity. 11 92 3.5. Ingress Node BGP Path Selection Behavior............ 12 93 3.5.1. AppMetaData Influenced BGP Path Selection...... 12 94 3.5.2. Forwarding Behavior............................ 12 95 3.5.3. Forwarding Behavior after a UE moving to a new 5G 96 Site.................................................. 13 97 4. The Sub-TLVs for App-Meta-Data........................... 14 98 4.1. Load Measurement sub-TLV format..................... 14 99 4.2. Capacity Index sub-TLV format....................... 15 100 4.3. The Site Preference Index sub-TLV format............ 15 101 5. AppMetaData Propagation Scope............................ 16 102 6. Soft Anchoring of an ANYCAST Flow........................ 16 103 7. Manageability Considerations............................. 18 104 8. Security Considerations.................................. 18 105 9. IANA Considerations...................................... 18 106 10. References.............................................. 18 107 10.1. Normative References............................... 18 108 10.2. Informative References............................. 19 109 11. Acknowledgments......................................... 20 111 1. Introduction 113 This document describes a new subTLV, AppMetaData, for egress 114 routers to advertise the running status and environment of the 115 directly attached Edge Computing servers. The AppMetaData can 116 be used by the ingress routers in the 5G Local Data Network to 117 make intelligent path selection for flows from UEs. The goal 118 is to improve latency and performance for 5G Edge Computing 119 services. 121 1.1. 5G Edge Computing Background 123 In 5G Edge Computing (EC), one Application can be hosted on 124 multiple Application Servers in different EC data centers that 125 are close in proximity. The network connecting the EC data 126 centers with the 5G Base stations consists of small number of 127 routers dedicated for the 5G Local Data Network (LDN), to 128 minimize latency and optimize the user experience. 130 When a User Equipment (UE) initiates application packets using 131 the destination address from a DNS reply or its cache, the 132 packets from the UE are carried in a PDU session through 5G 134 Internet-Draft AppMetaData for 5G EC Service 136 Core [5GC] to the 5G UPF-PSA (User Plan Function - PDU Session 137 Anchor). The UPF-PSA decapsulates the 5G GTP outer header and 138 forwards the packets from the UEs to its directly connected 139 Ingress router of the 5G LDN. The LDN for 5G EC, which is the 140 IP Networks from the 5GC perspective, is responsible for 141 forwarding the packets to the intended destinations. 143 When the UE moves out of coverage of its current gNB (next- 144 generation Node B) (gNB1), handover procedures are initiated, 145 and the 5G SMF (Session Management Function) selects a new 146 UPF-PSA. The standard handover procedures described in 3GPP TS 147 23.501 and TS 23.502 are followed. When the handover process 148 is complete, the UE is anchored to the new UPF-PSA, meaning 149 the packets to/from the UE is carried by the GTP tunnel to the 150 new UPF-PSA. The UE usually maintains its IP address when 151 anchored to the new UPF-PSA unless the new UFP-PSA belongs to 152 different mobile operators. 5GC may maintain a path from the 153 old UPF to new the UPF for a short time for the SSC [Session 154 and Service Continuity] mode 3 to make the handover process 155 more seamless. 157 1.2. 5G Edge Computing Network Properties 159 In this document, 5G Edge Computing Network refers to multiple 160 Local IP Data Networks (LDN) in one region that interconnect 161 the Edge Computing data centers. Those IP LDN networks are the 162 N6 interfaces from 3GPP 5G perspective. 164 The ingress routers to the 5G Edge Computing Network are the 165 routers directly connected to 5G UPFs. The egress routers to 166 the 5G Edge Computing Network are the routers that have a 167 direct link to the Edge Computing servers. The servers and the 168 egress routers are co-located. Some of those Edge Computing 169 Data centers may have Virtual switches or Top of Rack switches 170 between the egress routers and the servers. But transmission 171 delay between the egress routers and the Edge Computing 172 servers is too small to be considered in this document. 174 When one EC data center has multiple EC Servers attached to 175 one App Layer Load Balancer, only the App Layer Load Balancer 176 is visible to the 5G Edge Computing Network. How the App Layer 178 Internet-Draft AppMetaData for 5G EC Service 180 Load balancer manages the individual servers is out of the 181 scope of the network layer. 183 The 5G EC Services are specially managed services optimized by 184 utilizing the network topology and multiple servers with the 185 same IP address (ANYCAST) in multiple EC Data Centers. Many 186 services by the UEs are not part of the registered 5G EC 187 Services. 189 +--+ 190 |UE|---\+---------+ +------------------+ 191 +--+ | 5G | +--------+ | S1: aa08::4450 | 192 +--+ | Site +--+-+---+ +----+ | 193 |UE|----| A |PSA1| Ra| | R1 | S2: aa08::4460 | 194 +--+ | +----+---+ +----+ | 195 +---+ | | | | | S3: aa08::4470 | 196 |UE1|---/+---------+ | | +------------------+ 197 +---+ |IP Network | L-DN1 198 |(3GPP N6) | 199 | | | +------------------+ 200 | UE1 | | | S1: aa08::4450 | 201 | moves to | +----+ | 202 | Site B | | R3 | S2: aa08::4460 | 203 v | +----+ | 204 | | | S3: aa08::4470 | 205 | | +------------------+ 206 | | L-DN3 207 +--+ | | 208 |UE|---\+---------+ | | +------------------+ 209 +--+ | 5G | | | | S1: aa08::4450 | 210 +--+ | Site +--+--+---+ +----+ | 211 |UE|----| B |PSA2| Rb | | R2 | S2: aa08::4460 | 212 +--+ | +--+-+----+ +----+ | 213 +--+ | | +-----------+ | S3: aa08::4470 | 214 |UE|---/+---------+ +------------------+ 215 +--+ L-DN2 216 Figure 1: App Servers in different edge DCs 218 Internet-Draft AppMetaData for 5G EC Service 220 1.3. Problem#1: ANYCAST in 5G EC Environment 222 Increasingly, Anycast is used extensively by various 223 application providers and CDNs because ANYCAST makes it 224 possible to dynamically load balance across server locations 225 based on network conditions. 227 Using Anycast address leverages the proximity information 228 present in the network (routing) layer. It eliminates the 229 single point of failure and bottleneck at the DNS resolvers 230 and application layer load balancers. Another benefit of using 231 the ANYCAST address is removing the dependency on UEs. Some 232 UEs (or clients) might use their cached IP addresses, instead 233 of querying DNS, for an extended period. 235 But, having multiple locations of the same ANYCAST address in 236 the 5G EC environment can be problematic because all those EC 237 Data Centers can be close in proximity. There might be a very 238 small difference in the routing cost to reach the Application 239 Servers in different EC DCs. This list elaborates the issues 240 in detail: 242 a) Path Selection: When a new flow comes to an ingress node 243 (Ra), how to select the optimal egress router to reach an 244 ANYCAST server. 246 The mechanism described in this draft is for solving this 247 Path Selection problem. 249 b) How Ingress node keeps the packets from one flow to the 250 same ANYCAST server. 252 a.k.a. Flow Affinity, or Flow-based load balancing, which 253 is supported by many commercial routers. 255 The ingress node, (Ra/Rb) uses Flow ID (in IPv6 header) 256 or UDP/TCP port number combined with the source address 257 to enforce packets in one flow being placed in one tunnel 258 to one Egress router. No new features are needed. 260 c) When a UE moves to a new Cell Tower, a method is needed 261 to stick the flow to the same ANYCAST server, which is 262 required by 5G Edge Computing: 3GPP TR 23.748. 264 Internet-Draft AppMetaData for 5G EC Service 266 This problem is Out of scope for this draft. [5g-edge- 267 compute-sticky-service] describes several approaches to 268 solve this problem. 270 From BGP perspective, the multiple servers with the same IP 271 address (ANYCAST)attached to different egress routers is the 272 same as multiple next hops for the IP address. 274 This draft describes the BGP UPDATE to enable ingress routers 275 to take the App Server load, the capacity index, and the 276 location preference into consideration when computing the 277 optimal path to egress routers. 279 1.4. Problem #2: Unbalanced Anycast Distribution due to UE 280 Mobility 282 UEs frequent moving from one 5G site to another can make it 283 difficult to plan where and how many to deploy the App 284 servers. When one App server is heavily utilized, other 285 servers of the same App close-by can be very underutilized. 286 Since the condition can be short-lived, it is difficult for 287 the application controller to anticipate the move and adjust. 289 1.5. Problem 3: Application Server Relocation 291 When an Application Server is added to, moved, or deleted from 292 a 5G EC Data Center, the routing protocol needs to propagate 293 the changes to 5G PSA or the PSA adjacent routers. After the 294 change, the cost associated with the site might change as 295 well. 297 Note: for ease of description, the Edge Application Server and 298 Application Server are used interchangeably throughout this 299 document. 301 2. Conventions used in this document 303 A-ER: Egress Router to an Application Server, [A-ER] is 304 used to describe the last router that the 305 Application Server is attached. For a 5G EC 307 Internet-Draft AppMetaData for 5G EC Service 309 environment, the A-ER can be the gateway router to 310 a (mini) Edge Computing Data Center. 312 Application Server: An application server is a physical or 313 virtual server that hosts the software system for 314 the application. 316 Application Server Location: Represent a cluster of servers at 317 one location serving the same Application. One 318 application may have a Layer 7 Load balancer, 319 whose address(es) are reachable from an external 320 IP network, in front of a set of application 321 servers. From an IP network perspective, this 322 whole group of servers is considered as the 323 Application server at the location. 325 Edge Application Server: used interchangeably with Application 326 Server throughout this document. 328 EC: Edge Computing 330 Edge Hosting Environment: An environment providing the support 331 required for Edge Application Server's execution. 333 NOTE: The above terminologies are the same as 334 those used in 3GPP TR 23.758 336 Edge DC: Edge Data Center, which provides the Edge 337 Computing Hosting Environment. An Edge DC might 338 host 5G core functions in addition to the 339 frequently used application servers. 341 gNB next generation Node B 343 L-DN: Local Data Network 345 PSA: PDU Session Anchor (UPF) 347 SSC: Session and Service Continuity 349 UE: User Equipment 351 Internet-Draft AppMetaData for 5G EC Service 353 UPF: User Plane Function 355 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 356 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT 357 RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be 358 interpreted as described in BCP 14 [RFC2119] [RFC8174] when, 359 and only when, they appear in all capitals, as shown here. 361 3. Usage of App-Meta-Data for 5G Edge Computing 363 3.1. Assumptions 365 From IP Layer, the Application servers are identified by their 366 IP (ANYCAST) addresses. Here are some assumptions about the 5G 367 EC services: 368 - Only the registered EC services, which are only a small 369 portion of the services, need to include the AppMetadata 370 in path selection. 371 - The 5G EC controller or management system push down the 372 policies (e.g., ACLs) on the relevant routers to filter 373 out those registered EC services. 374 - The ingress routers' local BGP path compute algorithm 375 includes a special plugin that can compute the path to 376 the optimal Next Hop (egress router) based on the BGP 377 AppMetaData TLV received for the registered EC services. 379 The proposed solution is for the egress routers, i.e. A-ER, 380 that have direct links to the Application Servers to collect 381 various measurements about the Servers' running status [5G-EC- 382 Metrics] and advertise the metrics to other routers in 5G EC 383 LDN (Local Data Network). 385 3.2. IP Layer Metrics to Gauge Application Behavior 387 [5G-EC-Metrics] describes the IP Layer Metrics that can gauge 388 the application servers running status and environment: 390 - IP-Layer Metric for App Server Load Measurement: 391 The Load Measurement to an App Server is a weighted 392 combination of the number of packets/bytes to the App Server 393 and the number of packets/bytes from the App Server which 395 Internet-Draft AppMetaData for 5G EC Service 397 are collected by the A-ER to which the App Server is 398 directly attached. 399 The A-ER is configured with an ACL that can filter out the 400 packets for the Application Server. 401 - Capacity Index 402 a numeric number, configured on all A-ERs in the domain 403 consistently, is used to represent the capacity of the 404 application server attached to an A-ER. At some sites, the 405 IP address exposed to the A-ER is the App Layer Load 406 balancer that have many instances attached. At other sites, 407 the IP address exposed is the server instance itself. 408 - Site preference index: 409 is used to describe some sites are more preferred than 410 others. For example, a site with higher bandwidth has a 411 higher preference number than other. 413 In this document, the term "Application Server Egress Router" 414 [A-ER] is used to describe the last router that an Application 415 Server is attached. For the 5G EC environment, the A-ER can be 416 the gateway router to the EC DC where multiple Application 417 servers are hosted. 419 From IP Layer, an Application Server is identified by its IP 420 (ANYCAST) Address. Those IP addresses are called the 421 Application Server IDs throughout this document. 423 3.3. AppMetaData Constrained Optimal Path Selection 425 The main benefit of using ANYCAST is to leverage the network 426 layer information to select an optimal path among multiple 427 application Server locations of the same application 428 identified by its ANYCAST addresses. 430 Internet-Draft AppMetaData for 5G EC Service 432 For the 5G EC environment, the ingress routers to the LDN need 433 to be notified of the Load Index and Capacity Index of the App 434 Servers at different EC data centers to make the intelligent 435 decision on where to forward the traffic for the application 436 from UEs. 438 Here is an algorithm that computes the cost to reach the App 439 Servers attached to Site-i relative to another site, say Site- 440 b. When the reference site, Site-b, is plugged in the formula, 441 the cost is 1. So, if the formula returns a value less than 1, 442 the cost to reach Site-i is less than reaching Site-b. 444 CP-b * Load-i Pref-b * Network-Delay-i 445 Cost-i= (w *(----------------) + (1-w) *(-------------------------)) 446 CP-i * Load-b Pref-i * Network-Delay-b 448 Load-i: Load Index at Site-i, it is the weighted 449 combination of the total packets or/and bytes sent to and 450 received from the Application Server at Site-i during a 451 fixed time period. 453 CP-i: capacity index at Site-i, a higher value means higher 454 capacity. 456 Delay-i: Network latency measurement (RTT) to the A-ER that 457 has the Application Server attached at the site-i. 459 Pref-i: Preference index for the Site-i, a higher value 460 means higher preference. 462 w: Weight for load and site information, which is a value 463 between 0 and 1. If smaller than 0.5, Network latency and 464 the site Preference have more influence; otherwise, Server 465 load and its capacity have more influence. 467 3.4. BGP Protocol Extension to advertise Load & Capacity 469 The goal of the protocol extension: 470 - Propagate the Load Measurement Index for the attached App 471 Servers to other routers in the LDN. 473 - Propagate the Capacity Index & 475 Internet-Draft AppMetaData for 5G EC Service 477 - Propagate Site Preference Index. 479 The BGP extension is to include the Load Index Sub-TLV, 480 Capacity Sub-TLV, and the Site Preference Sub-TLV in the 481 Tunnel Encap Path Attribute associated with the routes. 483 3.5. Ingress Node BGP Path Selection Behavior 485 3.5.1. AppMetaData Influenced BGP Path Selection 487 In this scenario, an ingress router will receive one ANYCAST 488 address's multiple routes from different egress routers that 489 have the direct links to the ANYCAST servers. The ingress 490 router's BGP engine will do path selection, select the best 491 route, and download to FIB. And BGP engine will also download 492 the other paths to FIB that with the AppMetaData taken into 493 the consideration. 495 Assume that both Ra and Rb in Figure-1 have BGP Multipath 496 enabled. As a result, Dst Address: S1:aa08::4450 is resolved 497 via multiple NextHop: R1, R2, R3. 499 Suppose the local BGP special Plugin for AppMetaData finds R1 500 is the best for the flow towards S1:aa08::4450. Then this 501 special Plugin can insert a higher weight for the path R1 so 502 that BGP Best Path is locally influenced by the weight 503 parameter based on the local decision. 505 3.5.2. Forwarding Behavior 507 When the ingress router receives a packet and lookup the FIB, 508 it gets the destination prefix's whole path and AppMetaData. 509 The Forwarding Plane will do computing for the packet and 510 choose the suitable path as the result of the computing. Then 511 the Forwarding Plane encapsulates the packet destined towards 512 the optimal egress node. 514 For subsequent packets belonging to the same flow, the ingress 515 router needs to forward them to the same egress router unless 516 the selected egress router is no longer reachable. Keeping 517 packets from one flow to the same egress router, a.k.a. Flow 518 Affinity, is supported by many commercial routers. 520 How Flow Affinity is implemented is out of the scope for this 521 document. Here is one example to illustrate how Flow Affinity 522 can be achieved. This illustration is not to be standardized. 524 Internet-Draft AppMetaData for 5G EC Service 526 For the registered EC services, the ingress node keeps a 527 table of 528 - Service ID (i.e., ANYCAST address) 529 - Flow-ID 530 - Sticky Egress ID (egress router loopback address) 531 - A timer 533 The Flow-ID in this table is to identify a flow, initialized 534 to NULL. How Flow-ID is constructed is out of the scope for 535 this document. Here is one example of constructing the Flow- 536 ID: 537 - For IPv6, the Flow-ID can be the Flow-ID extracted from 538 the IPv6 packet header with or without the source 539 address. 540 - For IPv4, the Flow-ID can be the combination of the 541 Source Address with or without the TCP/UDP Port number. 543 The Sticky Egress ID is the egress node address for the same 544 flow. [5G-Sticky-Service] describes several methods to 545 derive the Sticky Egress ID. 547 The Timer is always refreshed when a packet with the 548 matching EC Service ID (ANYCAST address) is received by the 549 node. 551 If there is no Stick Egress ID present in the table for the 552 EC Service ID, the forwarding plane computes the optimal 553 path to an egress (NextHop) with the AppMetaData taken into 554 consideration. The forwarding plane encapsulates the packet 555 with a tunnel to the chosen egress (NextHop). The chosen 556 NextHop and the Flow ID are recorded in the table entry of 557 the EC Service ID. 559 When the selected optimal egress router is no longer 560 reachable, refer to Section 6 Soft Anchoring on how another 561 path is selected. 563 3.5.3. Forwarding Behavior after a UE moving to a new 5G Site 565 When a UE moves to a new 5G Site, the new ingress router might 566 use the pre-computed Egress Router which is passed from the 567 neighboring router. [5G-Edge-Sticky] describes the method for 568 the ingress router connected to the UPF in the new site to 569 take into consideration the information passed from other 570 ingress routers in selecting the optimal paths. The detailed 571 algorithm is out of the scope of this document. 573 Internet-Draft AppMetaData for 5G EC Service 575 4. The Sub-TLVs for App-Meta-Data 577 The App-Meta-Data attribute is encoded in an optional subTLV 578 within the Tunnel Encap [RFC9012] Path Attribute. 580 4.1. Load Measurement sub-TLV format 582 Two types of Load Measurement Sub-TLVs are specified. One is 583 to carry the aggregated cost Index based on a weighted 584 combination of the collected measurements; another one is to 585 carry the raw measurements of packets/bytes to/from the App 586 Server address. The raw measurement is useful when the egress 587 routers cannot be configured with a consistent algorithm to 588 compute the aggregated load index and the raw measurements are 589 needed by a central analytic system. 591 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 592 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 593 | Type (TBD1) | Length | 594 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 595 | Measurement Period | 596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 597 | Aggregated Load Index to reach the App Server | 598 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 599 Figure 2: Aggregated Load Index Sub-TLV 601 Raw Load Measurement sub-TLV has the following format: 603 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 604 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 605 | Type (TBD2) | Length | 606 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 607 | Measurement Period | 608 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 609 | total number of packets to the AppServer | 610 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 611 | total number of packets from the AppServer | 612 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 613 | total number of bytes to the AppServer | 614 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 615 | total number of bytes from the AppServer | 616 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 617 Figure 5: Raw Load Measurement Sub-TLV 619 Internet-Draft AppMetaData for 5G EC Service 621 Type =TBD1: Aggregated Load Measurement Index derived from 622 the Weighted combination of bytes/packets sent to/received 623 from the App server: 625 Index=w1*ToPackets+w2*FromPackes+w3*ToBytes+w4*FromBytes 627 Where wi is a value between 0 and 1; w1+ w2+ w3+ w4 = 1; 629 Type= TBD2: Raw measurements of packets/bytes to/from the 630 App Server address; 632 Measure Period: BGP Update period or user-specified period. 634 4.2. Capacity Index sub-TLV format 636 The Capacity Index sub-TLV has the following format: 638 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 639 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 640 | Type (TBD3) | Length | 641 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 642 | Capacity Index | 643 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 645 Note: "Capacity Index" can be more stable for each site. If 646 those values are configured to nodes, they might not need to 647 be included in every BGP UPDATE. 649 4.3. The Site Preference Index sub-TLV format 651 The site Preference Index is used to achieve Soft Anchoring 652 [Section 5] an application flow from a UE to a specific 653 location when the UE moves from one 5G site to another. 655 The Preference Index sub-TLV has the following format: 657 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 658 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 659 | Type (TBD4) | Length | 660 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 661 | Preference Index | 662 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 664 Internet-Draft AppMetaData for 5G EC Service 666 Note: "Site Preference Index" can be more stable for each 667 site. If those values are configured to nodes, they might not 668 need to be included in every BGP UPDATE. 670 5. AppMetaData Propagation Scope 672 AppMetaData is only to be distributed to the relevant ingress 673 nodes of the 5G EC local data networks. Only the ingress 674 routers that are configured with the 5G EC services ACLs need 675 to receive the AppMetaData for specific services. 677 For each registered EC service, a corresponding filter group 678 can be formed on RR to represent the interested ingress 679 routers that are interested in receiving the corresponding 680 AppMetaData information. 682 6. Soft Anchoring of an ANYCAST Flow 683 "Sticky Service" in the 3GPP Edge Computing specification 684 (3GPP TR 23.748) requires a UE to a specific ANYCAST location 685 when the UE moves from one 5G Site to another. 687 "Soft Anchoring" is referring to forwarding the Application 688 flow from a UE to a preferred location of the ANYCAST servers 689 when the preferred location is in good condition. But if 690 there is any failure reaching the preferred location, the 691 Application flow from the UE will be forwarded to another 692 location of the ANYCAST servers. 694 This section describes a solution that can softly anchor an 695 application flow from a UE to a preferred location. 697 Lets assume one application "App.net" is instantiated on four 698 servers that are attached to four different routers R1, R2, 699 R3, and R4 respectively. It is desired for packets to the 700 "App.net" from UE-1 to stick with one server, say the App 701 Server attached to R1, even when the UE moves from one 5G 702 site to another. When there is a failure reaching R1 or the 703 Application Server attached to R1, the packets of the flow 704 "App.net" from UE-1 need to be forwarded to the Application 705 Server attached to R2, R3, or R4. 707 We call this kind of sticky service "Soft Anchoring", meaning 708 that anchoring to the site of R1 is preferred, but other 709 sites can be chosen when the preferred site encounters a 710 failure. 712 Internet-Draft AppMetaData for 5G EC Service 714 Here are the details of this solution: 716 - Assign a group of ANYCAST addresses to one application. 717 For example, "App.net" is assigned with 4 ANYCAST 718 addresses, L1, L2, L3, and L4. L1/L2/L3/L4 represents 719 the location preferred ANYCAST addresses. 720 - For the App.net Server attached to a router, the router 721 has four Stub links to the same Server, L1, L2, L3, and 722 L4 respectively. The cost to L1, L2, L3, and L4 is 723 assigned differently for different routers. For example, 724 o When attached to R1, the L1 has the lowest cost, 725 say 10, when attached to R2, R3, and R4, the L1 can 726 have a higher cost, say 30. 727 o ANYCAST L2 has the lowest cost when attached to R2, 728 higher cost when attached to R1, R3, R4 729 respectively. 730 o ANYCAST L3 has the lowest cost when attached to R3, 731 higher cost when attached to R1, R2, R4 732 respectively, and 733 o ANYCAST L4 has the lowest cost when attached to R4, 734 higher cost when attached to R1, R2, R3 735 respectively 736 - When a UE queries for the "App.net" for the first time, 737 the DNS reply has the location preferred ANYCAST 738 address, say L1, based on where the query is initiated. 739 - When the UE moves from one 5G site-A to Site-B, UE 740 continues sending packets of the "App.net" to ANYCAST 741 address L1. The routers will continue sending packets to 742 R1 because the total cost for the App.net instance for 743 ANYCAST L1 is lowest at R1. If any failure occurs making 744 R1 not reachable, the packets of the "App.net" from UE-1 745 will be sent to R2, R3, or R4 (depending on the total 746 cost to reach each of them). 748 If the Application Server supports the HTTP redirect, more 749 optimal forwarding can be achieved. 751 Internet-Draft AppMetaData for 5G EC Service 753 - When a UE queries for the "App.net" for the first time, 754 the global DNS reply has the ANYCAST address G1, which 755 has the same cost regardless of where the Application 756 servers are attached. 757 - When the UE initiates the communication to G1, the 758 packets from the UE will be sent to the Application 759 Server that has the lowest cost, say the Server attached 760 to R1. The Application server is instructed with HTTPs 761 Redirect to reply with a location-specific URL, say 762 App.net-Loc1. The client on the UE will query the DNS 763 for App.net-Loc1 and get the response of ANYCAST L1. The 764 subsequent packets from the UE-1 for App.net are sent to 765 L1. 767 7. Manageability Considerations 769 To be added. 771 8. Security Considerations 773 To be added. 775 9. IANA Considerations 777 To be added. 779 10. References 781 10.1. Normative References 783 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 784 Requirement Levels", BCP 14, RFC 2119, March 1997. 786 [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private 787 networks (VPNs)", Feb 2006. 789 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in 790 RFC 2119 Key Words", BCP 14, RFC 8174, DOI 791 10.17487/RFC8174, May 2017, . 794 Internet-Draft AppMetaData for 5G EC Service 796 [RFC8200] s. Deering R. Hinden, "Internet Protocol, Version 6 797 (IPv6) Specification", July 2017 799 10.2. Informative References 801 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 802 Partnership Project; Technical Specification Group 803 Services and System Aspects; Study on enhancement of 804 support for Edge Computing in 5G Core network 805 (5GC)", Release 17 work in progress, Aug 2020. 807 [5G-EC-Metrics] L. Dunbar, H. Song, J. Kaippallimalil, "IP 808 Layer Metrics for 5G Edge Computing Service", draft- 809 dunbar-ippm-5g-edge-compute-ip-layer-metrics-00, 810 work-in-progress, Oct 2020. 812 [5G-Edge-Sticky] L. Dunbar, J. Kaippallimalil, "IPv6 Solution 813 for 5G Edge Computing Sticky Service", draft-dunbar- 814 6man-5g-ec-sticky-service-00, work-in-progress, Oct 815 2020. 817 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 818 Subsequent Address Family Identifier (SAFI) and the 819 BGP Tunnel Encapsulation Attribute", April 2009. 821 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension 822 for SDWAN Overlay Networks", draft-dunbar-idr-bgp- 823 sdwan-overlay-ext-03, work-in-progress, Nov 2018. 825 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 826 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 827 draft-dunbar-idr-sdwan-edge-discovery-00, work-in- 828 progress, July 2020. 830 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 831 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 832 2018. 834 Internet-Draft AppMetaData for 5G EC Service 836 11. Acknowledgments 838 Acknowledgements to Donald Eastlake for their review and 839 contributions. 841 This document was prepared using 2-Word-v2.0.template.dot. 843 Authors' Addresses 845 Linda Dunbar 846 Futurewei 847 Email: ldunbar@futurewei.com 849 Kausik Majumdar 850 CommScope 851 350 W Java Drive, Sunnyvale, CA 94089 852 Email: kausik.majumdar@commscope.com 854 Haibo Wang 855 Huawei 856 Email: rainsword.wang@huawei.com