idnits 2.17.1 draft-dunbar-idr-5g-edge-compute-app-meta-data-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (March 8, 2021) is 1146 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: '5GC' is mentioned on line 137, but not defined == Missing Reference: 'A-ER' is mentioned on line 420, but not defined == Missing Reference: 'IPv6-StickyService' is mentioned on line 415, but not defined == Missing Reference: '5G-Sticky-Service' is mentioned on line 550, but not defined == Missing Reference: 'Section 5' is mentioned on line 719, but not defined == Unused Reference: 'RFC4364' is defined on line 855, but no explicit reference was found in the text == Unused Reference: 'RFC8200' is defined on line 865, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 870, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 886, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 890, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 894, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 899, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 4364 == Outdated reference: A later version (-02) exists of draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-00 -- No information found for draft-dunbar-6man-5g-ec-sticky-service - is the name correct? == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 1 error (**), 0 flaws (~~), 17 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft Futurewei 3 Intended status: Standard K. Majumdar 4 Expires: September 8, 2021 CommScope 5 H. Wang 6 Huawei 7 March 8, 2021 9 BGP NLRI App Meta Data for 5G Edge Computing Service 10 draft-dunbar-idr-5g-edge-compute-app-meta-data-02 12 Abstract 13 This draft describes a new BGP Network Layer Reachability 14 Information (BGP NLRI) Path Attribute, AppMetaData, for egress 15 router to advertise the running status and environment of the 16 directly attached 5G Edge Computing servers. The AppMetaData 17 can be used by the ingress routers in the 5G Local Data 18 Network to make intelligent path selection for flows from UEs. 19 The goal is to improve latency and performance for 5G Edge 20 Computing services. 22 The extension enables a feature, called soft anchoring, which 23 makes one Edge Computing Server at one specific location to be 24 more preferred than others for the same application to receive 25 packets from a specific source (UE). 27 Status of this Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. This document may not be 34 modified, and derivative works of it may not be created, 35 except to publish it as an RFC and to translate it into 36 languages other than English. 38 Internet-Drafts are working documents of the Internet 39 Engineering Task Force (IETF), its areas, and its working 40 groups. Note that other groups may also distribute working 41 documents as Internet-Drafts. 43 Internet-Draft AppMetaData NLRI for 5G EC Service 45 Internet-Drafts are draft documents valid for a maximum of six 46 months and may be updated, replaced, or obsoleted by other 47 documents at any time. It is inappropriate to use Internet- 48 Drafts as reference material or to cite them other than as 49 "work in progress." 51 The list of current Internet-Drafts can be accessed at 52 http://www.ietf.org/ietf/1id-abstracts.txt 54 The list of Internet-Draft Shadow Directories can be accessed 55 at http://www.ietf.org/shadow.html 57 This Internet-Draft will expire on April 7, 2021. 59 Copyright Notice 61 Copyright (c) 2021 IETF Trust and the persons identified as 62 the document authors. All rights reserved. 64 This document is subject to BCP 78 and the IETF Trust's Legal 65 Provisions Relating to IETF Documents 66 (http://trustee.ietf.org/license-info) in effect on the date 67 of publication of this document. Please review these documents 68 carefully, as they describe your rights and restrictions with 69 respect to this document. Code Components extracted from this 70 document must include Simplified BSD License text as described 71 in Section 4.e of the Trust Legal Provisions and are provided 72 without warranty as described in the Simplified BSD License. 74 Table of Contents 76 1. Introduction.............................................. 3 77 1.1. 5G Edge Computing Background......................... 3 78 1.2. 5G Edge Computing Network Properties................. 4 79 1.3. Problem#1: ANYCAST in 5G EC Environment.............. 6 80 1.4. Problem #2: Unbalanced Anycast Distribution due to UE 81 Mobility.................................................. 7 82 1.5. Problem 3: Application Server Relocation............. 7 83 2. Conventions used in this document......................... 8 84 3. Usage of App-Meta-Data for 5G Edge Computing.............. 9 86 Internet-Draft AppMetaData NLRI for 5G EC Service 88 3.1. Assumptions.......................................... 9 89 3.2. IP Layer Metrics to Gauge Application Behavior....... 9 90 3.3. To Equalize among Multiple ANYCAST Locations........ 11 91 3.4. BGP Protocol Extension to advertise Load & Capacity. 11 92 3.5. Ingress Node BGP Path Selection Behavior............ 12 93 3.5.1. AppMetaData Influenced BGP Path Selection...... 12 94 3.5.2. Forwarding Behavior............................ 12 95 3.5.3. Forwarding Behavior after a UE moving to a new 5G 96 Site.................................................. 13 97 4. The NLRI Path Attribute for App-Meta-Data................ 14 98 4.1. Load Measurement sub-TLV format..................... 16 99 4.2. Capacity Index sub-TLV format....................... 17 100 4.3. The Site Preference Index sub-TLV format............ 17 101 5. AppMetaData Propagation Scope............................ 18 102 6. Soft Anchoring of an ANYCAST Flow........................ 18 103 7. Manageability Considerations............................. 20 104 8. Security Considerations.................................. 20 105 9. IANA Considerations...................................... 20 106 10. References.............................................. 20 107 10.1. Normative References............................... 20 108 10.2. Informative References............................. 21 109 11. Acknowledgments......................................... 22 111 1. Introduction 113 This document describes a new BGP Network Layer Reachability 114 Information (BGP NLRI) Path Attribute, AppMetaData, for egress 115 routers to advertise the running status and environment of the 116 directly attached Edge Computing servers. The AppMetaData can 117 be used by the ingress routers in the 5G Local Data Network to 118 make intelligent path selection for flows from UEs. The goal 119 is to improve latency and performance for 5G Edge Computing 120 services. 122 1.1. 5G Edge Computing Background 124 As described in [5G-EC-Metrics], one Application can have 125 multiple Application Servers hosted in different Edge 126 Computing data centers that are close in proximity. Those Edge 127 Computing (mini) data centers are usually very close to or co- 128 located with the 5G base stations, to minimize latency and 129 optimize the user experience. 131 When a UE (User Equipment) initiates application packets using 132 the destination address from a DNS reply or its cache, the 134 Internet-Draft AppMetaData NLRI for 5G EC Service 136 packets from the UE are carried in a PDU session through 5G 137 Core [5GC] to the 5G UPF-PSA (User Plan Function - PDU Session 138 Anchor). The UPF-PSA decapsulates the 5G GTP outer header and 139 forwards the packets from the UEs to the Ingress router of the 140 Edge Computing (EC) Local Data Network (LDN). The LDN for 5G 141 EC, which is the IP Networks from the 5GC perspective, is 142 responsible for forwarding the packets to the intended 143 destinations. 145 When the UE moves out of coverage of its current gNB (next- 146 generation Node B) (gNB1), handover procedures are initiated 147 and the 5G SMF (Session Management Function) also selects a 148 new UPF-PSA. The standard handover procedures described in 149 3GPP TS 23.501 and TS 23.502 are followed. When the handover 150 process is complete, the UE has a new IP address and the IP 151 point of attachment is to the new UPF-PSA. 5GC may maintain a 152 path from the old UPF to new the UPF for a short time for the 153 SSC [Session and Service Continuity] mode 3 to make the 154 handover process more seamless. 156 1.2. 5G Edge Computing Network Properties 158 In this document, 5G Edge Computing Network refers to multiple 159 Local IP Data Networks (LDN) in one region that interconnect 160 the Edge Computing mini-data centers. Those IP LDN networks 161 are the N6 interfaces from 3GPP 5G perspective. 163 The ingress routers to the 5G Edge Computing Network are the 164 routers directly connected to 5G UPFs. The egress routers to 165 the 5G Edge Computing Network are the routers that have a 166 direct link to the Edge Computing servers. The servers and the 167 egress routers are co-located. Some of those mini Edge 168 Computing Data centers may have Virtual switches or Top of 169 Rack switches between the egress routers and the servers. But 170 transmission delay between the egress routers and the Edge 171 Computing servers is too small to be considered in this 172 document. 174 When one mini data center has multiple Edge Computing Servers 175 attached to one App Layer Load Balancer, only the App Layer 176 Load Balancer is visible to the 5G Edge Computing Network. How 178 Internet-Draft AppMetaData NLRI for 5G EC Service 180 the App Layer Load balancer manages the individual servers is 181 out of the scope of the network layer. 183 The Edge Computer Services are specially managed services that 184 need to utilize the network topology and balance among 185 multiple mini Edge Computing Data Centers with the same 186 ANYCAST address. UEs can access many services that are not 187 part of the registered 5G Edge Computing Services. 189 +--+ 190 |UE|---\+---------+ +------------------+ 191 +--+ | 5G | +--------+ | S1: aa08::4450 | 192 +--+ | Site +--+-+---+ +----+ | 193 |UE|----| A |PSA1| Ra| | R1 | S2: aa08::4460 | 194 +--+ | +----+---+ +----+ | 195 +---+ | | | | | S3: aa08::4470 | 196 |UE1|---/+---------+ | | +------------------+ 197 +---+ |IP Network | L-DN1 198 |(3GPP N6) | 199 | | | +------------------+ 200 | UE1 | | | S1: aa08::4450 | 201 | moves to | +----+ | 202 | Site B | | R3 | S2: aa08::4460 | 203 v | +----+ | 204 | | | S3: aa08::4470 | 205 | | +------------------+ 206 | | L-DN3 207 +--+ | | 208 |UE|---\+---------+ | | +------------------+ 209 +--+ | 5G | | | | S1: aa08::4450 | 210 +--+ | Site +--+--+---+ +----+ | 211 |UE|----| B |PSA2| Rb | | R2 | S2: aa08::4460 | 212 +--+ | +--+-+----+ +----+ | 213 +--+ | | +-----------+ | S3: aa08::4470 | 214 |UE|---/+---------+ +------------------+ 215 +--+ L-DN2 216 Figure 1: App Servers in different edge DCs 218 Internet-Draft AppMetaData NLRI for 5G EC Service 220 1.3. Problem#1: ANYCAST in 5G EC Environment 222 Increasingly, Anycast is used extensively by various 223 application providers and CDNs because ANYCAST makes it 224 possible to dynamically load balance across server locations 225 based on network conditions. 227 Using Anycast address leverages the proximity information 228 present in the network (routing) layer and eliminates the 229 single point of failure and bottleneck at the DNS resolvers 230 and application layer load balancers. Another benefit of using 231 the ANYCAST address is removing the dependency on UEs. Some 232 UEs (or clients) might use their cached IP addresses instead 233 of querying DNS for an extended period. 235 But, having multiple locations of the same ANYCAST address in 236 the 5G Edge Computing environment can be problematic because 237 all those edge computing Data Centers can be close in 238 proximity. There might be a very small difference in the 239 routing cost to reach the Application Servers in different 240 Edge DCs. This list elaborates the issues in detail: 242 a) Path Selection: When a new flow comes to an ingress node 243 (Ra), how to select the optimal egress router to reach an 244 ANYCAST server. 246 The mechanism described in this draft is for solving this 247 Path Selection problem. 249 b) How Ingress node keeps the packets from one flow to the 250 same ANYCAST server. 252 a.k.a. Flow Affinity, or Flow-based load balancing, which 253 is supported by many commercial routers. 255 The ingress node, (Ra/Rb) uses Flow ID (in IPv6 header) 256 or UDP/TCP port number combined with the source address 257 to enforce packets in one flow being placed in one tunnel 258 to one Egress router. No new features are needed. 260 c) When a UE moves to a new Cell Tower, a method is needed 261 to stick the flow to the same ANYCAST server, which is 262 required by 5G Edge Computing: 3GPP TR 23.748. 264 Internet-Draft AppMetaData NLRI for 5G EC Service 266 This problem is Out of scope for this draft. [5g-edge- 267 compute-sticky-service] describes several approaches to 268 solve this problem. 270 BGP is an integral part of the way IP Anycast usually 271 functions. Within BGP routing there are multiple routes for 272 the same IP address which are pointing to different locations. 274 This draft describes the BGP UPDATE extension to allow the App 275 Servers Running status and environment to be included in the 276 BGP UPDATE messages, so that ingress routers can optimize its 277 path selection algorithm to select an optimal ANYCAST location 278 based on the combination of network delay, the App Server load 279 index, the location capacity index and the location 280 preference. 282 1.4. Problem #2: Unbalanced Anycast Distribution due to UE 283 Mobility 285 UEs frequent moving from one 5G site to another can make it 286 difficult to plan where the App ANYCAST servers should be 287 hosted. When one App server is heavily utilized, other App 288 servers of the same address close-by can be very 289 underutilized. Since the condition can be short-lived, it is 290 difficult for the application controller to anticipate the 291 move and adjust. 293 1.5. Problem 3: Application Server Relocation 295 When an Application Server is added to, moved, or deleted from 296 a 5G Edge Computing Data Center, the routing protocol needs to 297 propagate the changes to 5G PSA or the PSA adjacent routers. 298 After the change, the cost associated with the site [5G-EC- 299 Metrics] might change as well. 301 Note: for ease of description, the Edge Application Server and 302 Application Server are used interchangeably throughout this 303 document. 305 Internet-Draft AppMetaData NLRI for 5G EC Service 307 2. Conventions used in this document 309 A-ER: Egress Router to an Application Server, [A-ER] is 310 used to describe the last router that the 311 Application Server is attached. For a 5G EC 312 environment, the A-ER can be the gateway router to 313 a (mini) Edge Computing Data Center. 315 Application Server: An application server is a physical or 316 virtual server that hosts the software system for 317 the application. 319 Application Server Location: Represent a cluster of servers at 320 one location serving the same Application. One 321 application may have a Layer 7 Load balancer, 322 whose address(es) are reachable from an external 323 IP network, in front of a set of application 324 servers. From an IP network perspective, this 325 whole group of servers is considered as the 326 Application server at the location. 328 Edge Application Server: used interchangeably with Application 329 Server throughout this document. 331 EC: Edge Computing 333 Edge Hosting Environment: An environment providing the support 334 required for Edge Application Server's execution. 336 NOTE: The above terminologies are the same as 337 those used in 3GPP TR 23.758 339 Edge DC: Edge Data Center, which provides the Edge 340 Computing Hosting Environment. An Edge DC might 341 host 5G core functions in addition to the 342 frequently used application servers. 344 gNB next generation Node B 346 L-DN: Local Data Network 348 Internet-Draft AppMetaData NLRI for 5G EC Service 350 PSA: PDU Session Anchor (UPF) 352 SSC: Session and Service Continuity 354 UE: User Equipment 356 UPF: User Plane Function 358 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 359 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT 360 RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be 361 interpreted as described in BCP 14 [RFC2119] [RFC8174] when, 362 and only when, they appear in all capitals, as shown here. 364 3. Usage of App-Meta-Data for 5G Edge Computing 366 3.1. Assumptions 368 From IP Layer, the Application servers are identified by their 369 IP (ANYCAST) addresses. Here are some assumptions about the 5G 370 Edge Computing services: 371 - Only the registered Edge Computing services need special 372 consideration in path selection. 373 - The 5G Edge Computing controller or management system can 374 configure the ACLs to filter out those applications on 375 the routers adjacent to the 5G PSA and the routers to 376 which the Application servers are directly attached. 377 - The ingress routers' local BGP path compute algorithm 378 includes a special Plugin that can compute the path to 379 the optimal Next Hop (egress router) based on the BGP 380 AppMetaData TLV received for the registered Edge 381 Computing services. 383 The proposed solution is for the egress routers, i.e. A-ER, 384 that have direct links to the Application Servers to collect 385 various measurements about the Servers' running status [5G-EC- 386 Metrics] and advertise the metrics to other routers in 5G EC 387 LDN (Local Data Network). 389 3.2. IP Layer Metrics to Gauge Application Behavior 391 Internet-Draft AppMetaData NLRI for 5G EC Service 393 [5G-EC-Metrics] describes the IP Layer Metrics that can gauge 394 the application servers running status and environment: 396 - IP-Layer Metric for App Server Load Measurement: 397 The Load Measurement to an App Server is a weighted 398 combination of the number of packets/bytes to the App Server 399 and the number of packets/bytes from the App Server which 400 are collected by the A-ER to which the App Server is 401 directly attached. 402 The A-ER is configured with an ACL that can filter out the 403 packets for the Application Server. 404 - Capacity Index 405 Capacity Index is used to differentiate the running 406 environment of the application server. Some data centers can 407 have hundreds, or thousands, of servers behind an 408 Application Server's App Layer Load Balancer that is 409 reachable from an external world. Other data centers can 410 have a very small number of servers for the application 411 server. "Capacity Index", which is a numeric number, is used 412 to represent the capacity of the application server in a 413 specific location. 414 - Site preference index: 415 [IPv6-StickyService] describes a scenario that some sites 416 are more preferred for handling an application server than 417 others for flows from a specific UE. 419 In this document, the term "Application Server Egress Router" 420 [A-ER] is used to describe the last router that an Application 421 Server is attached. For the 5G EC environment, the A-ER can be 422 the gateway router to the EC DC where multiple Application 423 servers are hosted. 425 From IP Layer, an Application Server is identified by its IP 426 (ANYCAST) Address. Those IP addresses are called the 427 Application Server IDs throughout this document. 429 Internet-Draft AppMetaData NLRI for 5G EC Service 431 3.3. To Equalize among Multiple ANYCAST Locations 433 The main benefit of using ANYCAST is to leverage the network 434 layer information to equalize the traffic among multiple 435 Application Server locations of the same Application, which is 436 identified by its ANYCAST addresses. 438 For the 5G Edge Computing environment, the ingress routers to 439 the LDN need to be notified of the Load Index and Capacity 440 Index of the App Servers at different EC data centers to make 441 the intelligent decision on where to forward the traffic for 442 the application from UEs. 444 [5G-EC-Metrics] describes the algorithms that can be used by 445 the routers directly attached to the 5G PSA to compare the 446 cost to reach the App Servers between the Site-i or Site-j: 448 Load-i * CP-j Pref-j * Delay-i 449 Cost-i=min(w *(----------------) + (1-w) *(------------------)) 450 Load-j * CP-i Pref-i * Delay-j 452 Load-i: Load Index at Site-i, it is the weighted 453 combination of the total packets or/and bytes sent to and 454 received from the Application Server at Site-i during a 455 fixed time period. 457 CP-i: capacity index at Site-i, a higher value means higher 458 capacity. 460 Delay-i: Network latency measurement (RTT) to the A-ER that 461 has the Application Server attached at the site-i. 463 Pref-i: Preference index for the Site-i, a higher value 464 means higher preference. 466 w: Weight for load and site information, which is a value 467 between 0 and 1. If smaller than 0.5, Network latency and 468 the site Preference have more influence; otherwise, Server 469 load and its capacity have more influence. 471 3.4. BGP Protocol Extension to advertise Load & Capacity 473 The goal of the protocol extension: 475 Internet-Draft AppMetaData NLRI for 5G EC Service 477 - Propagate the Load Measurement Index for the attached App 478 Servers to other routers in the LDN. 480 - Propagate the Capacity Index & 482 - Propagate Site Preference Index. 484 The BGP extension is to add the Load Index Sub-TLV, Capacity 485 Sub-TLV, and the Site Preference Sub-TLV in the NLRI 486 associated with the routes. 488 3.5. Ingress Node BGP Path Selection Behavior 490 3.5.1. AppMetaData Influenced BGP Path Selection 492 In this scenario, an ingress router will receive one ANYCAST 493 address's multiple routes from different egress routers that 494 have the direct links to the ANYCAST servers. The ingress 495 router's BGP engine will do path selection, select the best 496 route, and download to FIB. And BGP engine will also download 497 the other paths to FIB that with the AppMetaData taken into 498 the consideration. 500 Assume that both Ra and Rb in Figure 1 have BGP Multipath 501 enabled. As a result, Dst Address: S1:aa08::4450 is resolved 502 via multiple NextHop: R1, R2, R3. 504 Suppose the local BGP special Plugin for AppMetaData finds R1 505 is the best for the flow towards S1:aa08::4450. Then this 506 special Plugin can insert a higher weight for the path R1 so 507 that BGP Best Path is locally influenced by the weight 508 parameter based on the local decision. 510 3.5.2. Forwarding Behavior 512 When the ingress router receives a packet and lookup the FIB, 513 get the destination prefix's whole path and AppMetaData. The 514 Forwarding Plane will do computing for the packet and choose 515 the suitable path as the result of the computing. Then the 516 Forwarding Plane encapsulates the packet destined towards the 517 optimal Nexthop node. 519 For subsequent packets belonging to the same flow, the ingress 520 router needs to forward them to the same egress router unless 521 the selected egress router is no longer reachable. Keeping 522 packets from one flow to the same egress router, a.k.a. Flow 523 Affinity, is supported by many commercial routers. 525 Internet-Draft AppMetaData NLRI for 5G EC Service 527 How Flow Affinity is implemented is out of the scope for this 528 document. Here is one example to illustrate how Flow Affinity 529 can be achieved. This illustration is not to be standardized. 531 For the registered Edge Computing services, the ingress node 532 keeps a table of 533 - Service ID (i.e. ANYCAST address) 534 - Flow-ID 535 - Sticky Egress ID 536 - A timer 538 The Flow-ID in this table is to identify a flow, initialized 539 to NULL. How Flow-ID is constructed is out of the scope for 540 this document. Here is one example of constructing the Flow- 541 ID: 542 - For IPv6, the Flow-ID can be the Flow-ID extracted from 543 the IPv6 packet header with or without the source 544 address. 545 - For IPv4, the Flow-ID can be the combination of the 546 Source Address with or without the TCP/UDP Port number. 548 The Sticky Egress ID is to record the egress node address 549 that the packets of the same flow that have been forwarded 550 to. [5G-Sticky-Service] describes several methods to derive 551 the Sticky Egress ID. 553 The Timer is always refreshed when a packet with the 554 matching ANYCAST address is received by the node. 556 If there is no Stick Egress ID present in the table for the 557 ANYCAST address, the forwarding plane computes the optimal 558 path to a NextHop with the AppMetaData taken into 559 consideration. The forwarding plane encapsulate the packet 560 with the tunnel to the chosen NextHop. The chosen NextHop 561 and the Flow ID are recorded in the table entry of the 562 ANYCAST ID. 564 When the selected optimal egress router is no longer 565 reachable, refer to Section 6 Soft Anchoring on how another 566 path is selected. 568 3.5.3. Forwarding Behavior after a UE moving to a new 5G Site 570 When a UE moves to a new 5G Site, the new ingress router might 571 use the pre-computed Egress Router which is passed from the 572 neighboring router. [5G-Edge-Sticky] describes the method for 573 the ingress router connected to the UPF in the new site to 575 Internet-Draft AppMetaData NLRI for 5G EC Service 577 take into consideration the information passed from other 578 ingress routers in selecting the optimal egress router. The 579 detailed algorithm is out of the scope of this document. 581 4. The NLRI Path Attribute for App-Meta-Data 583 The App-Meta-Data attribute is an optional transitive BGP Path 584 attribute to carry application-specific data, such as running 585 status, capacity, and site preference. Will need IANA to 586 assign a value as the type code of the attribute. The 587 attribute is composed of a set of Type-Length-Value (TLV) 588 encodings. Each TLV contains information corresponding to 589 metrics to a specific Application Server. An App-Meta-Data 590 TLV is structured as shown in Figure 1: 592 0 1 2 3 593 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 594 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 595 | AppMetaData Type (2 Octets) | Length (2 Octets) | 596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 597 | | 598 | Value | 599 | | 600 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 601 Figure 2: App Meta Data TLV Value Field 603 AppMetaData Type (2 octets): identifies a type of Application 604 related metadata. The field contains values from the IANA 605 Registry "BGP AppMetaData Types". To be added. 607 o Length (2 octets): the total number of octets of the 608 Value field. 610 o Value (variable): comprised of multiple sub-TLVs. 612 Each sub-TLV consists of three fields: a 1-octet type, a 1- 613 octet or 2-octet length field (depending on the type), and 614 zero or more octets of value. A sub-TLV is structured as 615 shown in Figure 2: 617 Internet-Draft AppMetaData NLRI for 5G EC Service 619 +--------------------------------+ 620 | Sub-TLV Type (1 Octet) | 621 +--------------------------------+ 622 | Sub-TLV Length (1 or 2 Octets) | 623 +--------------------------------+ 624 | Sub-TLV Value (Variable) | 625 +--------------------------------+ 627 Figure 3: App Metadata Sub-TLV Value Field 629 o Sub-TLV Type (1 octet): each sub-TLV type defines a 630 certain property about the AppMetaData TLV that contains 631 this sub-TLV. The field contains values from the IANA 632 Registry "BGP AppMetaData Attribute Sub-TLVs". 634 o Sub-TLV Length (1 or 2 octets): the total number of 635 octets of the sub-TLV value field. The Sub-TLV Length field 636 contains 1 octet if the Sub-TLV Type field contains a value 637 in the range from 0-127. The Sub-TLV Length field contains 638 two octets if the Sub-TLV Type field contains a value in the 639 range from 128-255. 641 o Sub-TLV Value (variable): encodings of the value field 642 depend on the sub-TLV type as enumerated above. The 643 following sub-sections define the encoding in detail. 645 Internet-Draft AppMetaData NLRI for 5G EC Service 647 4.1. Load Measurement sub-TLV format 649 Two types of Load Measurement Sub-TLVs are specified. One is 650 to carry the aggregated cost Index based on a weighted 651 combination of the collected measurements; another one is to 652 carry the raw measurements of packets/bytes to/from the App 653 Server address. The raw measurement is useful when the egress 654 routers cannot be configured with a consistent algorithm to 655 compute the aggregated load index and the raw measurements are 656 needed by a central analytic system. 658 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 659 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 660 | Type (TBD2) | Length | 661 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 662 | Measurement Period | 663 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 664 | Aggregated Load Index to reach the App Server | 665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 666 Figure 4: Aggregated Load Index Sub-TLV 668 Raw Load Measurement sub-TLV has the following format: 670 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 672 | Type (TBD3) | Length | 673 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 674 | Measurement Period | 675 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 676 | total number of packets to the AppServer | 677 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 678 | total number of packets from the AppServer | 679 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 680 | total number of bytes to the AppServer | 681 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 682 | total number of bytes from the AppServer | 683 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 684 Figure 5: Raw Load Measurement Sub-TLV 686 Type =TBD2: Aggregated Load Measurement Index derived from 687 the Weighted combination of bytes/packets sent to/received 688 from the App server: 690 Index=w1*ToPackets+w2*FromPackes+w3*ToBytes+w4*FromBytes 692 Where wi is a value between 0 and 1; w1+ w2+ w3+ w4 = 1; 694 Internet-Draft AppMetaData NLRI for 5G EC Service 696 Type= TBD3: Raw measurements of packets/bytes to/from the 697 App Server address; 699 Measure Period: BGP Update period or user-specified period. 701 4.2. Capacity Index sub-TLV format 703 The Capacity Index sub-TLV has the following format: 705 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 706 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 707 | Type (TBD4) | Length | 708 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 709 | Capacity Index | 710 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 712 Note: "Capacity Index" can be more stable for each site. If 713 those values are configured to nodes, they might not need to 714 be included in every BGP UPDATE. 716 4.3. The Site Preference Index sub-TLV format 718 The site Preference Index is used to achieve Soft Anchoring 719 [Section 5] an application flow from a UE to a specific 720 location when the UE moves from one 5G site to another. 722 The Preference Index sub-TLV has the following format: 724 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 725 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 726 | Type (TBD5) | Length | 727 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 728 | Preference Index | 729 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 731 Note: "Site Preference Index" can be more stable for each 732 site. If those values are configured to nodes, they might not 733 need to be included in every BGP UPDATE. 735 Internet-Draft AppMetaData NLRI for 5G EC Service 737 5. AppMetaData Propagation Scope 739 AppMetaData is only to be distributed to the relevant ingress 740 nodes of the 5G Edge Computing local data networks. Only the 741 ingress routers that are configured with the 5G Edge Computing 742 services ACLs need to receive the AppMetaData for specific 743 services. 745 For each registered Edge Computing service, a corresponding 746 filter group can be formed on RR to represent the interested 747 ingress routers that are interested in receiving the 748 corresponding AppMetaData information. 750 6. Soft Anchoring of an ANYCAST Flow 751 "Sticky Service" in the 3GPP Edge Computing specification 752 (3GPP TR 23.748) requires a UE to a specific ANYCAST location 753 when the UE moves from one 5G Site to another. 755 "Soft Anchoring" is referring to forwarding the Application 756 flow from a UE to a preferred location of the ANYCAST servers 757 when the preferred location is in good condition. But if 758 there is any failure reaching the preferred location, the 759 Application flow from the UE will be forwarded to another 760 location of the ANYCAST servers. 762 This section describes a solution that can softly anchor an 763 application flow from a UE to a preferred location. 765 Lets' assume one application "App.net" is instantiated on 766 four servers that are attached to four different routers R1, 767 R2, R3, and R4 respectively. It is desired for packets to the 768 "App.net" from UE-1 to stick with one server, say the App 769 Server attached to R1, even when the UE moves from one 5G 770 site to another. When there is a failure reaching R1 or the 771 Application Server attached to R1, the packets of the flow 772 "App.net" from UE-1 need to be forwarded to the Application 773 Server attached to R2, R3, or R4. 775 We call this kind of sticky service "Soft Anchoring", meaning 776 that anchoring to the site of R1 is preferred, but other 777 sites can be chosen when the preferred site encounters a 778 failure. 780 Here are the details of this solution: 782 Internet-Draft AppMetaData NLRI for 5G EC Service 784 - Assign a group of ANYCAST addresses to one application. 785 For example, "App.net" is assigned with 4 ANYCAST 786 addresses, L1, L2, L3, and L4. L1/L2/L3/L4 represents 787 the location preferred ANYCAST addresses. 788 - For the App.net Server attached to a router, the router 789 has four Stub links to the same Server, L1, L2, L3, and 790 L4 respectively. The cost to L1, L2, L3, and L4 is 791 assigned differently for different routers. For example, 792 o When attached to R1, the L1 has the lowest cost, 793 say 10, when attached to R2, R3, and R4, the L1 can 794 have a higher cost, say 30. 795 o ANYCAST L2 has the lowest cost when attached to R2, 796 higher cost when attached to R1, R3, R4 797 respectively. 798 o ANYCAST L3 has the lowest cost when attached to R3, 799 higher cost when attached to R1, R2, R4 800 respectively, and 801 o ANYCAST L4 has the lowest cost when attached to R4, 802 higher cost when attached to R1, R2, R3 803 respectively 804 - When a UE queries for the "App.net" for the first time, 805 the DNS reply has the location preferred ANYCAST 806 address, say L1, based on where the query is initiated. 807 - When the UE moves from one 5G site-A to Site-B, UE 808 continues sending packets of the "App.net" to ANYCAST 809 address L1. The routers will continue sending packets to 810 R1 because the total cost for the App.net instance for 811 ANYCAST L1 is lowest at R1. If any failure occurs making 812 R1 not reachable, the packets of the "App.net" from UE-1 813 will be sent to R2, R3, or R4 (depending on the total 814 cost to reach each of them). 816 If the Application Server supports the HTTP redirect, more 817 optimal forwarding can be achieved. 819 - When a UE queries for the "App.net" for the first time, 820 the global DNS reply has the ANYCAST address G1, which 822 Internet-Draft AppMetaData NLRI for 5G EC Service 824 has the same cost regardless of where the Application 825 servers are attached. 826 - When the UE initiates the communication to G1, the 827 packets from the UE will be sent to the Application 828 Server that has the lowest cost, say the Server attached 829 to R1. The Application server is instructed with HTTPs 830 Redirect to reply with a location-specific URL, say 831 App.net-Loc1. The client on the UE will query the DNS 832 for App.net-Loc1 and get the response of ANYCAST L1. The 833 subsequent packets from the UE-1 for App.net are sent to 834 L1. 836 7. Manageability Considerations 838 To be added. 840 8. Security Considerations 842 To be added. 844 9. IANA Considerations 846 To be added. 848 10. References 850 10.1. Normative References 852 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 853 Requirement Levels", BCP 14, RFC 2119, March 1997. 855 [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private 856 networks (VPNs)", Feb 2006. 858 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in 859 RFC 2119 Key Words", BCP 14, RFC 8174, DOI 860 10.17487/RFC8174, May 2017, . 863 Internet-Draft AppMetaData NLRI for 5G EC Service 865 [RFC8200] s. Deering R. Hinden, "Internet Protocol, Version 6 866 (IPv6) Specification", July 2017 868 10.2. Informative References 870 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 871 Partnership Project; Technical Specification Group 872 Services and System Aspects; Study on enhancement of 873 support for Edge Computing in 5G Core network 874 (5GC)", Release 17 work in progress, Aug 2020. 876 [5G-EC-Metrics] L. Dunbar, H. Song, J. Kaippallimalil, "IP 877 Layer Metrics for 5G Edge Computing Service", draft- 878 dunbar-ippm-5g-edge-compute-ip-layer-metrics-00, 879 work-in-progress, Oct 2020. 881 [5G-Edge-Sticky] L. Dunbar, J. Kaippallimalil, "IPv6 Solution 882 for 5G Edge Computing Sticky Service", draft-dunbar- 883 6man-5g-ec-sticky-service-00, work-in-progress, Oct 884 2020. 886 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 887 Subsequent Address Family Identifier (SAFI) and the 888 BGP Tunnel Encapsulation Attribute", April 2009. 890 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension 891 for SDWAN Overlay Networks", draft-dunbar-idr-bgp- 892 sdwan-overlay-ext-03, work-in-progress, Nov 2018. 894 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 895 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 896 draft-dunbar-idr-sdwan-edge-discovery-00, work-in- 897 progress, July 2020. 899 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 900 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 901 2018. 903 Internet-Draft AppMetaData NLRI for 5G EC Service 905 11. Acknowledgments 907 Acknowledgements to Donald Eastlake for their review and 908 contributions. 910 This document was prepared using 2-Word-v2.0.template.dot. 912 Authors' Addresses 914 Linda Dunbar 915 Futurewei 916 Email: ldunbar@futurewei.com 918 Kausik Majumdar 919 CommScope 920 350 W Java Drive, Sunnyvale, CA 94089 921 Email: kausik.majumdar@commscope.com 923 Haibo Wang 924 Huawei 925 Email: rainsword.wang@huawei.com