idnits 2.17.1 draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 384 has weird spacing: '... status to ne...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 26, 2020) is 1271 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: '5GC' is mentioned on line 108, but not defined == Missing Reference: 'A-ER' is mentioned on line 235, but not defined == Missing Reference: 'IPv6-StickyService' is mentioned on line 441, but not defined == Missing Reference: 'ISPF-EXT-EC' is mentioned on line 441, but not defined == Missing Reference: 'RFC4684' is mentioned on line 561, but not defined == Unused Reference: 'RFC4364' is defined on line 595, but no explicit reference was found in the text == Unused Reference: 'RFC8200' is defined on line 603, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 614, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 618, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 624, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 629, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 4364 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 1 error (**), 0 flaws (~~), 16 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft H. Song 3 J. Kaippallimalil 4 Intended status: Standard Futurewei 5 Expires: April 26, 2021 7 October 26, 2020 9 IP Layer Metrics for 5G Edge Computing Service 10 draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-00 12 Abstract 14 This draft describes the IP Layer metrics and methods to 15 measure the Edge Computing running status in order for IP 16 networks to select the optimal application server location in 17 5G Edge Computing (EC) environment. Those measurements are 18 for network to dynamically optimize the forwarding of 5G edge 19 computing service without any knowledge above IP layer. 21 Status of this Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. This document may not be 28 modified, and derivative works of it may not be created, 29 except to publish it as an RFC and to translate it into 30 languages other than English. 32 Internet-Drafts are working documents of the Internet 33 Engineering Task Force (IETF), its areas, and its working 34 groups. Note that other groups may also distribute working 35 documents as Internet-Drafts. 37 Internet-Drafts are draft documents valid for a maximum of 38 six months and may be updated, replaced, or obsoleted by 39 other documents at any time. It is inappropriate to use 40 Internet-Drafts as reference material or to cite them other 41 than as "work in progress." 42 IP Layer Metrics for 5G Edge Computing Services 44 The list of current Internet-Drafts can be accessed at 45 http://www.ietf.org/ietf/1id-abstracts.txt 47 The list of Internet-Draft Shadow Directories can be accessed 48 at http://www.ietf.org/shadow.html 50 This Internet-Draft will expire on April 7, 2021. 52 Copyright Notice 54 Copyright (c) 2020 IETF Trust and the persons identified as 55 the document authors. All rights reserved. 57 This document is subject to BCP 78 and the IETF Trust's Legal 58 Provisions Relating to IETF Documents 59 (http://trustee.ietf.org/license-info) in effect on the date 60 of publication of this document. Please review these 61 documents carefully, as they describe your rights and 62 restrictions with respect to this document. Code Components 63 extracted from this document must include Simplified BSD 64 License text as described in Section 4.e of the Trust Legal 65 Provisions and are provided without warranty as described in 66 the Simplified BSD License. 68 Table of Contents 70 1. Introduction..............................................3 71 1.1. 5G Edge Computing Background.........................3 72 1.2. Problem 1: Selecting 5G Edge Application Location....4 73 1.3. Problem 2: UE mobility creates unbalanced anycast 74 distribution..............................................5 75 2. Conventions used in this document.........................7 76 3. IP-Layer Metrics Definitions for 5G EC services...........9 77 3.1. IP-Layer Application ID..............................9 78 3.2. IP-Layer metric for App Server Load Measurement......9 79 3.3. Capacity Index in the overall cost..................11 80 3.4. Site Preference Index in the overall cost...........11 81 3.5. RTT to an ANYCAST Address in 5G EC..................12 82 4. Algorithm in Selecting the optimal Target Location.......13 83 5. Scope of IP Layer Metrics Advertisement..................14 84 6. Manageability Considerations.............................14 85 7. Security Considerations..................................14 86 8. IANA Considerations......................................15 87 IP Layer Metrics for 5G Edge Computing Services 89 9. References...............................................15 90 9.1. Normative References................................15 91 9.2. Informative References..............................15 92 10. Acknowledgments.........................................16 94 1. Introduction 95 1.1. 5G Edge Computing Background 97 In 5G Edge Computing environment [3GPP-EdgeComputing], one 98 Application can have multiple Application Servers hosted in 99 different Edge Computing data centers that are close in 100 proximity. Those Edge Computing (mini) data centers are 101 usually very close to, or co-located with, 5G base stations, 102 with the goal to minimize latency and optimize the user 103 experience. 105 When a UE (User Equipment) initiates application packets 106 using the destination address from a DNS reply or from its 107 own cache, the packets from the UE are carried in a PDU 108 session through 5G Core [5GC] to the 5G UPF-PSA (User Plan 109 Function - PDU Session Anchor). The UPF-PSA decapsulate the 110 5G GTP outer header and forwards the packets from the UEs to 111 the Ingress router of the Edge Computing (EC) Local Data 112 Network (LDN). The LDN for 5G EC, which is the IP Networks 113 from 5GC perspective, is responsible for forwarding the 114 packets to the intended destinations. 116 Routers in the local IP network should be able to select the 117 "best" or "closest" application server location out of many. 118 However, simply using distance alone as a metric may not be 119 sufficient as there may be many locations in close proximity. 120 Moreover, one of the main aims of locating application 121 servers so close to the user is to provide lower latency. 122 When a user moves and attaches from another access router 123 (UPF), the local IP network should be able to continue 124 routing to the established application server. As a user 125 keeps moving further away, a closer application server maybe 126 able to serve the user better. Network measurements, 127 including latency of various paths are provided to the 128 application domain to assist in reselection. These problems 129 are outlined in sections 1.2, and 1.3. 131 IP Layer Metrics for 5G Edge Computing Services 133 1.2. Problem 1: Selecting 5G Edge Application Location 135 Having multiple locations closer to UEs to host one 136 Application server can greatly improve the user experience. 137 But selecting an optimal location for the application traffic 138 from a UE may not be that simple. 140 Using DNS to reply with the address of the Application Server 141 location closest to the requesting UE can encounter issues 142 like: 143 - UE can cache results indefinitely, when the UE moves to a 144 5G cell site very far away, the cached address may still 145 be used, which can incur large network delay. 146 - The Application Server at a specific location whose 147 address replied by the DNS might be heavily loaded 148 causing slow or no response, when there are available low 149 utilized Application Server, for the same application, at 150 different locations very close in proximity. 151 - No inherent leverage of proximity information present in 152 the network (routing) layer, resulting in loss of 153 performance 154 - Local DNS resolver become the unit of traffic management 156 Increasingly, Anycast is used extensively by various 157 application providers and CDNs because ANYCAST makes it 158 possible to dynamically load balance across locations that 159 host the Application server based on network conditions. 160 Application server location selection using Anycast address 161 leverages the proximity information present in the network 162 (routing) layer and eliminates the single point of failure 163 and bottleneck at the DNS resolvers and application layer 164 load balancers. Another benefit of using ANYCAST address is 165 removing the dependency on UEs that use their cached 166 destination IP addresses for extended period. 168 But selection of an ANYCAST location purely based on the 169 network condition can encounter issue of the location 170 selected by network routing information being overutilized 171 while there are available underutilized locations close by. 173 IP Layer Metrics for 5G Edge Computing Services 175 1.3. Problem 2: UE mobility creates unbalanced anycast 176 distribution 178 Another problem of using ANYCAST address for multiple 179 locations of an Application Server in 5G environment is that 180 UEs' frequent moving from one 5G site to another. The 181 frequent move of UEs can make it difficult to plan where 182 Application server should be hosted. When a large number of 183 UEs using a particular Application congregate together 184 unpredictably, the ANYCAST location selected based on routing 185 distance can be heavily utilized, while the same Application 186 Server at other locations close-by are underutilized. 188 IP Layer Metrics for 5G Edge Computing Services 190 +--+ 191 |UE|---\+---------+ +------------------+ 192 +--+ | 5G | +-----------+ | S1: aa08::4450 | 193 +--+ | Site A +----+ +----+ | 194 |UE|----| | Ra | | R1 | S2: aa08::4460 | 195 +--+ | +----+ +----+ | 196 +---+ | | | | | S3: aa08::4470 | 197 |UE1|---/+---------+ | | +------------------+ 198 +---+ |IP Network | L-DN1 199 |(3GPP N6) | 200 | | | +------------------+ 201 | | | | S1: aa08::4450 | 202 | | +----+ | 203 | | | R3 | S2: aa08::4460 | 204 v | +----+ | 205 | | | S3: aa08::4470 | 206 | | +------------------+ 207 | | L-DN3 208 +--+ | | 209 |UE|---\+---------+ | | +------------------+ 210 +--+ | 5G | | | | S1: aa08::4450 | 211 +--+ | Site B +----+ +----+ | 212 |UE|----| | Rb | | R2 | S2: aa08::4460 | 213 +--+ | +----+ +----+ | 214 +--+ | | +-----------+ | S3: aa08::4470 | 215 |UE|---/+---------+ +------------------+ 216 +--+ L-DN2 217 Figure 1: multiple ANYCAST instances in different edge DCs 219 This document describes the measurements at IP Layer that can 220 reflect the application server running status and environment at 221 the specific locations. This document also describes the method 222 of incorporating those measurements with network routing cost to 223 come up with a more optimal criteria in selecting ANYCAST 224 locations. 226 Note: for the ease of description, the Edge Application Server 227 and Application Server are used interchangeably throughout this 228 document. 230 IP Layer Metrics for 5G Edge Computing Services 232 2. Conventions used in this document 234 A-ER: Egress Router to an Application Server Instance, 235 [A-ER] is used to describe the last router that 236 the application instance is attached. For 5G EC 237 environment, the A-ER can be the gateway router 238 to the Edge Computing Data Center. 240 ANYCAST Instance: refer to the Application Server Gateway at 241 a specific location which is reachable by the 242 ANYCAST address. 244 Application Server: An application server is a physical or 245 virtual server that host the software system for 246 the application. 248 Application Server Location: Represent a cluster of servers 249 at one location serving the same Application. One 250 application may have a Layer 7 Load balancer, 251 whose address(es) are reachable from external IP 252 network, in front of a set of application 253 servers. From IP network perspective, this whole 254 group of servers are considered as the 255 Application server at the location. 257 EC: Edge Computing 259 Edge Application Server: used interchangeably with 260 Application Server throughout this document. 262 Edge Computing Hosting Environment: An environment, such as 263 psychical or virtual machines, providing support 264 required for Edge Application Server's execution. 266 NOTE: The above terminologies are the same as 267 those used in 3GPP TR 23.758 269 IP Layer Metrics for 5G Edge Computing Services 271 Edge DC: Edge Data Center, which provides the Edge Hosting 272 Environment. It might be co-located with 5G Base 273 Station and not only host 5G core functions, but 274 also host frequently used Edge server instances. 276 gNB next generation Node B 278 Instance: the term "Instance" if used in the context of 279 Application Server, is referring to one location 280 of an application server; if used in the ANYCAST 281 context, is referring to one location of the 282 Application server with the same ANYCAST address. 284 L-DN: Local Data Network 286 PSA: PDU Session Anchor (UPF) 288 RTT: Round Trip Time 290 RTT-ANYCAST: A list of Round trip times to a group of 291 routers that have the ANYCAST instances directly 292 attached. 294 SSC: Session and Service Continuity 296 UE: User Equipment 298 UPF: User Plane Function 300 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 301 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT 302 RECOMMENDED", "MAY", and "OPTIONAL" in this document are to 303 be interpreted as described in BCP 14 [RFC2119] [RFC8174] 304 when, and only when, they appear in all capitals, as shown 305 here. 307 IP Layer Metrics for 5G Edge Computing Services 309 3. IP-Layer Metrics Definitions for 5G EC services 311 3.1. IP-Layer Application ID 313 From network perspective, an application server has an 314 Identifier, or IP Layer Application Server ID, which is 315 usually an ANYCAST address that can represent multiple 316 locations that host the application server. 318 3.2. IP-Layer metric for App Server Load Measurement 320 There are many network techniques and protocols to optimize 321 forwarding and ensuring QoS for applications, such as 322 DSCP/DiffServ, Traffic Engineered (TE) solutions, Segment 323 Routing, etc. But the reality is that most application 324 servers don't expose their internal logics to network 325 operators. Their communications are generally encrypted. Most 326 of them do not even respond to PING or ICMP messages 327 initiated by routers or network gears. 329 Even without knowledge of application internal logics, 330 network layer or IP Layer can monitor the traffic patterns 331 to/from the Application Server at each location to gauge the 332 running status of the application server at the location. 334 First, the network needs to discover which router(s) has the 335 application server attached. Those routers are called 336 Application Server Egress Router, or A-ER for short. A-ER is 337 usually the Gateway Router to an Edge Computing Data Center. 338 To discover if a (Gateway) router is the A-ER for a specific 339 Edge Application Server, the (Gateway) router can 340 periodically send reverse ARP (IPv4) or Neighbor Discovery 341 scan with the address of the Application Server to discover 342 if the Application Servers are hosted in its edge computing 343 data center. If yes, the router or routers are identified as 344 the A-ER for the Application Server. For one Application 345 Server, there can be many A-ERs at different EC Data Centers. 347 For an Application Server at a specific location, which is 348 identified by the address of the application server at the IP 349 layer, the A-ER can measure the amount of traffic destined 350 towards the address & the amount of the traffic from the 351 specific address, such as: 353 IP Layer Metrics for 5G Edge Computing Services 355 - Total number of packets to the attached App Server 356 (ToPackets); 357 - Total number of packets from the attached App Server 358 (FromPackets); 359 - Total number of Bytes to the attached App Server 360 (ToBytes); 361 - Total number of bytes from the attached App Server 362 (FromBytes); 364 The A-ER can advertise those measurements periodically, by BGP 365 UPDATE messages or OSPF/ISIS Link statement Advertisements, to 366 a group of routers that have traffic destined towards the 367 ANYCAST addresses of those application servers. 369 The actual load measurement to the App Server attached to an A- 370 ER can be based on one of the metrics above or including all 371 four metrics with different weights applied to each, such as: 373 LoadIndex = 374 w1*ToPackets+w2*FromPackes+w3*ToBytes+w4*FromBytes 376 Where 1. 603 [RFC8200] s. Deering R. Hinden, "Internet Protocol, Version 6 604 (IPv6) Specification", July 2017 606 9.2. Informative References 608 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 609 Partnership Project; Technical Specification Group 610 Services and System Aspects; Study on enhancement 611 of support for Edge Computing in 5G Core network 612 (5GC)", Release 17 work in progress, Aug 2020. 614 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 615 Subsequent Address Family Identifier (SAFI) and the 616 BGP Tunnel Encapsulation Attribute", April 2009. 618 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension 619 for SDWAN Overlay Networks", draft-dunbar-idr-bgp- 620 sdwan-overlay-ext-03, work-in-progress, Nov 2018. 622 IP Layer Metrics for 5G Edge Computing Services 624 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 625 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 626 draft-dunbar-idr-sdwan-edge-discovery-00, work-in- 627 progress, July 2020. 629 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 630 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 631 2018. 633 10. Acknowledgments 635 Acknowledgements to Donald Eastlake for their review and 636 contributions. 638 This document was prepared using 2-Word-v2.0.template.dot. 640 IP Layer Metrics for 5G Edge Computing Services 642 Authors' Addresses 644 Linda Dunbar 645 Futurewei 646 Email: ldunbar@futurewei.com 648 HaoYu Song 649 Futurewei 650 Email: haoyu.song@futurewei.com 652 John Kaippallimalil 653 Futurewei 654 Email: john.kaippallimalil@futurewei.com