idnits 2.17.1 draft-dunbar-6man-5g-edge-compute-sticky-service-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. (A line matching the expected section header was found, but with an unexpected indentation: ' 8. Security Considerations' ) ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) (A line matching the expected section header was found, but with an unexpected indentation: ' 9. IANA Considerations' ) ** The abstract seems to contain references ([APN6]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (February 22, 2021) is 1160 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: '5GC' is mentioned on line 113, but not defined == Missing Reference: 'A-ER' is mentioned on line 269, but not defined == Unused Reference: 'RFC4364' is defined on line 709, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 721, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 738, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 742, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 746, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 750, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 4364 == Outdated reference: A later version (-02) exists of draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-00 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: A later version (-22) exists of draft-ietf-idr-tunnel-encaps-10 Summary: 4 errors (**), 0 flaws (~~), 13 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft J. Kaippallimalil 3 Intended status: Standard Futurewei 4 Expires: August 22, 2021 5 February 22, 2021 7 IPv6 Solution for 5G Edge Computing Sticky Service 8 draft-dunbar-6man-5g-edge-compute-sticky-service-02 10 Abstract 12 This draft describes an IPv6 solution that enables packets from an 13 application on a UE (User Equipment) sticking to the same ANYCAST 14 server location when the UE moves from one 5G cell site to another. 16 The proposed solution expands the Application-aware ID and Service- 17 Para options specified by [APN6] by including the ANYCAST Sticky 18 Service location information in the IPv6 Hop-by-Hop or Destination 19 Optional Header. 21 Status of this Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. This document may not be modified, 28 and derivative works of it may not be created, except to publish it 29 as an RFC and to translate it into languages other than English. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF), its areas, and its working groups. Note that 33 other groups may also distribute working documents as Internet- 34 Drafts. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 The list of current Internet-Drafts can be accessed at 42 http://www.ietf.org/ietf/1id-abstracts.txt 44 Internet-Draft IPv6 for 5G Edge Sticky Service 46 The list of Internet-Draft Shadow Directories can be accessed at 47 http://www.ietf.org/shadow.html 49 This Internet-Draft will expire on April 7, 2021. 51 Copyright Notice 53 Copyright (c) 2021 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction.................................................. 3 69 1.1. 5G Edge Computing Background.......................... 3 70 1.2. 5G Edge Computing Network Properties.................. 3 71 1.3. Problem #1: ANYCAST in 5G EC Environment.............. 5 72 1.4. Problem #2: sticking to original App Server........... 6 73 1.5. Problem #3: Application Server Relocation............. 6 74 2. Conventions used in this document............................. 7 75 3. Sticky Egress node to an ANYCAST Server....................... 8 76 4. End-Node based Sticky Service Solution........................ 9 77 4.1. Sticky Egress Address Discovery...................... 10 78 4.2. Sticky-Dst-SubTLV in Destination Extension Header.... 11 79 4.3. Expected behavior at the UE.......................... 12 80 4.4. Processing at the Ingress router..................... 12 81 5. Tunnel based Sticky Service Solutions........................ 13 82 5.1. Ingress and Egress Routers Processing Behavior....... 13 83 5.2. Scenario 1: Without Communication with 5G system..... 15 84 5.3. Scenario 2: With communication with 5G system........ 15 85 6. Expanding APN6 for Sticky Service information................ 16 86 6.1. Sticky Service ID encoded in the Application-aware ID 16 87 6.2. Sticky Service Sub-TLV encoded in APN6 Service-para 88 option.................................................... 16 89 7. Manageability Considerations................................. 17 90 8. Security Considerations...................................... 17 91 9. IANA Considerations.......................................... 17 92 Internet-Draft IPv6 for 5G Edge Sticky Service 94 10. References.................................................. 17 95 10.1. Normative References................................ 17 96 10.2. Informative References.............................. 17 97 11. Acknowledgments............................................. 18 99 1. Introduction 101 1.1. 5G Edge Computing Background 103 As described in [5G-EC-Metrics], one Application in 5G Edge Computing 104 environment can have multiple Application Servers hosted in different 105 Edge Computing data centers that are close in proximity. Those Edge 106 Computing (mini) data centers are usually very close to, or co- 107 located with, 5G base stations, with the goal to minimize latency and 108 optimize the user experience. 110 When a UE (User Equipment) initiates application packets using the 111 destination address from a DNS reply or from its own cache, the 112 packets from the UE are carried in a PDU session through 5G Core 113 [5GC] to the 5G UPF-PSA (User Plan Function - PDU Session Anchor). 114 The UPF-PSA decapsulate the 5G GTP outer header and forwards the 115 packets from the UEs to the Ingress router of the Edge Computing (EC) 116 Local Data Network (LDN). The LDN for 5G EC, which is the IP Networks 117 from 5GC perspective, is responsible for forwarding the packets to 118 the intended destinations. 120 When the UE moves out of coverage of its current gNB (next generation 121 Node B) (gNB1), handover procedures are initiated and the 5G SMF 122 (Session Management Function) also selects a new UPF-PSA. The 123 standard handover procedures described in 3GPP TS 23.501 and TS 124 23.502 are followed. When the handover process is complete, the UE 125 has a new IP address and the IP point of attachment is to the new 126 UPF-PSA. 5GC may maintain a path from the old UPF to new the UPF for 127 a short period of time for SSC [Session and Service Continuity] mode 128 3 to make the handover process more seamless. 130 1.2. 5G Edge Computing Network Properties 132 In this document, 5G Edge Computing Network refers to multiple 133 Local IP Data Networks (LDN) in one region that interconnect 134 the Edge Computing mini-data centers. Those IP LDN networks are 135 the N6 interfaces from 3GPP 5G perspective. 137 The ingress routers to the 5G Edge Computing Network are the 138 routers directly connected to 5G UPFs. The egress routers to 140 Internet-Draft IPv6 for 5G Edge Sticky Service 142 the 5G Edge Computing Network are the routers that have direct 143 link to the Edge Computing servers. The servers and the egress 144 routers are co-located. Some of those mini Edge Computing Data 145 centers may have Virtual switches or Top of Rack switches 146 between the egress routers and the servers. But transmission 147 delay between the egress routers and the Edge Computing servers 148 is so small, therefore can be considered negligible in this 149 document. 151 When one mini data center has multiple Edge Computing Servers 152 attached to one App Layer Load Balancer, only the App Layer 153 Load Balancer is visible to the 5G Edge Computing Network. How 154 the App Layer Load balancer manages the individual servers is 155 out of the scope of the network layer. 157 The Edge Computer Services are specially managed services that 158 need to utilize the network topology and balance among multiple 159 mini Edge Computing Data Centers with the same ANYCAST address. 160 UEs can access many services that are not part of the 161 registered 5G Edge Computing Services. 163 Internet-Draft IPv6 for 5G Edge Sticky Service 165 +--+ 166 |UE|---\+---------+ +------------------+ 167 +--+ | 5G | +---------+ | S1: aa08::4450 | 168 +--+ | Site +--++---+ +----+ | 169 |UE|----| A |PSA| Ra| | R1 | S2: aa08::4460 | 170 +--+ | +---+---+ +----+ | 171 +---+ | | | | | S3: aa08::4470 | 172 |UE1|---/+---------+ | | +------------------+ 173 +---+ |IP Network | L-DN1 174 |(3GPP N6) | 175 | | | +------------------+ 176 | UE1 | | | S1: aa08::4450 | 177 | moves to | +----+ | 178 | Site B | | R3 | S2: aa08::4460 | 179 v | +----+ | 180 | | | S3: aa08::4470 | 181 | | +------------------+ 182 | | L-DN3 183 +--+ | | 184 |UE|---\+---------+ | | +------------------+ 185 +--+ | 5G | | | | S1: aa08::4450 | 186 +--+ | Site +--++-+--+ +----+ | 187 |UE|----| B |PSA| Rb | | R2 | S2: aa08::4460 | 188 +--+ | +--++----+ +----+ | 189 +--+ | | +-----------+ | S3: aa08::4470 | 190 |UE|---/+---------+ +------------------+ 191 +--+ L-DN2 192 Figure 1: App Servers in different edge DCs 194 1.3. Problem #1: ANYCAST in 5G EC Environment 196 Increasingly, ANYCAST is used extensively by various application 197 providers and CDNs because it is possible to dynamically load balance 198 across multiple locations of the same address based on network 199 conditions. BGP is an integral part in the way IP anycast usually 200 functions. Within BGP routing there are multiple routes for the same 201 IP address which are pointing to different locations. 203 Application Server location selection using Anycast address leverages 204 the proximity information present in the network (routing) layer and 205 eliminates the single point of failure and bottleneck at the DNS 206 resolvers and application layer load balancers. Another benefit of 207 using ANYCAST address is removing the dependency on UEs that use 209 Internet-Draft IPv6 for 5G Edge Sticky Service 211 their cached IP addresses instead of querying DNS when they move to a 212 new location. 214 But, having multiple locations for the same ANYCAST address in 5G 215 Edge Computing environment can be problematic because all those edge 216 computing Data Centers can be close in proximity. There might not be 217 any difference in the routing cost to reach the Application Servers 218 in different Edge DCs. Same routing cost to multiple ANYCAST 219 locations can cause packets from one flow to be forwarded to 220 different locations, which can cause service glitches. 222 1.4. Problem #2: sticking to original App Server 224 When a UE moves to a new location but continues the same application 225 flow, routers at the new location might choose the App Server closer 226 to the new location. As shown in the figure below, when the UE1 in 227 5G-site-A moves to the 5G-Site-B, the router directly connected to 5G 228 PSA2 might forward the packets destined towards the S1: aa08::4450 to 229 the server located in L-DN2 because L-DN2 has the lowest cost based 230 on routing. 232 This is not the desired behavior for some services, which are called 233 Sticky Services in this document. 235 Even for some advanced applications with built-in mechanisms to re- 236 sync the communications at the application layer after switching to a 237 new location, service glitches are very often experienced by users. 239 It worth noting that not all services need to be sticky. We assume 240 only a subset of services are, and the Network is informed of the 241 services that need to be sticky, usually by requests from application 242 developers or controllers. 244 This document describes an IPv6 based network layer solution to stick 245 the packets belonging to the same flow of a UE to its original App 246 Server location after the UE is anchored to a new UPF-PSA. 248 1.5. Problem #3: Application Server Relocation 250 When an Application Server is added to, moved, or deleted from a 5G 251 Edge Computing Data Center, the routing protocol has to propagate the 252 changes to 5G PSA or the PSA adjacent routers. After the change, the 253 cost associated with the site [5G-EC-Metrics] might change as well. 255 Note: for the ease of description, the Edge Computing Server, 256 Application Server, or App Server are used interchangeably throughout 257 this document. 259 Internet-Draft IPv6 for 5G Edge Sticky Service 261 2. Conventions used in this document 263 APN6 Application aware network using IPv6. The term 264 "Application" has very broad meanings. In this document 265 the term "Application" refers to any applications that 266 use ANYCAST servers in the 5G Edge Computing 267 Environment. 269 A-ER: Egress Router to an Application Server, [A-ER] is used 270 to describe the last router that the Application Server 271 is attached. For 5G EC environment, the A-ER can be the 272 gateway router to a (mini) Edge Computing Data Center. 274 Application Server: An application server is a physical or virtual 275 server that host the software system for the 276 application. 278 Application Server Location: Represent a cluster of servers at one 279 location serving the same Application. One application 280 may have a Layer 7 Load balancer, whose address(es) are 281 reachable from external IP network, in front of a set of 282 application servers. From IP network perspective, this 283 whole group of servers are considered as the Application 284 server at the location. 286 Edge Application Server: used interchangeably with Application Server 287 throughout this document. 289 EC: Edge Computing 291 Edge Hosting Environment: An environment providing support required 292 for Edge Application Server's execution. 294 NOTE: The above terminologies are the same as those used 295 in 3GPP TR 23.758 297 Edge DC: Edge Data Center, which provides the Edge Computing Hosting 298 Environment. It might be co-located with 5G Base Station 299 and not only host 5G core functions. 301 gNB next generation Node B 303 Internet-Draft IPv6 for 5G Edge Sticky Service 305 L-DN: Local Data Network 307 PSA: PDU Session Anchor (UPF) 309 SSC: Session and Service Continuity 311 UE: User Equipment 313 UPF: User Plane Function 315 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 316 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 317 "OPTIONAL" in this document are to be interpreted as described in BCP 318 14 [RFC2119] [RFC8174] when, and only when, they appear in all 319 capitals, as shown here. 321 3. Sticky Egress node to an ANYCAST Server 323 From Local Data Network perspective, achieving Sticky Service for a 324 flow from a UE to a specific ANYCAST server is same as delivering the 325 packets of the flow to the same egress router, e.g. R1 in Figure 1, 326 to which the ANYCAST Server is attached. The egress router, say R1 in 327 Figure 1, should deliver the packets destined towards the ANYCAST 328 address to its directly attached server even through the same address 329 is also reachable from other routers, unless the directly attached 330 server is no longer reachable due to Server or Link failure. 332 The egress router, e.g. R1, to which the Edge Computing server is 333 directly attached, is called the Sticky Egress node to the Edge 334 Computing Server at the location. The Sticky Egress node has a 335 unicast address. 337 When there are multiple Edge Computing Servers with the same ANYCAST 338 address located in different mini Edge Computing Data Centers, each 339 location is identified by its Sticky Egress nodes unicast 340 address(es). To the LDN's ingress routers, e.g. Ra or Rb in the 341 Figure 1, achieving sticky service is to send the packets belonging 342 to the same flow to the same Sticky Egress node's unicast address. 344 From Local Data Network perspective, the Sticky Egress nodes' unicast 345 addresses are the Next Hops (i.e. R1, R2, and R3) to reach the Edge 346 Computing server ANYCAST address. 348 The Flow Affinity feature, which is supported by most commercial 349 routers today, can ensure packets belonging to one flow be forwarded 351 Internet-Draft IPv6 for 5G Edge Sticky Service 353 along the same path to the same egress router which then delivers the 354 packets to the attached Edge Computing Server. 356 Editor's note: for IPv6 traffic, Flow Affinity can be supported by 357 the routers of the Local Data Network (LDN) forwarding the packets 358 with the same Flow Label in the packets' IPv6 Header along the same 359 path towards the same egress router. For IPv4 traffic, 5 tuples in 360 the IPv4 header can be used to achieve the Flow Affinity 362 The solution described in this document is to ensure a flow from a UE 363 to be forwarded to the same egress router of the App Server after the 364 UE moves to a new 5G Site which result in the UE being assigned a new 365 IP address and attached to a new access router. 367 4. End-Node based Sticky Service Solution 369 The End-Node based Sticky Service solution needs IPv6 end nodes to 370 insert Destination Option header to the IPv6 header. Section 5 371 describes a Sticky Service solution that do not require end nodes 372 performing anything. Here are some assumptions for the End-Node based 373 Sticky Service solution: 375 - There is an EdgeCompute-Controller that can exchange information 376 with the Edge Computing servers. 377 - Network is aware of the Sticky Services by the Sticky Service 378 Identifiers, which can be ANYCAST addresses or regular IP 379 addresses. If an Edge Computing service needs to be sticky in 380 the 5G Edge Computing environment, the service ID is registered 381 with the 5G Edge Computing controller. 383 Here is the overview of the End-Node based Sticky Service solution: 384 - Each ANYCAST Edge Computing server either learns or is informed 385 of the unicast Sticky Egress address (Section 3). The goal of 386 the network is to deliver packets belonging to one flow to the 387 same Sticky Egress address for the ANYCAST address. Section 4.1 388 describes how Edge Computing Servers discover their 389 corresponding Sticky Egress unicast addresses. 390 - When an Edge Computing server sends data packets to a UE (or 391 client), it inserts the Sticky-Dst-SubTLV (described in Section 392 4.2) into the packets' Destination Option Header. 393 - UE (or client) needs to copy the Destination Option Header from 394 the received packet to the next packet's Destination Header if 395 the next packet belongs to the same flow as the previous packet. 397 Internet-Draft IPv6 for 5G Edge Sticky Service 399 - If the following conditions are true, the ingress router 400 encapsulates the packet from the UE in a tunnel whose outer 401 destination address is set to the Sticky Egress Address 402 extracted from the packet's Sticky-Dst-SubTLV: 403 o The destination of the packet from the UE side matches 404 with one of the Sticky Service ACLs configured on the 405 ingress router of the LDN, 406 o the packet header has the Destination Option present with 407 Sticky-Dst-SubTLV. 409 - Else (i.e. one of the conditions above is not true), the ingress 410 node uses its own algorithm, such as the least cost as described 411 in [5G-EC-Metrics], to select the optimal Sticky Egress address 412 for forwarding the packet. 414 4.1. Sticky Egress Address Discovery 416 To an App server with ANYCAST address, the Sticky Egress address is 417 same as its default Gateway address. 419 To prevent malicious UEs (or clients) sending DDOS attacks to routers 420 within 5G EC LDN, e.g. the Sticky Egress address that is encoded in 421 the Destination option header in the packets sent back to the UEs (or 422 clients), a proxy Sticky Egress address can be encoded in the 423 Destination option header. The proxy Sticky Egress address is only 424 recognizable by the 5G EC LDN ingress nodes, i.e. the Ra and Rb in 425 the Figure 1, but not routable in other networks. The LDN ingress 426 routers can translate the proxy Sticky Egress to a routable address 427 for the Sticky Egress node after the source addresses of the packets 428 are authenticated. 430 Internet-Draft IPv6 for 5G Edge Sticky Service 432 4.2. Sticky-Dst-SubTLV in Destination Extension Header 434 A new Sticky-Dst-SubTLV is specified as below, which can be inserted 435 into the IPv6 Destination Options header. The IPv6 Destination Option 436 Header is specified by [RFC8200] as having a Next Header value of 60: 438 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 439 | Next Header | Hdr Ext Len | | 440 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 441 | | 442 | Sticky-Dst-SubTLV | 443 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 445 Sticky-Dst-SubTLV is specified as: 447 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 448 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 449 | Sticky-Type | Len | AFI | Reserved | 450 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 451 |Sticky Egress address (IPv4 or IPv6) for reaching the ANYCAST | 452 ~ ~ 453 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 455 Sticky-Type = 1: indicate the Sticky Egress unicast address at 456 encoded in the Sticky-Dst-SubTLV. 458 Internet-Draft IPv6 for 5G Edge Sticky Service 460 4.3. Expected behavior at the UE 462 For edge computing services that need sticky service while UEs move 463 across multiple 5G sites, the UEs (the TCP/IP layer of the UEs' OS to 464 be precise) need to extract the Destination Extension Header field 465 from packets received from the App Server and inserts the extracted 466 Destination Extension Header into the subsequent packets belonging to 467 the same flow. 469 Granted, it might take some time for IPv6 TCP/IP Layer to adopt the 470 practice of copying the IPv6 Destination Extension Header field from 471 the received packets to the subsequent packets belonging to the same 472 flow. However, once the egress routers and ingress routers for 5G 473 local data network support the feature, more and more Edge Computing 474 services would want to utilize this special feature by adding this 475 step. 477 Section 4 describes the network layer processing if UEs do not 478 perform the steps described here. 480 4.4. Processing at the Ingress router 482 - An Ingress router is configured with an ACL for filtering out 483 the applications that need sticky service. 485 Note, not all applications need sticky service. Using ACL can 486 greatly reduce the processing on the routers. 488 - When an Ingress router receives a packet from the 5G side that 489 matches the ACL, the Ingress router extracts the Sticky-Dst- 490 SubTLV from the packet IPv6 header if the field exists in the 491 packet header. 492 - Encapsulate the packet with the tunnel type that supported by 493 the original Sticky Egress node, using the extracted Sticky 494 Egress address in the destination field of the outer header, and 495 forward the packet. 497 Note: if the proxy Sticky Egress address is encoded in the 498 Sticky-Dst-SubTLV, the ingress router needs to translate 499 the proxy Sticky Egress address to a routable address. 501 If none of the above conditions are met, the ingress router uses 502 its own algorithm to select the optimal Sticky Egress node to 503 forward the packet. 505 Internet-Draft IPv6 for 5G Edge Sticky Service 507 5. Tunnel based Sticky Service Solutions 509 Before all UEs can change their processing behavior as described in 510 the Section 4, a Tunnel based Sticky Service solution can be used. 511 This solution doesn't depend on UE behavior, therefore, more 512 processing on the LDN ingress routers is needed. 513 5.1. Ingress and Egress Routers Processing Behavior 515 The solution assumes that both ingress routers and egress routers 516 support at least one type of tunnel and are configured with ACLs to 517 filter out packets whose destination or source addresses match with 518 the Sticky Service Identifier. The solution also assumes there are 519 only limited number of Sticky Services to be supported. 520 An ingress router needs to build a Sticky-Service-Table, with the 521 minimum following attributes. The Sticky-Service-Table is initialized 522 to be empty. 523 - Sticky Service ID 524 - Flow Label 525 - Sticky Egress address 526 - Timer 528 Editor's Note: 529 When a UE moves from one 5G Site to another, the same UE will have 530 a new IP address. "Flow Label + Sticky Service ID" stays the same 531 when a UE is anchored to a new PSA. Therefore, this solution use 532 "Flow Label + Sticky Service ID" to identify a sticky flow. Since 533 the chance of different UEs sending packets to the same ANYCAST 534 address using the same Flow Label is very low, it is with high 535 probability that "Flow Label + Sticky Service ID" can uniquely 536 identify a flow. When multiple UEs using the same Flow Label 537 sending packets to the same ANYCAST address, the solution described 538 in this section will stick the flows to the same ANYCAST server 539 attached to the Sticky Egress router. This behavior doesn't cause 540 any harm. 542 Each entry in the Sticky-Service-Table has a Timer because a sticky 543 service is no longer sticky if there are no packets of the same flow 544 destined towards the service ID for a period of time. The Timer 545 should be larger than a typical TCP session Timeout value. An entry 546 is automatically removed from the Sticky-Service-Table when its timer 547 expires. 549 Internet-Draft IPv6 for 5G Edge Sticky Service 551 Note: since there are only small number of Sticky services, the 552 Sticky-Service-Table is not very large. 553 When an ingress router receives a packet from a UE matching with one 554 of the Sticky Service ACLs and there is no entry in the Sticky- 555 Service-Table matching the Flow Label and the Sticky Service ID, the 556 ingress router considers the packet to be the first packet of the 557 flow. There is no need to sticking the packet to any location. The 558 ingress router uses its own algorithm to select the optimal egress 559 node as the Sticky Egress address for the ANYCAST address, 560 encapsulates the packet with a tunnel that is supported by the egress 561 node. The tunnel's destination address is set to the egress node 562 address. 564 When an egress router receives a packet from an attached host with 565 the packet's source address matching with one of the Sticky Service 566 IDs, the egress router encapsulates the packet with a tunnel that is 567 supported by the ingress router and the tunnel's destination address 568 is set to the ingress router address. An Egress router learns the 569 ingress router address for a UE IP address via BGP UPDATE messages. 571 When an ingress router receives a packet in a tunnel from any egress 572 router and the packet's source address matches with a Sticky Service 573 ID, the egress router address is set as the Sticky Egress address for 574 the Sticky Service ID. The ingress router adds the entry of "Sticky- 575 Service-ID + Flow Label + the associated Sticky Egress address + 576 Timer" to the Sticky-Service-Table if the entry doesn't exist yet in 577 the table. If the entry exists, the ingress router refreshes the 578 Timer of the entry in the table. 580 When the ingress router receives the subsequent packets of a flow 581 from the 5G side matching with an Sticky Service ID and the Sticky- 582 Service ID exists in the Sticky-Service-Table, the ingress router 583 uses the Sticky Egress address found in the Sticky-Service-Table to 584 encapsulate the packet and refresh the Timer of the entry. If the 585 Sticky-Service ID doesn't exist in the table, the ingress router 586 considers the packet as the first packet of a flow. 588 The subsequent sections describe how ingress nodes prorogate their 589 Sticky-Service-Table to their neighboring ingress nodes. The 590 propagation is for neighboring ingress nodes to be informed of the 591 Sticky Egress address to a sticky service if a UE moves to a new 592 neighboring 5G site resulting in anchoring to a new ingress node. 594 Internet-Draft IPv6 for 5G Edge Sticky Service 596 5.2. Scenario 1: Without Communication with 5G system 598 When a UE moves to a very far away 5G site, say a different 599 geographic region, the benefit of sticking to the original ANYCAST 600 server is out weighted by network delay. Then, there is no point 601 sending packets to the Sticky Egress node if the ingress router very 602 far away. Therefore, it is necessary for each ingress router to have 603 a group of neighboring ingress routers that are not too far away from 604 the potential Sticky Egress nodes selected by the ingress router. 605 This group of ingress routers is called the Neighboring Ingress 606 Group. Each ingress router can either automatically discover its 607 Neighboring Ingress Group by routing protocols or is configured by 608 its controller. It is out of the scope of this document on how 609 ingress nodes discover its Neighboring Ingress Group. 611 Each ingress node needs to periodically advertise its Sticky-Service- 612 Table to the routers within its Neighboring Ingress Group. 614 Upon receiving the Sticky-Service-Table from routers in its 615 Neighboring Ingress Group, each ingress router merges the entries 616 from the received Sticky-Service-Table to its own. 618 The ingress and the egress nodes perform the same actions as 619 described in Section 5.1. 621 5.3. Scenario 2: With communication with 5G system 623 In this scenario, there is communication with 5G System and network 624 get notified by a UE is anchored to a new PSA. 626 When a UE is re-anchoring from PSA1 to PSA2, 5GC EC management system 627 sends a notification to the router that is directly connected to 628 PSA1. The notification includes the address of the new PSA that the 629 UE is to be anchored, i.e. the PSA2, and the UE's new IP address. 631 In this scenario, the Sticky Service can be uniquely identified by 632 "Sticky Service ID" + "UE address". the Sticky-Service-Table should 633 include the following attributes: 635 - Sticky Service ID 636 - UE address 637 - Sticky Egress address 638 - Timer 640 Upon receiving the notification from the 5G EC management system, the 641 ingress router (i.e. the one directly connected to the old PSA) sends 642 the specific entry of the Sticky-Service Table, i.e. "Sticky Service 644 Internet-Draft IPv6 for 5G Edge Sticky Service 646 ID" + UE address + Sticky Egress + Timer to the router directly 647 connected to the new PSA. 649 Upon receiving the entry, the ingress router merges the entry into 650 its own Sticky-Service-Table. 652 The ingress and egress router processing are the same as described in 653 Section 5.1 except a flow is now uniquely identified by the "Sticky 654 Service ID" + "UE address" instead of "Sticky Service ID" + "Flow 655 Label". 657 6. Expanding APN6 for Sticky Service information 659 The Application-aware ID and Service-Para Option described 660 [APN6] can be expanded to include the sticky service 661 information. 663 6.1. Sticky Service ID encoded in the Application-aware ID 665 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 666 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 667 | Sticky Level |StickyServiceID| Reserved | Flow ID | 668 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 670 Sticky Level: represent how important for an application to stick to 671 its ANYCAST servers. Some applications may prefer one flow sticking 672 to the original ANYCAST server, but not required. Some applications 673 may require the stickiness. 675 StickyServiceID: the ANYCAST address of the application servers. 677 The Reserved field can be used for future to identifier the 5G access 678 domain for the flow. 680 Flow ID: the identifier for the flow that needs to stick to a 681 specific ANYCAST server. 683 6.2. Sticky Service Sub-TLV encoded in APN6 Service-para option 685 The Sticky-Dst-SubTLV described in the Section 4.2 of this document 686 can be included in the Service-Para Sub-TLVs field. 688 Internet-Draft IPv6 for 5G Edge Sticky Service 690 7. Manageability Considerations 692 To be added. 694 8. Security Considerations 696 To be added. 698 9. IANA Considerations 700 To be added. 702 10. References 704 10.1. Normative References 706 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 707 Requirement Levels", BCP 14, RFC 2119, March 1997. 709 [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private networks 710 (VPNs)", Feb 2006. 712 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 713 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 714 2017, . 716 [RFC8200] s. Deering R. Hinden, "Internet Protocol, Version 6 (IPv6) 717 Specification", July 2017 719 10.2. Informative References 721 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation Partnership 722 Project; Technical Specification Group Services and System 723 Aspects; Study on enhancement of support for Edge 724 Computing in 5G Core network (5GC)", Release 17 work in 725 progress, Aug 2020. 727 [5G-EC-Metrics] L. Dunbar, H. Song, J. Kaippallimalil, "IP Layer 728 Metrics for 5G Edge Computing Service", draft-dunbar-ippm- 729 5g-edge-compute-ip-layer-metrics-00, work-in-progress, Oct 730 2020. 732 Internet-Draft IPv6 for 5G Edge Sticky Service 734 [APN6] Z. Li, et al, "Application-aware IPv6 Networking (APN6) 735 Encapsulation", draft-li-6man-app-aware-ipv6-network-03, 736 work-in-progress, Feb 2021. 738 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 739 Address Family Identifier (SAFI) and the BGP Tunnel 740 Encapsulation Attribute", April 2009. 742 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension for SDWAN 743 Overlay Networks", draft-dunbar-idr-bgp-sdwan-overlay-ext- 744 03, work-in-progress, Nov 2018. 746 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. Majumdar, 747 "BGP UPDATE for SDWAN Edge Discovery", draft-dunbar-idr- 748 sdwan-edge-discovery-00, work-in-progress, July 2020. 750 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 751 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 2018. 753 11. Acknowledgments 755 Acknowledgements to Ron Bonica and Donald Eastlake for their review 756 and contributions. 758 This document was prepared using 2-Word-v2.0.template.dot. 760 Authors' Addresses 762 Linda Dunbar 763 Futurewei 764 Email: ldunbar@futurewei.com 766 John Kaippallimalil 767 Futurewei 768 Email: john.kaippallimalil@futurewei.com