dmm Z. Zhang Internet-Draft Juniper Networks Intended status: Informational K. Patel Expires: 12 January 2023 Arrcus T. Jiang China Mobile L. Contreras Telefonica 11 July 2022 5G Distributed UPFs draft-zzhang-dmm-5g-distributed-upf-01 Abstract This document describes evolution of mobile user plane in 5G, including distributed UPFs and alternative user plane implementations that some vendors/operators are pushing without changing 3GPP architecture/signaling. This also sets the stage for discussions in a companion document about potentially integrating UPF and Acess Node (AN) in a future generation (xG) of mobile network. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 12 January 2023. Copyright Notice Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Zhang, et al. Expires 12 January 2023 [Page 1] Internet-Draft 5G Distributed UPFs July 2022 Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Current User Plane in 5G . . . . . . . . . . . . . . . . . . 2 2. MUP Evolution in 5G: Distributed UPFs . . . . . . . . . . . . 3 2.1. Advantages of Distributed PSA UPFs . . . . . . . . . . . 5 2.2. Extent of Distribution and Open-RAN . . . . . . . . . . . 6 2.3. Enablers of Distributed PSA UPFs . . . . . . . . . . . . 7 3. MUP Evolution in 5G: Alternative Implementation Options . . . 7 3.1. GTP vs. SRv6 vs. MPLS tunneling . . . . . . . . . . . . . 7 3.2. Routing Based UPF . . . . . . . . . . . . . . . . . . . . 8 4. Security Considerations . . . . . . . . . . . . . . . . . . . 9 5. Informative References . . . . . . . . . . . . . . . . . . . 9 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9 1. Current User Plane in 5G Mobile User Plane (MUP) in 5G [_3GPP-23.501] has two distinct parts: the Access Network part between UE and AN/gNB, and the Core Network part between AN/gNB and UPF. N3 N9 N6 UE AN(gNB) | I-UPF | PSA UPF | +---------+ | | | |App Layer| | | routing | __ +---------+ | |+--/---+---\-+| ( ) |PDU Layer| relay | relay || PDU | || ( ) +---------+ +---/--+--\---+|+---/--+--\---+|+------+IP+L2|| ( ) | | | |GTP-U |||GTP-U |GTP-U |||GTP-U | || ( DN ) | 5G-AN | |5G-AN +------+||------+------+||------+ or || ( ) | | | |UDP+IP|||UDP+IP|UDP+IP|||UDP+IP| || ( ) | Proto | |Proto +------+||------+------+||------+Ether|| ( ) | | | | L2 ||| L2 | L2 ||| L2 | || -- | Layers | |Layers+------+||------+------+||------+-----+| | | | | L1 ||| L1 | L1 ||| L1 | L1 || +---------+ +------+------+|+------+------+|+------+-----+| | | | For the core network (CN) part, N3 interface extends the PDU layer from AN/gNB towards the PSA UPF, optionally through I-UPFs and in that case N9 interface is used between I-UPF and PSA UPF. Traditionally, UPFs are deployed at central locations and the N3/N9 tunnels extend the PDU layer to them. The N3/N9 interface uses GTP-U Zhang, et al. Expires 12 January 2023 [Page 2] Internet-Draft 5G Distributed UPFs July 2022 tunnels that are typically over a VPN over a transport infrastructure. While N6 is a 3GPP defined interface, it is for reference only and there is no tunneling or specification involved - it is simply a direct IP (in case of IP PDU session) or Ethernet (in case of Ethernet PDU session) connection to the DN. At the AN/gNB, relay is done between the radio layer and the GTP-U layer. At the PSA UPF, routing/switching is done for IP/Ethernet before GTP-U encapsulation (for downlink traffic) or after GTP-U decapsulation (for uplink traffic). 2. MUP Evolution in 5G: Distributed UPFs With MEC, ULCL UPFs are deployed closer to gNBs, while centralized PSA UPFs are still used to provide persistent IP addresses to UEs. In fact, even PSA UPFs could be distributed closer to gNBs and then the N3 interface becomes very simple - over a direct or short transport connection between gNB and UPF (or even an internal connection if the gNB and UPF are hosted on the same server). On the other hand, since the UPF to DN connection is direct, the DN becomes a VPN (e.g., IP VPN in case of IP PDU sessions or EVPN in case of Ethernet PDU sessions) over a transport infrastructure, most likely the same transport infrastructure for the VPN supporting the N3/N9 tunneling in centralized PSA UPF case, as shown in the following picture: Zhang, et al. Expires 12 January 2023 [Page 3] Internet-Draft 5G Distributed UPFs July 2022 N3 N6 UE1 AN1/gNB1 | PSA UPF1 | +---------+ | | |App Layer| | routing | +---------+ |+--/---+---\-+| |PDU Layer| relay || PDU | || PE1 +---------+ +---/--+--\---+|+------+IP+L2|| +----+--+ | | | |GTP-U |||GTP-U | ||----+VRF1| | | 5G-AN | |5G-AN +------+||------+ or || +----+ | | | | |UDP+IP|||UDP+IP| || |VRFn| | | Proto | |Proto +------+||------+Ether|| +----+--+ | | | | L2 ||| L2 | || ( ) | Layers | |Layers+------+||------+-----+| ( ) | | | | L1 ||| L1 | L1 || ( Transport ) +---------+ +------+------+|+------+-----+| ( ) | | ( Network ) PE3 | | ( +--+----+ UE2 AN2/gNB2 | PSA UPF2 | ( | |VRF1| +---------+ | | ( | |----+ |App Layer| | routing | ( | |VRFn| +---------+ |+--/---+---\-+| ( +--+----+ |PDU Layer| relay || PDU | || ( ) +---------+ +---/--+--\---+|+------+IP+L2|| ( ) | | | |GTP-U |||GTP-U | || ( ) | 5G-AN | |5G-AN +------+||------+ or || +----+--+ | | | |UDP+IP|||UDP+IP| ||----+VRF1| | | Proto | |Proto +------+||------+Ether|| +----+ | | | | | L2 ||| L2 | || |VRFn| | | Layers | |Layers+------+||------+-----+| +----+--+ | | | | L1 ||| L1 | L1 || PE2 +---------+ +------+------+|+------+-----+| | | The central PSA UPF is no longer needed in this case. Distributed UPF1/UPF2 connect to VRF1 on PE1/PE2 and VRF1 is for the VPN of the DN that UE1/UE2 access. There is also a PE3 for other sites of the VPN, which could be wireline sites including sites providing Internet access. UEs may keep their persistent IP addresses even when they re-anchor from one PSA UPF to another. In that case, for downlink traffic to be sent to the right UPF, when a UE anchors at a UPF the UPF advertises a host route for the UE and when a UE de-achors from a UPF the UPF withdraws the host route. While this relies on host routes to direct to-UE traffic to the right UPF, it does not introduce additional scaling burden compared to centralized PSA UPF model, as the centralized UPFs need to maintain Zhang, et al. Expires 12 January 2023 [Page 4] Internet-Draft 5G Distributed UPFs July 2022 per-UE forwarding state (in the form of PDRs/FARs) and the number is the same as the number of host routes that a hub DN router (e.g. vrf1 on PE3 for internet access) need to maintain in the distributed PSA UPFs model. Since the host routes may be lighter-weighted than the PDRs/FARs, the total amount of state may be actually smaller in the distributed model. For UE-UE traffic, the distributed PSA UPFs may maintain host routes that they learn from each other. With that the UE-UE traffic may take direct UPF-UPF path instead of going through a hub router in the DN (equivalent of central UPF). That is important in LAN-type services that require low delay. Alternatively, the distributed UPFs may maintain only a default route pointing to the hub router like PE3 (besides the host routes for locally anchored UEs). That way, they don't need to maintain many host routes though UPF-UPF traffic has to go through the hub router (and that is similar to all traffic going through a central PSA UPF). Optionally, even the host routes for locally anchored UEs can be omitted in the FIB of local UPF. Traffic among local UEs can be simply routed to the hub router following the default route, who will then send back to local UPF using VPN tunnels (MPLS or SRv6) that are stitched to GTP tunnels for destination UEs. 2.1. Advantages of Distributed PSA UPFs Distributed PSA UPFs have the following advantages: * MEC becomes much simpler - no need for centralized PSA UPF plus ULCL UPFs, and no need for special procedures for location based edge server discovery. * For LAN-type services, UE-UE traffic can be optimized (no need to go through centralized PSA UPFs) when UPFs maintain host routes. It also allows seamless integration of services across wireline/ wireless-connected customer sites. * N3/N9 tunneling is simplified In particular, there is now only short/simple N3 tunneling between AN/gNB and local UPFs in proximity. Among the distributed UPFs and other DN sites, versatile IETF/wireline VPN technologies are used instead. For example: Zhang, et al. Expires 12 January 2023 [Page 5] Internet-Draft 5G Distributed UPFs July 2022 * Any tunneling technology - MPLS, SR-MPLS or SRV6 - with any traffic engineering/differentiation capabilities can be used. Removal of the GTP/UDP header (and IPv4/IPv6 header in case of MPLS data plane) brings additional bandwidth savings in the transport infrastructure. * Any control plane model for VPN can be used - traditional distributed or newer controller based route advertisement. In short, the distributed PSA UPFs model achieves "N3/N9/N6 shortcut and central UPF bypass", which is desired by many operators. Notice that, since UPF has routing functions, depending on the capability of a UPF device, it may even be possible for a UPF device to act as a VPN PE. That can be done in one of the two models: * The UPF function and VPN PE function are separate but co-hosted on the same device with a logical/internal N6 connection between them. * The UPF and VPN PE function are integrated and the PDU sessions become VPN PE-CE links. The second model is especially useful when a UE is multi-homed to different EVPN PEs in case of Ethernet PDU sessions - EVPN's all- active multihoming procedures can be utilized. 2.2. Extent of Distribution and Open-RAN The UPFs can be distributed as close to the gNB as being co-located with it - either with a direct local link in between or both running as virtual functions on the same compute server. In the Open-RAN architecture [ORAN-Arch], the gNB function is split into gNB-CU (O-RAN CU or O-CU, for Central Unit) and gNB-DU (O-RAN DU or O-DU, for Distributed Unit). O-CU is the N3 GTP tunnel endpoint and is what gNB refers to in this document. Thus, the centralization process of the O-CU component can converge with the distribution process of the UPF up to some optimal and convenient location in the network. Zhang, et al. Expires 12 January 2023 [Page 6] Internet-Draft 5G Distributed UPFs July 2022 2.3. Enablers of Distributed PSA UPFs To distribute PSA UPFs, if persistent addresses must be used for UEs, the SMF must be able to allocate persistent IP addresses from a central pool even when a UE re-anchors at different PSA UPFs (e.g. due to mobility). If DHCPv4 is used, either the SMF acts as a central DHCP server or it relays DCHP requests to a central DHCP server on the DN. The distributed PSA UPFs must be able to advertise host routes in the DN. This should not be a problem since a UPF is essentially a router in that it routes traffic between DN and UEs (that are connected via PDU sessions). Notice that, advertising host routes for persistent IP addresses is no different from advertising MAC addresses in case of Ethernet PDU sessions. 3. MUP Evolution in 5G: Alternative Implementation Options 3.1. GTP vs. SRv6 vs. MPLS tunneling 3GPP specifies that all tunneling (e.g. N3/N9) use GTP, whose encapsulation includes IP header, UDP header and GTP header. The tunnel is between 3GPP NFs (e.g. gNBs and UPFs) over an IP transport, and the IP transport may be a VPN over the multi-service transport infrastructure of an operator. There have been proposals to replace GTP with SRv6 tunnels for the following benefits: * Traffic Engineering (TE) and Service Function Chaining (SFC) capability provided by SRv6 * Bandwidth savings because UDP and GTP headers are no longer needed While 3GPP has not adopted the proposal, and GTP can be transported over SRv6 (as overlay, instead of SRv6 replacing GTP), some operators still prefer to replace GTP with SRv6 "under the hood". That is, while RAN/UPF still use N2/N4 signaling, the actual tunnel are no longer GTP but SRv6 based on GTP parameters signaled by N2/N4. The SRv6 tunnel could be between two NFs, or a GW could be attached to an NF that still use traditional GTP and the GW will convert GTP to/from SRv6. This is specified in [I-D.ietf-dmm-srv6-mobile-uplane]. Similarly, if an operator prefers to use MPLS, a GTP tunnel can also be replaced with an MPLS PW instead of an SRv6 tunnel. Compared with SRv6, it is even more bandwidth efficient (no need for a minimum Zhang, et al. Expires 12 January 2023 [Page 7] Internet-Draft 5G Distributed UPFs July 2022 40-byte IPv6 header) and SR-MPLS can also provide TE/SFC capabilities. This is specified in [I-D.zzhang-pals-pw-for-ip-udp-payload]. Note that, While only IPv6 can scale to the 5G requirements for the transport infrastructure, it does not mean MPLS can not be used as data plane in the IPv6 network. 3.2. Routing Based UPF Traditionally, a UPF is implemented to follow 3GPP specifications. Specifically, N4 signaling is used for SMF to instruct a UPF to set up its session state in terms of PDRs/FARs. On N6 side, a UPF receives downlink traffic with destination addresses that are covered by the UPF's address range for its anchored UEs. The packet is matched against the installed PDRs and forwarded according to the associated FARs. On N3 side, a UPF decapsulates GTP+UDP+IP header of uplink traffic and uses the TEID to identify the DN where inner IP routing or Ethernet switching is done. [I-D.mhkk-dmm-srv6mup-architecture] specifies a new SRv6 based MUP architecture. When it is applied to a 3GPP based mobile architecture: * BGP signaling from a MUP Controller replaces N4 signaling from SMF. N4 signaling is still used between the MUP Controller and SMF - from SMF's point of view it is just interacting with a traditional UPF as usual. * A MUP GW becomes a distributed UPF for uplink traffic. * A MUP PE, which is different from a usually central PSA UPF, becomes a UPF for downlink traffic, in that traffic to each UE is placed into a different tunnel that is stitched to a GTP tunnel for that UE by a MUP GW (no route lookup is needed on the MUP GW for the downlink traffic). In this approach UE to UE traffic may still optionally go through the central PSA UPF. This is similar to that a hub router may be used in Section 2. This approach can be viewed as a specific way of implementing/ deploying distributed UPFs discussed in Section 2. It does have the advantage that from SMF's point of view, nothing is different from before - both from N4 signaling and deployment model point of view. Zhang, et al. Expires 12 January 2023 [Page 8] Internet-Draft 5G Distributed UPFs July 2022 While the above is specific to SRv6, a similar MPLS based approach will be specified separately for operators who prefer MPLS data plane, and it can even be SR-agnostic. 4. Security Considerations To be provided. 5. Informative References [I-D.ietf-dmm-srv6-mobile-uplane] Matsushima, S., Filsfils, C., Kohno, M., Garvia, P. C., Voyer, D., and C. E. Perkins, "Segment Routing IPv6 for Mobile User Plane", Work in Progress, Internet-Draft, draft-ietf-dmm-srv6-mobile-uplane-21, 9 May 2022, . [I-D.mhkk-dmm-srv6mup-architecture] Matsushima, S., Horiba, K., Khan, A., Kawakami, Y., Murakami, T., Patel, K., Kohno, M., Kamata, T., Garvia, P. C., Voyer, D., Zadok, S., Meilik, I., Agrawal, A., Perumal, K., and J. Horn, "Segment Routing IPv6 Mobile User Plane Architecture for Distributed Mobility Management", Work in Progress, Internet-Draft, draft-mhkk- dmm-srv6mup-architecture-03, 20 March 2022, . [I-D.zzhang-pals-pw-for-ip-udp-payload] Zhang, Z. and K. Patel, "PW for IP/UDP Payload without IP/ UDP Headers", Work in Progress, Internet-Draft, draft- zzhang-pals-pw-for-ip-udp-payload-01, 4 March 2022, . [ORAN-Arch] "O-RAN Architecture Description, V06.00", 2022. [_3GPP-23.501] "System architecture for the 5G System (5GS), V17.3.0", December 2021. Authors' Addresses Zhaohui Zhang Juniper Networks Zhang, et al. Expires 12 January 2023 [Page 9] Internet-Draft 5G Distributed UPFs July 2022 Email: zzhang@juniper.net Keyur Patel Arrcus Email: keyur@arrcus.com Tianji Jiang China Mobile Email: tianjijiang@chinamobile.com Luis M. Contreras Telefonica Email: luismiguel.contrerasmurillo@telefonica.com Zhang, et al. Expires 12 January 2023 [Page 10]