idnits 2.17.1 draft-ietf-dmm-5g-uplane-analysis-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 8, 2019) is 1753 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'GTP-U-1' is mentioned on line 683, but not defined == Missing Reference: 'GTP-U-2' is mentioned on line 685, but not defined == Missing Reference: 'GTP-U-3' is mentioned on line 687, but not defined == Missing Reference: 'Uplink' is mentioned on line 303, but not defined == Missing Reference: 'Downlink' is mentioned on line 323, but not defined == Missing Reference: 'GTP-U-4' is mentioned on line 690, but not defined == Missing Reference: 'GTP-U-5' is mentioned on line 693, but not defined == Missing Reference: 'GTP-U-6' is mentioned on line 696, but not defined == Missing Reference: 'GTP-U-7' is mentioned on line 699, but not defined == Missing Reference: 'GTP-U-8' is mentioned on line 703, but not defined == Missing Reference: 'GTP-U-9' is mentioned on line 706, but not defined == Missing Reference: 'GTP-U-10' is mentioned on line 708, but not defined == Missing Reference: 'GTP-U-11' is mentioned on line 710, but not defined == Missing Reference: 'GTP-U-12' is mentioned on line 712, but not defined == Missing Reference: 'GTP-U-13' is mentioned on line 715, but not defined == Missing Reference: 'GTP-U-14' is mentioned on line 717, but not defined == Missing Reference: 'F-SEID' is mentioned on line 926, but not defined == Missing Reference: 'PDR-ID' is mentioned on line 930, but not defined == Missing Reference: 'FAR-ID' is mentioned on line 936, but not defined == Missing Reference: 'QER-ID' is mentioned on line 949, but not defined == Missing Reference: 'URR-ID' is mentioned on line 959, but not defined == Missing Reference: 'BAR-ID' is mentioned on line 966, but not defined == Missing Reference: 'Traffic-Endpoint-ID' is mentioned on line 969, but not defined -- Obsolete informational reference (is this intentional?): RFC 2460 (Obsoleted by RFC 8200) Summary: 1 error (**), 0 flaws (~~), 25 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DMM Working Group S. Homma 3 Internet-Draft NTT 4 Intended status: Informational T. Miyasaka 5 Expires: January 9, 2020 KDDI Research 6 S. Matsushima 7 SoftBank 8 D. Voyer 9 Bell Canada 10 July 8, 2019 12 User Plane Protocol and Architectural Analysis on 3GPP 5G System 13 draft-ietf-dmm-5g-uplane-analysis-02 15 Abstract 17 This document analyzes the mobile user plane protocol and the 18 architecture specified in 3GPP 5G documents. The analysis work is to 19 clarify those specifications, extract protocol and architectural 20 requirements and derive evaluation aspects for user plane protocols 21 on IETF side. This work is corresponding to the User Plane Protocol 22 Study work on 3GPP side. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on January 9, 2020. 41 Copyright Notice 43 Copyright (c) 2019 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Current Status of Mobile User Plane for 5G . . . . . . . 3 60 1.2. Our Way of Analysis Work . . . . . . . . . . . . . . . . 4 61 2. Terms and Abbreviations . . . . . . . . . . . . . . . . . . . 4 62 3. GTP-U Specification and Observation . . . . . . . . . . . . . 6 63 3.1. GTP-U Tunnel . . . . . . . . . . . . . . . . . . . . . . 6 64 3.2. GTP-U Header Format . . . . . . . . . . . . . . . . . . . 10 65 3.3. Control Plane Protocol for GTP-U . . . . . . . . . . . . 12 66 3.4. GTP-U message . . . . . . . . . . . . . . . . . . . . . . 13 67 3.5. Packet Format . . . . . . . . . . . . . . . . . . . . . . 14 68 3.6. Observations Summary . . . . . . . . . . . . . . . . . . 16 69 4. 5GS Architectural Requirements for User Plane Protocols . . . 16 70 4.1. Overview of 5G System Architecture . . . . . . . . . . . 16 71 4.1.1. UPF Functionalities . . . . . . . . . . . . . . . . . 18 72 4.1.2. UP Traffic Detection . . . . . . . . . . . . . . . . 19 73 4.1.3. User Plane Configuration . . . . . . . . . . . . . . 21 74 4.2. Architectural Requirements for User Plane Protocols . 22 75 5. Evaluation Aspects . . . . . . . . . . . . . . . . . . . . . 28 76 5.1. Supporting PDU Session Type Variations . . . . . . . . . 29 77 5.2. Nature of Data Path . . . . . . . . . . . . . . . . . . . 29 78 5.3. Supporting Transport Variations . . . . . . . . . . . . . 30 79 5.4. Data Path Management . . . . . . . . . . . . . . . . . . 30 80 5.5. QoS Control . . . . . . . . . . . . . . . . . . . . . . . 31 81 5.6. Traffic Detection and Flow Handling . . . . . . . . . . . 32 82 5.7. Supporting Network Slicing Diversity . . . . . . . . . . 32 83 5.8. Reliable Communication support . . . . . . . . . . . . . 33 84 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 33 85 7. Security Consideration . . . . . . . . . . . . . . . . . . . 34 86 8. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 34 87 9. Informative References . . . . . . . . . . . . . . . . . . . 34 88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 38 90 1. Introduction 92 This document analyzes the mobile user plane protocol and the 93 architecture specified by 3GPP 5G documents. The background of the 94 work is that 3GPP requests through a liaison statement that the IETF 95 to provide any information for the User Plane Protocol Study work in 96 3GPP [CP-180116-3GPP]. Justification and the objectives of the study 97 can be found from [CP-173160-3GPP]. 99 We understand that the current user plane protocol, GTP-U 100 [TS.29.281-3GPP], has been well developed in 3GPP, and deployed very 101 widely as the successor of legacy network technologies, such as TDM 102 circuit, or ATM virtual circuit. That GTP-U success seems based on 103 IP overlay technique that is dramatically scaled compare to the 104 previous ones because it successfully isolates mobile session states 105 from the user plane transport network. 107 Even after that big success, it is definitely worth that 3GPP has 108 decided to revisit user plane which seems to response to IPv6 109 deployment growth and [IAB-Statement] that encourages the industry to 110 develop strategies for IPv6-only operation. It can be seen from the 111 justification section in [CP-173160-3GPP]. 113 The study description mentions that the study would be based on 114 Release 16 requirement while only Release 15 specifications has been 115 available now. However we believe that to provide adequate 116 information for 3GPP, we need to clearly understand what the current 117 user plane protocol is in Release 15, and architectural requirements 118 for the user plane. 120 As the liaison statement indicates 3GPP specifications related to 121 user plane, those documents should be a good start point to clarify 122 their specifications and to extract protocol and architectural 123 requirements from them. 125 1.1. Current Status of Mobile User Plane for 5G 127 3GPP RAN and CT4 decided to use GTP-U as the 5G user plane 128 encapsulation protocol over N3 and N9 that respectively described 129 in[TS.38.300-3GPP] and [TR.29.891-3GPP]. N3 is an interface between 130 RAN and UPF and N9 is an interface between different UPFs 131 [TS.23.501-3GPP]. 133 In [TR.29.891-3GPP], it captured user plane requirements and 134 concluded that GTP-U is adopted for the user plane protocol. It 135 seems that GTP-U was only option to be chose and it focused on how to 136 carry 5G specific QoS information between UPF and access networks. 137 That is described in section 5.2 and 11.2 of [TR.29.891-3GPP]. 138 Another aspects of user plane requirements couldn't be found. 140 1.2. Our Way of Analysis Work 142 First, we analyze [TS.29.281-3GPP] for clarifying it as the current 143 user plane protocol in the 5G system. [TR.29.891-3GPP] describes how 144 GTP-U is selected as the user plane protocol for 5G in 3GPP. 145 Clarified characteristics of the protocol are described in Section 3. 147 Then, to clarify what are required to the user plane protocol in 148 architecture level, we analyze [TS.23.501-3GPP] as the 5G system 149 architecture specification. [TS.23.502-3GPP] is the specification of 150 system procedures that helps us to understand how the system works in 151 the architecture. [TS.23.503-3GPP] is also helpful to find the role 152 of user plane in the architecture that influences user plane 153 protocol. Extracted architectural requirements are described in 154 Section 4. 156 Based on the results of above, we identify some aspects where there 157 might be gap between the current user plane protocol and the 158 architectural requirements on which [TR.29.891-3GPP] does not 159 discuss. That aspects are discussed Section 5. That's what we 160 intend to be as a part of the reply to 3GPP. CT4 WG in 3GPP can 161 utilize it as an input to evaluate the candidate protocols for user 162 plane to the 5G system including the current protocol. 164 2. Terms and Abbreviations 166 This section describes terms of functions and interfaces relevant to 167 user plane protocol which we extract from the 3GPP specifications 168 since this document focuses on user plane. 170 In those specifications, there are so many unique terms and 171 abbreviations in the 3GPP context which IETF community seems not 172 familiar with. We will try to bring those terms with brief 173 explanations to make sure common understanding for them. 175 GTP: GPRS Tunneling Protocol 177 GTP-U: User Plane part of GTP 179 Noted that GTP version 1 (GTPv1-U) is the user-plane protocol 180 specification which is defined in [TS.29.281-3GPP]. Unless there 181 is no specific annotation, we refer GTP-U to GTPv1-U in this 182 document. 184 PDU: Protocol Data Unit of end-to-end user protocol packet. 186 Noted that the PDU in 3GPP includes IP header in case that PDU 187 session type is IPv4 or IPv6. In contrast, in IETF it is supposed 188 that PDU is the payload of IP packet so that it doesn't include 189 IP/TCP/UDP header in end-to-end. 191 T-PDU: Transport PDU. 193 G-PDU: GTP encapsulated user Plane Data Unit. 195 GTP-U has above two notions on PDU. T-PDU is a PDU that GTP-U 196 header encapsulates. G-PDU is a PDU that includes GTP-U header. 197 A G-PDU may include a T-PDU. G-PDU can be sent without T-PDU, but 198 just with extension headers or TLV elements. It can be used for 199 OAM related operations. 201 PDU session: Association between the UE and a Data Network that 202 provides a PDU connectivity service. 204 Data Network (DN): The network of operator services, Internet access 205 or 3rd party services. 207 User Plane (UP): Encapsulating user end-to-end PDU. 209 In fact, we can't find exact text that defines UP in the 210 architecture specification. However when we see the figure 211 8.3.1-1 in [TS.23.501-3GPP], we specify UP as the layer right 212 under PDU that directly encapsulates PDU. Underneath layers of UP 213 are UP transport, such as IP/UDP, L2 and L1. 215 However 3GPP is consistent to use the term user plane when they 216 indicate that layer. In IETF, we can see the terms data plane, or 217 forwarding plane as variations which often makes us tend to be 218 confused in terminology. 220 QFI: QoS Flow Identifier 222 UPF: User Plane Function 224 SMF: Session Management Function 226 SMF is a control plane function which provides session management 227 service that handling PDU sessions in the control plane. SMF 228 allocates tunnels corresponding to the PDU sessions and configure 229 the tunnel to the UPF. 231 PFCP: Packet Forwarding Control Protocol 233 PFCP is used on N4 interface between SMF and UPF to configure the 234 rules of packet detection, forwarding action, QoS enforcement, 235 usage report and buffering for each PDU session. 237 PDR: Packet Detection Rule 239 FAR: Fowarding Action Rule 241 RAN: Radio Access Network 243 Noted that UP protocol provides a RAN to connect UPF. But the UP 244 protocol is not appeared on the air in the RAN. 246 3. GTP-U Specification and Observation 248 In this section we analyze the GTP-U specification and summarize 249 clarified characteristic of GTP-U to see if GTP-U meets the 250 requirements of 5G architecture for user plane in later section. 252 3.1. GTP-U Tunnel 254 GTP-U is a tunneling protocol between given a pair of GTP-U tunnel 255 endpoint nodes and encapsulates T-PDU from/to UE on top of IP/UDP. A 256 Tunnel Endpoint Identifier (TEID) value allocated on each end point 257 indicates which tunnel a particular T-PDU belongs to. 259 The receiving endpoint individually allocate a TEID and the sender 260 tunnel endpoint node encapsulates the IP packet from/to UE with the 261 TEID which is present in GTP-U header on top of IPv4 or IPv6, and 262 UDP. That is described in section 4.2.1 of [TS.29.281-3GPP]. 264 [GTP-U-1]: GTP-U is an unidirectional Point-to-Point tunneling 265 protocol. 267 Figure 1 shows an example of GTP-U protocol stack for uplink (UL) and 268 downlink (DL) traffic flow. Two GTP-U tunnels are required to form 269 one bi-directional tunnel. 271 UL: From RAN to UPF1 (TEID=1), and from UPF1 to UPF2 (TEID=2) 273 DL: From UPF2 to UPF1 (TEID=3), and from UPF1 to RAN (TEID=4) 275 In 5GS, GTP-U tunnel is established at following interfaces to 276 provide PDU Session between UE and 5GC. 278 N3: Between RAN and UPF 280 N9: Between different UPFs 282 GTP-U allows one tunnel endpoint node to send out a G-PDU to be 283 received by multiple tunnel endpoints by utilizing IP multicast 284 capability of underlay IP networks. That is described in section 285 4.2.6 of [TS.29.281-3GPP]. It looks GTP-U has Point-to-Multipoint 286 (P2MP) tunneling capability. The P2MP tunneling is used for MBMS 287 (Multimedia Broadcast Multicast Service) through GTP-U tunnel. 289 [GTP-U-2]: GTP-U supports Point-to-Multipoint tunneling. 291 UDP is utilized for GTP-U encapsulation and UDP destination port is 292 2152 which is assigned by IANA. Allocation of UDP source port 293 depends on sender tunnel endpoint node and GTP-U supports dynamic 294 allocation of UDP source port for load balancing objective. The 295 specification of this dynamic allocation is described in section 296 4.4.2.0 of [TS.29.281-3GPP], however specific procedure, e.g., 297 5-tuple hashing, is not described in the document and depends on the 298 implementation of GTP-U tunnel endpoint node. 300 [GTP-U-3]: GTP-U supports load balancing by using dynamic UDP source 301 port allocation. 303 [Uplink] 304 .- .-+--------+ - - - - +--------+ - - - - +--------+ 305 | | |Payload | |Payload | |Payload | 306 | PDU < +--------+ +--------+ +--------+ 307 |(3GPP)| |Inner IP| |Inner IP| |Inner IP| 308 | '-+--------+ - - - - +--------+ - - - - +--------+ 309 | | GTP-U | | GTP-U | | L2 | 310 | |(TEID=1)| |(TEID=2)| +--------+ 311 GTP< +--------+ +--------+ | L1 | 312 Pkt | | UDP | | UDP | +--------+ 313 | +--------+ +--------+ 314 | |Outer IP| |Outer IP| 315 | +--------+ +--------+ 316 | | L2 | | L2 | 317 | +--------+ +--------+ 318 | | L1 | | L1 | 319 '- +--------+ - - - - +--------+ 321 ================ Traffic Direction =======================> 323 [Downlink] 324 .- .-+--------+ - - - - +--------+ - - - - +--------+ 325 | | |Payload | |Payload | |Payload | 326 | PDU < +--------+ +--------+ +--------+ 327 |(3GPP)| |Inner IP| |Inner IP| |Inner IP| 328 | '-+--------+ - - - - +--------+ - - - - +--------+ 329 | | GTP-U | | GTP-U | | L2 | 330 | |(TEID=4)| |(TEID=3)| +--------+ 331 GTP< +--------+ +--------+ | L1 | 332 Pkt | | UDP | | UDP | +--------+ 333 | +--------+ +--------+ 334 | |Outer IP| |Outer IP| 335 | +--------+ +--------+ 336 | | L2 | | L2 | 337 | +--------+ +--------+ 338 | | L1 | | L1 | 339 '- +--------+ - - - - +--------+ 341 <=============== Traffic Direction ======================== 343 +-----+ N3 +------+ N9 +------+ N6 344 -----+ RAN +-----------+ UPF1 +------------+ UPF2 +---------- 345 +-----+ +------+ +------+ 347 Figure 1: Protocol Stack by GTPv1-U for Uplink and Downlink Traffic 348 Flow 350 IPv6 flow label [RFC6437] is also candidate method for load balancing 351 especially for IP-in-IPv6 tunnel [RFC6438] like GTP-U. However, how 352 to use IPv6 flow label of GTP-U is not described in [TS.29.281-3GPP]. 353 Though this method is limited to a case of IPv6 encapsulated GTP-U 354 tunnel, it is worth to study usage of IPv6 flow label in 3GPP. 356 [GTP-U-4]: GTP-U does not support IPv6 flow label for load balancing 357 in case of IPv6 transport. 359 GTP-U supports both IPv4 and IPv6 as underlying transport layer 360 protocol. As for IPv6, GTP-U specification refers [RFC2460], which 361 is described in section 4.2.3 of [TS.29.281-3GPP]. As [RFC2460] does 362 not allow the tunnel protocols on top of UDP to set the checksum 363 value to zero, the GTP-U specification inherits it while the IPv4 364 transport for GTP-U case allows UDP zero checksum. It is noted that 365 the IPv6 specification in IETF has been updated to [RFC8200] which 366 allows UDP zero checksum for the tunnel. [RFC6935] describes 367 benefits of zero checksum for tunnel protocol over UDP. If GTP-U 368 support UFP zero checksum in future version, possible 369 interoperability issue between previous generations which does not 370 support zero checksum may raise. 372 [GTP-U-5]: UDP zero checksum is not available in case of IPv6 373 transport. 375 "Unnecessary fragmentation should be avoided" is recommended and to 376 avoid the fragmentation operator should configure MTU size at UE 377 [TS.29.281-3GPP]. However, there's no reference and specification of 378 Path MTU Discovery for IPv6 transport. If encapsulated IPv6 packet 379 is too big on a network link between tunnel endpoint nodes, UE may 380 not receive ICMPv6 Packet Too Big message and causes Path MTU 381 Discovery black hole. 383 [GTP-U-6]: GTP-U does not support to response ICMP PTB for Path MTU 384 Discovery. 386 Section 9.3 of [TS.23.060-3GPP] specifies advertisement of inner IPv6 387 link MTU size for UE by IPv6 RA message [RFC4861]. However, this 388 document doesn't specify a procedure to measure MTU size in mobile 389 network system and mobile network operator need to calculate MTU size 390 for UE like Annex C of [TS.23.060-3GPP]. If link MTU of a router in 391 a transport network is accidentally modified, UE cannot detect the 392 event and send packet with initial MTU size, which may cause service 393 disruption due to MTU exceed in the router link. 395 3.2. GTP-U Header Format 397 Figure 2 shows general and mandatory GTP-U header and Figure 3 shows 398 extension GTP-U header. 400 [GTP-U-7]: GTP-U supports sequence number option in the header, but 401 it is not recommended to be used by almost GTP-U 402 entities. 404 GTP-U header has Sequence Number field to reorder incoming packets 405 based on the sequence number. If Sequence Number Flag is set to '1' 406 it indicates that Sequence Number Filed exists in GTP-U header and 407 examined at receiving tunnel endpoint node to reorder incoming 408 packets. However, the sequence number flag is set to '1' only for 409 RAT HO procedure and sequence number flag should be set to '0' in 410 normal case. Therefore, in normal case receiver tunnel endpoint node 411 doesn't examine sequence number and can't reorder GTP-U packets based 412 on the sequence number. This specification is described in section 413 5.1 of [TS.29.281-3GPP]. In 3GPP, sequential delivery is required 414 only during handover procedure and is used by only RAN entities. 416 0 1 2 3 417 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 418 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 419 | Ver |P|R|E|S|N| Message Type | Length | 420 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 421 | Tunnel Endpoint Identifier | 422 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 423 | Sequence Number | N-PDU Number | Next-Ext-Hdr | 424 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 426 Figure 2: GTP-U Header 428 o Ver: Version field (Set to '1') 430 o P: Protocol Type (Set to '1') 432 o R: Reserved bit (Set to '0') 434 o E: Extension Header Flag (Set to '1' if extension header exists) 436 o S: Sequence Number Flag (Set to '1' if sequence number exists) 438 o N: N-PDU Number Flag (Set to '1' if N-PDU number exists) 440 o Message Type: Indicates the type of GTP-U message 442 o Length: Indicates the length in octets of the payload 443 o Tunnel Endpoint Identifier (TEID) 445 o Sequence Number: Indicates increasing sequence number for T-PDUs 446 is transmitted via GTP-U tunnels 448 o N-PDU Number: It is used only for inter SGSN, 2G-3G handover case, 449 etc. 451 o Next-Ext-Hdr: Indicates following extension header type 453 0 1 2 3 454 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 455 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 456 | Ext-Hdr Length| | 457 +-+-+-+-+-+-+-+-+ | 458 | Extension Header Content | 459 . . 460 . +-+-+-+-+-+-+-+-+ 461 | | Next-Ext-Hdr | 462 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 464 Figure 3: Extension GTP-U Header 466 o Ext-Hdr Length: Represents the length of the Extension header in 467 units of 4 octets 469 o Extension Header Content: Contains 3GPP related information 471 o Next-Ext-Hdr: Indicates following extension header type 473 The extension GTP-U header is a variable-length and extendable header 474 and contains 3GPP specific information. Following list summarizes 475 every extension header which is used for user plane protocol. These 476 extension headers are defined in [TS.29.281-3GPP]. In this list 477 Next-Ext-Hdr is represented in binary. 479 o No more extension headers (Next-Ext-Hdr = 00000000) 481 o Service Class Indicator (Next-Ext-Hdr = 00100000) 483 o UDP Port (Next-Ext-Hdr = 01000000) 485 o RAN Container (Next-Ext-Hdr = 10000001) 487 o Long PDCP PDU Number (Next-Ext-Hdr = 10000010) 489 o Xw RAN Container (Next-Ext-Hdr = 10000011) 490 o NR RAN Container (Next-Ext-Hdr = 10000100) 492 o PDU Session Container (Next-Ext-Hdr = 10000101) 494 o PDCP PDU Number (Next-Ext-Hdr = 11000000) 496 [GTP-U-8]: GTP-U supports carrying QoS Identifiers transparently for 497 Access Networks in an extension header. 499 GTP-U is designed to carry 3GPP specific information with extension 500 headers. 3GPP creates PDU Session Container extension header for 501 NGRAN of 5G to carry QFI. It is described in section 5.2.2.7 of 502 [TS.29.281-3GPP]. 504 [GTP-U-9]: GTP-U supports DSCP marking based on the QFI. 506 DSCP marking on outer IPv4 or IPv6 shall be set by sender tunnel 507 endpoint node based on the QFI. This specification is described in 508 section 4.4.1 of [TS.29.281-3GPP]. 510 [GTP-U-10]: GTP-U does not specify extension header order. 512 In general, multiple GTP-U extension headers are able to contained in 513 one GTP-U packet and the order of those extension headers is not 514 specified by [TS.29.281-3GPP]. Thereby the receiving endpoint can't 515 predict exact position where the target extension headers are. This 516 could impact on header lookup performance on the node. 518 As for PDU Session Container extension header, there is a note in 519 [TS.29.281-3GPP] as "For a G-PDU with several Extension Headers, the 520 PDU Session Container should be the first Extension Header". This 521 note was added at the version 15.3.0 of [TS.29.281-3GPP] which is 522 published on June 2018 in order to accelerate the processing of GTP-U 523 packet at UPF and RAN. It is only one rule regarding the extension 524 header order. 526 [GTP-U-11]: GTP-U does not support to indicate next protocol type. 528 When Next-Ext-Hdr is set to 0x00 it indicates that no more extension 529 headers follow. As GTP is designed to indicate protocol types for 530 T-PDU by control-plane signaling, GTP-U doesn't have Next-Protocol- 531 Header field to indicate the T-PDU type in the header. 533 3.3. Control Plane Protocol for GTP-U 535 Control plane protocol for GTP-U signals TEID between tunnel endpoint 536 nodes. GTPv2-C [TS.29.274-3GPP] is the original control plane 537 protocol tied with GTP-U in previous generation architectures before 538 CUPS (Control and User Plane Separation). 540 3GPP decided to use extended PFCP (Packet Forwarding Control 541 Protocol) [TS.29.244-3GPP] for N4 interface [TR.29.891-3GPP] to 542 signal tunnel states from SMF to UPF. 544 3.4. GTP-U message 546 GTP-U supports in-band messaging to signal OAM. Currently GTP-U 547 supports following messages [TS.29.281-3GPP]. 549 o Echo Request 551 o Echo Response 553 o Supported Extension Headers Notification 555 o Error Indication 557 o End Marker 559 [GTP-U-12]: GTP-U supports active OAM as a path management message 560 "Echo Request/Response". 562 A GTP-U tunnel endpoint node sends a Echo Request message to another 563 nodes for keep-alive and received node sends a Echo Response message 564 to sender node as acknowledgment. Echo Request message and Echo 565 Response message are described in section 7.2.1 and section 7.2.2 of 566 [TS.29.281-3GPP] respectively. [TR.29.891-3GPP] recommends not to 567 send Echo Request message more often than 60s on each path. 569 Supported Extension Headers Notification message indicates a list of 570 supported that tunnel endpoint node can support. This message is 571 sent only in case a tunnel endpoint node receives GTP-U packet with 572 unsupported extension header. 574 [GTP-U-13]: GTP-U supports tunnel management messages "Error 575 Indication". 577 GTP-U has Error Indication message to notify that the receiving 578 endpoint discard packets of which no session exist to the sending 579 endpoint. Error Indication message is described in section 7.3.1 of 580 [TS.29.281-3GPP]. 582 [GTP-U-14]: GTP-U supports tunnel management messages "End Marker". 584 GTP-U has End Marker message to indicate the end of the payload 585 stream that needs to be sent on a GTP-U tunnel. End Marker message 586 is described in section 7.3.2 of [TS.29.281-3GPP]. 588 3.5. Packet Format 590 Figure 4 shows a packet format example of GTP-U over IPv6 that 591 carries an extension header for QFI and an IPv6 PDU. All values in 592 the example are illustration purpose only. The encoding of PDU 593 Session Container for QFI refers to [TS.38.415-3GPP]. 595 Outer IPv6 Header's DSCP value(EF) in Figure 4 is marked at sender 596 tunnel endpoint node based on QFI value which is contained in GTP-U 597 Extension Header (PDU Session Container). 599 0 1 2 3 600 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 601 Outer IPv6 Header 602 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 603 |Version| DSCP=EF | Flow Label | 604 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 605 | Payload Length | NxtHdr=17(UDP)| Hop Limit | 606 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 607 | | 608 + + 609 | | 610 + Source IPv6 Address + 611 | 2001:db8:1:1::1 | 612 + + 613 | | 614 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 615 | | 616 + + 617 | | 618 + Destination IPv6 Address + 619 | 2001:db8:1:2::1 | 620 + + 621 | | 622 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 624 Outer UDP Header 625 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 626 | Source Port = xxxx | Dest Port = 2152 | 627 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 628 | UDP Length | UDP Checksum (Non-zero) | 629 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 631 GTP-U header 632 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 633 | 0x1 |1|0|1|0|0| 0xff | Length | 634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 635 | TEID = 1654 | 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 | Sequence Number = 0 |N-PDU Number=0 |NextExtHdr=0x85| 638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 640 GTP-U Extension Header (PDU Session Container) 641 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 642 | ExtHdrLen=2 |Type=0 | Spare |0|0| QFI | PPI | Spare | 643 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 644 | Padding |NextExtHdr=0x0 | 645 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 647 Inner IPv6 Header 648 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 649 |Version| DSCP=0 | Flow Label | 650 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 651 | Payload Length | NexttHdr | Hop Limit | 652 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 653 | | 654 + + 655 | | 656 + Source IPv6 Address + 657 | 2001:db8:2:1::1 | 658 + + 659 | | 660 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 661 | | 662 + + 663 | | 664 + Destination IPv6 Address + 665 | 2001:db8:3:1::1 | 666 + + 667 | | 668 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 670 Payload 671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 672 | | 673 | | 674 . TCP/UDP/etc., Data . 675 . . 676 | | 677 | | 678 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 679 Figure 4: GTP-U Protocol Stack Example 681 3.6. Observations Summary 683 [GTP-U-1]: An unidirectional Point-to-Point tunneling protocol. 685 [GTP-U-2]: Supports Point-to-Multipoint tunneling. 687 [GTP-U-3]: Supports load balancing by using dynamic UDP port 688 allocation. 690 [GTP-U-4]: Does not support IPv6 flow label for load balancing in 691 case of IPv6 transport. 693 [GTP-U-5]: UDP zero checksum is not available in case of IPv6 694 transport. 696 [GTP-U-6]: Does not support to response ICMP PTB for Path MTU 697 Discovery. 699 [GTP-U-7]: Supports sequence number option and sequence number flag 700 in the header, but it is not recommended to be used by 701 almost GTP-U entities. 703 [GTP-U-8]: Supports carrying QoS Identifiers transparently for 704 Access Networks in extension headers. 706 [GTP-U-9]: Supports DSCP marking based on the QFI. 708 [GTP-U-10]: Does not specify the rule for the extension header order. 710 [GTP-U-11]: Does not support an indication of next-header type. 712 [GTP-U-12]: Supports active OAM as a path management message "Echo 713 Request/Response". 715 [GTP-U-13]: Supports tunnel management messages "Error Indication". 717 [GTP-U-14]: Supports tunnel management messages "End Marker". 719 4. 5GS Architectural Requirements for User Plane Protocols 721 4.1. Overview of 5G System Architecture 723 The 5G system is designed for applying to diverse devices and 724 services due to factors such as the diffusion of IoT devices, and the 725 UP protocol is required to have capabilities for satisfying their 726 requirements. 728 As a principle of the 5G system, User Plane (UP) functions are 729 separated from the Control Plane (CP) functions for allowing 730 independent scalability, evolution and flexible deployments. 732 Network slicing is also one of the fundamental concepts of the 5G 733 system, and it provides logical network separation. In terms of user 734 plane, multiple network slices can be comprised of UPFs on top of 735 same physical network resources. Allocated resources and structures 736 may be differentiated among the slices by which the required features 737 or capabilities. 739 The 3GPP 5G architecture [TS.23.501-3GPP] defines slice types which 740 are eMBB, URLLC and MIoT from Rel-15. In addition to that, V2X slice 741 type is defined from Rel-16. 743 The architecture overview is shown in Figure 5. The details of 744 functions are described in [TS.23.501-3GPP]. A UPF handles UP paths 745 on N3, N9 and N6 interface, and the setup is controlled by SMF via N4 746 interface. A UP path will be manipulated based on application 747 requirements for the PDU session corresponding to the path. An SMF 748 is also capable to receive information regarding routing path with 749 API from AF via NEF, PCF, and SMF. 751 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 752 |NSSF | | NEF | | NRF | | PCF | | UDM | | AF | 753 +--+--+ +--+--+ +--+--+ +--+--+ +--+--+ +--+--+ 754 Nnssf| Nnef| Nnrf| Npcf| Nudm| |Naf 755 ---+--------+--+-----+--+--------+--+-----+--------+- 756 Nausf| Namf| Nsmf| 757 +--+--+ +--+--+ +--+-------+ 758 |AUSR | | AMF | | SMF | 759 +-----+ ++-+--+ +--+-----+-+ 760 / | | \ 761 Control Plane N1/ |N2 |N4 \N4 762 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 763 User Plane / | | \ 764 +--++ +--+--+ N3 +--+--+ N9 +-+---+ N6 +-----+ 765 |UE +--+(R)AN+-----+ UPF +-----+ UPF +----+ DN | 766 +---+ +-----+ +--+--+ +-----+ +-----+ 767 |N6 768 +--+--+ 769 | DN | 770 +-----+ 772 Figure 5: 5GS Architecture and Service-based Interfaces 774 This document mainly focuses on requirements for N9 interface as 775 relevant to UP protocol of 5G system. 777 4.1.1. UPF Functionalities 779 UPF has a role to handle UP traffic, and provides functionalities to 780 look up user data traffic and enforce the appropriate policies to it. 782 The followings are defined as UPF functionalities defined in the 783 section 6.2.3 of [TS.23.501-3GPP] 785 o Anchor point for Intra-/Inter-RAT mobility (when applicable). 787 o External PDU Session point of interconnect to Data Network. 789 o Packet routing and forwarding (e.g. support of Uplink classifier 790 to route traffic flows to an instance of a data network, support 791 of Branching point to support multi-homed PDU Session). 793 o Packet inspection (e.g. Application detection based on service 794 data flow template and the optional PFDs received from the SMF in 795 addition). 797 o User Plane part of policy rule enforcement, e.g. Gating, 798 Redirection, Traffic steering). 800 o Lawful intercept (UP collection). 802 o Traffic usage reporting. 804 o QoS handling for user plane, e.g. UL/DL rate enforcement, 805 Reflective QoS marking in DL. 807 o Uplink Traffic verification (SDF to QoS Flow mapping). 809 o Transport level packet marking in the uplink and downlink. 811 o Downlink packet buffering and downlink data notification 812 triggering. 814 o Sending and forwarding of one or more "end marker" to the source 815 NG-RAN node. 817 o ARP proxying and / or IPv6 Neighbour Solicitation Proxying for the 818 Ethernet PDUs. 820 o Packet duplication in downlink direction and elimination in uplink 821 direction in UP protocol layer. 823 o TSN Translator functionality to hold and forward user plane 824 packets for de-jittering when 5G System is integrated as a bridge 825 with the TSN network. 827 4.1.2. UP Traffic Detection 829 The traffic detection is described in the section 5.8.2.4 of 830 [TS.23.501-3GPP]. In 3GPP UP packet forwarding model, UPF detects UP 831 traffic flow which belong to a N4 session configured by SMF. 833 The protocol of N4 interface, PFCP, brings a set of traffic detection 834 information from SMF to UPF as Packet Detection Information (PDI) in 835 a PDR to establish/modify the N4 PFCP session. It is defined in 836 section 7.5.2.2 of [TS.29.244-3GPP]. 838 Combination of the following information is used for the traffic 839 detection: 841 o For IPv4 or IPv6 PDU Session type 843 * CN tunnel info (Tunnel ID and the endpoint IP address of 5G 844 Core) 846 * Network instance 848 * QFI 850 * IP Packet Filter Set 852 * Application Identifier: The Application ID is an index to a set 853 of application detection rules configured in UPF 855 o For Ethernet PDU Session type 857 * CN tunnel info(Tunnel ID and the endpoint IP address of 5G 858 Core) 860 * Network instance 862 * QFI 864 * Ethernet Packet Filter Set 866 It is noted that Network Instance is encoded as Octet String in PFCP, 867 and is NOT appeared in UP packet over the wire. It is expected like 868 an attribute of the receiving IP interface of the UPF. It supports 869 UPF to be able to connect to different IP domains of N3, N9 or N6, 870 which run each independent policy in routing and addressing. The UPF 871 detects traffic flow with Network Instance which the receiving 872 interface attributed to. 874 The IP Packet Filter Set and Ethernet Packet Filter Set defined in 875 clause 5.7.6 of [TS.23.501-3GPP] are following: 877 o IP Packet Filter Set: 879 * Source/destination IP address or IPv6 prefix 881 * Source/destination port number 883 * Protocol ID of the protocol above IP/Next header type 885 * Type of Service (TOS) (IPv4) / Traffic class (IPv6) and Mask. 887 * Flow Label (IPv6) 889 * Security parameter index 891 * Packet filter direction 893 o Ethernet Packet Filter Set: 895 * Source/destination MAC address 897 * Ethertype as defined in IEEE 802.3 899 * Customer-VLAN tag(C-TAG) and/or Service-VLAN tag(S-TAG) VID 900 fields as defined in IEEE 802.1Q 902 * Customer-VLAN tag(C-TAG) and/or Service-VLAN tag(S-TAG) PCP/DEI 903 fields as defined in IEEE 802.1Q 905 * IP Packet Filter Set, in case Ethertype indicates IPv4/IPv6 906 payload 908 * Packet filter direction 910 4.1.3. User Plane Configuration 912 User Plane configuration on a UPF is managed by an SMF through PFCP 913 [TS.29.244-3GPP]. The SMF establishes PFCP sessions on the UPF per 914 PDU session basis. The UPF maintains each configured PFCP session 915 states during the sessions exist. 917 A PFCP session consists of the rules of packet detection, forwarding 918 action, QoS enforcement, usage reporting and buffering action. 919 Figure 6 depicts overview of the PFCP session state structure. 921 The listed information in Section 4.1.2 indicates packet detection 922 information of packet detection rule for that the rest of related 923 rules within the PFCP session to be derived. All rules are per 924 session unique and no rules are shared with other sessions. 926 PFCP-Session* [F-SEID] 927 +- F-SEID(Full Qualified Session Endpoint ID) uint64 928 +- PDU-Session-Type [IPv4|IPv6|IPv4v6|Ether|Unstrct] 929 +- DNN(Data Network Name) 930 +- PDR(Packet Detection Rule)* [PDR-ID] 931 | +- PDR-ID uint16 932 | +- PDI (Packet Detection Information) 933 | | +- Traffic-Endpoint-ID? -> Traffic-Endpoint-ID reference 934 | | +- .... 935 | +- FAR/URR/QER-ID -> FAR/URR/QER-ID references 936 +- FAR(Forwarding Action Rule)* [FAR-ID] 937 | +- FAR-ID uint32 938 | +- Forwarding-Parameters 939 | | +- Network-Instance? Octet String 940 | | +- Outer-Header-Creation 941 | | | +- Outer-Hdr-Creation-Desc [GTPoUDP/IPv4|IPv6, etc.,] 942 | | | +- TEID, outer IP-Address for N3/N9 943 | | | +- C/S-TAG, UDP Port-number for N6 944 | | +- Forwarding-Policy-ID? Octet String 945 | | +- .... 946 | +- Duplicating-Parameters 947 | | +- .... 948 | +- BAR-ID? -> BAR-ID reference 949 +- QER(QoS Enforcement Rule)* [QER-ID] 950 | +- QER-ID uint32 951 | +- MBR(Maximum Bit Rate) 952 | | +- UL/DL-MBR? bitrate_in_kbps (0..10000000) 953 | +- GBR(Guaranteed Bit Rate) 954 | | +- UL/DL-GBR? bitrate_in_kbps (0..10000000) 955 | +- QoS-flow-identifier? QFI value(6-bits) 956 | +- Reflective-QoS? boolean 957 | +- Paging-Policy-Indicator? PPI value(3-bits) 958 | +- .... 959 +- URR(Usage Reporting Rule)* [URR-ID] 960 | +- URR-ID uint32 961 | +- Measurement-Method, Period, Reporting-Triggers? 962 | +- Volume/Event/Time Threshold, Quota? 963 | +- Quota-Holding-Time? 964 | +- FAR-ID for Quota action? -> FAR-ID reference 965 | +- .... 966 +- BAR(Buffering Action Rule)* [BAR-ID] 967 | +- BAR-ID uint8 968 | +- Suggested-Buffering-Packets-Count 969 +- Traffic-Endpoint* [Traffic-Endpoint-ID] 970 +- Traffic-Endpoint-ID uint8 971 +- TEID, Tunnle IP Address, UE Address...? 973 Figure 6: User Plane Configuration Model 975 4.2. Architectural Requirements for User Plane Protocols 977 This section lists the requirements for the UP protocol on the 5G 978 system. The requirements are picked up from [TS.23.501-3GPP]. In 979 addition, some of service requirements described in [TS.22.261-3GPP] 980 are referred to clarify the originations of architectural 981 requirements. 983 According to [TS.23.501-3GPP], the specifications potentially have 984 assumptions that the UP protocol is a tunnel representing a single 985 TEID between a pair of UPFs and it is corresponding to a single PDU 986 session. In short, the UP protocol is a tunnel and it is assumed to 987 be managed under per PDU session handling. Also, it should be a 988 stateful tunnel in the UPFs along with the PDU session. 990 The requirements for UP protocols are described below: 992 ARCH-Req-1: Supporting IPv4, IPv6, Ethernet and Unstructured PDU 994 The 5G system defines four types of PDU session as IPv4, IPv6, 995 Ethernet, and Unstructured. Therefore, UP protocol must support to 996 convey all of these PDU session types. This is described in 997 [TS.23.501-3GPP]. 999 Note: In TS 23.501 v15.2.0, IPv4v6 is added as a PDU session type. 1001 ARCH-Req-2: Supporting IP connectivity for N3, N6, and N9 interfaces 1003 The 5G system requires IP connectivity for N3, N6, and N9 interfaces. 1004 The IP connectivity is assumed that it comprises of IP routing and 1005 L1/L2 transport networks which are outside of 3GPP specifications. 1007 It is desirable that the IP connectivity built on IPv6 networks when 1008 it comes to address space for end-to-end user plane coverage. But it 1009 is expected to take certain time. During the IPv6 networks are not 1010 deployed for all the coverage, UP protocol should support RANs and 1011 DNs running on IPv4 transport connect to UPF running on IPv6 1012 transport. 1014 Furthermore, on N6 interface, point-to-point tunneling based on UDP/ 1015 IPv6 may be used to deliver unstructured PDU type data. Then, the 1016 content information of the PDU may be mapped into UDP port number, 1017 and the UDP port numbers is pre-configured in the UPF and DN. This 1018 is described in the section 9.2 of [TS.29.561-3GPP]. 1020 ARCH-Req-3: Supporting deployment of multiple UPFs as anchors for a 1021 single PDU session 1023 The 5G system allows to deploy multiple UPFs as anchors for a single 1024 PDU session, and supports multihoming of a single PDU session for 1025 such anchor UPFs. 1027 Multihoming is provided with Branching Point (BP). BP provides 1028 forwarding of UL traffic towards the different PDU Session Anchors 1029 based on the source IPv6 prefixes and merge of DL traffic to the UE. 1030 IPv6 multihoming only means multiple source IPv6 prefixes are used 1031 for a PDU session. It is identical to one classified as scenario 1 1032 in [RFC7157]. 1034 Up link classifier (UL CL) is to forward uplink packets to multiple 1035 anchor UPFs based on the destination IP of the T-PDU regardless of 1036 the source IP address. Noted that single source IP address/prefix 1037 PDU session is not defined as multihoming PDU session in 5GCS even 1038 though a PDU session has multiple anchor UPFs. 1040 On UL side, P2P tunnels are established per destination anchor UPFs 1041 basis from one UL CL UPF to the anchor UPFs for the PDU session. 1043 On DL side, one single multipoint-to-point (MP2P) tunnel exists from 1044 the source anchor UPFs to the destination BP UPF for the PDU session. 1045 It means that the paths from the anchor UPFs are merged into just one 1046 tunnel state at the destination BP UPF. 1048 Multiple P2P paths on DL could also be used for multihoming. However 1049 it should be the multiple PDU sessions multihoming case where the 1050 destination gNB or UPF needs to maintain multiple tunnel states under 1051 the one PDU session to one UP tunnel architectural principle. It 1052 causes increase of load on tunnel states management in UPF due to 1053 increment of the anchor UPF for the PDU session. 1055 However, P2P tunneling could increase explosively the number of 1056 states in UPF as the anchor UPF/DN incremented to the PDU session. 1057 Thereby single PDU session multihoming with MP2P path should be a 1058 better option for multihoming in terms of reducing total number of 1059 tunnel states. 1061 SSC mode 3 for session continuity in hand-over case uses a single PDU 1062 multihoming with BP to make sure make-before-break. It is described 1063 in the section 5.6.4 and 5.6.9 of [TS.23.501-3GPP]. 1065 Multihoming is also assumed to be used for edge computing scenario. 1066 Edge computing enables some services to be hosted close to the UE's 1067 access point of attachment, and achieves an efficient service 1068 delivery through the reduced end-to-end latency and load on the 1069 transport network. In edge computing, local user's traffic is routed 1070 or steered to application in the local DN by UPF. This refers the 1071 section 5.13 of [TS.23.501-3GPP]. 1073 ARCH-Req-4: Supporting flexible UPF selection for PDU 1075 The appropriate UPFs are selected for a PDU session based on 1076 parameters and information such as UPF's dynamic load or UE location 1077 information. Examples of parameters and information are described in 1078 the section 6.3.3 of [TS.23.501-3GPP]. 1080 This means that it is possible to make routing on user plane more 1081 efficient in the 5GS. For example, in case that UPFs are distributed 1082 geographically, decision of the destination UPF based on locations of 1083 end hosts (e.g., UE or NF in DN) enables to forward PDUs with a route 1084 connecting between UPFs nearby the hosts directly. This would be 1085 useful UE-to-UE or UE-to-local_DN communication, and such usage is 1086 described in the section 6.5 of [TS.22.261-3GPP]. 1088 The 5GS allows operators to select parameters used for UPF selection. 1089 (In other words, any specific schemes on UPF selection are not 1090 defined in the current 3GPP documents.) 1092 ARCH-Req-5: No limitation for number of UPFs in a data path 1094 The number of UPF in the data path is not constrained by 3GPP 1095 specifications. This specification is described in the section 8.3.1 1096 of [TS.23.501-3GPP]. 1098 Putting multiple UPFs, which provides specific function, in a data 1099 path enables flexible function deployment to make sure load 1100 distribution optimizations, etc. 1102 Meanwhile, each UPF in a data path shall be controlled by an SMF via 1103 N4 interface. Thus putting an excess of UPF for data paths might 1104 cause increase of load of an SMF. Pragmatically, the number of UPF 1105 put in a data path is one or two (e.g., for MEC or roaming cases), 1106 and, at most, it would be three (e.g., for case where UE moves during 1107 a session). 1109 It is expected that multiple UPFs with per session tunnel handling 1110 for a PDU session becomes complicated task more and more for a SMF by 1111 increasing number of UPFs. 1113 ARCH-Req-6: Supporting aggregation of multiple QoS Flow indicated 1114 with QFI into a PDU Session 1116 Against to the previous generation, 5G enables UPF to multiplex QoS 1117 Flows, equivalent with IP-CAN bearers in the previous generation, 1118 into one single PDU session. That means that a single tunnel 1119 includes multiple QFIs contrast to just one QoS Flow (a bearer) to 1120 one tunnel before 5G. 1122 In even the 5GS, each flow is forwarded based on the appropriate QoS 1123 rules. QoS rules are configured by SMF as QoS profiles to UP 1124 components and these components perform QoS controls to PDUs based on 1125 rules. In downlink, a UPF pushes QFI into an extension header, and 1126 transmits the PDU to RAN or another UPF. Then, such UPF may perform 1127 transport level QoS packet marking (e.g., DSCP marking in the outer 1128 header). In uplink, each UE obtains the QoS rule from SMF, and 1129 transmit PDUs with QFI containing the QoS rules to the RAN. The 1130 following RAN and UPFs perform enforcement of QoS control and 1131 charging based on the QFI. 1133 This specification is described in 5.7.1 of [TS.23.501-3GPP]. 1135 ARCH-Req-7: Supporting network slicing 1137 The 5GS fundamentally supports network slicing for provision the 1138 appropriate end-to-end communication to various services. In the 1139 relevant documents (e.g., [TS.23.501-3GPP], [TS.28.530-3GPP]), a 1140 network slice is defined as virtual network and it is structured with 1141 5GS NF instances, such as SMF, UPF including IP transport 1142 connectivity between RANs and DNS. Each network slice is independent 1143 and its user plane (including network functions and links) should be 1144 noninteractive against the others. 1146 The 5G architecture specification has been updated with that Network 1147 Instance is defined as the glue of network slice between 5G slice and 1148 corresponding IP transport slice in addition to the original role of 1149 separating IP domains, which is described in Section 4.1.2. 1151 It has been appeared from version 15.2.0 of [TS.23.501-3GPP] in 1152 section 5.6.12. 1154 UP underlay transport networks and UPFs may be shared by 5G slices, 1155 as described in section 4 of [TS.28.530-3GPP]. The data model 1156 defined in [TS.29.510-3GPP] allows that a Network Instance, a UPF and 1157 its interfaces can belong to multiple slices as same as other type of 1158 NFs. UP endpoint IP prefix/address of an interface can also be 1159 shared with multiple interfaces on the UPF as the model doesn't make 1160 them slice unique. 1162 The slice lifecycle managements is described in the relevant 1163 documents: [TS.28.531-3GPP], [TS.28.532-3GPP], and [TS.28.533-3GPP]. 1165 ARCH-Req-8: End Marker support 1167 The construction of End Marker packets specified in [TS.23.501-3GPP] 1168 may either be done in the CP/UP functions for indicating the end of 1169 the payload stream on a given UP tunnel. PDU packets arrive after an 1170 End Marker message on the tunnel may be silently discarded. For 1171 example, End Maker is used for handover procedures, and it can 1172 prevent reordering of arriving packets due to switch of anchor UPFs. 1174 ARCHI-Req-9: Supporting URLLC 1176 The 5G is expected to support services which are latency sensitive 1177 and require high reliability. Communication to realize such services 1178 is called Ultra-Reliable and Low-Latency Communication or URLLC. In 1179 URLLC, redundancy of QoS flows is required for providing highly 1180 reliable communication. For instance, a set of UP NFs (e.g., UPF or 1181 gNB) and interfaces between UE and DN are redundant, and packets are 1182 replicated and forwarded via each route. UEs and DN support dual 1183 connectivity and drop duplicated received packets. The scheme of 1184 packet dropping at UE is out of responsibility of 3GPP. The overview 1185 is shown in Figure 7. 1187 ---+-----------+----------+--- 1188 Namf| Nsmf| Nsmf| 1189 +--+--+ +--+--+ +--+--+ 1190 | AMF | | SMF1| | SMF3| 1191 +-+---+ +--+--+ +-+---+ 1192 / / / 1193 N2/ N4/ N4/ 1194 / / / 1195 +----++ N3 +--+---+ / N6 +----+ 1196 |M-RAN+--------+ UPF1 +--------------+ | 1197 ++-+--+ +-+--+-+ / | | 1198 / | | | / | | 1199 +----+ / | +--+ / | | 1200 | UE +< |Xn N9 / | DN | 1201 +----+ \ | / | | 1202 \ | / | | 1203 ++-+--+ N3 +----+-+ N6 | | 1204 |S-RAN+--------+ UPF2 +--------------+ | 1205 +-----+ +-+--+-+ +----+ 1206 | | 1207 +--+ 1208 N9 1209 *Legends 1210 M-RAN: Master RAN 1211 S-RAN: Secondary RAN 1213 Figure 7: Redundant UP paths using dual connectivity 1215 Otherwise, in case that RAN nodes and UPFs have enough reliability 1216 and they are not redundant by dual devices, reliable connectivity of 1217 QoS flows is provided by dual N3 tunnels between RAN and UPFs. Such 1218 tunnels are treated as individual ones, but they have the same 1219 sequence number. UP NFs identifies the duplication of PDU packets 1220 based on sequence number content in the UP tunnel headers. For 1221 uplink packets, a RAN node replicates each packet from a UE. An 1222 anchor UPF receives the duplicated packets, and drops ones which 1223 reach later in each duplicated packet pare. On the other hand, for 1224 downlink packets, a UPF replicates packets received from DN, and a 1225 RAN node drops the duplicated packets as well. The overviews of the 1226 ways are shown in Figure 8. 1228 ----+-----------+-----------+--- 1229 Namf| Nsmf| Npcf| 1230 +--+--+ +--+--+ +--+--+ 1231 | AMF | | SMF | | PCF | 1232 +-+---+ +--+--+ +-----+ 1233 / | 1234 N2/ N4| 1235 / N3 Tunnel1 | 1236 +----+ +--+--+__________+--+--+ N6 +----+ 1237 | UE +----+ RAN |__________| UPF |-----+ DN | 1238 +----+ +-----+ +-----+ +----+ 1239 N3 Tunnel2 1241 Figure 8: Redundant UP transmission with two N3 tunnels 1243 In addition, there is a case that two intermediate UPFs (I-UPFs) 1244 between anchor UPF and RAN are used to support the redundant 1245 transmission based on two N3 and N9 tunnels between single anchor UPF 1246 and RAN node. The RAN node and anchor UPF support the packet 1247 replication and dropping of duplicated packets as described above. 1248 As described above, anchor UPF and RAN node detect packet duplication 1249 with sequence number of UP tunnels, and thus I-UPFs would forward the 1250 packets with the same sequence number on N3 and N9 tunnels. The 1251 overview is shown in Figure 9. 1253 ----+-------------+--------------+--- 1254 Namf| Nsmf| Npcf| 1255 +--+--+ +---+---+ +--+--+ 1256 | AMF | | SMF + | PCF | 1257 +--+--+ +-+-+-+-+ +-----+ 1258 / | | | 1259 N2 / N4| | +----------------+ 1260 / | | | 1261 / N3 Tunnel1 +-+-+---+ N9 Tunnel1 | 1262 / +-----+I-UPF1 +----+ | 1263 +----+ +--+--+______| +-------+ |______+--+--+ N6 +----+ 1264 | UE +---+ RAN |______ | ______| UPF +-----+ DN | 1265 +----+ +-----+ N3 | +---+---+ | N9 +-----+ +----+ 1266 +-----+I-UPF2 +----+ 1267 N3 Tunnel2 +-------+ N9 Tunnel2 1269 Figure 9: Redundant UP transmission with two I-UPF and N3/N9 tunnels 1271 5. Evaluation Aspects 1273 This section provides UP protocol evaluation aspects that are mainly 1274 we derived from the architectural requirements described in 1275 Section 4. Those aspects are not prioritized by the order here. 1277 Expected deployment scenarios explain the evaluations purpose in the 1278 corresponding aspects. 1280 As we were noticed that the gaps between GTP-U specifications and 5G 1281 architectural requirements through the analysis, those each gap are 1282 briefly described in the evaluation aspect associated to it. 1284 Since it is obvious that 5G system should be able to interwork with 1285 existing previous generation based systems, any aspects from 1286 coexisting and interworking point of view are not particularly 1287 articulated here. It may be described in a next version. 1289 5.1. Supporting PDU Session Type Variations 1291 Given that UP protocol is required to support all PDU session types: 1292 IPv4, IPv6, Ethernet, and Unstructured. However, it is expected that 1293 some deployment cases allow candidate protocol to adopt only one or 1294 few PDU session type(s) for simplicity of operations. As we can 1295 expect that IPv4 connectivity services will be available through 1296 IPv6-only PDU session that enabled by bunch of IPv6 transition 1297 solutions already available in the field. 1299 For this, the expected evaluation points from this aspect should be 1300 whether there is substitutional means to cover other PDU session 1301 types. And how much it makes simple the system than deploying 1302 original PDU session types. 1304 5.2. Nature of Data Path 1306 As it is described in Section 4.2, the single PDU session multi- 1307 homing case requires multipoint-to-point (MP2P) data path. It should 1308 be much scalable than multi-homing with multiple PDU sessions because 1309 number of required path states in the UPFs are reduced as closed to 1310 egress endpoint. Against that point-to-point (P2P) protocol requires 1311 same number of states in each UPF throughout the path, and it could 1312 increase explosively the load on management of tunnel states. 1314 From this point of view, the expected evaluation points from this 1315 aspect is whether the nature of candidate UP protocols are to utilize 1316 MP2P data path. Supporting MP2P data path by GTP-U could be a gap 1317 since GTP-U is a point-to-point tunneling protocol as it is described 1318 in Section 3. 1320 Noted that 3GPP CT WG4 pointed out GTP-U was already required to 1321 allow one single tunnel endpoint to receive packets from multiple 1322 source endpoints ([C4-185491-3GPP]). It was an architectural 1323 requirement of 3GPP system from a previous generation. It means that 1324 MP2P data path requirement for UP protocol has been existed before 1325 the 5G system. 1327 5.3. Supporting Transport Variations 1329 The 5G system will be expected that the new radio spectrums in high 1330 frequency bands require operators to deploy their base stations much 1331 dense for much wider areas compare to previous generation footprints. 1332 To make sure that density and coverage, all available types of 1333 transport in the field must be employed between RAN to UPF, or UPF to 1334 UPF. 1336 It is also expected that MTU size of each transport could be varied. 1337 Because one could be own fiber which the operator configure the MTU 1338 size as they like while others are third-party provided L2/L3 VPN 1339 lines which MTU size can't be controlled by the operators. 1341 The MTU between RAN and UPF can be discovered by offline means and 1342 the operator takes into account the MTU that is transferable on the 1343 radio interface and based on this the operator configures the right 1344 MTU to be used. That is then signaled to the UE either via PCO (for 1345 IPv4 case) or the IPv6 RA message (for IPv6 case). 1347 In addition, for cases that third-parties provide VPN lines, it would 1348 be recommended MTU size discovery for each data path and dynamic MTU 1349 size adjustment mechanisms, while GTP-U does not support those 1350 mechanisms. 1352 As the study item in 3GPP mentioned, IPv6 is preferable address 1353 family not only for UEs, but also for the UP transport, in terms of 1354 size of available address space to support dense and wide footprint 1355 of base stations. However it increases header size from 20bytes to 1356 40bytes compare to IPv4. It could be a problem if the MTU size is 1357 uncontrollable, or only limited MTU size available to carry committed 1358 PDU size on the user plane. 1360 The expected evaluation points from this aspect should be that the 1361 candidate protocols are able to dynamically adjust path MTU size with 1362 appropriate MTU size discovery mechanism. It also should be that how 1363 the candidate protocols leverage IPv6 to deal with header size 1364 increasing. 1366 5.4. Data Path Management 1368 As Section 4.2 described, the 5G systems allows user plane that 1369 flexible UPF selection, multiple anchor UPFs, and no limit on how 1370 many UPFs chained for the data path of the PDU session. UPF 1371 deployments in the field will thereby be distributed to be able to 1372 optimize the data path based on various logics and service scenarios. 1374 That powerful user plane capability could make data path management 1375 through the control plane, or operation support systems (OSS) be 1376 complicated and difficult. Perhaps it could be the case where the UP 1377 protocol nature is P2P and it only supports per session base data 1378 path handling. Therefore it would be better that UP protocol could 1379 support to aggregate several PDU sessions into a tunnel or shall be a 1380 session-less tunnel. 1382 Because it increases data path states by number of sessions, and 1383 number of endpoints of UPFs that makes data path handling much hectic 1384 and the control plane tend to be overloaded by not only usual 1385 attach/detach/hand-over operations, but also existing session 1386 manipulation triggered by UPF and transport nodes/paths restoration, 1387 etc., 1389 The expected evaluation points from this aspect should be that how 1390 much the candidate protocols can reduce data path management loads 1391 both on the control plane NFs and UPFs compare to the per session 1392 based handling for P2P paths. It could possibly include N3 and N6 in 1393 addition to N9 while it supports flexible user plane data path 1394 optimizations for some example scenarios. 1396 5.5. QoS Control 1398 The QoS model is based on QoS flows to which QFI indicates in the 5G 1399 system that allows multiple QoS flows are aggregated into a single 1400 PDU session. So that it is given that the UP protocol should convey 1401 QFIs for a PDU session and the UPF needs to lookup them. It makes 1402 sure that reflects QoS policy in the 5G system to corresponding 1403 forwarding policy in the user plane IP transports. 1405 The expected evaluation points from this aspect should be whether the 1406 candidate protocols can provide stable ID space for QFI which 1407 shouldn't be a big deal since QFI just requires 6-bits space. 1409 As we pointed out in Section 3.2, the lookup process could impact UPF 1410 performance if the QFI container position in the header is 1411 unpredictable. It could happen many times along the path because the 1412 each UPFs should do it again and again in case that various different 1413 QoS policies are deployed in the networks under the UP as we 1414 discussed in Section 5.3. 1416 As [TS.29.281-3GPP] updated in version 15.3.0, it is recommended that 1417 the first extension header is the PDU session container in which QFI 1418 is present. 1420 5.6. Traffic Detection and Flow Handling 1422 As described in Section 4.1.1, UPF need to detect traffic flow 1423 specified by SMF within a PDU session, and enforce some processes to 1424 the PDU based on the pre-configured policy rule. 1426 As similar with QoS flow lookup described in Section 5.5, UPFs along 1427 the path are repeatedly detecting an specified traffic flow in inner 1428 PDU. It could increase redundant flow detection load on every UPFs 1429 that could be avoided if the upstream UPF put some identifier which 1430 abstracts the detected flow into the packets. It enables following 1431 UPFs just find the ID to detect the indicated flow from the packet. 1433 The expected evaluation points from this aspect should be whether the 1434 candidate protocols can provide means to reduce that redundant flow 1435 detection that could be enough bits space on stable ID space to put 1436 abstracted detected flow identifier. 1438 5.7. Supporting Network Slicing Diversity 1440 Network Instance has been defined as the glue of network slice 1441 between 5G and IP transport in addition to IP domain separation, as 1442 described in Section 4.1.2. It is expected that SMF is able to 1443 configure UPF to send UP packet to corresponding transport slice by 1444 indicating Network Instance in FAR so that UPF can determine outgoing 1445 interface for the UP packet. 1447 It is assumed that IP transport networks are Network Instance 1448 agnostic, i.e., transport slices are independently instantiated and 1449 not bound to specific IP address space in the 5GC, for preventing 1450 increase of routing table size. 1452 As a transport slice may be shared with multiple IP domains, Network 1453 Instance could be instantiated for all combination of IP domains and 1454 transport slice. To indicate those combination in UP packet over the 1455 wire, the 5G architecture expects VPN solutions as described in 1456 section 5.6.12 of [TS.23.501-3GPP]. 1458 Binding Network Instance with corresponding VPN would be varied per 1459 VPN solutions and FAR is not able to do. Hence it is out of scope of 1460 3GPP and it may be covered by IETF, or other SDOs. 1462 Apart from binding, if it is the case where MPLS based VPNs, such as 1463 [RFC4364] and [RFC4664] are expected as the existing VPN solution 1464 which bound to Network Instance, there are some avaiable deployment 1465 options, such as 1). PE router integrates UPF, 2). CE router 1466 integrates UPF, 3). UPF connects to the VPN behind the CE router. 1468 Option 1 could work since all legacy MPLS or Segment Routing 1469 [RFC8402] based solution are available for both VPN and transport 1470 slicing at the UPF integrated PE router. However it is hard to 1471 expect it in multi-vendor deployment case, where the PE routers 1472 providing vendor is different from the vendor who provides UPFs, for 1473 example. 1475 Option 2 and 3 are expected as IP domain separation, but it is hard 1476 to see that it is able to indicate transport slice in addition to the 1477 IP domain. Other L2 and tunneling solutions should be same with 1478 those options. 1480 The expected evaluation points from this aspect should be whether the 1481 candidate protocols can contain forwarding information associated to 1482 the assigned IP domain and transport slice for all possible 1483 deployment cases. 1485 5.8. Reliable Communication support 1487 As Section 4.2 described, more than two UP paths are required for a 1488 QoS flow of a PDU session between the anchor UPF and gNB. Those UP 1489 paths are to convey redundant duplicated packets. 1491 To support reliable communication with above requirements, UPF and 1492 gNB must replicate the sending UP packets and eliminate the received 1493 duplicated UP packets. Not to mention that UP protocol should be 1494 able to make sure that the paths are not in fate sharing, the UP 1495 packet must have sequence number to indicate duplicate packets per 1496 QoS flow basis. 1498 The expected evaluation points from this aspect should be whether the 1499 candidate protocols can indicate packet sequence and diversed paths 1500 in the context of QoS flow, not in UP tunnel context. The packet 1501 sequence information should be transparent through I-UPF(s) exist in 1502 the middle of the path even in case that the UP tunnels are 1503 terminated at the I-UPF(s). 1505 6. Conclusion 1507 We analyzed the 3GPP specifications of the 5G architecture in terms 1508 of user plane and the current protocol adopted to the user plane. 1509 After the analysis work, we believe that the results described in 1510 this document shows that we reach at certain level of understanding 1511 on the 5G systems and ready to provide our inputs to 3GPP. 1513 We clarified GTP-U through the analysis and listed observed 1514 characteristics in Section 3.6. We also clarified the architectural 1515 requirements for UP protocol described in Section 4.2. 1517 Our conclusion here is that it is hopefull if the evaluation aspects 1518 described in Section 5 help for the study progress. It is worth to 1519 study possible candidate UP protocols for the 5G system including 1520 current one based from the aspects. 1522 7. Security Consideration 1524 TBD 1526 8. Acknowledgement 1528 The authors would like to thank Tom Herbert, Takashi Ito, John Leddy, 1529 Pablo Camarillo, Daisuke Yokota, Satoshi Watanabe, Koji Tsubouchi and 1530 Miya Kohno for their detailed reviews, comments, and contributions. 1532 A special thank you goes to Arashmid Akhavain for his technical 1533 review and feedback. 1535 Lastly, the authors would like to thank 3GPP CT WG4 folks for their 1536 review and feedback. 1538 9. Informative References 1540 [C4-185491-3GPP] 1541 3rd Generation Partnership Project (3GPP), "LS OUT on User 1542 Plane Analysis", July 2018, 1543 . 1546 [CP-173160-3GPP] 1547 3rd Generation Partnership Project (3GPP), "New Study Item 1548 on User Plane Protocol in 5GC", December 2017, 1549 . 1552 [CP-180116-3GPP] 1553 3rd Generation Partnership Project (3GPP), "LS on user 1554 plane protocol study", March 2018, 1555 . 1558 [IAB-Statement] 1559 Internet Architecture Board (IAB), "IAB Statement on 1560 IPv6", November 2016, 1561 . 1563 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1564 (IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460, 1565 December 1998, . 1567 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 1568 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 1569 2006, . 1571 [RFC4664] Andersson, L., Ed. and E. Rosen, Ed., "Framework for Layer 1572 2 Virtual Private Networks (L2VPNs)", RFC 4664, 1573 DOI 10.17487/RFC4664, September 2006, 1574 . 1576 [RFC4861] Narten, T., Nordmark, E., Simpson, W., and H. Soliman, 1577 "Neighbor Discovery for IP version 6 (IPv6)", RFC 4861, 1578 DOI 10.17487/RFC4861, September 2007, 1579 . 1581 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 1582 "IPv6 Flow Label Specification", RFC 6437, 1583 DOI 10.17487/RFC6437, November 2011, 1584 . 1586 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 1587 for Equal Cost Multipath Routing and Link Aggregation in 1588 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 1589 . 1591 [RFC6935] Eubanks, M., Chimento, P., and M. Westerlund, "IPv6 and 1592 UDP Checksums for Tunneled Packets", RFC 6935, 1593 DOI 10.17487/RFC6935, April 2013, 1594 . 1596 [RFC7157] Troan, O., Ed., Miles, D., Matsushima, S., Okimoto, T., 1597 and D. Wing, "IPv6 Multihoming without Network Address 1598 Translation", RFC 7157, DOI 10.17487/RFC7157, March 2014, 1599 . 1601 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1602 (IPv6) Specification", STD 86, RFC 8200, 1603 DOI 10.17487/RFC8200, July 2017, 1604 . 1606 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1607 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1608 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1609 July 2018, . 1611 [TR.29.891-3GPP] 1612 3rd Generation Partnership Project (3GPP), "3GPP TR 29.891 1613 (V15.0.0): 5G System Phase 1, CT WG4 Aspects", December 1614 2017, . 1617 [TS.22.261-3GPP] 1618 3rd Generation Partnership Project (3GPP), "3GPP TS 22.261 1619 (V15.7.0): Service requirements for 5G system stage 1", 1620 December 2018, . 1623 [TS.23.060-3GPP] 1624 3rd Generation Partnership Project (3GPP), "3GPP TS 23.060 1625 (V15.3.0): General Packet Radio Service (GPRS); Service 1626 description; Stage 2", June 2018, 1627 . 1630 [TS.23.501-3GPP] 1631 3rd Generation Partnership Project (3GPP), "3GPP TS 23.501 1632 (V16.1.0): System Architecture for 5G System; Stage 2", 1633 June 2019, . 1636 [TS.23.502-3GPP] 1637 3rd Generation Partnership Project (3GPP), "3GPP TS 23.502 1638 (V15.4.0): Procedures for 5G System; Stage 2", December 1639 2018, . 1642 [TS.23.503-3GPP] 1643 3rd Generation Partnership Project (3GPP), "3GPP TS 23.503 1644 (V15.4.0): Policy and Charging Control System for 5G 1645 Framework; Stage 2", December 2018, 1646 . 1649 [TS.28.530-3GPP] 1650 3rd Generation Partnership Project (3GPP), "3GPP TS 28.530 1651 (V15.1.0): Management and orchestration of networks and 1652 network slicing; Concepts, use cases and requirements 1653 (work in progress)", December 2018, 1654 . 1657 [TS.28.531-3GPP] 1658 3rd Generation Partnership Project (3GPP), "3GPP TS 28.531 1659 (V15.1.0): Management and orchestration of networks and 1660 network slicing; Provisioning; Stage 1 (Release 15)", 1661 December 2018, . 1664 [TS.28.532-3GPP] 1665 3rd Generation Partnership Project (3GPP), "3GPP TS 28.532 1666 (V15.1.0): Management and orchestration of networks and 1667 network slicing; Provisioning; Stage 2 and stage 3 1668 (Release 15)", Decempber 2018, 1669 . 1672 [TS.28.533-3GPP] 1673 3rd Generation Partnership Project (3GPP), "3GPP TS 28.533 1674 (V15.1.0): Management and orchestration of networks and 1675 network slicing; Management and orchestration architecture 1676 (Release 15)", December 2018, 1677 . 1680 [TS.29.244-3GPP] 1681 3rd Generation Partnership Project (3GPP), "3GPP TS 29.244 1682 (V15.1.0): Interface between the Control Plane and the 1683 User Plane Nodes; Stage 3", December 2018, 1684 . 1687 [TS.29.274-3GPP] 1688 3rd Generation Partnership Project (3GPP), "3GPP TS 29.274 1689 (V15.4.0): 3GPP Evolved Packet System (EPS); Evolved 1690 General Packet Radio Service (GPRS) Tunneling Protocol for 1691 Control plane (GTPv2-C); Stage 3", June 2018, 1692 . 1695 [TS.29.281-3GPP] 1696 3rd Generation Partnership Project (3GPP), "3GPP TS 29.281 1697 (V15.5.0): GPRS Tunneling Protocol User Plane (GTPv1-U)", 1698 December 2018, . 1701 [TS.29.510-3GPP] 1702 3rd Generation Partnership Project (3GPP), "3GPP TS 29.510 1703 (V15.2.0): 5G System; Network Function Repository 1704 Services; Stage 3", December 2018, 1705 . 1708 [TS.29.561-3GPP] 1709 3rd Generation Partnership Project (3GPP), "3GPP TS 29.561 1710 (V15.1.0): 5G System; Interworking between 5G Network and 1711 external Data Networks; Stage 3", September 2018, 1712 . 1715 [TS.38.300-3GPP] 1716 3rd Generation Partnership Project (3GPP), "3GPP TS 38.300 1717 (v15.4.0): NR and NG-RAN Overall Description; Stage 2", 1718 December 2018, . 1721 [TS.38.401-3GPP] 1722 3rd Generation Partnership Project (3GPP), "3GPP TS 38.401 1723 (v15.4.0): NG-RAN; Architecture Description", December 1724 2018, . 1727 [TS.38.415-3GPP] 1728 3rd Generation Partnership Project (3GPP), "3GPP TS 38.415 1729 (v15.2.0): NG-RAN; PDU Session User Plane protocol", 1730 December 2018, . 1733 Authors' Addresses 1735 Shunsuke Homma 1736 NTT 1738 Email: homma.shunsuke@lab.ntt.co.jp 1740 Takuya Miyasaka 1741 KDDI Research 1743 Email: ta-miyasaka@kddi-research.jp 1744 Satoru Matsushima 1745 SoftBank 1747 Email: satoru.matsushima@g.softbank.co.jp 1749 Daniel Voyer 1750 Bell Canada 1752 Email: daniel.voyer@bell.ca