idnits 2.17.1 draft-deconinck-quic-multipath-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (5 March 2020) is 1506 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-13) exists of draft-ietf-quic-invariants-07 == Outdated reference: A later version (-34) exists of draft-ietf-quic-recovery-26 == Outdated reference: A later version (-34) exists of draft-ietf-quic-tls-27 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-27 -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 6824 (Obsoleted by RFC 8684) Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group Q. De Coninck 3 Internet-Draft O. Bonaventure 4 Intended status: Standards Track UCLouvain 5 Expires: 6 September 2020 5 March 2020 7 Multipath Extensions for QUIC (MP-QUIC) 8 draft-deconinck-quic-multipath-04 10 Abstract 12 This document specifies extensions to the QUIC protocol to enable the 13 simultaneous usage of multiple paths for a single connection. The 14 proposed extensions remain compliant with the current single-path 15 QUIC design and preserve the QUIC privacy features. 17 Status of This Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at https://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on 6 September 2020. 34 Copyright Notice 36 Copyright (c) 2020 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 41 license-info) in effect on the date of publication of this document. 42 Please review these documents carefully, as they describe your rights 43 and restrictions with respect to this document. Code Components 44 extracted from this document must include Simplified BSD License text 45 as described in Section 4.e of the Trust Legal Provisions and are 46 provided without warranty as described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 51 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 4 52 2.1. Notation Conventions . . . . . . . . . . . . . . . . . . 4 53 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 5 54 3.1. Moving from Bidirectional Paths to Uniflows . . . . . . . 5 55 3.2. Beyond Connection Migration . . . . . . . . . . . . . . . 7 56 3.3. Establishment of a Multipath QUIC Connection . . . . . . 8 57 3.4. Architecture of Multipath QUIC . . . . . . . . . . . . . 8 58 3.5. Uniflow Establishment . . . . . . . . . . . . . . . . . . 11 59 3.6. Exchanging Data over Multiple Uniflows . . . . . . . . . 12 60 3.7. Exchanging Addresses . . . . . . . . . . . . . . . . . . 14 61 3.8. Coping with Address Removals . . . . . . . . . . . . . . 15 62 3.9. Uniflow Migration . . . . . . . . . . . . . . . . . . . . 15 63 3.10. Congestion Control . . . . . . . . . . . . . . . . . . . 15 64 4. Mapping Uniflow IDs to Connection IDs . . . . . . . . . . . . 15 65 5. Using Multiple Uniflows . . . . . . . . . . . . . . . . . . . 16 66 5.1. Multipath Negotiation . . . . . . . . . . . . . . . . . . 16 67 5.1.1. Transport Parameter Definition . . . . . . . . . . . 16 68 5.2. Coping with Additional Remote Addresses . . . . . . . . . 17 69 5.3. Receiving Uniflow State . . . . . . . . . . . . . . . . . 17 70 5.4. Sending Uniflow State . . . . . . . . . . . . . . . . . . 18 71 5.5. Losing Addresses . . . . . . . . . . . . . . . . . . . . 20 72 6. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 20 73 6.1. MP_NEW_CONNECTION_ID Frame . . . . . . . . . . . . . . . 20 74 6.2. MP_RETIRE_CONNECTION_ID Frame . . . . . . . . . . . . . . 21 75 6.3. MP_ACK Frame . . . . . . . . . . . . . . . . . . . . . . 22 76 6.4. ADD_ADDRESS Frame . . . . . . . . . . . . . . . . . . . . 23 77 6.5. REMOVE_ADDRESS Frame . . . . . . . . . . . . . . . . . . 24 78 6.6. UNIFLOWS Frame . . . . . . . . . . . . . . . . . . . . . 25 79 7. Extension of the Meaning of Existing QUIC Frames . . . . . . 26 80 8. Security Considerations . . . . . . . . . . . . . . . . . . . 27 81 8.1. Nonce Computation . . . . . . . . . . . . . . . . . . . . 27 82 8.2. Validation of Exchanged Addresses . . . . . . . . . . . . 27 83 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 84 9.1. QUIC Transport Parameter Registry . . . . . . . . . . . . 27 85 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 28 86 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 28 87 11.1. Normative References . . . . . . . . . . . . . . . . . . 28 88 11.2. Informative References . . . . . . . . . . . . . . . . . 29 89 Appendix A. Comparison with Multipath TCP . . . . . . . . . . . 30 90 A.1. Multipath TCP Bidirectional Paths vs. QUIC Uniflows . . . 30 91 A.2. Uniflow Establishment . . . . . . . . . . . . . . . . . . 31 92 A.3. Exchanging Data over Multiple Uniflows . . . . . . . . . 31 93 A.4. Congestion Control . . . . . . . . . . . . . . . . . . . 31 94 A.5. ACK Frame . . . . . . . . . . . . . . . . . . . . . . . . 31 95 Appendix B. To move in companion drafts . . . . . . . . . . . . 31 96 B.1. Uniflow Establishment . . . . . . . . . . . . . . . . . . 32 97 B.2. Exchanging Addresses . . . . . . . . . . . . . . . . . . 32 98 B.3. Uniflow Migration . . . . . . . . . . . . . . . . . . . . 32 99 B.4. Scheduling Strategies . . . . . . . . . . . . . . . . . . 34 100 Appendix C. Change Log . . . . . . . . . . . . . . . . . . . . . 34 101 C.1. Since draft-deconinck-quic-multipath-03 . . . . . . . . . 34 102 C.2. Since draft-deconinck-quic-multipath-02 . . . . . . . . . 35 103 C.3. Since draft-deconinck-quic-multipath-01 . . . . . . . . . 35 104 C.4. Since draft-deconinck-quic-multipath-00 . . . . . . . . . 35 105 C.5. Since draft-deconinck-multipath-quic-00 . . . . . . . . . 35 106 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 108 1. Introduction 110 Endhosts have evolved. Today's endhosts are equipped with several 111 network interfaces and users expect to be able to seamlessly switch 112 from one to another or use them simultaneously to aggregate bandwidth 113 whenever needed. During the last years, several multipath extensions 114 to transport protocols have been proposed [RFC6824],[MPRTP], 115 [SCTPCMT]. Multipath TCP [RFC6824] is the most mature one. It is 116 already deployed on popular smartphones, but also for other use cases 117 [RFC8041][IETFJ]. 119 With regular TCP and UDP, all the packets that belong to a given flow 120 share the same 5-tuple that acts as an identifier for this flow. 121 This prevents these flows from using multiple paths. QUIC 122 [I-D.ietf-quic-transport] does not use the 5-tuple as an implicit 123 connection identifier. A QUIC flow is identified by a Connection ID. 124 This enables QUIC to cope with events affecting the 5-tuple, such as 125 NAT rebinding or IP address changes. The QUIC connection migration 126 feature, specified in [I-D.ietf-quic-transport], enables migrating a 127 flow from one 5-tuple to another one, sustaining a connection over 128 different network paths. Still, there is a void to specify 129 simultaneous usage of available network paths for a single 130 connection. Use cases such as bandwidth aggregation or seamless 131 network handovers would be applicable to QUIC, as they are now with 132 Multipath TCP [RFC8041][IETFJ]. Experience with Multipath TCP on 133 smartphones shows that the ability to simultaneously use Wi-Fi and 134 cellular during handovers improves the user perceived quality of 135 experience. A performance evaluation of an early solution for such 136 use cases and a comparison between Multipath QUIC and Multipath TCP 137 may be found in [MPQUIC]. 139 In this document, we leverage many of the lessons learned from the 140 design of Multipath TCP and the comments received on the first 141 versions of this document to propose extensions to the current QUIC 142 design to enable it to simultaneously use several network paths. 144 This document focuses mainly on network paths that are 145 distinguishable by the endpoints. 147 This document is organized as follows. It first provides in 148 Section 3 an overview of the operation of Multipath QUIC. It then 149 states the required changes in the current QUIC design 150 [I-D.ietf-quic-transport] and specifies in Section 5 the usage of 151 multiple paths. Finally, it discusses some security considerations. 153 2. Conventions and Definitions 155 The words "MUST", "MUST NOT", "SHOULD", and "MAY" are used in this 156 document. It's not shouting; when they are capitalized, they have 157 the special meaning defined in [RFC2119]. 159 We assume that the reader is familiar with the terminology used in 160 [I-D.ietf-quic-transport]. In addition, we define the following 161 terms: 163 * Uniflow: A unidirectional flow of packets between a QUIC host and 164 its peer. This flow is identified by an internal Uniflow ID. 165 Packets sent over a uniflow use a Destination Connection ID that 166 may change during the lifetime of the connection. When being in 167 use, an uniflow is temporarily bound to a 4-tuple (Source IP 168 Address, Source Port Number, Destination IP Address, Destination 169 Port Number). 171 * Initial Uniflows: The two uniflows used by peers for the 172 establishment of a QUIC connection. One is the uniflow from the 173 client to the server and the other is the uniflow from the server 174 to the client. The cryptographic handshake is done on these 175 uniflows. These are identified by Uniflow ID 0. 177 2.1. Notation Conventions 179 Packet and frame diagrams use the format described in Section 2.1 of 180 [I-D.ietf-quic-transport]. 182 3. Overview 184 The current design of QUIC [I-D.ietf-quic-transport] provides 185 reliable transport with multiplexing, confidentiality, integrity and 186 authenticity of data flows. A wide range of devices on today's 187 Internet are multihomed. Examples include smartphones equipped with 188 both WLAN and cellular interfaces, but also regular dual-stack hosts 189 that use both IPv4 and IPv6. Experience with Multipath TCP has shown 190 that the ability to combine different paths during the lifetime of a 191 connection provides various benefits including bandwidth aggregation 192 or seamless handovers [RFC8041],[IETFJ]. 194 The current design of QUIC does not enable multihomed devices to 195 efficiently use different paths simultaneously. This draft proposes 196 multipath extensions with the following design goals: 198 * Each host keeps control on the number of uniflows being used over 199 the connection 201 * The simultaneous usage of multiple uniflows should not introduce 202 new privacy concerns 204 * Hosts must ensure that all the paths it uses actually reach its 205 peer to avoid packet flooding towards a victim (see 206 Section 21.12.3 of [I-D.ietf-quic-transport]) 208 * The multipath extensions should handle the asymmetrical nature of 209 networks 211 We first explain why a multipath extension would be beneficial to 212 QUIC and then describe it at a high level. 214 3.1. Moving from Bidirectional Paths to Uniflows 216 To understand the overall architecture of the multipath extensions, 217 let us first refine the notion of "path". Since the early days of 218 TCP/IP, a path was denoted by a 4-tuple (Source IP Address, Source 219 Port Number, Destination IP Address, Destination Port Number). In 220 QUIC, this is namely a UDP path from the local host to the remote 221 one. Considering a smartphone interacting with a single-homed 222 server, the smartphone might want to use one path over the WLAN 223 network and another over the cellular one. Those paths are not 224 necessarily disjoint. For example, when interacting with a dual- 225 stack server, a smartphone may create two paths over the Wi-Fi 226 network: one over IPv4 and the other one over IPv6. 228 A regular QUIC connection is composed of two independent active 229 packet flows. The first flow gathers the packets from the client to 230 the server and the other the packets from the server to the client. 231 To illustrate this, let us consider the Figure 1 below. The client 232 has two IP addresses: IPc1 and IPc2. The server has a single 233 address: IPs1. 235 Probed flow IPc2 to IPs1 236 +---------------------------------------------------------+ 237 | | 238 | From IPc1 to IPs1 v 239 | +--------+ Client to Server Flow +--------+ 240 | | | =====================================> | | 241 +-- | Client | | Server | 242 | | <===================================== | | 243 +--------+ Server to Client Flow +--------+ 244 From IPc1* to IPs1* 246 Figure 1: Identifying Unidirectional Flows in QUIC 248 The client initiates the QUIC connection by sending packets towards 249 the server. The server then replies to the client. If the handshake 250 succeeds, the connection is established. Still, this "path" actually 251 consists in two independent UDP flows. Each host has its own view of 252 i) the 4-tuple used to send packets and ii) the 4-tuple on which it 253 receives packets. While the 4-tuple used by the client to send 254 packets may be the same as the one seen and used by the server, this 255 is nt always the case since middleboxes that may alter the 4-tuple of 256 packets (e.g., NATs). To further emphasize on this flow asymmetry, 257 QUIC embeds a path validation mechanism [I-D.ietf-quic-transport] 258 that checks whether a host can reach its peer through a given 259 4-tuple. This process is unidirectional, i.e., the sender checks 260 that it can reach the receiver, but not the reverse. A host 261 receiving a PATH_CHALLENGE frame on a new 4-tuple may in turn 262 initiate a path validation, but this is up to the peer. 264 A QUIC connection is a collection of unidirectional flows, or 265 uniflows. A plain QUIC connection is composed of a main uniflow from 266 client to server and another main uniflow from server to client. 267 These uniflows have their own Connection ID. They are host-specific, 268 i.e., the uniflow(s) from A to B are different from the ones from B 269 to A. This potentially enables the use of unidirectional links such 270 as non-broadcast satellite links [RFC3077], which cannot be used with 271 TCP. 273 3.2. Beyond Connection Migration 275 Unlike TCP [RFC0793], QUIC is not bound to a particular 4-tuple 276 during the lifetime of a connection. A QUIC connection is identified 277 by a (set of) Connection ID(s), placed in the public header of each 278 QUIC packet. This enables hosts to continue the connection even if 279 the 4-tuple changes due to, e.g., NAT rebinding. This ability to 280 shift a connection from one 4-tuple to another is called Connection 281 Migration. One of its use cases is fail-over when the IP address in 282 use fails but another one is available. A smartphone losing the WLAN 283 connectivity can then continue the connection over its cellular 284 interface, for instance. 286 A QUIC connection can thus start on a given set of uniflows, denoted 287 as the initial uniflows, and end on another ones. However, the 288 current QUIC design [I-D.ietf-quic-transport] assumes that only one 289 pair of uniflows is in use for a given connection. The current 290 transport specification [I-D.ietf-quic-transport] does not provide 291 means to distinguish path migration from simultaneous usage of 292 available uniflows for a given connection. 294 This document fills that void. It first proposes mechanisms to 295 communicate endhost addresses to the peer. It then leverages the 296 Address Validation procedure with PATH_CHALLENGE and PATH_RESPONSE 297 frames described in Section 8 of [I-D.ietf-quic-transport] to verify 298 whether the additional addresses advertised by the host are 299 reachable. In this case, those addresses can be used to initiate new 300 uniflows to spread packets over several networks following a packet 301 scheduling policy that is out of scope of this document. 303 TODO: Add a companion document discussing the packet scheduling and 304 path management considerations. 306 The example of Figure 2 illustrates a data exchange between a dual- 307 homed client sending a request spanning two packets and a single- 308 homed server. Uniflow IDs are idependently chosen by each host. In 309 the presented example, the client sends packets over WLAN on uniflow 310 0 and over LTE on uniflow 1, while the packets sent by the server 311 over WLAN are on uniflow 2 and those over LTE are on uniflow 1. 313 Server Phone Server 314 via WLAN via LTE 315 ------- ------- ----- 316 | Pkt(DCID=A,PN=5,frames=[ | | 317 | STREAM("Request (1/2)")]) | Pkt(DCID=B,PN=1,frames=[ | 318 |<----------------------------| STREAM("Request (2/2)")]) | 319 | Pkt(DCID=E,PN=1,frames=[ |-------- | 320 | ACK(LargestAcked=5)]) | |---------- | 321 |---------------------------->| |-------- | 322 | Pkt(DCID=E,PN=2,frames=[ | |-->| 323 | STREAM("Response 1")]) | Pkt(DCID=D,PN=1,frames=[ | 324 |---------------------------->| MPACK(UID=1,LargestAck=1), | 325 | | STREAM("Response 2")]) ---| 326 | Pkt(DCID=A,PN=6,frames=[ | ---------| | 327 | MPACK(UID=2,LargestAck=2), | ----------| | 328 | MPACK(UID=1,LargestAck=1)])|<------| | 329 |<----------------------------| | 330 | Pkt(DCID=E,PN=3,frames=[ | Pkt(DCID=D,PN=2,frames=[ | 331 | STREAM("Response 3")]) | STREAM("Response 4")]) | 332 |---------------------------->| ----| 333 | | ------| | 334 | ... | ... <---------| | 336 Figure 2: Data flow with Multipath QUIC 338 The remaining of this section presents a high-level overview of the 339 multipath operations in QUIC. 341 3.3. Establishment of a Multipath QUIC Connection 343 A Multipath QUIC connection starts as a regular QUIC connection 344 [I-D.ietf-quic-transport] [I-D.ietf-quic-tls]. The multipath 345 extensions defined in this document are negotiated using the 346 "max_sending_uniflow_id" transport parameter. Any value for this 347 transport parameter advertises the support of the multipath 348 extensions. 350 Notice that a host advertising a value of 0 for the 351 "max_sending_uniflow_id" transport parameter indicates that it does 352 not want additional uniflows to send packets, but it still supports 353 the multipath extensions. Such situation might be useful when the 354 host does not require multiple uniflows for packet sending but still 355 wants to let the peer use multiple uniflows to reach it. 357 3.4. Architecture of Multipath QUIC 359 To illustrate the architecture of a Multipath QUIC connection, 360 consider Figure 3. 362 +--------+ CID A - Uniflow ID 1 +--------+ 363 | | =====================================> | | 364 | | CID B - Uniflow ID 0 | | 365 | | =====================================> | | 366 | | | | 367 | Client | CID C - Uniflow ID 0 | Server | 368 | | <===================================== | | 369 | | CID D - Uniflow ID 1 | | 370 | | <===================================== | | 371 | | CID E - Uniflow ID 2 | | 372 +--------+ <===================================== +--------+ 374 Figure 3: An example of uniflow distribution over a Multipath 375 QUIC connection 377 Once established, a Multipath QUIC connection consists in one or more 378 uniflow(s) from the client to the server and one or more uniflow(s) 379 from the server to the client. The number of uniflows in one 380 direction can be different from the one in the other direction. The 381 example in Figure 3 shows two uniflows from the client to the server 382 and three uniflows from the server to the client. From the end- 383 hosts' viewpoint, they observe two kinds of uniflows: 385 * Sending uniflows: uniflows over which the host can send packets 387 * Receiving uniflows: uniflows over which the host can receive 388 packets 390 Reconsidering the example in Figure 3, the client has two sending 391 uniflows and three receiving uniflows. The server has three sending 392 uniflows and two receiving uniflows. There is thus a one-to-one 393 mapping between the sending uniflows of a host and the receiving 394 uniflows of its peer. A uniflow is seen as a sending uniflow from 395 the sender's perspective and as a receiving uniflow from the 396 receiver's viewpoint. 398 Each uniflow is associated with a specific four-tuple and identified 399 by a Uniflow ID, as shown in Figure 4. 401 Client state 402 +-----------------------------------------------------+ 403 | Connection | 404 | +-----------+ +-----------+ ... +-------------+ | 405 | | Sending | | Sending | ... | Sending | | 406 | | Uniflow 0 | | Uniflow 1 | | Uniflow N-1 | | 407 | | UCID set A| | UCID set B| ... | UCID set C | | 408 | | Tuple A | | Tuple B | ... | Tuple C | | 409 | | PNS A | | PNS B | | PNS C | | 410 | +-----------+ +-----------+ ... +-------------+ | 411 | +-----------+ +-----------+ ... +-------------+ | 412 | | Receiving | | Receiving | ... | Receiving | | 413 | | Uniflow 0 | | Uniflow 1 | | Uniflow M-1 | | 414 | | UCID set X| | UCID set Y| ... | UCID set Z | | 415 | | Tuple X | | Tuple Y | ... | Tuple Z | | 416 | | PNS X | | PNS Y | | PNS Z | | 417 | +-----------+ +-----------+ ... +-------------+ | 418 +-----------------------------------------------------+ 420 Server state 421 +-----------------------------------------------------+ 422 | Connection | 423 | +-----------+ +-----------+ ... +-------------+ | 424 | | Sending | | Sending | ... | Sending | | 425 | | Uniflow 0 | | Uniflow 1 | | Uniflow M-1 | | 426 | | UCID set X| | UCID set Y| ... | UCID set Z | | 427 | | Tuple X* | | Tuple Y* | ... | Tuple Z* | | 428 | | PNS X | | PNS Y | | PNS Z | | 429 | +-----------+ +-----------+ ... +-------------+ | 430 | +-----------+ +-----------+ ... +-------------+ | 431 | | Receiving | | Receiving | ... | Receiving | | 432 | | Uniflow 0 | | Uniflow 1 | | Uniflow N-1 | | 433 | | UCID set A| | UCID set B| ... | UCID set C | | 434 | | Tuple A* | | Tuple B* | ... | Tuple C* | | 435 | | PNS A | | PNS B | | PNS C | | 436 | +-----------+ +-----------+ ... +-------------+ | 437 +-----------------------------------------------------+ 439 Figure 4: Architectural view of Multipath QUIC for a host having 440 N sending uniflows and M receiving uniflows 442 As described before, a Multipath QUIC connection starts using two 443 Initial Uniflows, identified by Uniflow ID 0 on each peer. The 444 packets can then be spread over several uniflows. Each uniflow has 445 its (set of) Uniflow Connection ID(s) (UCID) packets that are used to 446 explicitly mark where they belong to. Depending on the direction of 447 the uniflow, the host keeps either the Uniflow Source Connection ID 448 (USCID, for the receiving uniflows) or the Uniflow Destination 449 Connection ID (USCID, for the sending uniflows). Notice that the 450 (set of) UDCID(s) of a sending uniflow of a host is the same as the 451 (set of) USCID(s) of the corresponding receive uniflow of the remote 452 peer. Preventing the linkability of different uniflows is an 453 important requirement for the multipath extensions 454 [I-D.huitema-quic-mpath-req]. We address it by using UCIDs as 455 implicit uniflow identifiers. This makes the linkability harder than 456 having explicit signaling as in earlier version of this draft. 457 Furthermore, it does not require any public header change and thus 458 preserves the QUIC invariants [I-D.ietf-quic-invariants]. 460 When an uniflow is in use, each endhost associates it with a network 461 path. In practice, this consists in a particular 4-tuple over which 462 packets are sent (resp. received) on a sending (resp. receiving) 463 uniflow. Each endhost has a specific vision of the 4-tuple, which 464 might differ between endhosts. For instance, a client might send 465 data from a private IP address behind a NAT and the server will 466 receive packets coming from the NAT's public IP address. Notice that 467 while uniflows may share a common network path, this is not 468 mandatory. 470 Each uniflow is an independent flow of packets over a given network 471 path. Uniflows can experience very different network conditions 472 (latency, bandwidth,...). To handle this, each uniflow has its own 473 packet sequence number space. 475 In addition to the UCIDs, 4-tuple and packet number space, some 476 additional information is kept for each uniflow. The Uniflow ID 477 identifies the uniflow at the frame level and ensures uniqueness of 478 the nonce (see Section 8.1 for details) while limiting the number of 479 concurrently used uniflows. 481 3.5. Uniflow Establishment 483 The "max_sending_uniflow_id" transport parameter exchanged during the 484 cryptographic handshake fixes an upper bound on the number of sending 485 uniflows a host wants to support. Then, hosts provide to their peer 486 Uniflow Connection IDs to use on uniflows. Both hosts dynamically 487 control how many sending uniflows can currently be in use by the 488 peer, i.e., the number of different Uniflow IDs proposed to the peer. 489 While the sender determines the upper bound of sending paths it can 490 have, it is the receiver that initializes uniflows, as the sender 491 needs a UCID communicated by the receiver before using a uniflow. 492 Notice that the peers might advertise different values for the 493 "max_sending_uniflow_id" transport parameters, setting different 494 upper bounds to the sending and receiving uniflows of each host. 496 Hosts initiate the creation of their receiving uniflows by sending 497 MP_NEW_CONNECTION_ID frames (see Section 6.1) which are an extended 498 version of the NEW_CONNECTION_ID frame. This frame associates a UCID 499 to a uniflow. Upon reception of the MP_NEW_CONNECTION_ID frame, a 500 host can start using the proposed sending uniflow having the 501 referenced Uniflow ID by marking sent packets with the provided UCID. 502 Therefore, once a host sends a MP_NEW_CONNECTION_ID frame, it 503 announces that it is ready to receive packets from that Uniflow ID 504 with the proposed UCID. As frames are encrypted, adding new uniflows 505 over a QUIC connection does not leak cleartext identifiers 506 [I-D.huitema-quic-mpath-req]. 508 A server might provide several Uniflow Connection IDs for the same 509 Uniflow IDs with multiple MP_NEW_CONNECTION_ID frames. This can be 510 useful to cope with migration cases, as described in Section 3.9. 511 Multipath QUIC is thus asymmetrical. 513 3.6. Exchanging Data over Multiple Uniflows 515 A QUIC packet acts as a container for one or more frames. Multipath 516 QUIC uses the same STREAM frames as QUIC to carry data. A byte 517 offset is associated to the data payload. One of the key design 518 decisions of (Multipath) QUIC is that frames are independent of the 519 packets carrying them. This implies that a frame transmitted over 520 one uniflow could be retransmitted later on another uniflow without 521 any change. Furthermore, all current QUIC frames are idempotent and 522 could be optimistically duplicated over several uniflows. 524 The uniflow on which data is sent is a packet-level information. 525 This means that a frame can be sent regardless of the uniflow of the 526 packet carrying it. Other flow control considerations like the 527 stream receive window advertised by the MAX_STREAM_DATA frame remain 528 unchanged when there are multiple sending uniflows. 530 As previously described, Multipath QUIC might face reordering at 531 packet-level when using uniflows having different latencies. The 532 presence of different Uniflow Connection IDs ensures that the packets 533 sent over a given uniflow will contain monotonically increasing 534 packet numbers. To ensure more flexibility and potentially to reduce 535 the ACK block section of the (MP_)ACK frame when aggregating 536 bandwidth of uniflows exhibiting different network characteristics, 537 each uniflow keeps its own monotonically increasing Packet Number 538 space. This potentially allows sending up to 2 * 2^64 packets on a 539 QUIC connection since each uniflow has its own packet number space 540 (see Section 8.1 for the detail of this limit). 542 With the introduction of multiple uniflows, there is a need to 543 acknowledge packets sent on different uniflows separately. The 544 packets sent on Initial Uniflows (with Uniflow ID 0) are still 545 acknowledged with regular ACK frames, such that no modification is 546 introduced in a core frame. For the other uniflows, the multipath 547 extensions introduce a MP_ACK frame which prepends the ACK frame with 548 a Uniflow ID field indicating from which receiving uniflow the host 549 acknowledges packets. To better explain this, let us consider the 550 situation illustrated in Figure 5. 552 Sending uniflow 0 - CID A | Receiving uniflow 0 - CID A 553 Sending uniflow 1 - CID B | Receiving uniflow 1 - CID B 554 Receiving uniflow 0 - CID C | Sending uniflow 0 - CID C 555 Receiving uniflow 1 - CID D | Sending uniflow 1 - CID D 556 Receiving uniflow 2 - CID E | Sending uniflow 2 - CID E 558 Client Server 559 ------ ------ 560 | | 561 | Pkt(DCID=B,PN=42,frames=[STREAM("Request")]) | 562 |--------------------------- | 563 | |---------------------------->| 564 | | 565 | Pkt(DCID=E,PN=58,frames=[ | 566 | STREAM("Response"), MP_ACK(UID=1,LargestAcked=42)]) | 567 | ------------------------------| 568 |<-------------------------| | 569 | | 571 Figure 5: Acknowledging packets sent on uniflows 573 Here five uniflows are in use, two in the client to server direction 574 and three in the reverse one. The client first sends a packet on its 575 sending uniflow 1 (linked to CID B). The server receives the packet 576 on its receiving uniflow 1. Therefore, it generates a MP_ACK frame 577 for Uniflow ID 1 and transmits it to the client. The server can 578 choose any of its sending uniflows to transmit this frame. In the 579 provided situation, the server sends its packets on uniflow 2. The 580 client thus receives this packet on its receiving uniflow 2. 582 Similarly, packets sent over a given uniflow might be acknowledged by 583 (MP_)ACK frames sent on another uniflow that does not share the same 584 network path. Looking at Figure 2 again, "Response 2" packet on 585 server's sending uniflow 1 with DCID D using the LTE network is 586 acknowledged by a MP_ACK frame received on a uniflow using the Wi-Fi 587 network. 589 3.7. Exchanging Addresses 591 When a multi-homed mobile device connects to a dual-stacked server 592 using its IPv4 address, it is aware of its local addresses (e.g., the 593 Wi-Fi and the cellular ones) and the IPv4 remote address used to 594 establish the QUIC connection. If the client wants to create new 595 uniflows and use them over the IPv6 network, it needs to learn the 596 other addresses of the remote peer. 598 This is possible with the ADD_ADDRESS frames that are sent by a 599 Multipath QUIC host to advertise its current addresses. Each 600 advertised address is identified by an Address ID. The addresses 601 attached to a host can vary during the lifetime of a Multipath QUIC 602 connection. A new ADD_ADDRESS frame is transmitted when a host has a 603 new address. This ADD_ADDRESS frame is protected as other QUIC 604 control frames, which implies that it cannot be spoofed by attackers. 605 The communicated address MUST first be validated by the receiving 606 host before it starts using it as described in Section 8 of 607 [I-D.ietf-quic-invariants]. This process ensures that the advertised 608 address actually belongs to the peer and that the peer can receive 609 packets sent by the host on the provided address. It also prevents 610 hosts from launching amplification attacks to a victim address. 612 If the client is behind a NAT, it could announce a private address in 613 an ADD_ADDRESS frame. In such situations, the server would not be 614 able to validate the communicated address. The client might still 615 use its NATed addresses to start using its sending uniflows. To 616 enable the server to make the link between the private and the public 617 addresses and hence conciliate the different 4-tuple views, Multipath 618 QUIC provides the UNIFLOWS frame that lists the current active 619 sending Uniflow IDs along with their associated local Address ID. 620 Notice that a host might also discover the public addresses of its 621 peer by observing its remote IP addresses associated to the 622 connection. 624 A receiving uniflow is active as soon as the host has sent the 625 MP_NEW_CONNECTION_ID frames proposing the corresponding Uniflow 626 Connection IDs to its peer. A sending uniflow is active when it has 627 received its Uniflow Connection IDs and is bound to a validated 628 4-tuple. The UNIFLOWS frame indicates the local Address IDs that the 629 uniflow uses from the sender's perspective. With this information, 630 the remote host can validate the public address and associate the 631 advertised one with the perceived addresses. 633 3.8. Coping with Address Removals 635 During the lifetime of a QUIC connection, a host might lose some of 636 its addresses. A concrete example is a smartphone going out of reach 637 of a Wi-Fi network or shutting off one of its network interfaces. 638 Such address removals are advertised using REMOVE_ADDRESS frames. 639 The REMOVE_ADDRESS frame contains the Address ID of the lost address 640 previously communicated through ADD_ADDRESS. Notice that because a 641 given Address ID might encounter several events that need to be 642 ordered (e.g., ADD_ADDRESS, REMOVE_ADDRESS and ADD_ADDRESS again), 643 both ADD_ADDRESS and REMOVE_ADDRESS frames include an Address ID 644 related Sequence Number. 646 3.9. Uniflow Migration 648 At a given time, a Multipath QUIC endpoint gathers a set of active 649 sending and receiving uniflows, each associated to a 4-tuple. To 650 address privacy issues due to the linkability of addresses with 651 Connection IDs, hosts should avoid changing the 4-tuple used by a 652 sending uniflow, as described in Section 8 of 653 [I-D.ietf-quic-invariants]. Multipath QUIC keeps the path migration 654 ability and applies it to each uniflow independently. 656 3.10. Congestion Control 658 The QUIC congestion control scheme is defined in 659 [I-D.ietf-quic-recovery]. This congestion control scheme is not 660 suitable when several sending uniflows are active. Using the 661 congestion control scheme defined in [I-D.ietf-quic-recovery] with 662 Multipath QUIC would result in unfairness. Each sending uniflow of a 663 Multipath QUIC connection MUST have its own congestion control state. 664 As for Multipath TCP, the windows of the different sending uniflows 665 MUST be coupled together [RFC6356]. 667 4. Mapping Uniflow IDs to Connection IDs 669 As described in the overview section, hosts need to identify on which 670 uniflows packets are sent. The Uniflow ID must then be inferred from 671 the public header. This is done by using the Destination Connection 672 ID field of Short Header packets. 674 The Initial Uniflow Connection IDs are determined during the 675 cryptographic handshake and actually correspond to both Connection 676 IDs in the current single-path QUIC design [I-D.ietf-quic-transport]. 677 Additional Uniflow Connection IDs for the Initial Uniflows can be 678 provided with the regular NEW_CONNECTION_ID frames. The Uniflow 679 Connection IDs of the other uniflows are determined when the 680 MP_NEW_CONNECTION_ID frames are exchanged. 682 Hosts MUST accept packets coming from their peer using the UCIDs they 683 proposed in the (MP_)NEW_CONNECTION_ID frames they sent and associate 684 them with the corresponding receiving Uniflow ID. Upon reception of 685 a (MP_)NEW_CONNECTION_ID frame, hosts MUST acknowledge it and MUST 686 store the advertised Uniflow Destination Connection ID and the 687 Uniflow ID of the proposed sending uniflow. 689 Hosts MUST ensure that all advertised Uniflow Connection IDs are 690 available for the whole connection lifetime, unless they have been 691 retired by their peer in the meantime by the reception of a 692 (MP_)RETIRE_CONNECTION_ID. 694 A host MUST NOT send MP_NEW_CONNECTION_ID frames with a Uniflow ID 695 greater than the value of "max_sending_uniflow_id" advertised by its 696 peer. 698 5. Using Multiple Uniflows 700 This section describes in details the Multipath QUIC operations. 702 5.1. Multipath Negotiation 704 The Multipath negotiation takes place during the cryptographic 705 handshake with the "max_sending_uniflow_id" transport parameter. A 706 QUIC connection is initially single-path in QUIC. During this 707 handshake, hosts advertise their support for multipath operations. 708 When a host advertises a value for the "max_sending_uniflow_id" 709 transport parameter, it indicates that it supports the multipath 710 extensions, i.e., the extensions defined in this document (not to be 711 mixed with the availability of local multiple network paths). If any 712 host does not advertise the "max_sending_uniflow_id" transport 713 parameter, multipath extensions are disabled. 715 The usage of multiple uniflows relies on the ability to use several 716 Connection IDs over a same QUIC connection. Therefore, zero-length 717 Connection IDs MUST NOT be used if the peer advertises a value 718 different from 0 for the "max_sending_uniflow_id" transport 719 parameter. 721 5.1.1. Transport Parameter Definition 723 A host MAY use the following transport parameter: 725 max_sending_uniflow_id (0x40): Indicate the support of the multipath 726 extension presented in this document, regardless of the carried 727 value. Its integer value puts an upper bound on the number of 728 sending uniflows the host advertising the value is ready to 729 support. If absent, this means that the host does not agree to 730 use the multipath extension over the connection. 732 5.2. Coping with Additional Remote Addresses 734 Hosts can learn remote addresses either by receiving ADD_ADDRESS 735 frames or observing the 4-tuple of incoming packets. Hosts MUST 736 first validate the newly learned remote IP addresses before starting 737 sending packets to those addresses. This requirement is explained in 738 Section 8.2. Hosts MUST initiate Address Validation Procedure as 739 specified in [I-D.ietf-quic-transport]. 741 A host MAY cache a validated address for a limited amount of time. 743 5.3. Receiving Uniflow State 745 When proposing uniflows to their peer, hosts need to maintain some 746 state for their receiving uniflows. This state is created upon the 747 sending of a first MP_NEW_CONNECTION_ID frame proposing the 748 corresponding Uniflow ID. As long as there is still one active 749 Uniflow Connection ID for this receiving uniflow (i.e., one UCID 750 which was not retired yet using a MP_RETIRE_CONNECTION_ID), the host 751 MUST accept packets over the receiving uniflow. Once created, hosts 752 MUST keep the following receiving uniflow information: 754 Uniflow ID: An integer that uniquely identifies the receiving 755 uniflow in the connection. This value is immutable. 757 Uniflow Connection IDs: Possible values for the Connection ID field 758 of packets belonging to this receiving uniflow. This value 759 contains the sequence of active UCIDs that were advertised in 760 previously sent MP_NEW_CONNECTION_ID frames. Notice that this 761 sequence might be empty, e.g., when all advertised UCIDs have been 762 retired by the peer. 764 Packet Number Space: Packet number space dedicated to this receiving 765 uniflow. Packet number considerations described in Section 12.3 766 of [I-D.ietf-quic-transport] apply within a given receiving 767 uniflow. 769 Associated 4-tuple: The tuple (sIP, dIP, sport, dport) currently 770 observed to receive packets over this uniflow. This value is 771 mutable, because a host might receive a packet with a different 772 (possibly) validated remote address and/or port than the one 773 previously recorded. If a host observes a change in the 4-tuple 774 of the receiving uniflow, it follows the considerations of 775 Section 9.5 of [I-D.ietf-quic-transport]. 777 Associated local Address ID: The Address ID advertised in 778 ADD_ADDRESS frames sent by the peer corresponding to the local 779 address used to receive packets. This helps to generate UNIFLOWS 780 frames advertising the mapping between uniflows and addresses. 781 The addresses on which the connection was established have Address 782 ID 0. 784 Hosts can also collect network measurements on a per-uniflow basis, 785 like the number of packets received. 787 5.4. Sending Uniflow State 789 During the Multipath QUIC connection, hosts maintain some state for 790 sending uniflows. The state of the sending uniflow determines 791 information that hosts are required to store. The possible sending 792 uniflow states are depicted in Figure 6. 794 TODO: intermediate state for address validation? Yes 796 o 797 | 798 | receive a first MP_NEW_CONNECTION_ID 799 | with the associated Uniflow ID 800 | 801 v path usage over a validated 4-tuple 802 +----------+ ------------------------------------> +----------+ 803 | UNUSED | | ACTIVE | 804 +----------+ <------------------------------------ +----------+ 805 address change or retired UCID 807 Figure 6: Finite-State Machine describing the possible states of 808 a sending uniflow 810 Once a sending uniflow has been proposed by the peer in a received 811 MP_NEW_CONNECTION_ID frame, it is in the UNUSED state. In this 812 situation, hosts MUST keep the following sending uniflow information: 814 Uniflow ID: An integer that uniquely identifies the sending uniflow 815 in the connection. This value is immutable. 817 Uniflow Connection IDs: Possible values for the Connection ID field 818 of packets belonging to this sending uniflow. This value contains 819 the sequence of active UCIDs that were advertised in previously 820 received MP_NEW_CONNECTION_ID frames. Notice that this sequence 821 might be empty, e.g., when all advertised UCIDs have been retired. 823 Sending Uniflow State: The current state of the sending uniflow, 824 being one of the values presented in Figure 6. 826 Packet Number Space: Packet number space dedicated to this sending 827 uniflow. Packet number considerations described in Section 12.3 828 of [I-D.ietf-quic-transport] apply within a given sending uniflow. 830 When the host wants to start using the sending uniflow over a 831 validated address, the sending uniflow goes to the ACTIVE state. 832 This is the state where a sending uniflow can be used to send 833 packets. Having an uniflow in ACTIVE state only guarantees that it 834 can be used, but the host is not forced to. In addition to the 835 fields required in the UNUSED state, the following elements MUST be 836 tracked: 838 Associated 4-tuple: The tuple (sIP, dIP, sport, dport) currently 839 used to packets over this uniflow. This value is mutable, as the 840 host might decide to change its local (or remote) address and/or 841 port. A host that changes the 4-tuple of a sending uniflow SHOULD 842 migrate it. 844 Associated (local Address ID, remote Address ID) tuple: Those 845 identifiers come from the ADD_ADDRESS sent (local) and received 846 (remote). This enables a host to temporarily stop using a sending 847 uniflow when, e.g., the remote Address ID is declared as lost in a 848 REMOVE_ADDRESS. The addresses on which the connection was 849 established have Address ID 0. The reception of UNIFLOWS frames 850 helps hosts associate the remote Address ID used by the sending 851 uniflow. 853 Congestion controller: A congestion window limiting the transmission 854 rate of the sending uniflow. 856 Performance metrics: Basic statistics such as one-way delay or the 857 number of packets sent. This information can be useful when a 858 host needs to perform packet scheduling decisions and flow 859 control. 861 It might happen that a sending path is temporarily unavailable, 862 because one of the endpoint's addresses is no more available or 863 because the host retired all the UCIDs of the sending uniflow. In 864 such cases, the path goes back to the UNUSED state. When performing 865 a transition back to the UNUSED state, hosts MUST reset the 866 additional state added by the ACTIVE state. In the UNUSED state, the 867 host MUST NOT send non-probing packets on it. At this state, the 868 host might want to restart using the uniflow over another validated 869 4-tuple, switching the uniflow state back to the ACTIVE state. 870 However, its congestion controller state MUST be restarted and its 871 performance metrics SHOULD be reset. 873 5.5. Losing Addresses 875 During the lifetime of a connection, a host might lose addresses, 876 e.g., a network interface that was shut down. All the ACTIVE sending 877 uniflows that were using that local address MUST stop sending packets 878 from that address. To advertise the loss of an address to the peer, 879 the host MUST send a REMOVE_ADDRESS frame indicating which local 880 Address IDs has been lost. The host MUST also send an UNIFLOWS frame 881 indicating the status of the remaining ACTIVE uniflows. 883 Upon reception of the REMOVE_ADDRESS, the receiving host MUST stop 884 using the ACTIVE sending uniflows affected by the address removal. 886 Hosts MAY reuse one of these sending uniflows by changing the 887 assigned 4-tuple. In this case, it MUST send an UNIFLOWS frame 888 describing that change. 890 6. New Frames 892 To support the multipath operations, new frames have been defined to 893 coordinate hosts. All frames defined in this document MUST be 894 exchanged in 1-RTT packets. A host receiving one of the following 895 frames in other encryption context MUST close the connection with a 896 PROTOCOL_VIOLATION error. 898 6.1. MP_NEW_CONNECTION_ID Frame 900 The MP_NEW_CONNECTION_ID frame (type=0x40) is an extension of the 901 NEW_CONNECTION_ID frame defined by [I-D.ietf-quic-transport]. It 902 provides the peer with alternative Connection IDs and associates them 903 to a particular uniflow using the Uniflow ID. 905 The format of the MP_NEW_CONNECTION_ID frame is as follows. 907 0 1 2 3 908 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 909 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 910 | Uniflow ID (i) ... 911 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 912 | Sequence Number (i) ... 913 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 914 | Retire Prior To (i) ... 915 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 916 | Length (8) | | 917 +-+-+-+-+-+-+-+-+ Connection ID (8..160) + 918 | ... 919 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 920 | | 921 + + 922 | | 923 + Stateless Reset Token (128) + 924 | | 925 + + 926 | | 927 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 929 Figure 7: MP_NEW_CONNECTION_ID frame 931 Compared to the frame specified in [I-D.ietf-quic-transport], an 932 Uniflow ID varint field of is prefixed to associate the Connection ID 933 with a uniflow. This frame can be sent by both hosts. Upon 934 reception of the frame with a specified Uniflow ID, the peer MUST 935 update the related sending uniflow state and store the communicated 936 Connection ID. 938 To limit the delay of the multipath usage upon handshake completion, 939 hosts SHOULD send MP_NEW_CONNECTION_ID frames for receive uniflows 940 they allow using as soon the connection establishment completes. 942 The generation of Connection ID MUST follow the same considerations 943 as presented in Section 5.1 of [I-D.ietf-quic-transport]. 945 6.2. MP_RETIRE_CONNECTION_ID Frame 947 The MP_RETIRE_CONNECTION_ID frame (type=0x41) is an extension of the 948 RETIRE_CONNECTION_ID frame defined by [I-D.ietf-quic-transport]. It 949 indicates that the end-host will no longer use a Connection ID 950 related to a given uniflow that was issued by its peer. 952 The format of the MP_RETIRE_CONNECTION_ID frame is shown below. 954 0 1 2 3 955 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 956 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 957 | Uniflow ID (i) ... 958 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 959 | Sequence Number (i) ... 960 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 962 Figure 8: MP_RETIRE_CONNECTION_ID frame 964 The frame is handled as described in [I-D.ietf-quic-transport] on an 965 uniflow basis. 967 6.3. MP_ACK Frame 969 The MP_ACK frame (types 0x42 and 0x43) is an extension of the ACK 970 frame defined by [I-D.ietf-quic-transport]. It allows hosts to 971 acknowledge packets that were sent on non-initial uniflows. 973 The format of the MP_ACK frame is shown below. 975 0 1 2 3 976 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 977 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 978 | Uniflow ID (i) ... 979 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 980 | Largest Acknowledged (i) ... 981 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 982 | ACK Delay (i) ... 983 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 984 | ACK Range Count (i) ... 985 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 986 | First ACK Range (i) ... 987 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 988 | ACK Ranges (*) ... 989 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 990 | [ECN Counts] ... 991 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 993 Figure 9: ACK frame adapted to Multipath 995 Compared to the ACK frame, the MP_ACK frame is prefixed by a varint 996 Uniflow ID field indicating to which uniflow the acknowledged packet 997 sequence numbers relate. 999 6.4. ADD_ADDRESS Frame 1001 The ADD_ADDRESS frame (type=0x44) is used by a host to advertise its 1002 currently reachable addresses. 1004 The format of the ADD_ADDRESS frame is shown below. 1006 0 1 2 3 1007 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1008 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1009 |0|0|0|P|IPVers.|Address ID (8) | 1010 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1011 | Sequence Number (i) ... 1012 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1013 |Interface T.(8)| 1014 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1015 | IP Address (32/128) ... 1016 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1017 | [Port (16)] | 1018 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1020 Figure 10: ADD_ADDRESS Frame 1022 The ADD_ADDRESS frame contains the following fields. 1024 Reserved bits: The three most-significant bits of the first byte are 1025 set to 0, and are reserved for future use. 1027 P bit: The fourth most-significant bit of the first byte indicates, 1028 if set, the presence of the Port field. 1030 IPVers.: The remaining four least-significant bits of the first byte 1031 contain the version of the IP address contained in the IP Address 1032 field. 1034 Address ID: An unique identifier for the advertised address for 1035 tracking and removal purposes. This is needed when, e.g., a NAT 1036 changes the IP address such that both hosts see different IP 1037 addresses for a same network path. 1039 Sequence Number: An Address ID related sequence number of the event. 1040 The sequence number space is shared with REMOVE_ADDRESS frames 1041 mentioning the same Address ID. 1043 Interface Type: Used to provide an indication about the interface 1044 type to which this address is bound. The following values are 1045 defined: 1047 * 0: fixed. Used as default value. 1049 * 1: WLAN 1051 * 2: cellular 1053 IP Address: The advertised IP address, in network order. 1055 Port: This optional field indicates the port number related to the 1056 advertised IP address. When this field is present, it indicates 1057 that an uniflow can use the 2-tuple (IP addr, port). 1059 Upon reception of an ADD_ADDRESS frame, the receiver SHOULD store the 1060 communicated address for future use. 1062 The receiver MUST NOT send packets others than validation ones to the 1063 communicated address without having validated it as specified in 1064 Section 8 of [I-D.ietf-quic-transport]. ADD_ADDRESS frames SHOULD 1065 contain globally reachable addresses. Link-local and possibly 1066 private addresses SHOULD NOT be exchanged. 1068 6.5. REMOVE_ADDRESS Frame 1070 The REMOVE_ADDRESS frame (type=0x45) is used by a host to signal that 1071 a previously announced address was lost. 1073 The format of the REMOVE_ADDRESS frame is shown below. 1075 0 1 2 3 1076 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1077 +-+-+-+-+-+-+-+-+ 1078 |Address ID (8) | 1079 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1080 | Sequence Number (i) ... 1081 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1083 Figure 11: REMOVE_ADDRESS Frame 1085 The REMOVE_ADDRESS frame contains the following fields. 1087 Address ID: The identifier of the address to remove. 1089 Sequence Number: An Address ID related sequence number of the event. 1090 The sequence number space is shared with ADD_ADDRESS frames 1091 mentioning the same Address ID. This help the receiver figure out 1092 that a REMOVE_ADDRESS might have been sent before an ADD_ADDRESS 1093 frame implying the same Address ID, even if for some reason the 1094 REMOVE_ADDRESS reaches the receiver after the newer ADD_ADDRESS 1095 one. 1097 A host SHOULD stop using sending uniflows using the removed address 1098 and set them in the UNUSED state. If the REMOVE_ADDRESS contains an 1099 Address ID that was not previously announced, the receiver MUST 1100 silently ignore the frame. 1102 6.6. UNIFLOWS Frame 1104 The UNIFLOWS frame (type=0x46) communicates the uniflows' state of 1105 the sending host to the peer. It allows the sender to communicate 1106 its active uniflows to the peer in order to detect potential 1107 connectivity issue over uniflows. 1109 The format of the UNIFLOWS frame is shown below. 1111 0 1 2 3 1112 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1113 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1114 | Sequence (i) ... 1115 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1116 | ReceivingUniflows (i) ... 1117 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1118 | ActiveSendingUniflows (i) ... 1119 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1120 | Receiving Uniflow Info Section (*) ... 1121 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1122 | Sending Uniflow Info Section (*) ... 1123 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1125 Figure 12: UNIFLOWS Frame 1127 The UNIFLOWS frame contains the following fields. 1129 Sequence: A variable-length integer. This value starts at 0 and 1130 increases by 1 for each UNIFLOWS frame sent by the host. It 1131 allows identifying the most recent UNIFLOWS frame. 1133 ReceivingUniflows: The current number of receiving uniflows 1134 considered as being usable from the sender's point of view. 1136 ActiveSendingUniflows: The current number of sending uniflows in the 1137 ACTIVE state from the sender's point of view. 1139 Receiving Uniflow Info Section: Contains information about the 1140 receiving uniflows (there are ReceivingUniflows entries). 1142 Sending Uniflow Info Section: Contains information about the sending 1143 uniflows in ACTIVE state (there are ActiveSendingUniflows 1144 entries). 1146 Both Receiving Uniflow Info and Sending Uniflow Info Sections share 1147 the same format, which is shown below. 1149 0 1 2 3 1150 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1151 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1152 | Uniflow ID 0 (i) ... 1153 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1154 |LocAddrID 0 (8)| 1155 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1156 ... 1157 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1158 | Uniflow ID N (i) ... 1159 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1160 |LocAddrID N (8)| 1161 +-+-+-+-+-+-+-+-+ 1163 Figure 13: Uniflow Info Section 1165 The fields in the Uniflow Info Section are the following. 1167 Uniflow ID: The Uniflow ID of the uniflow the sending host provides 1168 information about. 1170 LocAddrID: The local Address ID of the address currently used by the 1171 uniflow. 1173 The Uniflow Info section only contains the local Address ID so far, 1174 but this section can be extended later with other potentially useful 1175 information. 1177 7. Extension of the Meaning of Existing QUIC Frames 1179 The multipath extensions do not modify the wire format of existing 1180 QUIC frames. However, they extend the meaning of a few of them while 1181 keeping this addition transparent and consistent with the single-path 1182 QUIC design. The concerned frames (and their extended meaning) are 1183 the following. 1185 NEW_CONNECTION_ID: Equivalent to a MP_NEW_CONNECTION_ID frame with 1186 Uniflow ID set to 0. 1188 RETIRE_CONNECTION_ID: Equivalent to a MP_RETIRE_CONNECTION_ID frame 1189 with Uniflow ID set to 0. 1191 ACK: Equivalent to a MP_ACK frame with Uniflow ID set to 0. 1193 8. Security Considerations 1195 8.1. Nonce Computation 1197 With Multipath QUIC, each uniflow has its own packet number space. 1198 With the current nonce computation [I-D.ietf-quic-tls], using twice 1199 the same packet number over two different uniflows on the same 1200 direction leads to the same cryptographic nonce. Using twice the 1201 same nonce MUST NOT happen, hence MP-QUIC has a different nonce 1202 computation than [I-D.ietf-quic-tls] 1204 the left most bits of nonce MUST be the Uniflow ID that identifies 1205 the current uniflow up to max_sending_uniflow_id. The remaining bits 1206 of the nonce is formed by an exclusive OR of the least signicant bits 1207 of the packet protection IV with the padded packet number (left- 1208 padded with 0s). The nonce MUST be left-padded with a 0 if 1209 max_sending_uniflow_id <= 2, and the max_sending_uniflow_id MUST NOT 1210 be higher than 2^61. If a uniflow has sent 1211 2^62-max_sending_uniflow_id packets, another uniflow MUST be used to 1212 avoid re-using the same nonce. 1214 8.2. Validation of Exchanged Addresses 1216 To use addresses communicated by the peer through ADD_ADDRESS frames, 1217 hosts are required to validate them before using uniflows to these 1218 addresses as described in Section 8 of [I-D.ietf-quic-transport]. 1219 Section 21.12.3 of [I-D.ietf-quic-transport] provides additional 1220 motivation for this process. In addition, hosts MUST send ADD 1221 ADDRESS frames in 1-RTT frames to prevent off-path attacks. 1223 9. IANA Considerations 1225 9.1. QUIC Transport Parameter Registry 1227 This document defines a new transport parameter for the negotiation 1228 of multiple paths. The following entry in Table 1 should be added to 1229 the "QUIC Transport Parameters" registry under the "QUIC Protocol" 1230 heading. 1232 +-------+------------------------+---------------+ 1233 | Value | Parameter Name | Specification | 1234 +=======+========================+===============+ 1235 | 0x40 | max_sending_uniflow_id | Section 5.1.1 | 1236 +-------+------------------------+---------------+ 1238 Table 1: Addition to QUIC Transport Parameters 1239 Entries 1241 10. Acknowledgments 1243 We would like to thank Masahiro Kozuka and Kazuho Oku for their 1244 numerous comments on the first version of this draft. We also thank 1245 Philipp Tiesel for his early comments that led to the current design, 1246 and Ian Swett for later feedback. We also want to thank Christian 1247 Huitema for his draft about multipath requirements to identify 1248 critical elements about the multipath feature. Mohamed Boucadair 1249 provided lot of useful inputs on the second version of this document. 1250 Maxime Piraux and Florentin Rochet helped us to improve the last 1251 versions of this draft. This project was partially supported by the 1252 MQUIC project funded by the Walloon Government. 1254 11. References 1256 11.1. Normative References 1258 [I-D.ietf-quic-invariants] 1259 Thomson, M., "Version-Independent Properties of QUIC", 1260 Work in Progress, Internet-Draft, draft-ietf-quic- 1261 invariants-07, 11 September 2019, . 1264 [I-D.ietf-quic-recovery] 1265 Iyengar, J. and I. Swett, "QUIC Loss Detection and 1266 Congestion Control", Work in Progress, Internet-Draft, 1267 draft-ietf-quic-recovery-26, 21 February 2020, 1268 . 1271 [I-D.ietf-quic-tls] 1272 Thomson, M. and S. Turner, "Using TLS to Secure QUIC", 1273 Work in Progress, Internet-Draft, draft-ietf-quic-tls-27, 1274 21 February 2020, . 1277 [I-D.ietf-quic-transport] 1278 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 1279 and Secure Transport", Work in Progress, Internet-Draft, 1280 draft-ietf-quic-transport-27, 21 February 2020, 1281 . 1284 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1285 Requirement Levels", BCP 14, RFC 2119, 1286 DOI 10.17487/RFC2119, March 1997, 1287 . 1289 11.2. Informative References 1291 [I-D.huitema-quic-1wd] 1292 Huitema, C., "Quic ACK Timestamps For Measuring One-Way 1293 Delays", Work in Progress, Internet-Draft, draft-huitema- 1294 quic-1wd-00, 3 January 2020, . 1297 [I-D.huitema-quic-mpath-req] 1298 Huitema, C., "QUIC Multipath Requirements", Work in 1299 Progress, Internet-Draft, draft-huitema-quic-mpath-req-01, 1300 7 January 2018, . 1303 [IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments", 1304 IETF Journal , November 2016. 1306 [MPQUIC] De Coninck, Q. and O. Bonaventure, "Multipath QUIC: Design 1307 and Evaluation", 13th International Conference on emerging 1308 Networking EXperiments and Technologies (CoNEXT 2017). 1309 http://multipath-quic.org , December 2017. 1311 [MPRTP] Singh, V., Ahsan, S., and J. Ott, "MPRTP: Multipath 1312 considerations for real-time media", Proceedings of the 1313 4th ACM Multimedia Systems Conference , 2013. 1315 [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J.- 1316 Y. Le Boudec, "MPTCP is not pareto-optimal: performance 1317 issues and a possible solution", Proceedings of the 8th 1318 international conference on Emerging networking 1319 experiments and technologies, ACM , 2012. 1321 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1322 RFC 793, DOI 10.17487/RFC0793, September 1981, 1323 . 1325 [RFC3077] Duros, E., Dabbous, W., Izumiyama, H., Fujii, N., and Y. 1326 Zhang, "A Link-Layer Tunneling Mechanism for 1327 Unidirectional Links", RFC 3077, DOI 10.17487/RFC3077, 1328 March 2001, . 1330 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 1331 Congestion Control for Multipath Transport Protocols", 1332 RFC 6356, DOI 10.17487/RFC6356, October 2011, 1333 . 1335 [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, 1336 "TCP Extensions for Multipath Operation with Multiple 1337 Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, 1338 . 1340 [RFC7050] Savolainen, T., Korhonen, J., and D. Wing, "Discovery of 1341 the IPv6 Prefix Used for IPv6 Address Synthesis", 1342 RFC 7050, DOI 10.17487/RFC7050, November 2013, 1343 . 1345 [RFC7225] Boucadair, M., "Discovering NAT64 IPv6 Prefixes Using the 1346 Port Control Protocol (PCP)", RFC 7225, 1347 DOI 10.17487/RFC7225, May 2014, 1348 . 1350 [RFC8041] Bonaventure, O., Paasch, C., and G. Detal, "Use Cases and 1351 Operational Experience with Multipath TCP", RFC 8041, 1352 DOI 10.17487/RFC8041, January 2017, 1353 . 1355 [SCTPCMT] Iyengar, J., Amer, P., and R. Stewart, "Concurrent 1356 multipath transfer using SCTP multihoming over independent 1357 end-to-end paths", IEEE/ACM Transactions on networking, 1358 Vol. 14, no 5 , 2006. 1360 Appendix A. Comparison with Multipath TCP 1362 Multipath TCP [RFC6824] is currently the most widely deployed 1363 multipath transport protocol on the Internet. While its design 1364 impacted the initial versions of the Multipath extensions for the 1365 QUIC protocol, there are now major differences between boths 1366 protocols that we now highlight. 1368 A.1. Multipath TCP Bidirectional Paths vs. QUIC Uniflows 1370 The notion of bidirectional paths, i.e., paths where packets flow in 1371 both directions, became a de facto standard with TCP. The Multipath 1372 TCP extension [RFC6824] combines several TCP connections to spread a 1373 single data stream over them. Hence, all the paths of a Multipath 1374 TCP connection must be bidirectional. However, networking 1375 experiences showed that packets following a direction do not always 1376 share the exact same road as the packets in the opposite direction. 1377 Furthermore, QUIC does not require a network path to be bidirectional 1378 in order to be used. 1380 A.2. Uniflow Establishment 1382 Unlike Multipath TCP [RFC6824], both hosts dynamically control how 1383 many sending uniflows can currently be in use by the peer. 1385 A.3. Exchanging Data over Multiple Uniflows 1387 The uniflow on which data is sent is a packet-level information. 1388 This means that a frame can be sent regardless of the uniflow of the 1389 packet carrying it. Furthermore, because the data offset is a frame- 1390 level information, there is no need to define additional sequence 1391 numbers to cope with reordering across uniflows, unlike Multipath TCP 1392 [RFC6824] that uses a Data Sequence Number at the Multipath TCP 1393 level. 1395 Decoupling the network paths of data with their acknowledgment can be 1396 useful to limit the latency due to (MP_)ACK transmissions on high- 1397 latency network paths and to enable the usage of unidirectional 1398 networks. Such scheduling decision would not have been possible in 1399 Multipath TCP [RFC6824] which must acknowledge data on the 1400 (bidirectional) path it was received on. 1402 A.4. Congestion Control 1404 Multipath TCP uses the LIA congestion control scheme specified in 1405 [RFC6356]. This scheme can immediately be adapted to Multipath QUIC. 1406 Other coupled congestion control schemes have been proposed for 1407 Multipath TCP such as [OLIA]. 1409 A.5. ACK Frame 1411 Since frames are independent of packets, and the uniflow notion 1412 relates to the packets, the (MP_)ACK frames can be sent on any 1413 uniflow, unlike Multipath TCP [RFC6824] which is constrained to send 1414 ACKs on the same path. 1416 Appendix B. To move in companion drafts 1417 B.1. Uniflow Establishment 1419 Sending useful data on a fresh new sending uniflow might lead to poor 1420 performance as the network path used by the QUIC uniflow might not be 1421 usable. A typical case is when a server wants to initiate a new 1422 sending uniflow to a client behind a NAT. The client would possibly 1423 never receive this packet, leading to connectivity issues on that 1424 uniflow. In addition, an opportunistic usage of network paths might 1425 also lead to possible attacks, such as a client advertising the IP 1426 address of a victim hoping that the server will flood it. To avoid 1427 these issues, a remote address MUST have been validated as described 1428 in [I-D.ietf-quic-transport] before associating it on a sending 1429 uniflows. 1431 Because attaching to new networks may be volatile and an endpoint 1432 does not have full visibility on multiple paths that may be available 1433 (e.g., hosts connected to a CPE), a Multipath QUIC capable endhost 1434 SHOULD advertise a "max_sending_uniflow_id" value of at least 4 and 1435 SHOULD propose at least 4 receiving uniflows to its peer. 1437 B.2. Exchanging Addresses 1439 Likewise, the client may be located behind a NAT64. As such it may 1440 announce an IPv6 address in an ADD_ADDRESS frame, that will be 1441 received over IPv4 by an IPv4-only server. The server should not 1442 discard that address, even if it is not IPv6-capable. 1444 An IPv6-only client may also receive from the server an ADD_ADDRESS 1445 frame which may contain an IPv4 address. The client should rely on 1446 means, such as [RFC7050] or [RFC7225], to learn the IPv6 prefix to 1447 build an IPv4-converted IPv6 address. 1449 Hosts that are connected behind an address sharing mechanism may 1450 collect the external IP address and port numbers assigned to the 1451 hosts and then use their addresses in the ADD_ADDRESS. Means to 1452 gather such information include, but not limited to, UPnP IGD, PCP, 1453 or STUN. 1455 B.3. Uniflow Migration 1457 At a given time, a Multipath QUIC endpoint gathers a set of active 1458 sending and receiving uniflows, each associated to a 4-tuple. To 1459 address privacy issues due to the linkability of addresses with 1460 Connection IDs, hosts should avoid changing the 4-tuple used by a 1461 sending uniflow. There still remain situations where this change is 1462 unavoidable. These can be categorized into two groups: host-aware 1463 changes (e.g., network handover from Wi-Fi to cellular) and host- 1464 unaware changes (e.g., NAT rebinding). 1466 For the host-aware case, let us consider the case of a Multipath QUIC 1467 connection where the client is a smartphone with both Wi-Fi and 1468 cellular. It advertised both addresses and the server currently 1469 enables only one client's sending uniflow, the initial one. The 1470 Initial Uniflow uses the Wi-Fi address. Then, for some reason, the 1471 Wi-Fi address becomes unusable. To preserve connectivity, the client 1472 might then decide to use the cellular address for its Initial sending 1473 uniflow. It thus sends a REMOVE_ADDRESS announcing the loss of the 1474 Wi-Fi address and an UNIFLOWS frame to inform that its Initial 1475 sending uniflow is now using the cellular address. If the cellular 1476 address validation succeeds (which could have been done as soon as 1477 the cellular address was advertised), the server can continue 1478 exchanging data through the cellular address. 1480 TODO: I don't think we have to change the Uniflow ID, we can just 1481 rely on (MP_)NEW_CONNECTION_ID and (MP_)RETIRE_CONNECTION_ID - hence, 1482 not sure the PATH_UPDATE frame is required, but will depend on the 1483 use cases 1485 TODO: update following paragraph to remove PATH_UPDATE 1487 However, both server and client might want to change the uniflow used 1488 on the cellular address for privacy concerns. If the server provides 1489 an additional uniflow (e.g., with Uniflow ID 1) through 1490 MP_NEW_CONNECTION_ID frame at the beginning of the connection, the 1491 client can perform the network path change directly and avoid using 1492 the Initial Uniflow Connection ID on the cellular network. This can 1493 be done using the PATH_UPDATE frame. It can indicate that the host 1494 stopped to use the Initial sending uniflow to use the one with 1495 Uniflow ID 1 instead. This frame is placed in the first packet sent 1496 to the new sending uniflow with its corresponding UCID. The client 1497 can then send the REMOVE_ADDRESS and UNIFLOWS frames on this new 1498 uniflow. Compared to the previous case, it is harder to link the 1499 uniflows with the IP addresses to observe that they belong to the 1500 same Multipath QUIC connection. 1502 For the host-unaware case, the situation is similar. In case of NAT 1503 rebinding, the server will observe a change in the 2-tuple (source 1504 IP, source port) of the receiving uniflow of the packet. The server 1505 first validates that the 2-tuple actually belongs to the client 1506 [I-D.ietf-quic-transport]. If it is the case, the server can send a 1507 PATH_UPDATE frame on a previously communicated but unused Uniflow ID. 1508 The client might have sent some packets with a given UCID on a 1509 different 4-tuple, but the server did not use the given UCID on that 1510 4-tuple. Because some on-path devices may rewrite the source IP 1511 address to forward packets via the available network attachments 1512 (e.g., a host located behind a multi-homed CPE), the server may 1513 inadvertently conclude that an uniflow is not anymore valid leading 1514 thus to frequently sending PATH_UPDATE frames as a function of the 1515 traffic distribution scheme enforced by the on-path device. To 1516 prevent such behavior, the server SHOULD wait for at least X seconds 1517 to ensure this is about a connection migration and not a side effect 1518 of an on-path multi-interfaced device. 1520 B.4. Scheduling Strategies 1522 The current QUIC design [I-D.ietf-quic-transport] offers a single 1523 scheduling space, i.e., which frames will be packed inside a given 1524 packet. With the simultaneous use of several sending uniflows, a 1525 second dimension is added, i.e., the sending uniflow on which the 1526 packet will be sent. This dimension can have a non negligible impact 1527 on the operations of Multipath QUIC, especially if the available 1528 sending uniflows exhibit very different network characteristics. 1530 The progression of the data flow depends on the reception of the 1531 MAX_DATA and MAX_STREAM_DATA frames. Those frames SHOULD be 1532 duplicated on several or all ACTIVE sending uniflows. This helps to 1533 limit the head-of-line blocking issue due to the transmission of the 1534 frames over a slow or lossy network path. 1536 The sending path on which (MP_)ACK frames are sent impacts the peer. 1537 The (MP_)ACK frame is notably used to determine the latency of a 1538 combination of uniflows. In particular, the peer can compute the 1539 round-trip-time of the combination of its sending uniflow with its 1540 receive one. The peer would compute the latency as the sum of the 1541 forward delay of the acknowledged uniflow and the return delay of the 1542 uniflow used to send the (MP_)ACK frame. Choosing between 1543 acknowledging packets symmetrically (on uniflow B to A if packet was 1544 sent on A to B) or not is up to the implementation, if only possible. 1545 However, hosts SHOULD keep a consistent acknowledgment strategy. 1546 Selecting a random uniflow to acknowledge packets may affect the 1547 performance of the connection. Notice that the inclusion of a 1548 timestamp field in the (MP_)ACK frame, as proposed by 1549 [I-D.huitema-quic-1wd], may help hosts estimate more precisely the 1550 one-way delay of each uniflow, therefore leading to improved 1551 scheduling decisions. Unlike MAX_DATA and MAX_STREAM_DATA, (MP_)ACK 1552 frames SHOULD NOT be systematically duplicated on several sending 1553 uniflows as they can induce a large network overhead. 1555 Appendix C. Change Log 1557 C.1. Since draft-deconinck-quic-multipath-03 1559 * Clarify the notion of asymmetric paths by introducing uniflows 1561 * Remove the PATH_UPDATE frame 1562 * Rename PATHS frame to UNIFLOWS frame and adapt its content 1564 * Add a sequence number to frames involving Address ID events (#4) 1566 * Disallow Zero-length connection ID (#2) 1568 * Correctly handle nonce computation (thanks to Florentin Rochet) 1570 * Focus on the core concepts of multipath and delegate algorithms to 1571 companion drafts 1573 * Updated text to match draft-ietf-quic-transport-27 1575 C.2. Since draft-deconinck-quic-multipath-02 1577 * Consider asymmetric paths 1579 C.3. Since draft-deconinck-quic-multipath-01 1581 * Include path policies considerations 1583 * Add practical considerations thanks to Mohamed Boucadair inputs 1585 * Adapt the RETIRE_CONNECTION_ID frame 1587 * Updated text to match draft-ietf-quic-transport-18 1589 C.4. Since draft-deconinck-quic-multipath-00 1591 * Comply with asymmetric Connection IDs 1593 * Add max_paths transport parameter to negotiate initial number of 1594 active paths 1596 * Path ID as a regular varint 1598 * Remove max_path_id transport parameter 1600 * Updated text to match draft-ietf-quic-transport-14 1602 C.5. Since draft-deconinck-multipath-quic-00 1604 * Added PATH_UPDATE frame 1606 * Added MAX_PATHS frame 1608 * No more packet header change 1609 * Implicit Path ID notification using Connection ID and 1610 NEW_CONNECTION_ID frames 1612 * Variable-length encoding for Path ID 1614 * Updated text to match draft-ietf-quic-transport-10 1616 * Fixed various typos 1618 Authors' Addresses 1620 Quentin De Coninck 1621 UCLouvain 1623 Email: quentin.deconinck@uclouvain.be 1625 Olivier Bonaventure 1626 UCLouvain 1628 Email: olivier.bonaventure@uclouvain.be