idnits 2.17.1 draft-deconinck-quic-multipath-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (20 August 2020) is 1346 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-13) exists of draft-ietf-quic-invariants-09 == Outdated reference: A later version (-34) exists of draft-ietf-quic-recovery-29 == Outdated reference: A later version (-34) exists of draft-ietf-quic-tls-29 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-29 -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 6824 (Obsoleted by RFC 8684) Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group Q. De Coninck 3 Internet-Draft O. Bonaventure 4 Intended status: Standards Track UCLouvain 5 Expires: 21 February 2021 20 August 2020 7 Multipath Extensions for QUIC (MP-QUIC) 8 draft-deconinck-quic-multipath-05 10 Abstract 12 This document specifies extensions to the QUIC protocol to enable the 13 simultaneous usage of multiple paths for a single connection. 15 These extensions are compliant with the single-path QUIC design and 16 preserve QUIC privacy features. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on 21 February 2021. 35 Copyright Notice 37 Copyright (c) 2020 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 42 license-info) in effect on the date of publication of this document. 43 Please review these documents carefully, as they describe your rights 44 and restrictions with respect to this document. Code Components 45 extracted from this document must include Simplified BSD License text 46 as described in Section 4.e of the Trust Legal Provisions and are 47 provided without warranty as described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 52 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 4 53 2.1. Notation Conventions . . . . . . . . . . . . . . . . . . 5 54 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 5 55 3.1. Moving from Bidirectional Paths to Uniflows . . . . . . . 5 56 3.2. Beyond Connection Migration . . . . . . . . . . . . . . . 7 57 3.3. Establishment of a Multipath QUIC Connection . . . . . . 8 58 3.4. Architecture of Multipath QUIC . . . . . . . . . . . . . 8 59 3.5. Uniflow Establishment . . . . . . . . . . . . . . . . . . 11 60 3.6. Exchanging Data over Multiple Uniflows . . . . . . . . . 12 61 3.7. Exchanging Addresses . . . . . . . . . . . . . . . . . . 14 62 3.8. Coping with Address Removals . . . . . . . . . . . . . . 15 63 3.9. Uniflow Migration . . . . . . . . . . . . . . . . . . . . 15 64 3.10. Congestion Control . . . . . . . . . . . . . . . . . . . 15 65 4. Mapping Uniflow IDs to Connection IDs . . . . . . . . . . . . 15 66 5. Using Multiple Uniflows . . . . . . . . . . . . . . . . . . . 16 67 5.1. Multipath Negotiation . . . . . . . . . . . . . . . . . . 16 68 5.1.1. Transport Parameter Definition . . . . . . . . . . . 17 69 5.2. Coping with Additional Remote Addresses . . . . . . . . . 17 70 5.3. Receiving Uniflow State . . . . . . . . . . . . . . . . . 17 71 5.4. Sending Uniflow State . . . . . . . . . . . . . . . . . . 18 72 5.5. Losing Addresses . . . . . . . . . . . . . . . . . . . . 20 73 6. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 20 74 6.1. MP_NEW_CONNECTION_ID Frame . . . . . . . . . . . . . . . 20 75 6.2. MP_RETIRE_CONNECTION_ID Frame . . . . . . . . . . . . . . 21 76 6.3. MP_ACK Frame . . . . . . . . . . . . . . . . . . . . . . 22 77 6.4. ADD_ADDRESS Frame . . . . . . . . . . . . . . . . . . . . 23 78 6.5. REMOVE_ADDRESS Frame . . . . . . . . . . . . . . . . . . 24 79 6.6. UNIFLOWS Frame . . . . . . . . . . . . . . . . . . . . . 25 80 7. Extension of the Meaning of Existing QUIC Frames . . . . . . 26 81 8. Security Considerations . . . . . . . . . . . . . . . . . . . 27 82 8.1. Nonce Computation . . . . . . . . . . . . . . . . . . . . 27 83 8.2. Validation of Exchanged Addresses . . . . . . . . . . . . 27 84 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 85 9.1. QUIC Transport Parameter Registry . . . . . . . . . . . . 27 86 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 28 87 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 28 88 11.1. Normative References . . . . . . . . . . . . . . . . . . 28 89 11.2. Informative References . . . . . . . . . . . . . . . . . 29 90 Appendix A. Comparison with Multipath TCP . . . . . . . . . . . 30 91 A.1. Multipath TCP Bidirectional Paths vs. QUIC Uniflows . . . 31 92 A.2. Uniflow Establishment . . . . . . . . . . . . . . . . . . 31 93 A.3. Exchanging Data over Multiple Uniflows . . . . . . . . . 31 94 A.4. Congestion Control . . . . . . . . . . . . . . . . . . . 31 95 A.5. ACK Frame . . . . . . . . . . . . . . . . . . . . . . . . 31 96 Appendix B. To move in companion drafts . . . . . . . . . . . . 32 97 B.1. Uniflow Establishment . . . . . . . . . . . . . . . . . . 32 98 B.2. Exchanging Addresses . . . . . . . . . . . . . . . . . . 32 99 B.3. Uniflow Migration . . . . . . . . . . . . . . . . . . . . 32 100 B.4. Scheduling Strategies . . . . . . . . . . . . . . . . . . 34 101 Appendix C. Change Log . . . . . . . . . . . . . . . . . . . . . 34 102 C.1. Since draft-deconinck-quic-multipath-04 . . . . . . . . . 34 103 C.2. Since draft-deconinck-quic-multipath-03 . . . . . . . . . 35 104 C.3. Since draft-deconinck-quic-multipath-02 . . . . . . . . . 35 105 C.4. Since draft-deconinck-quic-multipath-01 . . . . . . . . . 35 106 C.5. Since draft-deconinck-quic-multipath-00 . . . . . . . . . 35 107 C.6. Since draft-deconinck-multipath-quic-00 . . . . . . . . . 36 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 110 1. Introduction 112 Today's endhosts are equipped with several network interfaces. Users 113 expect to be able to seamlessly switch from one interface to another 114 one or use them simultaneously to aggregate bandwidth whenever 115 needed. During the last years, several multipath extensions to 116 transport protocols have been proposed (e.g., [RFC6824], [MPRTP], or 117 [SCTPCMT]). Multipath TCP [RFC6824] is the most mature one. It is 118 already deployed on popular smartphones, but also for other use cases 119 [RFC8041] [IETFJ]. 121 With regular TCP and UDP, all the packets that belong to a given flow 122 share the same 4-tuple {source IP address, source port number, 123 destination IP address, destination port number} that acts as an 124 identifier for this flow. This prevents these flows from using 125 multiple paths. 127 QUIC [I-D.ietf-quic-transport] does not use the 4-tuple as an 128 implicit connection identifier. Instead, a QUIC flow is identified 129 by a Connection ID. This enables QUIC to cope with events affecting 130 the 4-tuple, such as NAT rebinding or IP address changes. The QUIC 131 connection migration feature, specified in Section 9 of 132 [I-D.ietf-quic-transport], enables migrating a flow from one 4-tuple 133 to another one, sustaining in particular a connection over different 134 network paths. 136 Still, there is a void to specify simultaneous usage of QUIC over 137 available network paths for a single connection. Use cases such as 138 bandwidth aggregation or seamless network handovers would be 139 applicable to QUIC, as they are now with Multipath TCP 140 [RFC8041][IETFJ]. Experience with Multipath TCP on smartphones shows 141 that the ability to simultaneously use WLAN and cellular during 142 handovers improves the user perceived quality of experience. A 143 performance evaluation of an early solution for such use cases and a 144 comparison between Multipath QUIC and Multipath TCP may be found in 145 [MPQUIC]. 147 In this document, we leverage many of the lessons learned from the 148 design of Multipath TCP and the comments received on the first 149 versions of this document to propose extensions to the current QUIC 150 design to enable it to simultaneously use several network paths. 151 This document focuses mainly on network paths that are 152 distinguishable by the endpoints. 154 This document is organized as follows. It first provides in 155 Section 3 an overview of the operation of Multipath QUIC. It then 156 states the required changes in the current QUIC design 157 [I-D.ietf-quic-transport] and specifies in Section 5 the usage of 158 multiple paths. Finally, it discusses some security considerations. 160 2. Conventions and Definitions 162 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 163 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 164 "OPTIONAL" in this document are to be interpreted as described in BCP 165 14 [RFC2119] [RFC8174] when, and only when, they appear in all 166 capitals, as shown here. 168 We assume that the reader is familiar with the terminology used in 169 [I-D.ietf-quic-transport]. In addition, we define the following 170 terms: 172 * Uniflow: A unidirectional flow of packets between a QUIC host and 173 its peer. This flow is identified by an internal identifier, 174 called Uniflow ID. Packets sent over a uniflow use a Destination 175 Connection ID that may change during the lifetime of the 176 connection. When being in use, an uniflow is temporarily bound to 177 a 4-tuple (Source IP Address, Source Port Number, Destination IP 178 Address, Destination Port Number). 180 * Initial Uniflows: The two uniflows used by peers for the 181 establishment of a QUIC connection. One is the uniflow from the 182 client to the server and the other is the uniflow from the server 183 to the client. The cryptographic handshake is done on these 184 uniflows. These are identified by Uniflow ID 0. 186 2.1. Notation Conventions 188 Packet and frame diagrams use the format described in Section 12 of 189 [I-D.ietf-quic-transport]. 191 3. Overview 193 The design of QUIC [I-D.ietf-quic-transport] provides reliable 194 transport with multiplexing, confidentiality, integrity, and 195 authenticity of data flows. A wide range of devices on today's 196 Internet are multihomed. Examples include smartphones equipped with 197 both WLAN and cellular interfaces, but also regular dual-stack hosts 198 that use both IPv4 and IPv6. 200 The current design of QUIC does not enable multihomed devices to 201 efficiently use different paths simultaneously. This document 202 proposes multipath extensions with the following design goals: 204 * Each host keeps control on the number of uniflows being used over 205 the connection. 207 * The simultaneous usage of multiple uniflows should not introduce 208 new privacy concerns. 210 * A host must ensure that all the paths it uses actually reach its 211 peer to avoid packet flooding towards a victim (see 212 Section 21.12.3 of [I-D.ietf-quic-transport]) 214 * The multipath extensions should handle the asymmetrical nature of 215 paths between two peers. 217 We first explain why a multipath extension would be beneficial to 218 QUIC and then describe it at a high level. 220 3.1. Moving from Bidirectional Paths to Uniflows 222 To understand the overall architecture of the multipath extensions, 223 let us first refine the notion of "path". A path may be denoted by a 224 4-tuple (Source IP Address, Source Port Number, Destination IP 225 Address, Destination Port Number). In QUIC, this is namely a UDP 226 path from the local host to the remote one. Considering a smartphone 227 interacting with a single-homed server, the smartphone might want to 228 use one path over the WLAN network and another over the cellular one. 229 Those paths are not necessarily disjoint. For example, when 230 interacting with a dual-stack server, a smartphone may create two 231 paths over WLAN: one over IPv4 and the other one over IPv6. 233 A regular QUIC connection is composed of two independent active 234 packet flows. The first flow gathers the packets from the client to 235 the server and the other the packets from the server to the client. 236 To illustrate this, let us consider the example depicted in Figure 1. 237 In this example, the client has two IP addresses: IPc1 and IPc2. The 238 server has one single address: IPs1. 240 Probed flow IPc2 to IPs1 241 +---------------------------------------------------------+ 242 | | 243 | From IPc1 to IPs1 v 244 | +--------+ Client to Server Flow +--------+ 245 | | | =====================================> | | 246 +-- | Client | | Server | 247 | | <===================================== | | 248 +--------+ Server to Client Flow +--------+ 249 From IPc1* to IPs1* 251 Figure 1: Identifying Unidirectional Flows in QUIC 253 The client initiates the QUIC connection by sending packets towards 254 the server. The server then replies to the client if it accepts the 255 connection. If the handshake succeeds, the connection is 256 established. Still, this "path" actually consists in two independent 257 UDP flows. Each host has its own view of (i) the 4-tuple used to 258 send packets and (ii) the 4-tuple on which it receives packets. 259 While the 4-tuple used by the client to send packets may be the same 260 as the one seen and used by the server, this is not always the case 261 since middleboxes (e.g., NATs) may alter the 4-tuple of packets. 263 To further emphasize on this flow asymmetry, QUIC embeds a path 264 validation mechanism [I-D.ietf-quic-transport] assessing whether a 265 host can reach its peer through a given 4-tuple. This process is 266 unidirectional, i.e., the sender checks that it can reach the 267 receiver, but not the reverse. A host receiving a PATH_CHALLENGE 268 frame on a new 4-tuple may in turn initiate a path validation, but 269 this is up to the peer. 271 A QUIC connection is a collection of unidirectional flows (called, 272 uniflows). A plain QUIC connection is composed of a main uniflow 273 from client to server and another main uniflow from server to client. 274 These uniflows have their own Connection IDs. They are host- 275 specific, i.e., the uniflow(s) from A to B are different from the 276 ones from B to A. This potentially enables the use of unidirectional 277 links such as non-broadcast satellite links [RFC3077], which cannot 278 be used with TCP. 280 3.2. Beyond Connection Migration 282 Unlike TCP [RFC0793], QUIC is not bound to a particular 4-tuple 283 during the lifetime of a connection. A QUIC connection is identified 284 by a (set of) Connection ID(s), placed in the public header of each 285 QUIC packet. This enables hosts to continue the connection even if 286 the 4-tuple changes due to, e.g., NAT rebinding. This ability to 287 shift a connection from one 4-tuple to another is called: Connection 288 Migration. One of its use cases is fail-over when the IP address in 289 use fails but another one is available. A smartphone losing the WLAN 290 connectivity can then continue the connection over its cellular 291 interface, for instance. 293 A QUIC connection can thus start on a given set of uniflows, denoted 294 as the initial uniflows, and end on another ones. However, the 295 current QUIC design [I-D.ietf-quic-transport] assumes that only one 296 pair of uniflows is in use for a given connection. The current 297 transport specification [I-D.ietf-quic-transport] does not provide 298 means to distinguish path migration from simultaneous usage of 299 available uniflows for a given connection. 301 This document fills that void. It first proposes mechanisms to 302 communicate endhost addresses to the peer. It then leverages the 303 Address Validation procedure with PATH_CHALLENGE and PATH_RESPONSE 304 frames described in Section 8 of [I-D.ietf-quic-transport] to verify 305 whether the additional addresses advertised by the host are 306 reachable. In this case, those addresses can be used to initiate new 307 uniflows to spread packets over several network paths following a 308 packet scheduling policy that is out of scope of this document. 310 TODO: Add a companion document discussing the packet scheduling and 311 path management considerations. 313 The example of Figure 2 illustrates a data exchange between a dual- 314 homed client sending a request spanning two packets and a single- 315 homed server. Uniflow IDs are independently chosen by each host. In 316 the presented example, the client sends packets over WLAN on Uniflow 317 0 and over LTE on Uniflow 1, while the packets sent by the server 318 over WLAN are on Uniflow 2 and those over LTE are on Uniflow 1. 320 Server Phone Server 321 via WLAN via LTE 322 ------- ------- ----- 323 | Pkt(DCID=A,PN=5,frames=[ | | 324 | STREAM("Request (1/2)")]) | Pkt(DCID=B,PN=1,frames=[ | 325 |<----------------------------| STREAM("Request (2/2)")]) | 326 | Pkt(DCID=E,PN=1,frames=[ |-------- | 327 | ACK(LargestAcked=5)]) | |---------- | 328 |---------------------------->| |-------- | 329 | Pkt(DCID=E,PN=2,frames=[ | |-->| 330 | STREAM("Response 1")]) | Pkt(DCID=D,PN=1,frames=[ | 331 |---------------------------->| MPACK(UID=1,LargestAck=1), | 332 | | STREAM("Response 2")]) ---| 333 | Pkt(DCID=A,PN=6,frames=[ | ---------| | 334 | MPACK(UID=2,LargestAck=2), | ----------| | 335 | MPACK(UID=1,LargestAck=1)])|<------| | 336 |<----------------------------| | 337 | Pkt(DCID=E,PN=3,frames=[ | Pkt(DCID=D,PN=2,frames=[ | 338 | STREAM("Response 3")]) | STREAM("Response 4")]) | 339 |---------------------------->| ----| 340 | | ------| | 341 | ... | ... <---------| | 343 Figure 2: Data flow with Multipath QUIC 345 The remaining of this section presents a high-level overview of the 346 multipath operations in QUIC. 348 3.3. Establishment of a Multipath QUIC Connection 350 A Multipath QUIC connection starts as a regular QUIC connection 351 [I-D.ietf-quic-transport] [I-D.ietf-quic-tls]. The multipath 352 extensions defined in this document are negotiated using the 353 "max_sending_uniflow_id" transport parameter. Any value for this 354 transport parameter advertises the support of the multipath 355 extensions. 357 Notice that a host advertising a value of 0 for the 358 "max_sending_uniflow_id" transport parameter indicates that it does 359 not want additional uniflows to send packets, but it still supports 360 the multipath extensions. Such situation might be useful when the 361 host does not require multiple uniflows for packet sending but still 362 wants to let the peer use multiple uniflows to reach it. 364 3.4. Architecture of Multipath QUIC 366 To illustrate the architecture of a Multipath QUIC connection, 367 consider Figure 3. 369 +--------+ CID A - Uniflow ID 1 +--------+ 370 | | =====================================> | | 371 | | CID B - Uniflow ID 0 | | 372 | | =====================================> | | 373 | | | | 374 | Client | CID C - Uniflow ID 0 | Server | 375 | | <===================================== | | 376 | | CID D - Uniflow ID 1 | | 377 | | <===================================== | | 378 | | CID E - Uniflow ID 2 | | 379 | | <===================================== | | 380 +--------+ +--------+ 382 Figure 3: An Example of Uniflow Distribution over a Multipath 383 QUIC Connection 385 Once established, a Multipath QUIC connection consists in one or more 386 uniflows from the client to the server and one or more uniflows from 387 the server to the client. The number of uniflows in one direction 388 can be different from the one in the other direction. The example in 389 Figure 3 shows two uniflows from the client to the server and three 390 uniflows from the server to the client. From the end-hosts' 391 viewpoint, they observe two kinds of uniflows: 393 * Sending uniflows: uniflows over which the host can send packets 395 * Receiving uniflows: uniflows over which the host can receive 396 packets 398 Reconsidering the example in Figure 3, the client has two sending 399 uniflows and three receiving uniflows. The server has three sending 400 uniflows and two receiving uniflows. There is thus a one-to-one 401 mapping between the sending uniflows of a host and the receiving 402 uniflows of its peer. A uniflow is seen as a sending uniflow from 403 the sender's perspective and as a receiving uniflow from the 404 receiver's viewpoint. 406 Each uniflow is associated with a specific four-tuple and identified 407 by a Uniflow ID, as shown in Figure 4. 409 Client state 410 +-----------------------------------------------------+ 411 | Connection | 412 | +-----------+ +-----------+ ... +-------------+ | 413 | | Sending | | Sending | ... | Sending | | 414 | | Uniflow 0 | | Uniflow 1 | | Uniflow N-1 | | 415 | | UCID set A| | UCID set B| ... | UCID set C | | 416 | | Tuple A | | Tuple B | ... | Tuple C | | 417 | | PNS A | | PNS B | | PNS C | | 418 | +-----------+ +-----------+ ... +-------------+ | 419 | +-----------+ +-----------+ ... +-------------+ | 420 | | Receiving | | Receiving | ... | Receiving | | 421 | | Uniflow 0 | | Uniflow 1 | | Uniflow M-1 | | 422 | | UCID set X| | UCID set Y| ... | UCID set Z | | 423 | | Tuple X | | Tuple Y | ... | Tuple Z | | 424 | | PNS X | | PNS Y | | PNS Z | | 425 | +-----------+ +-----------+ ... +-------------+ | 426 +-----------------------------------------------------+ 428 Server state 429 +-----------------------------------------------------+ 430 | Connection | 431 | +-----------+ +-----------+ ... +-------------+ | 432 | | Sending | | Sending | ... | Sending | | 433 | | Uniflow 0 | | Uniflow 1 | | Uniflow M-1 | | 434 | | UCID set X| | UCID set Y| ... | UCID set Z | | 435 | | Tuple X* | | Tuple Y* | ... | Tuple Z* | | 436 | | PNS X | | PNS Y | | PNS Z | | 437 | +-----------+ +-----------+ ... +-------------+ | 438 | +-----------+ +-----------+ ... +-------------+ | 439 | | Receiving | | Receiving | ... | Receiving | | 440 | | Uniflow 0 | | Uniflow 1 | | Uniflow N-1 | | 441 | | UCID set A| | UCID set B| ... | UCID set C | | 442 | | Tuple A* | | Tuple B* | ... | Tuple C* | | 443 | | PNS A | | PNS B | | PNS C | | 444 | +-----------+ +-----------+ ... +-------------+ | 445 +-----------------------------------------------------+ 447 Figure 4: Architectural view of Multipath QUIC for a host having 448 N sending uniflows and M receiving uniflows 450 A Multipath QUIC connection starts using two Initial Uniflows, 451 identified by Uniflow ID 0 on each peer. The packets can then be 452 spread over several uniflows. Each uniflow has its (set of) Uniflow 453 Connection ID(s) (UCID) packets that are used to explicitly mark 454 where they belong to. Depending on the direction of the uniflow, the 455 host keeps either the Uniflow Source Connection ID (USCID, for the 456 receiving uniflows) or the Uniflow Destination Connection ID (USCID, 457 for the sending uniflows). Notice that the (set of) UDCID(s) of a 458 sending uniflow of a host is the same as the (set of) USCID(s) of the 459 corresponding receive uniflow of the remote peer. 461 Preventing the linkability of different uniflows is an important 462 requirement for the multipath extensions 463 [I-D.huitema-quic-mpath-req]. We address it by using UCIDs as 464 implicit uniflow identifiers. This makes the linkability harder than 465 having explicit signaling as in earlier version of this draft. 466 Furthermore, it does not require any public header change and thus 467 preserves the QUIC invariants [I-D.ietf-quic-invariants]. 469 When a uniflow is in use, each endhost associates it with a network 470 path. In practice, this consists in a particular 4-tuple over which 471 packets are sent (or received) on a sending (or receiving) uniflow. 472 Each endhost has a specific vision of the 4-tuple, which might differ 473 between endhosts. For instance, a client located behind a NAT sends 474 data from a private IP address and the server will receive packets 475 coming from the NAT's public IP address. Notice that while uniflows 476 may share a common network path, this is not mandatory. 478 Each uniflow is an independent flow of packets over a given network 479 path. Uniflows can experience very different network conditions 480 (latency, bandwidth, ...). To handle this, each uniflow has its own 481 packet sequence number space. 483 In addition to the UCIDs, 4-tuple and packet number space, some 484 additional information is maintained for each uniflow. The Uniflow 485 ID identifies the uniflow at the frame level and ensures uniqueness 486 of the nonce (see Section 8.1 for details) while limiting the number 487 of concurrently used uniflows. 489 3.5. Uniflow Establishment 491 The "max_sending_uniflow_id" transport parameter exchanged during the 492 cryptographic handshake fixes an upper bound on the number of sending 493 uniflows a host wants to support. Then, hosts provide to their peer 494 Uniflow Connection IDs to use on uniflows. Both hosts dynamically 495 control how many sending uniflows can currently be in use by the 496 peer, i.e., the number of different Uniflow IDs proposed to the peer. 497 While the sender determines the upper bound of sending paths it can 498 have, it is the receiver that initializes uniflows, as the sender 499 needs a UCID communicated by the receiver before using a uniflow. 501 Notice that the peers might advertise different values for the 502 "max_sending_uniflow_id" transport parameters, setting different 503 upper bounds to the sending and receiving uniflows of each host. 505 Hosts initiate the creation of their receiving uniflows by sending 506 MP_NEW_CONNECTION_ID frames (see Section 6.1) which are an extended 507 version of the NEW_CONNECTION_ID frame. This frame associates a UCID 508 to a uniflow. Upon reception of the MP_NEW_CONNECTION_ID frame, a 509 host can start using the proposed sending uniflow having the 510 referenced Uniflow ID by marking sent packets with the provided UCID. 511 Therefore, once a host sends a MP_NEW_CONNECTION_ID frame, it 512 announces that it is ready to receive packets from that Uniflow ID 513 with the proposed UCID. As frames are encrypted, adding new uniflows 514 over a QUIC connection does not leak cleartext identifiers 515 [I-D.huitema-quic-mpath-req]. 517 A server might provide several Uniflow Connection IDs for the same 518 Uniflow ID with multiple MP_NEW_CONNECTION_ID frames. This can be 519 useful to cope with migration cases, as described in Section 3.9. 520 Multipath QUIC is thus asymmetrical. 522 3.6. Exchanging Data over Multiple Uniflows 524 A QUIC packet acts as a container for one or more frames. Multipath 525 QUIC uses the same STREAM frames as QUIC to carry data. A byte 526 offset is associated with the data payload. One of the key design of 527 (Multipath) QUIC is that frames are independent of the packets 528 carrying them. This implies that a frame transmitted over one 529 uniflow could be retransmitted later on another uniflow without any 530 change. Furthermore, all current QUIC frames are idempotent and 531 could be optimistically duplicated over several uniflows. 533 The uniflow on which data is sent is a packet-level information. 534 This means that a frame can be sent regardless of the uniflow of the 535 packet carrying it. Other flow control considerations like the 536 stream receive window advertised by the MAX_STREAM_DATA frame remain 537 unchanged when there are multiple sending uniflows. 539 As previously described, Multipath QUIC might face reordering at 540 packet-level when using uniflows having different latencies. The 541 presence of different Uniflow Connection IDs ensures that the packets 542 sent over a given uniflow will contain monotonically increasing 543 packet numbers. To ensure more flexibility and potentially to reduce 544 the ACK block section of the (MP_)ACK frame when aggregating 545 bandwidth of uniflows exhibiting different network characteristics, 546 each uniflow keeps its own monotonically increasing Packet Number 547 space. This potentially allows sending up to 2 * 2^64 packets on a 548 QUIC connection since each uniflow has its own packet number space 549 (see Section 8.1 for the detail of this limit). 551 With the introduction of multiple uniflows, there is a need to 552 acknowledge packets sent on different uniflows separately. The 553 packets sent on Initial Uniflows (with Uniflow ID 0) are still 554 acknowledged with regular ACK frames, such that no modification is 555 introduced in a core frame. For the other uniflows, the multipath 556 extensions introduce a MP_ACK frame which prefixes the ACK frame with 557 a Uniflow ID field indicating from which receiving uniflow the host 558 acknowledges packets. To better explain this, let us consider the 559 situation illustrated in Figure 5. 561 Sending uniflow 0 - CID A | Receiving uniflow 0 - CID A 562 Sending uniflow 1 - CID B | Receiving uniflow 1 - CID B 563 Receiving uniflow 0 - CID C | Sending uniflow 0 - CID C 564 Receiving uniflow 1 - CID D | Sending uniflow 1 - CID D 565 Receiving uniflow 2 - CID E | Sending uniflow 2 - CID E 567 Client Server 568 ------ ------ 569 | | 570 | Pkt(DCID=B,PN=42,frames=[STREAM("Request")]) | 571 |--------------------------- | 572 | |---------------------------->| 573 | | 574 | Pkt(DCID=E,PN=58,frames=[ | 575 | STREAM("Response"), MP_ACK(UID=1,LargestAcked=42)]) | 576 | ------------------------------| 577 |<-------------------------| | 578 | | 580 Figure 5: Acknowledging Packets Sent on Uniflows 582 Here five uniflows are in use, two in the client to server direction 583 and three in the reverse one. The client first sends a packet on its 584 sending Uniflow 1 (linked to CID B). The server receives the packet 585 on its receiving Uniflow 1. Therefore, it generates a MP_ACK frame 586 for Uniflow ID 1 and transmits it to the client. The server can 587 choose any of its sending uniflows to transmit this frame. In the 588 provided situation, the server sends its packets on Uniflow 2. The 589 client thus receives this packet on its receiving Uniflow 2. 591 Similarly, packets sent over a given uniflow might be acknowledged by 592 (MP_)ACK frames sent on another uniflow that does not share the same 593 network path. Looking at Figure 2 again, "Response 2" packet on 594 server's sending uniflow 1 with DCID D using the LTE network is 595 acknowledged by a MP_ACK frame received on a uniflow using the WLAN 596 network. 598 3.7. Exchanging Addresses 600 When a multi-homed device connects to a dual-stacked server using its 601 IPv4 address, it is aware of its local addresses (e.g., the WLAN and 602 the cellular ones) and the IPv4 remote address used to establish the 603 QUIC connection. If the client wants to create new uniflows and use 604 them over the IPv6 network, it needs to learn the other addresses of 605 the remote peer. 607 This is possible with the ADD_ADDRESS frames that are sent by a 608 Multipath QUIC host to advertise its current addresses. Each 609 advertised address is identified by an Address ID. The addresses 610 attached to a host can vary during the lifetime of a Multipath QUIC 611 connection. A new ADD_ADDRESS frame is transmitted when a host has a 612 new address. This ADD_ADDRESS frame is protected as other QUIC 613 control frames, which implies that it cannot be spoofed by attackers. 614 The communicated address MUST first be validated by the receiving 615 host before it starts using it as described in Section 8 of 616 [I-D.ietf-quic-transport]. This process ensures that the advertised 617 address actually belongs to the peer and that the peer can receive 618 packets sent by the host on the provided address. It also prevents 619 hosts from launching amplification attacks to a victim address. 621 If the client is behind a NAT, it could announce a private address in 622 an ADD_ADDRESS frame. In such situations, the server would not be 623 able to validate the communicated address. The client might still 624 use its NATed addresses to start using its sending uniflows. To 625 enable the server to make the link between the private and the public 626 addresses and hence conciliate the different 4-tuple views, Multipath 627 QUIC provides the UNIFLOWS frame that lists the current active 628 sending Uniflow IDs along with their associated local Address ID. 629 Notice that a host might also discover the public addresses of its 630 peer by observing its remote IP addresses associated to the 631 connection. 633 A receiving uniflow is active as soon as the host has sent the 634 MP_NEW_CONNECTION_ID frames proposing the corresponding Uniflow 635 Connection IDs to its peer. A sending uniflow is active when it has 636 received its Uniflow Connection IDs and is bound to a validated 637 4-tuple. The UNIFLOWS frame indicates the local Address IDs that the 638 uniflow uses from the sender's perspective. With this information, 639 the remote host can validate the public address and associate the 640 advertised one with the perceived addresses. 642 3.8. Coping with Address Removals 644 During the lifetime of a QUIC connection, a host might lose some of 645 its addresses. A concrete example is a smartphone going out of reach 646 of a WLAN network or shutting off one of its network interfaces. 647 Such address removals are advertised using REMOVE_ADDRESS frames. 648 The REMOVE_ADDRESS frame contains the Address ID of the lost address 649 previously communicated through ADD_ADDRESS. Notice that because a 650 given Address ID might encounter several events that need to be 651 ordered (e.g., ADD_ADDRESS, REMOVE_ADDRESS and ADD_ADDRESS again), 652 both ADD_ADDRESS and REMOVE_ADDRESS frames include an Address ID 653 related Sequence Number. 655 3.9. Uniflow Migration 657 At a given time, a Multipath QUIC endpoint gathers a set of active 658 sending and receiving uniflows, each associated to a 4-tuple. This 659 association is mutable. Hosts can change the 4-tuple used by their 660 sending uniflows at any time, enabling QUIC to migrate uniflows from 661 one network path to another. Yet, to address privacy issues due to 662 the linkability of addresses, hosts should avoid reusing the same 663 Connection ID used by a sending uniflow when the 4-tuple changes, as 664 described in Section 9.5 of [I-D.ietf-quic-transport]. 666 3.10. Congestion Control 668 The QUIC congestion control scheme is defined in 669 [I-D.ietf-quic-recovery]. This congestion control scheme is not 670 suitable when several sending uniflows are active. Using the 671 congestion control scheme defined in [I-D.ietf-quic-recovery] with 672 Multipath QUIC would result in unfairness. Each sending uniflow of a 673 Multipath QUIC connection MUST have its own congestion control state. 674 As for Multipath TCP, the windows of the different sending uniflows 675 MUST be coupled together [RFC6356]. 677 4. Mapping Uniflow IDs to Connection IDs 679 As described in the overview section, hosts need to identify on which 680 uniflows packets are sent. The Uniflow ID must then be inferred from 681 the public header. This is done by using the Destination Connection 682 ID field of Short Header packets. 684 The Initial Uniflow Connection IDs are determined during the 685 cryptographic handshake and actually correspond to both Connection 686 IDs in the current single-path QUIC design [I-D.ietf-quic-transport]. 687 Additional Uniflow Connection IDs for the Initial Uniflows can be 688 provided with the regular NEW_CONNECTION_ID frames. The Uniflow 689 Connection IDs of the other uniflows are determined when the 690 MP_NEW_CONNECTION_ID frames are exchanged. 692 Hosts MUST accept packets coming from their peer using the UCIDs they 693 proposed in the (MP_)NEW_CONNECTION_ID frames they sent and associate 694 them with the corresponding receiving Uniflow ID. Upon reception of 695 a (MP_)NEW_CONNECTION_ID frame, hosts MUST acknowledge it and MUST 696 store the advertised Uniflow Destination Connection ID and the 697 Uniflow ID of the proposed sending uniflow. 699 Hosts MUST ensure that all advertised Uniflow Connection IDs are 700 available for the whole connection lifetime, unless they have been 701 retired by their peer in the meantime by the reception of a 702 (MP_)RETIRE_CONNECTION_ID. 704 A host MUST NOT send MP_NEW_CONNECTION_ID frames with a Uniflow ID 705 greater than the value of "max_sending_uniflow_id" advertised by its 706 peer. 708 5. Using Multiple Uniflows 710 This section describes in details the Multipath QUIC operations. 712 5.1. Multipath Negotiation 714 The Multipath negotiation takes place during the cryptographic 715 handshake with the "max_sending_uniflow_id" transport parameter. A 716 QUIC connection is initially single-path in QUIC. During this 717 handshake, hosts advertise their support for multipath operations. 718 When a host advertises a value for the "max_sending_uniflow_id" 719 transport parameter, it indicates that it supports the multipath 720 extensions, i.e., the extensions defined in this document (not to be 721 mixed with the availability of local multiple network paths). If any 722 host does not advertise the "max_sending_uniflow_id" transport 723 parameter, multipath extensions are disabled. 725 The usage of multiple uniflows relies on the ability to use several 726 Connection IDs over a same QUIC connection. Therefore, zero-length 727 Connection IDs MUST NOT be used if the peer advertises a value 728 different from 0 for the "max_sending_uniflow_id" transport 729 parameter. 731 5.1.1. Transport Parameter Definition 733 A host MAY use the following transport parameter: 735 max_sending_uniflow_id (0x40): Indicates the support of the 736 multipath extension presented in this document, regardless of the 737 carried value. Its integer value puts an upper bound on the 738 number of sending uniflows the host advertising the value is ready 739 to support. If absent, this means that the host does not agree to 740 use the multipath extension over the connection. 742 5.2. Coping with Additional Remote Addresses 744 Hosts can learn remote addresses either by receiving ADD_ADDRESS 745 frames or observing the 4-tuple of incoming packets. Hosts MUST 746 first validate the newly learned remote IP addresses before starting 747 sending packets to those addresses. This requirement is explained in 748 Section 8.2. Hosts MUST initiate Address Validation Procedure as 749 specified in [I-D.ietf-quic-transport]. 751 A host MAY cache a validated address for a limited amount of time. 753 5.3. Receiving Uniflow State 755 When proposing uniflows to their peer, hosts need to maintain some 756 state for their receiving uniflows. This state is created upon the 757 sending of a first MP_NEW_CONNECTION_ID frame proposing the 758 corresponding Uniflow ID. As long as there is still one active 759 Uniflow Connection ID for this receiving uniflow (i.e., one UCID 760 which was not retired yet using a MP_RETIRE_CONNECTION_ID), the host 761 MUST accept packets over the receiving uniflow. Once created, hosts 762 MUST keep the following receiving uniflow information: 764 Uniflow ID: An integer that uniquely identifies the receiving 765 uniflow in the connection. This value is immutable. 767 Uniflow Connection IDs: Possible values for the Connection ID field 768 of packets belonging to this receiving uniflow. This value 769 contains the sequence of active UCIDs that were advertised in 770 previously sent MP_NEW_CONNECTION_ID frames. Notice that this 771 sequence might be empty, e.g., when all advertised UCIDs have been 772 retired by the peer. 774 Packet Number Space: Packet number space dedicated to this receiving 775 uniflow. Packet number considerations described in Section 12.3 776 of [I-D.ietf-quic-transport] apply within a given receiving 777 uniflow. 779 Associated 4-tuple: The tuple (sIP, dIP, sport, dport) currently 780 observed to receive packets over this uniflow. This value is 781 mutable, because a host might receive a packet with a different 782 (possibly) validated remote address and/or port than the one 783 previously recorded. If a host observes a change in the 4-tuple 784 of the receiving uniflow, it follows the considerations of 785 Section 9.5 of [I-D.ietf-quic-transport]. 787 Associated local Address ID: The Address ID advertised in 788 ADD_ADDRESS frames sent by the peer corresponding to the local 789 address used to receive packets. This helps to generate UNIFLOWS 790 frames advertising the mapping between uniflows and addresses. 791 The addresses on which the connection was established have Address 792 ID 0. 794 Hosts can also collect network measurements on a per-uniflow basis, 795 like the number of packets received. 797 5.4. Sending Uniflow State 799 During a Multipath QUIC connection, hosts maintain some state for 800 sending uniflows. The state of the sending uniflow determines 801 information that hosts are required to store. The possible sending 802 uniflow states are depicted in Figure 6. 804 TODO: intermediate state for address validation? Yes 806 o 807 | 808 | receive a first MP_NEW_CONNECTION_ID 809 | with the associated Uniflow ID 810 | 811 v path usage over a validated 4-tuple 812 +----------+ ------------------------------------> +----------+ 813 | UNUSED | | ACTIVE | 814 +----------+ <------------------------------------ +----------+ 815 address change or retired UCID 817 Figure 6: Finite-State Machine describing the possible states of 818 a sending uniflow 820 Once a sending uniflow has been proposed by the peer in a received 821 MP_NEW_CONNECTION_ID frame, it is in the UNUSED state. In this 822 situation, hosts MUST keep the following sending uniflow information: 824 Uniflow ID: An integer that uniquely identifies the sending uniflow 825 in the connection. This value is immutable. 827 Uniflow Connection IDs: Possible values for the Connection ID field 828 of packets belonging to this sending uniflow. This value contains 829 the sequence of active UCIDs that were advertised in previously 830 received MP_NEW_CONNECTION_ID frames. Notice that this sequence 831 might be empty, e.g., when all advertised UCIDs have been retired. 833 Sending Uniflow State: The current state of the sending uniflow, 834 being one of the values presented in Figure 6. 836 Packet Number Space: Packet number space dedicated to this sending 837 uniflow. Packet number considerations described in Section 12.3 838 of [I-D.ietf-quic-transport] apply within a given sending uniflow. 840 When the host wants to start using the sending uniflow over a 841 validated address, the sending uniflow goes to the ACTIVE state. 842 This is the state where a sending uniflow can be used to send 843 packets. Having an uniflow in ACTIVE state only guarantees that it 844 can be used, but the host is not forced to. In addition to the 845 fields required in the UNUSED state, the following elements MUST be 846 tracked: 848 Associated 4-tuple: The tuple (sIP, dIP, sport, dport) currently 849 used to packets over this uniflow. This value is mutable, as the 850 host might decide to change its local (or remote) address and/or 851 port. A host that changes the 4-tuple of a sending uniflow SHOULD 852 migrate it. 854 Associated (local Address ID, remote Address ID) tuple: Those 855 identifiers come from the ADD_ADDRESS sent (local) and received 856 (remote). This enables a host to temporarily stop using a sending 857 uniflow when, e.g., the remote Address ID is declared as lost in a 858 REMOVE_ADDRESS. The addresses on which the connection was 859 established have Address ID 0. The reception of UNIFLOWS frames 860 helps hosts associate the remote Address ID used by the sending 861 uniflow. 863 Congestion controller: A congestion window limiting the transmission 864 rate of the sending uniflow. 866 Performance metrics: Basic statistics such as one-way delay or the 867 number of packets sent. This information can be useful when a 868 host needs to perform packet scheduling decisions and flow 869 control. 871 It might happen that a sending path is temporarily unavailable, 872 because one of the endpoint's addresses is no more available or 873 because the host retired all the UCIDs of the sending uniflow. In 874 such cases, the path goes back to the UNUSED state. When performing 875 a transition back to the UNUSED state, hosts MUST reset the 876 additional state added by the ACTIVE state. In the UNUSED state, the 877 host MUST NOT send non-probing packets on it. At this state, the 878 host might want to restart using the uniflow over another validated 879 4-tuple, switching the uniflow state back to the ACTIVE state. 880 However, its congestion controller state MUST be restarted and its 881 performance metrics SHOULD be reset. 883 5.5. Losing Addresses 885 During the lifetime of a connection, a host might lose addresses, 886 e.g., a network interface that was shut down. All the ACTIVE sending 887 uniflows that were using that local address MUST stop sending packets 888 from that address. To advertise the loss of an address to the peer, 889 the host MUST send a REMOVE_ADDRESS frame indicating which local 890 Address IDs has been lost. The host MUST also send an UNIFLOWS frame 891 indicating the status of the remaining ACTIVE uniflows. 893 Upon reception of the REMOVE_ADDRESS, the receiving host MUST stop 894 using the ACTIVE sending uniflows affected by the address removal. 896 Hosts MAY reuse one of these sending uniflows by changing the 897 assigned 4-tuple. In this case, it MUST send an UNIFLOWS frame 898 describing that change. 900 6. New Frames 902 To support the multipath operations, new frames have been defined to 903 coordinate hosts. All frames defined in this document MUST be 904 exchanged in 1-RTT packets. A host receiving one of the following 905 frames in other encryption context MUST close the connection with a 906 PROTOCOL_VIOLATION error. 908 6.1. MP_NEW_CONNECTION_ID Frame 910 The MP_NEW_CONNECTION_ID frame (type=0x40) is an extension of the 911 NEW_CONNECTION_ID frame defined by [I-D.ietf-quic-transport]. It 912 provides the peer with alternative Connection IDs and associates them 913 to a particular uniflow using the Uniflow ID. 915 The format of the MP_NEW_CONNECTION_ID frame is as follows. 917 0 1 2 3 918 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 919 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 920 | Uniflow ID (i) ... 921 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 922 | Sequence Number (i) ... 923 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 924 | Retire Prior To (i) ... 925 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 926 | Length (8) | | 927 +-+-+-+-+-+-+-+-+ Connection ID (8..160) + 928 | ... 929 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 930 | | 931 + + 932 | | 933 + Stateless Reset Token (128) + 934 | | 935 + + 936 | | 937 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 939 Figure 7: MP_NEW_CONNECTION_ID frame 941 Compared to the frame specified in [I-D.ietf-quic-transport], an 942 Uniflow ID varint field of is prefixed to associate the Connection ID 943 with a uniflow. This frame can be sent by both hosts. Upon 944 reception of the frame with a specified Uniflow ID, the peer MUST 945 update the related sending uniflow state and store the communicated 946 Connection ID. 948 To limit the delay of the multipath usage upon handshake completion, 949 hosts SHOULD send MP_NEW_CONNECTION_ID frames for receive uniflows 950 they allow using as soon the connection establishment completes. 952 The generation of Connection ID MUST follow the same considerations 953 as presented in Section 5.1 of [I-D.ietf-quic-transport]. 955 6.2. MP_RETIRE_CONNECTION_ID Frame 957 The MP_RETIRE_CONNECTION_ID frame (type=0x41) is an extension of the 958 RETIRE_CONNECTION_ID frame defined by [I-D.ietf-quic-transport]. It 959 indicates that the end-host will no longer use a Connection ID 960 related to a given uniflow that was issued by its peer. 962 The format of the MP_RETIRE_CONNECTION_ID frame is shown below. 964 0 1 2 3 965 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 966 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 967 | Uniflow ID (i) ... 968 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 969 | Sequence Number (i) ... 970 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 972 Figure 8: MP_RETIRE_CONNECTION_ID frame 974 The frame is handled as described in [I-D.ietf-quic-transport] on an 975 uniflow basis. 977 6.3. MP_ACK Frame 979 The MP_ACK frame (types 0x42 and 0x43) is an extension of the ACK 980 frame defined by [I-D.ietf-quic-transport]. It allows hosts to 981 acknowledge packets that were sent on non-initial uniflows. 983 The format of the MP_ACK frame is shown below. 985 0 1 2 3 986 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 987 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 988 | Uniflow ID (i) ... 989 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 990 | Largest Acknowledged (i) ... 991 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 992 | ACK Delay (i) ... 993 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 994 | ACK Range Count (i) ... 995 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 996 | First ACK Range (i) ... 997 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 998 | ACK Ranges (*) ... 999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1000 | [ECN Counts] ... 1001 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1003 Figure 9: ACK frame adapted to Multipath 1005 Compared to the ACK frame, the MP_ACK frame is prefixed by a varint 1006 Uniflow ID field indicating to which uniflow the acknowledged packet 1007 sequence numbers relate. 1009 6.4. ADD_ADDRESS Frame 1011 The ADD_ADDRESS frame (type=0x44) is used by a host to advertise its 1012 currently reachable addresses. 1014 The format of the ADD_ADDRESS frame is shown below. 1016 0 1 2 3 1017 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1018 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1019 |0|0|0|P|IPVers.|Address ID (8) | 1020 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1021 | Sequence Number (i) ... 1022 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1023 |Interface T.(8)| 1024 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1025 | IP Address (32/128) ... 1026 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1027 | [Port (16)] | 1028 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1030 Figure 10: ADD_ADDRESS Frame 1032 The ADD_ADDRESS frame contains the following fields. 1034 Reserved bits: The three most-significant bits of the first byte are 1035 set to 0, and are reserved for future use. 1037 P bit: The fourth most-significant bit of the first byte indicates, 1038 if set, the presence of the Port field. 1040 IPVers.: The remaining four least-significant bits of the first byte 1041 contain the version of the IP address contained in the IP Address 1042 field. 1044 Address ID: An unique identifier for the advertised address for 1045 tracking and removal purposes. This is needed when, e.g., a NAT 1046 changes the IP address such that both hosts see different IP 1047 addresses for a same network path. 1049 Sequence Number: An Address ID related sequence number of the event. 1050 The sequence number space is shared with REMOVE_ADDRESS frames 1051 mentioning the same Address ID. 1053 Interface Type: Used to provide an indication about the interface 1054 type to which this address is bound. The following values are 1055 defined: 1057 * 0: fixed. Used as default value. 1059 * 1: WLAN 1061 * 2: cellular 1063 IP Address: The advertised IP address, in network order. 1065 Port: This optional field indicates the port number related to the 1066 advertised IP address. When this field is present, it indicates 1067 that an uniflow can use the 2-tuple (IP addr, port). 1069 Upon reception of an ADD_ADDRESS frame, the receiver SHOULD store the 1070 communicated address for future use. 1072 The receiver MUST NOT send packets others than validation ones to the 1073 communicated address without having validated it as specified in 1074 Section 8 of [I-D.ietf-quic-transport]. ADD_ADDRESS frames SHOULD 1075 contain globally reachable addresses. Link-local and possibly 1076 private addresses SHOULD NOT be exchanged. 1078 6.5. REMOVE_ADDRESS Frame 1080 The REMOVE_ADDRESS frame (type=0x45) is used by a host to signal that 1081 a previously announced address was lost. 1083 The format of the REMOVE_ADDRESS frame is shown below. 1085 0 1 2 3 1086 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1087 +-+-+-+-+-+-+-+-+ 1088 |Address ID (8) | 1089 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1090 | Sequence Number (i) ... 1091 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1093 Figure 11: REMOVE_ADDRESS Frame 1095 The REMOVE_ADDRESS frame contains the following fields. 1097 Address ID: The identifier of the address to remove. 1099 Sequence Number: An Address ID related sequence number of the event. 1100 The sequence number space is shared with ADD_ADDRESS frames 1101 mentioning the same Address ID. This help the receiver figure out 1102 that a REMOVE_ADDRESS might have been sent before an ADD_ADDRESS 1103 frame implying the same Address ID, even if for some reason the 1104 REMOVE_ADDRESS reaches the receiver after the newer ADD_ADDRESS 1105 one. 1107 A host SHOULD stop using sending uniflows using the removed address 1108 and set them in the UNUSED state. If the REMOVE_ADDRESS contains an 1109 Address ID that was not previously announced, the receiver MUST 1110 silently ignore the frame. 1112 6.6. UNIFLOWS Frame 1114 The UNIFLOWS frame (type=0x46) communicates the uniflows' state of 1115 the sending host to the peer. It allows the sender to communicate 1116 its active uniflows to the peer in order to detect potential 1117 connectivity issue over uniflows. 1119 The format of the UNIFLOWS frame is shown below. 1121 0 1 2 3 1122 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1123 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1124 | Sequence (i) ... 1125 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1126 | ReceivingUniflows (i) ... 1127 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1128 | ActiveSendingUniflows (i) ... 1129 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1130 | Receiving Uniflow Info Section (*) ... 1131 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1132 | Sending Uniflow Info Section (*) ... 1133 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1135 Figure 12: UNIFLOWS Frame 1137 The UNIFLOWS frame contains the following fields. 1139 Sequence: A variable-length integer. This value starts at 0 and 1140 increases by 1 for each UNIFLOWS frame sent by the host. It 1141 allows identifying the most recent UNIFLOWS frame. 1143 ReceivingUniflows: The current number of receiving uniflows 1144 considered as being usable from the sender's point of view. 1146 ActiveSendingUniflows: The current number of sending uniflows in the 1147 ACTIVE state from the sender's point of view. 1149 Receiving Uniflow Info Section: Contains information about the 1150 receiving uniflows (there are ReceivingUniflows entries). 1152 Sending Uniflow Info Section: Contains information about the sending 1153 uniflows in ACTIVE state (there are ActiveSendingUniflows 1154 entries). 1156 Both Receiving Uniflow Info and Sending Uniflow Info Sections share 1157 the same format, which is shown below. 1159 0 1 2 3 1160 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1161 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1162 | Uniflow ID 0 (i) ... 1163 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1164 |LocAddrID 0 (8)| 1165 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1166 ... 1167 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1168 | Uniflow ID N (i) ... 1169 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1170 |LocAddrID N (8)| 1171 +-+-+-+-+-+-+-+-+ 1173 Figure 13: Uniflow Info Section 1175 The fields in the Uniflow Info Section are the following. 1177 Uniflow ID: The Uniflow ID of the uniflow the sending host provides 1178 information about. 1180 LocAddrID: The local Address ID of the address currently used by the 1181 uniflow. 1183 The Uniflow Info section only contains the local Address ID so far, 1184 but this section can be extended later with other potentially useful 1185 information. 1187 7. Extension of the Meaning of Existing QUIC Frames 1189 The multipath extensions do not modify the wire format of existing 1190 QUIC frames. However, they extend the meaning of a few of them while 1191 keeping this addition transparent and consistent with the single-path 1192 QUIC design. The concerned frames (and their extended meaning) are 1193 the following. 1195 NEW_CONNECTION_ID: Equivalent to a MP_NEW_CONNECTION_ID frame with 1196 Uniflow ID set to 0. 1198 RETIRE_CONNECTION_ID: Equivalent to a MP_RETIRE_CONNECTION_ID frame 1199 with Uniflow ID set to 0. 1201 ACK: Equivalent to a MP_ACK frame with Uniflow ID set to 0. 1203 8. Security Considerations 1205 8.1. Nonce Computation 1207 With Multipath QUIC, each uniflow has its own packet number space. 1208 With the current nonce computation [I-D.ietf-quic-tls], using twice 1209 the same packet number over two different uniflows on the same 1210 direction leads to the same cryptographic nonce. Using twice the 1211 same nonce MUST NOT happen, hence MP-QUIC has a different nonce 1212 computation than [I-D.ietf-quic-tls] 1214 The left most bits of nonce MUST be the Uniflow ID that identifies 1215 the current uniflow up to max_sending_uniflow_id. The remaining bits 1216 of the nonce is formed by an exclusive OR of the least significant 1217 bits of the packet protection IV with the padded packet number (left- 1218 padded with 0s). The nonce MUST be left-padded with a 0 if 1219 max_sending_uniflow_id <= 2, and the max_sending_uniflow_id MUST NOT 1220 be higher than 2^61. If a uniflow has sent 1221 2^62-max_sending_uniflow_id packets, another uniflow MUST be used to 1222 avoid re-using the same nonce. 1224 8.2. Validation of Exchanged Addresses 1226 To use addresses communicated by the peer through ADD_ADDRESS frames, 1227 hosts are required to validate them before using uniflows to these 1228 addresses as described in Section 8 of [I-D.ietf-quic-transport]. 1229 Section 21.12.3 of [I-D.ietf-quic-transport] provides additional 1230 motivation for this process. In addition, hosts MUST send ADD 1231 ADDRESS frames in 1-RTT frames to prevent off-path attacks. 1233 9. IANA Considerations 1235 9.1. QUIC Transport Parameter Registry 1237 This document defines a new transport parameter for the negotiation 1238 of multiple paths. The following entry in Table 1 should be added to 1239 the "QUIC Transport Parameters" registry under the "QUIC Protocol" 1240 heading. 1242 +-------+------------------------+---------------+ 1243 | Value | Parameter Name | Specification | 1244 +=======+========================+===============+ 1245 | 0x40 | max_sending_uniflow_id | Section 5.1.1 | 1246 +-------+------------------------+---------------+ 1248 Table 1: Addition to QUIC Transport Parameters 1249 Entries 1251 10. Acknowledgments 1253 We would like to thank Masahiro Kozuka and Kazuho Oku for their 1254 numerous comments on the first version of this draft. We also thank 1255 Philipp Tiesel for his early comments that led to the current design, 1256 and Ian Swett for later feedback. We also want to thank Christian 1257 Huitema for his draft about multipath requirements to identify 1258 critical elements about the multipath feature. Mohamed Boucadair 1259 provided lot of useful inputs on the second version of this document. 1260 Maxime Piraux and Florentin Rochet helped us to improve the last 1261 versions of this draft. This project was partially supported by the 1262 MQUIC project funded by the Walloon Government. 1264 11. References 1266 11.1. Normative References 1268 [I-D.ietf-quic-invariants] 1269 Thomson, M., "Version-Independent Properties of QUIC", 1270 Work in Progress, Internet-Draft, draft-ietf-quic- 1271 invariants-09, 9 June 2020, . 1274 [I-D.ietf-quic-recovery] 1275 Iyengar, J. and I. Swett, "QUIC Loss Detection and 1276 Congestion Control", Work in Progress, Internet-Draft, 1277 draft-ietf-quic-recovery-29, 9 June 2020, 1278 . 1281 [I-D.ietf-quic-tls] 1282 Thomson, M. and S. Turner, "Using TLS to Secure QUIC", 1283 Work in Progress, Internet-Draft, draft-ietf-quic-tls-29, 1284 9 June 2020, . 1287 [I-D.ietf-quic-transport] 1288 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 1289 and Secure Transport", Work in Progress, Internet-Draft, 1290 draft-ietf-quic-transport-29, 9 June 2020, 1291 . 1294 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1295 Requirement Levels", BCP 14, RFC 2119, 1296 DOI 10.17487/RFC2119, March 1997, 1297 . 1299 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1300 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1301 May 2017, . 1303 11.2. Informative References 1305 [I-D.huitema-quic-1wd] 1306 Huitema, C., "Quic ACK Timestamps For Measuring One-Way 1307 Delays", Work in Progress, Internet-Draft, draft-huitema- 1308 quic-1wd-00, 3 January 2020, . 1311 [I-D.huitema-quic-mpath-req] 1312 Huitema, C., "QUIC Multipath Requirements", Work in 1313 Progress, Internet-Draft, draft-huitema-quic-mpath-req-01, 1314 7 January 2018, . 1317 [IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments", 1318 IETF Journal , November 2016. 1320 [MPQUIC] De Coninck, Q. and O. Bonaventure, "Multipath QUIC: Design 1321 and Evaluation", 13th International Conference on emerging 1322 Networking EXperiments and Technologies (CoNEXT 2017). 1323 http://multipath-quic.org , December 2017. 1325 [MPRTP] Singh, V., Ahsan, S., and J. Ott, "MPRTP: Multipath 1326 considerations for real-time media", Proceedings of the 1327 4th ACM Multimedia Systems Conference , 2013. 1329 [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J.- 1330 Y. Le Boudec, "MPTCP is not pareto-optimal: performance 1331 issues and a possible solution", Proceedings of the 8th 1332 international conference on Emerging networking 1333 experiments and technologies, ACM , 2012. 1335 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1336 RFC 793, DOI 10.17487/RFC0793, September 1981, 1337 . 1339 [RFC3077] Duros, E., Dabbous, W., Izumiyama, H., Fujii, N., and Y. 1340 Zhang, "A Link-Layer Tunneling Mechanism for 1341 Unidirectional Links", RFC 3077, DOI 10.17487/RFC3077, 1342 March 2001, . 1344 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 1345 Congestion Control for Multipath Transport Protocols", 1346 RFC 6356, DOI 10.17487/RFC6356, October 2011, 1347 . 1349 [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, 1350 "TCP Extensions for Multipath Operation with Multiple 1351 Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, 1352 . 1354 [RFC7050] Savolainen, T., Korhonen, J., and D. Wing, "Discovery of 1355 the IPv6 Prefix Used for IPv6 Address Synthesis", 1356 RFC 7050, DOI 10.17487/RFC7050, November 2013, 1357 . 1359 [RFC7225] Boucadair, M., "Discovering NAT64 IPv6 Prefixes Using the 1360 Port Control Protocol (PCP)", RFC 7225, 1361 DOI 10.17487/RFC7225, May 2014, 1362 . 1364 [RFC8041] Bonaventure, O., Paasch, C., and G. Detal, "Use Cases and 1365 Operational Experience with Multipath TCP", RFC 8041, 1366 DOI 10.17487/RFC8041, January 2017, 1367 . 1369 [SCTPCMT] Iyengar, J., Amer, P., and R. Stewart, "Concurrent 1370 multipath transfer using SCTP multihoming over independent 1371 end-to-end paths", IEEE/ACM Transactions on networking, 1372 Vol. 14, no 5 , 2006. 1374 Appendix A. Comparison with Multipath TCP 1376 Multipath TCP [RFC6824] is currently the most widely deployed 1377 multipath transport protocol on the Internet. While its design 1378 impacted the initial versions of the Multipath extensions for the 1379 QUIC protocol, there are now major differences between both protocols 1380 that we now highlight. 1382 A.1. Multipath TCP Bidirectional Paths vs. QUIC Uniflows 1384 The notion of bidirectional paths, i.e., paths where packets flow in 1385 both directions, became a de facto standard with TCP. The Multipath 1386 TCP extension [RFC6824] combines several TCP connections to spread a 1387 single data stream over them. Hence, all the paths of a Multipath 1388 TCP connection must be bidirectional. However, networking 1389 experiences showed that packets following a direction do not always 1390 share the exact same road as the packets in the opposite direction. 1391 Furthermore, QUIC does not require a network path to be bidirectional 1392 in order to be used. 1394 A.2. Uniflow Establishment 1396 Unlike Multipath TCP [RFC6824], both hosts dynamically control how 1397 many sending uniflows can currently be in use by the peer. 1399 A.3. Exchanging Data over Multiple Uniflows 1401 The uniflow on which data is sent is a packet-level information. 1402 This means that a frame can be sent regardless of the uniflow of the 1403 packet carrying it. Furthermore, because the data offset is a frame- 1404 level information, there is no need to define additional sequence 1405 numbers to cope with reordering across uniflows, unlike Multipath TCP 1406 [RFC6824] that uses a Data Sequence Number at the Multipath TCP 1407 level. 1409 Decoupling the network paths of data with their acknowledgment can be 1410 useful to limit the latency due to (MP_)ACK transmissions on high- 1411 latency network paths and to enable the usage of unidirectional 1412 networks. Such scheduling decision would not have been possible in 1413 Multipath TCP [RFC6824] which must acknowledge data on the 1414 (bidirectional) path it was received on. 1416 A.4. Congestion Control 1418 Multipath TCP uses the LIA congestion control scheme specified in 1419 [RFC6356]. This scheme can immediately be adapted to Multipath QUIC. 1420 Other coupled congestion control schemes have been proposed for 1421 Multipath TCP such as [OLIA]. 1423 A.5. ACK Frame 1425 Since frames are independent of packets, and the uniflow notion 1426 relates to the packets, the (MP_)ACK frames can be sent on any 1427 uniflow, unlike Multipath TCP [RFC6824] which is constrained to send 1428 ACKs on the same path. 1430 Appendix B. To move in companion drafts 1432 B.1. Uniflow Establishment 1434 Sending useful data on a fresh new sending uniflow might lead to poor 1435 performance as the network path used by the QUIC uniflow might not be 1436 usable. A typical case is when a server wants to initiate a new 1437 sending uniflow to a client behind a NAT. The client would possibly 1438 never receive this packet, leading to connectivity issues on that 1439 uniflow. In addition, an opportunistic usage of network paths might 1440 also lead to possible attacks, such as a client advertising the IP 1441 address of a victim hoping that the server will flood it. To avoid 1442 these issues, a remote address MUST have been validated as described 1443 in [I-D.ietf-quic-transport] before associating it on a sending 1444 uniflows. 1446 Because attaching to new networks may be volatile and an endpoint 1447 does not have full visibility on multiple paths that may be available 1448 (e.g., hosts connected to a CPE), a Multipath QUIC capable endhost 1449 SHOULD advertise a "max_sending_uniflow_id" value of at least 4 and 1450 SHOULD propose at least 4 receiving uniflows to its peer. 1452 B.2. Exchanging Addresses 1454 Likewise, the client may be located behind a NAT64. As such it may 1455 announce an IPv6 address in an ADD_ADDRESS frame, that will be 1456 received over IPv4 by an IPv4-only server. The server should not 1457 discard that address, even if it is not IPv6-capable. 1459 An IPv6-only client may also receive from the server an ADD_ADDRESS 1460 frame which may contain an IPv4 address. The client should rely on 1461 means, such as [RFC7050] or [RFC7225], to learn the IPv6 prefix to 1462 build an IPv4-converted IPv6 address. 1464 Hosts that are connected behind an address sharing mechanism may 1465 collect the external IP address and port numbers assigned to the 1466 hosts and then use their addresses in the ADD_ADDRESS. Means to 1467 gather such information include, but not limited to, UPnP IGD, PCP, 1468 or STUN. 1470 B.3. Uniflow Migration 1472 At a given time, a Multipath QUIC endpoint gathers a set of active 1473 sending and receiving uniflows, each associated to a 4-tuple. To 1474 address privacy issues due to the linkability of addresses with 1475 Connection IDs, hosts should avoid changing the 4-tuple used by a 1476 sending uniflow. There still remain situations where this change is 1477 unavoidable. These can be categorized into two groups: host-aware 1478 changes (e.g., network handover from Wi-Fi to cellular) and host- 1479 unaware changes (e.g., NAT rebinding). 1481 For the host-aware case, let us consider the case of a Multipath QUIC 1482 connection where the client is a smartphone with both Wi-Fi and 1483 cellular. It advertised both addresses and the server currently 1484 enables only one client's sending uniflow, the initial one. The 1485 Initial Uniflow uses the Wi-Fi address. Then, for some reason, the 1486 Wi-Fi address becomes unusable. To preserve connectivity, the client 1487 might then decide to use the cellular address for its Initial sending 1488 uniflow. It thus sends a REMOVE_ADDRESS announcing the loss of the 1489 Wi-Fi address and an UNIFLOWS frame to inform that its Initial 1490 sending uniflow is now using the cellular address. If the cellular 1491 address validation succeeds (which could have been done as soon as 1492 the cellular address was advertised), the server can continue 1493 exchanging data through the cellular address. 1495 TODO: I don't think we have to change the Uniflow ID, we can just 1496 rely on (MP_)NEW_CONNECTION_ID and (MP_)RETIRE_CONNECTION_ID - hence, 1497 not sure the PATH_UPDATE frame is required, but will depend on the 1498 use cases 1500 TODO: update following paragraph to remove PATH_UPDATE 1502 However, both server and client might want to change the uniflow used 1503 on the cellular address for privacy concerns. If the server provides 1504 an additional uniflow (e.g., with Uniflow ID 1) through 1505 MP_NEW_CONNECTION_ID frame at the beginning of the connection, the 1506 client can perform the network path change directly and avoid using 1507 the Initial Uniflow Connection ID on the cellular network. This can 1508 be done using the PATH_UPDATE frame. It can indicate that the host 1509 stopped to use the Initial sending uniflow to use the one with 1510 Uniflow ID 1 instead. This frame is placed in the first packet sent 1511 to the new sending uniflow with its corresponding UCID. The client 1512 can then send the REMOVE_ADDRESS and UNIFLOWS frames on this new 1513 uniflow. Compared to the previous case, it is harder to link the 1514 uniflows with the IP addresses to observe that they belong to the 1515 same Multipath QUIC connection. 1517 For the host-unaware case, the situation is similar. In case of NAT 1518 rebinding, the server will observe a change in the 2-tuple (source 1519 IP, source port) of the receiving uniflow of the packet. The server 1520 first validates that the 2-tuple actually belongs to the client 1521 [I-D.ietf-quic-transport]. If it is the case, the server can send a 1522 PATH_UPDATE frame on a previously communicated but unused Uniflow ID. 1523 The client might have sent some packets with a given UCID on a 1524 different 4-tuple, but the server did not use the given UCID on that 1525 4-tuple. Because some on-path devices may rewrite the source IP 1526 address to forward packets via the available network attachments 1527 (e.g., a host located behind a multi-homed CPE), the server may 1528 inadvertently conclude that an uniflow is not anymore valid leading 1529 thus to frequently sending PATH_UPDATE frames as a function of the 1530 traffic distribution scheme enforced by the on-path device. To 1531 prevent such behavior, the server SHOULD wait for at least X seconds 1532 to ensure this is about a connection migration and not a side effect 1533 of an on-path multi-interfaced device. 1535 B.4. Scheduling Strategies 1537 The current QUIC design [I-D.ietf-quic-transport] offers a single 1538 scheduling space, i.e., which frames will be packed inside a given 1539 packet. With the simultaneous use of several sending uniflows, a 1540 second dimension is added, i.e., the sending uniflow on which the 1541 packet will be sent. This dimension can have a non negligible impact 1542 on the operations of Multipath QUIC, especially if the available 1543 sending uniflows exhibit very different network characteristics. 1545 The progression of the data flow depends on the reception of the 1546 MAX_DATA and MAX_STREAM_DATA frames. Those frames SHOULD be 1547 duplicated on several or all ACTIVE sending uniflows. This helps to 1548 limit the head-of-line blocking issue due to the transmission of the 1549 frames over a slow or lossy network path. 1551 The sending path on which (MP_)ACK frames are sent impacts the peer. 1552 The (MP_)ACK frame is notably used to determine the latency of a 1553 combination of uniflows. In particular, the peer can compute the 1554 round-trip-time of the combination of its sending uniflow with its 1555 receive one. The peer would compute the latency as the sum of the 1556 forward delay of the acknowledged uniflow and the return delay of the 1557 uniflow used to send the (MP_)ACK frame. Choosing between 1558 acknowledging packets symmetrically (on uniflow B to A if packet was 1559 sent on A to B) or not is up to the implementation, if only possible. 1560 However, hosts SHOULD keep a consistent acknowledgment strategy. 1561 Selecting a random uniflow to acknowledge packets may affect the 1562 performance of the connection. Notice that the inclusion of a 1563 timestamp field in the (MP_)ACK frame, as proposed by 1564 [I-D.huitema-quic-1wd], may help hosts estimate more precisely the 1565 one-way delay of each uniflow, therefore leading to improved 1566 scheduling decisions. Unlike MAX_DATA and MAX_STREAM_DATA, (MP_)ACK 1567 frames SHOULD NOT be systematically duplicated on several sending 1568 uniflows as they can induce a large network overhead. 1570 Appendix C. Change Log 1572 C.1. Since draft-deconinck-quic-multipath-04 1573 * Mostly editorial and reference fixes 1575 C.2. Since draft-deconinck-quic-multipath-03 1577 * Clarify the notion of asymmetric paths by introducing uniflows 1579 * Remove the PATH_UPDATE frame 1581 * Rename PATHS frame to UNIFLOWS frame and adapt its content 1583 * Add a sequence number to frames involving Address ID events (#4) 1585 * Disallow Zero-length connection ID (#2) 1587 * Correctly handle nonce computation (thanks to Florentin Rochet) 1589 * Focus on the core concepts of multipath and delegate algorithms to 1590 companion drafts 1592 * Updated text to match draft-ietf-quic-transport-27 1594 C.3. Since draft-deconinck-quic-multipath-02 1596 * Consider asymmetric paths 1598 C.4. Since draft-deconinck-quic-multipath-01 1600 * Include path policies considerations 1602 * Add practical considerations thanks to Mohamed Boucadair inputs 1604 * Adapt the RETIRE_CONNECTION_ID frame 1606 * Updated text to match draft-ietf-quic-transport-18 1608 C.5. Since draft-deconinck-quic-multipath-00 1610 * Comply with asymmetric Connection IDs 1612 * Add max_paths transport parameter to negotiate initial number of 1613 active paths 1615 * Path ID as a regular varint 1617 * Remove max_path_id transport parameter 1619 * Updated text to match draft-ietf-quic-transport-14 1621 C.6. Since draft-deconinck-multipath-quic-00 1623 * Added PATH_UPDATE frame 1625 * Added MAX_PATHS frame 1627 * No more packet header change 1629 * Implicit Path ID notification using Connection ID and 1630 NEW_CONNECTION_ID frames 1632 * Variable-length encoding for Path ID 1634 * Updated text to match draft-ietf-quic-transport-10 1636 * Fixed various typos 1638 Authors' Addresses 1640 Quentin De Coninck 1641 UCLouvain 1643 Email: quentin.deconinck@uclouvain.be 1645 Olivier Bonaventure 1646 UCLouvain 1648 Email: olivier.bonaventure@uclouvain.be