idnits 2.17.1 draft-deconinck-quic-multipath-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (3 May 2021) is 1088 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-02) exists of draft-bonaventure-iccrg-schedulers-01 -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group Q. De Coninck 3 Internet-Draft O. Bonaventure 4 Intended status: Standards Track UCLouvain 5 Expires: 4 November 2021 3 May 2021 7 Multipath Extensions for QUIC (MP-QUIC) 8 draft-deconinck-quic-multipath-07 10 Abstract 12 This document specifies extensions to the QUIC protocol to enable the 13 simultaneous usage of multiple paths for a single connection. These 14 extensions are compliant with the single-path QUIC design and 15 preserve QUIC privacy features. 17 Discussion about this draft is encouraged either on the QUIC IETF 18 mailing list quic@ietf.org or on the GitHub repository which contains 19 the draft: https://github.com/qdeconinck/draft-deconinck-multipath- 20 quic. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at https://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on 4 November 2021. 39 Copyright Notice 41 Copyright (c) 2021 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 46 license-info) in effect on the date of publication of this document. 47 Please review these documents carefully, as they describe your rights 48 and restrictions with respect to this document. Code Components 49 extracted from this document must include Simplified BSD License text 50 as described in Section 4.e of the Trust Legal Provisions and are 51 provided without warranty as described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 56 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 4 57 2.1. Notation Conventions . . . . . . . . . . . . . . . . . . 5 58 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 3.1. Moving from Bidirectional Paths to Uniflows . . . . . . . 5 60 3.2. Beyond Connection Migration . . . . . . . . . . . . . . . 7 61 3.3. Establishment of a Multipath QUIC Connection . . . . . . 8 62 3.4. Architecture of Multipath QUIC . . . . . . . . . . . . . 9 63 3.5. Uniflow Establishment . . . . . . . . . . . . . . . . . . 11 64 3.6. Exchanging Data over Multiple Uniflows . . . . . . . . . 12 65 3.7. Exchanging Addresses . . . . . . . . . . . . . . . . . . 14 66 3.8. Coping with Address Removals . . . . . . . . . . . . . . 15 67 3.9. Uniflow Migration . . . . . . . . . . . . . . . . . . . . 15 68 3.10. Handling Multiple Network Paths . . . . . . . . . . . . . 15 69 3.11. Congestion Control . . . . . . . . . . . . . . . . . . . 16 70 4. Mapping Uniflow IDs to Connection IDs . . . . . . . . . . . . 16 71 5. Using Multiple Uniflows . . . . . . . . . . . . . . . . . . . 16 72 5.1. Multipath Negotiation . . . . . . . . . . . . . . . . . . 17 73 5.1.1. Transport Parameter Definition . . . . . . . . . . . 17 74 5.2. Coping with Additional Remote Addresses . . . . . . . . . 17 75 5.3. Receiving Uniflow State . . . . . . . . . . . . . . . . . 18 76 5.4. Sending Uniflow State . . . . . . . . . . . . . . . . . . 19 77 5.5. Losing Addresses . . . . . . . . . . . . . . . . . . . . 20 78 6. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 21 79 6.1. MP_NEW_CONNECTION_ID Frames . . . . . . . . . . . . . . . 22 80 6.2. MP_RETIRE_CONNECTION_ID Frame . . . . . . . . . . . . . . 22 81 6.3. MP_ACK Frame . . . . . . . . . . . . . . . . . . . . . . 23 82 6.4. ADD_ADDRESS Frame . . . . . . . . . . . . . . . . . . . . 24 83 6.5. REMOVE_ADDRESS Frame . . . . . . . . . . . . . . . . . . 25 84 6.6. UNIFLOWS Frame . . . . . . . . . . . . . . . . . . . . . 26 85 7. Extension of the Meaning of Existing QUIC Frames . . . . . . 27 86 8. Security Considerations . . . . . . . . . . . . . . . . . . . 27 87 8.1. Nonce Computation . . . . . . . . . . . . . . . . . . . . 27 88 8.2. Validation of Exchanged Addresses . . . . . . . . . . . . 28 89 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 28 90 9.1. QUIC Transport Parameter Registry . . . . . . . . . . . . 28 91 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 29 92 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 29 93 11.1. Normative References . . . . . . . . . . . . . . . . . . 29 94 11.2. Informative References . . . . . . . . . . . . . . . . . 30 95 Appendix A. Comparison with Multipath TCP . . . . . . . . . . . 31 96 A.1. Multipath TCP Bidirectional Paths vs. QUIC Uniflows . . . 31 97 A.2. Negotiating the Multipath Extensions . . . . . . . . . . 32 98 A.3. Uniflow Establishment . . . . . . . . . . . . . . . . . . 32 99 A.4. Exchanging Data over Multiple Uniflows . . . . . . . . . 33 100 A.5. Advertising Host's Addresses . . . . . . . . . . . . . . 33 101 A.6. Congestion Control . . . . . . . . . . . . . . . . . . . 34 102 Appendix B. Change Log . . . . . . . . . . . . . . . . . . . . . 34 103 B.1. Since draft-deconinck-quic-multipath-06 . . . . . . . . . 34 104 B.2. Since draft-deconinck-quic-multipath-05 . . . . . . . . . 34 105 B.3. Since draft-deconinck-quic-multipath-04 . . . . . . . . . 34 106 B.4. Since draft-deconinck-quic-multipath-03 . . . . . . . . . 34 107 B.5. Since draft-deconinck-quic-multipath-02 . . . . . . . . . 35 108 B.6. Since draft-deconinck-quic-multipath-01 . . . . . . . . . 35 109 B.7. Since draft-deconinck-quic-multipath-00 . . . . . . . . . 35 110 B.8. Since draft-deconinck-multipath-quic-00 . . . . . . . . . 35 111 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 113 1. Introduction 115 Today's endhosts are equipped with several network interfaces. Users 116 expect to be able to seamlessly switch from one interface to another 117 one or use them simultaneously to aggregate bandwidth whenever 118 needed. During the last years, several multipath extensions to 119 transport protocols have been proposed (e.g., [RFC8684], [MPRTP], or 120 [SCTPCMT]). Multipath TCP [RFC8684] is the most mature one. It is 121 already deployed on popular smartphones, but also for other use cases 122 [RFC8041] [IETFJ]. 124 With regular TCP and UDP, all the packets that belong to a given flow 125 share the same 4-tuple {source IP address, source port number, 126 destination IP address, destination port number} that acts as an 127 identifier for this flow. This prevents these flows from using 128 multiple paths. 130 QUIC [I-D.ietf-quic-transport] does not use the 4-tuple as an 131 implicit connection identifier. Instead, a QUIC flow is identified 132 by a Connection ID. This enables QUIC to cope with events affecting 133 the 4-tuple, such as NAT rebinding or IP address changes. The QUIC 134 connection migration feature, specified in Section 9 of 135 [I-D.ietf-quic-transport], enables migrating a flow from one 4-tuple 136 to another one, sustaining in particular a connection over different 137 network paths. 139 Still, there is a void to specify simultaneous usage of QUIC over 140 available network paths for a single connection. Use cases such as 141 bandwidth aggregation or seamless network handovers would be 142 applicable to QUIC, as they are now with Multipath TCP 143 [RFC8041][IETFJ]. Experience with Multipath TCP on smartphones shows 144 that the ability to simultaneously use WLAN and cellular during 145 handovers improves the user perceived quality of experience. A 146 performance evaluation of an early solution for such use cases and a 147 comparison between Multipath QUIC and Multipath TCP may be found in 148 [MPQUIC]. A recent publication [MFQUIC] shows how the design 149 presented in this draft enables implementations to take advantage of 150 highly asymmetrical network paths such as satellite links. 152 In this document, we leverage many of the lessons learned from the 153 design of Multipath TCP and the comments received on the first 154 versions of this document to propose extensions to the current QUIC 155 design to enable it to simultaneously use several network paths. 156 This document focuses mainly on network paths that are 157 distinguishable by the endpoints. 159 This document is organized as follows. It first provides in 160 Section 3 an overview of the operation of Multipath QUIC. It then 161 states in Section 4 how Connection IDs can map to different 162 unidirectional flows (called uniflows) in use. Section 5 specifies 163 the required changes in the current QUIC design 164 [I-D.ietf-quic-transport] to enable the simultaneous usage of 165 multiple network paths. These extensions introduce new frames that 166 are described in Section 6 and Section 7. Finally, Section 8 167 discusses some security considerations. 169 2. Conventions and Definitions 171 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 172 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 173 "OPTIONAL" in this document are to be interpreted as described in BCP 174 14 [RFC2119] [RFC8174] when, and only when, they appear in all 175 capitals, as shown here. 177 We assume that the reader is familiar with the terminology used in 178 [I-D.ietf-quic-transport]. In addition, we define the following 179 terms: 181 * Uniflow: A unidirectional flow of packets between a QUIC host and 182 its peer. This flow is identified by an internal identifier, 183 called Uniflow ID. Packets sent over a uniflow use a Destination 184 Connection ID that may change during the lifetime of the 185 connection. When being in use, an uniflow is temporarily bound to 186 a 4-tuple (Source IP Address, Source Port Number, Destination IP 187 Address, Destination Port Number). 189 * Initial Uniflows: The two uniflows used by peers for the 190 establishment of a QUIC connection. One is the uniflow from the 191 client to the server and the other is the uniflow from the server 192 to the client. The cryptographic handshake is done on these 193 uniflows. These are identified by Uniflow ID 0. 195 2.1. Notation Conventions 197 Packet and frame diagrams use the format described in Section 12 of 198 [I-D.ietf-quic-transport]. 200 3. Overview 202 The design of QUIC [I-D.ietf-quic-transport] provides reliable 203 transport with multiplexing, confidentiality, integrity, and 204 authenticity of data flows. A wide range of devices on today's 205 Internet are multihomed. Examples include smartphones equipped with 206 both WLAN and cellular interfaces, but also regular dual-stack hosts 207 that use both IPv4 and IPv6. 209 The current design of QUIC does not enable multihomed devices to 210 efficiently use different paths simultaneously. This document 211 proposes multipath extensions with the following design goals: 213 * Each host keeps control on the number of uniflows being used over 214 the connection. 216 * The simultaneous usage of multiple uniflows should not introduce 217 new privacy concerns. 219 * A host must ensure that all the paths it uses actually reach its 220 peer to avoid packet flooding towards a victim (see 221 Section 21.12.3 of [I-D.ietf-quic-transport]) 223 * The multipath extensions should handle the asymmetrical nature of 224 paths between two peers. 226 We first explain why a multipath extension would be beneficial to 227 QUIC and then describe it at a high level. 229 3.1. Moving from Bidirectional Paths to Uniflows 231 To understand the overall architecture of the multipath extensions, 232 let us first refine the notion of "path". A path may be denoted by a 233 4-tuple (Source IP Address, Source Port Number, Destination IP 234 Address, Destination Port Number). In QUIC, this is namely a UDP 235 path from the local host to the remote one. Considering a smartphone 236 interacting with a single-homed server, the smartphone might want to 237 use one path over the WLAN network and another over the cellular one. 238 Those paths are not necessarily disjoint. For example, when 239 interacting with a dual-stack server, a smartphone may create two 240 paths over WLAN: one over IPv4 and the other one over IPv6. 242 A regular QUIC connection is composed of two independent active 243 packet flows. The first flow gathers the packets from the client to 244 the server and the other the packets from the server to the client. 245 To illustrate this, let us consider the example depicted in Figure 1. 246 In this example, the client has two IP addresses: IPc1 and IPc2. The 247 server has one single address: IPs1. 249 Probed flow IPc2 to IPs1 250 +---------------------------------------------------------+ 251 | | 252 | From IPc1 to IPs1 v 253 | +--------+ Client to Server Flow +--------+ 254 | | | =====================================> | | 255 +-- | Client | | Server | 256 | | <===================================== | | 257 +--------+ Server to Client Flow +--------+ 258 From IPc1* to IPs1* 260 Figure 1: Identifying Unidirectional Flows in QUIC 262 The client initiates the QUIC connection by sending packets towards 263 the server. The server then replies to the client if it accepts the 264 connection. If the handshake succeeds, the connection is 265 established. Still, this "path" actually consists in two independent 266 UDP flows. Each host has its own view of (i) the 4-tuple used to 267 send packets and (ii) the 4-tuple on which it receives packets. 268 While the 4-tuple used by the client to send packets may be the same 269 as the one seen and used by the server, this is not always the case 270 since middleboxes (e.g., NATs) may alter the 4-tuple of packets. 272 To further emphasize on this flow asymmetry, QUIC embeds a path 273 validation mechanism [I-D.ietf-quic-transport] assessing whether a 274 host can reach its peer through a given 4-tuple. This process is 275 unidirectional, i.e., the sender checks that it can reach the 276 receiver, but not the reverse. A host receiving a PATH_CHALLENGE 277 frame on a new 4-tuple may in turn initiate a path validation, but 278 this is up to the peer. 280 A QUIC connection is a collection of unidirectional flows (called, 281 uniflows). A plain QUIC connection is composed of a main uniflow 282 from client to server and another main uniflow from server to client. 283 These uniflows have their own Connection IDs. They are host- 284 specific, i.e., the uniflow(s) from A to B are different from the 285 ones from B to A. This potentially enables the use of unidirectional 286 links such as non-broadcast satellite links [RFC3077], which cannot 287 be used with TCP. 289 3.2. Beyond Connection Migration 291 Unlike TCP [RFC0793], QUIC is not bound to a particular 4-tuple 292 during the lifetime of a connection. A QUIC connection is identified 293 by a (set of) Connection ID(s), placed in the public header of each 294 QUIC packet. This enables hosts to continue the connection even if 295 the 4-tuple changes due to, e.g., NAT rebinding. This ability to 296 shift a connection from one 4-tuple to another is called: Connection 297 Migration. One of its use cases is fail-over when the IP address in 298 use fails but another one is available. A smartphone losing the WLAN 299 connectivity can then continue the connection over its cellular 300 interface, for instance. 302 A QUIC connection can thus start on a given set of uniflows, denoted 303 as the initial uniflows, and end on another ones. However, the 304 current QUIC design [I-D.ietf-quic-transport] assumes that only one 305 pair of uniflows is in use for a given connection. The current 306 transport specification [I-D.ietf-quic-transport] does not provide 307 means to distinguish path migration from simultaneous usage of 308 available uniflows for a given connection. 310 This document fills that void. It first proposes mechanisms to 311 communicate endhost addresses to the peer. It then leverages the 312 Address Validation procedure with PATH_CHALLENGE and PATH_RESPONSE 313 frames described in Section 8 of [I-D.ietf-quic-transport] to verify 314 whether the additional addresses advertised by the host are 315 reachable. In this case, those addresses can be used to initiate new 316 uniflows to spread packets over several network paths following a 317 packet scheduling policy that is out of scope of this document. 319 The example of Figure 2 illustrates a data exchange between a dual- 320 homed client sending a request spanning two packets and a single- 321 homed server. Uniflow IDs are independently chosen by each host. In 322 the presented example, the client sends packets over WLAN on Uniflow 323 0 and over LTE on Uniflow 1, while the packets sent by the server 324 over WLAN are on Uniflow 2 and those over LTE are on Uniflow 1. 326 Server Phone Server 327 via WLAN via LTE 328 ------- ------- ----- 329 | Pkt(DCID=A,PN=5,frames=[ | | 330 | STREAM("Request (1/2)")]) | Pkt(DCID=B,PN=1,frames=[ | 331 |<----------------------------| STREAM("Request (2/2)")]) | 332 | Pkt(DCID=E,PN=1,frames=[ |-------- | 333 | ACK(LargestAcked=5)]) | |---------- | 334 |---------------------------->| |-------- | 335 | Pkt(DCID=E,PN=2,frames=[ | |-->| 336 | STREAM("Response 1")]) | Pkt(DCID=D,PN=1,frames=[ | 337 |---------------------------->| MPACK(UID=1,LargestAck=1), | 338 | | STREAM("Response 2")]) ---| 339 | Pkt(DCID=A,PN=6,frames=[ | ---------| | 340 | MPACK(UID=2,LargestAck=2), | ----------| | 341 | MPACK(UID=1,LargestAck=1)])|<------| | 342 |<----------------------------| | 343 | Pkt(DCID=E,PN=3,frames=[ | Pkt(DCID=D,PN=2,frames=[ | 344 | STREAM("Response 3")]) | STREAM("Response 4")]) | 345 |---------------------------->| ----| 346 | | ------| | 347 | ... | ... <---------| | 349 Figure 2: Data flow with Multipath QUIC 351 The remaining of this section presents a high-level overview of the 352 multipath operations in QUIC. 354 3.3. Establishment of a Multipath QUIC Connection 356 A Multipath QUIC connection starts as a regular QUIC connection 357 [I-D.ietf-quic-transport] [I-D.ietf-quic-tls]. The multipath 358 extensions defined in this document are negotiated using the 359 "max_sending_uniflow_id" transport parameter. Any value for this 360 transport parameter advertises the support of the multipath 361 extensions. 363 When negotiating the multipath extensions, the host puts a upper 364 bound on the number of sending uniflows that it will use over the 365 connection. For instance, if a host wants to spread connection's 366 packets on at most two network paths, it advertises a 367 "max_sending_uniflow_id" of 1. Note that the creation of a second 368 sending uniflow will depend on the peer, as described later. 370 Notice that a host advertising a value of 0 for the 371 "max_sending_uniflow_id" transport parameter indicates that it does 372 not want additional uniflows to send packets, but it still supports 373 the multipath extensions. Such situation might be useful when the 374 host does not require multiple uniflows for packet sending but still 375 wants to let the peer use multiple uniflows to reach it. 377 3.4. Architecture of Multipath QUIC 379 To illustrate the architecture of a Multipath QUIC connection, 380 consider Figure 3. 382 +--------+ CID A - Uniflow ID 1 +--------+ 383 | | =====================================> | | 384 | | CID B - Uniflow ID 0 | | 385 | | =====================================> | | 386 | | | | 387 | Client | CID C - Uniflow ID 0 | Server | 388 | | <===================================== | | 389 | | CID D - Uniflow ID 1 | | 390 | | <===================================== | | 391 | | CID E - Uniflow ID 2 | | 392 | | <===================================== | | 393 +--------+ +--------+ 395 Figure 3: An Example of Uniflow Distribution over a Multipath 396 QUIC Connection 398 Once established, a Multipath QUIC connection consists in one or more 399 uniflows from the client to the server and one or more uniflows from 400 the server to the client. The number of uniflows in one direction 401 can be different from the one in the other direction. The example in 402 Figure 3 shows two uniflows from the client to the server and three 403 uniflows from the server to the client. From the end-hosts' 404 viewpoint, they observe two kinds of uniflows: 406 * Sending uniflows: uniflows over which the host can send packets 408 * Receiving uniflows: uniflows over which the host can receive 409 packets 411 Reconsidering the example in Figure 3, the client has two sending 412 uniflows and three receiving uniflows. The server has three sending 413 uniflows and two receiving uniflows. There is thus a one-to-one 414 mapping between the sending uniflows of a host and the receiving 415 uniflows of its peer. A uniflow is seen as a sending uniflow from 416 the sender's perspective and as a receiving uniflow from the 417 receiver's viewpoint. 419 Each uniflow is associated with a specific four-tuple and identified 420 by a Uniflow ID, as shown in Figure 4. 422 Client state 423 +-----------------------------------------------------+ 424 | Connection | 425 | +-----------+ +-----------+ ... +-------------+ | 426 | | Sending | | Sending | ... | Sending | | 427 | | Uniflow 0 | | Uniflow 1 | | Uniflow N-1 | | 428 | | UCID set A| | UCID set B| ... | UCID set C | | 429 | | Tuple A | | Tuple B | ... | Tuple C | | 430 | | PNS A | | PNS B | | PNS C | | 431 | +-----------+ +-----------+ ... +-------------+ | 432 | +-----------+ +-----------+ ... +-------------+ | 433 | | Receiving | | Receiving | ... | Receiving | | 434 | | Uniflow 0 | | Uniflow 1 | | Uniflow M-1 | | 435 | | UCID set X| | UCID set Y| ... | UCID set Z | | 436 | | Tuple X | | Tuple Y | ... | Tuple Z | | 437 | | PNS X | | PNS Y | | PNS Z | | 438 | +-----------+ +-----------+ ... +-------------+ | 439 +-----------------------------------------------------+ 441 Server state 442 +-----------------------------------------------------+ 443 | Connection | 444 | +-----------+ +-----------+ ... +-------------+ | 445 | | Sending | | Sending | ... | Sending | | 446 | | Uniflow 0 | | Uniflow 1 | | Uniflow M-1 | | 447 | | UCID set X| | UCID set Y| ... | UCID set Z | | 448 | | Tuple X* | | Tuple Y* | ... | Tuple Z* | | 449 | | PNS X | | PNS Y | | PNS Z | | 450 | +-----------+ +-----------+ ... +-------------+ | 451 | +-----------+ +-----------+ ... +-------------+ | 452 | | Receiving | | Receiving | ... | Receiving | | 453 | | Uniflow 0 | | Uniflow 1 | | Uniflow N-1 | | 454 | | UCID set A| | UCID set B| ... | UCID set C | | 455 | | Tuple A* | | Tuple B* | ... | Tuple C* | | 456 | | PNS A | | PNS B | | PNS C | | 457 | +-----------+ +-----------+ ... +-------------+ | 458 +-----------------------------------------------------+ 460 Figure 4: Architectural view of Multipath QUIC for a host having 461 N sending uniflows and M receiving uniflows 463 A Multipath QUIC connection starts using two Initial Uniflows, 464 identified by Uniflow ID 0 on each peer. The packets can then be 465 spread over several uniflows. Each uniflow has its (set of) Uniflow 466 Connection ID(s) (UCID) packets that are used to explicitly mark 467 where they belong to. Depending on the direction of the uniflow, the 468 host keeps either the Uniflow Source Connection ID (USCID, for the 469 receiving uniflows) or the Uniflow Destination Connection ID (USCID, 470 for the sending uniflows). Notice that the (set of) UDCID(s) of a 471 sending uniflow of a host is the same as the (set of) USCID(s) of the 472 corresponding receive uniflow of the remote peer. 474 Preventing the linkability of different uniflows is an important 475 requirement for the multipath extensions 476 [I-D.huitema-quic-mpath-req]. We address it by using UCIDs as 477 implicit uniflow identifiers. This makes the linkability harder than 478 having explicit signaling as in earlier version of this draft. 479 Furthermore, it does not require any public header change and thus 480 preserves the QUIC invariants [I-D.ietf-quic-invariants]. 482 When a uniflow is in use, each endhost associates it with a network 483 path. In practice, this consists in a particular 4-tuple over which 484 packets are sent (or received) on a sending (or receiving) uniflow. 485 Each endhost has a specific vision of the 4-tuple, which might differ 486 between endhosts. For instance, a client located behind a NAT sends 487 data from a private IP address and the server will receive packets 488 coming from the NAT's public IP address. Notice that while uniflows 489 may share a common network path, this is not mandatory. 491 Each uniflow is an independent flow of packets over a given network 492 path. Uniflows can experience very different network conditions 493 (latency, bandwidth, ...). To handle this, each uniflow has its own 494 packet sequence number space. 496 In addition to the UCIDs, 4-tuple and packet number space, some 497 additional information is maintained for each uniflow. The Uniflow 498 ID identifies the uniflow at the frame level and ensures uniqueness 499 of the nonce (see Section 8.1 for details) while limiting the number 500 of concurrently used uniflows. 502 3.5. Uniflow Establishment 504 The "max_sending_uniflow_id" transport parameter exchanged during the 505 cryptographic handshake fixes an upper bound on the number of sending 506 uniflows a host wants to support. Then, hosts provide to their peer 507 Uniflow Connection IDs to use on uniflows. Both hosts dynamically 508 control how many sending uniflows can currently be in use by the 509 peer, i.e., the number of different Uniflow IDs proposed to the peer. 510 While the sender determines the upper bound of sending paths it can 511 have, it is the receiver that initializes uniflows, as the sender 512 needs a UCID communicated by the receiver before using a uniflow. 514 Notice that the peers might advertise different values for the 515 "max_sending_uniflow_id" transport parameters, setting different 516 upper bounds to the sending and receiving uniflows of each host. 518 Hosts initiate the creation of their receiving uniflows by sending 519 MP_NEW_CONNECTION_ID frames (see Section 6.1) which are an extended 520 version of the NEW_CONNECTION_ID frame. This frame associates a UCID 521 to a uniflow. Upon reception of the MP_NEW_CONNECTION_ID frame, a 522 host can start using the proposed sending uniflow having the 523 referenced Uniflow ID by marking sent packets with the provided UCID. 524 Therefore, once a host sends a MP_NEW_CONNECTION_ID frame, it 525 announces that it is ready to receive packets from that Uniflow ID 526 with the proposed UCID. As frames are encrypted, adding new uniflows 527 over a QUIC connection does not leak cleartext identifiers 528 [I-D.huitema-quic-mpath-req]. 530 A server might provide several Uniflow Connection IDs for the same 531 Uniflow ID with multiple MP_NEW_CONNECTION_ID frames. This can be 532 useful to cope with migration cases, as described in Section 3.9. 533 Multipath QUIC is thus asymmetrical. 535 3.6. Exchanging Data over Multiple Uniflows 537 A QUIC packet acts as a container for one or more frames. Multipath 538 QUIC uses the same STREAM frames as QUIC to carry data. A byte 539 offset is associated with the data payload. One of the key design of 540 (Multipath) QUIC is that frames are independent of the packets 541 carrying them. This implies that a frame transmitted over one 542 uniflow could be retransmitted later on another uniflow without any 543 change. Furthermore, all current QUIC frames are idempotent and 544 could be optimistically duplicated over several uniflows. 546 The uniflow on which data is sent is a packet-level information. 547 This means that a frame can be sent regardless of the uniflow of the 548 packet carrying it. Other flow control considerations like the 549 stream receive window advertised by the MAX_STREAM_DATA frame remain 550 unchanged when there are multiple sending uniflows. 552 As previously described, Multipath QUIC might face reordering at 553 packet-level when using uniflows having different latencies. The 554 presence of different Uniflow Connection IDs ensures that the packets 555 sent over a given uniflow will contain monotonically increasing 556 packet numbers. To ensure more flexibility and potentially to reduce 557 the ACK block section of the (MP_)ACK frame when aggregating 558 bandwidth of uniflows exhibiting different network characteristics, 559 each uniflow keeps its own monotonically increasing Packet Number 560 space. This potentially allows sending up to 2 * 2^64 packets on a 561 QUIC connection since each uniflow has its own packet number space 562 (see Section 8.1 for the detail of this limit). 564 With the introduction of multiple uniflows, there is a need to 565 acknowledge packets sent on different uniflows separately. The 566 packets sent on Initial Uniflows (with Uniflow ID 0) are still 567 acknowledged with regular ACK frames, such that no modification is 568 introduced in a core frame. For the other uniflows, the multipath 569 extensions introduce a MP_ACK frame which prefixes the ACK frame with 570 a Uniflow ID field indicating from which receiving uniflow the host 571 acknowledges packets. To better explain this, let us consider the 572 situation illustrated in Figure 5. 574 Sending uniflow 0 - CID A | Receiving uniflow 0 - CID A 575 Sending uniflow 1 - CID B | Receiving uniflow 1 - CID B 576 Receiving uniflow 0 - CID C | Sending uniflow 0 - CID C 577 Receiving uniflow 1 - CID D | Sending uniflow 1 - CID D 578 Receiving uniflow 2 - CID E | Sending uniflow 2 - CID E 580 Client Server 581 ------ ------ 582 | | 583 | Pkt(DCID=B,PN=42,frames=[STREAM("Request")]) | 584 |--------------------------- | 585 | |---------------------------->| 586 | | 587 | Pkt(DCID=E,PN=58,frames=[ | 588 | STREAM("Response"), MP_ACK(UID=1,LargestAcked=42)]) | 589 | ------------------------------| 590 |<-------------------------| | 591 | | 593 Figure 5: Acknowledging Packets Sent on Uniflows 595 Here five uniflows are in use, two in the client to server direction 596 and three in the reverse one. The client first sends a packet on its 597 sending Uniflow 1 (linked to CID B). The server receives the packet 598 on its receiving Uniflow 1. Therefore, it generates a MP_ACK frame 599 for Uniflow ID 1 and transmits it to the client. The server can 600 choose any of its sending uniflows to transmit this frame. In the 601 provided situation, the server sends its packets on Uniflow 2. The 602 client thus receives this packet on its receiving Uniflow 2. 604 Similarly, packets sent over a given uniflow might be acknowledged by 605 (MP_)ACK frames sent on another uniflow that does not share the same 606 network path. Looking at Figure 2 again, "Response 2" packet on 607 server's sending uniflow 1 with DCID D using the LTE network is 608 acknowledged by a MP_ACK frame received on a uniflow using the WLAN 609 network. 611 3.7. Exchanging Addresses 613 When a multi-homed device connects to a dual-stacked server using its 614 IPv4 address, it is aware of its local addresses (e.g., the WLAN and 615 the cellular ones) and the IPv4 remote address used to establish the 616 QUIC connection. If the client wants to create new uniflows and use 617 them over the IPv6 network, it needs to learn the other addresses of 618 the remote peer. 620 This is possible with the ADD_ADDRESS frames that are sent by a 621 Multipath QUIC host to advertise its current addresses. Each 622 advertised address is identified by an Address ID. The addresses 623 attached to a host can vary during the lifetime of a Multipath QUIC 624 connection. A new ADD_ADDRESS frame is transmitted when a host has a 625 new address. This ADD_ADDRESS frame is protected as other QUIC 626 control frames, which implies that it cannot be spoofed by attackers. 627 The communicated address MUST first be validated by the receiving 628 host before it starts using it as described in Section 8 of 629 [I-D.ietf-quic-transport]. This process ensures that the advertised 630 address actually belongs to the peer and that the peer can receive 631 packets sent by the host on the provided address. It also prevents 632 hosts from launching amplification attacks to a victim address. 634 If the client is behind a NAT, it could announce a private address in 635 an ADD_ADDRESS frame. In such situations, the server would not be 636 able to validate the communicated address. The client might still 637 use its NATed addresses to start using its sending uniflows. To 638 enable the server to make the link between the private and the public 639 addresses and hence conciliate the different 4-tuple views, Multipath 640 QUIC provides the UNIFLOWS frame that lists the current active 641 sending Uniflow IDs along with their associated local Address ID. 642 Notice that a host might also discover the public addresses of its 643 peer by observing its remote IP addresses associated to the 644 connection. 646 A receiving uniflow is active as soon as the host has sent the 647 MP_NEW_CONNECTION_ID frames proposing the corresponding Uniflow 648 Connection IDs to its peer. A sending uniflow is active when it has 649 received its Uniflow Connection IDs and is bound to a validated 650 4-tuple. The UNIFLOWS frame indicates the local Address IDs that the 651 uniflow uses from the sender's perspective. With this information, 652 the remote host can validate the public address and associate the 653 advertised one with the perceived addresses. 655 3.8. Coping with Address Removals 657 During the lifetime of a QUIC connection, a host might lose some of 658 its addresses. A concrete example is a smartphone going out of reach 659 of a WLAN network or shutting off one of its network interfaces. 660 Such address removals are advertised using REMOVE_ADDRESS frames. 661 The REMOVE_ADDRESS frame contains the Address ID of the lost address 662 previously communicated through ADD_ADDRESS. Notice that because a 663 given Address ID might encounter several events that need to be 664 ordered (e.g., ADD_ADDRESS, REMOVE_ADDRESS and ADD_ADDRESS again), 665 both ADD_ADDRESS and REMOVE_ADDRESS frames include an Address ID 666 related Sequence Number. 668 3.9. Uniflow Migration 670 At a given time, a Multipath QUIC endpoint gathers a set of active 671 sending and receiving uniflows, each associated to a 4-tuple. This 672 association is mutable. Hosts can change the 4-tuple used by their 673 sending uniflows at any time, enabling QUIC to migrate uniflows from 674 one network path to another. Yet, to address privacy issues due to 675 the linkability of addresses, hosts should avoid reusing the same 676 Connection ID used by a sending uniflow when the 4-tuple changes, as 677 described in Section 9.5 of [I-D.ietf-quic-transport]. 679 3.10. Handling Multiple Network Paths 681 The simultaneous usage of several sending uniflows introduces new 682 algorithms (packet scheduling, path management) whose specifications 683 are out of scope of this document. Nevertheless, these algorithms 684 are actually present in any multipath-enabled transport protocol like 685 Multipath TCP, CMT-SCTP and Multipath DCCP. A companion draft 686 [I-D.bonaventure-iccrg-schedulers] provides several general-purpose 687 packet schedulers depending on the application goals. A similar 688 document can be created to discuss path/uniflow management 689 considerations. 691 3.11. Congestion Control 693 The QUIC congestion control scheme is defined in 694 [I-D.ietf-quic-recovery]. This congestion control scheme is not 695 suitable when several sending uniflows are active. Using the 696 congestion control scheme defined in [I-D.ietf-quic-recovery] with 697 Multipath QUIC would result in unfairness. Each sending uniflow of a 698 Multipath QUIC connection MUST have its own congestion control state. 699 As for Multipath TCP, the windows of the different sending uniflows 700 MUST be coupled together [RFC6356]. 702 4. Mapping Uniflow IDs to Connection IDs 704 As described in the overview section, hosts need to identify on which 705 uniflows packets are sent. The Uniflow ID must then be inferred from 706 the public header. This is done by using the Destination Connection 707 ID field of Short Header packets. 709 The Initial Uniflow Connection IDs are determined during the 710 cryptographic handshake and actually correspond to both Connection 711 IDs in the current single-path QUIC design [I-D.ietf-quic-transport]. 712 Additional Uniflow Connection IDs for the Initial Uniflows can be 713 provided with the regular NEW_CONNECTION_ID frames. The Uniflow 714 Connection IDs of the other uniflows are determined when the 715 MP_NEW_CONNECTION_ID frames are exchanged. 717 Hosts MUST accept packets coming from their peer using the UCIDs they 718 proposed in the (MP_)NEW_CONNECTION_ID frames they sent and associate 719 them with the corresponding receiving Uniflow ID. Upon reception of 720 a (MP_)NEW_CONNECTION_ID frame, hosts MUST acknowledge it and MUST 721 store the advertised Uniflow Destination Connection ID and the 722 Uniflow ID of the proposed sending uniflow. 724 Hosts MUST ensure that all advertised Uniflow Connection IDs are 725 available for the whole connection lifetime, unless they have been 726 retired by their peer in the meantime by the reception of a 727 (MP_)RETIRE_CONNECTION_ID. 729 A host MUST NOT send MP_NEW_CONNECTION_ID frames with a Uniflow ID 730 greater than the value of "max_sending_uniflow_id" advertised by its 731 peer. 733 5. Using Multiple Uniflows 735 This section describes in details the Multipath QUIC operations. 737 5.1. Multipath Negotiation 739 The Multipath negotiation takes place during the cryptographic 740 handshake with the "max_sending_uniflow_id" transport parameter. A 741 QUIC connection is initially single-path in QUIC. During this 742 handshake, hosts advertise their support for multipath operations. 743 When a host advertises a value for the "max_sending_uniflow_id" 744 transport parameter, it indicates that it supports the multipath 745 extensions, i.e., the extensions defined in this document (not to be 746 mixed with the availability of local multiple network paths). If any 747 host does not advertise the "max_sending_uniflow_id" transport 748 parameter, multipath extensions are disabled. 750 The usage of multiple uniflows relies on the ability to use several 751 Connection IDs over a same QUIC connection. Therefore, zero-length 752 Connection IDs MUST NOT be used if the peer advertises a value 753 different from 0 for the "max_sending_uniflow_id" transport 754 parameter. 756 5.1.1. Transport Parameter Definition 758 A host MAY use the following transport parameter: 760 max_sending_uniflow_id (0x40): Indicates the support of the 761 multipath extension presented in this document, regardless of the 762 carried value. Its integer value puts an upper bound on the 763 number of sending uniflows the host advertising the value is ready 764 to support. If absent, this means that the host does not agree to 765 use the multipath extension over the connection. 767 5.2. Coping with Additional Remote Addresses 769 Hosts can learn remote addresses either by receiving ADD_ADDRESS 770 frames or observing the 4-tuple of incoming packets. Hosts MUST 771 first validate the newly learned remote IP addresses before starting 772 sending packets to those addresses. This requirement is explained in 773 Section 8.2. Hosts MUST initiate Address Validation Procedure as 774 specified in [I-D.ietf-quic-transport]. 776 A host MAY cache a validated address for a limited amount of time. 778 5.3. Receiving Uniflow State 780 When proposing uniflows to their peer, hosts need to maintain some 781 state for their receiving uniflows. This state is created upon the 782 sending of a first MP_NEW_CONNECTION_ID frame proposing the 783 corresponding Uniflow ID. As long as there is still one active 784 Uniflow Connection ID for this receiving uniflow (i.e., one UCID 785 which was not retired yet using a MP_RETIRE_CONNECTION_ID), the host 786 MUST accept packets over the receiving uniflow. Once created, hosts 787 MUST keep the following receiving uniflow information: 789 Uniflow ID: An integer that uniquely identifies the receiving 790 uniflow in the connection. This value is immutable. 792 Uniflow Connection IDs: Possible values for the Connection ID field 793 of packets belonging to this receiving uniflow. This value 794 contains the sequence of active UCIDs that were advertised in 795 previously sent MP_NEW_CONNECTION_ID frames. Notice that this 796 sequence might be empty, e.g., when all advertised UCIDs have been 797 retired by the peer. 799 Packet Number Space: Packet number space dedicated to this receiving 800 uniflow. Packet number considerations described in Section 12.3 801 of [I-D.ietf-quic-transport] apply within a given receiving 802 uniflow. 804 Associated 4-tuple: The tuple (sIP, dIP, sport, dport) currently 805 observed to receive packets over this uniflow. This value is 806 mutable, because a host might receive a packet with a different 807 (possibly) validated remote address and/or port than the one 808 previously recorded. If a host observes a change in the 4-tuple 809 of the receiving uniflow, it follows the considerations of 810 Section 9.5 of [I-D.ietf-quic-transport]. 812 Associated local Address ID: The Address ID advertised in 813 ADD_ADDRESS frames sent by the peer corresponding to the local 814 address used to receive packets. This helps to generate UNIFLOWS 815 frames advertising the mapping between uniflows and addresses. 816 The addresses on which the connection was established have Address 817 ID 0. 819 Hosts can also collect network measurements on a per-uniflow basis, 820 like the number of packets received. 822 5.4. Sending Uniflow State 824 During a Multipath QUIC connection, hosts maintain some state for 825 sending uniflows. The state of the sending uniflow determines 826 information that hosts are required to store. The possible sending 827 uniflow states are depicted in Figure 6. 829 o 830 | 831 | receive a first MP_NEW_CONNECTION_ID 832 | with the associated Uniflow ID 833 | 834 v path usage over a validated 4-tuple 835 +----------+ ------------------------------------> +----------+ 836 | UNUSED | | ACTIVE | 837 +----------+ <------------------------------------ +----------+ 838 address change or retired UCID 840 Figure 6: Finite-State Machine describing the possible states of 841 a sending uniflow 843 Once a sending uniflow has been proposed by the peer in a received 844 MP_NEW_CONNECTION_ID frame, it is in the UNUSED state. In this 845 situation, hosts MUST keep the following sending uniflow information: 847 Uniflow ID: An integer that uniquely identifies the sending uniflow 848 in the connection. This value is immutable. 850 Uniflow Connection IDs: Possible values for the Connection ID field 851 of packets belonging to this sending uniflow. This value contains 852 the sequence of active UCIDs that were advertised in previously 853 received MP_NEW_CONNECTION_ID frames. Notice that this sequence 854 might be empty, e.g., when all advertised UCIDs have been retired. 856 Sending Uniflow State: The current state of the sending uniflow, 857 being one of the values presented in Figure 6. 859 Packet Number Space: Packet number space dedicated to this sending 860 uniflow. Packet number considerations described in Section 12.3 861 of [I-D.ietf-quic-transport] apply within a given sending uniflow. 863 Notice that a UNUSED sending uniflow MAY send probing packets to 864 validate a given 4-tuple. 866 When the host wants to start using the sending uniflow over a 867 validated address, the sending uniflow goes to the ACTIVE state. 868 This is the state where a sending uniflow can be used to send 869 packets. Having an uniflow in ACTIVE state only guarantees that it 870 can be used, but the host is not forced to. In addition to the 871 fields required in the UNUSED state, the following elements MUST be 872 tracked: 874 Associated 4-tuple: The tuple (sIP, dIP, sport, dport) currently 875 used to packets over this uniflow. This value is mutable, as the 876 host might decide to change its local (or remote) address and/or 877 port. A host that changes the 4-tuple of a sending uniflow SHOULD 878 migrate it. 880 Associated (local Address ID, remote Address ID) tuple: Those 881 identifiers come from the ADD_ADDRESS sent (local) and received 882 (remote). This enables a host to temporarily stop using a sending 883 uniflow when, e.g., the remote Address ID is declared as lost in a 884 REMOVE_ADDRESS. The addresses on which the connection was 885 established have Address ID 0. The reception of UNIFLOWS frames 886 helps hosts associate the remote Address ID used by the sending 887 uniflow. 889 Congestion controller: A congestion window limiting the transmission 890 rate of the sending uniflow. 892 Performance metrics: Basic statistics such as one-way delay or the 893 number of packets sent. This information can be useful when a 894 host needs to perform packet scheduling decisions and flow 895 control. 897 It might happen that a sending path is temporarily unavailable, 898 because one of the endpoint's addresses is no more available or 899 because the host retired all the UCIDs of the sending uniflow. In 900 such cases, the path goes back to the UNUSED state. When performing 901 a transition back to the UNUSED state, hosts MUST reset the 902 additional state added by the ACTIVE state. In the UNUSED state, the 903 host MUST NOT send non-probing packets on it. At this state, the 904 host might want to restart using the uniflow over another validated 905 4-tuple, switching the uniflow state back to the ACTIVE state. 906 However, its congestion controller state MUST be restarted and its 907 performance metrics SHOULD be reset. 909 5.5. Losing Addresses 911 During the lifetime of a connection, a host might lose addresses, 912 e.g., a network interface that was shut down. All the ACTIVE sending 913 uniflows that were using that local address MUST stop sending packets 914 from that address. To advertise the loss of an address to the peer, 915 the host MUST send a REMOVE_ADDRESS frame indicating which local 916 Address IDs has been lost. The host MUST also send an UNIFLOWS frame 917 indicating the status of the remaining ACTIVE uniflows. 919 Upon reception of the REMOVE_ADDRESS, the receiving host MUST stop 920 using the ACTIVE sending uniflows affected by the address removal. 922 Hosts MAY reuse one of these sending uniflows by changing the 923 assigned 4-tuple. In this case, it MUST send an UNIFLOWS frame 924 describing that change. 926 6. New Frames 928 To support the multipath operations, new frames have been defined to 929 coordinate hosts. The following table summarizes the added frames. 931 +=============+=========================+=============+======+======+ 932 | Type Value | Frame Type Name | Definition | Pkts | Spec | 933 +=============+=========================+=============+======+======+ 934 | 0x40 | MP_NEW_CONNECTION_ID | Section 6.1 | ___1 | P | 935 +-------------+-------------------------+-------------+------+------+ 936 | 0x41 | MP_RETIRE_CONNECTION_ID | Section 6.2 | ___1 | | 937 +-------------+-------------------------+-------------+------+------+ 938 | 0x42 - | MP_ACK | Section 6.3 | ___1 | NC | 939 | 0x43 | | | | | 940 +-------------+-------------------------+-------------+------+------+ 941 | 0x44 | ADD_ADDRESS | Section 6.4 | ___1 | | 942 +-------------+-------------------------+-------------+------+------+ 943 | 0x45 | REMOVE_ADDRESS | Section 6.5 | ___1 | | 944 +-------------+-------------------------+-------------+------+------+ 945 | 0x46 | UNIFLOWS | Section 6.6 | ___1 | | 946 +-------------+-------------------------+-------------+------+------+ 948 Table 1: Multipath-related Frame Types 950 Table 1 uses the same notation convention as the Table 3 of 951 [I-D.ietf-quic-transport] for the Pkts and Spec columns. In 952 particular, all frames defined in this document MUST be exchanged in 953 1-RTT packets. A host receiving one of the multipath-related frames 954 in other encryption context MUST close the connection with a 955 PROTOCOL_VIOLATION error. All the frames are ack-eliciting except 956 the MP_ACK frame. The MP_ACK frame does not count towards bytes in 957 flight if the packet containing it only carries either ACK or MP_ACK 958 frames. The MP_NEW_CONNECTION_ID frame is the only new frame that 959 can be sent to probe new network paths. 961 The remaining of this document uses the notation convention described 962 in [I-D.ietf-quic-transport]. 964 6.1. MP_NEW_CONNECTION_ID Frames 966 The MP_NEW_CONNECTION_ID frame (type=0x40) is an extension of the 967 NEW_CONNECTION_ID frame defined by [I-D.ietf-quic-transport]. It 968 provides the peer with alternative Connection IDs and associates them 969 to a particular uniflow using the Uniflow ID. 971 The format of the MP_NEW_CONNECTION_ID frame is as follows. 973 MP_NEW_CONNECTION_ID Frame { 974 Type (i) = 0x40, 975 Uniflow ID (i), 976 Sequence Number (i), 977 Retire Prior To (i), 978 Length (8), 979 Connection ID (8..160), 980 Stateless Reset Token (128), 981 } 983 Figure 7: MP_NEW_CONNECTION_ID Frame Format 985 Compared to the NEW_CONNECTION_ID frame specified in 986 [I-D.ietf-quic-transport], the following field is added. 988 Uniflow ID: Indicates to which uniflow the provided Connection ID 989 relates. 991 The remaining fields keep the same semantic as for the 992 NEW_CONNECTION_ID frame. 994 This frame can be sent by both hosts. Upon reception of the frame 995 with a specified Uniflow ID, the peer MUST update the related sending 996 uniflow state and store the communicated Connection ID. 998 To limit the delay of the multipath usage upon handshake completion, 999 hosts SHOULD send MP_NEW_CONNECTION_ID frames for receive uniflows 1000 they allow using as soon the connection establishment completes. 1002 The generation of Connection ID MUST follow the same considerations 1003 as presented in Section 5.1 of [I-D.ietf-quic-transport]. 1005 6.2. MP_RETIRE_CONNECTION_ID Frame 1007 The MP_RETIRE_CONNECTION_ID frame (type=0x41) is an extension of the 1008 RETIRE_CONNECTION_ID frame defined by [I-D.ietf-quic-transport]. It 1009 indicates that the end-host will no longer use a Connection ID 1010 related to a given uniflow that was issued by its peer. 1012 The format of the MP_RETIRE_CONNECTION_ID frame is shown below. 1014 MP_RETIRE_CONNECTION_ID Frame { 1015 Type (i) = 0x41, 1016 Uniflow ID (i), 1017 Sequence Number (i), 1018 } 1020 Figure 8: MP_RETIRE_CONNECTION_ID Frame Format 1022 Compared to the RETIRE_CONNECTION_ID frame specified in 1023 [I-D.ietf-quic-transport], the following field is added. 1025 Uniflow ID: Indicates on which uniflow the Connection ID is retired. 1027 The frame is handled as the RETIRE_CONNECTION_ID frame described in 1028 [I-D.ietf-quic-transport] on an uniflow basis. 1030 6.3. MP_ACK Frame 1032 The MP_ACK frame (types 0x42 and 0x43) is an extension of the ACK 1033 frame defined by [I-D.ietf-quic-transport]. It allows hosts to 1034 acknowledge packets that were sent on non-initial uniflows. If the 1035 frame type is 0x43, MP_ACK frames also contain the sum of QUIC 1036 packets with associated ECN marks received on the connection up to 1037 this point. 1039 The format of the MP_ACK frame is shown below. 1041 MP_ACK Frame { 1042 Type (i) = 0x02..0x03, 1043 Uniflow ID (i), 1044 Largest Acknowledged (i), 1045 ACK Delay (i), 1046 ACK Range Count (i), 1047 First ACK Range (i), 1048 ACK Range (..) ..., 1049 [ECN Counts (..)], 1050 } 1052 Figure 9: MP_ACK Frame Format 1054 Compared to the ACK frame specified in [I-D.ietf-quic-transport], the 1055 following field is added. 1057 Uniflow ID: Indicates on which uniflow the acknowledged packet 1058 sequence numbers relate. 1060 6.4. ADD_ADDRESS Frame 1062 The ADD_ADDRESS frame (type=0x44) is used by a host to advertise its 1063 currently reachable addresses. 1065 The format of the ADD_ADDRESS frame is shown below. 1067 ADD_ADDRESS Frame { 1068 Type (i) = 0x44, 1069 Rsv (3) = 0, 1070 P (1), 1071 IP Version (4), 1072 Address ID (8), 1073 Sequence Number (i), 1074 Interface Type (8), 1075 IP Address (32/128), 1076 [Port (16)], 1077 } 1079 Figure 10: ADD_ADDRESS Frame Format 1081 The ADD_ADDRESS frame contains the following fields. 1083 Rsv: These bits are set to 0, and are reserved for future use. 1085 P bit: This bit indicates, if set, the presence of the Port field. 1087 IP Version: Written on 4 bits, contains the version of the IP 1088 address contained in the IP Address field. 1090 Address ID: An one-byte unique identifier for the advertised address 1091 for tracking and removal purposes. This is needed when, e.g., a 1092 NAT changes the IP address such that both hosts see different IP 1093 addresses for a same network path. 1095 Sequence Number: An Address ID related sequence number of the event, 1096 encoded as a variable-length integer. The sequence number space 1097 is shared with REMOVE_ADDRESS frames mentioning the same Address 1098 ID. 1100 Interface Type: A one-byte field providing an indication about the 1101 interface type to which this address is bound. The following 1102 values are defined: 1104 * 0: fixed. Used as default value. 1106 * 1: WLAN 1107 * 2: cellular 1109 IP Address: The advertised IP address, in network order. 1111 Port: This optional field indicates the port number related to the 1112 advertised IP address. When this field is present, it indicates 1113 that an uniflow can use the 2-tuple (IP addr, port). 1115 Upon reception of an ADD_ADDRESS frame, the receiver SHOULD store the 1116 communicated address for future use. 1118 The receiver MUST NOT send packets others than validation ones to the 1119 communicated address without having validated it as specified in 1120 Section 8 of [I-D.ietf-quic-transport]. ADD_ADDRESS frames SHOULD 1121 contain globally reachable addresses. Link-local and possibly 1122 private addresses SHOULD NOT be exchanged. 1124 6.5. REMOVE_ADDRESS Frame 1126 The REMOVE_ADDRESS frame (type=0x45) is used by a host to signal that 1127 a previously announced address was lost. 1129 The format of the REMOVE_ADDRESS frame is shown below. 1131 REMOVE_ADDRESS Frame { 1132 Type (i) = 0x45, 1133 Address ID (8), 1134 Sequence Number (i), 1135 } 1137 Figure 11: REMOVE_ADDRESS Frame Format 1139 The REMOVE_ADDRESS frame contains the following fields. 1141 Address ID: The one-byte identifier of the address to remove. 1143 Sequence Number: An Address ID related sequence number of the event, 1144 encoded as a variable-length integer. The sequence number space 1145 is shared with ADD_ADDRESS frames mentioning the same Address ID. 1146 This help the receiver figure out that a REMOVE_ADDRESS might have 1147 been sent before an ADD_ADDRESS frame implying the same Address 1148 ID, even if for some reason the REMOVE_ADDRESS reaches the 1149 receiver after the newer ADD_ADDRESS one. 1151 A host SHOULD stop using sending uniflows using the removed address 1152 and set them in the UNUSED state. If the REMOVE_ADDRESS contains an 1153 Address ID that was not previously announced, the receiver MUST 1154 silently ignore the frame. 1156 6.6. UNIFLOWS Frame 1158 The UNIFLOWS frame (type=0x46) communicates the uniflows' state of 1159 the sending host to the peer. It allows the sender to communicate 1160 its active uniflows to the peer in order to detect potential 1161 connectivity issue over uniflows. It also enables hosts to map 1162 Address IDs to seen 4-tuples when middleboxes affecting them (e.g., 1163 NATs,...) are present. 1165 The format of the UNIFLOWS frame is shown below. 1167 UNIFLOWS Frame { 1168 Type (i) = 0x46, 1169 Sequence Number (i), 1170 Receiving Uniflows (i), 1171 Active Sending Uniflows (i), 1172 Receiving Uniflow Info Section (..) ..., 1173 Active Sending Uniflow Info Section (..) ..., 1174 } 1176 Figure 12: UNIFLOWS Frame Format 1178 The UNIFLOWS frame contains the following fields. 1180 Sequence Number: A variable-length integer. This value starts at 0 1181 and increases by 1 for each UNIFLOWS frame sent by the host. It 1182 allows identifying the most recent UNIFLOWS frame. 1184 Receiving Uniflows: The current number of receiving uniflows from 1185 the sender's point of view. 1187 Active Sending Uniflows: The current number of sending uniflows in 1188 the ACTIVE state from the sender's point of view. 1190 Receiving Uniflow Info Section: Contains information about the 1191 receiving uniflows (there are Receiving Uniflows entries). 1193 Sending Uniflow Info Section: Contains information about the sending 1194 uniflows in ACTIVE state (there are Active Sending Uniflows 1195 entries). 1197 Both Receiving Uniflow Info and Active Sending Uniflow Info Sections 1198 share the same format which is shown below. 1200 Uniflow Info Section { 1201 Uniflow ID (i), 1202 Local Address ID (8), 1203 } 1204 Figure 13: Uniflow Info Section Format 1206 The fields in the Uniflow Info Section are the following. 1208 Uniflow ID: The Uniflow ID of the uniflow the sending host provides 1209 information about. 1211 LocAddrID: The local Address ID of the address currently used by the 1212 uniflow. 1214 The Uniflow Info section only contains the local Address ID so far, 1215 but this section can be extended later with other potentially useful 1216 information. 1218 7. Extension of the Meaning of Existing QUIC Frames 1220 The multipath extensions do not modify the wire format of existing 1221 QUIC frames. However, they extend the meaning of a few of them while 1222 keeping this addition transparent and consistent with the single-path 1223 QUIC design. The concerned frames (and their extended meaning) are 1224 the following. 1226 NEW_CONNECTION_ID: Equivalent to a MP_NEW_CONNECTION_ID frame with 1227 Uniflow ID set to 0. 1229 RETIRE_CONNECTION_ID: Equivalent to a MP_RETIRE_CONNECTION_ID frame 1230 with Uniflow ID set to 0. 1232 ACK: Equivalent to a MP_ACK frame with Uniflow ID set to 0. 1234 8. Security Considerations 1236 8.1. Nonce Computation 1238 With Multipath QUIC, each uniflow has its own packet number space. 1239 With the current nonce computation [I-D.ietf-quic-tls], using twice 1240 the same packet number over two different uniflows on the same 1241 direction leads to the same cryptographic nonce. Using twice the 1242 same nonce MUST NOT happen, hence MP-QUIC has a different nonce 1243 computation than [I-D.ietf-quic-tls] 1244 The left most bits of nonce MUST be the Uniflow ID that identifies 1245 the current uniflow up to max_sending_uniflow_id. The remaining bits 1246 of the nonce is formed by an exclusive OR of the least significant 1247 bits of the packet protection IV with the padded packet number (left- 1248 padded with 0s). The nonce MUST be left-padded with a 0 if 1249 max_sending_uniflow_id <= 2, and the max_sending_uniflow_id MUST NOT 1250 be higher than 2^61. If a uniflow has sent 1251 2^62-max_sending_uniflow_id packets, another uniflow MUST be used to 1252 avoid re-using the same nonce. 1254 8.2. Validation of Exchanged Addresses 1256 To use addresses communicated by the peer through ADD_ADDRESS frames, 1257 hosts are required to validate them before using uniflows to these 1258 addresses as described in Section 8 of [I-D.ietf-quic-transport]. 1259 Section 21.12.3 of [I-D.ietf-quic-transport] provides additional 1260 motivation for this process. In addition, hosts MUST send ADD 1261 ADDRESS frames in 1-RTT frames to prevent off-path attacks. 1263 9. IANA Considerations 1265 9.1. QUIC Transport Parameter Registry 1267 This document defines a new transport parameter for the negotiation 1268 of multiple paths. The following entry in Table 2 should be added to 1269 the "QUIC Transport Parameters" registry under the "QUIC Protocol" 1270 heading. 1272 +=======+========================+===============+ 1273 | Value | Parameter Name | Specification | 1274 +=======+========================+===============+ 1275 | 0x40 | max_sending_uniflow_id | Section 5.1.1 | 1276 +-------+------------------------+---------------+ 1278 Table 2: Addition to QUIC Transport Parameters 1279 Entries 1281 10. Acknowledgments 1283 We would like to thank Masahiro Kozuka and Kazuho Oku for their 1284 numerous comments on the first version of this draft. We also thank 1285 Philipp Tiesel for his early comments that led to the current design, 1286 and Ian Swett for later feedback. We also want to thank Christian 1287 Huitema for his draft about multipath requirements to identify 1288 critical elements about the multipath feature. Mohamed Boucadair 1289 provided lot of useful inputs on the second version of this document. 1290 Maxime Piraux and Florentin Rochet helped us to improve the last 1291 versions of this draft. This project was partially supported by the 1292 MQUIC project funded by the Walloon Government. 1294 11. References 1296 11.1. Normative References 1298 [I-D.ietf-quic-invariants] 1299 Thomson, M., "Version-Independent Properties of QUIC", 1300 Work in Progress, Internet-Draft, draft-ietf-quic- 1301 invariants-13, 14 January 2021, 1302 . 1305 [I-D.ietf-quic-recovery] 1306 Iyengar, J. and I. Swett, "QUIC Loss Detection and 1307 Congestion Control", Work in Progress, Internet-Draft, 1308 draft-ietf-quic-recovery-34, 14 January 2021, 1309 . 1312 [I-D.ietf-quic-tls] 1313 Thomson, M. and S. Turner, "Using TLS to Secure QUIC", 1314 Work in Progress, Internet-Draft, draft-ietf-quic-tls-34, 1315 14 January 2021, . 1318 [I-D.ietf-quic-transport] 1319 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 1320 and Secure Transport", Work in Progress, Internet-Draft, 1321 draft-ietf-quic-transport-34, 14 January 2021, 1322 . 1325 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1326 Requirement Levels", BCP 14, RFC 2119, 1327 DOI 10.17487/RFC2119, March 1997, 1328 . 1330 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1331 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1332 May 2017, . 1334 11.2. Informative References 1336 [I-D.bonaventure-iccrg-schedulers] 1337 Bonaventure, O., Piraux, M., Coninck, Q. D., Baerts, M., 1338 Paasch, C., and M. Amend, "Multipath schedulers", Work in 1339 Progress, Internet-Draft, draft-bonaventure-iccrg- 1340 schedulers-01, 9 September 2020, 1341 . 1344 [I-D.huitema-quic-mpath-req] 1345 Huitema, C., "QUIC Multipath Requirements", Work in 1346 Progress, Internet-Draft, draft-huitema-quic-mpath-req-01, 1347 7 January 2018, . 1350 [IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments", 1351 IETF Journal , November 2016. 1353 [MFQUIC] De Coninck, Q. and O. Bonaventure, "Multiflow QUIC: A 1354 Generic Multipath Transport Protocol", To appear in IEEE 1355 Communications Magazine. A preprint is available at 1356 https://hdl.handle.net/2078.1/243486 , May 2021. 1358 [MPQUIC] De Coninck, Q. and O. Bonaventure, "Multipath QUIC: Design 1359 and Evaluation", 13th International Conference on emerging 1360 Networking EXperiments and Technologies (CoNEXT 2017). 1361 http://multipath-quic.org , December 2017. 1363 [MPRTP] Singh, V., Ahsan, S., and J. Ott, "MPRTP: Multipath 1364 considerations for real-time media", Proceedings of the 1365 4th ACM Multimedia Systems Conference , 2013. 1367 [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J.- 1368 Y. Le Boudec, "MPTCP is not pareto-optimal: performance 1369 issues and a possible solution", Proceedings of the 8th 1370 international conference on Emerging networking 1371 experiments and technologies, ACM , 2012. 1373 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1374 RFC 793, DOI 10.17487/RFC0793, September 1981, 1375 . 1377 [RFC3077] Duros, E., Dabbous, W., Izumiyama, H., Fujii, N., and Y. 1378 Zhang, "A Link-Layer Tunneling Mechanism for 1379 Unidirectional Links", RFC 3077, DOI 10.17487/RFC3077, 1380 March 2001, . 1382 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 1383 Congestion Control for Multipath Transport Protocols", 1384 RFC 6356, DOI 10.17487/RFC6356, October 2011, 1385 . 1387 [RFC8041] Bonaventure, O., Paasch, C., and G. Detal, "Use Cases and 1388 Operational Experience with Multipath TCP", RFC 8041, 1389 DOI 10.17487/RFC8041, January 2017, 1390 . 1392 [RFC8684] Ford, A., Raiciu, C., Handley, M., Bonaventure, O., and C. 1393 Paasch, "TCP Extensions for Multipath Operation with 1394 Multiple Addresses", RFC 8684, DOI 10.17487/RFC8684, March 1395 2020, . 1397 [SCTPCMT] Iyengar, J., Amer, P., and R. Stewart, "Concurrent 1398 multipath transfer using SCTP multihoming over independent 1399 end-to-end paths", IEEE/ACM Transactions on networking, 1400 Vol. 14, no 5 , 2006. 1402 Appendix A. Comparison with Multipath TCP 1404 Multipath TCP [RFC8684] is currently the most widely deployed 1405 multipath transport protocol on the Internet. While its design 1406 impacted the initial versions of the Multipath extensions for the 1407 QUIC protocol, there are now major differences between both protocols 1408 that we now highlight. 1410 A.1. Multipath TCP Bidirectional Paths vs. QUIC Uniflows 1412 TCP ensures reliable data delivery by sending back acknowledgments to 1413 the data sender. Because data flows in one direction and 1414 acknowledgments in the other, the notion of bidirectional paths 1415 became a de facto standard with TCP. The Multipath TCP extension 1416 [RFC8684] combines several TCP connections to spread a single data 1417 stream over them. Hence, all the paths of a Multipath TCP connection 1418 must be bidirectional. However, networking experiences showed that 1419 packets following a direction do not always share the exact same road 1420 as the packets in the opposite direction. Furthermore, QUIC does not 1421 require a network path to be bidirectional in order to be used, as 1422 many parts of its design handle possible network asymmetries 1423 (unidirectional Connection IDs, PATH_RESPONSE that can flow on a 1424 different network path than the elliciting PATH_CHALLENGE,...). 1426 A.2. Negotiating the Multipath Extensions 1428 A Multipath TCP connection relies on the TCP option field of the TCP 1429 header to negotiate the multipath extensions. During the handshake, 1430 Multipath TCP uses the MP_CAPABLE TCP option to exchange connection's 1431 keys used to authenticate additional Multipath TCP subflows and 1432 generate tokens used to identify the connection. However, TCP 1433 options are sent in clear-text. Any on-path network observer may 1434 record the connection's keys and create Multipath TCP subflows on it. 1435 Multipath QUIC does not face this security issue. The multipath 1436 extensions are negotiated using authenticated transport parameters of 1437 the QUIC handshake. Then, Multipath QUIC leverages the encryption 1438 feature of QUIC to hide information from network observers. 1440 A.3. Uniflow Establishment 1442 To create additional subflows to a Multipath TCP connection, hosts 1443 initiate a TCP handshake by negotitating the MP_JOIN TCP option. 1444 This option carries a token matching the TCP subflow to a Multipath 1445 TCP connection and a HMAC value computed using the exchanged 1446 connection's keys. In addition that connection's keys were exchanged 1447 in clear-text during the handshake and that the token is also in 1448 clear-text, the HMAC value (using SHA-256) is truncated to the 1449 leftmost 64 bits (in SYN/ACK) or 160 bits (in third ACK) because of 1450 the TCP option length limitation to 40 bytes. Multipath QUIC avoids 1451 all these issues. The negotiation and usage of additional uniflows 1452 are performed using encrypted messages and the length of frames are 1453 only limited by the size of the packet that carries them. 1455 Another difference comes in the control of the number of paths/ 1456 uniflows in use. Multipath TCP has a new subflow when the 1457 corresponding TCP handshake succeeds. However, it is not possible to 1458 restrict in advance the number of paths that a Multipath TCP 1459 connection can use. Therefore, the only way for a server to control 1460 the number of paths is to adopt a reactive approach, i.e., to reset 1461 or blackhole exceeding subflows from the client. In Multipath QUIC, 1462 each host sets a upper limit on the number of sending uniflows that 1463 it wants to use, while keeping control on the number of sending 1464 uniflows it provides to its peer. This path management is hence 1465 proactive. 1467 A.4. Exchanging Data over Multiple Uniflows 1469 One of the key design decision of Multipath TCP is that all its 1470 subflows have to behave like regular TCP connections to handle 1471 network interference. Each Multipath TCP subflow has to keep in- 1472 sequence data delivery and dedicates its TCP sequence number to this 1473 end. To handle multipath reordering, an additional Data Sequence 1474 Number at the Multipath TCP level is needed. In Multipath QUIC, the 1475 uniflow on which data is sent is a packet-level information. This 1476 means that a frame can be sent regardless of the uniflow of the 1477 packet carrying it. Furthermore, because the (STREAM) data offset is 1478 a frame-level information, there is no need to define additional 1479 sequence numbers to cope with reordering across uniflows. 1481 In addition to this signaling overhead, Multipath TCP faces 1482 performance issues due to this acknoweldgment constraint. Consider 1483 the following scenario. Some data was first transmitted on a lossy 1484 path A, such that the peer never receives it. The sender can 1485 (successfully) retransmit the same data over a working path B (and 1486 gets the corresponding acknowledgment). However, the sender cannot 1487 send new data on the path A as long as the initial lost data was not 1488 delivered on that path (because of the TCP behavior constraint). 1489 Such transmission overhead is not present in Multipath QUIC, as there 1490 is no rule that a Multipath QUIC uniflow has to behave like a single- 1491 path QUIC one. 1493 Another consequence of the "Multipath TCP subflows must behave like 1494 regular TCP connections" is that acknowkedgments have to stay on the 1495 same network path as the one used by the data. This constraint on 1496 the acknowledgment strategy is not present (and hardly enforceable) 1497 in Multipath QUIC as frames are independent of packets. Multipath 1498 QUIC can better benefit from high-latency paths and enable the usage 1499 of unidirectional networks. 1501 A.5. Advertising Host's Addresses 1503 Multipath TCP enables host to communicate their local IP addresses to 1504 its peer by using the ADD_ADDR TCP option. Similarly, REMOVE_ADDR 1505 TCP option is used to advertise the loss of a local address to its 1506 peer. Their usage is however subject to security issues, as these 1507 options are communicated in clear-text, possibly leaking the host's 1508 IP addresses to the network. This security concern does not affect 1509 Multipath QUIC as all this information is encrypted in frames. 1511 Another point is related to the reliability of the address 1512 advertisement. In Multipath TCP, the ADD_ADDR and REMOVE_ADDR 1513 options are sent unreliably, i.e., there is no acknowledgment 1514 mechanism for their reception. While Multipath TCP [RFC8684] 1515 provides an "echo" mechanism to the ADD_ADDR, there is no such 1516 equivalent for REMOVE_ADDR. In Multipath QUIC, ADD_ADDRESS and 1517 REMOVE_ADDRESS frames are ack-elliciting, making them reliable. 1519 A.6. Congestion Control 1521 Multipath TCP uses the LIA congestion control scheme specified in 1522 [RFC6356]. This scheme can immediately be adapted to Multipath QUIC. 1523 Other coupled congestion control schemes have been proposed for 1524 Multipath TCP such as [OLIA]. 1526 Appendix B. Change Log 1528 B.1. Since draft-deconinck-quic-multipath-06 1530 * Link with recent publication 1532 B.2. Since draft-deconinck-quic-multipath-05 1534 * Summarize frame types in a table 1536 * Update frame format to structure style 1538 * Update text to match draft-ietf-quic-transport-32 1540 * Link to scheduling companion draft 1542 * Remove dangling to-dos 1544 B.3. Since draft-deconinck-quic-multipath-04 1546 * Mostly editorial and reference fixes 1548 B.4. Since draft-deconinck-quic-multipath-03 1550 * Clarify the notion of asymmetric paths by introducing uniflows 1552 * Remove the PATH_UPDATE frame 1554 * Rename PATHS frame to UNIFLOWS frame and adapt its content 1556 * Add a sequence number to frames involving Address ID events (#4) 1558 * Disallow Zero-length connection ID (#2) 1560 * Correctly handle nonce computation (thanks to Florentin Rochet) 1561 * Focus on the core concepts of multipath and delegate algorithms to 1562 companion drafts 1564 * Updated text to match draft-ietf-quic-transport-27 1566 B.5. Since draft-deconinck-quic-multipath-02 1568 * Consider asymmetric paths 1570 B.6. Since draft-deconinck-quic-multipath-01 1572 * Include path policies considerations 1574 * Add practical considerations thanks to Mohamed Boucadair inputs 1576 * Adapt the RETIRE_CONNECTION_ID frame 1578 * Updated text to match draft-ietf-quic-transport-18 1580 B.7. Since draft-deconinck-quic-multipath-00 1582 * Comply with asymmetric Connection IDs 1584 * Add max_paths transport parameter to negotiate initial number of 1585 active paths 1587 * Path ID as a regular varint 1589 * Remove max_path_id transport parameter 1591 * Updated text to match draft-ietf-quic-transport-14 1593 B.8. Since draft-deconinck-multipath-quic-00 1595 * Added PATH_UPDATE frame 1597 * Added MAX_PATHS frame 1599 * No more packet header change 1601 * Implicit Path ID notification using Connection ID and 1602 NEW_CONNECTION_ID frames 1604 * Variable-length encoding for Path ID 1606 * Updated text to match draft-ietf-quic-transport-10 1608 * Fixed various typos 1610 Authors' Addresses 1612 Quentin De Coninck 1613 UCLouvain 1615 Email: quentin.deconinck@uclouvain.be 1617 Olivier Bonaventure 1618 UCLouvain 1620 Email: olivier.bonaventure@uclouvain.be