idnits 2.17.1 draft-deconinck-multipath-quic-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (October 30, 2017) is 2369 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-34) exists of draft-ietf-quic-recovery-06 == Outdated reference: A later version (-34) exists of draft-ietf-quic-tls-07 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-07 -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 6824 (Obsoleted by RFC 8684) Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group Q. De Coninck 3 Internet-Draft O. Bonaventure 4 Intended status: Standards Track UCLouvain 5 Expires: May 3, 2018 October 30, 2017 7 Multipath Extension for QUIC 8 draft-deconinck-multipath-quic-00 10 Abstract 12 Multipath TCP has shown how a reliable transport protocol can 13 efficiently use multiple paths for a given connection. We leverage 14 the experience gained with Multipath TCP to propose simple extensions 15 that enable QUIC to efficiently use multiple paths during the 16 lifetime of a QUIC connection. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at http://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on May 3, 2018. 35 Copyright Notice 37 Copyright (c) 2017 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (http://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3 54 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4 55 3.1. What is a Path? . . . . . . . . . . . . . . . . . . . . . 4 56 3.2. Going Further than Connection Migration . . . . . . . . . 4 57 3.3. Starting a Multipath QUIC Connection . . . . . . . . . . 6 58 3.4. Multipath QUIC Architecture . . . . . . . . . . . . . . . 7 59 3.5. Exchanging Data over Multiple Paths . . . . . . . . . . . 7 60 3.6. Starting to Use Paths . . . . . . . . . . . . . . . . . . 8 61 3.7. Communicating New Addresses . . . . . . . . . . . . . . . 9 62 3.8. Path Migration . . . . . . . . . . . . . . . . . . . . . 9 63 3.9. Coping with Address Removals . . . . . . . . . . . . . . 10 64 3.10. Congestion Control . . . . . . . . . . . . . . . . . . . 11 65 4. Packet Format Changes . . . . . . . . . . . . . . . . . . . . 11 66 5. Using Multiple Paths . . . . . . . . . . . . . . . . . . . . 12 67 5.1. Multipath Negotiation . . . . . . . . . . . . . . . . . . 12 68 5.1.1. Transport Parameter Definition . . . . . . . . . . . 13 69 5.2. Path State . . . . . . . . . . . . . . . . . . . . . . . 13 70 6. Modifications to QUIC frames . . . . . . . . . . . . . . . . 14 71 6.1. ACK Frame . . . . . . . . . . . . . . . . . . . . . . . . 14 72 7. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 15 73 7.1. ADD_ADDRESS Frame . . . . . . . . . . . . . . . . . . . . 15 74 7.2. REMOVE_ADDRESS Frame . . . . . . . . . . . . . . . . . . 16 75 7.3. PATHS Frame . . . . . . . . . . . . . . . . . . . . . . . 16 76 8. Security Considerations . . . . . . . . . . . . . . . . . . . 17 77 8.1. Nonce Computation . . . . . . . . . . . . . . . . . . . . 17 78 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 79 9.1. QUIC Transport Parameter Registry . . . . . . . . . . . . 18 80 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 81 10.1. Normative References . . . . . . . . . . . . . . . . . . 18 82 10.2. Informative References . . . . . . . . . . . . . . . . . 19 83 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 85 1. Introduction 87 Endhosts have evolved. Today's endhosts are equipped with several 88 network interfaces and users expect to be able to seamlessly switch 89 from one to another or use them simultaneously to aggregate 90 bandwidth. During the last years, several multipath extensions to 91 transport protocols have been proposed [RFC6824],[MPRTP],[SCTPCMT]. 92 Multipath TCP [RFC6824] is the most mature one. It is already 93 deployed on popular smartphones and for other use cases [RFC8041]. 95 With regular TCP and UDP, all the packets that belong to a given flow 96 contain the same 5-tuple that acts as an identifier for this flow. 97 This prevents flows from using multiple paths. QUIC 99 [I-D.ietf-quic-transport] does not use the 5-tuple as an implicit 100 connection identifier. A QUIC flow is identified by its Connection 101 ID. This enables flows to survive to events such as NAT rebinding or 102 mobility cases where the IP address or the port of one of the 103 communicating peer changes. This connection migration feature is key 104 for QUIC to migrate a flow from one path to another. However, this 105 path change is implicit and the current design 106 [I-D.ietf-quic-transport] still assumes single-path flows. Seamless 107 handovers between wireless networks on smartphones are one of the 108 motivations for connection migration in QUIC. However, experience 109 with Multipath TCP shows that the handover between different wireless 110 networks is not an abrupt process [Cellnet],[IETFJ] . To support 111 seamless handovers, it is important to be able to use two (or more) 112 paths simultaneously during the handover. 114 Bringing Multipath to QUIC allows hosts to aggregate several networks 115 while providing better handover support, and potentially opens new 116 use cases. A detailed performance evaluation and a comparison 117 between Multipath QUIC and Multipath TCP may be found in [MPQUIC]. 119 In this draft, we leverage many of the lessons learned from the 120 design of Multipath TCP and propose extensions to the current QUIC 121 design to enable it to simultaneously use several paths. This 122 document is organized as follows. It first provides an overview of 123 the operation of Multipath QUIC. It then states changes required in 124 the packet format and specifies the usage of multiple paths. It also 125 defines new frames to perform multipath operations. Finally, it 126 provides some security and IANA considerations. 128 2. Conventions and Definitions 130 The words "MUST", "MUST NOT", "SHOULD", and "MAY" are used in this 131 document. It's not shouting; when they are capitalized, they have 132 the special meaning defined in [RFC2119]. 134 We assume that the reader is familiar with the terminology used in 135 the QUIC documents [I-D.ietf-quic-transport]. In addition, we 136 define: 138 o Path: A logical association between two hosts over which packets 139 can be sent. A path is identified by a Path ID. 141 o Initial Path: The path used for the establishment of the QUIC 142 connection. The cryptographic handshake is done on this path. It 143 is identified by Path ID 0. 145 3. Overview 147 The current design of QUIC [I-D.ietf-quic-transport] provides 148 reliable transport with multiplexing and security. A wide range of 149 devices on today's Internet are multihomed. Examples include 150 smartphones equipped with both WiFi and cellular interfaces, but also 151 regular dual-stack hosts that use both IPv4 and IPv6. Experience 152 with Multipath TCP has shown that the ability to combine different 153 paths during the lifetime of a connection provides various benefits 154 including bandwidth aggregation or seamless handovers 155 [RFC8041],[IETFJ]. 157 The current design of QUIC does not enable multihomed devices to 158 efficiently use different paths. We first explain why a multipath 159 extension would be beneficial to QUIC and them describe it at a high 160 level. 162 3.1. What is a Path? 164 Before going into details, let's first define what is called a 165 "path". A path is a UDP flow between two hosts denoted by a 4-tuple 166 (IP source address, IP destination address, source port, destination 167 port). On a smartphone interacting with a single-homed server, the 168 mobile device could decide to use one path over the WiFi network and 169 another over the cellular one. Those paths are not necessarily 170 completely disjoint. For example, when interacting with a dual-stack 171 server, a smartphone may create two paths over the WiFi network, one 172 over IPv4 and the other one over IPv6. 174 3.2. Going Further than Connection Migration 176 Unlike TCP [RFC0793], QUIC is not bound to a particular 4-tuple 177 during the lifetime of a connection. A QUIC connection is identified 178 by a Connection ID, placed in the public header of QUIC packets. 179 This enables hosts to continue the connection even if the 4-tuple 180 changed due to, e.g., NAT rebinding. This ability to shift a 181 connection from one 4-tuple to another is called Connection 182 Migration. Another of its use cases is fail-over when the address in 183 use fails but another one is available. A mobile device loosing the 184 WiFi connectivity can then continue the connection over its cellular 185 interface. 187 A QUIC connection can thus start on a given path and end on another 188 one. However, the current QUIC design [I-D.ietf-quic-transport] 189 assumes that only one path is in use for a given connection. This 190 connection migration feature is not intended to support the 191 simultaneous usage of multiple paths. To illustrate this point, 192 consider the following scenario where a smartphone connected to both 193 WiFi and LTE networks sends a POST request fitting in 2 packets and 194 receives a large response from the single-homed server. 196 Server Phone Server 197 via WiFi via LTE 198 ------- ------- ----- 199 | Pkt(CID=A, PN=1, frames=[ | 200 | STREAM("Request (1/2)")]) | Pkt(CID=A, PN=2, frames=[ | 201 |<----------------------------| STREAM("Request (2/2)")]) | 202 | Pkt(CID=A, PN=1, frames=[ |-------- | 203 | ACK(LargestAcked=1)]) | |---------- | 204 |---------------------------->| |---------- | 205 | | |-->| 206 | | Pkt(CID=A, PN=2, frames=[ | 207 | | ACK(LargestAcked=2), | 208 | | STREAM("Response 1")]) ----| 209 | | ----------| | 210 | | ----------| | 211 | |<------| | 212 | | Pkt(CID=A, PN=3, frames=[ | 213 | | ACK (LargestAcked=2)]) | 214 | |-------- | 215 | | |---------- | 216 | | |---------- | 217 | | |-->| 218 | | Pkt(CID=A, PN=2, frames=[ | 219 | | STREAM("Response 2")]) | 220 | | ----| 221 | | ... <---------| | 223 Figure 1: Single-path QUIC with multiple paths 225 Assume the client wants to aggregate all the network interfaces it 226 has. It sends the first packet with the STREAM frame containing the 227 beginning of the request on the WiFi interface. It then sends the 228 second one on its LTE interface. If the WiFi network exhibits a 229 lower latency than the LTE one, the server will first receive the 230 packet from the WiFi network and acknowledge it by sending an ACK 231 frame on the WiFi. Then, it receives the second packet from the LTE 232 network. Thanks to the Connection ID (CID) in the public header of 233 the QUIC packet, the server detects that this packet belongs to the 234 same connection and performs a connection migration over this new 235 4-tuple. At this point, the connection remains stuck on the cellular 236 network, unless the smartphone sends a new packet over the WiFi 237 network. This is because there is currently no way for the server to 238 know that the remote peer uses two different networks paths with 239 potentially different properties. The smartphone might have sent the 240 first response ACK frame on the WiFi to drift the connection towards 241 the WiFi network, but the server would lose its ability to use the 242 cellular one, and would possibly observe large RTT variance over the 243 connection. The multipath extension of QUIC aims to achieve an 244 efficient usage of multiple paths by making them explicit to peers. 246 With the proposed multipath extension to QUIC, the example presented 247 in Figure 1 can become the one presented below. 249 Server Phone Server 250 via WiFi via LTE 251 ------- ------- ----- 252 | Pkt(CID=A,PID=1,PN=1,frames=[ | | 253 | STREAM("Request (1/2)")]) | Pkt(CID=A,PID=3,PN=1,frames=[ | 254 |<------------------------------| STREAM("Request (2/2)")]) | 255 | Pkt(CID=A,PID=1,PN=1,frames=[ |-------- | 256 | ACK(PID=1,LargestAcked=1)]) | |---------- | 257 |------------------------------>| |---------- | 258 | Pkt(CID=A,PID=1,PN=2,frames=[ | |-->| 259 | STREAM("Response 1")]) | Pkt(CIA=A,PID=3,PN=1,frames=[ | 260 |------------------------------>| ACK(PID=3,LargestAcked=1), | 261 | | STREAM("Response 2")]) ----| 262 | Pkt(CID=A,PID=1,PN=2,frames=[ | ----------| | 263 | ACK(PID=1, LargestAcked=2), | ----------| | 264 | ACK(PID=3, LargestAcked=1)]) |<------| | 265 |<------------------------------| | 266 | Pkt(CID=A,PID=1,PN=3,frames=[ | Pkt(CIA=A,PID=3,PN=2,frames=[ | 267 | STREAM("Response 3")]) | STREAM("Response 4")]) | 268 |------------------------------>| ----| 269 | | ----------| | 270 | ... | ... <---------| | 272 Figure 2: Dataflow with Multipath QUIC 274 With the notion of multiple paths explicitly advertised, the server 275 is aware that multiple paths using different networks are present, 276 and can potentially be used simultaneously since it knows the 4-tuple 277 to use for each path. When the server receives the request that was 278 carried over two different paths, it can then use both of them to 279 transfer back the response to the client. The remaining of this 280 section focuses on giving a high-level view of the multipath 281 operations in QUIC. 283 3.3. Starting a Multipath QUIC Connection 285 Before using multiple paths, the QUIC connection must be established. 286 A Multipath QUIC connection always starts over an initial path where 287 the cryptographic handshake takes place over the dedicated stream 288 with Stream ID 0. The establishment is thus performed as in the 289 current QUIC design [I-D.ietf-quic-transport] [I-D.ietf-quic-tls]. 290 The negotiation of multipath is performed during the cryptographic 291 handshake with the max_path_id transport parameter, where both hosts 292 advertise how many paths they are willing to use. The number of 293 paths that can then be used over the connection once handshake 294 completes is the minimum between both advertised values. The path on 295 which the cryptographic handshake and the path number negotiation are 296 performed is called the Initial Path and is identified by the Path ID 297 0. 299 3.4. Multipath QUIC Architecture 301 Once the connection is established, QUIC can start to use as many 302 paths as negotiated. A Multipath QUIC connection is composed of 303 several paths. Each path is associated with a different four-tuple 304 and identified by a Path ID, as shown in Figure 3. 306 +---------------------------------------------+ 307 | Connection | 308 | +--------+ +--------+ ... +------------+ | 309 | | Path 0 | | Path 1 | | Path N - 1 | | 310 | | Tuple | | Tuple' | ... | Tuple" | | 311 | | PN | | PN' | | PN" | | 312 | +--------+ +--------+ ... +------------+ | 313 +---------------------------------------------+ 315 Figure 3: Architectural view of Multipath QUIC 317 A Multipath QUIC connection starts on the Initial Path, identified by 318 Path ID 0. In Multipath QUIC, each packet (except the handshake 319 packets at the beginning of the connection) explicitly contains the 320 path identifier of the path it belongs to. This Path ID is included 321 in the QUIC public header of each packet. This design simplifies the 322 detection of packet losses on a per-path basis and enables a receiver 323 to easily detect out-of-order packets on a given path. Hosts can 324 also collect network information about each path, such as round-trip- 325 time measurements and maintain a per-path congestion window. 327 3.5. Exchanging Data over Multiple Paths 329 A QUIC packet acts as a container of one of more frames. Multipath 330 QUIC uses the same STREAM frames as QUIC to carry data. A byte 331 offset is associated to the data payload. One of the key design 332 decision of (Multipath) QUIC is that frames are independent of the 333 packets carrying them. This implies that a frame transmitted over 334 one path could be retransmitted later on another path without any 335 change. 337 However, the path on which data is sent is a packet-level 338 information. This means a frame can be sent regardless of the path 339 of the packet carrying it. Furthermore, because the data offset is a 340 frame-level information, there is no need to define additional 341 sequence numbers to cope with reordering across paths, unlike 342 Multipath TCP [RFC6824] that uses a Data Sequence Number at MPTCP 343 level. Other flow control considerations like the stream receive 344 window advertised by the MAX_STREAM_DATA frame remain unchanged when 345 there are multiple paths. 347 However, Multipath QUIC might face reordering at packet-level when 348 using paths having different latencies. The presence of the Path ID 349 in the public header ensures that the packets sent over a given path 350 will contain monotonically increasing packet numbers. To ensure more 351 flexibility and potentially to reduce the ACK block section of the 352 ACK frame when aggregating bandwidth of paths exhibiting different 353 network characteristics, each path keeps its own monotonically 354 increasing Packet Number space. This potentially allows sending 256 355 * 2^64 packets on a QUIC connection since each path (with a Path ID 356 encoded on 1 byte) has its own packet number space. 358 The ACK frame is also modified to allow per-path packet 359 acknowledgments. This remains compliant with the design decision of 360 the independence between packets and frames while providing more 361 flexibility to hosts to decide on which path they want to send path 362 acknowledgments. Looking again in Figure 2, packets that were sent 363 over a given path (e.g., the response2 packet on path 3) can be 364 acknowledged on another path (here, path 1) to limit the latency due 365 to ACK transmission on high-latency paths. Such scheduling decision 366 would not have been possible in Multipath TCP [RFC6824] which must 367 acknowledge data coming from a given path on the same path. 369 3.6. Starting to Use Paths 371 The cryptographic handshake determines how many paths, in addition to 372 the initial one, can be used. Once this handshake completes, hosts 373 can start using them simply by transmitting QUIC packets with the 374 associated Path ID. In contrast with Multipath TCP, Multipath QUIC 375 does not require a per-path handshake. This reduces the time 376 required to use a not used yet path. Multipath QUIC is fully 377 symmetrical. Both client and server can start using new paths. 379 Although a path can be first used by any host, it might not be 380 practical for one of the peers to start using new paths. A possible 381 cause is when a server wants to initiate the usage of a new path to a 382 NAT'd client. The client would possibly not receive the packet, 383 leading to connectivity issues on that path. To detect such issues, 384 the PATHS frame provides a list of the current active paths of the 385 sending hosts to the peer. A path is called active when a functional 386 network 4-tuple (on which either packets were sent or received on it 387 or host received acknowledgments for packets over that path) is 388 assigned to it. Furthermore, the PATHS frame indicates which 4-tuple 389 the path is currently using. It also contains some path status 390 metrics such as the round-trip-time estimated by the sending host 391 over a given path in order to provide a global view of the path 392 performance. 394 3.7. Communicating New Addresses 396 When a multi-homed mobile device connects to a dual-stacked server on 397 its IPv4 address, it is aware of its local addresses (e.g., the WiFi 398 and the cellular ones) and the IPv4 remote address used to establish 399 connection. If the client wants to create new paths over IPv6, it 400 needs to learn the other addresses of the remote peer. 402 This is possible with the ADD_ADDRESS frames that are sent by a 403 Multipath QUIC host to advertise its current addresses. Each 404 advertised address has an Address ID given by the sending host. The 405 addresses assigned to a host can vary during the lifetime of a 406 Multipath QUIC connection. A new ADD_ADDRESS frame is transmitted 407 when host has a new address. This ADD_ADDRESS frame is protected as 408 other QUIC control frames, which implies that it cannot be spoofed by 409 attackers. 411 3.8. Path Migration 413 At a given time, a Multipath QUIC connection gathers a set of paths, 414 each denoted by a 4-tuple. The 4-tuple that is associated to a path 415 is not fixed. It may change during the lifetime of a connection. 416 Those changes can be caused by NAT rebindings or handovers for 417 example. 419 Host 420 ---- 421 UDP(sIP= 192.0.2.0, dIP=198.51.100.0, sport=S, dport=D, payload= | 422 QUICPkt(CID=A, PN=X, PathID=1)) | 423 --------------------------------------------------------> | 424 | 425 UDP(sIP= 198.51.100.0, dIP=192.0.2.0, sport=D, dport=S, payload= | 426 QUICPkt(CID=A, PN=Y, PathID=1)) | 427 <-------------------------------------------------------- | 428 | 429 UDP(sIP= 203.0.113.0, dIP=198.51.100.0, sport=T, dport=D, payload= | 430 QUICPkt(CID=A, PN=X+1, PathID=1)) | 431 --------------------------------------------------------> | 432 | 433 UDP(sIP= 198.51.100.0, dIP=203.0.113.0, sport=D, dport=T, payload= | 434 QUICPkt(CID=A, PN=Y+1, PathID=1)) | 435 <-------------------------------------------------------- | 436 | 438 Figure 4: Example of path migration 440 Figure 4 shows an example of path whose 4-tuple changes during the 441 connection. Initially, from the host perspective, path 1 is using 442 the 4-tuple (198.51.100.0, 192.0.2.0, D, S). The first packet it 443 receives still uses it and the host uses that 4-tuple to generate the 444 next packet on that path. Then, host receives a packet on path 1 445 with a different 4-tuple (198.51.100.0, 203.0.113.0, D, T), with both 446 changed remote IP and port. Because the QUIC packet contains both 447 the Connection ID and the Path ID it belongs to, host can simply 448 adapt the path 4-tuple to the one of the last packet received, and 449 next packets on path 1 will use that 4-tuple. Because the Path ID is 450 its only identifier, a path is not bound to a particular 4-tuple, and 451 can shift to another one. Multipath QUIC thus integrates the 452 Connection Migration ability at path level, providing the Path 453 Migration ability. 455 3.9. Coping with Address Removals 457 During the life of the QUIC connection, an host might lose some of 458 its addresses . A concrete example is a smartphone going out of 459 reachability of a WiFi network or shutting off one of its network 460 interfaces. Such address removals are advertised by REMOVE_ADDRESS 461 frames. Those frames contain the Address ID of the lost address. 463 Thanks to this frame, an host can stop using a path whose 4-tuple 464 contains the removed address. However, if a NAT'd client losses its 465 private address and advertises this event, the sever will not know to 466 which public address this advertisement is intended and might still 467 use paths using that particular address. The PATHS frame helps the 468 server to perform the mapping between both addresses. It contains 469 for each active path the local Address ID used by the sending host. 470 With both the PATHS and the REMOVE_ADDRESS frames, the server can 471 identify which paths will be affected by the address removal to stop 472 their usage before being migrated to another 4-tuple. 474 3.10. Congestion Control 476 The QUIC congestion control scheme is defined in 477 [I-D.ietf-quic-recovery]. This congestion control scheme is not 478 suitable when several paths are active. Using the congestion control 479 scheme defined in [I-D.ietf-quic-recovery] with Multipath QUIC would 480 result in unfairness. Each path of a Multipath QUIC connection MUST 481 have its own congestion window. The windows of the different paths 482 MUST be coupled together. Multipath TCP uses the LIA congestion 483 control scheme specified in [RFC6356]. This scheme can immediately 484 be adapted to Multipath QUIC. Other coupled congestion control 485 schemes have been proposed for Multipath TCP such as [OLIA]. 487 4. Packet Format Changes 489 This section describes the changes required in the packet format. As 490 previously described, multipath capabilities must be enabled after 491 connection establishment. Because the initial path has Path ID 0, 492 packets with long headers are considered to be sent on Path 0, 493 therefore no change is required on the wire. Since only short 494 headers are expected to be sent once the connection is established, 495 only those require the Path ID as a field. The proposed short header 496 is presented in Figure 5. 498 0 1 2 3 499 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 500 +-+-+-+-+-+-+-+-+ 501 |0|C|K|M|Type(4)| 502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 503 | | 504 + [Connection ID (64)] + 505 | | 506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 507 | [Path ID (8)] | 508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 | Packet Number (8/16/32) ... 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 511 | Protected Payload (*) ... 512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 514 Figure 5: New Short Header for Packets 516 The usage of multiple paths is indicated with the M bit. If this bit 517 is set, the packet also contains the Path ID field indicating the 518 path the packet belongs to. If the multipath flag is not set, the 519 packet belongs to path 0. Notice that Path ID 0 can be used as a 520 regular path to exchange data, but a single-path connection MUST use 521 path 0 for all exchanged packets. 523 5. Using Multiple Paths 525 5.1. Multipath Negotiation 527 The Multipath Negotiation takes place during the cryptographic 528 handshake with the max_path_id transport parameter. A QUIC 529 connection is initially single-path in QUIC, and all packets prior to 530 handshake completion MUST be exchanged over the Initial Path. During 531 this process, hosts advertise the maximum path ID they are willing to 532 use. The maximum path ID that can be used over the connection is the 533 minimum between both advertised values. A connection can then use 534 any path with a Path ID comprised between 0 and the negotiated 535 maximum path ID inclusive. If one of the host does not provides the 536 max_path_id transport parameter during the cryptographic handshake, 537 the remote MUST assume a value of 0, leading to a single-path 538 connection over the Initial Path. Packets with a Path ID greater 539 than the negotiated maximum value MUST be ignored. 541 5.1.1. Transport Parameter Definition 543 An endhost MAY use the following transport parameter: 545 max_path_id (0x0007): The maximum path ID transport parameter 546 indicates the number of paths in addition to the Initial Path that 547 the host is willing to use over the connection, encoded as an 548 unsigned 8-bit integer. This indicates that received packets 549 larger than this limit will be dropped. The default value for 550 this parameter is 0, meaning that an host omitting this transport 551 parameter does not want to use multiple paths over the connection. 553 5.2. Path State 555 A path is associated to a UDP flow over which packets can be sent or 556 received. The following state is maintained for a given path. 558 o Path ID: this 1-byte number uniquely identifies a path in a 559 connection. This value is immutable. 560 o Packet Number Space: each path is associated with own monotonically 561 increasing packet number space. Each endpoint maintains a separate 562 packet number for sending and receiving. Packet number 563 considerations described in {{I-D.ietf-quic-transport}} apply 564 within a given path. 565 o Current 4-tuple: the tuple (sIP, dIP, sport, dport) used by the 566 path to send or receive packets. This value is mutable and can also 567 be empty when the path is not in use. If this value is set, the 568 path is considered as active. The tuple can change either because 569 the host decides to change its local address and/or port, or 570 because it receives a packet with a different remote address and/or 571 port than currently recorded. To cope with possible packet 572 reordering within a given path, the remote address and port 573 recorded by the host MUST match the one of the received packet with 574 the largest Packet Number. The Initial Path MUST remain active at 575 any time of the connection. 576 o Current (local Address ID, remote Address ID) tuple: those 577 identifiers come from the ADD_ADDRESS sent (local) and received 578 (remote). This enables host to detect a path as unusable when, 579 e.g., the remote Address ID is declared as lost by a 580 REMOVE_ADDRESS. The addresses on which the connection was 581 established have Address ID 0. The reception of PATHS frames 582 helps hosts to perform the matching between the remote Address ID 583 and the path. 584 o Performance metrics: basic statistics such as round-trip-time or 585 number of packets sent and received can be collected on a per-path 586 basis. This information can be useful when an host needs to perform 587 packet scheduling decisions and flow control management. 589 6. Modifications to QUIC frames 591 The multipath extension allows hosts to send packets over multiple 592 paths. Since nearly all QUIC frames are independent of packets, no 593 change is required. The only exception is the ACK frame that 594 contains packet-level information with the Largest Acknowledged 595 field. Since the Packet Number are now linked to paths, the ACK 596 frame must contain the Path ID it acknowledges. 598 6.1. ACK Frame 600 This draft changes the type byte of the ACK frame. The new type is 601 formatted as "10PNLLMM", and is parsed as follows: 603 o The two high-order bits must be set to 10 indicating that this is 604 an ACK frame. This changes from the 101 prefix of the current QUIC 605 design [I-D.ietf-quic-transport]. Note that the prefix 100 is 606 currently unused in [I-D.ietf-quic-transport]. 608 o The "P" bit indicates the presence of the Path ID field in the ACK 609 frame. If unset, the ACK frame relates to the default path (i.e., 610 Path ID 0). 612 o Other bits remain unchanged. 614 The format of the modified ACK frame is shown below. 616 0 1 2 3 617 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 618 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 619 | [Path ID (8)] |[Num Blocks(8)]| NumTS (8) | 620 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 621 | Largest Acknowledged (8/16/32/48) ... 622 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 623 | ACK Delay (16) | 624 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 625 | ACK Block Section (*) ... 626 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 627 | Timestamp Section (*) ... 628 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 630 Figure 6: ACK Frame adapted to Multipath 632 Compared to the ACK frame in the current QUIC design 633 [I-D.ietf-quic-transport], the ACK frame can contain an optional Path 634 ID field indicating to which path the acknowledged PSNs relate to. 635 Since frames are independent of packets, and the path notion relates 636 to the packets, the ACK frames can be sent on any path, unlike 637 Multipath TCP [RFC6824] which is constrained to send ACKs on the same 638 path. The impact of such a strategy on the latency estimation has to 639 be explored further. 641 7. New Frames 643 To support the multipath operations, new frames have been defined to 644 coordinate hosts. This draft uses a type field containing 0x10 to 645 indicate that the frame is related to multipath operations. 647 7.1. ADD_ADDRESS Frame 649 The ADD_ADDRESS frame is used by a host to advertise its currently 650 reachable addresses. The proposed type for the ADDRESS frame is 651 0x10. An ADD_ADDRESS frame is shown below. 653 0 1 2 3 654 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 655 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 656 |0|0|0|P|IPVers.|Address ID (8) | 657 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 658 | IP Address (32/128) ... 659 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 660 | [Port (16)] | 661 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 663 Figure 7: ADD_ADDRESS Frame 665 The ADD_ADDRESS frame contains the following fields: 667 o Reserved bits: the three most-significant bits of the first byte 668 are set to 0, and are reserved for future use. 670 o P bit: the fourth most-significant bit of the first byte indicates, 671 if set, the presence of the Port field. 673 o IPVers.: the remaining four least-significant bits of the first 674 byte contains the version of the IP address contained in the IP 675 Address field. 677 o IP Address: the advertised IP address, in network order. 679 o Port: this optional field indicates the port number related to the 680 advertised IP address. When this field is present, it indicates 681 that a path can use the 2-tuple (IP addr, port). 683 7.2. REMOVE_ADDRESS Frame 685 The REMOVE_ADDRESS frame is used by a host to signal that a 686 previously announced address was lost. The proposed type for the 687 REMOVE_ADDRESS frame is 0x11. A REMOVE_ADDRESS frame is shown below. 689 0 1 2 3 690 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 691 +-+-+-+-+-+-+-+-+ 692 |Address ID (8) | 693 +-+-+-+-+-+-+-+-+ 695 Figure 8: REMOVE_ADDRESS Frame 697 The frame contains only one field, Address ID, being the identifier 698 of the address to remove. A host SHOULD stop using paths using the 699 removed address until they have been migrated to another available 700 address. If the REMOVE_ADDRESS contains an Address ID that was not 701 previously announced, host MUST ignore the frame. 703 7.3. PATHS Frame 705 The PATHS frame communicates the paths state of the sending host to 706 the peer. It allows the sender to communicate its active paths to 707 the peer in order to detect potential connectivity issue over paths. 708 Its proposed type is 0x12. The format of the PATHS frame is shown 709 below. 711 0 1 2 3 712 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 713 +-+-+-+-+-+-+-+-+ 714 |ActivePaths (8)| 715 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 716 | Path Info Section (*) ... 717 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 719 Figure 9: PATHS Frame 721 The PATHS frame contains the following fields: 723 o ActivePaths: the current number of paths considered as active 724 sender point of view, excluding the Initial Path. ActivePaths MUST 725 be lower or equal to the maximum Path ID negotiated. 727 o Path Info Section: contains information about all the active paths 728 (i.e., there are ActivePaths + 1 entries). The format of this 729 section is shown below. 731 0 1 2 3 732 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 733 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 734 | Path ID 0 (8) |LocAddrID 0 (8)| 735 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 736 | RTT Path 0 (32) | 737 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 738 ... 739 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 740 | Path ID N (8) |LocAddrID N (8)| 741 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 742 | RTT Path N (32) | 743 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 745 Figure 10: Path Info Section 747 The fields in the Path Info Section are: 749 o Path ID: the Path ID of the active path the sending host provides 750 information about. 752 o LocAddrID: the local Address ID of the address currently used by 753 the path. 755 o RTT Path: the RTT experienced by the sending host over the path 756 with the provided Path ID. The formatting is similar to the one 757 used for the ACK delay field in the ACK frame. 759 The Path Info section currently contains the RTT of the sending host, 760 but this section can be extended to provide additional information in 761 order to get a global picture of the connection at both ends. 763 8. Security Considerations 765 8.1. Nonce Computation 767 With Multipath QUIC, each path has its own packet number space. With 768 the current nonce computation [I-D.ietf-quic-tls], using twice the 769 same packet number over two different paths leads to the same 770 cryptographic nonce. Depending on the size of the Initial Value (and 771 hence the nonce), there are two ways to mitigate this issue. 773 If the Initial Value has a length of 8 bytes, then a packet number 774 used on a given path MUST NOT be reused on another path of the 775 connection, and therefore at most 2^64 packets can be sent on a QUIC 776 connection. This means there will be packet number skipping at path 777 level, but the packet number will remain monotonically increasing on 778 each path. 780 If the Initial Value has a length of 9 or more, then the 781 cryptographic nonce computation is now performed as follow. The 782 nonce, N, is formed by combining the packet protection IV (either 783 client_pp_iv_n or server_pp_iv_n) with the Path ID and the packet 784 number. The 64 bits of the reconstructed QUIC packet number in 785 network byte order is left-padded with zeros to the size of the IV. 786 The 8 bits of the Path ID is right-padded with zeros to the size of 787 the IV. The Path IV is computed as the exclusive OR of the padded 788 Path ID and the IV. The exclusive OR of the padded packet number and 789 the Path IV forms the AEAD nonce. 791 9. IANA Considerations 793 9.1. QUIC Transport Parameter Registry 795 This document defines a new transport parameter for the negotiation 796 of multiple paths. The following entry in Table 1 should be added to 797 the "QUIC Transport Parameters" registry under the "QUIC Protocol" 798 heading. 800 +--------+----------------+---------------+ 801 | Value | Parameter Name | Specification | 802 +--------+----------------+---------------+ 803 | 0x0007 | max_path_id | Section 5.1.1 | 804 +--------+----------------+---------------+ 806 Table 1: Addition to QUIC Transport Parameters Entries 808 10. References 810 10.1. Normative References 812 [I-D.ietf-quic-recovery] 813 Iyengar, J. and I. Swett, "QUIC Loss Detection and 814 Congestion Control", draft-ietf-quic-recovery-06 (work in 815 progress), September 2017. 817 [I-D.ietf-quic-tls] 818 Thomson, M. and S. Turner, "Using Transport Layer Security 819 (TLS) to Secure QUIC", draft-ietf-quic-tls-07 (work in 820 progress), October 2017. 822 [I-D.ietf-quic-transport] 823 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 824 and Secure Transport", draft-ietf-quic-transport-07 (work 825 in progress), October 2017. 827 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 828 Requirement Levels", BCP 14, RFC 2119, 829 DOI 10.17487/RFC2119, March 1997, . 832 10.2. Informative References 834 [Cellnet] Paasch, C., Detal, G., Duchene, F., Raiciu, C., and O. 835 Bonaventure, "Exploring Mobile/WiFi Handover with 836 Multipath TCP", ACM SIGCOMM workshop on Cellular Networks 837 (Cellnet'12) , 2012. 839 [IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments", 840 IETF Journal , November 2016. 842 [MPQUIC] De Coninck, Q. and O. Bonaventure, "Multipath QUIC: Design 843 and Evaluation", 13th International Conference on emerging 844 Networking EXperiments and Technologies (CoNEXT 2017). 845 http://multipath-quic.org , December 2017. 847 [MPRTP] Singh, V., Ahsan, S., and J. Ott, "MPRTP: Multipath 848 considerations for real-time media", Proceedings of the 849 4th ACM Multimedia Systems Conference , 2013. 851 [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J. 852 Le Boudec, "MPTCP is not pareto-optimal: performance 853 issues and a possible solution", Proceedings of the 8th 854 international conference on Emerging networking 855 experiments and technologies, ACM , 2012. 857 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 858 RFC 793, DOI 10.17487/RFC0793, September 1981, 859 . 861 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 862 Congestion Control for Multipath Transport Protocols", 863 RFC 6356, DOI 10.17487/RFC6356, October 2011, 864 . 866 [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, 867 "TCP Extensions for Multipath Operation with Multiple 868 Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, 869 . 871 [RFC8041] Bonaventure, O., Paasch, C., and G. Detal, "Use Cases and 872 Operational Experience with Multipath TCP", RFC 8041, 873 DOI 10.17487/RFC8041, January 2017, . 876 [SCTPCMT] Iyengar, J., Amer, P., and R. Stewart, "Concurrent 877 multipath transfer using SCTP multihoming over independent 878 end-to-end paths", IEEE/ACM Transactions on networking, 879 Vol. 14, no 5 , 2006. 881 Authors' Addresses 883 Quentin De Coninck 884 UCLouvain 886 Email: quentin.deconinck@uclouvain.be 888 Olivier Bonaventure 889 UCLouvain 891 Email: Olivier.Bonaventure@uclouvain.be