idnits 2.17.1 draft-lmbdhk-quic-multipath-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Both endpoints, namely the client and the server, can close a path, by sending PATH_ABANDON frame (see Section 10.1) which abandons the path with a corresponding Path Identifier. Once a path is marked as "abandoned", it means that the resources related to the path, such as the used connection IDs, can be released. However, information related to data delivered over that path SHOULD not be released immediately as acknowledgments can still be received or other frames that also may trigger retransmission of data on another path. -- The document date (25 October 2021) is 914 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 762 == Missing Reference: 'X' is mentioned on line 812, but not defined == Missing Reference: 'Y' is mentioned on line 814, but not defined -- Looks like a reference, but probably isn't: '1' on line 818 == Missing Reference: 'U' is mentioned on line 816, but not defined -- Looks like a reference, but probably isn't: '2' on line 816 == Missing Reference: 'V' is mentioned on line 818, but not defined == Unused Reference: 'QUIC-Invariants' is defined on line 1090, but no explicit reference was found in the text == Outdated reference: A later version (-08) exists of draft-huitema-quic-ts-06 Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group Y. Liu 3 Internet-Draft Y. Ma 4 Intended status: Standards Track Alibaba Inc. 5 Expires: 28 April 2022 Q. De Coninck 6 O. Bonaventure 7 UCLouvain 8 C. Huitema 9 Private Octopus Inc. 10 M. Kuehlewind, Ed. 11 Ericsson 12 25 October 2021 14 Multipath Extension for QUIC 15 draft-lmbdhk-quic-multipath-00 17 Abstract 19 This document specifies a multipath extension for the QUIC protocol 20 to enable the simultaneous usage of multiple paths for a single 21 connection. 23 Discussion Venues 25 This note is to be removed before publishing as an RFC. 27 Discussion of this document takes place on the QUIC Working Group 28 mailing list (quic@ietf.org), which is archived at 29 https://mailarchive.ietf.org/arch/browse/quic/. 31 Source for this draft and an issue tracker can be found at 32 https://github.com/mirjak/draft-lmbdhk-quic-multipath. 34 Status of This Memo 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. 39 Internet-Drafts are working documents of the Internet Engineering 40 Task Force (IETF). Note that other groups may also distribute 41 working documents as Internet-Drafts. The list of current Internet- 42 Drafts is at https://datatracker.ietf.org/drafts/current/. 44 Internet-Drafts are draft documents valid for a maximum of six months 45 and may be updated, replaced, or obsoleted by other documents at any 46 time. It is inappropriate to use Internet-Drafts as reference 47 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on 28 April 2022. 50 Copyright Notice 52 Copyright (c) 2021 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 57 license-info) in effect on the date of publication of this document. 58 Please review these documents carefully, as they describe your rights 59 and restrictions with respect to this document. Code Components 60 extracted from this document must include Simplified BSD License text 61 as described in Section 4.e of the Trust Legal Provisions and are 62 provided without warranty as described in the Simplified BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 67 1.1. Conventions and Definitions . . . . . . . . . . . . . . . 4 68 2. Handshake Negotiation and Transport Parameter . . . . . . . . 5 69 3. Path Setup and Removal . . . . . . . . . . . . . . . . . . . 6 70 3.1. Path Initiation . . . . . . . . . . . . . . . . . . . . . 7 71 3.2. Path Close . . . . . . . . . . . . . . . . . . . . . . . 7 72 3.2.1. Use PATH_ABANDON Frame to Close a Path . . . . . . . 7 73 3.2.2. Effect of RETIRE_CONNECTION_ID Frame . . . . . . . . 8 74 3.2.3. Idle Timeout . . . . . . . . . . . . . . . . . . . . 9 75 3.3. Path States . . . . . . . . . . . . . . . . . . . . . . . 9 76 4. Congestion Control . . . . . . . . . . . . . . . . . . . . . 11 77 5. Computing Path RTT . . . . . . . . . . . . . . . . . . . . . 11 78 6. Packet Scheduling . . . . . . . . . . . . . . . . . . . . . . 12 79 7. Packet Number Space and Use of Connection ID . . . . . . . . 13 80 7.1. Using One Packet Number Space . . . . . . . . . . . . . . 13 81 7.1.1. Sending Acknowledgements and Handling Ranges . . . . 14 82 7.2. Using Multiple Packet Number Spaces . . . . . . . . . . . 15 83 7.2.1. Packet Protection for QUIC Multipath . . . . . . . . 15 84 7.2.2. Key Update for QUIC Multipath . . . . . . . . . . . . 16 85 8. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 17 86 8.1. Path Establishment . . . . . . . . . . . . . . . . . . . 17 87 8.2. Path Closure . . . . . . . . . . . . . . . . . . . . . . 18 88 9. Implementation Considerations . . . . . . . . . . . . . . . . 19 89 10. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 19 90 10.1. PATH_ABANDON Frame . . . . . . . . . . . . . . . . . . . 19 91 10.2. ACK_MP Frame . . . . . . . . . . . . . . . . . . . . . . 21 92 11. Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . 22 93 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 94 13. Security Considerations . . . . . . . . . . . . . . . . . . . 23 95 14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 23 96 15. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 97 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 98 16.1. Normative References . . . . . . . . . . . . . . . . . . 23 99 16.2. Informative References . . . . . . . . . . . . . . . . . 24 100 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 102 1. Introduction 104 This document specifies an extension to QUIC v1 [QUIC-TRANSPORT] to 105 enable the simultaneous usage of multiple paths for a single 106 connection. 108 This proposal is based on several basic design points: 110 * Re-use as much as possible mechanisms of QUIC-v1. In particular 111 this proposal uses path validation as specified for QUIC v1 and 112 aims to re-use as much as possible of QUIC's connection migration. 114 * Use the same packet header formats as QUIC v1 to avoid the risk of 115 packets being dropped by middleboxes (which may only support QUIC 116 v1) 118 * Congestion Control, RTT measurements and PMTU discovery should be 119 per-path (following [QUIC-TRANSPORT]) 121 * A path is determined by the 4-tuple of source and destination IP 122 address as well as source and destination port. Therefore there 123 can be at most one active paths/connection ID per 4-tuple. 125 The path management specified in section 9 of [QUIC-TRANSPORT] 126 fulfills multiple goals: it directs a peer to switch sending through 127 a new preferred path, and it allows the peer to release resources 128 associated with the old path. Multipath requires several changes to 129 that mechanism: 131 * Allow simultaneous transmission of non probing frames on multiple 132 paths. 134 * Continue using an existing path even if non-probing frames have 135 been received on another path. 137 * Manage the removal of paths that have been abandoned. 139 As such this extension specifies a departure from the specification 140 of path management in section 9 of [QUIC-TRANSPORT] and therefore 141 requires negotiation between the two endpoints using a new transport 142 parameter, as specified in Section 2. 144 This proposal supports the negotiation of either the use of one 145 packet number space for all paths or the use of separate packet 146 number spaces per path. While separate packet number spaces allow 147 for more efficient ACK encoding, especially when paths have highly 148 different latencies, this approach requires the use of a connection 149 ID. Therefore use of a single number space can be beneficial in 150 highly constrained networks that do not benefit from exposing the 151 connection ID in the header. While both approaches are supported by 152 the specification in this version of the document, the intention for 153 the final publication of a multipath extension for QUIC is to choose 154 one option in order to avoid incompatibility. More evaluation and 155 implementation experience is needed to select one approach before 156 final publication. Some discussion about pros and cons can be found 157 here: https://github.com/mirjak/draft-lmbdhk-quic- 158 multipath/blob/master/presentations/PacketNumberSpace_s.pdf 160 As currently defined in this version of the draft the use of multiple 161 packet number spaces requires the use of connection IDs is both 162 directions. Today's deployments often only use destination 163 connection ID when sending packets from the client to the server as 164 this addresses the most important use cases for migration, like NAT 165 rebinding or mobility events. Further discussion and work is 166 required to evaluate if the use of multiple packet number spaces 167 could be supported as well when the connection ID is only present in 168 one direction. 170 This proposal does not cover address discovery and management. 171 Addresses and the actual decision process to setup or tear down paths 172 are assumed to be handled by the application that is using the QUIC 173 multipath extension. Further, this proposal only specifies a simple 174 basic packet scheduling algorithm in order to provide some basic 175 implementation guidance. However, more advanced algorithms as well 176 as potential extensions to enhance signaling of the current path 177 state are expected as future work. 179 1.1. Conventions and Definitions 181 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 182 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 183 "OPTIONAL" in this document are to be interpreted as described in BCP 184 14 [RFC2119] [RFC8174] when, and only when, they appear in all 185 capitals, as shown here. 187 We assume that the reader is familiar with the terminology used in 188 [QUIC-TRANSPORT]. In addition, we define the following terms: 190 * Path Identifier (Path ID): An identifier that is used to identify 191 a path in a QUIC connection at an endpoint. Path Identifier is 192 used in multi-path control frames (etc. PATH_ABANDON frame) to 193 identify a path. By default, it is defined as the sequence number 194 of the destination Connection ID used for sending packets on that 195 particular path, but alternative definitions can be used if the 196 length of that connection ID is zero. 198 * Packet Number Space Identifier (PN Space ID): An identifier that 199 is used to distinguish packet number spaces for different paths. 200 It is used in 1-RTT packets and ACK_MP frames. Each node 201 maintains a list of "Received Packets" for each of the CID that it 202 provided to the peer, which is used for acknowledging packets 203 received with that CID. 205 The difference between Path Identifier and Packet Number Space 206 Identifier, is that the Path Identifier is used in multipath control 207 frames to identify a path, and the Packet Number Space Identifier is 208 used in 1-RTT packets and ACK_MP frames to distinguish packet number 209 spaces for different paths. Both identifiers have the same value, 210 which is the sequence number of the connection ID, if a non-zero 211 connection ID is used. If the connection ID is zero length, the 212 Packet Number Space Identifier is 0, while the Path Identifier is 213 selected on path establishment. 215 2. Handshake Negotiation and Transport Parameter 217 This extension defines a new transport parameter, used to negotiate 218 the use of the multipath extension during the connection handshake, 219 as specified in [QUIC-TRANSPORT]. The new transport parameter is 220 defined as follow: 222 * name: enable_multipath (TBD - experiments use 0xbabf) 224 * value: 0 (default) for disabled. Endpoints use 2-bits in the 225 value field for negotiating one or more PN spaces, available 226 option value for client and server are listed in Table 1 : 228 +===============+===========================+================+ 229 | Client Option | Definition | Allowed server | 230 | | | responses | 231 +===============+===========================+================+ 232 | 0x0 | don't support multi-path | 0x0 | 233 +---------------+---------------------------+----------------+ 234 | 0x1 | only support one PN space | 0x0 or 0x1 | 235 | | for multi-path | | 236 +---------------+---------------------------+----------------+ 237 | 0x2 | only support multiple PN | 0x0 or 0x2 | 238 | | spaces for multi-path | | 239 +---------------+---------------------------+----------------+ 240 | 0x3 | support both one PN space | 0x0, 0x1 or | 241 | | and multiple PN space | 0x2 | 242 +---------------+---------------------------+----------------+ 244 Table 1: Available value for enable_multipath 246 If the peer does not carry the enable_multipath transport parameter, 247 which means the peer does not support multipath, endpoint MUST 248 fallback to [QUIC-TRANSPORT] with single path and MUST NOT use any 249 frame or mechanism defined in this document. If endpoint receives 250 unexpected value for the transport parameter "enable_multipath", it 251 MUST treat this as a connection error of type MP_CONNECTION_ERROR and 252 close the connection. 254 Note that the transport parameter "active_connection_id_limit" 255 [QUIC-TRANSPORT] limits the number of usable Connection IDs, and also 256 limits the number of concurrent paths. For the QUIC multipath 257 extension this limit even applies when no connection ID is exposed in 258 the QUIC header. 260 3. Path Setup and Removal 262 After completing the handshake, endpoints have agreed to enable 263 multipath feature and can start using multiple paths. This document 264 does not discuss when a client decides to initiate a new path. We 265 delegate such discussion in separate documents. 267 This proposal adds one multi-path control frame for path management: 269 * PATH_ABANDON frame for the receiver side to abandon the path 270 Section 10.1 272 All the new frames are sent in 1-RTT packets [QUIC-TRANSPORT]. 274 3.1. Path Initiation 276 When the multipath option is negotiated, clients that want to use an 277 additional path MUST first initiate the Address Validation procedure 278 with PATH_CHALLENGE and PATH_RESPONSE frames described in Section 8 279 of [QUIC-TRANSPORT]. After receiving packets from the client on the 280 new paths, the servers MAY in turn attempt to validate these paths 281 using the same mechanisms. 283 If validation succeed, the client can send non-probing, 1-RTT packets 284 on the new paths. In contrast with the specification in section 9 of 285 [QUIC-TRANSPORT], the server MUST NOT assume that receiving non- 286 probing packets on a new path indicates an attempt to migrate to that 287 path. Instead, servers SHOULD consider new paths over which non- 288 probing packets have been received as available for transmission. 290 3.2. Path Close 292 Each endpoint manages the set of paths that are available for 293 transmission. At any time in the connection, each endpoint can 294 decide to abandon one of these paths, following for example changes 295 in local connectivity or changes in local preferences. After an 296 endpoint abandons a path, the peer will not receive any more non- 297 probing packets on that path. 299 An endpoint that wants to close a path SHOULD NOT rely on implicit 300 signals like idle time or packet losses, but instead SHOULD use 301 explicit request to terminate path by sending the PATH_ABANDON frame 302 (see Section 10.1). 304 3.2.1. Use PATH_ABANDON Frame to Close a Path 306 Both endpoints, namely the client and the server, can close a path, 307 by sending PATH_ABANDON frame (see Section 10.1) which abandons the 308 path with a corresponding Path Identifier. Once a path is marked as 309 "abandoned", it means that the resources related to the path, such as 310 the used connection IDs, can be released. However, information 311 related to data delivered over that path SHOULD not be released 312 immediately as acknowledgments can still be received or other frames 313 that also may trigger retransmission of data on another path. 315 The endpoint sending the PATH_ABANDON frame SHOULD consider a path as 316 abandoned when the packet that contained the PATH_ABANDON frame is 317 acknowledged. When releasing resources of a path, the endpoint 318 SHOULD send a RETIRE_CONNECTION_ID frame for the connection IDs used 319 on the path, if any. 321 The receiver of a PATH_ABANDON frame SHOULD NOT release its resources 322 immediately but SHOULD wait for the receive of the 323 RETIRE_CONNECTION_ID frame for the used connection IDs or 3 RTOs. 325 Usually it is expected that the PATH_ABANDON frame is used by the 326 client to indicate to the server that path conditions have changed 327 such that the path is or will be not usable anymore, e.g. in case of 328 an mobility event. The PATH_ABANDON frame therefore indicates to the 329 receiving peer that the sender does not intend to send any packets on 330 that path anymore but also recommends to the receiver that no packets 331 should be sent in either direction. The receiver of an PATH_ABANDON 332 frame MAY also send an PATH_ABANDON frame to signal its own 333 willingness to not send any packet on this path anymore. 335 If connection IDs are used, PATH_ABANDON frames can be sent on any 336 path, not only the path that is intended to be closed. Thus a 337 connection can be abandoned even if connectivity on that path is 338 already broken. If no connection IDs are used and the PATH_ABANDON 339 frame has to sent on the path that is intended to be closed, it is 340 possible that the packet containing the PATH_ABANDON frame or the 341 packet containing the ACK for the PATH_ABANDON frame cannot be 342 received anymore and the endpoint might need to rely on an idle time 343 out to close the path, as described in Section Section 3.2.3. 345 Retransmittable frames, that have previously been send on the 346 abandoned path and are considered lost, SHOULD be retransmitted on a 347 different path. 349 If a PATH_ABANDON frame is received for the only active path of a 350 QUIC connection, the receiving peer SHOULD send a CONNECTION_CLOSE 351 frame and enters the closing state. If the client received a 352 PATH_ABANDON frame for the last open path, it MAY instead try to open 353 a new path, if available, and only initiate connection closure if 354 path validation fails or a CONNECTION_CLOSE frame is received from 355 the server. Similarly the server MAY wait for a short, limited time 356 such as one RTO if a path probing packet is received on a new path 357 before sending the CONNECTION_CLOSE frame. 359 3.2.2. Effect of RETIRE_CONNECTION_ID Frame 361 Receiving a RETIRE_CONNECTION_ID frame causes the endpoint to discard 362 the resources associated with that connection ID. If the connection 363 ID was used by the peer to identify a path from the peer to this 364 endpoint, the resources include the list of received packets used to 365 send acknowledgements. The peer MAY decide to keep sending data 366 using the same IP addresses and UDP ports previously associated with 367 the connection ID, but MUST use a different connection ID when doing 368 so. 370 3.2.3. Idle Timeout 372 [QUIC-TRANSPORT] allows for closing of connections if they stay idle 373 for too long. The connection idle timeout in multipath QUIC is 374 defined as "no packet received on any path for the duration of the 375 idle timeout". When only one path is available, servers MUST follow 376 the specifications in [QUIC-TRANSPORT]. 378 When more than one path is available, servers shall monitor the 379 arrival of non-probing packets on the available paths. Servers 380 SHOULD stop sending traffic on paths through where no non-probing 381 packet was received in the last 3 path RTTs, but MAY ignore that rule 382 if it would disqualify all available paths. Server MAY release the 383 resource associated with paths for which no non-probing packet was 384 received for a sufficiently long path-idle delay, but SHOULD only 385 release resource for the last available path if no traffic is 386 received for the duration of the idle timeout, as specified in 387 section 10.1 of [QUIC-TRANSPORT]. This means if all paths remain 388 idle for the idle timeout, the connection is implicitly closed. 390 Server implementations need to select the sub-path idle timeout as a 391 trade- off between keeping resources, such as connection IDs, in use 392 for an excessive time or having to promptly reestablish a path after 393 a spurious estimate of path abandonment by the client. 395 3.3. Path States 397 Figure 1 shows the states that an endpoint's path can have. 399 o 400 | PATH_CHALLENGE sent/received on new path 401 v 402 +------------+ Path validation abandoned 403 | Validating |----------------------------------+ 404 +------------+ | 405 | | 406 | PATH_RESPONSE received | 407 | | 408 v Associated CID have been retired | 409 +------------+ Path's idle timeout | 410 | Active |----------------------------------+ 411 +------------+ | 412 | | 413 | PATH_ABANDONED sent/received | 414 v | 415 +------------+ | 416 | Closing | | 417 +------------+ | 418 | | 419 | Associated CID have been retired | 420 | Path's idle timeout | 421 v | 422 +------------+ | 423 | Closed |<---------------------------------+ 424 +------------+ 426 Figure 1: States of a path 428 In non-final states, hosts have to track the following information. 430 * Associated 4-tuple: The tuple (source IP, source port, destination 431 IP, destination port) used by the endhost to send packets over the 432 path. 434 * Associated Destination Connection ID: The Connection ID used to 435 send packets over the path. 437 If multiple packet number spaces are used over the connection, hosts 438 MUST also track the following information. 440 * Path Packet Number Space: The endpoint maintains a separate packet 441 number for sending and receiving packets over this path. Packet 442 number considerations described in [QUIC-TRANSPORT] apply within 443 the given path. 445 In the "Active" state, hosts MUST also track the following 446 information. 448 * Associated Source Connection ID: The Connection ID used to receive 449 packets over the path. 451 A path in the "Validating" state performs path validation as 452 described in Section 8.2 of [QUIC-TRANSPORT]. An endhost should not 453 send non-probing frames on a path in "Validating" state, as it has no 454 guarantee that packets will actually reach the peer. 456 The endhost can use all the paths in the "Active" state, provided 457 that the congestion control and flow control currently allow sending 458 of new data on a path. 460 In the "Closing" state, the endhost SHOULD NOT send packets on this 461 path anymore, as there is no guarantee that the peer can still map 462 the packets to the connection. The endhost SHOULD wait for the 463 acknowledgment of the PATH_ABANDONED frame before moving the path to 464 the "Closed" state to ensure a graceful termination of the path. 466 When a path reaches the "Closed" state, the endhost releases all the 467 path's associated resources. Consequently, the endhost is not able 468 to send nor receive packets on this path anymore. 470 4. Congestion Control 472 Senders MUST manage per-path congestion status, and MUST NOT send 473 more data on a given path than congestion control on that path 474 allows. This is already a requirement of [QUIC-TRANSPORT]. 476 When a Multipath QUIC connection uses two or more paths, there is no 477 guarantee that these paths are fully disjoint. When two (or more 478 paths) share the same bottleneck, using a standard congestion control 479 scheme could result in an unfair distribution of the bandwidth with 480 the multipath connection getting more bandwidth than competing single 481 paths connections. Multipath TCP uses the LIA congestion control 482 scheme specified in [RFC6356] to solve this problem. This scheme can 483 immediately be adapted to Multipath QUIC. Other coupled congestion 484 control schemes have been proposed for Multipath TCP such as [OLIA]. 486 5. Computing Path RTT 488 Acknowledgement delays are the sum of two one-way delays, the delay 489 on the packet sending path and the delay on the return path chosen 490 for the acknowledgements. When different paths have different 491 characteristics, this can cause acknowledgement delays to vary 492 widely. Consider for example multipath transmission using both a 493 terrestrial path, with a latency of 50ms in each direction, and a 494 geostationary satellite path, with a latency of 300ms in both 495 directions. The acknowledgement delay will depend on the combination 496 of paths used for the packet transmission and the ACK transmission, 497 as shown in Table 2. 499 +======================+=============+===========+ 500 | ACK Path \ Data path | Terrestrial | Satellite | 501 +======================+=============+===========+ 502 | Terrestrial | 100ms | 350ms | 503 +----------------------+-------------+-----------+ 504 | Satellite | 350ms | 600ms | 505 +----------------------+-------------+-----------+ 507 Table 2: Example of ACK delays using multiple 508 paths 510 Using the default algorithm specified in [QUIC-RECOVERY] would result 511 in suboptimal performance, computing average RTT and standard 512 deviation from series of different delay measurements of different 513 combined paths. At the same time, early tests showed that it is 514 desirable to send ACKs through the shortest path, because a shorter 515 ACK delay results in a tighter control loop and better performances. 516 The tests also showed that it is desirable to send copies of the ACKs 517 on multiple paths, for robustness if a path experiences sudden 518 losses. 520 An early implementation mitigated the delay variation issue by using 521 time stamps, as specified in [QUIC-Timestamp]. When the timestamps 522 are present, the implementation can estimate the transmission delay 523 on each one-way path, and can then use these one way delays for more 524 efficient implementations of recovery and congestion control 525 algorithms. 527 If timestamps are not available, implementations could estimate one 528 way delays using statistical techniques. For example, in the example 529 shown in Table 1, implementations can use use "same path" 530 measurements to estimate the one way delay of the terrestrial path to 531 about 50ms in each direction, and that of the satellite path to about 532 300ms. Further measurements can then be used to maintain estimates 533 of one way delay variations, using logical similar to Kalman filters. 534 But statistical processing is error-prone, and using time stamps 535 provides more robust measurements. 537 6. Packet Scheduling 539 The transmission of QUIC packets on a regular QUIC connection is 540 regulated by the arrival of data from the application and the 541 congestion control scheme. QUIC packets can only be sent when the 542 congestion window of at least one path is open. 544 Multipath QUIC implementations also need to include a packet 545 scheduler that decides, among the paths whose congestion window is 546 open, the path over which the next QUIC packet will be sent. Many 547 factors can influence the definition of these algorithms and their 548 precise definition is outside the scope of this document. Various 549 packet schedulers have been proposed and implemented, notably for 550 Multipath TCP. A companion draft [I-D.bonaventure-iccrg-schedulers] 551 provides several general-purpose packet schedulers depending on the 552 application goals. 554 7. Packet Number Space and Use of Connection ID 556 If the connection ID is present (non-zero length) in the packet 557 header, the connection ID is used to identify the path. If no 558 connection ID is present, the 4 tuple identifies the path. The 559 initial path that is used during the handshake (and multipath 560 negotiation) has the path ID 0 and therefore all 0-RTT packets are 561 also tracked and processed with the path ID 0. For 1-RTT packets the 562 path ID is the sequence number of the Destination Connection ID 563 present in the packet header, as defined in Section 5.1.1 of 564 [QUIC-TRANSPORT], or also 0 if the Connection ID is zero-length. 566 If non-zero-length Connection IDs are used, an endpoint MUST use 567 different Connection IDs on different paths. Still, the receiver may 568 observe the same Connection ID used on different 4-tuples due to, 569 e.g., NAT rebinding. In such case, the receiver reacts as specified 570 in Section 9.3 of [QUIC-TRANSPORT]. 572 Acknowledgements of Initial and Handshake packets MUST be carried 573 using ACK frames, as specified in [QUIC-TRANSPORT]. The ACK frames, 574 as defined in [QUIC-TRANSPORT], do not carry path identifiers. If 575 for any reason ACK frames are received in 1-RTT packets while the 576 state of multipath negotiation is ambiguous, they MUST be interpreted 577 as acknowledging packets sent on path 0. 579 7.1. Using One Packet Number Space 581 If the multipath option is negotiated to use one packet number space 582 for all paths, the packet sequence numbers are allocated from the 583 common number space, so that, for example, packet number N could be 584 sent on one path and packet number N+1 on another. 586 ACK frames report the numbers of packets that have been received so 587 far, regardless of the path on which they have been received. That 588 means the senders needs to maintain an association between sent 589 packet numbers and the path over which these packets were sent. This 590 is necessary to implement per path congestion control. 592 When a packet is acknowledged, the state of the congestion control 593 MUST be updated for the path where the acknowledged packet was 594 originally sent. The RTT is calculated based on the delay between 595 the transmission of that packet and its first acknowledgement (see 596 Section 5) and is used to update the RTT statistics for the sending 597 path. 599 Also loss detection MUST be adapted to allow for different RTTs on 600 different paths. For example, timer computations should take into 601 account the RTT of the path on which a packet was sent. Detections 602 based on packet numbers shall compare a given packet number to the 603 highest packet number received for that path. 605 7.1.1. Sending Acknowledgements and Handling Ranges 607 If senders decide to send packets on paths with different 608 transmission delays, some packets will very likely be received out of 609 order. This will cause the ACK frames to carry multiple ranges of 610 received packets. The large number of range increases the size of 611 ACK frames, causing transmission and processing overhead. 613 The size and overhead of the ACK frames can be controlled by the 614 combination of one or several of the following: 616 * Not transmitting again ACK ranges that were present in an ACK 617 frame acknowledged by the peer. 619 * Delay acknowledgements to allow for arrival of "hole filling" 620 packets. 622 * Limit the total number of ranges sent in an ACK frame. 624 * Limiting the number of transmissions of a specific ACK range, on 625 the assumption that a sufficient number of transmissions almost 626 certainly ensures reception by the peer. 628 * Send multiple messages for a given path in a single socket 629 operation, so that a series of packets sent from a single path 630 uses a series of consecutive sequence numbers without creating 631 holes. 633 7.2. Using Multiple Packet Number Spaces 635 If the multipath option is enabled with a value of 2, each path has 636 its own packet number space for transmitting 1-RTT packets and a new 637 ACK frame format is used as specified in Section 10.2. Compared to 638 the QUIC v1 ACK frame, the MP_ACK frames additionally contains a 639 Packet Number Space Identifier (PN Space ID). The PN Space ID used 640 to distinguish packet number spaces for different paths and is simply 641 derived from the sequence number of Destination Connection ID. 642 Therefore, the packet number space for 1-RTT packets can be 643 identified based on the Destination Connection ID in each packets. 645 As soon as the negotiation of multipath support with value 2 is 646 completed, endpoints SHOULD use ACK_MP frames instead of ACK frames 647 for acknowledgements of 1-RTT packets on path 0, as well as for 0-RTT 648 packets that are acknowledged after the handshake concluded. 650 Following [QUIC-TRANSPORT], each endpoint uses NEW_CONNECTION_ID 651 frames to issue usable connections IDs to reach it. Before an 652 endpoint adds a new path by initiating path validation, it MUST check 653 whether at least one unused Connection ID is available for each side. 655 If the transport parameter "active_connection_id_limit" is negotiated 656 as N, the server provided N Connection IDs, and the client is already 657 actively using N paths, the limit is reached. If the client wants to 658 start a new path, it has to retire one of the established paths. 660 ACK_MP frame Section 10.2 can be returned via either a different 661 path, or the same path identified by the Path Identifier, based on 662 different strategies of sending ACK_MP frames. 664 Using multiple packet number spaces requires changes in the way AEAD 665 is applied for packet protection, as explained in Section 7.2.1, and 666 tighter constraints for key updates, as explained in Section 7.2.2. 668 7.2.1. Packet Protection for QUIC Multipath 670 Packet protection for QUIC v1 is specified is Section 5 of 671 [QUIC-TLS]. The general principles of packet protection are not 672 changed for QUIC Multipath. No changes are needed for setting packet 673 protection keys, initial secrets, header protection, use of 0-RTT 674 keys, receiving out-of-order protected packets, receiving protected 675 packets, or retry packet integrity. However, the use of multiple 676 number spaces for 1-RTT packets requires changes in AEAD usage. 678 Section 5.3 of [QUIC-TLS] specifies AEAD usage, and in particular the 679 use of a nonce, N, formed by combining the packet protection IV with 680 the packet number. If multiple packet number spaces are used, the 681 packet number alone would not guarantee the uniqueness of the nonce. 683 In order to guarantee the uniqueness of the None, the nonce N is 684 calculated by combining the packet protection IV with the packet 685 number and with the path identifier. 687 The path ID for 1-RTT packets is the sequence number of of 688 [QUIC-TRANSPORT], or zero if the Connection ID is zero-length. 689 Section 19 of [QUIC-TRANSPORT] encodes the Connection ID Sequence 690 Number as a variable-length integer, allowing values up to 2^62-1; in 691 this specification a range of less than 2^32-1 values MUST be used 692 before updating the packet protection key. 694 To calculate the nonce, a 96 bit path-and-packet-number is composed 695 of the 32 bit Connection ID Sequence Number in byte order, two zero 696 bits, and the 62 bits of the reconstructed QUIC packet number in 697 network byte order. If the IV is larger than 96 bits, the path-and- 698 packet-number is left-padded with zeros to the size of the IV. The 699 exclusive OR of the padded packet number and the IV forms the AEAD 700 nonce. 702 For example, assuming the IV value is 6b26114b9cba2b63a9e8dd4f, the 703 connection ID sequence number is 3, and the packet number is aead, 704 the nonce will be set to 6b2611489cba2b63a9e873e2. 706 7.2.2. Key Update for QUIC Multipath 708 The Key Phase bit update process for QUIC v1 is specified in 709 Section 6 of [QUIC-TLS]. The general principles of key update are 710 not changed in this specification. Following QUIC v1, the Key Phase 711 bit is used to indicate which packet protection keys are used to 712 protect the packet. The Key Phase bit is toggled to signal each 713 subsequent key update. Because of network delays, packets protected 714 with the older key might arrive later than the packets protected with 715 the new key. Therefore, the endpoint needs to retain old packet keys 716 to allow these delayed packets to be processed and it must 717 distinguish between the new key and the old key. In QUIC V1, this is 718 done using packet numbers so that the rule is made simple: Use the 719 older key if packet number is lower than any packet number frame the 720 current key phase. 722 When using multiple packet number spaces on different paths, some 723 care is needed when initiating the Key Update process, as different 724 paths use different packet number spaces but share a single key. 725 When a key update is initiated on one path, packets sent to another 726 path needs to know when the transition is complete. Otherwise, it is 727 possible that the other paths send packets with the old keys, but 728 skip sending any packets in the current key phase and directly jump 729 to sending packet in the next key phase. When that happens, as the 730 endpoint can only retain two sets of packet protection keys with the 731 1-bit Key Phase bit, the other paths cannot distinguish which key 732 should be used to decode received packets, which results in a key 733 rotation synchronization problem. 735 To address such a synchronization issue, if key update is initialized 736 on one path, the sender SHOULD send at least one packet with the new 737 key on all active paths. Further, an endpoint MUST NOT initiate a 738 subsequent key update until a packet with the current key has been 739 acknowledged on each path. 741 Following Section 5.4 of [QUIC-TLS], the Key Phase bit is protected, 742 so sending multiple packets with Key Phase bit flipping at the same 743 time should not cause linkability issue. 745 8. Examples 747 8.1. Path Establishment 749 Figure 2 illustrates an example of new path establishment using 750 multiple packet number spaces. 752 Client Server 754 (Exchanges start on default path) 755 1-RTT[]: NEW_CONNECTION_ID[C1, Seq=1] --> 756 <-- 1-RTT[]: NEW_CONNECTION_ID[S1, Seq=1] 757 <-- 1-RTT[]: NEW_CONNECTION_ID[S2, Seq=2] 758 ... 759 (starts new path) 760 1-RTT[0]: DCID=S2, PATH_CHALLENGE[X] --> 761 Checks AEAD using nonce(CID sequence 2, PN 0) 762 <-- 1-RTT[0]: DCID=C1, PATH_RESPONSE[X], PATH_CHALLENGE[Y], 763 ACK_MP[Seq=2,PN=0] 764 Checks AEAD using nonce(CID sequence 1, PN 0) 765 1-RTT[1]: DCID=S2, PATH_RESPONSE[Y], 766 ACK_MP[Seq=1, PN=0], ... --> 768 Figure 2: Example of new path establishment 770 In Figure Figure 2, the endpoints first exchange new available 771 Connection IDs with the NEW_CONNECTION_ID frame. In this example the 772 client provides one Connection ID (C1 with sequence number 1), and 773 server provides two Connection IDs (S1 with sequence number 1, and S2 774 with sequence number 2). 776 Before the client opens a new path by sending an packet on that path 777 with a PATH_CHALLENGE frame, it has to check. whether there is an 778 unused Connection IDs available for each side. In this example the 779 client chooses the Connection ID S2 as the Destination Connection ID 780 in the new path. 782 If the client has used all the allocated CID, it is supposed to 783 retire those that are not used anymore, and the server is supposed to 784 provide replacements, as specified in [QUIC-TRANSPORT]. Usually it 785 is desired to provide one more connection ID as currently in used, to 786 allow for new paths or migration. 788 8.2. Path Closure 790 In this example the client detects the network environment change 791 (client's 4G/Wi-Fi is turned off, Wi-Fi signal is fading to a 792 threshold, or the quality of RTT or loss rate is becoming worse) and 793 wants to close the initial path. 795 In Figure Figure 3 the server's 1-RTT packets use DCID C1, which has 796 a sequence number of 1, for the first path; the client's 1-RTT 797 packets use DCID S2, which has a sequence number of 2. For the 798 second path, the server's 1-RTT packets use DCID C2, which has a 799 sequence number of 2; the client's 1-RTT packets use CID S3, which 800 has a sequence number of 3. Note that two paths use different packet 801 number space. 803 Thee client initiates the path closure for the path with ID 1 by 804 sending a packet with an PATH_ABANDON frame. When the server 805 received the PATH_ABANDON frame, it also sends an PATH_ABANDON frame 806 in the next packet. Afterwards the connection IDs in both directions 807 can be retired using the RETIRE_CONNECTION_ID frame. 809 Client Server 811 (client tells server to abandon a path) 812 1-RTT[X]: DCID=S2 PATH_ABANDON[path_id=1]-> 813 (server tells client to abandon a path) 814 <-1-RTT[Y]: DCID=C1 PATH_ABANDON[path_id=2], ACK_MP[Seq=2, PN=X] 815 (client abandons the path that it is using) 816 1-RTT[U]: DCID=S3 RETIRE_CONNECTION_ID[2], ACK_MP[Seq=1, PN=Y] -> 817 (server abandons the path that it is using) 818 <- 1-RTT[V]: DCID=C2 RETIRE_CONNECTION_ID[1], ACK_MP[Seq=3, PN=U] 820 Figure 3: Example of closing a path (path id type=0x00) 822 9. Implementation Considerations 824 TDB 826 10. New Frames 828 All the new frames MUST only be sent in 1-RTT packet, and MUST NOT 829 use other encryption levels. 831 If an endpoint receives multipath-specific frames from packets of 832 other encryption levels, it MUST return MP_PROTOCOL_VIOLATION as a 833 connection error and close the connection. 835 10.1. PATH_ABANDON Frame 837 The PATH_ABANDON frame informs the peer to abandon a path. More 838 complex path management can be made possible with additional 839 extensions (e.g., PATH_STATUS frame in [I-D.liu-multipath-quic] ). 841 PATH_ABANDON frames are formatted as shown in Figure 4. 843 PATH_ABANDON Frame { 844 Type (i) = TBD-03 (experiments use 0xbaba05), 845 Path Identifier (..), 846 Error Code (i), 847 Reason Phrase Length (i), 848 Reason Phrase (..), 849 } 851 Figure 4: PATH_ABANDON Frame Format 853 PATH_ABANDON frames contain the following fields: 855 Path Identifier: An identifier of the path, which is formatted as 856 shown in Figure 5. 858 * Identifier Type: Identifier Type field is set to indicate the type 859 of path identifier. 861 - Type 0: Refer to the connection identifier used by the sender 862 of the control frame when sending data over the specified path. 863 This method SHOULD be used if this connection identifier is 864 non-zero length. This method MUST NOT be used if this 865 connection identifier is zero-length. 867 - Type 1: Refer to the connection identifier used by the receiver 868 of the control frame when sending data over the specified path. 869 This method MUST NOT be used if this connection identifier is 870 zero-length. 872 - Type 2: Refer to the path over which the control frame is sent 873 or received. 875 * Path Identifier Content: A variable-length integer specifying the 876 path identifier. If Identifier Type is 2, the Path Identifier 877 Content MUST be empty. 879 Path Identifier { 880 Identifier Type (i) = 0x00..0x02, 881 [Path Identifier Content (i)], 882 } 884 Figure 5: Path Identifier Format 886 Note: If the receiver of the PATH_ABANDON frame is using non-zero 887 length Connection ID on that path, endpoint SHOULD use type 0x00 for 888 path identifier in the control frame. If the receiver of the 889 PATH_ABANDON frame is using zero-length Connection ID, but the peer 890 is using non-zero length Connection ID on that path, endpoints SHOULD 891 use type 0x01 for path identifier. If both endpoints are using 892 0-length Connection IDs on that path, endpoints SHOULD only use type 893 0x02 for path identifier. 895 Error Code: A variable-length integer that indicates the reason for 896 abandoning this path. 898 Reason Phrase Length: A variable-length integer specifying the 899 length of the reason phrase in bytes. Because an PATH_ABANDON 900 frame cannot be split between packets, any limits on packet size 901 will also limit the space available for a reason phrase. 903 Reason Phrase: Additional diagnostic information for the closure. 905 This can be zero length if the sender chooses not to give details 906 beyond the Error Code value. This SHOULD be a UTF-8 encoded 907 string [RFC3629], though the frame does not carry information, 908 such as language tags, that would aid comprehension by any entity 909 other than the one that created the text. 911 PATH_ABANDON frames SHOULD be acknowledged. If a packet containing a 912 PATH_ABANDON frame is considered lost, the peer SHOULD repeat it. 914 If the Identifier Type is 0x00 or 0x01, PATH_ABANDON frames MAY be 915 sent on any path, not only the path identified by the Path Identifier 916 Content field. If the Identifier Type if 0x02, the PATH_ABANDON 917 frame MUST only be sent on the path that is intended to be abandoned. 919 10.2. ACK_MP Frame 921 The ACK_MP frame (types TBD-00 and TBD-01; experiments use 922 0xbaba00..0xbaba01) is an extension of the ACK frame defined by 923 [QUIC-TRANSPORT]. It is used to acknowledge packets that were sent 924 on different paths when using multiple packet number spaces. If the 925 frame type is TBD-01, ACK_MP frames also contain the sum of QUIC 926 packets with associated ECN marks received on the connection up to 927 this point. 929 ACK_MP frame is formatted as shown in Figure 6. 931 ACK_MP Frame { 932 Type (i) = TBD-00..TBD-01 (experiments use 0xbaba00..0xbaba01), 933 Packet Number Space Identifier (i), 934 Largest Acknowledged (i), 935 ACK Delay (i), 936 ACK Range Count (i), 937 First ACK Range (i), 938 ACK Range (..) ..., 939 [ECN Counts (..)], 940 } 942 Figure 6: ACK_MP Frame Format 944 Compared to the ACK frame specified in [QUIC-TRANSPORT], the 945 following field is added. 947 Packet Number Space Identifier: An identifier of the path packet 948 number space, which is the sequence number of Destination Connection 949 ID of the 1-RTT packets which are acknowledged by the ACK_MP frame. 950 If the endpoint receives 1-RTT packets with zero-length Connection 951 ID, it SHOULD use Packet Number Space Identifier 0 in ACK_MP frames. 952 If an endpoint receives a ACK_MP frame with a non-existing packet 953 number space ID, it MUST treat this as a connection error of type 954 MP_PROTOCOL_VIOLATION and close the connection. 956 When using a single packet number space, endhosts MUST NOT send 957 ACK_MP frames. If an endhost receives an ACK_MP frame while a single 958 packet number space was negotiated, it MUST treat this as a 959 connection error of type MP_PROTOCOL_VIOLATION and close the 960 connection. 962 11. Error Codes 964 Multi-path QUIC transport error codes are 62-bit unsigned integers 965 following [QUIC-TRANSPORT]. 967 This section lists the defined multipath QUIC transport error codes 968 that can be used in a CONNECTION_CLOSE frame with a type of 0x1c. 969 These errors apply to the entire connection. 971 MP_PROTOCOL_VIOLATION (experiments use 0xba01): An endpoint detected 972 an error with protocol compliance that was not covered by more 973 specific error codes. 975 12. IANA Considerations 977 This document defines a new transport parameter for the negotiation 978 of enable multiple paths for QUIC, and two new frame types. The 979 draft defines provisional values for experiments, but we expect IANA 980 to allocate short values if the draft is approved. 982 The following entry in Table 3 should be added to the "QUIC Transport 983 Parameters" registry under the "QUIC Protocol" heading. 985 +==============================+==================+===============+ 986 | Value | Parameter Name. | Specification | 987 +==============================+==================+===============+ 988 | TBD (experiments use 0xbabf) | enable_multipath | Section 2 | 989 +------------------------------+------------------+---------------+ 991 Table 3: Addition to QUIC Transport Parameters Entries 993 The following frame types defined in Table 4 should be added to the 994 "QUIC Frame Types" registry under the "QUIC Protocol" heading. 996 +==============================+==============+===============+ 997 | Value | Frame Name | Specification | 998 +==============================+==============+===============+ 999 | TBD-00 - TBD-01 (experiments | ACK_MP | Section 10.2 | 1000 | use 0xbaba00-0xbaba01) | | | 1001 +------------------------------+--------------+---------------+ 1002 | TBD-02 (experiments use | PATH_ABANDON | Section 10.1 | 1003 | 0xbaba05) | | | 1004 +------------------------------+--------------+---------------+ 1006 Table 4: Addition to QUIC Frame Types Entries 1008 The following transport error code defined in Table 5 should be added 1009 to the "QUIC Transport Error Codes" registry under the "QUIC 1010 Protocol" heading. 1012 +==============+=======================+============+===============+ 1013 | Value | Code |Description | Specification | 1014 +==============+=======================+============+===============+ 1015 | TBD | MP_PROTOCOL_VIOLATION |Multi-path | Section 11 | 1016 | (experiments | |protocol | | 1017 | use 0xba01) | |violation | | 1018 +--------------+-----------------------+------------+---------------+ 1020 Table 5: Error Code for Multi-path QUIC 1022 13. Security Considerations 1024 TBD 1026 14. Contributors 1028 This document is a collaboration of authors that combines work from 1029 three proposals. Further contributors that were also involved one of 1030 the original proposals are: 1032 * Qing An 1034 * Zhenyu Li 1036 15. Acknowledgments 1038 TBD 1040 16. References 1042 16.1. Normative References 1044 [QUIC-TLS] Thomson, M., Ed. and S. Turner, Ed., "Using TLS to Secure 1045 QUIC", RFC 9001, DOI 10.17487/RFC9001, May 2021, 1046 . 1048 [QUIC-TRANSPORT] 1049 Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 1050 Multiplexed and Secure Transport", RFC 9000, 1051 DOI 10.17487/RFC9000, May 2021, 1052 . 1054 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1055 Requirement Levels", BCP 14, RFC 2119, 1056 DOI 10.17487/RFC2119, March 1997, 1057 . 1059 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 1060 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 1061 2003, . 1063 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1064 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1065 May 2017, . 1067 16.2. Informative References 1069 [I-D.bonaventure-iccrg-schedulers] 1070 Bonaventure, O., Piraux, M., Coninck, Q. D., Baerts, M., 1071 Paasch, C., and M. Amend, "Multipath schedulers", Work in 1072 Progress, Internet-Draft, draft-bonaventure-iccrg- 1073 schedulers-02, 25 October 2021, 1074 . 1077 [I-D.liu-multipath-quic] 1078 Liu, Y., Ma, Y., Huitema, C., An, Q., and Z. Li, 1079 "Multipath Extension for QUIC", Work in Progress, 1080 Internet-Draft, draft-liu-multipath-quic-04, 5 September 1081 2021, . 1084 [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J.- 1085 Y. Le Boudec, "MPTCP is not pareto-optimal: performance 1086 issues and a possible solution", Proceedings of the 8th 1087 international conference on Emerging networking 1088 experiments and technologies, ACM , 2012. 1090 [QUIC-Invariants] 1091 Thomson, M., "Version-Independent Properties of QUIC", 1092 RFC 8999, DOI 10.17487/RFC8999, May 2021, 1093 . 1095 [QUIC-RECOVERY] 1096 Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection 1097 and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, 1098 May 2021, . 1100 [QUIC-Timestamp] 1101 Huitema, C., "Quic Timestamps For Measuring One-Way 1102 Delays", Work in Progress, Internet-Draft, draft-huitema- 1103 quic-ts-06, 12 September 2021, 1104 . 1107 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 1108 Congestion Control for Multipath Transport Protocols", 1109 RFC 6356, DOI 10.17487/RFC6356, October 2011, 1110 . 1112 Authors' Addresses 1114 Yanmei Liu 1115 Alibaba Inc. 1117 Email: miaoji.lym@alibaba-inc.com 1119 Yunfei Ma 1120 Alibaba Inc. 1122 Email: yunfei.ma@alibaba-inc.com 1124 Quentin De Coninck 1125 UCLouvain 1127 Email: quentin.deconinck@uclouvain.be 1129 Olivier Bonaventure 1130 UCLouvain 1132 Email: olivier.bonaventure@uclouvain.be 1133 Christian Huitema 1134 Private Octopus Inc. 1136 Email: huitema@huitema.net 1138 Mirja Kuehlewind (editor) 1139 Ericsson 1141 Email: mirja.kuehlewind@ericsson.com