idnits 2.17.1 draft-ietf-quic-recovery-24.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [3], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 04, 2019) is 1627 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 1008 -- Looks like a reference, but probably isn't: '2' on line 1010 -- Looks like a reference, but probably isn't: '3' on line 1012 == Missing Reference: 'Initial' is mentioned on line 1259, but not defined == Outdated reference: A later version (-34) exists of draft-ietf-quic-tls-24 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-24 == Outdated reference: A later version (-15) exists of draft-ietf-tcpm-rack-05 -- Obsolete informational reference (is this intentional?): RFC 8312 (Obsoleted by RFC 9438) Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC J. Iyengar, Ed. 3 Internet-Draft Fastly 4 Intended status: Standards Track I. Swett, Ed. 5 Expires: May 7, 2020 Google 6 November 04, 2019 8 QUIC Loss Detection and Congestion Control 9 draft-ietf-quic-recovery-24 11 Abstract 13 This document describes loss detection and congestion control 14 mechanisms for QUIC. 16 Note to Readers 18 Discussion of this draft takes place on the QUIC working group 19 mailing list (quic@ietf.org), which is archived at 20 https://mailarchive.ietf.org/arch/search/?email_list=quic [1]. 22 Working Group information can be found at https://github.com/quicwg 23 [2]; source code and issues list for this draft can be found at 24 https://github.com/quicwg/base-drafts/labels/-recovery [3]. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on May 7, 2020. 43 Copyright Notice 45 Copyright (c) 2019 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 61 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 4 62 3. Design of the QUIC Transmission Machinery . . . . . . . . . . 5 63 3.1. Relevant Differences Between QUIC and TCP . . . . . . . . 5 64 3.1.1. Separate Packet Number Spaces . . . . . . . . . . . . 6 65 3.1.2. Monotonically Increasing Packet Numbers . . . . . . . 6 66 3.1.3. Clearer Loss Epoch . . . . . . . . . . . . . . . . . 6 67 3.1.4. No Reneging . . . . . . . . . . . . . . . . . . . . . 7 68 3.1.5. More ACK Ranges . . . . . . . . . . . . . . . . . . . 7 69 3.1.6. Explicit Correction For Delayed Acknowledgements . . 7 70 4. Estimating the Round-Trip Time . . . . . . . . . . . . . . . 7 71 4.1. Generating RTT samples . . . . . . . . . . . . . . . . . 7 72 4.2. Estimating min_rtt . . . . . . . . . . . . . . . . . . . 8 73 4.3. Estimating smoothed_rtt and rttvar . . . . . . . . . . . 8 74 5. Loss Detection . . . . . . . . . . . . . . . . . . . . . . . 9 75 5.1. Acknowledgement-based Detection . . . . . . . . . . . . . 10 76 5.1.1. Packet Threshold . . . . . . . . . . . . . . . . . . 10 77 5.1.2. Time Threshold . . . . . . . . . . . . . . . . . . . 10 78 5.2. Probe Timeout . . . . . . . . . . . . . . . . . . . . . . 11 79 5.2.1. Computing PTO . . . . . . . . . . . . . . . . . . . . 11 80 5.3. Handshakes and New Paths . . . . . . . . . . . . . . . . 12 81 5.3.1. Sending Probe Packets . . . . . . . . . . . . . . . . 13 82 5.3.2. Loss Detection . . . . . . . . . . . . . . . . . . . 14 83 5.4. Handling Retry Packets . . . . . . . . . . . . . . . . . 14 84 5.5. Discarding Keys and Packet State . . . . . . . . . . . . 14 85 6. Congestion Control . . . . . . . . . . . . . . . . . . . . . 15 86 6.1. Explicit Congestion Notification . . . . . . . . . . . . 15 87 6.2. Slow Start . . . . . . . . . . . . . . . . . . . . . . . 16 88 6.3. Congestion Avoidance . . . . . . . . . . . . . . . . . . 16 89 6.4. Recovery Period . . . . . . . . . . . . . . . . . . . . . 16 90 6.5. Ignoring Loss of Undecryptable Packets . . . . . . . . . 16 91 6.6. Probe Timeout . . . . . . . . . . . . . . . . . . . . . . 16 92 6.7. Persistent Congestion . . . . . . . . . . . . . . . . . . 17 93 6.8. Pacing . . . . . . . . . . . . . . . . . . . . . . . . . 18 94 6.9. Under-utilizing the Congestion Window . . . . . . . . . . 18 95 7. Security Considerations . . . . . . . . . . . . . . . . . . . 19 96 7.1. Congestion Signals . . . . . . . . . . . . . . . . . . . 19 97 7.2. Traffic Analysis . . . . . . . . . . . . . . . . . . . . 19 98 7.3. Misreporting ECN Markings . . . . . . . . . . . . . . . . 19 99 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 100 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 101 9.1. Normative References . . . . . . . . . . . . . . . . . . 20 102 9.2. Informative References . . . . . . . . . . . . . . . . . 20 103 9.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 22 104 Appendix A. Loss Recovery Pseudocode . . . . . . . . . . . . . . 22 105 A.1. Tracking Sent Packets . . . . . . . . . . . . . . . . . . 22 106 A.1.1. Sent Packet Fields . . . . . . . . . . . . . . . . . 22 107 A.2. Constants of interest . . . . . . . . . . . . . . . . . . 23 108 A.3. Variables of interest . . . . . . . . . . . . . . . . . . 23 109 A.4. Initialization . . . . . . . . . . . . . . . . . . . . . 24 110 A.5. On Sending a Packet . . . . . . . . . . . . . . . . . . . 24 111 A.6. On Receiving an Acknowledgment . . . . . . . . . . . . . 25 112 A.7. On Packet Acknowledgment . . . . . . . . . . . . . . . . 26 113 A.8. Setting the Loss Detection Timer . . . . . . . . . . . . 27 114 A.9. On Timeout . . . . . . . . . . . . . . . . . . . . . . . 29 115 A.10. Detecting Lost Packets . . . . . . . . . . . . . . . . . 29 116 Appendix B. Congestion Control Pseudocode . . . . . . . . . . . 30 117 B.1. Constants of interest . . . . . . . . . . . . . . . . . . 30 118 B.2. Variables of interest . . . . . . . . . . . . . . . . . . 31 119 B.3. Initialization . . . . . . . . . . . . . . . . . . . . . 32 120 B.4. On Packet Sent . . . . . . . . . . . . . . . . . . . . . 32 121 B.5. On Packet Acknowledgement . . . . . . . . . . . . . . . . 32 122 B.6. On New Congestion Event . . . . . . . . . . . . . . . . . 33 123 B.7. Process ECN Information . . . . . . . . . . . . . . . . . 33 124 B.8. On Packets Lost . . . . . . . . . . . . . . . . . . . . . 34 125 Appendix C. Change Log . . . . . . . . . . . . . . . . . . . . . 34 126 C.1. Since draft-ietf-quic-recovery-23 . . . . . . . . . . . . 34 127 C.2. Since draft-ietf-quic-recovery-22 . . . . . . . . . . . . 35 128 C.3. Since draft-ietf-quic-recovery-21 . . . . . . . . . . . . 35 129 C.4. Since draft-ietf-quic-recovery-20 . . . . . . . . . . . . 35 130 C.5. Since draft-ietf-quic-recovery-19 . . . . . . . . . . . . 35 131 C.6. Since draft-ietf-quic-recovery-18 . . . . . . . . . . . . 36 132 C.7. Since draft-ietf-quic-recovery-17 . . . . . . . . . . . . 36 133 C.8. Since draft-ietf-quic-recovery-16 . . . . . . . . . . . . 36 134 C.9. Since draft-ietf-quic-recovery-14 . . . . . . . . . . . . 37 135 C.10. Since draft-ietf-quic-recovery-13 . . . . . . . . . . . . 37 136 C.11. Since draft-ietf-quic-recovery-12 . . . . . . . . . . . . 38 137 C.12. Since draft-ietf-quic-recovery-11 . . . . . . . . . . . . 38 138 C.13. Since draft-ietf-quic-recovery-10 . . . . . . . . . . . . 38 139 C.14. Since draft-ietf-quic-recovery-09 . . . . . . . . . . . . 38 140 C.15. Since draft-ietf-quic-recovery-08 . . . . . . . . . . . . 38 141 C.16. Since draft-ietf-quic-recovery-07 . . . . . . . . . . . . 38 142 C.17. Since draft-ietf-quic-recovery-06 . . . . . . . . . . . . 39 143 C.18. Since draft-ietf-quic-recovery-05 . . . . . . . . . . . . 39 144 C.19. Since draft-ietf-quic-recovery-04 . . . . . . . . . . . . 39 145 C.20. Since draft-ietf-quic-recovery-03 . . . . . . . . . . . . 39 146 C.21. Since draft-ietf-quic-recovery-02 . . . . . . . . . . . . 39 147 C.22. Since draft-ietf-quic-recovery-01 . . . . . . . . . . . . 39 148 C.23. Since draft-ietf-quic-recovery-00 . . . . . . . . . . . . 39 149 C.24. Since draft-iyengar-quic-loss-recovery-01 . . . . . . . . 39 150 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 40 151 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 40 153 1. Introduction 155 QUIC is a new multiplexed and secure transport atop UDP. QUIC builds 156 on decades of transport and security experience, and implements 157 mechanisms that make it attractive as a modern general-purpose 158 transport. The QUIC protocol is described in [QUIC-TRANSPORT]. 160 QUIC implements the spirit of existing TCP loss recovery mechanisms, 161 described in RFCs, various Internet-drafts, and also those prevalent 162 in the Linux TCP implementation. This document describes QUIC 163 congestion control and loss recovery, and where applicable, 164 attributes the TCP equivalent in RFCs, Internet-drafts, academic 165 papers, and/or TCP implementations. 167 2. Conventions and Definitions 169 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 170 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 171 "OPTIONAL" in this document are to be interpreted as described in BCP 172 14 [RFC2119] [RFC8174] when, and only when, they appear in all 173 capitals, as shown here. 175 Definitions of terms that are used in this document: 177 ACK-only: Any packet containing only one or more ACK frame(s). 179 In-flight: Packets are considered in-flight when they have been sent 180 and are not ACK-only, and they are not acknowledged, declared 181 lost, or abandoned along with old keys. 183 Ack-eliciting Frames: All frames other than ACK, PADDING, and 184 CONNECTION_CLOSE are considered ack-eliciting. 186 Ack-eliciting Packets: Packets that contain ack-eliciting frames 187 elicit an ACK from the receiver within the maximum ack delay and 188 are called ack-eliciting packets. 190 Crypto Packets: Packets containing CRYPTO data sent in Initial or 191 Handshake packets. 193 Out-of-order Packets: Packets that do not increase the largest 194 received packet number for its packet number space by exactly one. 195 Packets arrive out of order when earlier packets are lost or 196 delayed. 198 3. Design of the QUIC Transmission Machinery 200 All transmissions in QUIC are sent with a packet-level header, which 201 indicates the encryption level and includes a packet sequence number 202 (referred to below as a packet number). The encryption level 203 indicates the packet number space, as described in [QUIC-TRANSPORT]. 204 Packet numbers never repeat within a packet number space for the 205 lifetime of a connection. Packet numbers monotonically increase 206 within a space, preventing ambiguity. 208 This design obviates the need for disambiguating between 209 transmissions and retransmissions and eliminates significant 210 complexity from QUIC's interpretation of TCP loss detection 211 mechanisms. 213 QUIC packets can contain multiple frames of different types. The 214 recovery mechanisms ensure that data and frames that need reliable 215 delivery are acknowledged or declared lost and sent in new packets as 216 necessary. The types of frames contained in a packet affect recovery 217 and congestion control logic: 219 o All packets are acknowledged, though packets that contain no ack- 220 eliciting frames are only acknowledged along with ack-eliciting 221 packets. 223 o Long header packets that contain CRYPTO frames are critical to the 224 performance of the QUIC handshake and use shorter timers for 225 acknowledgement. 227 o Packets containing frames besides ACK or CONNECTION_CLOSE frames 228 count toward congestion control limits and are considered in- 229 flight. 231 o PADDING frames cause packets to contribute toward bytes in flight 232 without directly causing an acknowledgment to be sent. 234 3.1. Relevant Differences Between QUIC and TCP 236 Readers familiar with TCP's loss detection and congestion control 237 will find algorithms here that parallel well-known TCP ones. 238 Protocol differences between QUIC and TCP however contribute to 239 algorithmic differences. We briefly describe these protocol 240 differences below. 242 3.1.1. Separate Packet Number Spaces 244 QUIC uses separate packet number spaces for each encryption level, 245 except 0-RTT and all generations of 1-RTT keys use the same packet 246 number space. Separate packet number spaces ensures acknowledgement 247 of packets sent with one level of encryption will not cause spurious 248 retransmission of packets sent with a different encryption level. 249 Congestion control and round-trip time (RTT) measurement are unified 250 across packet number spaces. 252 3.1.2. Monotonically Increasing Packet Numbers 254 TCP conflates transmission order at the sender with delivery order at 255 the receiver, which results in retransmissions of the same data 256 carrying the same sequence number, and consequently leads to 257 "retransmission ambiguity". QUIC separates the two: QUIC uses a 258 packet number to indicate transmission order, and any application 259 data is sent in one or more streams, with delivery order determined 260 by stream offsets encoded within STREAM frames. 262 QUIC's packet number is strictly increasing within a packet number 263 space, and directly encodes transmission order. A higher packet 264 number signifies that the packet was sent later, and a lower packet 265 number signifies that the packet was sent earlier. When a packet 266 containing ack-eliciting frames is detected lost, QUIC rebundles 267 necessary frames in a new packet with a new packet number, removing 268 ambiguity about which packet is acknowledged when an ACK is received. 269 Consequently, more accurate RTT measurements can be made, spurious 270 retransmissions are trivially detected, and mechanisms such as Fast 271 Retransmit can be applied universally, based only on packet number. 273 This design point significantly simplifies loss detection mechanisms 274 for QUIC. Most TCP mechanisms implicitly attempt to infer 275 transmission ordering based on TCP sequence numbers - a non-trivial 276 task, especially when TCP timestamps are not available. 278 3.1.3. Clearer Loss Epoch 280 QUIC ends a loss epoch when a packet sent after loss is declared is 281 acknowledged. TCP waits for the gap in the sequence number space to 282 be filled, and so if a segment is lost multiple times in a row, the 283 loss epoch may not end for several round trips. Because both should 284 reduce their congestion windows only once per epoch, QUIC will do it 285 correctly once for every round trip that experiences loss, while TCP 286 may only do it once across multiple round trips. 288 3.1.4. No Reneging 290 QUIC ACKs contain information that is similar to TCP SACK, but QUIC 291 does not allow any acked packet to be reneged, greatly simplifying 292 implementations on both sides and reducing memory pressure on the 293 sender. 295 3.1.5. More ACK Ranges 297 QUIC supports many ACK ranges, opposed to TCP's 3 SACK ranges. In 298 high loss environments, this speeds recovery, reduces spurious 299 retransmits, and ensures forward progress without relying on 300 timeouts. 302 3.1.6. Explicit Correction For Delayed Acknowledgements 304 QUIC endpoints measure the delay incurred between when a packet is 305 received and when the corresponding acknowledgment is sent, allowing 306 a peer to maintain a more accurate round-trip time estimate (see 307 Section 13.2 of [QUIC-TRANSPORT]). 309 4. Estimating the Round-Trip Time 311 At a high level, an endpoint measures the time from when a packet was 312 sent to when it is acknowledged as a round-trip time (RTT) sample. 313 The endpoint uses RTT samples and peer-reported host delays (see 314 Section 13.2 of [QUIC-TRANSPORT]) to generate a statistical 315 description of the connection's RTT. An endpoint computes the 316 following three values: the minimum value observed over the lifetime 317 of the connection (min_rtt), an exponentially-weighted moving average 318 (smoothed_rtt), and the variance in the observed RTT samples 319 (rttvar). 321 4.1. Generating RTT samples 323 An endpoint generates an RTT sample on receiving an ACK frame that 324 meets the following two conditions: 326 o the largest acknowledged packet number is newly acknowledged, and 328 o at least one of the newly acknowledged packets was ack-eliciting. 330 The RTT sample, latest_rtt, is generated as the time elapsed since 331 the largest acknowledged packet was sent: 333 latest_rtt = ack_time - send_time_of_largest_acked 334 An RTT sample is generated using only the largest acknowledged packet 335 in the received ACK frame. This is because a peer reports host 336 delays for only the largest acknowledged packet in an ACK frame. 337 While the reported host delay is not used by the RTT sample 338 measurement, it is used to adjust the RTT sample in subsequent 339 computations of smoothed_rtt and rttvar Section 4.3. 341 To avoid generating multiple RTT samples using the same packet, an 342 ACK frame SHOULD NOT be used to update RTT estimates if it does not 343 newly acknowledge the largest acknowledged packet. 345 An RTT sample MUST NOT be generated on receiving an ACK frame that 346 does not newly acknowledge at least one ack-eliciting packet. A peer 347 does not send an ACK frame on receiving only non-ack-eliciting 348 packets, so an ACK frame that is subsequently sent can include an 349 arbitrarily large Ack Delay field. Ignoring such ACK frames avoids 350 complications in subsequent smoothed_rtt and rttvar computations. 352 A sender might generate multiple RTT samples per RTT when multiple 353 ACK frames are received within an RTT. As suggested in [RFC6298], 354 doing so might result in inadequate history in smoothed_rtt and 355 rttvar. Ensuring that RTT estimates retain sufficient history is an 356 open research question. 358 4.2. Estimating min_rtt 360 min_rtt is the minimum RTT observed over the lifetime of the 361 connection. min_rtt is set to the latest_rtt on the first sample in 362 a connection, and to the lesser of min_rtt and latest_rtt on 363 subsequent samples. 365 An endpoint uses only locally observed times in computing the min_rtt 366 and does not adjust for host delays reported by the peer. Doing so 367 allows the endpoint to set a lower bound for the smoothed_rtt based 368 entirely on what it observes (see Section 4.3), and limits potential 369 underestimation due to erroneously-reported delays by the peer. 371 4.3. Estimating smoothed_rtt and rttvar 373 smoothed_rtt is an exponentially-weighted moving average of an 374 endpoint's RTT samples, and rttvar is the endpoint's estimated 375 variance in the RTT samples. 377 The calculation of smoothed_rtt uses path latency after adjusting RTT 378 samples for host delays. For packets sent in the ApplicationData 379 packet number space, a peer limits any delay in sending an 380 acknowledgement for an ack-eliciting packet to no greater than the 381 value it advertised in the max_ack_delay transport parameter. 383 Consequently, when a peer reports an Ack Delay that is greater than 384 its max_ack_delay, the delay is attributed to reasons out of the 385 peer's control, such as scheduler latency at the peer or loss of 386 previous ACK frames. Any delays beyond the peer's max_ack_delay are 387 therefore considered effectively part of path delay and incorporated 388 into the smoothed_rtt estimate. 390 When adjusting an RTT sample using peer-reported acknowledgement 391 delays, an endpoint: 393 o MUST ignore the Ack Delay field of the ACK frame for packets sent 394 in the Initial and Handshake packet number space. 396 o MUST use the lesser of the value reported in Ack Delay field of 397 the ACK frame and the peer's max_ack_delay transport parameter. 399 o MUST NOT apply the adjustment if the resulting RTT sample is 400 smaller than the min_rtt. This limits the underestimation that a 401 misreporting peer can cause to the smoothed_rtt. 403 On the first RTT sample in a connection, the smoothed_rtt is set to 404 the latest_rtt. 406 smoothed_rtt and rttvar are computed as follows, similar to 407 [RFC6298]. On the first RTT sample in a connection: 409 smoothed_rtt = latest_rtt 410 rttvar = latest_rtt / 2 412 On subsequent RTT samples, smoothed_rtt and rttvar evolve as follows: 414 ack_delay = min(Ack Delay in ACK Frame, max_ack_delay) 415 adjusted_rtt = latest_rtt 416 if (min_rtt + ack_delay < latest_rtt): 417 adjusted_rtt = latest_rtt - ack_delay 418 smoothed_rtt = 7/8 * smoothed_rtt + 1/8 * adjusted_rtt 419 rttvar_sample = abs(smoothed_rtt - adjusted_rtt) 420 rttvar = 3/4 * rttvar + 1/4 * rttvar_sample 422 5. Loss Detection 424 QUIC senders use both ack information and timeouts to detect lost 425 packets, and this section provides a description of these algorithms. 427 If a packet is lost, the QUIC transport needs to recover from that 428 loss, such as by retransmitting the data, sending an updated frame, 429 or abandoning the frame. For more information, see Section 13.3 of 430 [QUIC-TRANSPORT]. 432 5.1. Acknowledgement-based Detection 434 Acknowledgement-based loss detection implements the spirit of TCP's 435 Fast Retransmit [RFC5681], Early Retransmit [RFC5827], FACK [FACK], 436 SACK loss recovery [RFC6675], and RACK [RACK]. This section provides 437 an overview of how these algorithms are implemented in QUIC. 439 A packet is declared lost if it meets all the following conditions: 441 o The packet is unacknowledged, in-flight, and was sent prior to an 442 acknowledged packet. 444 o Either its packet number is kPacketThreshold smaller than an 445 acknowledged packet (Section 5.1.1), or it was sent long enough in 446 the past (Section 5.1.2). 448 The acknowledgement indicates that a packet sent later was delivered, 449 while the packet and time thresholds provide some tolerance for 450 packet reordering. 452 Spuriously declaring packets as lost leads to unnecessary 453 retransmissions and may result in degraded performance due to the 454 actions of the congestion controller upon detecting loss. 455 Implementations that detect spurious retransmissions and increase the 456 reordering threshold in packets or time MAY choose to start with 457 smaller initial reordering thresholds to minimize recovery latency. 459 5.1.1. Packet Threshold 461 The RECOMMENDED initial value for the packet reordering threshold 462 (kPacketThreshold) is 3, based on best practices for TCP loss 463 detection [RFC5681] [RFC6675]. 465 Some networks may exhibit higher degrees of reordering, causing a 466 sender to detect spurious losses. Implementers MAY use algorithms 467 developed for TCP, such as TCP-NCR [RFC4653], to improve QUIC's 468 reordering resilience. 470 5.1.2. Time Threshold 472 Once a later packet packet within the same packet number space has 473 been acknowledged, an endpoint SHOULD declare an earlier packet lost 474 if it was sent a threshold amount of time in the past. To avoid 475 declaring packets as lost too early, this time threshold MUST be set 476 to at least kGranularity. The time threshold is: 478 kTimeThreshold * max(smoothed_rtt, latest_rtt, kGranularity) 479 If packets sent prior to the largest acknowledged packet cannot yet 480 be declared lost, then a timer SHOULD be set for the remaining time. 482 Using max(smoothed_rtt, latest_rtt) protects from the two following 483 cases: 485 o the latest RTT sample is lower than the smoothed RTT, perhaps due 486 to reordering where the acknowledgement encountered a shorter 487 path; 489 o the latest RTT sample is higher than the smoothed RTT, perhaps due 490 to a sustained increase in the actual RTT, but the smoothed RTT 491 has not yet caught up. 493 The RECOMMENDED time threshold (kTimeThreshold), expressed as a 494 round-trip time multiplier, is 9/8. 496 Implementations MAY experiment with absolute thresholds, thresholds 497 from previous connections, adaptive thresholds, or including RTT 498 variance. Smaller thresholds reduce reordering resilience and 499 increase spurious retransmissions, and larger thresholds increase 500 loss detection delay. 502 5.2. Probe Timeout 504 A Probe Timeout (PTO) triggers sending one or two probe datagrams 505 when ack-eliciting packets are not acknowledged within the expected 506 period of time or the handshake has not been completed. A PTO 507 enables a connection to recover from loss of tail packets or 508 acknowledgements. The PTO algorithm used in QUIC implements the 509 reliability functions of Tail Loss Probe [RACK], RTO [RFC5681] and 510 F-RTO algorithms for TCP [RFC5682], and the timeout computation is 511 based on TCP's retransmission timeout period [RFC6298]. 513 5.2.1. Computing PTO 515 When an ack-eliciting packet is transmitted, the sender schedules a 516 timer for the PTO period as follows: 518 PTO = smoothed_rtt + max(4*rttvar, kGranularity) + max_ack_delay 520 kGranularity, smoothed_rtt, rttvar, and max_ack_delay are defined in 521 Appendix A.2 and Appendix A.3. 523 The PTO period is the amount of time that a sender ought to wait for 524 an acknowledgement of a sent packet. This time period includes the 525 estimated network roundtrip-time (smoothed_rtt), the variance in the 526 estimate (4*rttvar), and max_ack_delay, to account for the maximum 527 time by which a receiver might delay sending an acknowledgement. 529 The PTO value MUST be set to at least kGranularity, to avoid the 530 timer expiring immediately. 532 When a PTO timer expires, the PTO period MUST be set to twice its 533 current value. This exponential reduction in the sender's rate is 534 important because the PTOs might be caused by loss of packets or 535 acknowledgements due to severe congestion. The life of a connection 536 that is experiencing consecutive PTOs is limited by the endpoint's 537 idle timeout. 539 A sender computes its PTO timer every time an ack-eliciting packet is 540 sent. A sender might choose to optimize this by setting the timer 541 fewer times if it knows that more ack-eliciting packets will be sent 542 within a short period of time. 544 The probe timer is not set if the time threshold Section 5.1.2 loss 545 detection timer is set. The time threshold loss detection timer is 546 expected to both expire earlier than the PTO and be less likely to 547 spuriously retransmit data. 549 5.3. Handshakes and New Paths 551 The initial probe timeout for a new connection or new path SHOULD be 552 set to twice the initial RTT. Resumed connections over the same 553 network SHOULD use the previous connection's final smoothed RTT value 554 as the resumed connection's initial RTT. If no previous RTT is 555 available, the initial RTT SHOULD be set to 500ms, resulting in a 1 556 second initial timeout as recommended in [RFC6298]. 558 A connection MAY use the delay between sending a PATH_CHALLENGE and 559 receiving a PATH_RESPONSE to seed initial_rtt for a new path, but the 560 delay SHOULD NOT be considered an RTT sample. 562 Until the server has validated the client's address on the path, the 563 amount of data it can send is limited to three times the amount of 564 data received, as specified in Section 8.1 of [QUIC-TRANSPORT]. If 565 no data can be sent, then the PTO alarm MUST NOT be armed. 567 Since the server could be blocked until more packets are received 568 from the client, it is the client's responsibility to send packets to 569 unblock the server until it is certain that the server has finished 570 its address validation (see Section 8 of [QUIC-TRANSPORT]). That is, 571 the client MUST set the probe timer if the client has not received an 572 acknowledgement for one of its Handshake or 1-RTT packets. 574 Prior to handshake completion, when few to none RTT samples have been 575 generated, it is possible that the probe timer expiration is due to 576 an incorrect RTT estimate at the client. To allow the client to 577 improve its RTT estimate, the new packet that it sends MUST be ack- 578 eliciting. If Handshake keys are available to the client, it MUST 579 send a Handshake packet, and otherwise it MUST send an Initial packet 580 in a UDP datagram of at least 1200 bytes. 582 Initial packets and Handshake packets may never be acknowledged, but 583 they are removed from bytes in flight when the Initial and Handshake 584 keys are discarded. 586 5.3.1. Sending Probe Packets 588 When a PTO timer expires, a sender MUST send at least one ack- 589 eliciting packet as a probe, unless there is no data available to 590 send. An endpoint MAY send up to two full-sized datagrams containing 591 ack-eliciting packets, to avoid an expensive consecutive PTO 592 expiration due to a single lost datagram. 594 When the PTO timer expires, and there is new or previously sent 595 unacknowledged data, it MUST be sent. Data that was previously sent 596 with Initial encryption MUST be sent before Handshake data and data 597 previously sent at Handshake encryption MUST be sent before any 598 ApplicationData data. 600 It is possible the sender has no new or previously-sent data to send. 601 As an example, consider the following sequence of events: new 602 application data is sent in a STREAM frame, deemed lost, then 603 retransmitted in a new packet, and then the original transmission is 604 acknowledged. When there is no data to send, the sender SHOULD send 605 a PING or other ack-eliciting frame in a single packet, re-arming the 606 PTO timer. 608 Alternatively, instead of sending an ack-eliciting packet, the sender 609 MAY mark any packets still in flight as lost. Doing so avoids 610 sending an additional packet, but increases the risk that loss is 611 declared too aggressively, resulting in an unnecessary rate reduction 612 by the congestion controller. 614 Consecutive PTO periods increase exponentially, and as a result, 615 connection recovery latency increases exponentially as packets 616 continue to be dropped in the network. Sending two packets on PTO 617 expiration increases resilience to packet drops, thus reducing the 618 probability of consecutive PTO events. 620 Probe packets sent on a PTO MUST be ack-eliciting. A probe packet 621 SHOULD carry new data when possible. A probe packet MAY carry 622 retransmitted unacknowledged data when new data is unavailable, when 623 flow control does not permit new data to be sent, or to 624 opportunistically reduce loss recovery delay. Implementations MAY 625 use alternative strategies for determining the content of probe 626 packets, including sending new or retransmitted data based on the 627 application's priorities. 629 When the PTO timer expires multiple times and new data cannot be 630 sent, implementations must choose between sending the same payload 631 every time or sending different payloads. Sending the same payload 632 may be simpler and ensures the highest priority frames arrive first. 633 Sending different payloads each time reduces the chances of spurious 634 retransmission. 636 5.3.2. Loss Detection 638 Delivery or loss of packets in flight is established when an ACK 639 frame is received that newly acknowledges one or more packets. 641 A PTO timer expiration event does not indicate packet loss and MUST 642 NOT cause prior unacknowledged packets to be marked as lost. When an 643 acknowledgement is received that newly acknowledges packets, loss 644 detection proceeds as dictated by packet and time threshold 645 mechanisms; see Section 5.1. 647 5.4. Handling Retry Packets 649 A Retry packet causes a client to send another Initial packet, 650 effectively restarting the connection process. A Retry packet 651 indicates that the Initial was received, but not processed. A Retry 652 packet cannot be treated as an acknowledgment, because it does not 653 indicate that a packet was processed or specify the packet number. 655 Clients that receive a Retry packet reset congestion control and loss 656 recovery state, including resetting any pending timers. Other 657 connection state, in particular cryptographic handshake messages, is 658 retained; see Section 17.2.5 of [QUIC-TRANSPORT]. 660 The client MAY compute an RTT estimate to the server as the time 661 period from when the first Initial was sent to when a Retry or a 662 Version Negotiation packet is received. The client MAY use this 663 value in place of its default for the initial RTT estimate. 665 5.5. Discarding Keys and Packet State 667 When packet protection keys are discarded (see Section 4.9 of 668 [QUIC-TLS]), all packets that were sent with those keys can no longer 669 be acknowledged because their acknowledgements cannot be processed 670 anymore. The sender MUST discard all recovery state associated with 671 those packets and MUST remove them from the count of bytes in flight. 673 Endpoints stop sending and receiving Initial packets once they start 674 exchanging Handshake packets (see Section 17.2.2.1 of 675 [QUIC-TRANSPORT]). At this point, recovery state for all in-flight 676 Initial packets is discarded. 678 When 0-RTT is rejected, recovery state for all in-flight 0-RTT 679 packets is discarded. 681 If a server accepts 0-RTT, but does not buffer 0-RTT packets that 682 arrive before Initial packets, early 0-RTT packets will be declared 683 lost, but that is expected to be infrequent. 685 It is expected that keys are discarded after packets encrypted with 686 them would be acknowledged or declared lost. Initial secrets however 687 might be destroyed sooner, as soon as handshake keys are available 688 (see Section 4.9.1 of [QUIC-TLS]). 690 6. Congestion Control 692 QUIC's congestion control is based on TCP NewReno [RFC6582]. NewReno 693 is a congestion window based congestion control. QUIC specifies the 694 congestion window in bytes rather than packets due to finer control 695 and the ease of appropriate byte counting [RFC3465]. 697 QUIC hosts MUST NOT send packets if they would increase 698 bytes_in_flight (defined in Appendix B.2) beyond the available 699 congestion window, unless the packet is a probe packet sent after a 700 PTO timer expires, as described in Section 5.2. 702 Implementations MAY use other congestion control algorithms, such as 703 Cubic [RFC8312], and endpoints MAY use different algorithms from one 704 another. The signals QUIC provides for congestion control are 705 generic and are designed to support different algorithms. 707 6.1. Explicit Congestion Notification 709 If a path has been verified to support ECN, QUIC treats a Congestion 710 Experienced codepoint in the IP header as a signal of congestion. 711 This document specifies an endpoint's response when its peer receives 712 packets with the Congestion Experienced codepoint. As discussed in 713 [RFC8311], endpoints are permitted to experiment with other response 714 functions. 716 6.2. Slow Start 718 QUIC begins every connection in slow start and exits slow start upon 719 loss or upon increase in the ECN-CE counter. QUIC re-enters slow 720 start anytime the congestion window is less than ssthresh, which only 721 occurs after persistent congestion is declared. While in slow start, 722 QUIC increases the congestion window by the number of bytes 723 acknowledged when each acknowledgment is processed. 725 6.3. Congestion Avoidance 727 Slow start exits to congestion avoidance. Congestion avoidance in 728 NewReno uses an additive increase multiplicative decrease (AIMD) 729 approach that increases the congestion window by one maximum packet 730 size per congestion window acknowledged. When a loss is detected, 731 NewReno halves the congestion window and sets the slow start 732 threshold to the new congestion window. 734 6.4. Recovery Period 736 Recovery is a period of time beginning with detection of a lost 737 packet or an increase in the ECN-CE counter. Because QUIC does not 738 retransmit packets, it defines the end of recovery as a packet sent 739 after the start of recovery being acknowledged. This is slightly 740 different from TCP's definition of recovery, which ends when the lost 741 packet that started recovery is acknowledged. 743 The recovery period limits congestion window reduction to once per 744 round trip. During recovery, the congestion window remains unchanged 745 irrespective of new losses or increases in the ECN-CE counter. 747 6.5. Ignoring Loss of Undecryptable Packets 749 During the handshake, some packet protection keys might not be 750 available when a packet arrives. In particular, Handshake and 0-RTT 751 packets cannot be processed until the Initial packets arrive, and 752 1-RTT packets cannot be processed until the handshake completes. 753 Endpoints MAY ignore the loss of Handshake, 0-RTT, and 1-RTT packets 754 that might arrive before the peer has packet protection keys to 755 process those packets. 757 6.6. Probe Timeout 759 Probe packets MUST NOT be blocked by the congestion controller. A 760 sender MUST however count these packets as being additionally in 761 flight, since these packets add network load without establishing 762 packet loss. Note that sending probe packets might cause the 763 sender's bytes in flight to exceed the congestion window until an 764 acknowledgement is received that establishes loss or delivery of 765 packets. 767 6.7. Persistent Congestion 769 When an ACK frame is received that establishes loss of all in-flight 770 packets sent over a long enough period of time, the network is 771 considered to be experiencing persistent congestion. Commonly, this 772 can be established by consecutive PTOs, but since the PTO timer is 773 reset when a new ack-eliciting packet is sent, an explicit duration 774 must be used to account for those cases where PTOs do not occur or 775 are substantially delayed. This duration is computed as follows: 777 (smoothed_rtt + 4 * rttvar + max_ack_delay) * 778 kPersistentCongestionThreshold 780 For example, assume: 782 smoothed_rtt = 1 rttvar = 0 max_ack_delay = 0 783 kPersistentCongestionThreshold = 3 785 If an ack-eliciting packet is sent at time = 0, the following 786 scenario would illustrate persistent congestion: 788 +-----+------------------------+ 789 | t=0 | Send Pkt #1 (App Data) | 790 +-----+------------------------+ 791 | t=1 | Send Pkt #2 (PTO 1) | 792 | | | 793 | t=3 | Send Pkt #3 (PTO 2) | 794 | | | 795 | t=7 | Send Pkt #4 (PTO 3) | 796 | | | 797 | t=8 | Recv ACK of Pkt #4 | 798 +-----+------------------------+ 800 The first three packets are determined to be lost when the ACK of 801 packet 4 is received at t=8. The congestion period is calculated as 802 the time between the oldest and newest lost packets: (3 - 0) = 3. 803 The duration for persistent congestion is equal to: (1 * 804 kPersistentCongestionThreshold) = 3. Because the threshold was 805 reached and because none of the packets between the oldest and the 806 newest packets are acknowledged, the network is considered to have 807 experienced persistent congestion. 809 When persistent congestion is established, the sender's congestion 810 window MUST be reduced to the minimum congestion window 811 (kMinimumWindow). This response of collapsing the congestion window 812 on persistent congestion is functionally similar to a sender's 813 response on a Retransmission Timeout (RTO) in TCP [RFC5681] after 814 Tail Loss Probes (TLP) [RACK]. 816 6.8. Pacing 818 This document does not specify a pacer, but it is RECOMMENDED that a 819 sender pace sending of all in-flight packets based on input from the 820 congestion controller. For example, a pacer might distribute the 821 congestion window over the smoothed RTT when used with a window-based 822 controller, and a pacer might use the rate estimate of a rate-based 823 controller. 825 An implementation should take care to architect its congestion 826 controller to work well with a pacer. For instance, a pacer might 827 wrap the congestion controller and control the availability of the 828 congestion window, or a pacer might pace out packets handed to it by 829 the congestion controller. Timely delivery of ACK frames is 830 important for efficient loss recovery. Packets containing only ACK 831 frames should therefore not be paced, to avoid delaying their 832 delivery to the peer. 834 Sending multiple packets into the network without any delay between 835 them creates a packet burst that might cause short-term congestion 836 and losses. Implementations MUST either use pacing or limit such 837 bursts to the initial congestion window, which is recommended to be 838 the minimum of 10 * max_datagram_size and max(2* max_datagram_size, 839 14720)), where max_datagram_size is the current maximum size of a 840 datagram for the connection, not including UDP or IP overhead. 842 As an example of a well-known and publicly available implementation 843 of a flow pacer, implementers are referred to the Fair Queue packet 844 scheduler (fq qdisc) in Linux (3.11 onwards). 846 6.9. Under-utilizing the Congestion Window 848 When bytes in flight is smaller than the congestion window and 849 sending is not pacing limited, the congestion window is under- 850 utilized. When this occurs, the congestion window SHOULD NOT be 851 increased in either slow start or congestion avoidance. This can 852 happen due to insufficient application data or flow control credit. 854 A sender MAY use the pipeACK method described in section 4.3 of 855 [RFC7661] to determine if the congestion window is sufficiently 856 utilized. 858 A sender that paces packets (see Section 6.8) might delay sending 859 packets and not fully utilize the congestion window due to this 860 delay. A sender should not consider itself application limited if it 861 would have fully utilized the congestion window without pacing delay. 863 A sender MAY implement alternative mechanisms to update its 864 congestion window after periods of under-utilization, such as those 865 proposed for TCP in [RFC7661]. 867 7. Security Considerations 869 7.1. Congestion Signals 871 Congestion control fundamentally involves the consumption of signals 872 - both loss and ECN codepoints - from unauthenticated entities. On- 873 path attackers can spoof or alter these signals. An attacker can 874 cause endpoints to reduce their sending rate by dropping packets, or 875 alter send rate by changing ECN codepoints. 877 7.2. Traffic Analysis 879 Packets that carry only ACK frames can be heuristically identified by 880 observing packet size. Acknowledgement patterns may expose 881 information about link characteristics or application behavior. 882 Endpoints can use PADDING frames or bundle acknowledgments with other 883 frames to reduce leaked information. 885 7.3. Misreporting ECN Markings 887 A receiver can misreport ECN markings to alter the congestion 888 response of a sender. Suppressing reports of ECN-CE markings could 889 cause a sender to increase their send rate. This increase could 890 result in congestion and loss. 892 A sender MAY attempt to detect suppression of reports by marking 893 occasional packets that they send with ECN-CE. If a packet marked 894 with ECN-CE is not reported as having been marked when the packet is 895 acknowledged, the sender SHOULD then disable ECN for that path. 897 Reporting additional ECN-CE markings will cause a sender to reduce 898 their sending rate, which is similar in effect to advertising reduced 899 connection flow control limits and so no advantage is gained by doing 900 so. 902 Endpoints choose the congestion controller that they use. Though 903 congestion controllers generally treat reports of ECN-CE markings as 904 equivalent to loss [RFC8311], the exact response for each controller 905 could be different. Failure to correctly respond to information 906 about ECN markings is therefore difficult to detect. 908 8. IANA Considerations 910 This document has no IANA actions. Yet. 912 9. References 914 9.1. Normative References 916 [QUIC-TLS] 917 Thomson, M., Ed. and S. Turner, Ed., "Using TLS to Secure 918 QUIC", draft-ietf-quic-tls-24 (work in progress), November 919 2019. 921 [QUIC-TRANSPORT] 922 Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 923 Multiplexed and Secure Transport", draft-ietf-quic- 924 transport-24 (work in progress), November 2019. 926 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 927 Requirement Levels", BCP 14, RFC 2119, 928 DOI 10.17487/RFC2119, March 1997, 929 . 931 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 932 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 933 May 2017, . 935 [RFC8311] Black, D., "Relaxing Restrictions on Explicit Congestion 936 Notification (ECN) Experimentation", RFC 8311, 937 DOI 10.17487/RFC8311, January 2018, 938 . 940 9.2. Informative References 942 [FACK] Mathis, M. and J. Mahdavi, "Forward Acknowledgement: 943 Refining TCP Congestion Control", ACM SIGCOMM , August 944 1996. 946 [RACK] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "RACK: 947 a time-based fast loss detection algorithm for TCP", 948 draft-ietf-tcpm-rack-05 (work in progress), April 2019. 950 [RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte 951 Counting (ABC)", RFC 3465, DOI 10.17487/RFC3465, February 952 2003, . 954 [RFC4653] Bhandarkar, S., Reddy, A., Allman, M., and E. Blanton, 955 "Improving the Robustness of TCP to Non-Congestion 956 Events", RFC 4653, DOI 10.17487/RFC4653, August 2006, 957 . 959 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 960 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 961 . 963 [RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata, 964 "Forward RTO-Recovery (F-RTO): An Algorithm for Detecting 965 Spurious Retransmission Timeouts with TCP", RFC 5682, 966 DOI 10.17487/RFC5682, September 2009, 967 . 969 [RFC5827] Allman, M., Avrachenkov, K., Ayesta, U., Blanton, J., and 970 P. Hurtig, "Early Retransmit for TCP and Stream Control 971 Transmission Protocol (SCTP)", RFC 5827, 972 DOI 10.17487/RFC5827, May 2010, 973 . 975 [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, 976 "Computing TCP's Retransmission Timer", RFC 6298, 977 DOI 10.17487/RFC6298, June 2011, 978 . 980 [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The 981 NewReno Modification to TCP's Fast Recovery Algorithm", 982 RFC 6582, DOI 10.17487/RFC6582, April 2012, 983 . 985 [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., 986 and Y. Nishida, "A Conservative Loss Recovery Algorithm 987 Based on Selective Acknowledgment (SACK) for TCP", 988 RFC 6675, DOI 10.17487/RFC6675, August 2012, 989 . 991 [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, 992 "Increasing TCP's Initial Window", RFC 6928, 993 DOI 10.17487/RFC6928, April 2013, 994 . 996 [RFC7661] Fairhurst, G., Sathiaseelan, A., and R. Secchi, "Updating 997 TCP to Support Rate-Limited Traffic", RFC 7661, 998 DOI 10.17487/RFC7661, October 2015, 999 . 1001 [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and 1002 R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", 1003 RFC 8312, DOI 10.17487/RFC8312, February 2018, 1004 . 1006 9.3. URIs 1008 [1] https://mailarchive.ietf.org/arch/search/?email_list=quic 1010 [2] https://github.com/quicwg 1012 [3] https://github.com/quicwg/base-drafts/labels/-recovery 1014 Appendix A. Loss Recovery Pseudocode 1016 We now describe an example implementation of the loss detection 1017 mechanisms described in Section 5. 1019 A.1. Tracking Sent Packets 1021 To correctly implement congestion control, a QUIC sender tracks every 1022 ack-eliciting packet until the packet is acknowledged or lost. It is 1023 expected that implementations will be able to access this information 1024 by packet number and crypto context and store the per-packet fields 1025 (Appendix A.1.1) for loss recovery and congestion control. 1027 After a packet is declared lost, the endpoint can track it for an 1028 amount of time comparable to the maximum expected packet reordering, 1029 such as 1 RTT. This allows for detection of spurious 1030 retransmissions. 1032 Sent packets are tracked for each packet number space, and ACK 1033 processing only applies to a single space. 1035 A.1.1. Sent Packet Fields 1037 packet_number: The packet number of the sent packet. 1039 ack_eliciting: A boolean that indicates whether a packet is ack- 1040 eliciting. If true, it is expected that an acknowledgement will 1041 be received, though the peer could delay sending the ACK frame 1042 containing it by up to the MaxAckDelay. 1044 in_flight: A boolean that indicates whether the packet counts 1045 towards bytes in flight. 1047 sent_bytes: The number of bytes sent in the packet, not including 1048 UDP or IP overhead, but including QUIC framing overhead. 1050 time_sent: The time the packet was sent. 1052 A.2. Constants of interest 1054 Constants used in loss recovery are based on a combination of RFCs, 1055 papers, and common practice. Some may need to be changed or 1056 negotiated in order to better suit a variety of environments. 1058 kPacketThreshold: Maximum reordering in packets before packet 1059 threshold loss detection considers a packet lost. The RECOMMENDED 1060 value is 3. 1062 kTimeThreshold: Maximum reordering in time before time threshold 1063 loss detection considers a packet lost. Specified as an RTT 1064 multiplier. The RECOMMENDED value is 9/8. 1066 kGranularity: Timer granularity. This is a system-dependent value. 1067 However, implementations SHOULD use a value no smaller than 1ms. 1069 kInitialRtt: The RTT used before an RTT sample is taken. The 1070 RECOMMENDED value is 500ms. 1072 kPacketNumberSpace: An enum to enumerate the three packet number 1073 spaces. 1075 enum kPacketNumberSpace { 1076 Initial, 1077 Handshake, 1078 ApplicationData, 1079 } 1081 A.3. Variables of interest 1083 Variables required to implement the congestion control mechanisms are 1084 described in this section. 1086 latest_rtt: The most recent RTT measurement made when receiving an 1087 ack for a previously unacked packet. 1089 smoothed_rtt: The smoothed RTT of the connection, computed as 1090 described in [RFC6298] 1092 rttvar: The RTT variance, computed as described in [RFC6298] 1094 min_rtt: The minimum RTT seen in the connection, ignoring ack delay. 1096 max_ack_delay: The maximum amount of time by which the receiver 1097 intends to delay acknowledgments for packets in the 1098 ApplicationData packet number space. The actual ack_delay in a 1099 received ACK frame may be larger due to late timers, reordering, 1100 or lost ACKs. 1102 loss_detection_timer: Multi-modal timer used for loss detection. 1104 pto_count: The number of times a PTO has been sent without receiving 1105 an ack. 1107 time_of_last_sent_ack_eliciting_packet: The time the most recent 1108 ack-eliciting packet was sent. 1110 largest_acked_packet[kPacketNumberSpace]: The largest packet number 1111 acknowledged in the packet number space so far. 1113 loss_time[kPacketNumberSpace]: The time at which the next packet in 1114 that packet number space will be considered lost based on 1115 exceeding the reordering window in time. 1117 sent_packets[kPacketNumberSpace]: An association of packet numbers 1118 in a packet number space to information about them. Described in 1119 detail above in Appendix A.1. 1121 A.4. Initialization 1123 At the beginning of the connection, initialize the loss detection 1124 variables as follows: 1126 loss_detection_timer.reset() 1127 pto_count = 0 1128 latest_rtt = 0 1129 smoothed_rtt = 0 1130 rttvar = 0 1131 min_rtt = 0 1132 max_ack_delay = 0 1133 time_of_last_sent_ack_eliciting_packet = 0 1134 for pn_space in [ Initial, Handshake, ApplicationData ]: 1135 largest_acked_packet[pn_space] = infinite 1136 loss_time[pn_space] = 0 1138 A.5. On Sending a Packet 1140 After a packet is sent, information about the packet is stored. The 1141 parameters to OnPacketSent are described in detail above in 1142 Appendix A.1.1. 1144 Pseudocode for OnPacketSent follows: 1146 OnPacketSent(packet_number, pn_space, ack_eliciting, 1147 in_flight, sent_bytes): 1148 sent_packets[pn_space][packet_number].packet_number = 1149 packet_number 1150 sent_packets[pn_space][packet_number].time_sent = now 1151 sent_packets[pn_space][packet_number].ack_eliciting = 1152 ack_eliciting 1153 sent_packets[pn_space][packet_number].in_flight = in_flight 1154 if (in_flight): 1155 if (ack_eliciting): 1156 time_of_last_sent_ack_eliciting_packet = now 1157 OnPacketSentCC(sent_bytes) 1158 sent_packets[pn_space][packet_number].size = sent_bytes 1159 SetLossDetectionTimer() 1161 A.6. On Receiving an Acknowledgment 1163 When an ACK frame is received, it may newly acknowledge any number of 1164 packets. 1166 Pseudocode for OnAckReceived and UpdateRtt follow: 1168 OnAckReceived(ack, pn_space): 1169 if (largest_acked_packet[pn_space] == infinite): 1170 largest_acked_packet[pn_space] = ack.largest_acked 1171 else: 1172 largest_acked_packet[pn_space] = 1173 max(largest_acked_packet[pn_space], ack.largest_acked) 1175 // Nothing to do if there are no newly acked packets. 1176 newly_acked_packets = DetermineNewlyAckedPackets(ack, pn_space) 1177 if (newly_acked_packets.empty()): 1178 return 1180 // If the largest acknowledged is newly acked and 1181 // at least one ack-eliciting was newly acked, update the RTT. 1182 if (sent_packets[pn_space].contains(ack.largest_acked) && 1183 IncludesAckEliciting(newly_acked_packets)): 1184 latest_rtt = 1185 now - sent_packets[pn_space][ack.largest_acked].time_sent 1186 ack_delay = 0 1187 if (pn_space == ApplicationData): 1188 ack_delay = ack.ack_delay 1189 UpdateRtt(ack_delay) 1191 // Process ECN information if present. 1192 if (ACK frame contains ECN information): 1193 ProcessECN(ack, pn_space) 1195 for acked_packet in newly_acked_packets: 1196 OnPacketAcked(acked_packet.packet_number, pn_space) 1198 DetectLostPackets(pn_space) 1200 pto_count = 0 1202 SetLossDetectionTimer() 1204 UpdateRtt(ack_delay): 1205 // First RTT sample. 1206 if (smoothed_rtt == 0): 1207 min_rtt = latest_rtt 1208 smoothed_rtt = latest_rtt 1209 rttvar = latest_rtt / 2 1210 return 1212 // min_rtt ignores ack delay. 1213 min_rtt = min(min_rtt, latest_rtt) 1214 // Limit ack_delay by max_ack_delay 1215 ack_delay = min(ack_delay, max_ack_delay) 1216 // Adjust for ack delay if plausible. 1217 adjusted_rtt = latest_rtt 1218 if (latest_rtt > min_rtt + ack_delay): 1219 adjusted_rtt = latest_rtt - ack_delay 1221 rttvar = 3/4 * rttvar + 1/4 * abs(smoothed_rtt - adjusted_rtt) 1222 smoothed_rtt = 7/8 * smoothed_rtt + 1/8 * adjusted_rtt 1224 A.7. On Packet Acknowledgment 1226 When a packet is acknowledged for the first time, the following 1227 OnPacketAcked function is called. Note that a single ACK frame may 1228 newly acknowledge several packets. OnPacketAcked must be called once 1229 for each of these newly acknowledged packets. 1231 OnPacketAcked takes two parameters: acked_packet, which is the struct 1232 detailed in Appendix A.1.1, and the packet number space that this ACK 1233 frame was sent for. 1235 Pseudocode for OnPacketAcked follows: 1237 OnPacketAcked(acked_packet, pn_space): 1238 if (acked_packet.in_flight): 1239 OnPacketAckedCC(acked_packet) 1240 sent_packets[pn_space].remove(acked_packet.packet_number) 1242 A.8. Setting the Loss Detection Timer 1244 QUIC loss detection uses a single timer for all timeout loss 1245 detection. The duration of the timer is based on the timer's mode, 1246 which is set in the packet and timer events further below. The 1247 function SetLossDetectionTimer defined below shows how the single 1248 timer is set. 1250 This algorithm may result in the timer being set in the past, 1251 particularly if timers wake up late. Timers set in the past SHOULD 1252 fire immediately. 1254 Pseudocode for SetLossDetectionTimer follows: 1256 // Returns the earliest loss_time and the packet number 1257 // space it's from. Returns 0 if all times are 0. 1258 GetEarliestLossTime(): 1259 time = loss_time[Initial] 1260 space = Initial 1261 for pn_space in [ Handshake, ApplicationData ]: 1262 if (loss_time[pn_space] != 0 && 1263 (time == 0 || loss_time[pn_space] < time)): 1264 time = loss_time[pn_space]; 1265 space = pn_space 1266 return time, space 1268 PeerNotAwaitingAddressValidation(): 1269 # Assume clients validate the server's address implicitly. 1270 if (endpoint is server): 1271 return true 1272 # Servers complete address validation when a 1273 # protected packet is received. 1274 return has received Handshake ACK || 1275 has received 1-RTT ACK 1277 SetLossDetectionTimer(): 1278 loss_time, _ = GetEarliestLossTime() 1279 if (loss_time != 0): 1280 // Time threshold loss detection. 1281 loss_detection_timer.update(loss_time) 1282 return 1284 if (no ack-eliciting packets in flight && 1285 PeerNotAwaitingAddressValidation()): 1286 loss_detection_timer.cancel() 1287 return 1289 // Use a default timeout if there are no RTT measurements 1290 if (smoothed_rtt == 0): 1291 timeout = 2 * kInitialRtt 1292 else: 1293 // Calculate PTO duration 1294 timeout = smoothed_rtt + max(4 * rttvar, kGranularity) + 1295 max_ack_delay 1296 timeout = timeout * (2 ^ pto_count) 1298 loss_detection_timer.update( 1299 time_of_last_sent_ack_eliciting_packet + timeout) 1301 A.9. On Timeout 1303 When the loss detection timer expires, the timer's mode determines 1304 the action to be performed. 1306 Pseudocode for OnLossDetectionTimeout follows: 1308 OnLossDetectionTimeout(): 1309 loss_time, pn_space = GetEarliestLossTime() 1310 if (loss_time != 0): 1311 // Time threshold loss Detection 1312 DetectLostPackets(pn_space) 1313 SetLossDetectionTimer() 1314 return 1316 if (endpoint is client without 1-RTT keys): 1317 // Client sends an anti-deadlock packet: Initial is padded 1318 // to earn more anti-amplification credit, 1319 // a Handshake packet proves address ownership. 1320 if (has Handshake keys): 1321 SendOneAckElicitingHandshakePacket() 1322 else: 1323 SendOneAckElicitingPaddedInitialPacket() 1324 else: 1325 // PTO. Send new data if available, else retransmit old data. 1326 // If neither is available, send a single PING frame. 1327 SendOneOrTwoAckElicitingPackets() 1329 pto_count++ 1330 SetLossDetectionTimer() 1332 A.10. Detecting Lost Packets 1334 DetectLostPackets is called every time an ACK is received and 1335 operates on the sent_packets for that packet number space. 1337 Pseudocode for DetectLostPackets follows: 1339 DetectLostPackets(pn_space): 1340 assert(largest_acked_packet[pn_space] != infinite) 1341 loss_time[pn_space] = 0 1342 lost_packets = {} 1343 loss_delay = kTimeThreshold * max(latest_rtt, smoothed_rtt) 1345 // Minimum time of kGranularity before packets are deemed lost. 1346 loss_delay = max(loss_delay, kGranularity) 1348 // Packets sent before this time are deemed lost. 1349 lost_send_time = now() - loss_delay 1351 foreach unacked in sent_packets[pn_space]: 1352 if (unacked.packet_number > largest_acked_packet[pn_space]): 1353 continue 1355 // Mark packet as lost, or set time when it should be marked. 1356 if (unacked.time_sent <= lost_send_time || 1357 largest_acked_packet[pn_space] >= 1358 unacked.packet_number + kPacketThreshold): 1359 sent_packets[pn_space].remove(unacked.packet_number) 1360 if (unacked.in_flight): 1361 lost_packets.insert(unacked) 1362 else: 1363 if (loss_time[pn_space] == 0): 1364 loss_time[pn_space] = unacked.time_sent + loss_delay 1365 else: 1366 loss_time[pn_space] = min(loss_time[pn_space], 1367 unacked.time_sent + loss_delay) 1369 // Inform the congestion controller of lost packets and 1370 // let it decide whether to retransmit immediately. 1371 if (!lost_packets.empty()): 1372 OnPacketsLost(lost_packets) 1374 Appendix B. Congestion Control Pseudocode 1376 We now describe an example implementation of the congestion 1377 controller described in Section 6. 1379 B.1. Constants of interest 1381 Constants used in congestion control are based on a combination of 1382 RFCs, papers, and common practice. Some may need to be changed or 1383 negotiated in order to better suit a variety of environments. 1385 kInitialWindow: Default limit on the initial amount of data in 1386 flight, in bytes. Taken from [RFC6928], but increased slightly to 1387 account for the smaller 8 byte overhead of UDP vs 20 bytes for 1388 TCP. The RECOMMENDED value is the minimum of 10 * 1389 max_datagram_size and max(2 * max_datagram_size, 14720)). 1391 kMinimumWindow: Minimum congestion window in bytes. The RECOMMENDED 1392 value is 2 * max_datagram_size. 1394 kLossReductionFactor: Reduction in congestion window when a new loss 1395 event is detected. The RECOMMENDED value is 0.5. 1397 kPersistentCongestionThreshold: Period of time for persistent 1398 congestion to be established, specified as a PTO multiplier. The 1399 rationale for this threshold is to enable a sender to use initial 1400 PTOs for aggressive probing, as TCP does with Tail Loss Probe 1401 (TLP) [RACK], before establishing persistent congestion, as TCP 1402 does with a Retransmission Timeout (RTO) [RFC5681]. The 1403 RECOMMENDED value for kPersistentCongestionThreshold is 3, which 1404 is approximately equivalent to having two TLPs before an RTO in 1405 TCP. 1407 B.2. Variables of interest 1409 Variables required to implement the congestion control mechanisms are 1410 described in this section. 1412 max_datagram_size: The sender's current maximum payload size. Does 1413 not include UDP or IP overhead. The max datagram size is used for 1414 congestion window computations. An endpoint sets the value of 1415 this variable based on its PMTU (see Section 14.1 of 1416 [QUIC-TRANSPORT]), with a minimum value of 1200 bytes. 1418 ecn_ce_counters[kPacketNumberSpace]: The highest value reported for 1419 the ECN-CE counter in the packet number space by the peer in an 1420 ACK frame. This value is used to detect increases in the reported 1421 ECN-CE counter. 1423 bytes_in_flight: The sum of the size in bytes of all sent packets 1424 that contain at least one ack-eliciting or PADDING frame, and have 1425 not been acked or declared lost. The size does not include IP or 1426 UDP overhead, but does include the QUIC header and AEAD overhead. 1427 Packets only containing ACK frames do not count towards 1428 bytes_in_flight to ensure congestion control does not impede 1429 congestion feedback. 1431 congestion_window: Maximum number of bytes-in-flight that may be 1432 sent. 1434 congestion_recovery_start_time: The time when QUIC first detects 1435 congestion due to loss or ECN, causing it to enter congestion 1436 recovery. When a packet sent after this time is acknowledged, 1437 QUIC exits congestion recovery. 1439 ssthresh: Slow start threshold in bytes. When the congestion window 1440 is below ssthresh, the mode is slow start and the window grows by 1441 the number of bytes acknowledged. 1443 B.3. Initialization 1445 At the beginning of the connection, initialize the congestion control 1446 variables as follows: 1448 congestion_window = kInitialWindow 1449 bytes_in_flight = 0 1450 congestion_recovery_start_time = 0 1451 ssthresh = infinite 1452 for pn_space in [ Initial, Handshake, ApplicationData ]: 1453 ecn_ce_counters[pn_space] = 0 1455 B.4. On Packet Sent 1457 Whenever a packet is sent, and it contains non-ACK frames, the packet 1458 increases bytes_in_flight. 1460 OnPacketSentCC(bytes_sent): 1461 bytes_in_flight += bytes_sent 1463 B.5. On Packet Acknowledgement 1465 Invoked from loss detection's OnPacketAcked and is supplied with the 1466 acked_packet from sent_packets. 1468 InCongestionRecovery(sent_time): 1469 return sent_time <= congestion_recovery_start_time 1471 OnPacketAckedCC(acked_packet): 1472 // Remove from bytes_in_flight. 1473 bytes_in_flight -= acked_packet.size 1474 if (InCongestionRecovery(acked_packet.time_sent)): 1475 // Do not increase congestion window in recovery period. 1476 return 1477 if (IsAppLimited()): 1478 // Do not increase congestion_window if application 1479 // limited. 1480 return 1481 if (congestion_window < ssthresh): 1482 // Slow start. 1483 congestion_window += acked_packet.size 1484 else: 1485 // Congestion avoidance. 1486 congestion_window += max_datagram_size * acked_packet.size 1487 / congestion_window 1489 B.6. On New Congestion Event 1491 Invoked from ProcessECN and OnPacketsLost when a new congestion event 1492 is detected. May start a new recovery period and reduces the 1493 congestion window. 1495 CongestionEvent(sent_time): 1496 // Start a new congestion event if packet was sent after the 1497 // start of the previous congestion recovery period. 1498 if (!InCongestionRecovery(sent_time)): 1499 congestion_recovery_start_time = Now() 1500 congestion_window *= kLossReductionFactor 1501 congestion_window = max(congestion_window, kMinimumWindow) 1502 ssthresh = congestion_window 1504 B.7. Process ECN Information 1506 Invoked when an ACK frame with an ECN section is received from the 1507 peer. 1509 ProcessECN(ack, pn_space): 1510 // If the ECN-CE counter reported by the peer has increased, 1511 // this could be a new congestion event. 1512 if (ack.ce_counter > ecn_ce_counters[pn_space]): 1513 ecn_ce_counters[pn_space] = ack.ce_counter 1514 CongestionEvent(sent_packets[ack.largest_acked].time_sent) 1516 B.8. On Packets Lost 1518 Invoked from DetectLostPackets when packets are deemed lost. 1520 InPersistentCongestion(largest_lost_packet): 1521 pto = smoothed_rtt + max(4 * rttvar, kGranularity) + 1522 max_ack_delay 1523 congestion_period = pto * kPersistentCongestionThreshold 1524 // Determine if all packets in the time period before the 1525 // newest lost packet, including the edges, are marked 1526 // lost 1527 return AreAllPacketsLost(largest_lost_packet, 1528 congestion_period) 1530 OnPacketsLost(lost_packets): 1531 // Remove lost packets from bytes_in_flight. 1532 for (lost_packet : lost_packets): 1533 bytes_in_flight -= lost_packet.size 1534 largest_lost_packet = lost_packets.last() 1535 CongestionEvent(largest_lost_packet.time_sent) 1537 // Collapse congestion window if persistent congestion 1538 if (InPersistentCongestion(largest_lost_packet)): 1539 congestion_window = kMinimumWindow 1541 Appendix C. Change Log 1543 *RFC Editor's Note:* Please remove this section prior to 1544 publication of a final version of this document. 1546 Issue and pull request numbers are listed with a leading octothorp. 1548 C.1. Since draft-ietf-quic-recovery-23 1550 o Define under-utilizing the congestion window (#2630, #2686, #2675) 1552 o PTO MUST send data if possible (#3056, #3057) 1554 o Connection Close is not ack-eliciting (#3097, #3098) 1556 o MUST limit bursts to the initial congestion window (#3160) 1558 o Define the current max_datagram_size for congestion control 1559 (#3041, #3167) 1561 o Separate PTO by packet number space (#3067, #3074, #3066) 1563 C.2. Since draft-ietf-quic-recovery-22 1565 o PTO should always send an ack-eliciting packet (#2895) 1567 o Unify the Handshake Timer with the PTO timer (#2648, #2658, #2886) 1569 o Move ACK generation text to transport draft (#1860, #2916) 1571 C.3. Since draft-ietf-quic-recovery-21 1573 o No changes 1575 C.4. Since draft-ietf-quic-recovery-20 1577 o Path validation can be used as initial RTT value (#2644, #2687) 1579 o max_ack_delay transport parameter defaults to 0 (#2638, #2646) 1581 o Ack Delay only measures intentional delays induced by the 1582 implementation (#2596, #2786) 1584 C.5. Since draft-ietf-quic-recovery-19 1586 o Change kPersistentThreshold from an exponent to a multiplier 1587 (#2557) 1589 o Send a PING if the PTO timer fires and there's nothing to send 1590 (#2624) 1592 o Set loss delay to at least kGranularity (#2617) 1594 o Merge application limited and sending after idle sections. Always 1595 limit burst size instead of requiring resetting CWND to initial 1596 CWND after idle (#2605) 1598 o Rewrite RTT estimation, allow RTT samples where a newly acked 1599 packet is ack-eliciting but the largest_acked is not (#2592) 1601 o Don't arm the handshake timer if there is no handshake data 1602 (#2590) 1604 o Clarify that the time threshold loss alarm takes precedence over 1605 the crypto handshake timer (#2590, #2620) 1607 o Change initial RTT to 500ms to align with RFC6298 (#2184) 1609 C.6. Since draft-ietf-quic-recovery-18 1611 o Change IW byte limit to 14720 from 14600 (#2494) 1613 o Update PTO calculation to match RFC6298 (#2480, #2489, #2490) 1615 o Improve loss detection's description of multiple packet number 1616 spaces and pseudocode (#2485, #2451, #2417) 1618 o Declare persistent congestion even if non-probe packets are sent 1619 and don't make persistent congestion more aggressive than RTO 1620 verified was (#2365, #2244) 1622 o Move pseudocode to the appendices (#2408) 1624 o What to send on multiple PTOs (#2380) 1626 C.7. Since draft-ietf-quic-recovery-17 1628 o After Probe Timeout discard in-flight packets or send another 1629 (#2212, #1965) 1631 o Endpoints discard initial keys as soon as handshake keys are 1632 available (#1951, #2045) 1634 o 0-RTT state is discarded when 0-RTT is rejected (#2300) 1636 o Loss detection timer is cancelled when ack-eliciting frames are in 1637 flight (#2117, #2093) 1639 o Packets are declared lost if they are in flight (#2104) 1641 o After becoming idle, either pace packets or reset the congestion 1642 controller (#2138, 2187) 1644 o Process ECN counts before marking packets lost (#2142) 1646 o Mark packets lost before resetting crypto_count and pto_count 1647 (#2208, #2209) 1649 o Congestion and loss recovery state are discarded when keys are 1650 discarded (#2327) 1652 C.8. Since draft-ietf-quic-recovery-16 1654 o Unify TLP and RTO into a single PTO; eliminate min RTO, min TLP 1655 and min crypto timeouts; eliminate timeout validation (#2114, 1656 #2166, #2168, #1017) 1658 o Redefine how congestion avoidance in terms of when the period 1659 starts (#1928, #1930) 1661 o Document what needs to be tracked for packets that are in flight 1662 (#765, #1724, #1939) 1664 o Integrate both time and packet thresholds into loss detection 1665 (#1969, #1212, #934, #1974) 1667 o Reduce congestion window after idle, unless pacing is used (#2007, 1668 #2023) 1670 o Disable RTT calculation for packets that don't elicit 1671 acknowledgment (#2060, #2078) 1673 o Limit ack_delay by max_ack_delay (#2060, #2099) 1675 o Initial keys are discarded once Handshake keys are available 1676 (#1951, #2045) 1678 o Reorder ECN and loss detection in pseudocode (#2142) 1680 o Only cancel loss detection timer if ack-eliciting packets are in 1681 flight (#2093, #2117) 1683 C.9. Since draft-ietf-quic-recovery-14 1685 o Used max_ack_delay from transport params (#1796, #1782) 1687 o Merge ACK and ACK_ECN (#1783) 1689 C.10. Since draft-ietf-quic-recovery-13 1691 o Corrected the lack of ssthresh reduction in CongestionEvent 1692 pseudocode (#1598) 1694 o Considerations for ECN spoofing (#1426, #1626) 1696 o Clarifications for PADDING and congestion control (#837, #838, 1697 #1517, #1531, #1540) 1699 o Reduce early retransmission timer to RTT/8 (#945, #1581) 1701 o Packets are declared lost after an RTO is verified (#935, #1582) 1703 C.11. Since draft-ietf-quic-recovery-12 1705 o Changes to manage separate packet number spaces and encryption 1706 levels (#1190, #1242, #1413, #1450) 1708 o Added ECN feedback mechanisms and handling; new ACK_ECN frame 1709 (#804, #805, #1372) 1711 C.12. Since draft-ietf-quic-recovery-11 1713 No significant changes. 1715 C.13. Since draft-ietf-quic-recovery-10 1717 o Improved text on ack generation (#1139, #1159) 1719 o Make references to TCP recovery mechanisms informational (#1195) 1721 o Define time_of_last_sent_handshake_packet (#1171) 1723 o Added signal from TLS the data it includes needs to be sent in a 1724 Retry packet (#1061, #1199) 1726 o Minimum RTT (min_rtt) is initialized with an infinite value 1727 (#1169) 1729 C.14. Since draft-ietf-quic-recovery-09 1731 No significant changes. 1733 C.15. Since draft-ietf-quic-recovery-08 1735 o Clarified pacing and RTO (#967, #977) 1737 C.16. Since draft-ietf-quic-recovery-07 1739 o Include Ack Delay in RTO(and TLP) computations (#981) 1741 o Ack Delay in SRTT computation (#961) 1743 o Default RTT and Slow Start (#590) 1745 o Many editorial fixes. 1747 C.17. Since draft-ietf-quic-recovery-06 1749 No significant changes. 1751 C.18. Since draft-ietf-quic-recovery-05 1753 o Add more congestion control text (#776) 1755 C.19. Since draft-ietf-quic-recovery-04 1757 No significant changes. 1759 C.20. Since draft-ietf-quic-recovery-03 1761 No significant changes. 1763 C.21. Since draft-ietf-quic-recovery-02 1765 o Integrate F-RTO (#544, #409) 1767 o Add congestion control (#545, #395) 1769 o Require connection abort if a skipped packet was acknowledged 1770 (#415) 1772 o Simplify RTO calculations (#142, #417) 1774 C.22. Since draft-ietf-quic-recovery-01 1776 o Overview added to loss detection 1778 o Changes initial default RTT to 100ms 1780 o Added time-based loss detection and fixes early retransmit 1782 o Clarified loss recovery for handshake packets 1784 o Fixed references and made TCP references informative 1786 C.23. Since draft-ietf-quic-recovery-00 1788 o Improved description of constants and ACK behavior 1790 C.24. Since draft-iyengar-quic-loss-recovery-01 1792 o Adopted as base for draft-ietf-quic-recovery 1794 o Updated authors/editors list 1795 o Added table of contents 1797 Acknowledgments 1799 Authors' Addresses 1801 Jana Iyengar (editor) 1802 Fastly 1804 Email: jri.ietf@gmail.com 1806 Ian Swett (editor) 1807 Google 1809 Email: ianswett@google.com