idnits 2.17.1 draft-ietf-quic-ack-frequency-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (25 October 2021) is 907 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC J. Iyengar 3 Internet-Draft Fastly 4 Intended status: Standards Track I. Swett 5 Expires: 28 April 2022 Google 6 25 October 2021 8 QUIC Acknowledgement Frequency 9 draft-ietf-quic-ack-frequency-01 11 Abstract 13 This document describes a QUIC extension for an endpoint to control 14 its peer's delaying of acknowledgements. 16 Note to Readers 18 Discussion of this draft takes place on the QUIC working group 19 mailing list (quic@ietf.org), which is archived at 20 https://mailarchive.ietf.org/arch/search/?email_list=quic. Source 21 code and issues list for this draft can be found at 22 https://github.com/quicwg/ack-frequency. 24 Working Group information can be found at https://github.com/quicwg. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on 28 April 2022. 43 Copyright Notice 45 Copyright (c) 2021 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 50 license-info) in effect on the date of publication of this document. 51 Please review these documents carefully, as they describe your rights 52 and restrictions with respect to this document. Code Components 53 extracted from this document must include Simplified BSD License text 54 as described in Section 4.e of the Trust Legal Provisions and are 55 provided without warranty as described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 1.1. Terms and Definitions . . . . . . . . . . . . . . . . . . 3 61 2. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 3 62 3. Negotiating Extension Use . . . . . . . . . . . . . . . . . . 4 63 4. ACK_FREQUENCY Frame . . . . . . . . . . . . . . . . . . . . . 5 64 5. Multiple ACK_FREQUENCY Frames . . . . . . . . . . . . . . . . 6 65 6. IMMEDIATE_ACK Frame . . . . . . . . . . . . . . . . . . . . . 7 66 7. Sending Acknowledgments . . . . . . . . . . . . . . . . . . . 7 67 7.1. Response to Out-of-Order Packets . . . . . . . . . . . . 8 68 7.2. Expediting Congestion Signals . . . . . . . . . . . . . . 8 69 7.3. Batch Processing of Packets . . . . . . . . . . . . . . . 9 70 8. Computation of Probe Timeout Period . . . . . . . . . . . . . 9 71 9. Implementation Considerations . . . . . . . . . . . . . . . . 10 72 9.1. Loss Detection . . . . . . . . . . . . . . . . . . . . . 10 73 9.2. New Connections . . . . . . . . . . . . . . . . . . . . . 10 74 9.3. Window-based Congestion Controllers . . . . . . . . . . . 10 75 9.4. Connection Migration . . . . . . . . . . . . . . . . . . 11 76 9.5. Path MTU Discovery . . . . . . . . . . . . . . . . . . . 11 77 10. Security Considerations . . . . . . . . . . . . . . . . . . . 11 78 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 79 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 80 12.1. Normative References . . . . . . . . . . . . . . . . . . 11 81 12.2. Informative References . . . . . . . . . . . . . . . . . 12 82 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 12 83 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 12 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 86 1. Introduction 88 This document describes a QUIC extension for an endpoint to control 89 its peer's delaying of acknowledgements. 91 1.1. Terms and Definitions 93 The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 94 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 95 "OPTIONAL" in this document are to be interpreted as described in BCP 96 14 [RFC2119] [RFC8174] when, and only when, they appear in all 97 capitals, as shown here. 99 In the rest of this document, "sender" refers to a QUIC data sender 100 (and acknowledgement receiver). Similarly, "receiver" refers to a 101 QUIC data receiver (and acknowledgement sender). 103 An "acknowledgement packet" refers to a QUIC packet that contains 104 only an ACK frame. 106 This document uses terms, definitions, and notational conventions 107 described in Section 1.2 and Section 1.3 of [QUIC-TRANSPORT]. 109 2. Motivation 111 A receiver acknowledges received packets, but it can delay sending 112 these acknowledgements. The delaying of acknowledgements can impact 113 connection throughput, loss detection and congestion controller 114 performance at a data sender, and CPU utilization at both a data 115 sender and a data receiver. 117 Reducing the frequency of acknowledgement packets can improve 118 connection and endpoint performance in the following ways: 120 * Sending UDP packets can be noticeably CPU intensive on some 121 platforms. Reducing the number of packets that only contain 122 acknowledgements can therefore reduce the amount of CPU consumed 123 at a data receiver. Experience shows that this cost reduction can 124 be significant for high bandwidth connections. 126 * Similarly, receiving and processing UDP packets can also be CPU 127 intensive, and reducing acknowledgement frequency reduces this 128 cost at a data sender. 130 * Severely asymmetric link technologies, such as DOCSIS, LTE, and 131 satellite links, connection throughput in the data direction 132 becomes constrained when the reverse bandwidth is filled by 133 acknowledgment packets. When traversing such links, reducing the 134 number of acknowledgments allows connection throughput to scale 135 much further. 137 As discussed in Section 9 however, there can be undesirable 138 consequences to congestion control and loss recovery if a receiver 139 uniltaerally reduces the acknowledgment frequency. A sender's 140 constraints on the acknowledgement frequency need to be taken into 141 account to maximize congestion controller and loss recovery 142 performance. 144 [QUIC-TRANSPORT] currently specifies a simple delayed acknowledgement 145 mechanism that a receiver can use: send an acknowledgement for every 146 other packet, and for every packet that is received out of order 147 (Section 13.2.1 of [QUIC-TRANSPORT]). This simple mechanism does not 148 allow a sender to signal its constraints. This extension provides a 149 mechanism to solve this problem. 151 3. Negotiating Extension Use 153 Endpoints advertise their support of the extension described in this 154 document by sending the following transport parameter (Section 7.2 of 155 [QUIC-TRANSPORT]): 157 min_ack_delay (0xff03de1a): A variable-length integer representing 158 the minimum amount of time in microseconds by which the endpoint 159 can delay an acknowledgement. This limit could be based on the 160 receiver's clock or timer granularity. 162 An endpoint's min_ack_delay MUST NOT be greater than its 163 max_ack_delay. Endpoints that support this extension MUST treat 164 receipt of a min_ack_delay that is greater than the received 165 max_ack_delay as a connection error of type 166 TRANSPORT_PARAMETER_ERROR. Note that while the endpoint's 167 max_ack_delay transport parameter is in milliseconds (Section 18.2 of 168 [QUIC-TRANSPORT]), min_ack_delay is specified in microseconds. 170 The min_ack_delay transport parameter is a unilateral indication of 171 support for receiving ACK_FREQUENCY frames. If an endpoint sends the 172 transport parameter, the peer is allowed to send ACK_FREQUENCY frames 173 independent of whether it also sends the min_ack_delay transport 174 parameter or not. 176 Receiving a min_ack_delay transport parameter indicates that the peer 177 might send ACK_FREQUENCY frames in the future. Until an 178 ACK_FREQUENCY frame is received, receiving this transport parameter 179 does not cause the endpoint to change its acknowledgement behavior. 181 Endpoints MUST NOT remember the value of the min_ack_delay transport 182 parameter they received. Consequently, ACK_FREQUENCY frames cannot 183 be sent in 0-RTT packets, as per Section 7.4.1 of [QUIC-TRANSPORT]. 185 This Transport Parameter is encoded as per Section 18 of 186 [QUIC-TRANSPORT]. 188 4. ACK_FREQUENCY Frame 190 Delaying acknowledgements as much as possible reduces both work done 191 by the endpoints and network load. An endpoint's loss detection and 192 congestion control mechanisms however need to be tolerant of this 193 delay at the peer. An endpoint signals the frequency it wants to 194 receive ACK frames to its peer using an ACK_FREQUENCY frame, shown 195 below: 197 ACK_FREQUENCY Frame { 198 Type (i) = 0xaf, 199 Sequence Number (i), 200 Ack-Eliciting Threshold (i), 201 Request Max Ack Delay (i), 202 Reserved (6), 203 Ignore CE (1), 204 Ignore Order (1) 205 } 207 Following the common frame format described in Section 12.4 of 208 [QUIC-TRANSPORT], ACK_FREQUENCY frames have a type of 0xaf, and 209 contain the following fields: 211 Sequence Number: A variable-length integer representing the sequence 212 number assigned to the ACK_FREQUENCY frame by the sender to allow 213 receivers to ignore obsolete frames, see Section 5. 215 Ack-Eliciting Threshold: A variable-length integer representing the 216 maximum number of ack-eliciting packets the recipient of this 217 frame can receive without sending an acknowledgment. In other 218 words, an acknowledgement is sent when more than this number of 219 ack-eliciting packets have been received. Since this is a maximum 220 value, a receiver can send an acknowledgement earlier. A value of 221 0 results in a receiver immediately acknowledging every ack- 222 eliciting packet. 224 Request Max Ack Delay: A variable-length integer representing the 225 value to which the endpoint requests the peer update its 226 max_ack_delay (Section 18.2 of [QUIC-TRANSPORT]). The value of 227 this field is in microseconds, unlike the 'max_ack_delay' 228 transport parameter, which is in milliseconds. Sending a value 229 smaller than the min_ack_delay advertised by the peer is invalid. 230 Receipt of an invalid value MUST be treated as a connection error 231 of type PROTOCOL_VIOLATION. 233 Reserved: This field has no meaning in this version of 234 ACK_FREQUENCY. The value of this field MUST be 0x00. Receipt of 235 any other value MUST be treated as a connection error of type 236 FRAME_ENCODING_ERROR. 238 Ignore Order: A 1-bit field representing a boolean truth value. 239 This field is set to true by an endpoint that does not wish to 240 receive an immediate acknowledgement when the peer receives a 241 packet out of order (Section 7.1). 0 represents 'false' and 1 242 represents 'true'. 244 Ignore CE: A 1-bit field representing a boolean truth value. This 245 field is set to true by an endpoint that does not wish to receive 246 an immediate acknowledgement when the peer receives CE-marked 247 packets (Section 7.1). 0 represents 'false' and 1 represents 248 'true'. 250 ACK_FREQUENCY frames are ack-eliciting. However, their loss does not 251 require retransmission if an ACK_FREQUENCY frame with a larger 252 Sequence Number value has been sent. 254 An endpoint MAY send ACK_FREQUENCY frames multiple times during a 255 connection and with different values. 257 An endpoint will have committed a max_ack_delay value to the peer, 258 which specifies the maximum amount of time by which the endpoint will 259 delay sending acknowledgments. When the endpoint receives an 260 ACK_FREQUENCY frame, it MUST update this maximum time to the value 261 proposed by the peer in the Request Max Ack Delay field. 263 5. Multiple ACK_FREQUENCY Frames 265 An endpoint can send multiple ACK_FREQUENCY frames, and each one of 266 them can have different values in all fields. An endpoint MUST use a 267 sequence number of 0 for the first ACK_FREQUENCY frame it constructs 268 and sends, and a strictly increasing value thereafter. 270 An endpoint MUST allow reordered ACK_FREQUENCY frames to be received 271 and processed, see Section 13.3 of [QUIC-TRANSPORT]. 273 On the first received ACK_FREQUENCY frame in a connection, an 274 endpoint MUST immediately record all values from the frame. The 275 sequence number of the frame is recorded as the largest seen sequence 276 number. The new Ack-Eliciting Threshold and Request Max Ack Delay 277 values MUST be immediately used for delaying acknowledgements; see 278 Section 7. 280 On a subsequently received ACK_FREQUENCY frame, the endpoint MUST 281 check if this frame is more recent than any previous ones, as 282 follows: 284 * If the frame's sequence number is not greater than the largest one 285 seen so far, the endpoint MUST ignore this frame. 287 * If the frame's sequence number is greater than the largest one 288 seen so far, the endpoint MUST immediately replace old recorded 289 state with values received in this frame. The endpoint MUST start 290 using the new values immediately for delaying acknowledgements; 291 see Section 7. The endpoint MUST also replace the recorded 292 sequence number. 294 6. IMMEDIATE_ACK Frame 296 A sender can use an ACK_FREQUENCY frame to reduce the number of 297 acknowledgements sent by a receiver, but doing so increases the 298 chances that time-sensitive feedback is delayed as well. For 299 example, as described in Section 9.1, delaying acknowledgements can 300 increase the time it takes for a sender to detect packet loss. The 301 IMMEDIATE_ACK frame helps mitigate this problem. 303 An IMMEDIATE_ACK frame can be useful in other situations as well. 304 For example, it can be used with a PING frame (Section 19.2 of 305 [QUIC-TRANSPORT]) if a sender wants an immediate RTT measurement or 306 if a sender wants to establish receiver liveness as quickly as 307 possible. 309 An endpoint SHOULD send a packet containing an ACK frame immediately 310 upon receiving an IMMEDIATE_ACK frame. An endpoint MAY delay sending 311 an ACK frame despite receiving an IMMEDIATE_ACK frame. For example, 312 an endpoint might do this if a large number of received packets 313 contain an IMMEDIATE_ACK or if the endpoint is under heavy load. 315 IMMEDIATE_ACK Frame { 316 Type (i) = 0xac, 317 } 319 7. Sending Acknowledgments 321 Prior to receiving an ACK_FREQUENCY frame, endpoints send 322 acknowledgements as specified in Section 13.2.1 of [QUIC-TRANSPORT]. 324 On receiving an ACK_FREQUENCY frame and updating its recorded 325 max_ack_delay and Ack-Eliciting Threshold values (Section 5), the 326 endpoint MUST send an acknowledgement when one of the following 327 conditions are met: 329 * Since the last acknowledgement was sent, the number of received 330 ack-eliciting packets is greater than or equal to the recorded 331 Ack-Eliciting Threshold. 333 * Since the last acknowledgement was sent, max_ack_delay amount of 334 time has passed. 336 Section 7.1, Section 7.2, and Section 7.3 describe exceptions to this 337 strategy. 339 An endpoint is expected to bundle acknowledgements when possible. 340 Every time an acknowledgement is sent, bundled or otherwise, all 341 counters and timers related to delaying of acknowledgments are reset. 343 The receiver of an ACK_FREQUENCY frame can continue to process 344 multiple available packets before determining whether to send an ACK 345 frame in response, as stated in Section 13.2.2 of [QUIC-TRANSPORT]. 347 7.1. Response to Out-of-Order Packets 349 As specified in Section 13.2.1 of [QUIC-TRANSPORT], endpoints are 350 expected to send an acknowledgement immediately on receiving a 351 reordered ack-eliciting packet. This extension modifies this 352 behavior. 354 If the endpoint has not yet received an ACK_FREQUENCY frame, or if 355 the most recent frame received from the peer has an Ignore Order 356 value of false (0x00), the endpoint MUST immediately acknowledge any 357 subsequent packets that are received out of order. 359 If the most recent ACK_FREQUENCY frame received from the peer has an 360 Ignore Order value of true (0x01), the endpoint does not make this 361 exception. That is, the endpoint MUST NOT send an immediate 362 acknowledgement in response to packets received out of order, and 363 instead continues to use the peer's Ack-Eliciting Threshold and 364 max_ack_delay thresholds for sending acknowledgements. 366 7.2. Expediting Congestion Signals 368 An endpoint SHOULD send an immediate acknowledgement when a packet 369 marked with the ECN Congestion Experienced (CE) codepoint in the IP 370 header is received and the previously received packet was not marked 371 CE. 373 Doing this maintains the peer's response time to congestion events, 374 while also reducing the ACK rate compared to Section 13.2.1 of 375 [QUIC-TRANSPORT] during extreme congestion or when peers are using 376 DCTCP [RFC8257] or other congestion controllers that mark more 377 frequently than classic ECN [RFC3168]. 379 7.3. Batch Processing of Packets 381 For performance reasons, an endpoint can receive incoming packets 382 from the underlying platform in a batch of multiple packets. This 383 batch can contain enough packets to cause multiple acknowledgements 384 to be sent. 386 To avoid sending multiple acknowledgements in rapid succession, an 387 endpoint MAY process all packets in a batch before determining 388 whether a threshold has been met and an acknowledgement is to be sent 389 in response. 391 8. Computation of Probe Timeout Period 393 On sending an update to the peer's max_ack_delay, an endpoint can use 394 this new value in later computations of its Probe Timeout (PTO) 395 period; see Section 5.2.1 of [QUIC-RECOVERY]. The endpoint MUST 396 however wait until the ACK_FREQUENCY frame that carries this new 397 value is acknowledged by the peer. 399 Until the frame is acknowledged, the endpoint MUST use the greater of 400 the current max_ack_delay and the value that is in flight when 401 computing the PTO period. Doing so avoids spurious PTOs that can be 402 caused by an update that increases the peer's max_ack_delay. 404 While it is expected that endpoints will have only one ACK_FREQUENCY 405 frame in flight at any given time, this extension does not prohibit 406 having more than one in flight. Generally, when using max_ack_delay 407 for PTO computations, endpoints MUST use the maximum of the current 408 value and all those in flight. 410 When the number of in-flight ack-eliciting packets is larger than the 411 ACK-Eliciting Threshold, an endpoint can expect that the peer will 412 not need to wait for its max_ack_delay period before sending an 413 acknowledgement. In such cases, the endpoint MAY therefore exclude 414 the peer's 'max_ack_delay' from its PTO calculation. Note that this 415 optimization requires some care in implementation, since it can cause 416 premature PTOs under packet loss when ignore_order is enabled. 418 9. Implementation Considerations 420 There are tradeoffs inherent in a sender sending an ACK_FREQUENCY 421 frame to the receiver. As such it is recommended that implementers 422 experiment with different strategies and find those which best suit 423 their applications and congestion controllers. There are, however, 424 noteworthy considerations when devising strategies for sending 425 ACK_FREQUENCY frames. 427 9.1. Loss Detection 429 A sender relies on receipt of acknowledgements to determine the 430 amount of data in flight and to detect losses, e.g. when packets 431 experience reordering, see [QUIC-RECOVERY]. Consequently, how often 432 a receiver sends acknowledgments determines how long it takes for 433 losses to be detected at the sender. 435 9.2. New Connections 437 Many congestion control algorithms have a startup mechanism during 438 the beginning phases of a connection. It is typical that in this 439 period the congestion controller will quickly increase the amount of 440 data in the network until it is signalled to stop. While the 441 mechanism used to achieve this increase varies, acknowledgments by 442 the peer are generally critical during this phase to drive the 443 congestion controller's machinery. A sender can send ACK_FREQUENCY 444 frames while its congestion controller is in this state, ensuring 445 that the receiver will send acknowledgments at a rate which is 446 optimal for the the sender's congestion controller. 448 9.3. Window-based Congestion Controllers 450 Congestion controllers that are purely window-based and strictly 451 adherent to packet conservation, such as the one defined in 452 [QUIC-RECOVERY], rely on receipt of acknowledgments to move the 453 congestion window forward and send additional data into the network. 454 Such controllers will suffer degraded performance if acknowledgments 455 are delayed excessively. Similarly, if these controllers rely on the 456 timing of peer acknowledgments (an "ACK clock"), delaying 457 acknowledgments will cause undesirable bursts of data into the 458 network. 460 9.4. Connection Migration 462 To avoid additional delays to connection migration confirmation when 463 using this extension, a client can bundle an IMMEDIATE_ACK frame with 464 the first non-probing frame (Section 9.2 of [QUIC-TRANSPORT]) it 465 sends or it can simply send an IMMEDIATE_ACK frame, which is a non- 466 probing frame. 468 An endpoint's congestion controller and RTT estimator are reset upon 469 confirmation of migration (Section 9.4 of [QUIC-TRANSPORT]), which 470 can impact the number of acknowledgements received after migration. 471 An endpoint that has sent an ACK_FREQUENCY frame earlier in the 472 connection SHOULD update and send a new ACK_FREQUENCY frame 473 immediately upon confirmation of connection migration. 475 9.5. Path MTU Discovery 477 A sender might use timers to detect loss of PMTUD probe packets. A 478 sender SHOULD bundle an IMMEDIATE_ACK frame with any PTMUD probes to 479 avoid triggering such timers. 481 10. Security Considerations 483 TBD. 485 11. IANA Considerations 487 TBD. 489 12. References 491 12.1. Normative References 493 [QUIC-TRANSPORT] 494 Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 495 Multiplexed and Secure Transport", RFC 9000, 496 DOI 10.17487/RFC9000, May 2021, 497 . 499 [QUIC-RECOVERY] 500 Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection 501 and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, 502 May 2021, . 504 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 505 Requirement Levels", BCP 14, RFC 2119, 506 DOI 10.17487/RFC2119, March 1997, 507 . 509 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 510 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 511 May 2017, . 513 12.2. Informative References 515 [RFC8257] Bensley, S., Thaler, D., Balasubramanian, P., Eggert, L., 516 and G. Judd, "Data Center TCP (DCTCP): TCP Congestion 517 Control for Data Centers", RFC 8257, DOI 10.17487/RFC8257, 518 October 2017, . 520 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 521 of Explicit Congestion Notification (ECN) to IP", 522 RFC 3168, DOI 10.17487/RFC3168, September 2001, 523 . 525 Appendix A. Change Log 527 *RFC Editor's Note:* Please remove this section prior to 528 publication of a final version of this document. 530 Acknowledgments 532 The following people directly contributed key ideas that shaped this 533 draft: Bob Briscoe, Kazuho Oku, Marten Seemann. 535 Authors' Addresses 537 Jana Iyengar 538 Fastly 540 Email: jri.ietf@gmail.com 542 Ian Swett 543 Google 545 Email: ian.swett@google.com