idnits 2.17.1 draft-fairhurst-quic-ack-scaling-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 28, 2020) is 1549 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-34) exists of draft-ietf-quic-recovery-25 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-25 == Outdated reference: A later version (-02) exists of draft-iyengar-quic-delayed-ack-00 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force G. Fairhurst 3 Internet-Draft A. Custura 4 Intended status: Informational T. Jones 5 Expires: July 31, 2020 University of Aberdeen 6 January 28, 2020 8 Changing the Default QUIC ACK Policy 9 draft-fairhurst-quic-ack-scaling-00 11 Abstract 13 On network paths where there is significant path asymmetry, 14 acknowledgments of data receipt can reduce the efficient use of 15 network capacity. This effect occurs when the return capacity is 16 significantly more constrained than the forward capacity, or the cost 17 of transmission per packet is a significant component of the total 18 transmission cost. In these cases, reducing the ratio of 19 acknowledgements to data can improve link utilization and reduce link 20 transmission costs. 22 This document proposes a change to the default acknowledgement policy 23 of the QUIC transport protocol to improve performance over paths with 24 appreciable asymmetry. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on July 31, 2020. 43 Copyright Notice 45 Copyright (c) 2020 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 61 2. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 2 62 2.1. ACK Ratio Impact on Asymmetric Paths . . . . . . . . . . 3 63 2.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 64 2.3. Adapting the ACK Ratio in Current Transport 65 Specifications . . . . . . . . . . . . . . . . . . . . . 3 66 2.4. Adapting the ACK Ratio in the Network . . . . . . . . . . 4 67 3. Updating the Default ACK Ratio for QUIC . . . . . . . . . . . 4 68 3.1. Considerations Updating the Default QUIC ACK Policy . . . 4 69 3.2. During Slow Start . . . . . . . . . . . . . . . . . . . . 5 70 3.3. After Slow Start . . . . . . . . . . . . . . . . . . . . 6 71 3.4. When Encountering Loss/Congestion . . . . . . . . . . . . 6 72 3.5. Further Tuning of the ACK Policy . . . . . . . . . . . . 6 73 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 74 5. Security Considerations . . . . . . . . . . . . . . . . . . . 6 75 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 6 76 6.1. Normative References . . . . . . . . . . . . . . . . . . 6 77 6.2. Informative References . . . . . . . . . . . . . . . . . 7 78 Appendix A. Summary of Recommended ACK Policy for TCP . . . . . 8 79 Appendix B. Experiments Exploring an ACK Ratio of 1:10 . . . . . 9 80 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 82 1. Introduction 84 The current design of QUIC currently proposes a default 85 acknowledgement (ACK) ratio of 1:2 (at least one ACK for each 2 data 86 packets) inspired by current recommendations for TCP, Appendix A, 87 see. This document proposes an increase in the ratio of ACK packets 88 to data packets from 1:2 to 1:10 for QUIC flows, once a flow has 89 exited slow start. 91 2. Motivation 93 When the characteristics of the forward and return paths are not 94 symmetric, the transmission of ACKs can adversely impact either 95 transport performance or the cost of sending data across a link. 97 2.1. ACK Ratio Impact on Asymmetric Paths 99 TCP Performance Implications of Network Path Asymmetry [RFC3449] 100 describes a series of problems and mitigations when transports use an 101 asymmetric path. Performance problems arise in several access 102 networks, including bandwidth-asymmetric networks (such as broadband 103 satellite access, DOCSIS cable networks, cellular mobile, etc). 105 Where the ACK rate is limited by the capacity of the return path, 106 this constrains the maxium throughput for the forward path. The ACK 107 traffic also constrains other traffic that shares a constrained 108 return path. This motivates the need to reduce the volume of ACK 109 traffic (increase the number of segments/packets that are 110 acknowledged by each ACK). 112 Capacity is not the only asymmetric path constraint. Sending ACKs 113 can consume significant transmission resources and the cost of 114 transmitting ACKs can become a significant part of the cost of 115 transmission when using a network segment. In many wireless 116 technologies, there is appreciable overhead for the transmission of 117 each packet burst (data and ACK). There can also be associated costs 118 (e.g. in radio resource management and transmission scheduling) that 119 are often different for the forward and return paths because they use 120 different technologies or configurations. This provides an incentive 121 to reduce the rate of ACK traffic. 123 2.2. Terminology 125 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 126 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 127 "OPTIONAL" in this document are to be interpreted as described in BCP 128 14 [RFC2119] [RFC8174] when, and only when, they appear in all 129 capitals, as shown here. 131 2.3. Adapting the ACK Ratio in Current Transport Specifications 133 Various methods have been proposed to modify the ACK Ratio used by 134 transport protocols. Two examples follow: 136 ACK-CC [RFC5690] proposed a method to control the rate of ACKs to 137 avoid the return path from becoming congested, but this did not 138 achieve wide-scale deployment. 140 The Datagram Congestion Control Protocol (DCCP) [RFC4340] has methods 141 to control the ACK ratio at the receiver. DCCP specifies a TCP- 142 friendly congestion control [RFC4341], which includes the ability to 143 use signaling to allow this sender to adjust the receiver ACK ratio 144 within certain parameters. This also was not widely used. 146 End-to-end transport methods do not have a way to detect and account 147 for the cost of ACK transmission, and are difficult to tune to adapt 148 to delays introduced by the path (especially when fair-queueing and 149 other link scheduling methods hide the effects of increased ACK 150 traffic). They therefore only provide a partial mitigation to the 151 impacts when sending ACKs over asymmetric paths. 153 2.4. Adapting the ACK Ratio in the Network 155 An alternative approach has been deployed for TCP that uses a 156 middlebox in the network to thin the rate of ACKs. This method is 157 used with paths that exhibit significant ACK symmetry to improve 158 performance of TCP when it uses an ACK Ratio of 1:2 [RFC3449]. 159 Performance Enhancing Proxies (PEPs) can implement these functions 160 when they detect/know an upstream link is filled with TCP ACKs. 162 Removing redundant ACKs (also known as "ACK Thinning") leads to TCP 163 stretch ACKs (where a single ACK acknowledges more than two TCP 164 segments). The introduction of TCP Appropriate Byte Counting (ABC) 165 i[RFC3465] partly mitigates the impact of stretch ACKs, and also 166 recommends burst mitigation techniques at a TCP sender. 168 These methods only work when ACKs can be observed by a device in the 169 network. QUIC uses an encrypted feedback packet to communicate an 170 ACK. The use of encryption intentionally prevents such in-network 171 optimisations. Compared to TCP, performance of QUIC is therefore 172 disadvantaged when QUIC uses an ACK Ratio of 1:2. 174 3. Updating the Default ACK Ratio for QUIC 176 Any ACK policy that changes the ACK ratio from 1:2 needs to 177 compensate for three issues: 179 o A reduced frequency of feedback can increase the time to detect 180 congestion, impacting the congestion control algorithm. 182 o A reduced frequency of feedback can increase the time to detect 183 loss, impacting the loss recovery algorithm; 185 o A reduced ACK rate can lead to bursts of acknowledged packets, and 186 introduces a need for burst mitigation at the sender; 188 3.1. Considerations Updating the Default QUIC ACK Policy 190 The QUIC transport protocol currently specifies a maximum ACK Delay, 191 which is communicated by the sender to indicate the maximum time an 192 endpoint will delay sending acknowledgments. A default of 25 193 milliseconds is recommended. QUIC also currently recommends a 194 default ACK Ratio of 1:2. 196 This document proposes changing the default QUIC behaviour to send an 197 ACK for at least every 10 received packets. Further background to 198 the proposed method is detailed in the Annexe (Appendix B). 200 Congestion controllers can benefit from frequent feedback during an 201 initial slow start period, where the sender is probing for available 202 path capacity. This update therefore does not change the recommended 203 ACK Ratio during the initial part of slow start. This ensures 204 stretch ACKs do not impact the initial rate of growth for the 205 congestion window. 207 The ACK Delay timer ensures ACKs are not unduly delayed. The effect 208 of a large delay could be significant when a stretched ACK 209 acknowledges more packets, and therefore the proposed method also 210 ensures feedback at least four times per RTT (less important when the 211 RTT is greater than 100ms and the ACK Delay forms a small part of the 212 total RTT). 214 Loss detection can be impacted by delayed acknowledgments. Although 215 timer-based methods (e.g., using the QUIC Probe Timeout (PTO), see 216 section 5.1 of [I-D.ietf-quic-recovery]) can reduce the reliance on 217 ACKs to detect loss, prompt communication of ACK ranges after loss is 218 still important to efficient loss recovery. For a short period of 219 time after detected loss, a QUIC receiver therefore ACKs each packet 220 (with an ACK Ratio 1:1). 222 Since the introduction of the specification to allow a larger TCP 223 Initial Window (IW) [RFC6928], there has been deployment experience 224 using TCP with an IW of 10 segments at startup. QUIC continues this 225 practice, which in part motivates the need for a QUIC transport 226 protocol to operate when a burst of acknowledgements for up to ten 227 packets is received. QUIC therefore transport recommends the use of 228 pacing to mitigate packet bursts being generated by a sender (see 229 section 6.8 of [I-D.ietf-quic-recovery]). 231 The proposed method seeks to allow QUIC to effectively operate over 232 asymmetric paths. 234 3.2. During Slow Start 236 For the first 100 received packets, the receiver should send an ACK 237 if max_ack_delay time has passed since the oldest unacknowledged data 238 was received OR upon receiving two unacknowledged packets (an ACK 239 Ratio of 1:2). 241 A receiver typically has no understanding of the senders congestion 242 control state. The number 100 reflects a trade-off, corresponding to 243 an appreciable opening of the sender's congestion window. 245 A congestion controller typically re-enters slow start after 246 congestion is detected. The need to probe to re-establish a working 247 congestion window is helped by the ACK policy after loss. 249 3.3. After Slow Start 251 A receiver sends an ACK if a period more than 252 MIN(max_ack_delay,min_rtt/4) has passed since receiving the oldest 253 unacknowledged data OR it has accumulated 10 unacknowledged packets. 255 3.4. When Encountering Loss/Congestion 257 Following detected loss or congestion, a receiver sends ACKs 258 according to section 13.2.1 of [I-D.ietf-quic-transport]. 260 3.5. Further Tuning of the ACK Policy 262 In situations where the default method is not sufficient, the ACK 263 Ratio might be further tuned by server, as described in 264 [I-D.iyengar-quic-delayed-ack]. This could also permit the ACK 265 method to be adapted to match the behaviour of new congestion control 266 algorithms. Reducing the rate of ACKs can also lower the 267 computational effort required to process ACKs at the sender and 268 receiver. For instance, this could reduce the workload for high 269 speed network interfaces by reducing the rate of cache ejection for 270 Generic Receiver Offload (GRO). 272 4. IANA Considerations 274 This memo includes no request to IANA. 276 5. Security Considerations 278 The security considerations for the QUIC transport protocol are 279 described in [I-D.ietf-quic-transport]. 281 6. References 283 6.1. Normative References 285 [I-D.ietf-quic-recovery] 286 Iyengar, J. and I. Swett, "QUIC Loss Detection and 287 Congestion Control", draft-ietf-quic-recovery-25 (work in 288 progress), January 2020. 290 [I-D.ietf-quic-transport] 291 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 292 and Secure Transport", draft-ietf-quic-transport-25 (work 293 in progress), January 2020. 295 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 296 Requirement Levels", BCP 14, RFC 2119, 297 DOI 10.17487/RFC2119, March 1997, 298 . 300 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 301 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 302 May 2017, . 304 6.2. Informative References 306 [I-D.iyengar-quic-delayed-ack] 307 Iyengar, J. and I. Swett, "Sender Control of 308 Acknowledgement Delays in QUIC", draft-iyengar-quic- 309 delayed-ack-00 (work in progress), January 2020. 311 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 312 Communication Layers", STD 3, RFC 1122, 313 DOI 10.17487/RFC1122, October 1989, 314 . 316 [RFC2525] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, 317 J., Heavens, I., Lahey, K., Semke, J., and B. Volz, "Known 318 TCP Implementation Problems", RFC 2525, 319 DOI 10.17487/RFC2525, March 1999, 320 . 322 [RFC3449] Balakrishnan, H., Padmanabhan, V., Fairhurst, G., and M. 323 Sooriyabandara, "TCP Performance Implications of Network 324 Path Asymmetry", BCP 69, RFC 3449, DOI 10.17487/RFC3449, 325 December 2002, . 327 [RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte 328 Counting (ABC)", RFC 3465, DOI 10.17487/RFC3465, February 329 2003, . 331 [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram 332 Congestion Control Protocol (DCCP)", RFC 4340, 333 DOI 10.17487/RFC4340, March 2006, 334 . 336 [RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion 337 Control Protocol (DCCP) Congestion Control ID 2: TCP-like 338 Congestion Control", RFC 4341, DOI 10.17487/RFC4341, March 339 2006, . 341 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 342 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 343 . 345 [RFC5690] Floyd, S., Arcia, A., Ros, D., and J. Iyengar, "Adding 346 Acknowledgement Congestion Control to TCP", RFC 5690, 347 DOI 10.17487/RFC5690, February 2010, 348 . 350 [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, 351 "Increasing TCP's Initial Window", RFC 6928, 352 DOI 10.17487/RFC6928, April 2013, 353 . 355 Appendix A. Summary of Recommended ACK Policy for TCP 357 RFC 5681 [RFC5681] clarifies the TCP ACK frequency described in 358 RFC 1122 [RFC1122] to recommend the ACK policy for a TCP receiver: 359 "an ACK SHOULD be generated for at least every second full-sized 360 segment, and MUST be generated within 500 ms of the arrival of the 361 first unacknowledged packet". 363 The TCP sender regards reception of an ACK as a positive indication 364 that data has been received across the path. The congestion control 365 algorithm uses this to increase the size of the congestion window, 366 cwnd RFC 5681 [RFC5681]. 368 To reduce the ACK Rate, a receiver can delay sending an ACK for a 369 period of time called the ACK Delay. This can increase network 370 efficiency. When the receiver delays ACKs, this reduces the rate of 371 growth of the cwnd. TCP implementations often use heuristics such as 372 DAAS (Delayed ACK after Slow Start) to mitigate this. This allows 373 the receiver to estimate when the sender could be in the slow start 374 phase of cwnd growth, and for a period of time sends an ACK for each 375 received segment/packet (i.e., an ACK Ratio of 1:1). 377 ACKs can be lost on the return path (either through packet loss, or 378 by intentional thinning of the ACK stream). If a sender does not 379 receive an ACK for every second segment, a stretch ACK has occurred. 380 RFC2525 [RFC2525] describes the significance of stretch ACK 381 violations: 383 "this behavior will cause TCP senders to generate burstier traffic, 384 which can degrade performance in congested environments. In 385 addition, generating fewer ACKs increases the amount of time needed 386 by the slow start algorithm to open the congestion window to an 387 appropriate point, which diminishes performance in environments with 388 large bandwidth-delay products. Finally, generating fewer ACKs may 389 cause needless retransmission timeouts in lossy environments, as it 390 increases the possibility that an entire window of ACKs is lost, 391 forcing a retransmission timeout." 393 RFC 3465 [RFC3465] further discusses the issue of bursts that may be 394 caused by the interaction between ACK processing and congestion 395 control. This motivates a need to deal with bursts within TCP. TCP 396 senders can mitigate bursts by using Appropriate Byte Counting (ABC), 397 which increases the congestion window in proportion to the amount of 398 data sent into the network, rather than upon the arrival of each ACK. 399 (This also defends against ACK Splitting, where multiple ACKs are 400 received for parts of the same segment/packet). 402 Appendix B. Experiments Exploring an ACK Ratio of 1:10 404 The performance of QUIC is at a disadvantage compared to other 405 transport protocols if designs use a conservative ACK Ratio, because 406 QUIC can not modified by in-network middleboxes (such as used for TCP 407 ACK Thinning). We argue that a default ratio of 1:2 is too 408 conservative. 410 We used an experimental approach to examine a change to QUIC's ACK 411 Policy . This 412 updated the default ACK Ratio from 1:2 to 1:10. Our tests show that 413 this did not negatively impact the protocol. This reduced the amount 414 of IP, UDP and ACK overhead by a factor of approximately 5. The 415 implemented congestion control, was also not negatively impacted. 416 These experiments were performed in January 2020 based on available 417 implementations at that time. 419 Unlike TCP, QUIC sends other types of data frames in addition to ACK 420 frames, increasing the total overhead on the return path. On 421 asymmetrical paths an ACK Ratio of 1:10 may still reduce the ACK 422 traffic, helping to avoid return path capacity limits impacting the 423 ability to use the forward path capacity. 425 Figure 1 presents a table with a set of asymmetric scenarios. The 426 columns present the rate of ACK traffic required (in kbps) to fill 427 each of the forward paths. The table shows the results for TCP 428 (without a PEP), both for lossless communication. It considers the 429 period after loss, when ACKs communicate the loss information. It 430 also shows the impact of using an ACK Ratio of 1:10 with QUIC. 432 An ACK Ratio of 1:10 reduces the utilisation of the return path. 433 Scenarios where the ACK traffic exceeds the return link capacity 434 (i.e. where this limits the forward path capacity that can be used) 435 are marked with a star. 437 +-------------------+-------------+---------------+-----------------+ 438 | | 10/2 Mbps | 50/10 Mbps | 250/3 Mbps | 439 +-------------------+-------------+---------------+-----------------+ 440 | TCP no loss | 133 - 346 | 650 -1,730 | 3250 - 8,650* | 441 | TCP loss | 346 - 560 | 1,730 - 2,800 | 8,650 - 14,000* | 442 | QUIC 1:2 no loss | 144 - 438 | 720 - 2,190 | 3,600 - 10,950* | 443 | QUIC 1:2 loss | 290 - * | 1450 - * | 7,250 - * | 444 | QUIC 1:10 no loss | 28.8 - 87.6 | 144 - 438 | 720 - 2,190 | 445 +-------------------+-------------+---------------+-----------------+ 447 Figure 1: ACK traffic required to fill the forward path in different 448 loss and asymmetry scenarios 450 Figure 2 presents a table with the numbers of packets sent by two 451 QUIC implementations using a 1:2 and a 1:10 ACK Ratio. This shows 452 between a four and five time reduction in the number of packets sent. 454 +-------------------------------+----------------+----------------+ 455 | | Chromium | Quicly | 456 | | Draft 23 | Draft 23 | 457 +-------------------------------+----------------+----------------+ 458 | Packets sent | 77,419 | 83,238 | 459 | Packets on return path (1:2) | 39,089 (50.4%) | 41,108 (49.3%) | 460 | Packets on return path (1:10) | 10,409 (13.4%) | 9650 (11.5%) | 461 +-------------------------------+----------------+----------------+ 463 Figure 2: Number of packets sent and received during a 100MB QUIC 464 transfer using different ACK ratios, for two implementations 466 Figure 3 presents a table with the number of bytes sent by one of the 467 QUIC implementations using a 1:2 and a 1:10 ACK Ratio. This shows a 468 reduction in the number of bytes on the return path from 2.7 to 0.7% 469 of the total bytes sent. In these scenarios, this is sufficient to 470 take full advantage of the forward path capacity. 472 +---------------------------------+----------------+ 473 | | Chromium | 474 | | Draft 23 | 475 +---------------------------------+----------------+ 476 | Bytes sent | 110M | 477 | Bytes on return path (1:2 AR) | 3,056KB (2.7%) | 478 | Bytes on return path (1:10 AR) | 810KB (0.7%) | 479 +---------------------------------+----------------+ 481 Figure 3: Number of bytes sent and received during a 100MB Chromium 482 QUIC transfer using different ACK ratios 484 The number of bytes and packets reduces as expected when using an ACK 485 Ratio of 1:10, without any increase in loss, on a high-latency path 486 with an asymmetry of 5:1. This offers a clear benefit for paths that 487 are capacity-constrained, as well as paths which would benefit from a 488 reduction in the ACK Rate. 490 Authors' Addresses 492 Godred Fairhurst 493 University of Aberdeen 494 School of Engineering 495 Fraser Noble Building 496 Aberdeen AB24 3UE 497 UK 499 Email: gorry@erg.abdn.ac.uk 501 Ana Custura 502 University of Aberdeen 503 School of Engineering 504 Fraser Noble Building 505 Aberdeen AB24 3UE 506 UK 508 Email: ana@erg.abdn.ac.uk 510 Tom Jones 511 University of Aberdeen 512 School of Engineering 513 Fraser Noble Building 514 Aberdeen AB24 3UE 515 UK 517 Email: tom@erg.abdn.ac.uk