Sender Control of Delayed Acknowledgments in TCP: Problem Statement, Requirements and Analysis of Potential Solutions

Delayed Acknowledgments (ACKs) were specified for TCP with the aim to reduce protocol overhead . With Delayed ACKs, a TCP delays sending an ACK by up to 500 ms (often 200 ms, with lower values in recent implementations such as ~50 ms also reported), and typically sends an ACK for at least every second segment received in a stream of full-sized segments. This allows combining several segments into a single one (e.g. the application layer response to an application layer data message, and the corresponding ACK), and it also saves up to one of every two ACKs under many traffic patterns (e.g. bulk transfers). The "SHOULD" requirement level for implementing Delayed ACKs in RFC 1122, along with its expected benefits, has led to a widespread deployment of this mechanism. However, there exist traffic patterns and scenarios for which Delayed ACKs can actually be detrimental to performance. When a segment carrying a message of a size up to one Maximum Segment Size (MSS) is transferred, if the message does not elicit an application-layer response, and a second data segment is not transferred earlier than the Delayed ACK timeout, the ACK is unnecessarily delayed, with a number of negative consequences. Furthermore, there may be reasons to allow a sender communicate the ACK ratio to be used in a TCP connection, and thus dynamically override (or restore) use of Delayed ACKs at the receiver. This document presents the problem statement regarding sender control of Delayed ACKs. The document discusses the scenarios and use cases in which sender control of Delayed ACKs offers advantages. Then, requirements for a potential solution are derived. Finally, a number of potential solutions are discussed, based on the requirements, and also considering pros and cons in each case.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

This section provides scenarios and use cases where performance issues arise due to Delayed ACKs.

During slow start, the congestion window (cwnd) increases by up to Sender Maximum Segment Size (SMSS) upon receipt of an ACK covering new data . However, use of Delayed ACKs reduces the amount of ACKs received by the sender, thus reducing the rate of cwnd growth, increasing transfer time and reducing throughput, when compared with sending an ACK for each incoming data segment. Note that, while Appropriate Byte Counting (ABC) might be used to address this problem, it remains as an experimental mechanism, not fully included in RFC 5681, which specifies standard TCP congestion control. Furthermore, Delayed ACKs preclude using sender behaviors intended to quickly and non-intrusively probe for available capacity during slow start. One example of such behaviors is chirping, where a sender follows a predetermined pattern to send data segments (e.g. decreasing intersegment time gaps) and measures the gaps between ACKs (see Appendix B.4 of revision -03 of the referenced document).

When the Nagle algorithm is used, in some cases the sender may be prevented from sending more data while awaiting a delayed ACK. In some high bit rate environment (e.g. Gigabit Ethernet) use cases, such a delay may be very large, and link utilitzation may be dramatically reduced, since the Delayed ACK timeout may be several orders of magnitude greater than the Round Trip Time (RTT) .

Delayed ACKs are also detrimental in Internet of Things (IoT) scenarios, where TCP is being increasingly used . Many IoT devices, such as sensors, transfer small messages (e.g. containing sensor readings) rather infrequently, therefore if the receiver uses Delayed ACKs, the ACK will often be unnecessarily delayed. The sender cannot release the memory resources associated to a transferred data segment until the ACK is received and processed. This may be a problem for many IoT devices, which are typically memory-constrained, and may even lead to subsequent packet drops if their scarce memory resources are blocked while awaiting an ACK. Moreover, if the IoT device uses a radio interface for communication, in some scenarios Delayed ACKs will lead to increased energy consumption (e.g. with the radio interface of the device staying in receive mode while awaiting the ACK). Since many IoT devices run on small batteries, the device lifetime may significantly decrease. Furthermore, the delay suffered by the ACK may interact negatively with layer two mechanisms, especially in wireless network technologies where devices remain in low-power states for long intervals , potentially leading to a further exacerbated delay (by even one or more orders of magnitude).

In some scenarios, it may be desirable to enable ACK transmission behaviors beyond the classic ones. For example, it may be beneficial to apply congestion control to ACKs . Reducing the amount of ACKs on a congested reverse path may allow alleviating congestion on that path, with minimal impact on a relatively independent forward path. On the other hand, in some scenarios, the rate at which ACKs arrive at the sender limits the achievable performance of data transfer. This happens due to forward and reverse path asymmetric capacity (with the latter being significantly limited, e.g. in terms of bandwidth) . In some environments, the issue is mitigated by using middleboxes that perform ACK thinning, i.e., deleting a subset of the ACKs. Examples of technologies where deployments have been reported to do ACK thinning include satellite links, DOCSIS cable networks, mobile cellular networks, among others.

This section provides the requirements for a potential solution to enable sender control of Delayed ACKs.

An assumption is that the sender knows when Delayed ACKs operation should be overriden. For example, the sender may know in advance the pattern of the traffic it will generate, or it may know whether an application-layer response will be sent by the receiving endpoint upon reception of a given message. Therefore, control of Delayed ACKs has to be sender-triggered.

One approach that cannot be recommended as a general solution for controlling Delayed ACKs is (permanently) disabling Delayed ACKs at the receiving TCP. In fact, the latter may interact with a wide variety of devices and many of those may still benefit from the advantages of Delayed ACKs. Another approach would be determining suppression of Delayed-ACKs per connection. However, within the same connection, a sender may offer a mixed traffic pattern comprising single data segments that will lead to unnecessarily delayed ACKs, with other data segments upon which Delayed ACKs will act as intended. Therefore, the solution has to be provided at a per-segment granularity.

Since the presented problem is about low performance in various scenarios, another requirement for the solution is to minimize incurring overhead in terms of header size increase or additional packets sent. For example, in IoT scenarios, every additional communicated byte consumes scarce resources (e.g. energy, bandwidth and computational resources). Another benefit of keeping a low header/message overhead is alleviating the processing workload of a receiving TCP.

For many of the scenarios and use cases described in Section 2, an ACK ratio of 1 (i.e. a receiver sending one ACK per incoming data segment) would solve the mentioned issues. However, the ability of enforcing a generic ACK ratio (including values different from 1 and 2) allows to enable a wider range of ACK behaviors, which may support congestion control for ACKs, sender behaviors not based on ACK-clocking, etc. The desired generic ACK ratio is intended to be in force for the current data segment, and for subsequent data segments (at least, during a time interval of a duration that may depend on several factors). The mechanism used to indicate the desired receiver Delayed ACK behavior might exploit soft state (i.e. using explicit information carried by data segments) or connection state (which needs to be stored by the receiver).

Deployment of new functionality for TCP faces the risk of packets being discarded by existing middleboxes upon detection of unexpected or discouraged formats, header field values or even traffic patterns. A solution for sender control of Delayed ACKs should offer relatively good middlebox traversal (to the extent possible).

A solution for sender control of Delayed ACKs must ensure that normal Delayed ACKs operation is in force by default, and also once temporary action on Delayed ACKs needs to end.

A solution for sender control of Delayed ACKs should not reduce the space of existing TCP functionality.

A solution for sender control of Delayed ACKs should not pose significant risk of preventing future TCP development. If an available resource (e.g. a reserved bit of the TCP header, a new TCP option, etc.) is used by a solution, careful analysis must be carried out regarding the risks and benefits of using such resource.

Sender control of Delayed ACKs might be achieved by using workarounds, such as implementation techniques that may produce the desired effect. However, such approaches may be suboptimal regarding implementation cleanliness, and may entail other performance issues (see section 5.6).

The receiver might not always be able to honour the ACK behavior desired by the sender. Therefore, the semantics of sender control of Delayed ACKs have to be of a hint, not a command. If the receiver is actually not able to apply the ACK behavior desired by the sender, then the former has a range of options with regard to communicating so to the latter, from remaining silent, to providing explicit feedback to the sender. Each option has associated trade-offs. For example, remaining silent might degrade performance if a sender relies on a receiver that uses the ACK behavior intended by the sender.

This section enumerates and discusses potential solutions that might be considered to enable sender control of Delayed ACKs. The list of solutions is not necessarily comprehensive. This section intends to illustrate the trade-offs that arise when considering potential solutions for sender control of Delayed ACKs. (Note: the analysis needs to be completed for many of the solutions below.)

In Acknowledgment Congestion Control (AckCC) , the sender tells the receiver the ACK ratio R to use, where the receiver sends one ACK per R data packets received. AckCC defines a 2-byte "TCP ACK Congestion Control Permitted Option" for negotiating use of AckCC, whereas it defines a 3-byte "ACK ratio TCP option" to communicate the ACK Ratio value from the sender to the receiver. Middlebox traversal of a new TCP option is often regarded as 'bad' (to be confirmed).

Tail Loss Probe (TLP) is intended to avoid RTO-expiration-based retransmission when tail loss occurs by inducing additional ACKs at the receiver. This is achieved by sending a probe segment after a probe time-out (PTO) when data have been sent but not confirmed. Of course, this means sending a whole new packet to trigger ACKs, which adds significant overhead. This approach might offer good middlebox traversal (to be confirmed).

One solution that has been proposed for sender control of Delayed ACKs is called 'TCP ACK Pull' . TCP ACK Pull defines the AKP flag as bit number 6 of the 13th byte of the TCP header. When a TCP sender needs a data segment to be acknowledged by the receiving TCP without additional delay, the sender sets the AKP flag of the data segment TCP header. Upon reception of a segment with the AKP flag set, a conforming receiving TCP behaves accordingly by sending the corresponding ACK without additional delay. This solution would entail zero header or message overhead. However, it would consume a TCP header bit, leaving only two available TCP header reserved bits. A question is thus whether one TCP header bit should be dedicated to this purpose or not. Middlebox traversal characteristics of bit 6 of the TCP header need to be assessed.

Another approach relies on defining a new option-kind-only TCP option with the same semantics as the AKP flag, which might be called 'ACK Pull Option' or 'AKP Option'. This solution would consume an available TCP Option Kind number. However, most of the 256 numbers in the TCP Option Kind number space are currently available. Therefore, consuming one such number does not appear to significantly limit future TCP development. The header overhead of the AKP Option is one byte. Middlebox traversal of a new TCP option is often regarded as 'bad' (to be confirmed).

Another approach that might be used to enable sender control of Delayed ACKs is based on reusing existing TCP header fields. For example, use of the Urgent pointer has been suggested (e.g. by reserving 3 of its 16 bits to encode an ACK ratio exponent that may be communicated by the sender to the receiver), when URG=0. A problem with this approach is that the semantics of the reused TCP header field may become overloaded. Therefore, in some cases either the original intended use of the reused TCP header field may become limited, or if it prevails, then sender control of Delayed ACKs might not always be available for use. Middlebox traversal characteristics of this approach might be relatively good (to be confirmed).

One approach that allows eliciting an immediate ACK after sending a data segment is sending a subsequent segment carrying a previously acknowledged data byte. However, in addition to the inefficiency of sending a byte that has previously been sent, this approach may require the transmission of a new packet (even carrying a single byte of data payload) just for that purpose, which represents significant overhead. Furthermore, sending a previously sent byte is not a clean solution from an implementation perspective. Another workaround intended to trigger an immediate ACK from the receiving TCP, which is used in the Contiki operating system (a popular operating system for constrained devices in IoT scenarios) is splitting the data to be sent into two segments of smaller size. A standard compliant TCP receiver will acknowledge the second MSS of data. However, this 'split hack' may not always work since a TCP receiver is required to acknowledge every second full-sized segment, but not two consecutive small segments. Furthermore, the overhead of sending two IP packets instead of one is another downside of the 'split hack'.

The next table summarizes whether the different solutions presented in Section 4 are able to satisfy the requirements stated in Section 4.

TBD

Bob Briscoe, Jonathan Morton, Richard Scheffenegger, Michael Tuexen and Jana Iyengar provided useful input for this document. Stuart Cheshire, Ted Lemon, Michael Scharf, and Christoph Paasch participated in a discussion that was seminal to the TCP ACK Pull proposal, which eventually led to this document. Carles Gomez has been funded in part by the Spanish Government (Ministerio de Ciencia, Innovacion y Universidades) through Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya 2017 SGR 376.